Ryo Okubo

Results 26 comments of Ryo Okubo

I will create a patch which implements 1) option if no problem.

@markya0616 I've fixed conflict. Could you review this? 🙏

`GuessByRegexp` looks useful guess rule! I'd like to support that but we need to think how to read regexp pattern string. We can some options, like below: - cli options...

Oh I missed existing `arrjson` might support arrow -> json conversion for this part `arrow -> map's -> json` https://github.com/apache/arrow/tree/master/go/arrow/internal/arrjson It's actually an internal package but reusable for this use...

I finally examine a mem pprof result. It show a lower usage than the current version's ( https://github.com/reproio/columnify/issues/44#issuecomment-654917629 ) The reduction effect is `543.88MB` -> `97.45MB`, 18% sized! But I'm...

To resolve https://github.com/reproio/columnify/issues/27, we need https://github.com/reproio/columnify/issues/49 ... 😭 . I created an another pullreq https://github.com/reproio/columnify/pull/50 to aim that.

Here's a quick performance test. I gave the below dummy input Avro file. ``` $ java -jar ~/tools/avro-tools-1.8.2.jar random --schema-file examples/primitives.avsc --count 1000000 tmp.avro log4j:WARN No appenders could be found...

> record type validations #27 Finally supported! If we have this schema: ``` $ cat columnifier/testdata/schema/primitives.avsc { "type": "record", "name": "Primitives", "fields" : [ {"name": "boolean", "type": "boolean"}, {"name": "int",...

> The elapsed time increased by 1.5x I will profile CPU usages next.

I added benchmark and profilings into the CI job. The cpu profiling was here: ``` flat flat% sum% cum cum% 5.59s 14.32% 14.32% 9.55s 24.46% runtime.scanobject 2.20s 5.64% 19.95% 2.59s...