clickhouse-java icon indicating copy to clipboard operation
clickhouse-java copied to clipboard

Is this library suitable to generate Native files?

Open mikeTWC1984 opened this issue 3 years ago • 2 comments

Wondering if this library can be used to convert flat files (e.g. CSV) to Clickhouse Native format (without connecting to server)?

mikeTWC1984 avatar Aug 29 '22 01:08 mikeTWC1984

As of now, we have clickhouse-cli-client, which is simply a wrapper of the native command-line clickhouse-client. It supports docker in the case when ClickHouse client is not installed locally. However, it does not support clickhouse-local at this point.

Actually I added LOCAL and FILE two protocols and planned to enhance clickhouse-cli-client accordingly. Would be great if some of you want to contribute :)

zhicwu avatar Aug 29 '22 08:08 zhicwu

OK. What I mean is something like pure library for writing native files, without any CH binary dependency. Something like parquet-mr for Parquet format. The idea is to use it with ETL tools (specifically NiFi). Those typically can generate CSV/JSON/Parquet or Avro, but looks like native format is noticably better for ingestion.

mikeTWC1984 avatar Aug 29 '22 19:08 mikeTWC1984