clickhouse-java icon indicating copy to clipboard operation
clickhouse-java copied to clipboard

batch insert into aggregatingMergeTree table

Open marlondu opened this issue 3 years ago • 1 comments

I have a table with aggregatingMergeTree engine, like:

CREATE TABLE db2.agg_dcc1_local ON CLUSTER ck_default_cluster( 
        id String,
        city String,
        codeUniq AggregateFunction(uniq,String), 
        codeCount AggregateFunction(count,String), 
        valueSum AggregateFunction(sum,Int32), 
        valueMin AggregateFunction(min,Int32), 
        valueMax AggregateFunction(max,Int32), 
        valueAvg AggregateFunction(avg,Int32), 
        create_time DateTime
)
ENGINE = AggregatingMergeTree() 
PARTITION BY toYYYYMM(create_time) 
ORDER BY (id,city)

So, How to batch insert data into the table with jdbc api ? I read ClickHousePreparedStatementImpl.java#executeBatch's code. because "insert into ... select ..." does not contains "VALUES", so it execute one by one. and it was too slow.

help. !!! my clickhosue version: 21.9.1.7426 clickhouse-jdbc: 0.3.2-patch8

marlondu avatar May 06 '22 09:05 marlondu

Sorry @marlondu the driver does not support Aggregatefunction so you probably should use materialized view or projection etc instead.

As to batch insert, actually it's not that slow as it streams serialized data in RowBinary format.

zhicwu avatar May 07 '22 03:05 zhicwu