ClickHouse icon indicating copy to clipboard operation
ClickHouse copied to clipboard

Issues with persistency in storage `File`/`S3`/… with setting `engine_file_allow_create_multiple_files`

Open CurtizJ opened this issue 1 year ago • 0 comments

Describe the unexpected behaviour All files created by storage with setting engine_file_allow_create_multiple_files except the first one are not visible by the storage after restart.

How to reproduce

DROP TABLE IF EXISTS test_file_insert;

CREATE TABLE test_file_insert (x UInt64) ENGINE = File(JSON, 'test_file.json');

SET engine_file_allow_create_multiple_files = 1;

INSERT INTO test_file_insert SELECT * FROM numbers(10);

INSERT INTO test_file_insert SELECT * FROM numbers(10);

SELECT _file, count() FROM test_file_insert GROUP BY _file ORDER BY _file ASC;

   ┌─_file────────────┬─count()─┐
1. │ test_file.1.json │      10 │
2. │ test_file.json   │      10 │
   └──────────────────┴─────────┘

DETACH TABLE test_file_insert;
ATTACH TABLE test_file_insert;

SELECT _file, count() FROM test_file_insert GROUP BY _file ORDER BY _file ASC;

   ┌─_file──────────┬─count()─┐
1. │ test_file.json │      10 │
   └────────────────┴─────────┘

Expected behavior The persistent storage with engine File should read from the same set of files after restart.

CurtizJ avatar May 17 '24 14:05 CurtizJ