citus icon indicating copy to clipboard operation
citus copied to clipboard

Columnar: options ignored during ALTER TABLE rewrite

Open jeff-davis opened this issue 3 years ago • 2 comments

CREATE TABLE test(i int) USING columnar;
SELECT alter_columnar_table_set('test'::regclass, compression=>'lz4');
INSERT INTO test VALUES(1);
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000006
total file size: 24576, total data size: 6
compression rate: 0.83x
total row count: 1, stripe count: 1, average rows per stripe: 1
chunk count: 1, containing data for dropped columns: 0, lz4 compressed: 1

Compressed with `lz4'.

ALTER TABLE test ALTER COLUMN i TYPE int8;
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000007
total file size: 24576, total data size: 18
compression rate: 0.50x
total row count: 1, stripe count: 1, average rows per stripe: 1
chunk count: 1, containing data for dropped columns: 0, zstd compressed: 1

Now it's compressed with zstd.

INSERT INTO test VALUES(2);
VACUUM VERBOSE test;
INFO:  statistics for "test":
storage id: 10000000007
total file size: 32768, total data size: 28
compression rate: 0.64x
total row count: 2, stripe count: 2, average rows per stripe: 1
chunk count: 2, containing data for dropped columns: 0, lz4 compressed: 1, zstd compressed: 1

New stripes are compressed with lz4 again.

jeff-davis avatar May 05 '22 22:05 jeff-davis

This is caused by using a regclass in columnar.options instead of the storage ID.

jeff-davis avatar May 06 '22 17:05 jeff-davis

As discussed with Jeff, the problem occurs because during the ALTER TABLE process, PG creates a new copy of the original table and the copy does not have the same table OID yet so it compresses with its default zstd format instead.

yxu2162 avatar Aug 25 '22 22:08 yxu2162