iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

JDBC Catalog create table DDL is not compatible for MYSQL 8.0.29

Open noneback opened this issue 3 years ago • 1 comments

env: Ubuntu 20.04LTS, MYSQL 8.0.29 with InnoDB, Iceberg-flink-runtime built from the main branch, Flink1.15.

I am trying to use JDBC as the catalog in flink with iceberg, and I cannot create those iceberg_namespace_properties table because the attributes are too long as an index key.

The index key prefix length limit is 3072 bytes but in iceberg_namespace_properties, the index is longer than 5500.

Ref: #5014

MySQL 8.0 Reference Manual

noneback avatar Jun 14 '22 03:06 noneback

NAMESPACE_PROPERTY_KEY was defined as VARCHAR(5500), it may not have to be that long. Some value less than 500 is good enough I guess?

Kontinuation avatar Aug 09 '22 12:08 Kontinuation

@noneback are you able to increase the limit on MySql? Usually different databases impose different limits, but I'll see if we can lower namespace properties from 5500.

nastra avatar Dec 01 '22 12:12 nastra

I have created to https://github.com/apache/iceberg/pull/6338 to make this less of an issue with MySql

nastra avatar Dec 01 '22 15:12 nastra