chenwyi2
chenwyi2
and i already use using use-starting-sequence-number variable, iceberg version is 0.13.1
we use cdc mode, the data is always changing, even if we use strategy to auto-retry, rewritedatafile will always be failed
Is this problem solved? i met the same error
@Shane-Yu i add 'write.metadata.metrics.default'='full' into table ,and i print the log messae with upper, "upper java.nio.HeapByteBuffer[pos=0 lim=16 cap=16],fromByteBuffer qbfs://online010", "pos=0 lim=16 cap=16" is still truncate 16 bit? it doesn't work?
ORC_BLOOM_FILTER_COLUMNS this property will work on spark? when i set write.orc.bloom.filter.columns=xx and used spark to write data, i found that bloomfilter had no effect on querying.
write.orc.bloom.filter.columns can take effect on spark 3.1? when i set write.orc.bloom.filter.columns=xx, wrote data by spark and query table by xx field, but i can not find any query improvement.
we also met this problem, and suppose the compact can cause some files are corrupted, Is there a solution?
maybe that's because of null value? we found that the file with null values can cause ParquetDecodingException, when filtering null value it's fine
https://github.com/apache/hive/pull/3888 changes HMS API , we can use "hive_metastoreConstants.EXPECTED_PARAMETER_KEY" or "hive_metastoreConstants.EXPECTED_PARAMETER_VALUE" to commit instead of 4 HMS calls
https://issues.apache.org/jira/browse/HIVE-26882 has been resolved, there will be a plan to make some improvement on hivelock?