abhiNB-star

Results 21 comments of abhiNB-star

@RameshkumarChikoti123 @ad1happy2go @danny0405 @yihua Any solution to this i tried every possible things that is added in this chain i am doing a fresh ingestion and still facing same error...

@danny0405 Hudi version : 0.14.1 SparK 3.1.1 Hadoop version : 3.3.4 Storage (HDFS/S3/GCS..) : GCS sparkConf: spark.executor.memoryOverhead: 1200M spark.kubernetes.executor.podNamePrefix: testing-party-job-old-hudi spark.serializer: org.apache.spark.serializer.KryoSerializer spark.kryo.registrator: org.apache.spark.HoodieSparkKryoRegistrar spark.executor.extraJavaOptions: "-Dlog4j.configuration=log4j.properties -verbose:class" spark.jars: "gs://dummy/nbdata/resources/jars/test/hbase-client-2.4.17.jar,gs://dummy/nbdata/resources/jars/test/hbase-common-2.4.17.jar,gs://dummy/nbdata/resources/jars/test/hbase-metrics-api-2.4.17.jar,gs://dummy/nbdata/resources/jars/test/hbase-server-2.4.17.jar"

@danny0405 @ad1happy2go `abhi@bastion-host-new:~/testing/new_test/party$ kubectl logs -f -n spark-hood testing-party-job-old-hudi-exec-1 | grep 'HFile' [149.207s][info][class,load] org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex source: file:/opt/spark/work-dir/./hudi-utilities-bundle_2.12-0.14.1.jar [150.231s][info][class,load] org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex$HFileBootstrapIndexWriter source: file:/opt/spark/work-dir/./hudi-utilities-bundle_2.12-0.14.1.jar [150.232s][info][class,load] org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex$HFileBootstrapIndexReader source: file:/opt/spark/work-dir/./hudi-utilities-bundle_2.12-0.14.1.jar [150.232s][info][class,load] org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex$HFilePathForReader source: file:/opt/spark/work-dir/./hudi-utilities-bundle_2.12-0.14.1.jar [150.241s][info][class,load] org.apache.hudi.org.apache.hadoop.hbase.io.hfile.HFile$CachingBlockReader...

hoodie.archive.merge.enable: true hoodie.auto.adjust.lock.configs: true hoodie.bulkinsert.shuffle.parallelism: 2 hoodie.clean.automatic: true hoodie.cleaner.fileversions.retained: 5 hoodie.cleaner.parallelism: 200 hoodie.cleaner.policy: KEEP_LATEST_FILE_VERSIONS hoodie.cleaner.policy.failed.writes: LAZY hoodie.datasource.hive_sync.assume_date_partitioning: false hoodie.datasource.hive_sync.database: test hoodie.datasource.hive_sync.enable: true hoodie.datasource.hive_sync.jdbcurl: jdbc:hive2://192.168.3.56:10000 hoodie.datasource.hive_sync.metastore.uris: thrift://192.168.3.30:9083 hoodie.datasource.hive_sync.mode: hms hoodie.datasource.hive_sync.omit_metadata_fields: true...

I am trying to ingest the data using spark+kafka streaming to hudi table with the RLI index. but unfortunately ingesting records is throwing the below issue. Steps to reproduce the...

@codope [hoodie_bundle.tar.gz](https://github.com/user-attachments/files/20724365/hoodie_bundle.tar.gz)