Harsha Gudladona
Harsha Gudladona
Any update on this?
It also appears that, for non-aws environments, there is no logic for the remote write client to reload the credentials from the config file periodically, It's a common usecase that...
We encountered this issue as well, we think this is due to the SparkAvroPostProcessor ignoring the default values from the passed target schema [here](https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala#L154) - when converting from struct type...
@edenhill any insights here? Kindly take a look.
We are facing a similar issue during this phase kafka fetch errors (at INFO level) are as follows: 2022-09-02 06:26:12,606 INFO [Executor task launch worker for task 9.1 in stage...
Assessment and workaround provided here: https://github.com/aws/aws-sdk-java/issues/3110
After profiling the driver this is what we found. There are 100s of Jetty qtp(Queued Thread Pool) threads that are blocked and remained blocked when we encountered the socket timeouts....
We may have some indicators on what is causing this problem we have a small file limit of 100MB, it appears that this works well (makes larger files and cleans...
> what are the memory configs you're using? This failed on driver even with MAX heap of 32G. The threshold for failure can be relative to the number of files...