TarunMootala
TarunMootala
@ad1happy2go Thanks for your inputs. I don't think it was related to loading of archival timeline. When this error occurred, the first option I've tried is cleaning of archival timeline...
@ad1happy2go `.hoodie/` folder is 350 MB and it has 3435 files (this includes active and archival timelines) `.hoodie/archived/` is 327 MB and it has 695 files (only archival timelines) Attached...
@ad1happy2go, When AWS Glue encounters OOME it kills the JVM immediately. It could be reason for the error not being available in driver logs. However, the error is present in...
@ad1happy2go > Can you share the timeline? Can you elaborate on this ? > Do you know how many file groups are there in the clean instant? Are you referring...
Related issue https://github.com/apache/hudi/issues/8199
I'm facing the exact OOME error with the Cleaner. Is there a fix or workaround ? Hudi Version: 0.12.1 Spark 3.3.0 Glue: 4.0 Spark Streaming batch interval: 120 seconds ####...
@gudladona What did you do to handle this issue?