Shawn Chang
Shawn Chang
Opened this PR for testing hadoop 3.4 on top of Spark4. When testing Spark4 I saw many tests failed to `java.lang.NoSuchMethodError: 'boolean org.apache.hadoop.fs.FileStatus.hasAcl()'`. `FileStatus#hasAcl` only exists in Hadoop 3+
This PR is still WIP, I'm trying to trigger Azure CI to detect potential failures
@hudi-bot run azure
> @CTTY Are you planning to apply this to Amazon EMR Hudi once merged? @Gatsby-Lee Yes, the support would be released along with Spark 4.0.0 on EMR
@Gatsby-Lee No, but we should have better idea after Spark 4 offcial release is out
Hi @liurenjie1024 , thanks for having this! Would it be ok if I take up the commit path work? I can start working on a POC tomorrow
Hi @stevie9868 , Thanks for being willing to work on this! However, I don't think we need a new trait like `PartitionFIleWriter` to achieve partition-aware writing. Some of my thoughts:...
Hi all, I'm currently working on a draft for this and will post it for comments/feedback soon!
Hi folks, I've drafted this doc to summarize what have been discussed here plus some code snippets to showcase what the new trait would look like: https://docs.google.com/document/d/1-CEvRvb52vPTDLnzwJRBx5KLpej7oSlTu_rg0qKEGZ8/edit?tab=t.0#heading=h.a0d7lqurg3dq Please feel free...
There is a critical bug with how we persist tables using `GlueCatalog` and would be nice if we can fix this in 0.8.0: https://github.com/apache/iceberg-rust/issues/1868