Drew Gallardo
Drew Gallardo
I believe in the Java implementation we have a concept of a [PositionOutputStream](https://github.com/apache/iceberg/blob/866021d7d34f274349ce7de1f29d113395e7f28c/api/src/main/java/org/apache/iceberg/io/PositionOutputStream.java#L24) which is used to keep track of bytes written to each file with a position/counter. What we...
Hey Fokko, I'll take references here ☝️
Your glue calls look, fine but your S3 calls are the problem. I was able to reproduce the issue by having the incorrect region for my AWS profile at `./aws/config`...
No Problem!! This could potentially be a bug if we assume that the catalog and FileIO (S3) share the same aws profile configs. On one side, having a single profile...
Interesting can you run `aws sts get-caller-identity` in the terminal to ensure the right identity is being used? you can also, explicitly set the S3FileIO by passing in the s3...
They added support for the arange function in pyarrow in https://github.com/apache/arrow/pull/46778. I pulled the latest Arrow branch, built it locally, and integrated it into PyIceberg. The updated implementation would looks...
Should we create sub-tasks tracking for adding each option type to the language bindings. That might help us avoid huge PRs, and we can surface some great first issues!
I can take up the Refs table if nobody is working on it!
Yeah, this definitely seems like an inconsistency in behavior between Python and Java, as PyIceberg allows the server to override the user's OAuth2 URL before initialization, whereas Java enforces that...