Mykhailo Osypov

@mgill25 Hi, great PR. I wonder whether you're planning on finish that. Also alternatively @houqp, we could merge it to the new feature branch instead and work on it collectively....

I like the idea with the separate struct. As for the repro steps, only in the isolated way as in this test https://github.com/delta-io/delta-rs/blob/main/rust/tests/checkpoint_writer_test.rs#L174-L195. E.g. manually create a broken tombstones.

Some context for the repro steps. The very first implementation of delta-rs checkpoint creation has missing this code https://github.com/delta-io/delta-rs/blob/abd0aaf7801d0ca53844caf94381424f93808dac/rust/src/checkpoints.rs#L186-L191. Which is `setting extended_file_metadata=false` if at least **one** tombstones has no...

Let's go with the example then: pre fix: ``` {path=tomb1, ..., extended_file_metadata=true, size=1} {path=tomb2, ..., extended_file_metadata=false} // is being written in checkpoint as {path=tomb1, ..., extended_file_metadata=true} {path=tomb2, ..., extended_file_metadata=false} ```...

An easy way would be just an additional option/flag. And if set, then we'd keep tomstones list empty, destipe it's writer or reader

@dispanser If we stick with an optional flag, then if it's set, then prevent of creating a checkpoint and telling user a meaningful error

From delta log https://github.com/delta-io/delta/blob/master/PROTOCOL.md#primitive-types > timestamp | Microsecond precision timestamp without a timezone From my personal delta-rs / spark interop, microsecond is the correct precision for timestamps which works with...

That is true, I made a false assumption in that comment due to initial implementation was failing in unit testing. But nevertheless I'm glad this has grown into separate issue/pr...

Hi! It's managed by delta-rs and rusoto-s3, the default instance creds should work just fine. In addition you can even assume role via sts, please see https://github.com/delta-io/delta-rs/blob/main/rust/src/storage/s3/mod.rs#L30-L86 and https://github.com/delta-io/delta-rs/blob/main/rust/src/storage/s3/mod.rs#L398

cc @xianwill