Christopher Watford

We're ramping up our usage of delta-rs (and with it arrow-datafusion) and would like to see resolved much of the weirdness with dates/times (ignoring the absolute pain Spark brings upstream)....

I'm interested and happy to help where I can. I'm putting together a data access layer over top of our delta lake, and it is mostly Rust right now. Consistency...

After digging into Spark's source code, it appears that each structured stream receives an _isolated_ `SparkSession`, meaning it would be safe to set idempotent details within a microbatch. This is...

Yes, you can check the functional tests to confirm: https://github.com/fabfuel/circuitbreaker/blob/bb7cf0ca8c4c9742a75fe86387e08d679ecb7047/tests/test_functional.py#L227-L231