Append mode when saving DataFrames to file

[zhabinka]

Greetings! I need save some big DataFrames to single file, but to_csv overwrites the file. And I use this code:

data
  |> Explorer.DataFrame.dump_csv!(header: false)
  |> then(fn data -> File.write("./data.csv", data, [:append]) end)

Is there any way to set the mode like a (append) mode in Pandas?

[billylanchantin]

Hello!

This is not currently possible in Explorer. Though it's definitely possible using Polars:

https://stackoverflow.com/a/75273080/5932228

But we'd need to implement it basically from scratch in Rust. (I think we'd optionally use a batched writer, I'm not sure.)

This is a good feature request. PRs are welcome!