Mark Klik
Mark Klik
That's an interesting idea. Off course, `.rdata` files can contain any number of objects, but files created with `saveRDS` contain only a single object, so that would be a better...
Yes, I completely understand. Unfortunately some more changes to the format are necessary to allow for multiple chunks (`rbind`) and adding columns (`rbind`), but also for storing key indexes and...
Hi @talegari , thanks for your feature request and posting your code! Yes, support for streaming would be nice, your example is already very useful for processing a _fst_ file...
Hi @xiaodaigh, thanks for your question! The ability to append `fst` files to an existing `fst` file is planned for release v0.9.0. That's the release after the coming CRAN release...
Hi @wei-wu-nyc, thanks for your question. The `fst` file format is already prepared to make column- and row binding possible. That means that the meta-data for current `fst` files has...
Hi @wei-wu-nyc, your specific use-case is interesting because that's exactly the type of problem that I would like `fst` to help solve (in this case memory shortage). The most memory...
What would you like to do with the resulting table? Can your data be logically partitioned? Perhaps you only need a few columns for your calculations?
Thanks, yes, `read_fst` only allocates the table once and then just overwrites the elements in-memory, so that should take about the same size as the resulting vector (plus some buffering),...
Hi @phillc73, yes, `v0.9.0` was released earlier because of the [CRAN issue with rchk](https://github.com/fstpackage/fst/issues/189). Apologies for the confusion, this feature should have been moved to the `v0.9.2` milestone, thanks for...
Hi @schelhorn, thanks for your kind words and submitting your request! In general, serializers in `R` take different approaches to the treatment of column names: ```r data.table::fwrite(df, "1.csv") # no...