explorer

explorer copied to clipboard

Introduce `_with` APIs

1

The goal is to introduce `filter_with`, `summarize_with`, `mutate_with`, `arrange_with`, and `distinct_with`. ## Attack plan * [x] Support `filter_with` with row-based series operations * [x] Support `summarize_with` with aggregation-based series operations...

josevalim

Add support for Arrow IPC Stream files

5

This is waiting on https://github.com/pola-rs/polars/pull/3783 being merged, once it is we can remove the reference to my repository, and just use the Polars version with this. - [x] Merged https://github.com/jorgecarleitao/arrow2/pull/1097...

joshuataylor

The range implementation should be a matter of: take(df, Enum.slice(0..n_rows(df), range))

josevalim

Ideally we will have a separate test file for testing how each of them works when grouped, removing all of .group calls from data_frame_test.exs. ### Group handling * [x] group_by...

josevalim

Provide complete range of IO operations

8

Each IO operation must have 8 functions: `dump`, `load`, `from` and `to` as well as the `!` version of them. This is an issue to add the missing APIs. We...

josevalim

Combining string columns

2

Hi! Thanks for the great library! Is there a way to combine two string columns e.g. by concatenating them? My use case is this. ```#Explorer.DataFrame< Polars[24 x 3] House number...

tomekowal

This is supported in polars and can permit some useful stuff like explode.

cigrainger

Split filter/2 in two functions

5

@cigrainger I was wondering if we should break `filter/2` in two functions: `mask(dataframe, series)` and `filter(dataframe, callback)`. The rationale for doing so is that we don't want to encourage `filter(dataframe,...

josevalim

Across APIs

6

@cigrainger and I were discussing ideas to improve the dataframe API. One of the ideas is to bring dplyr's [recent `across` functionality](https://gist.github.com/josevalim/342728a5b2434569704fe129ff84d228), so we can do: |> DF.filter(across(sepal_width < 10))...

josevalim

Proposal: Integration testing for CSV, IPC, NDJSON, Parquet

3

I think it would be worthwhile adding tests for CSV, IPC, NDJSON and Parquet, with all having the same data. On the other hand, [Apache supplies Arrow testing files](https://github.com/apache/arrow-testing) and...

joshuataylor