John C. Merfeld
John C. Merfeld
When we execute a query or column filter on a dataframe, under the hood it's actually a separate object. It might be convenient to be able to save this out...
Not sure this really matters, but wanted to document it: Showed datascroller to a friend last night. She went through the help box and tried out the features. But it...
As we've discussed, it would be cool to extend datascroller to other formats besides CSVs. Probably the next most useful datatype to support is object data – JSON. Setting aside...
When you're `scroll`ing a dataFrame, integer columns like `[0 1 2 3 4]` are displayed as `[0.0 1.0 2.0 3.0 4.0]`. Whatever the solution, this seems like undesirable behavior. Ints...
## Motivation Per [this ticket](https://educationanalytics.monday.com/boards/6058716254/pulses/6103710811), add a top-level `clean` command that removes the configured output directory (as well as earthmover_compiled.yml) and leaves the user with a clean project. Modeled after...
From this [Monday item](https://educationanalytics.monday.com/boards/6058716254/pulses/7287932813): > Earthmover doesn't support the following construction: > > ```yaml > config: > output_dir: ${OUTPUT_DIR} > parameter_defaults: > OUTPUT_DIR: './output/' > ``` > (result is the...
Pertaining to this [Monday item](https://educationanalytics.monday.com/boards/6058716254/pulses/6613536663). Two main ideas here: 1. **In order to upgrade to Python 3.12**, we need to upgrade Dask. Dask now uses a [query optimizer](https://blog.dask.org/2023/08/25/dask-expr-introduction) that does...
A bundle due for release in Runway uses a [preprocessor script](https://github.com/edanalytics/earthmover_edfi_bundles/blob/main/assessments/TX_KEA/python_pre_exec/tx_kea_pre_exec.py), which we do not want to allow. This change folds that functionality into Earthmover via the new `sparse_header_rows` option....
Rework how we think about the config block during project composition. Now uses the following precedence order for config options, from lowest to highest: 1. Hardcoded global `config_defaults` 2. `config`...