objectiv-analytics
objectiv-analytics copied to clipboard
Support new datastore: Athena
Step one: Bach basics
- [x] Test setup Bach:
- Create test infrastructure
- Add
--athenaoption for running tests - Add mark to annotate tests that support new datastore
- [x] Make first test work:
- make
DataFrame.from_pandas()- required by most functional tests to get an initial DataFrame with test data - Make super basic test work:
test_df.py::test_basic()
- make
- [x] Make tests for new datastore run in CI automatically
- [ ] Make all Series work with new datastore
- [x] Int
- [x] Float
- [x] Boolean
- [x] Uuid
- [x] String
- [x] Json
- [x] Date
- [x] DateTime
- [x] Time
- [x] TimeDelta
- [ ] Possibly more
- [ ] Check which tests work out of the box, and annotate them to always run for the new datastore
Step two: Modelhub and more Bach
- [ ] Test setup Modelhub
- [ ] Port DataPipeline to work with new datastore
- [ ] Make production data available
- [ ] Write platform specific DataPipeline parts
- [ ] Port custom Modelhub types
- [ ] Check which Modelhub tests work out of the box, and annotate them to always run for the new datastore
- [ ] Check what works in Notebooks out of the box
Step three
Work on anything that doesn't yet work in Bach/Modelhub, prioritize based on what is needed to make Modelhub and notebooks work