operation: data: Add/Update operations to handle time series data
Pain Point
We do not have operations to work with time-series data
Proposed Solution
Implement/Update operations for the following tasks:
- [ ] NaN Removal
- [ ] Outlier Removal
- [ ] Missing Timestamp Handling
- [ ] Scaling
- [ ] Imputation & Interpolation
- [ ] Shifting/lag generation
- [ ] Time Series Decomposition
Note: Some Topics may overlap with others in the list above, we need to make our operations as generic as possible. So that they can work with not only time-series data but other data formats as well if possible.
Hey, @programmer290399
You mentioned earlier that you guys generally avoid using 3rd party libraries, does that mean we'll be writing the code for Timestamp Handling on our own or can we use the datetime library?
Per the docs: https://intel.github.io/dffml/contributing/git.html#issue-and-pull-request-title-formatting
operation(s): data: means this will live in dffml.git/operations/data which means you can use anything you want to get the job done.
If it's in dffml.git/dffml then we stick to Python STDLIB
Per the docs: https://intel.github.io/dffml/contributing/git.html#issue-and-pull-request-title-formatting
operation(s): data:means this will live indffml.git/operations/datawhich means you can use anything you want to get the job done.If it's in
dffml.git/dffmlthen we stick to Python STDLIB
@pdxjohnny okay great!