dffml icon indicating copy to clipboard operation
dffml copied to clipboard

operation: data: Add/Update operations to handle time series data

Open programmer290399 opened this issue 3 years ago • 3 comments

Pain Point

We do not have operations to work with time-series data

Proposed Solution

Implement/Update operations for the following tasks:

  • [ ] NaN Removal
  • [ ] Outlier Removal
  • [ ] Missing Timestamp Handling
  • [ ] Scaling
  • [ ] Imputation & Interpolation
  • [ ] Shifting/lag generation
  • [ ] Time Series Decomposition

Note: Some Topics may overlap with others in the list above, we need to make our operations as generic as possible. So that they can work with not only time-series data but other data formats as well if possible.

programmer290399 avatar Mar 10 '22 01:03 programmer290399

Hey, @programmer290399 You mentioned earlier that you guys generally avoid using 3rd party libraries, does that mean we'll be writing the code for Timestamp Handling on our own or can we use the datetime library?

TirelessClock avatar Apr 10 '22 12:04 TirelessClock

Per the docs: https://intel.github.io/dffml/contributing/git.html#issue-and-pull-request-title-formatting

operation(s): data: means this will live in dffml.git/operations/data which means you can use anything you want to get the job done.

If it's in dffml.git/dffml then we stick to Python STDLIB

johnandersen777 avatar May 05 '22 20:05 johnandersen777

Per the docs: https://intel.github.io/dffml/contributing/git.html#issue-and-pull-request-title-formatting

operation(s): data: means this will live in dffml.git/operations/data which means you can use anything you want to get the job done.

If it's in dffml.git/dffml then we stick to Python STDLIB

@pdxjohnny okay great!

TirelessClock avatar May 06 '22 02:05 TirelessClock