python-runstats icon indicating copy to clipboard operation
python-runstats copied to clipboard

Time Based ExponentialMovingStatistics and ExponentialCovariance

Open alanmazankiewicz opened this issue 4 years ago • 8 comments

Main features:

  • Changed ExponentialStatistics name to ExponentialMovingStatistics (Issue #28)
  • Implemented time-based discounting for ExponentialMovingStatistics (Issue #28) (incl. Readme)
  • Implemented ExponentialMovingCovariance (incl. Readme)
    • https://reasonabledeviations.com/2018/08/15/exponential-covariance/
    • https://vlab.stern.nyu.edu/docs/correlation/EWMA-COV
    • Equivalent of ExponetialMovingStatistics for Covariance and Correlation

Smaller fixes/changes/features:

  • Updated Readme for link to Finch Paper for ExponetialMovingStatistics
  • Adjusted clear method of ExponentialMovingStatistics: Goes back to state at object construction, does not accept any arguments for modification. I think that fits better to the behavior of the other classes. Modification is still possible over other methods.
  • Extended tests for ExponentialMovingStatistics against a true (non-moving) exponential mean and variance formulation

Problems:

I have done my best to resolve all issues that are raised by the CI pipeline. However, I could not resolve all of them. You've been just changing the structure of the project including the CI pipeline and the way Cython is compiled. Thus, I guess you should be able to identify the sources of these problems fairly quickly.

  • Integration test fails: Its the lines in the Readme where RunStats_ is used. I don't know why this is happening.
docs/index.rst:4: D000 Unknown target name: "runstats".
docs/index.rst:30: D000 Unknown target name: "runstats".
docs/index.rst:60: D000 Unknown target name: "runstats".
docs/index.rst:85: D000 Unknown target name: "runstats".
docs/index.rst:380: D000 Unknown target name: "runstats".
  • Locally running tox also complains in Readme about:
>>> import runstats.core   # Pure-Python
>>> runstats.core.Statistics
 <class 'runstats.core.Statistics'>
Expected:
    <class 'runstats.core.Statistics'>
Got:
    <class 'runstats._core.Statistics'>

When changing in the Readme <class 'runstats.core.Statistics'> to <class 'runstats._core.Statistics'> this does not resolve the issue. Since currently in the main repositories master branch its <class 'runstats.core.Statistics'> I left it that way.

  • Test coverage: When locally running tox -e py it tells me that there is 98% test coverage and reports missing lines
402->405, 405, 412->413, 413, 418->419, 419

These are the lines where Exceptions are raised in the new freeze() and unfreeze() methods. However, these are being tested by test_raise_if_not_time_exp_stats()

  • I'd recommend merging the PR onto another branch and checking this out yourself.

alanmazankiewicz avatar Feb 28 '21 20:02 alanmazankiewicz

Thanks. I’ll try to take a look this week.

grantjenks avatar Feb 28 '21 21:02 grantjenks

One thing I forgot to mention: I have not implemented the ExponentialMovingCovariance in a time-based fashion because I wanted to first get feedback from you on the how it is implemented for the ExponentialMovingStatistics.

alanmazankiewicz avatar Feb 28 '21 23:02 alanmazankiewicz

Hi @alanmazankiewicz 👋 I updated the testing and Cythonizing. Could you rebase and get the CI executions passing?

grantjenks avatar Jul 03 '21 05:07 grantjenks

Hi there :) Yes sure, I'll get back to you when I am done.

alanmazankiewicz avatar Jul 03 '21 07:07 alanmazankiewicz

Can you have a look at the pipeline failure of this PR: https://github.com/alanmazankiewicz/python-runstats/pull/1/checks?check_run_id=2978450007. Its a merge from my "rebase" branch onto my "master" branch. I created it to test the pipeline. The pipeline fails at "test (ubuntu-latests, 3.x)" with

 _pytest.pathlib.ImportPathMismatchError: ('runstats.core', '/home/runner/work/python-runstats/python-runstats/.tox/py/lib/python3.8/site-packages/runstats/core.py', PosixPath('/home/runner/work/python-runstats/python-runstats/runstats/core.py'))

Have you encountered that issue before? When I locally checkout the remote/upstream/master branch (your current version) and run tox -e py or tox I run into the same error. I found following issue in it: https://github.com/pytest-dev/pytest/issues/2042. Trying out the proposed solution (export PY_IGNORE_IMPORTMISMATCH=1 ) locally did not solve it.

alanmazankiewicz avatar Jul 03 '21 15:07 alanmazankiewicz

I'm not sure how it worked before but it should be fixed now on master. Please rebase.

grantjenks avatar Jul 04 '21 05:07 grantjenks

Pull Request Rebased. The Problems mentioned in the description are not present anymore.

alanmazankiewicz avatar Jul 13 '21 19:07 alanmazankiewicz

Did you had the chance to look at changes?

alanmazankiewicz avatar Oct 18 '21 21:10 alanmazankiewicz