atoti icon indicating copy to clipboard operation
atoti copied to clipboard

Cumulative over window

Open HuifangYeo opened this issue 4 years ago • 7 comments

Steps to reproduce

I have created a rolling windows in Pandas:

t = temp.rolling("7D", on="trans_date")["amt"].count()

It gives me the below:

The 7 days is inclusive of the current member's value.

However, in atoti, the 7D is the 7 previous days + the current member, i.e. 8 days worth of value:

I'm aware that the documentation did indicate it's the previous number of days, but is it better to align it with Pandas which most people are acquainted with? I.e. 7 days refers to 6 previous days + current days.

Environment

  • atoti: 0.6.2
  • Python: 3.8.12
  • Operating System: win32

HuifangYeo avatar Nov 01 '21 07:11 HuifangYeo

Could you help to explain how the "-1D" works?
If we look at the previous day plus current day, then on 2021-09-06 (blue), we should be having 3 and not 5.
On 2021-09-07, shouldn't the first row be 2?

image

HuifangYeo avatar Nov 01 '21 10:11 HuifangYeo

Hi Hui Fang, I cannot reproduce this with a simple example on my side. Can you provide a minimal reproduction case please?

tibdex avatar Nov 02 '21 16:11 tibdex

Hi @tibdex,

sorry for the delay: Untitled.zip

HuifangYeo avatar Nov 25 '21 06:11 HuifangYeo

Indeed this is strange and not the same as excel or pandas :

m["Cumulative datetime"] = tt.agg.sum(
    m["contributors.COUNT"],
    scope=tt.scope.cumulative(l["datetime"], window=("-1D", None)),
)

atoti: image pandas: image excel: image

image

bloussou avatar Jan 31 '22 10:01 bloussou

atoti result for -1D is the same as pandas for 2d : image

bloussou avatar Jan 31 '22 10:01 bloussou

Hello, The next release will support cumulative window like pandas and Excel.

bloussou avatar Jun 27 '22 11:06 bloussou

Hi,

I just tried the CumulativeScope with mean aggregation:

m["Moving_average_1m"] = tt.agg.mean(
    m["Weighted average price"],
    scope=tt.CumulativeScope(level=l["time_1m"], window=range(-4, 0)),
)

It is accumulating over window of 5 instead of the 4 that I have intended: image

sample code: cumulative.zip

HuifangYeo avatar Mar 14 '23 05:03 HuifangYeo