arrow icon indicating copy to clipboard operation
arrow copied to clipboard

ARROW-17893: [Python] Test that reading of timedelta is stable (read_feather/to_pandas)

Open AlenkaF opened this issue 3 years ago • 3 comments

AlenkaF avatar Oct 27 '22 13:10 AlenkaF

https://issues.apache.org/jira/browse/ARROW-17893

github-actions[bot] avatar Oct 27 '22 14:10 github-actions[bot]

Can you move this test to test_pandas.py? Similarly as the other timestamp_as_object issue, I think the issue here is in the arrow->pandas conversion, and reading a feather file is only a way to get there. But so it should also be possible to test this with just creating a pyarrow.Table and converting to pandas.

jorisvandenbossche avatar Oct 27 '22 16:10 jorisvandenbossche

There is also an issue when comparing timedelta in the old version of pandas I need to look into:

>       assert result["timedelta_2"].item() == timedelta_2
E       assert 13223000000000 == datetime.timedelta(seconds=13223)
E        +  where 13223000000000 = <bound method IndexOpsMixin.item of 0   03:40:23\nName: timedelta_2, dtype: timedelta64[ns]>()
E        +    where <bound method IndexOpsMixin.item of 0   03:40:23\nName: timedelta_2, dtype: timedelta64[ns]> = 0   03:40:23\nName: timedelta_2, dtype: timedelta64[ns].item

AlenkaF avatar Oct 28 '22 04:10 AlenkaF

Benchmark runs are scheduled for baseline = 147b5c922efe19d34ef7e7cda635b7d8a07be2eb and contender = 0ef4510dffd31849aa4cfcef661f591a4360fe4c. 0ef4510dffd31849aa4cfcef661f591a4360fe4c is a master commit associated with this PR. Results will be available as each benchmark for each run completes. Conbench compare runs links: [Finished :arrow_down:0.0% :arrow_up:0.0%] ec2-t3-xlarge-us-east-2 [Finished :arrow_down:0.1% :arrow_up:0.0%] test-mac-arm [Finished :arrow_down:0.54% :arrow_up:0.0%] ursa-i9-9960x [Finished :arrow_down:0.35% :arrow_up:0.0%] ursa-thinkcentre-m75q Buildkite builds: [Finished] 0ef4510d ec2-t3-xlarge-us-east-2 [Finished] 0ef4510d test-mac-arm [Finished] 0ef4510d ursa-i9-9960x [Finished] 0ef4510d ursa-thinkcentre-m75q [Finished] 147b5c92 ec2-t3-xlarge-us-east-2 [Finished] 147b5c92 test-mac-arm [Finished] 147b5c92 ursa-i9-9960x [Finished] 147b5c92 ursa-thinkcentre-m75q Supported benchmarks: ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True test-mac-arm: Supported benchmark langs: C++, Python, R ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

ursabot avatar Nov 10 '22 04:11 ursabot