sagemaker-distribution icon indicating copy to clipboard operation
sagemaker-distribution copied to clipboard

s3fs unit test broken before commit tagged by 2025.2.0

Open TRNWWZ opened this issue 1 year ago • 0 comments

s3fs is one of our marquee dependency introduced in 2.3.0, its unit test keep failing due to error:

s3 = <s3fs.core.S3FileSystem object at 0x7fa76ea4f190>

    def test_info(s3):
        s3.touch(a)
        s3.touch(b)
        info = s3.info(a)
        linfo = s3.ls(a, detail=True)[0]
        assert abs(info.pop("LastModified") - linfo.pop("LastModified")).seconds < 1
        info.pop("VersionId")
        info.pop("ContentType")
        linfo.pop("Key")
        linfo.pop("Size")
>       assert info == linfo
E       assert {'ETag': '"d4...size': 0, ...} == {'ChecksumAlg.../test/a', ...}
E         
E         Omitting 5 identical items, use -vv to show
E         Right contains 1 more item:
E         {'ChecksumAlgorithm': ['CRC32']}
E         Use -v to get more diff

test_s3fs.py:271: AssertionError

The failure is due to a new change introduced since: https://github.com/fsspec/s3fs/blame/2025.2.0/s3fs/tests/test_s3fs.py#L272

There is no actual functionality issue, but we cannot fix this unit test failure until a new major version bump to upgrade s3fs version.

We may also consider not follow semiver rule for s3fs, as its version represents timestamp only.

TRNWWZ avatar Feb 05 '25 06:02 TRNWWZ