sagemaker-distribution
sagemaker-distribution copied to clipboard
s3fs unit test broken before commit tagged by 2025.2.0
s3fs is one of our marquee dependency introduced in 2.3.0, its unit test keep failing due to error:
s3 = <s3fs.core.S3FileSystem object at 0x7fa76ea4f190>
def test_info(s3):
s3.touch(a)
s3.touch(b)
info = s3.info(a)
linfo = s3.ls(a, detail=True)[0]
assert abs(info.pop("LastModified") - linfo.pop("LastModified")).seconds < 1
info.pop("VersionId")
info.pop("ContentType")
linfo.pop("Key")
linfo.pop("Size")
> assert info == linfo
E assert {'ETag': '"d4...size': 0, ...} == {'ChecksumAlg.../test/a', ...}
E
E Omitting 5 identical items, use -vv to show
E Right contains 1 more item:
E {'ChecksumAlgorithm': ['CRC32']}
E Use -v to get more diff
test_s3fs.py:271: AssertionError
The failure is due to a new change introduced since: https://github.com/fsspec/s3fs/blame/2025.2.0/s3fs/tests/test_s3fs.py#L272
There is no actual functionality issue, but we cannot fix this unit test failure until a new major version bump to upgrade s3fs version.
We may also consider not follow semiver rule for s3fs, as its version represents timestamp only.