dvc
dvc copied to clipboard
iterative
Can't instantiate abstract class AsyncPayload with abstract methods decode. When I push to oss
Bug Report
Can't instantiate abstract class AsyncPayload with abstract methods decode.
Issue name
I can't push my data to oss.
Description
dvc push --verbose
2024-10-15 15:03:18,167 DEBUG: v3.42.0 (pip), CPython 3.8.16 on Windows-10-10.0.22631-SP0
2024-10-15 15:03:18,167 DEBUG: command: D:\anaconda3\envs\work38\Scripts\dvc push --verbose
Collecting |0.00 [00:00, ?entry/s]
2024-10-15 15:03:18,761 DEBUG: Preparing to transfer data from 'E:\dvc_test\.dvc\cache\files\md5' to 'oss://sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,761 DEBUG: Preparing to collect status from 'sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,762 DEBUG: Collecting status from 'sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,763 DEBUG: Querying 1 oids via object_exists
2024-10-15 15:03:19,578 DEBUG: Estimated remote size: 4096 files
2024-10-15 15:03:19,579 DEBUG: Large remote (4 oids < 40.96 traverse weight), using object_exists for remaining oids
2024-10-15 15:03:19,580 DEBUG: Querying 4 oids via object_exists
2024-10-15 15:03:19,774 DEBUG: Preparing to collect status from 'E:\dvc_test\.dvc\cache\files\md5'
2024-10-15 15:03:19,774 DEBUG: Collecting status from 'E:\dvc_test\.dvc\cache\files\md5'
2024-10-15 15:03:19,865 DEBUG: transfer dir: md5: fdde843b2edc481dd70d4032e35d6226.dir with 3 files
2024-10-15 15:03:20,251 ERROR: failed to transfer '4186881925838e1c3136150473324c01' - Can't instantiate abstract class AsyncPayload with abstract methods decode
Traceback (most recent call last):
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 330, in transfer
_try_links(
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 262, in _try_links
return copy(
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 87, in copy
return _put(
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 156, in _put
return _put_one(from_paths[0], to_paths[0])
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 148, in _put_one
return put_file(from_path, to_path, callback=callback, **put_file_kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\callbacks.py", line 146, in sync_wrapper
res = fn(*args, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\callbacks.py", line 164, in sync_wrapper
return fn(path1, path2, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\base.py", line 643, in put_file
self.fs.put_file(os.fspath(from_file), to_info, callback=callback, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 103, in sync
raise return_result
File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 56, in _runner
result[0] = await coro
File "D:\anaconda3\envs\work38\lib\site-packages\ossfs\async_oss.py", line 396, in _put_file
await self._call_oss(
File "D:\anaconda3\envs\work38\lib\site-packages\ossfs\async_oss.py", line 162, in _call_oss
out = await method(*args, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 581, in put_object_from_file
return await self.put_object(
File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 291, in put_object
resp: "AioResponse" = await self._do_object(
File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 254, in _do_object
return await self._do(method, self.bucket_name, key, **kwargs)
File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 144, in _do
resp: "AioResponse" = await self.session.do_request(req, timeout=self.timeout)
File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\http.py", line 150, in do_request
resp = await self.session.request(
File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client.py", line 631, in _request
req = self._request_class(
File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client_reqrep.py", line 338, in __init__
self.update_body_from_data(data)
File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client_reqrep.py", line 588, in update_body_from_data
body = payload.PAYLOAD_REGISTRY.get(body, disposition=None)
File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\payload.py", line 116, in get
return factory(data, *args, **kwargs)
TypeError: Can't instantiate abstract class AsyncPayload with abstract methods decode
dvc doctor
DVC version: 3.42.0 (pip)
-------------------------
Platform: Python 3.8.16 on Windows-10-10.0.22631-SP0
Subprojects:
dvc_data = 3.8.0
dvc_objects = 3.0.6
dvc_render = 1.0.1
dvc_task = 0.40.2
scmrepo = 2.0.4
Supports:
http (aiohttp = 3.10.10, aiohttp-retry = 2.8.3),
https (aiohttp = 3.10.10, aiohttp-retry = 2.8.3),
oss (ossfs = 2023.12.0)
Config:
Global: C:\Users\Sigma Squares\AppData\Local\iterative\dvc
System: C:\ProgramData\iterative\dvc
Cache types: hardlink
Cache directory: NTFS on E:\
Caches: local
Remotes: oss
Workspace directory: NTFS on E:\
Repo: dvc, git
Repo.site_cache_dir: C:\ProgramData\iterative\dvc\Cache\repo\aee995bc5ba11f246beffb6ea62fed52
I tried both Windows and Linux systems, but they didn't work. I can manually upload cache to OSS and pull it locally, but I cannot push it to OSS through DVC
I downgraded the aiohttp version from 3.10.10 to 3.9.5, which temporarily resolved this issue. However, compatibility between DVC and the new version of aiohttpis still an issue, and the released PyInstaller packaged version of Windows still cannot be used properly.
Hey folks!
It is actually an error in aiooss2 library (which is transitive dependency). There is PR to it with fix: https://github.com/karajan1001/aiooss2/pull/97
If you're affected by this issue, please consider giving the PR a 👍 It may not be merged per se, but it's definitely adds some attention there :)
