DataDog
Unable to patch aiobotocore 2.3.0+
Which version of dd-trace-py are you using?
1.1.2
Which version of pip are you using?
22.0.4
Which version of the libraries are you using?
# Relevant:
aiobotocore==2.3.2
# Probably irrelevant:
attrs==21.4.0
ddsketch==2.0.2
packaging==21.3
protobuf==3.20.1
tenacity==8.0.1
six==1.16.0
How can we reproduce your problem?
import ddtrace
ddtrace.patch(aiobotocore=True)
What is the result that you get?
/usr/local/lib/python3.10/site-packages/ddtrace/contrib/aiobotocore/patch.py:8: in <module>
from aiobotocore.endpoint import ClientResponseContentProxy
E ImportError: cannot import name 'ClientResponseContentProxy' from 'aiobotocore.endpoint' (/usr/local/lib/python3.10/site-packages/aiobotocore/endpoint.py)
During handling of the above exception, another exception occurred:
/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py:261: in _attempt_patch_module
imported_module = importlib.import_module(path)
/usr/local/lib/python3.10/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
???
<frozen importlib._bootstrap>:1027: in _find_and_load
???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:688: in _load_unlocked
???
/usr/local/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:168: in exec_module
exec(co, module.__dict__)
/usr/local/lib/python3.10/site-packages/ddtrace/contrib/aiobotocore/__init__.py:28: in <module>
from .patch import patch
/usr/local/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:168: in exec_module
exec(co, module.__dict__)
/usr/local/lib/python3.10/site-packages/ddtrace/contrib/aiobotocore/patch.py:11: in <module>
from aiobotocore._endpoint_helpers import ClientResponseContentProxy
E ImportError: cannot import name 'ClientResponseContentProxy' from 'aiobotocore._endpoint_helpers' (/usr/local/lib/python3.10/site-packages/aiobotocore/_endpoint_helpers.py)
During handling of the above exception, another exception occurred:
/usr/local/lib/python3.10/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
???
<frozen importlib._bootstrap>:1027: in _find_and_load
???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:688: in _load_unlocked
???
/usr/local/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:168: in exec_module
exec(co, module.__dict__)
tests/conftest.py:17: in <module>
from apps import webhook_listener
apps.py:15: in <module>
ddtrace.patch(aiobotocore=True)
/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py:194: in patch
_patch_module(module, raise_errors=raise_errors)
/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py:223: in _patch_module
return _attempt_patch_module(module)
/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py:264: in _attempt_patch_module
raise ModuleNotFoundException(
E ddtrace._monkey.ModuleNotFoundException: integration module ddtrace.contrib.aiobotocore does not exist, module will not have tracing available
What is the result that you expected?
No error when patching aiobotocore.
Current understanding of issue:
aiobotocore version 2.3.0 removed aiobotocore._endpoint_helpers.ClientResponseContentProxy which is imported here:
https://github.com/DataDog/dd-trace-py/blob/8001193d73552fdf7442da290f70b124128a6a82/ddtrace/contrib/aiobotocore/patch.py#L7-L11
@meshy Thank you for the detailed issue.
Upon first look, the removal should not be too difficult to support in the aiobotocore integration. Rather than the response_class being set to ClientResponseContentProxy as a wrapper, an AioAWSResponse object is instantiated with the necessary fields. We only need the length of the response content.
@meshy If you want to submit a fix as a PR, we would be happy to work with you. Otherwise, it is something our team will prioritize to get out as a bug fix.
@majorgreys Thank you for the quick response, and your consideration.
I'm unable to prioritise this at the moment, but I'd be happy to test that any patch your team produces works on my end, when it's available.
Thanks for the report! Support for aiobotocore was added in #3849 and will be released this week in v1.5 🙂
