apify-client-python icon indicating copy to clipboard operation
apify-client-python copied to clipboard

Address accessing of non-existing field `_maybe_parsed_body` within `httpx.Response` object

Open vdusek opened this issue 2 years ago • 4 comments

It seems that the following code:

response: httpx.Response = await self.http_client.call(
    url=self._url(f'records/{key}'),
    method='GET',
    params=self._params(),
)

returns a response of type httpx.Response. Later the field _maybe_parsed_body is accessed:

return {
    'value': response._maybe_parsed_body,
    '...': '...',
}

There are 2 occurrences of this:

  • https://github.com/apify/apify-client-python/blob/v1.6.0/src/apify_client/clients/resource_clients/key_value_store.py#L120
  • https://github.com/apify/apify-client-python/blob/v1.6.0/src/apify_client/clients/resource_clients/key_value_store.py#L323

Check if http_client.call really returns a httpx.Response object and in such case fix the accessing of non-existing field _maybe_parsed_body.

vdusek avatar Nov 20 '23 11:11 vdusek

http_client.call returns a httpx.Response which is in some cases enhanced with a _maybe_parsed_body. It's a bit of a hack which was really useful when we still used requests, and simplified the code a lot, but when we switched to using httpx and did some more changes, it ended up being used only in one method, when fetching a key-value store record body. So now we could put the "maybe" response parsing into that method, instead of having it in the http_client, to have a better separation of concerns, and less hacks.

fnesveda avatar Nov 27 '23 05:11 fnesveda

My IDE was not able to recognize that, instead it raised an accessing of non-existing field error. The self.http_client.call is annotated to return httpx.Response object. So it might be the source of the problem.

vdusek avatar Nov 27 '23 11:11 vdusek

Yeah, that's probably it. I wonder how this ever passed lint without a #noqa comment 🤷

fnesveda avatar Nov 27 '23 13:11 fnesveda

Yeah, that's probably it. I wonder how this ever passed lint without a #noqa comment 🤷

I think because accessing fields is checked by Mypy. That's the reason why there is # type: ignore .

vdusek avatar Nov 27 '23 14:11 vdusek