crawlee-python
crawlee-python copied to clipboard
Curl Cffi Client
Since we use HTTPX , Isn't better to use curl_cffi instead or use it an optional client that switches when the HTTPX gets defeated in the AntiBot war.
I am thinking about intelligently switching between multiple http clients. (The same as of proxy rotation)
Hi, thanks for the issue. I think we can split this one into two parts. First, we can (probably, needs further investigation) switch from httpx to curl_cffi, which seems better meet our use-case. The second part would be the switching, which should be part of a separate issue/discussion.