scholarly icon indicating copy to clipboard operation
scholarly copied to clipboard

SingleProxy: Failed to create httpx.Client in Python 3.13

Open EgodPrime opened this issue 4 months ago • 0 comments

Describe the bug A clear and concise description of what the bug is.

To Reproduce

from scholarly import ProxyGenerator

pg = ProxyGenerator()
http_proxy_url = "http://anystr"
success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)

The above code outputs:

Traceback (most recent call last):
  File "/root/tracefuzz/test_scholarly.py", line 5, in <module>
    success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 124, in SingleProxy
    proxy_works = self._use_proxy(http=http, https=https)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 215, in _use_proxy
    self._new_session(proxies=proxies)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 491, in _new_session
    self._session = httpx.Client(**init_kwargs)
                    ~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'proxies'

Expected behavior success should be True.

Screenshots

Image

In this Python version, httpx.Client receives proxy (URL|str|Proxy) rather than proxies

Desktop (please complete the following information):

  • Proxy service: SingleProxy
  • python version: [3.13.4]
  • OS: [Ubuntu 22.04.5 LTS]
  • Version [1.7.11]

Do you plan on contributing? No, but I think it's a piece of cake to fix this issue, so the developer can make it.

# scholarly/_proxy_generator.py
# ...
if self._proxy_works:
            init_kwargs["proxies"] = proxies #.get("http", None)
            """==================================
                Add code like this
            =================================="""
            if sys.version_info >= (3, 13):
                init_kwargs["proxy"] = proxies.get("http") 
                init_kwargs.pop("proxies", None) 
             """==================================
             =================================="""
            self._proxies = proxies
            if self.proxy_mode is ProxyMode.SCRAPERAPI:
                # SSL Certificate verification must be disabled for
                # ScraperAPI requests to work.
                # https://www.scraperapi.com/documentation/
                init_kwargs["verify"] = False
        self._session = httpx.Client(**init_kwargs)
        self._webdriver = None

EgodPrime avatar Oct 15 '25 08:10 EgodPrime