VeNoMouS
create_scraper doesn't work with arguments
Hello, I get a TypeError every time I use create_scraper with any arguments
from cloudscraper import create_scraper
url = 'https://example.com'
proxie_list = ["###", "###"]
scraper = create_scraper(
rotating_proxies=proxie_list,
proxy_options={
'rotation_strategy': 'smart',
'ban_time': 300
},
interpreter='js2py',
delay=5,
enable_stealth=True
)
response = scraper.get(url=url)
print(response)
Output:
Traceback (most recent call last):
File "D:\project\pythonProject\smh\test.py", line 28, in
I use python 3.12.3 (3.8 or 3.9 doesn't work either) Any help?
@apostos are you able to install the latest version using pip? I am getting this error:
pip install 'cloudscraper>=3.0.0'
ERROR: Could not find a version that satisfies the requirement cloudscraper>=3.0.0 (from versions: 1.1.13, 1.1.14, 1.1.15, 1.1.16, 1.1.17, 1.1.18, 1.1.19, 1.1.20, 1.1.23, 1.1.24, 1.1.25, 1.1.26, 1.1.27, 1.1.28, 1.1.29, 1.1.30, 1.1.31, 1.1.32, 1.1.33, 1.1.34, 1.1.36, 1.1.39, 1.1.40, 1.1.41, 1.1.42, 1.1.43, 1.1.45, 1.1.46, 1.1.47, 1.2.0, 1.2.1, 1.2.2, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.2.11, 1.2.12, 1.2.13, 1.2.14, 1.2.16, 1.2.18, 1.2.20, 1.2.22, 1.2.23, 1.2.24, 1.2.26, 1.2.28, 1.2.30, 1.2.33, 1.2.34, 1.2.36, 1.2.38, 1.2.40, 1.2.42, 1.2.44, 1.2.46, 1.2.48, 1.2.50, 1.2.52, 1.2.54, 1.2.56, 1.2.58, 1.2.60, 1.2.61, 1.2.62, 1.2.63, 1.2.64, 1.2.65, 1.2.66, 1.2.67, 1.2.68, 1.2.69, 1.2.70, 1.2.71)
ERROR: No matching distribution found for cloudscraper>=3.0.0
Edit: I have been able to install the latest version using
pip install 'git+https://github.com/VeNoMouS/[email protected]#egg=cloudscraper'
But this should not be the case. The pip version and/or the documentation needs to be updated.
i did not deploy it to pip as the PR was a mess and broken and I'm rewriting what was pushed
Oh, OK. can you maybe create some issues where you may need help so we can also help contribute?
i did not deploy it to pip as the PR was a mess and broken and I'm rewriting what was pushed
Whoever made the PR seems to have used a lot of AI.
Edit: I have been able to install the latest version using
pip install 'git+https://github.com/VeNoMouS/[email protected]#egg=cloudscraper' But this should not be the case. The pip version and/or the documentation needs to be updated.
I'm not able install with that command
LAEY@DESKTOP-HB49UN0 MINGW64 ~
$ pip install 'git+https://github.com/VeNoMouS/[email protected]#egg=cloudscraper'
Collecting cloudscraper
Cloning https://github.com/VeNoMouS/cloudscraper.git (to revision 3.0.0) to c:\users\laey\appdata\local\temp\pip-install-8k10spp_\cloudscraper_83ddec773f2841e9808e1b761ce03da0
Running command git clone --filter=blob:none --quiet https://github.com/VeNoMouS/cloudscraper.git 'C:\Users\LAEY\AppData\Local\Temp\pip-install-8k10spp_\cloudscraper_83ddec773f2841e9808e1b761ce03da0'
Resolved https://github.com/VeNoMouS/cloudscraper.git to commit 9ea528a8675f1bebd49ff853d142e94988a95178
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [21 lines of output]
Traceback (most recent call last):
File "C:\Users\LAEY\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 389, in <module>
main()
File "C:\Users\LAEY\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\LAEY\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 143, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\LAEY\AppData\Local\Temp\pip-build-env-ft6p988k\overlay\Lib\site-packages\setuptools\build_meta.py", line 331, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\LAEY\AppData\Local\Temp\pip-build-env-ft6p988k\overlay\Lib\site-packages\setuptools\build_meta.py", line 301, in _get_build_requires
self.run_setup()
File "C:\Users\LAEY\AppData\Local\Temp\pip-build-env-ft6p988k\overlay\Lib\site-packages\setuptools\build_meta.py", line 317, in run_setup
exec(code, locals())
File "<string>", line 7, in <module>
File "C:\Users\LAEY\AppData\Local\Programs\Python\Python312\Lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 19267: character maps to <undefined>
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
I had similar issues myself. pip install cloudscraper kept installing an outdated version, and when I tried installing directly from GitHub, I got the subprocess-exited-with-error message. But I managed to fix it by doing the following:
Uninstalled the old version:
pip uninstall cloudscraper
Then I installed the latest version directly from GitHub:
pip install "git+https://github.com/VeNoMouS/cloudscraper.git"
However, after that I got this error:
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
└─> [21 lines of output]
I’m using Windows 10, and an AI assistant suggested setting this environment variable. It solved the installation problem for me:
For CMD:
set PYTHONUTF8=1
For PowerShell:
$env:PYTHONUTF8=1
After that, version 3.0.0 installed correctly.
