microsoft
[Feature] Support socks5 proxy with authentication
Playwright version
lastest
Operating system
No response
What browsers are you seeing the problem on?
No response
Other information
No response
What happened? / Describe the bug
_proxy = {
"server": f"{proxy.scheme}://{proxy.ip}:{proxy.port}",
"username": f"{proxy.username}",
"password": f"{proxy.password}"
}
this proxy is correct, i can use it via requests. but in playwright, it doesnt work . the doc says it support http or socks, how to use socks5 proxy ? thank yoy.
Code snippet to reproduce your bug
_proxy = {
"server": f"{proxy.scheme}://{proxy.ip}:{proxy.port}",
"username": f"{proxy.username}",
"password": f"{proxy.password}"
}
this proxy is correct, i can use it via requests. but in playwright, it doesnt work .
the doc says it support http or socks,
how to use socks5 proxy ?
thank yoy.
Relevant log output
No response
What kind of socks5 proxy are you using? Could you provide us e.g. a squid proxy configuration so we can repro?
Looking at the ecosystem:
- Chromium doesn't support it: https://issues.chromium.org/issues/40323993
- Firefox doesn't support it: https://bugzilla.mozilla.org/show_bug.cgi?id=122752
- WebKit networking stack for linux: libsoup seems to support it
- WebKit networking stack for Windows: libcurl seems to support it
- WebKit networking stack for macOS: cfnetwork seems to support it: kCFStreamPropertySOCKSUser
I think the dependency on this issue is Firefox and Chromium support. Without that its hard for us to drive this forward. I recommend having maybe a proxy between you and the actual proxy in the meantime.
Socks5 proxy w/ auth is not supported but major browsers or infrastructures. Could you share your use case?
import asyncio import requests from playwright.async_api import async_playwright
username = 'usertestme888' password = 'usertestme888' ip = '47.241.238.22' port = '2333'
async def mytest():
url = 'http://httpbin.org/ip'
proxies = {
'http': f'socks5://{username}:{password}@{ip}:{port}',
'https': f'socks5://{username}:{password}@{ip}:{port}'
}
res = requests.get(url, proxies=proxies, verify=False)
print(res.text)
async def main(): proxy = { "server": f'https://{ip}:{port}', "username": f"{username}", "password": f"{password}" }
user_data_dir = ''
playwright = await async_playwright().start()
browser = await playwright.chromium.launch_persistent_context(
user_data_dir,
proxy=proxy,
timeout=300000,
headless=False,
)
page = await browser.new_page()
await page.goto('https://baidu.com')
pass
if name == 'main':
asyncio.run(mytest())
asyncio.run(main())
pass
to those who keep posting the same comment - please just upvote the post at the top
for something productive here - I tried to bump the 'proxy-authorization' header, but ofcourse this triggers net::ERR_INVALID_ARGUMENT from inside puppeteer because overriding some headers is not supported
if self.proxy and self.proxy.get('server', '').lower().startswith('socks5://'):
import base64
auth_str = base64.b64encode(f"{self.proxy['username']}:{self.proxy['password']}".encode('utf8')).decode('ascii')
self.proxy['username'] = ''
self.proxy['password'] = ''
request_headers['proxy-authorization']= f"Basic {auth_str}"
https://github.com/puppeteer/puppeteer/issues/676 https://cri.dev/posts/2020-03-30-How-to-solve-Puppeteer-Chrome-Error-ERR_INVALID_ARGUMENT
However an interesting work-around is to use a local SOCKS proxy chain, so you bounce a local request to localhost socks server off to the real server, because SOCKS chains very well due to its low-level TCP socket type arrangement
https://medium.com/apifier-blog/how-to-make-headless-chrome-and-puppeteer-use-a-proxy-server-with-authentication-249a21a79212
I also tried self.page.evaluate('const aHandle = page.authenticate({"xxxx", "pppp"});') but that didnt work (unsure why) SyntaxError: Unexpected string at eval (<anonymous>) at UtilityScript.evaluate (<anonymous>:201:30) at UtilityScript.<anonymous> (<anonymous>:1:44)
