node-libcurl Proxy Doesn't Change

Hi, No matter how many request.session objects I create in the same Node.js process, it always uses the initially assigned proxy and ignores other proxies. Even if I pass a new proxy as a parameter when doing session.get, it still uses the first proxy.

const proxies = [
    {
        proxy: 'http://localhost:3000',
        username: 'user1',
        password: 'password1',
    },
    {
        proxy: 'http://localhost:3001',
        username: 'user2',
        password: 'password2',
    },
    {
        proxy: 'http://localhost:3002',
        username: 'user3',
        password: 'password3',
    },
];


for (const proxy of proxies) {
    const session = requests.session({
        proxy: proxy,
    });

    const response = session.get('https://ipinfo.io/json');
    console.log(response.json);

    const response2 = session.get('https://ipinfo.io/json', {
        proxy: proxies[1],
    });
    console.log(response2.json);

}

// Output:
// ip from proxy 0
// ip from proxy 0

// ip from proxy 0
// ip from proxy 0

// ip from proxy 0
// ip from proxy 0

Oct 19 '24 05:10 mehmetuken

Looks like a socket connect reuse problem ...

Oct 19 '24 06:10 Ossianaa

This issue is resolved, but in cases like Promise.all, it still uses the first assigned proxy.

const proxies = [
    {
        proxy: 'http://localhost:3000',
        username: 'user1',
        password: 'password1',
    },
    {
        proxy: 'http://localhost:3001',
        username: 'user2',
        password: 'password2',
    },
    {
        proxy: 'http://localhost:3002',
        username: 'user3',
        password: 'password3',
    },
];


async function sendRequest(proxy) {
    const session = requests.session({
        proxy: proxy,
    });

    const response = session.get('https://ipinfo.io/json');
    console.log(response.json);

    const response2 = session.get('https://ipinfo.io/json', {
        proxy: proxies[1],
    });
    console.log(response2.json);
}

const promises = proxies.map(proxy => sendRequest(proxy));
await Promise.all(promises);

Oct 19 '24 06:10 mehmetuken

give me a code

Oct 19 '24 06:10 Ossianaa

I edited and added the comment above.

Oct 19 '24 06:10 mehmetuken

look like CURLOPT_PIPEWAIT cause problem

Oct 19 '24 06:10 Ossianaa

It's solved. Thank you so much!

Oct 19 '24 07:10 mehmetuken

So, it is now impossible to reuse TCP connections, right? I experienced similar issue with proxies and was needed to set up connectReuse: false to solve it.

Oct 21 '24 09:10 iamoskvin

yes, If it is reuse connect, it may increase performance but it will do the wrong thing

Oct 21 '24 09:10 Ossianaa

Thanks. Is it possible to make curl reuse connects but with proper proxy handling? We need to take into account not only proxy host/port but also login and password (because proxy server can switch an external IP based on login/pwd). The curl does not do this by default, as far as I understand: it reuses a connection if it has the same host and password.

Oct 21 '24 09:10 iamoskvin

I will see if there is a better way, and if you have a solution, pr please :)

Oct 21 '24 09:10 Ossianaa

Thanks. I don't have a better solution currently.

Oct 21 '24 09:10 iamoskvin

If it was the username and password that caused the proxy reuse. I should be able to easily fix it and turn connectReuse back on

Oct 21 '24 09:10 Ossianaa

In the error I encountered, there were the same IP:PORT with different username and password combinations.

Oct 21 '24 09:10 mehmetuken

But I see in your code that you used different proxy ports: 3000, 3001, 3002.

Oct 21 '24 09:10 iamoskvin

You disabled CURLPIPE_MULTIPLEX to prevent issues with proxies. Do I understand correctly that the issues occur only in the case of proxies with user/password. If I use passwordless proxies with different hosts/ports, then it looks like there is no issue with connection reuse, right? Is connection pool bound to curllib instance of is common for all instances? Thank you.

Oct 25 '24 11:10 iamoskvin

it looks like there is no issue with connection reuse, right? I did not test the proxy for all cases, but disabling it CURLPIPE_MULTIPLEX that the proxy is not reused under http2

Is connection pool bound to curllib instance of is common for all instances? yes, all instance are bound pool

https://github.com/Ossianaa/node-libcurl/blob/95afc7655bdfd5540fd6e04e8ef20eccbe2bd8bf/src/libcurl/bao_curl_node_addon.cpp#L708

Oct 25 '24 11:10 Ossianaa

There may be a better solution but I've disabled all the relevant ones for now

Oct 25 '24 11:10 Ossianaa

work in v1.7.8

May 31 '25 20:05 Ossianaa