nodejs
SocketError: other side closed (with repro example)
Bug Description
SocketError: other side closed
Reproducible By
https://github.com/nordluf/fetch-socket-closed-example Here is the complete example
Expected Behavior
I expect fetch to stream any amount of data after any number of requests
Logs & Screenshots
TypeError: fetch failed
at node:internal/deps/undici/undici:12502:13
at async streamFetch (file:///srv/www/example.mjs:52:18)
at async file:///srv/www/example.mjs:16:1 {
[cause]: SocketError: other side closed
at Socket.
Environment
Node.js v20.13.1, Linux
Additional context
There are plenty of other users affected by this issue
I am also getting this error on Next.js 14 fetching data within a server component within a Suspense boundary. Error does not occur with regular (non-streamed) fetches (i.e. fetches not made within Suspense boundaries).
Hi! I have the exact same issue right now on node 20.15.
It seems to fail randomly. When it fails, the failed request doesn't even reach its destination. In my case I control both the client and the API, and the failed requests doesn't appear anywhere in the API logs.
I've seen numerous issues mentioning this, all closed without a solution. We're here to provides more info or context if needed.
Ran into this issue as well on v20 and also on v22 in a NextJS build where during buildtime a lot of fetches are made. Downgrading to v18 seemed to work, interestingly enough.
I still see the issue using node 18. You can change image: node:18-slim in docker-compose.yaml to see the failing output
The issue seems to root to the actual server. I've tried to reproduce your example, and although i reproduce it successfully, the issue goes to the server closing the connection.
If the server closes the connection, this will inevitably lead to undici not being able to send anymore request.
Currently, agents does not reconnect by themselves, so a new connection needs to be attempted.
Of course, if the server closes the connection the undici(fetch) is not guilty This is the smallest (and easiest to reuse by others) possible example of reproducing the issue. I have seen the error from undici (fetch) even with other HTTP servers like nginx. It always happens after fetching a lot of data.
If you need any other type of support or any other data from my side to be shared - feel free to ping me
Can you provide an Minimum Reproducible Example without Express to reproduce this?
I'd like to have something closer to what you are seeing to debug it better.
The issue seems to root to undici itself, not to fetch; but want to be sure what can be causing the problem
If you call res.write without a callback then you are basically flooding the socket without any flow control. Basically the server was crashing.
people who claim to run in a similar issue have probably a different issue where the SocketError is thrown.
'use strict'
const { fetch } = require('./undici-fetch.js');
const { Readable } = require('node:stream');
const { once } = require('node:events');
const { createServer } = require('node:http');
const { promisify } = require('node:util');
async function init() {
const server = createServer(async (req, res) => {
const url = new URL(req.url, `http://example.org`);
res.writeHead(200, { 'Content-Type': 'text/plain' });
const size = +url.searchParams.get('size');
const count = +url.searchParams.get('count')
const strObj = Buffer.allocUnsafe(+size).toString('ascii')
const resWrite = promisify(res.write.bind(res));
await resWrite('[');
for (let i = 0; i < count; i++) {
await resWrite(strObj + ',');
}
await resWrite(strObj);
await resWrite(']');
res.end();
})
server.listen(3000);
await once(server, 'listening');
return server;
}
async function streamFetch(size, count) {
const result = await fetch(`http://127.0.0.1:3000?size=${size}&count=${count}`, { isStream: true });
if (!result.ok) {
throw new Error(`HTTP error! Status: ${result.status}`);
}
if (!result.body) {
throw new Error('No result body!');
}
const stream = Readable.fromWeb(result.body, { encoding: 'utf8' });
console.log('Request sent.');
let kb = 0, lastkb = 0;
return new Promise((resolve, reject) => {
stream.on('error', reject);
stream.on('end', resolve);
stream.on('data', (chunk) => {
kb += chunk.length;
if (kb - lastkb > 10 * 1024 * 1024) {
console.log('Another 10M received');
lastkb = kb;
// console.log(chunk)
}
});
});
}
async function run() {
const server = await init();
await streamFetch(1024, 100);
await streamFetch(10240, 1000);
await streamFetch(10240, 10000);
await streamFetch(10240, 100000);
await streamFetch(10240, 1000000);
await streamFetch(10240, 10000000);
server.close();
}
run()
always happening occasionally and randomly with node.js fetch, with any server... never happens with deno.
I ran into this issue because my local setup was using HTTPS, but the requests were being made over HTTP. I have a proxy in my Next.js app that forwards requests to an external API, while attaching the auth cookies. For some reason, the requests weren’t being sent over HTTPS, which caused the problem.
Once I made sure the requests were being forwarded to the proxy with HTTPS, everything started working fine.
workaround is to retry fetch on failure. you can use an http library or a simple fetch wrapper like fetch-retry on npm (or write your own in a few lines)
