node icon indicating copy to clipboard operation
node copied to clipboard

fs: make `FileHandle.readableWebStream` always create byte streams

Open isker opened this issue 1 year ago • 1 comments

The original implementation of the experimental FileHandle.readableWebStream API created non-type: 'bytes' streams, which prevented callers from creating mode: 'byob' readers from the returned stream, which means they could not achieve the associated "zero-copy" performance characteristics.

Then, #46933 added a parameter allowing callers to pass the type parameter down to the ReadableStream constructor, exposing the same semantics to callers of FileHandle.readableWebStream.

But there is no point to giving callers this choice: FileHandle-derived streams are by their very nature byte streams. We should not require callers to explicitly opt in to having byte stream semantics. Moreover, I do not see a situation in which callers would ever want to have a non-bytes stream: bytes-streams only do anything differently than normal ones if mode: 'byob' is passed to getReader.

So, remove the options parameter and always create a ReadableStream with type: 'bytes'.

Fixes #54041.

isker avatar Oct 19 '24 23:10 isker

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.41%. Comparing base (5e5af29) to head (41c448c). Report is 800 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #55461      +/-   ##
==========================================
- Coverage   88.41%   88.41%   -0.01%     
==========================================
  Files         652      654       +2     
  Lines      186918   187855     +937     
  Branches    36077    36145      +68     
==========================================
+ Hits       165271   166095     +824     
- Misses      14901    15000      +99     
- Partials     6746     6760      +14     
Files with missing lines Coverage Δ
lib/internal/fs/promises.js 98.23% <100.00%> (-0.01%) :arrow_down:

... and 179 files with indirect coverage changes

codecov[bot] avatar Oct 20 '24 01:10 codecov[bot]

Assuming this lands in 23.X.0, we can remove the warning in 23.(X + 1).0

Although, because this is experimental, a warning isn't technically required, so you have the final say (IMO) whether there is one to add

avivkeller avatar Nov 07 '24 02:11 avivkeller

@RedYetiDev can you take another look?

isker avatar Nov 14 '24 00:11 isker

@nodejs/fs please take a look. Thanks!

isker avatar Dec 19 '24 15:12 isker

Maybe someone from @nodejs/streams could review?

isker avatar Dec 27 '24 15:12 isker

Thanks @jasnell. @avivkeller stated on slack that they don't think they're qualified to approve but I don't know how to withdraw the review request here. Besides that, I think there is nothing else here blocking a merge.

isker avatar Dec 28 '24 20:12 isker

Thanks!

isker avatar Dec 28 '24 20:12 isker

CI: https://ci.nodejs.org/job/node-test-pull-request/64256/

nodejs-github-bot avatar Dec 29 '24 18:12 nodejs-github-bot

Hi @jasnell, is there anything else I need to do to land this PR?

isker avatar Jan 12 '25 18:01 isker

CI: https://ci.nodejs.org/job/node-test-pull-request/64633/

nodejs-github-bot avatar Jan 23 '25 08:01 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/64669/ 💛

nodejs-github-bot avatar Jan 23 '25 20:01 nodejs-github-bot

Hi @mcollina or @jasnell, could you take another look at landing this? I'm not sure whether the orange stuff in Jenkins means CI failed or not.

isker avatar Feb 09 '25 23:02 isker

Landed in 1781f6363359f3d0d944360c13f9e6206938418d

nodejs-github-bot avatar Feb 10 '25 03:02 nodejs-github-bot

Thanks @jasnell!

isker avatar Feb 10 '25 04:02 isker

@isker @jasnell @mcollina @avivkeller Thanks for the PR, reviews and merge!

Excited to see this released - seems it's planned in Node.js 23.8.0:

  • https://github.com/nodejs/node/pull/57005

Will this also be backported to v22?

karlhorky avatar Feb 12 '25 10:02 karlhorky

I don't have any plans to attempt a backport myself.

isker avatar Feb 12 '25 15:02 isker

Now that this PR has been released in Node.js v23.8.0:

I can confirm that Node.js v23.8.0 FileHandle.readableWebStream() can be used without configuration to easily create Web Streams 🎉

import fs from "node:fs/promises";
import http from "node:http";
import path from "node:path";
import { Readable } from "node:stream";

const filePath = "./image.jpg";

const server = http.createServer(async (nodeRequest, nodeResponse) => {
  const stats = await fs.stat(filePath);
  const fileHandle = await fs.open(filePath);
  const webStream = fileHandle.readableWebStream();

  nodeResponse.writeHead(200, {
    "Content-Type": "image/jpeg",
    "Content-Length": String(stats.size),
  });

  const nodeStream = Readable.fromWeb(webStream);

  nodeStream.pipe(nodeResponse).on("finish", async () => {
    await fileHandle.close();
  });
});

const port = 3000;
server.listen(port, () => {
  console.log(`Server running at http://localhost:${port}/`);
});

CodeSandbox demo: https://codesandbox.io/p/devbox/filehandle-readablewebstream-with-new-response-forked-rly47y?file=%2Findex.js%3A9%2C35&workspaceId=ws_GfAuHrswXyA1DoeSwsjjjz

Screenshot 2025-02-14 at 17 47 35

karlhorky avatar Feb 14 '25 16:02 karlhorky