node icon indicating copy to clipboard operation
node copied to clipboard

fs: optimize `fs.cpSync` js calls

Open anonrig opened this issue 1 year ago • 2 comments

https://github.com/nodejs/node/pull/53612 is required to do a benchmark

A different approach to https://github.com/nodejs/node/pull/53541

cc @nodejs/performance @nodejs/cpp-reviewers @billywhizz @jasnell @nodejs/fs

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1573/

Small win on happy path, and probably high win on error path:

                           confidence improvement accuracy (*)   (**)   (***)
fs/bench-cpSync.js n=1            ***     21.96 %       ±6.85% ±9.14% ±11.97%
fs/bench-cpSync.js n=100          ***     14.91 %       ±2.86% ±3.81%  ±4.97%
fs/bench-cpSync.js n=10000        ***      9.74 %       ±1.28% ±1.71%  ±2.23%

anonrig avatar Jun 27 '24 18:06 anonrig

cc @nodejs/tsc @nodejs/fs appreciate your input on which path to take.

previous pr:

  • is a sem-ver major
  • introduces 2 different implementations (js only and cpp)
  • had limitations due to std::filesystem::copy

this pr:

  • is not a sem-ver major
  • optimizes all flags passed to cpSync
  • optimizes error cases as same as the previous pr
  • has a lot more optimization to do

anonrig avatar Jun 27 '24 21:06 anonrig

CI: https://ci.nodejs.org/job/node-test-pull-request/60000/

nodejs-github-bot avatar Jun 28 '24 00:06 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60175/

nodejs-github-bot avatar Jul 08 '24 18:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60191/

nodejs-github-bot avatar Jul 09 '24 06:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60190/

nodejs-github-bot avatar Jul 09 '24 06:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60257/

nodejs-github-bot avatar Jul 11 '24 19:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60274/

nodejs-github-bot avatar Jul 12 '24 10:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60277/

nodejs-github-bot avatar Jul 12 '24 15:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60366/

nodejs-github-bot avatar Jul 16 '24 19:07 nodejs-github-bot

Windows compilation is failing:

C:\workspace\node-compile-windows-debug\node\deps\v8\src\base\bits.h(477,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows-debug\node\tools\v8_gypfiles\mksnapshot.vcxproj]
  embedded-file-writer.cc
  mksnapshot.cc
  embedded-empty.cc
  platform-embedded-file-writer-base.cc
  platform-embedded-file-writer-generic.cc
  platform-embedded-file-writer-win.cc
  platform-embedded-file-writer-aix.cc
  platform-embedded-file-writer-mac.cc
  snapshot-empty.cc
  static-roots-gen.cc
     Creating library ..\..\out\Debug\mksnapshot.lib and object ..\..\out\Debug\mksnapshot.exp
  mksnapshot.vcxproj -> ..\..\out\Debug\\mksnapshot.exe
  generating: "..\..\out\Debug\obj\v8_snapshot\/snapshot.cc" "..\..\out\Debug\obj\v8_snapshot\/embedded.S"
  Assembling ..\..\out\Debug\obj\v8_snapshot\\embedded.S...
  setup-isolate-deserialize.cc
C:\workspace\node-compile-windows-debug\node\deps\v8\src\base\bits.h(477,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows-debug\node\tools\v8_gypfiles\v8_snapshot.vcxproj]
  snapshot.cc
C:\workspace\node-compile-windows-debug\node\deps\v8\src\base\bits.h(477,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows-debug\node\tools\v8_gypfiles\v8_snapshot.vcxproj]
  v8_snapshot.vcxproj -> ..\..\out\Debug\lib\v8_snapshot.lib

aduh95 avatar Jul 16 '24 21:07 aduh95

@nodejs/platform-windows any idea why this build is failing?

anonrig avatar Jul 17 '24 00:07 anonrig

Building locally, I got this:

C:\Users\vinic\Desktop\Projects\node\src\node_file.cc(3036,15): error C2664: 'void node::Environment::ThrowUVException(int,const char *,const char *,const char *,const char *)': cannot convert argument 4 from
 'const std::filesystem::path::value_type *' to 'const char *' [C:\Users\vinic\Desktop\Projects\node\libnode.vcxproj]

H4ad avatar Jul 17 '24 01:07 H4ad

Building locally, I got this:

C:\Users\vinic\Desktop\Projects\node\src\node_file.cc(3036,15): error C2664: 'void node::Environment::ThrowUVException(int,const char *,const char *,const char *,const char *)': cannot convert argument 4 from
 'const std::filesystem::path::value_type *' to 'const char *' [C:\Users\vinic\Desktop\Projects\node\libnode.vcxproj]

This is also in the log from the CI. Based on the error message, this looks very similar to what we had in https://github.com/nodejs/node/pull/53600 because std::filesystem::path::value_type is different on POSIX and Windows.

StefanStojanovic avatar Jul 17 '24 06:07 StefanStojanovic

CI: https://ci.nodejs.org/job/node-test-pull-request/60500/

nodejs-github-bot avatar Jul 21 '24 21:07 nodejs-github-bot

This is also in the log from the CI. Based on the error message, this looks very similar to what we had in https://github.com/nodejs/node/pull/53600 because std::filesystem::path::value_type is different on POSIX and Windows.

It seems I forgot a src_path.c_str() there. Thank you! @StefanStojanovic @H4ad

anonrig avatar Jul 21 '24 21:07 anonrig

CI: https://ci.nodejs.org/job/node-test-pull-request/60519/

nodejs-github-bot avatar Jul 21 '24 23:07 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/60528/

nodejs-github-bot avatar Jul 22 '24 06:07 nodejs-github-bot

Landed in 88027e84d812e5d4558fa0f27c4d8baf7a671ebf

nodejs-github-bot avatar Jul 22 '24 16:07 nodejs-github-bot