M1 Installation

[vaguue]

Hi, and thanks for your great project. I'm having issues installing duckdb for my M1 device with Node.js v21.5.0. Providing the logs below:

npm ERR! code 1
npm ERR! path /Users/seva/repositories/mono/node_modules/duckdb
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | darwin | arm64
npm ERR! gyp info find Python using Python version 3.12.0 found at "/Users/seva/.pyenv/versions/3.12.0/bin/python3"
npm ERR! gyp info spawn /Users/seva/.pyenv/versions/3.12.0/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/Users/seva/repositories/mono/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/seva/seva/repositories/mono/node_modules/duckdb/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/seva/repositories/mono/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/seva/Library/Caches/node-gyp/21.5.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/seva/Library/Caches/node-gyp/21.5.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/seva/repositories/mono/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/seva/Library/Caches/node-gyp/21.5.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/seva/repositories/mono/node_modules/duckdb',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: Undefined variable module_name in binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/Users/seva/repositories/mono/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Darwin 23.4.0
npm ERR! gyp ERR! command "/Users/seva/.nvm/versions/node/v21.5.0/bin/node" "/Users/seva/repositories/mono/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /Users/seva/repositories/mono/node_modules/duckdb
npm ERR! gyp ERR! node -v v21.5.0
npm ERR! gyp ERR! node-gyp -v v9.4.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/seva/.npm/_logs/2024-04-11T06_27_05_030Z-debug-0.log

I think this is the issue with node-pre-gyp, because as much as I know this module should provide the module_name, but I haven't used myself. This problem my affect other users who happen to have Node.js versions without prebuilds for them.

[noctarius]

Same happens now on MacOS 14.4.1, node 22, x64

[archiewood]

my understanding of this issue is:

• duckdb only precompiles binaries to some set of architectures / nodeJS versions
• if you do not have one of these combinations, then duckdb attempts to build from the source code.
• building from source requires a number of tools to be installed on your machine, many of which are not typical, including at least
  • Python
  • Microsoft Visual C++ Build Tools

there may also be some others i don't remember.

if your machine doesn't have all of the required tools, the npm install will throw

[vaguue]

That's not the issue with the tools, even having all these tools the build won't work. The only workaround is to use npm i duckdb --ignore-scripts, then cd node_modules/duckdb && npm i .

[vaguue]

Any progress on this? Perhaps if it is intended for the library to be like this I can close the issue

[archiewood]

if it's possible for you, you can use Node 18 or 20 on M1

[carlopi]

I am on M1 and node , just tried:

% node --version
v21.7.3
% npm uninstall duckdb
% npm install duckdb
% npm list duckdb
duckdb@ /Users/carlo/duckdb
└── [email protected]

and looks to be working as expected and in a few seconds (depending on network speed) you should have duckdb available.

@vaguue: Could you log the previous commands for you?

Original issue was pre 1.0.0, I think it could have been resolved with 1.0.0.

[vaguue]

The problem is you have the necessary prebuild for your system, but in case you won't the fallback to building from source won't work. It's not ok in my opinion, but up to you.

[carlopi]

I am not seeing something, since I believe we ship both darwin-x64 and darwin-arm64, for all relevant versions of node.

Do you have any steps on helping me to reproduce what you are seeing?

Possibly is tied to Python 3.12, that can be a difference, very much unsure.

[vaguue]

I think the simplest solution is to change the install script of package.json to something like this: node-pre-gyp install --fallback-to-build || node-gyp rebuild just in case, because let's imagine the new Node version comes out etc