Cannot find module './dist/tiktoken-node.linux-x64-gnu.node' when deployed on Vercel - Nuxt 3

[balsimpson]

[nuxt] [request error] [unhandled] [500] Cannot find module './dist/tiktoken-node.linux-x64-gnu.node' Require stack:

• /var/task/node_modules/tiktoken-node/index.mjs at Module._resolveFilename (node:internal/modules/cjs/loader:1039:15)
  at Module._load (node:internal/modules/cjs/loader:885:27)
  at Module.require (node:internal/modules/cjs/loader:1105:19)
  at require (node:internal/modules/cjs/helpers:103:18)
  at ./node_modules/tiktoken-node/index.mjs:11:21
  at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

It works perfectly fine in development mode in my local machine, but the deployed code gives me this error on the server.

[ceifa]

This is a problem on your publishing. Check if the .node file it's on the correct location.

[balsimpson]

I don't have enough understanding of that to correct it. My project is just standard Nuxt 3.

[mydaoyuan]

node:internal/modules/cjs/loader:1338 return process.dlopen(module, path.toNamespacedPath(filename)); ^ Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /app/node_modules/.pnpm/[email protected]/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node)

---

Same as me, I use Docker

[JanRuettinger]

@balsimpson Did you solve your issue? I run into the same issue.

[JanRuettinger]

This is a problem on your publishing. Check if the .node file it's on the correct location.

What .node file do you mean?

[balsimpson]

@balsimpson Did you solve your issue? I run into the same issue.

nope. just abandoned it ;-)

[balsimpson]

same issue. What a nightmare. What library you ended up using?

None. couldn't find a good one.

[ceifa]

@mydaoyuan actually your problem is related on #8. Do you have a minimal repro to it?

[ceifa]

Fix probably located here: https://github.com/napi-rs/package-template/blob/main/.github/workflows/CI.yml

[AaronFriel]

@mydaoyuan's issue is likely that they are using Alpine Linux or another MUSL. The gcompat package is needed to add the dependency. Adding that package before npm install does fix that specific error for me.

FROM node:lts-alpine
...
RUN apk add --no-cache libc6-compat gcompat
RUN apk update

Unfortunately, as a musl libc distribution I ran into another linker issue:

Error relocating /app/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node: fwrite: initial-exec TLS resolves to dynamic definition in /app/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node