apm-agent-nodejs icon indicating copy to clipboard operation
apm-agent-nodejs copied to clipboard
verified publisher icon elastic

next js output standalone - next:13.2.1, elastic-apm-node: 3.46.0

Open mindyourlifeguide opened this issue 2 years ago • 7 comments

Describe the bug

To Reproduce I have a monorepo with several applications. Package manager - pnpm v8.4.0.

my Dockerfile:

FROM node:16-alpine AS builder

ARG BUILD_ENV=prod
ARG PROJECT_NAME

WORKDIR /app

RUN apk add --no-cache build-base g++ cairo-dev jpeg-dev pango-dev giflib-dev
RUN npm install -g [email protected]

COPY . .

RUN pnpm config set store-dir /app/.pnpm-store
RUN --mount=type=secret,id=npmrc,dst=/root/.npmrc \
    --mount=type=tmpfs,target=/app/node_modules/ \
    pnpm install && pnpm build:${BUILD_ENV} --filter=${PROJECT_NAME}

# Production image, copy the necessary files and run next
FROM node:16-alpine AS runner
WORKDIR /app

# args needed to build the docker image
ARG PATH_TO_PROJECT
ARG PROJECT_NAME

# service env variables
ENV PATH_TO_PROJECT $PATH_TO_PROJECT
ENV PROJECT_NAME $PROJECT_NAME
ENV RUN_ENV prod
ENV PORT 3000

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/.next/standalone .
COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/public ./${PATH_TO_PROJECT}/public
COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/.next/static ./${PATH_TO_PROJECT}/.next/static

USER nextjs

EXPOSE 3000

CMD node NODE_OPTIONS=--require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js

I have a elastic-apm-node.js In normal mode, everything worked - the Dockerfile was a little different. When I turned on output: standalone - apm stopped working. You told me to import the api route package to include in the bundle. The package really dragged on and locally it seemed to even start. But when I build in ci - module not found at startup.

I solved this by removing the options for node NODE_OPTIONS=--require=elastic-apm-node/start-next.js from the Dockerfile and running apm agent manually in api routes. The agent worked, but now it says that the routes are unknown

Expected behavior

Working apm agent with output: standalone

Environment (please complete the following information)

  • OS: macOS 13.3.1 22E261 arm64
  • Node.js version: 18.16.0
  • "elastic-apm-node": "3.45.0",
  • "next": "13.2.1",

How are you starting the agent? (please tick one of the boxes)

  • [x] Calling agent.start() directly (e.g. require('elastic-apm-node').start(...)) - work, but routes are unknown
  • [ ] Requiring elastic-apm-node/start from within the source code
  • [x] Starting node with -r elastic-apm-node/start - module not found

mindyourlifeguide avatar May 11 '23 21:05 mindyourlifeguide 

CMD node NODE_OPTIONS=--require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js

That looks wrong. I think it should be either:

ENV NODE_OPTIONS=--require=elastic-apm-node/start-next.js
CMD node ./${PATH_TO_PROJECT}/server.js

or

CMD node --require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js

I'm not sure this is the issue you are having, however.

trentm avatar May 11 '23 21:05 trentm

Also, as I mentioned at https://github.com/elastic/apm-agent-nodejs/issues/1611#issuecomment-1541023864 I wonder if standalone mode is just completely broken in [email protected].

Can you should a recursive directory listing of /app in your "runner" container? find /app

trentm avatar May 11 '23 22:05 trentm

@trentm I tried:

CMD node --require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js
CMD node NODE_OPTIONS=--require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js
CMD node -r elastic-apm-node/start-next.js server.js

And always the result was the same. image

Standalone mode itself works in "next": "13.2.1". Only elastic-apm-node does not work. The same sentry works and doesn't require the api routes import hack.

mindyourlifeguide avatar May 11 '23 22:05 mindyourlifeguide

Can you should a recursive directory listing of /app in your "runner" container? find /app

what exactly are you interested in? image

mindyourlifeguide avatar May 11 '23 22:05 mindyourlifeguide

@trentm in order for everything to work in nextjs in output standalone mode, you need:

  • import 'elastic-apm-node/start-next.js' - I do it in _document.tsx
  • COPY --chown=nextjs:nodejs --from=builder /app/${PATH_TO_PROJECT}/elastic-apm-node.* ./ - need to add to Dockerfile

Library versions: "next": "13.2.1", "elastic-apm-node": "3.46.0",

My final Dockerfile is more complicated since I don't run apm everywhere, but a simplified version would look like this:

FROM node:16-alpine AS builder

ARG BUILD_ENV=prod
ARG PROJECT_NAME

WORKDIR /app

RUN apk add --no-cache build-base g++ cairo-dev jpeg-dev pango-dev giflib-dev
RUN npm install -g [email protected]

COPY . .

RUN pnpm config set store-dir /app/.pnpm-store
RUN --mount=type=secret,id=npmrc,dst=/root/.npmrc \
    --mount=type=tmpfs,target=/app/node_modules/ \
    pnpm install && pnpm build:${BUILD_ENV} --filter=${PROJECT_NAME}

# Production image, copy the necessary files and run next
FROM node:16-alpine AS runner
WORKDIR /app

# args needed to build the docker image
ARG PATH_TO_PROJECT
ARG PROJECT_NAME

# service env variables
ENV PATH_TO_PROJECT $PATH_TO_PROJECT
ENV PROJECT_NAME $PROJECT_NAME
ENV RUN_ENV prod
ENV PORT 3000

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/.next/standalone .
COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/public ./${PATH_TO_PROJECT}/public
COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/.next/static ./${PATH_TO_PROJECT}/.next/static
COPY --chown=nextjs:nodejs  --from=builder /app/${PATH_TO_PROJECT}/elastic-apm-node.* ./


USER nextjs

EXPOSE 3000

CMD node NODE_OPTIONS=--require=elastic-apm-node/start-next.js ./${PATH_TO_PROJECT}/server.js

mindyourlifeguide avatar May 22 '23 14:05 mindyourlifeguide

I am getting something similar with a more verbose error message below:

/app $ ELASTIC_APM_LOG_LEVEL=trace node --require=elastic-apm-node/start-next.js server.js

{"log.level":"debug","@timestamp":"2023-07-29T14:50:16.728Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"error":{"type":"Error","message":"Cannot find module './modules/next/dist/server/next.js'\nRequire stack:\n- /app/node_modules/elastic-apm-node/lib/instrumentation/index.js\n- /app/node_modules/elastic-apm-node/lib/agent.js\n- /app/node_modules/elastic-apm-node/index.js\n- /app/node_modules/elastic-apm-node/start.js\n- /app/node_modules/elastic-apm-node/start-next.js\n- internal/preload","stack_trace":"Error: Cannot find module './modules/next/dist/server/next.js'\nRequire stack:\n- /app/node_modules/elastic-apm-node/lib/instrumentation/index.js\n- /app/node_modules/elastic-apm-node/lib/agent.js\n- /app/node_modules/elastic-apm-node/index.js\n- /app/node_modules/elastic-apm-node/start.js\n- /app/node_modules/elastic-apm-node/start-next.js\n- internal/preload\n    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)\n    at /app/node_modules/next/dist/server/require-hook.js:113:36\n    at Module._load (node:internal/modules/cjs/loader:922:27)\n    at Module.require (node:internal/modules/cjs/loader:1143:19)\n    at Hook._require.Module.require (/app/node_modules/require-in-the-middle/index.js:167:34)\n    at require (node:internal/modules/cjs/helpers:110:18)\n    at /app/node_modules/elastic-apm-node/lib/instrumentation/index.js:142:14\n    at Instrumentation._patchModule (/app/node_modules/elastic-apm-node/lib/instrumentation/index.js:433:20)\n    at /app/node_modules/elastic-apm-node/lib/instrumentation/index.js:358:17\n    at Hook._require.Module.require (/app/node_modules/require-in-the-middle/index.js:262:28)\n    at require (node:internal/modules/cjs/helpers:110:18)\n    at Object.<anonymous> (/app/node_modules/next/dist/server/lib/render-server.js:40:54)\n    at Module._compile (node:internal/modules/cjs/loader:1256:14)\n    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)\n    at Module.load (node:internal/modules/cjs/loader:1119:32)\n    at Module._load (node:internal/modules/cjs/loader:960:12)\n    at Module.require (node:internal/modules/cjs/loader:1143:19)\n    at Hook._require.Module.require (/app/node_modules/require-in-the-middle/index.js:188:39)\n    at require (node:internal/modules/cjs/helpers:110:18)\n    at execMethod (/app/node_modules/next/dist/compiled/jest-worker/processChild.js:1:2366)\n    at process.messageListener (/app/node_modules/next/dist/compiled/jest-worker/processChild.js:1:1284)\n    at process.emit (node:events:514:28)\n    at emit (node:internal/child_process:937:14)\n    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)"},"message":"Elastic APM caught unhandled exception"}
{"log.level":"trace","@timestamp":"2023-07-29T14:50:16.773Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"azure metadata server responded, but there was an error parsing the result: {}"}
{"log.level":"debug","@timestamp":"2023-07-29T14:50:16.793Z","log":{"logger":"elastic-apm-node"},"filename":"/app/node_modules/next/dist/server/require-hook.js","ecs":{"version":"1.6.0"},"error":{"type":"Error","message":"Error reading sourcemap for file \"/app/node_modules/next/dist/server/require-hook.js\":\nENOENT: no such file or directory, open '/app/node_modules/next/dist/server/require-hook.js.map'","stack_trace":"Error: Error reading sourcemap for file \"/app/node_modules/next/dist/server/require-hook.js\":\nENOENT: no such file or directory, open '/app/node_modules/next/dist/server/require-hook.js.map'"},"message":"could not process file source map"}
{"log.level":"debug","@timestamp":"2023-07-29T14:50:16.794Z","log":{"logger":"elastic-apm-node"},"filename":"/app/node_modules/next/dist/server/lib/render-server.js","ecs":{"version":"1.6.0"},"error":{"type":"Error","message":"Error reading sourcemap for file \"/app/node_modules/next/dist/server/lib/render-server.js\":\nENOENT: no such file or directory, open '/app/node_modules/next/dist/server/lib/render-server.js.map'","stack_trace":"Error: Error reading sourcemap for file \"/app/node_modules/next/dist/server/lib/render-server.js\":\nENOENT: no such file or directory, open '/app/node_modules/next/dist/server/lib/render-server.js.map'"},"message":"could not process file source map"}

These files are indeed not present in the traced output. Can we make sure to add the proper require/include statement such that they are picked up? I believe NextJS uses this tool to do the tracing https://github.com/vercel/nft.

Note; I think the scope is broader to framework that use output file tracing solutions

sjoukedv avatar Jul 29 '23 14:07 sjoukedv

I had the same problem with this. I am sharing my solution. (Caution: This is a trick.)

  • Issue : When using Next.js in standalone mode, an issue occurs in which the elastic-amp-node/start-next.js module cannot be found when built and run in Docker.

  • Cause : Next.js' standAlone mode includes only the dependencies absolutely necessary for production builds in the node_modules folder. elastic-amp-node only installs dependencies, but since it is not actually used anywhere in the code, it is judged to be an unnecessary dependency and is not included in node_modules of standAlone, so the module cannot be found.

  • Solution

    • By importing and using elastic-apm-node/start-next in the dummy API, you can include the start-next.js file in standalone/node_modules after building. It may not actually work, but it was imported. image
    • After building Next.js, elastic-apm-node is placed in the node_modules folder in the standAlone folder. image
    • If you change the path to the last line in DockerFile from node_modules → /app/node_modules, it will run successfully. image

my package version

 "next": "13.2.3"
 "elastic-apm-node": "^4.1.0",

snowman95 avatar Feb 01 '24 09:02 snowman95