headplane icon indicating copy to clipboard operation
headplane copied to clipboard

Headplane cannot resolve DNS queries at times.

Open cowbe0x004 opened this issue 1 year ago • 13 comments

Trying to set up headplane and getting this error in the log,

headplane  | GET /admin/ 302 - - 1.923 ms
headplane  | GET /admin/machines 302 - - 4.647 ms
headplane  | TypeError: Cannot read properties of undefined (reading 'issuer')
headplane  |     at getOidcConfig (file:///app/build/server/index.js?t=1715826667000:492:27)
headplane  |     at getContext (file:///app/build/server/index.js?t=1715826667000:480:19)
headplane  |     at loader$9 (file:///app/build/server/index.js?t=1715826667000:605:20)
headplane  |     at Object.callRouteLoader (/app/node_modules/.pnpm/@[email protected][email protected]/node_modules/@remix-run/server-runtime/dist/data.js:66:16)
headplane  |     at /app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4266:21
headplane  |     at callLoaderOrAction (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4328:16)
headplane  |     at async Promise.all (index 0)
headplane  |     at callDataStrategyImpl (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4169:17)
headplane  |     at callDataStrategy (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:3503:19)
headplane  |     at loadRouteData (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:3447:19)
headplane  | GET /admin/login 500 - - 10.136 ms

Page shows "unexpected server error".

This is my docker-compose,

  headscale:
    image: headscale/headscale:0.23.0-alpha11
    container_name: headscale
    volumes:
      - ./headscale/config:/etc/headscale/
      - ./headscale/data:/var/lib/headscale
    #ports:
     # - 8080:8080
    command: serve
    restart: unless-stopped
    networks:
      reverseproxy-nw:

  headplane:
    container_name: headplane
    image: ghcr.io/tale/headplane:latest
    restart: unless-stopped
    volumes:
      - './headscale/data:/var/lib/headscale'
      - './headscale/config:/etc/headscale'
      - '/var/run/docker.sock:/var/run/docker.sock:ro'
    ports:
      - '3000:3000'
    networks:
      reverseproxy-nw:
    environment:
      # This is always required for Headplane to work
      COOKIE_SECRET: 'BunchFabulousGalvanizeHardcore'
      #CONFIG_FILE: './headscale/config/config.yaml'
      HEADSCALE_CONTAINER: 'headscale'
      DISABLE_API_KEY_LOGIN: 'true'
      HOST: '0.0.0.0'
      PORT: '3000'

Please let me know if you know what might be causing this, thank you.

cowbe0x004 avatar May 19 '24 18:05 cowbe0x004

You've disabled API key login, but Headplane is either unable to read your OIDC configuration correctly or it isn't able to read a configuration at all. Does your configuration file have all the required fields?

On another note, thanks for opening this issue, I definitely need to improve errors messages.

tale avatar May 19 '24 21:05 tale

You've disabled API key login, but Headplane is either unable to read your OIDC configuration correctly or it isn't able to read a configuration at all. Does your configuration file have all the required fields?

On another note, thanks for opening this issue, I definitely need to improve errors messages.

Do you mean the configuration file for headscale? I have clients registered and working so I assume it has all the required files. When I set the CONFIG_FILE to where config.yaml is, I got an error so I commented it out. So I set DISABLE_API_KEY_LOGIN to false, I'm still getting the same error. Same when I try to set API_KEY from 'headscale apikeys create'. I'm not using OIDC.

cowbe0x004 avatar May 19 '24 21:05 cowbe0x004

Through an oversight, DISABLE_API_KEY_LOGIN goes into effect if it's been set, remove the variable completely. That should at least enable login via API keys. Could you share more about the issue you get when the CONFIG_FILE variable is set?

tale avatar May 19 '24 23:05 tale

This is the error when "CONFIG_FILE: './headscale/config/config.yaml'" is set,

headplane  | Error: ENOENT: no such file or directory, open '/app/headscale/config/config.yaml'

Removing DISABLE_API_KEY_LOGIN I'm still getting the same error as above.

cowbe0x004 avatar May 19 '24 23:05 cowbe0x004

Yes that error is correct, put the full path to the file in CONFIG_FILE. It needs to be the path that's mounted in the container, not out of it!

tale avatar May 20 '24 00:05 tale

I changed it to this and it's seeing it now.

  CONFIG_FILE: '/etc/headscale/config.yaml'

But still getting this error,

headplane  | TypeError: Cannot read properties of undefined (reading 'issuer')
headplane  |     at getOidcConfig (file:///app/build/server/index.js?t=1715826667000:492:27)
headplane  |     at getContext (file:///app/build/server/index.js?t=1715826667000:480:19)
headplane  |     at loader$9 (file:///app/build/server/index.js?t=1715826667000:605:20)
headplane  |     at Object.callRouteLoader (/app/node_modules/.pnpm/@[email protected][email protected]/node_modules/@remix-run/server-runtime/dist/data.js:66:16)
headplane  |     at /app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4266:21
headplane  |     at callLoaderOrAction (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4328:16)
headplane  |     at async Promise.all (index 0)
headplane  |     at callDataStrategyImpl (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:4169:17)
headplane  |     at callDataStrategy (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:3503:19)
headplane  |     at loadRouteData (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/router/router.ts:3447:19)
headplane  | GET /admin/login 500 - - 163.250 ms

By the way I'm on headscale/headscale:0.23.0-alpha11, and latest headplane. Thanks for your suggestions so far.

cowbe0x004 avatar May 20 '24 01:05 cowbe0x004

I'm assuming you aren't using OIDC, the issue is that you've kept the DISABLE_API_KEY_LOGIN environment variable set (is my assumption). The configuration loading code I have is still slightly brittle so if you could remove the variable completely your problem might be fixed.

I'm also going to cut a release pretty soon that should gracefully handle the issue anyways!

tale avatar May 20 '24 02:05 tale

I'm not using OIDC, and I don't have DISABLE_API_KEY_LOGIN set.

image

cowbe0x004 avatar May 20 '24 02:05 cowbe0x004

Ah my apologies, I've identified an issue on my end, I'll cut a release shortly and let you know once it's out.

tale avatar May 20 '24 03:05 tale

Once 0.1.5 finishes building, can you try it and let me know if it makes a difference.

tale avatar May 20 '24 18:05 tale

Getting this that looks like a DNS issue? When I go to the API URL, I get unauthorized so I assume DNS is working. headscale and headplane are in the same docker network.

headplane  | [remix-serve] http://localhost:3000 (http://0.0.0.0:3000)
headplane  | GET /admin/login 200 - - 99.938 ms
headplane  | FetchError: request to https://*******/api/v1/apikey failed, reason: getaddrinfo EAI_AGAIN headscale.cowbe.me
headplane  |     at ClientRequest.<anonymous> (/app/node_modules/.pnpm/@[email protected]/node_modules/@remix-run/web-fetch/src/fetch.js:112:11)
headplane  |     at ClientRequest.emit (node:events:519:28)
headplane  |     at TLSSocket.socketErrorListener (node:_http_client:500:9)
headplane  |     at TLSSocket.emit (node:events:519:28)
headplane  |     at emitErrorNT (node:internal/streams/destroy:169:8)
headplane  |     at emitErrorCloseNT (node:internal/streams/destroy:128:3)
headplane  |     at processTicksAndRejections (node:internal/process/task_queues:82:21) {
headplane  |   type: 'system',
headplane  |   errno: 'EAI_AGAIN',
headplane  |   code: 'EAI_AGAIN',
headplane  |   erroredSysCall: 'getaddrinfo'
headplane  | }
headplane  | POST /admin/login?_data=routes%2Flogin 200 - - 5096.872 ms
headplane  | GET /admin/login?_data=routes%2Flogin 200 - - 1.676 ms

cowbe0x004 avatar May 20 '24 19:05 cowbe0x004

Most likely the container doesn't have access to the internet/the DNS available to the container has issues. Is it just limited to headplane or do all of your containers fail to make DNS requests?

tale avatar May 20 '24 22:05 tale

I only have caddy, headplane, and headscale on this box. No connectivity issue from caddy and headscale. image

But something might be going on with this box, I was able to login to headplane then I restarted it, now giving me the same getaddrinfo. Thank you for the help, I'll troubheshoot further.

cowbe0x004 avatar May 21 '24 00:05 cowbe0x004