OIDC connect with Google - Debug a 400's

[reynico]

Hola! 👋

I'm trying to setup Google's OIDC auth for Headscale, my headscale config.yml has the following configuration for oidc:

oidc:
  issuer: "https://accounts.google.com"
  client_id: "31928xxx-d09xxxxxxxxxx.apps.googleusercontent.com"
  client_secret: "GOxxxxxx"

• [x] I have an (external reachable) domain, with SSL configured.
• [x] I have added the callback URL https://tailscale.domain.com/oidc/callback to Google's OAuth authorized redirect URIs.
• [x] I have setup a consent page with "Internal" user type (as it's for my organization).

No errors seen on Headscale's logs:

2022-02-18T23:23:46Z DBG There has been updates since the last successful update to tailscale-prd handler=PollNetMapStream last_state_change=164522662
4 last_successful_update=1645226616 machine=tailscale-prd
2022-02-18T23:23:48Z DBG Redirecting to https://accounts.google.com/o/oauth2/v2/auth?client_id=31928xxxxx-d09xxxxxxxxxx.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Ftailscale.domain.com%3A8080%2Foidc%2Fcallback&response_type=code&scope=openid+profile+email&state=c63ed2
bcf4b5f58569ee7bbfea427b73 for authentication
[GIN] 2022/02/18 - 23:23:48 | 302 |     157.585µs | 190.191.xxx.xxx | GET      "/oidc/register/mkey:xxxxxxxxxxxxxx"
2022-02-18T23:23:56Z DBG Sending update request func=scheduledPollWorker machine=tailscale-prd
2022-02-18T23:23:56Z DBG There has been updates since the last successful update to tailscale-prd handler=PollNetMapStream last_state_change=1645226634 last_successful_update=1645226626 machine=tailscale-prd
2022-02-18T23:24:06Z DBG Sending update request func=scheduledPollWorker machine=tailscale-prd
2022-02-18T23:24:06Z DBG There has been updates since the last successful update to tailscale-prd handler=PollNetMapStream last_state_change=1645226644 last_successful_update=1645226636 machine=tailscale-prd

When I open the Login URL from my laptop I got redirected to Google's account selection, I pick my organization's account and then I just receive a 400 error:

400. That’s an error.

The server cannot process the request because it is malformed. It should not be retried. That’s all we know.

and the response from the server is:

)]}'

[[["er",null,null,null,null,400,null,null,null,3],["e",2,null,null,78]]]

Environment:

• EC2 Ubuntu instance
• Headscale v0.13.0
• Tailscale client 1.20.4 @ mac osx

Is there any way to debug this kind of error?

Thanks!

[reynico]

I think I found a race condition. I've noticed that the Google authentication request page was in Spanish, manually switching the language to English did the trick and the authentication was successful.

[kazauwa]

Encountered the exact same behavior with v0.15.0 Switching from "English (UK)" to "English (US)" helped.

[kradalby]

This one is a bit weird, not sure why that should matter. This part should be handled by the upstream library I believe.

Can any of you by any chance provide an example payloads that brakes it so we can compare?

[kazauwa]

Will do I also noticed that "English (UK)" breaks almost every Google OIDC integration I used for the past couple of months. Not sure why, but it seems that the problem may be elsewhere. Anyways, will get back to you with example payloads.

[kazauwa]

Okay, so I can't figure out which payload fields are sensitive and which are not, but I noticed one interesting thing. Here is the request path that fails for me when I try to log in from my browser: https://accounts.google.com/_/signin/oauth?authuser=2&hl=en-GB&_reqid=<redacted>&rt=j

The interesting thing is that this doesn't reproduce when using other languages. It also works with English (UK) chosen when doing a fresh log-in to a google account (I tried incognito mode and another browser). So I compared payloads and noticed that the failing request lacks hl=en-GB in form data. Every successful request I inspected included that field. Haven't figured out why it's being omitted, though.

[kradalby]

I will close this as it might have been fixed in newer version, please try a new release and reopen if it still does not work and if there is any progress in the investigation.