GitLab Integration fails - redirects to 504 Gateway Time-out
Self-Hosted Version
22.9.0
CPU Architecture
x86_64
Docker Version
20.10.18
Docker Compose Version
1.25.0
Steps to Reproduce
- Add GitLab integration
- Follow the setup steps
- Sign in to GitLab and allow Sentry access
Expected Result
GitLab Integration should be added to Sentry, the setup wizard should continue.
Actual Result
After authorizing in GitLab it redirects to 172.16.16.202:9000/extensions/gitlab/setup/?code=XXX&state=XXX
This results sometimes into a 504 Gateway Timeout and other times into a 502 Bad Gateway. I found the following in docker compose logs:
sentry-self-hosted-nginx-1| 2022/09/17 14:51:00 [error] 25#25: *3446 upstream prematurely closed connection while reading response header from upstream, client: 192.168.1.5, server: , request: "GET /extensions/gitlab/setup/?code=XXX&state=XXX HTTP/1.1", upstream: "http://172.18.0.23:9000/extensions/gitlab/setup/?code=XXX&state=XXX", host: "172.16.16.202:9000"
If i try to add a new GitLab Integration again, I have to log out and into Sentry again before I can try again.
This does seem like a pretty annoying issue to come across. Are the logs you linked the result when a 504 Gateaway timeout occurs?
Could you paste your nginx config?
Yes, this error occurs right after i get the 504 in the little setup window.
Sure, it should be the stock config Sentry comes with:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
reset_timedout_connection on;
keepalive_timeout 75s;
gzip off;
server_tokens off;
server_names_hash_bucket_size 64;
types_hash_max_size 2048;
types_hash_bucket_size 64;
client_max_body_size 100m;
proxy_http_version 1.1;
proxy_redirect off;
proxy_buffering off;
proxy_next_upstream error timeout invalid_header http_502 http_503 non_idempotent;
proxy_next_upstream_tries 2;
# Remove the Connection header if the client sends it,
# it could be "close" to close a keepalive connection
proxy_set_header Connection '';
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-Id $request_id;
proxy_read_timeout 30s;
proxy_send_timeout 5s;
upstream relay {
server relay:3000;
}
upstream sentry {
server web:9000;
}
server {
listen 80;
location /api/store/ {
proxy_pass http://relay;
}
location ~ ^/api/[1-9]\d*/ {
proxy_pass http://relay;
}
location / {
proxy_pass http://sentry;
}
}
}
Got it, thanks for that. I was wondering if you could try tweaking the numbers a bit for your proxy timeout. https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout
I played around a bit and increased the proxy timeouts. I still get a 502 but now an exception is triggered (and thankfully Sentry monitors it self \o/).
SafeHTTPSConnectionPool(host='xxx.xxx.xxx', port=443): Max retries exceeded with url: /oauth/token (Caused by ConnectTimeoutError(<sentry.net.http.SafeHTTPSConnection object at 0x7f64a95d1e50>, 'Connection to xxx.xxx.xxx timed out. (connect timeout=30)'))
I've checked that the machine running Sentry can reach GitLab (ping and curl) and it works fine...
Hmmm, this still sounds like some sort of networking issue of some sort. Have you attempted to ping and curl GitLab from inside the docker containers?
related with getsentry/self-hosted#1672 ??
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀