self-hosted icon indicating copy to clipboard operation
self-hosted copied to clipboard

GitLab Integration fails - redirects to 504 Gateway Time-out

Open jemx opened this issue 3 years ago • 5 comments

Self-Hosted Version

22.9.0

CPU Architecture

x86_64

Docker Version

20.10.18

Docker Compose Version

1.25.0

Steps to Reproduce

  1. Add GitLab integration
  2. Follow the setup steps
  3. Sign in to GitLab and allow Sentry access

Expected Result

GitLab Integration should be added to Sentry, the setup wizard should continue.

Actual Result

After authorizing in GitLab it redirects to 172.16.16.202:9000/extensions/gitlab/setup/?code=XXX&state=XXX

This results sometimes into a 504 Gateway Timeout and other times into a 502 Bad Gateway. I found the following in docker compose logs: sentry-self-hosted-nginx-1| 2022/09/17 14:51:00 [error] 25#25: *3446 upstream prematurely closed connection while reading response header from upstream, client: 192.168.1.5, server: , request: "GET /extensions/gitlab/setup/?code=XXX&state=XXX HTTP/1.1", upstream: "http://172.18.0.23:9000/extensions/gitlab/setup/?code=XXX&state=XXX", host: "172.16.16.202:9000"

If i try to add a new GitLab Integration again, I have to log out and into Sentry again before I can try again.

jemx avatar Sep 17 '22 15:09 jemx

This does seem like a pretty annoying issue to come across. Are the logs you linked the result when a 504 Gateaway timeout occurs?

Could you paste your nginx config?

hubertdeng123 avatar Sep 19 '22 16:09 hubertdeng123

Yes, this error occurs right after i get the 504 in the little setup window.

Sure, it should be the stock config Sentry comes with:

user nginx;
worker_processes auto;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;


events {
        worker_connections 1024;
}


http {
        default_type application/octet-stream;

        log_format main '$remote_addr - $remote_user [$time_local] "$request" '
        '$status $body_bytes_sent "$http_referer" '
        '"$http_user_agent" "$http_x_forwarded_for"';

        access_log /var/log/nginx/access.log main;

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        reset_timedout_connection on;

        keepalive_timeout 75s;

        gzip off;
        server_tokens off;

        server_names_hash_bucket_size 64;
        types_hash_max_size 2048;
        types_hash_bucket_size 64;
        client_max_body_size 100m;

        proxy_http_version 1.1;
        proxy_redirect off;
        proxy_buffering off;
        proxy_next_upstream error timeout invalid_header http_502 http_503 non_idempotent;
        proxy_next_upstream_tries 2;

        # Remove the Connection header if the client sends it,
        # it could be "close" to close a keepalive connection
        proxy_set_header Connection '';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Request-Id $request_id;
        proxy_read_timeout 30s;
        proxy_send_timeout 5s;

        upstream relay {
                server relay:3000;
        }

        upstream sentry {
                server web:9000;
        }

        server {
                listen 80;

                location /api/store/ {
                        proxy_pass http://relay;
                }
                location ~ ^/api/[1-9]\d*/ {
                        proxy_pass http://relay;
                }
                location / {
                        proxy_pass http://sentry;
                }
        }
}

jemx avatar Sep 19 '22 16:09 jemx

Got it, thanks for that. I was wondering if you could try tweaking the numbers a bit for your proxy timeout. https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout

hubertdeng123 avatar Sep 19 '22 17:09 hubertdeng123

I played around a bit and increased the proxy timeouts. I still get a 502 but now an exception is triggered (and thankfully Sentry monitors it self \o/).

SafeHTTPSConnectionPool(host='xxx.xxx.xxx', port=443): Max retries exceeded with url: /oauth/token (Caused by ConnectTimeoutError(<sentry.net.http.SafeHTTPSConnection object at 0x7f64a95d1e50>, 'Connection to xxx.xxx.xxx timed out. (connect timeout=30)'))

I've checked that the machine running Sentry can reach GitLab (ping and curl) and it works fine...

jemx avatar Sep 19 '22 17:09 jemx

Hmmm, this still sounds like some sort of networking issue of some sort. Have you attempted to ping and curl GitLab from inside the docker containers?

hubertdeng123 avatar Sep 21 '22 16:09 hubertdeng123

related with getsentry/self-hosted#1672 ??

victorelec14 avatar Sep 28 '22 06:09 victorelec14

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

github-actions[bot] avatar Oct 20 '22 00:10 github-actions[bot]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

github-actions[bot] avatar Nov 16 '22 00:11 github-actions[bot]