spring-cloud-gateway icon indicating copy to clipboard operation
spring-cloud-gateway copied to clipboard

WebSocketHandshakeException Invalid handshake response getStatus: 403 Forbidden

Open wangyang1749 opened this issue 2 years ago • 16 comments

Hi everyone, I want to use spring-cloud-gateway for reverse proxy of code-server. But I encountered the following error.

io.netty.handler.codec.http.websocketx.WebSocketHandshakeException: Invalid handshake response getStatus: 403 Forbidden
        at io.netty.handler.codec.http.websocketx.WebSocketClientHandshaker13.verify(WebSocketClientHandshaker13.java:274) ~[netty-codec-http-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.http.websocketx.WebSocketClientHandshaker.finishHandshake(WebSocketClientHandshaker.java:302) ~[netty-codec-http-4.1.51.Final.jar!/:4.1.51.Final]
        at reactor.netty.http.client.WebsocketClientOperations.onInboundNext(WebsocketClientOperations.java:118) ~[reactor-netty-0.9.10.RELEASE.jar!/:0.9.10.RELEASE]
        at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) [reactor-netty-0.9.10.RELEASE.jar!/:0.9.10.RELEASE]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:383) [netty-codec-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354) [netty-codec-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:311) [netty-codec-http-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:221) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818) [netty-transport-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [netty-common-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [netty-common-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) [netty-transport-native-epoll-4.1.51.Final-linux-x86_64.jar!/:4.1.51.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.51.Final.jar!/:4.1.51.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.51.Final.jar!/:4.1.51.Final]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_202]

The docker command of the code server I use is as follows.

docker run -it --name code-server --rm  -p 40001:8080    codercom/code-server:latest
# docker exec -it code-server bash 
# cat /home/coder/.config/code-server/config.yaml

The version of spring-cloud-gateway is as follows

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.2.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
 <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-gateway</artifactId>
   </dependency>

The configuration of the gateway is as follows.

server:
  port: 30000
spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
          lower-case-service-id: true
      filter:
        remove-non-proxy-headers:
          headers:
      httpclient:
        websocket:
          max-frame-payload-length: 3000000 
      routes:
        - id:  docker-app
          uri: http://192.168.3.60:80
          predicates:
            - Path=/app/**
        - id: vscode-cloud
          uri: http://192.168.3.60:40001
          predicates:
            - Path=/vscode/**
          filters:
            - RewritePath=/vscode/(?<segment>.*), /$\{segment}

When I access http://192.168.3.60:30000/vscode/?folder=/home/coder, the above error appears in the log of spring-cloud-gateway.

The above port 80 is the reverse proxy of nginx and can be accessed normally.

location /app/ {
      add_header X-debug-message "A static file was served" always;
      add_header aaa "$request_uri";
      if ($request_uri ~ /app/(.+)) {
            set $rightUrl $1;
      }
      add_header bbb "$rightUrl";
      proxy_pass http://192.168.3.60:$rightUrl;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      proxy_set_header Host $host;
      rewrite ^/app/$rightUrl(.*)$ /$1 break;
}

http://192.168.3.60/app/40001/?folder=/home/coder

But when I use spring-cloud-gateway to proxy nginx 80, I continue to report the above error. http://192.168.3.60:30000/app/40001/?folder=/home/coder

please help !

wangyang1749 avatar Dec 26 '23 10:12 wangyang1749

I also encountered it, have you solved it

SweetMojito avatar Jun 18 '24 01:06 SweetMojito

麻烦问下您解决了吗?我也遇到了同样的问题,不知道怎么解决。

birkhoff2017 avatar Jul 09 '24 08:07 birkhoff2017

我的解决了,跟spring-cluod-gateway没啥关系,你检查下你的ng或者代理的后端,是不是他们返回的。

SweetMojito avatar Jul 09 '24 08:07 SweetMojito

对,我检查了,是code server 的docker容器返回的403,这个是怎么解决的呢?

birkhoff2017 avatar Jul 09 '24 08:07 birkhoff2017

我的解决了,跟spring-cluod-gateway没啥关系,你检查下你的ng或者代理的后端,是不是他们返回的。

对,我检查了,是code server 的docker容器返回的403,这个是怎么解决的呢?

birkhoff2017 avatar Jul 09 '24 08:07 birkhoff2017

应该是你跨域了,看下code-server的后台日志

SweetMojito avatar Jul 09 '24 08:07 SweetMojito

我用docker logs查看code-server日志没有有用的信息,还有其他地方能查code-server的日志吗?

birkhoff2017 avatar Jul 09 '24 08:07 birkhoff2017

我用docker logs查看code-server日志没有有用的信息,还有其他地方能查code-server的日志吗?

你进入docker,重启code-server,前台启动,应该就有日志了。

SweetMojito avatar Jul 09 '24 08:07 SweetMojito

前台启动跟docker logs的输出一样,没有报错信息 您后来怎么解决的呢?

birkhoff2017 avatar Jul 09 '24 09:07 birkhoff2017

前台启动跟docker logs的输出一样,没有报错信息 您后来怎么解决的呢?

打开debug日志呢,肯定会有明显的报错信息的

SweetMojito avatar Jul 09 '24 09:07 SweetMojito

我的就是有明显的跨域禁止的报错,header有问题

---Original--- From: @.> Date: Tue, Jul 9, 2024 17:21 PM To: @.>; Cc: @.@.>; Subject: Re: [spring-cloud/spring-cloud-gateway] WebSocketHandshakeExceptionInvalid handshake response getStatus: 403 Forbidden (Issue #3198)

前台启动跟docker logs的输出一样,没有报错信息 您后来怎么解决的呢?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

SweetMojito avatar Jul 09 '24 10:07 SweetMojito

debug host ""10.3.4.26:8088"" does not match origin "10.3.4.26:8088"; 这是我debug日志的报错,您是怎么解决的呢

birkhoff2017 avatar Jul 09 '24 10:07 birkhoff2017

这个报错没看出来不匹配呢,不是一样的吗。我的报错不一样哎

---Original--- From: @.> Date: Tue, Jul 9, 2024 18:30 PM To: @.>; Cc: @.@.>; Subject: Re: [spring-cloud/spring-cloud-gateway] WebSocketHandshakeExceptionInvalid handshake response getStatus: 403 Forbidden (Issue #3198)

debug host ""10.3.4.26:8088"" does not match origin "10.3.4.26:8088"; 这是我debug日志的报错,您是怎么解决的呢

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

SweetMojito avatar Jul 09 '24 14:07 SweetMojito

https://github.com/coder/code-server/blob/main/src/node/http.ts#L362 但是看code server的代码,if (host !== origin) {,是这样判断的,这样的话host多了一个双引号,就和origin不相等了,然后就报错了。 您那边没有这个问题是吗?方便贴一下您使用的gateway的版本,和gateway中跟ws相关的配置文件吗?

birkhoff2017 avatar Jul 10 '24 01:07 birkhoff2017

问题解决了,是gateway转发的header里,forward设置为了proto=http;host="127.0.0.1:8088";for="127.0.0.1:59876",在设置里关闭forwarded和x-forwarded参数,然后设置带上host,让code server取host,然后就可以了。 forwarded: enabled: false x-forwarded: enabled: false

    filters:
    - PreserveHostHeader

birkhoff2017 avatar Jul 10 '24 07:07 birkhoff2017

前台启动跟docker logs的输出一样,没有报错信息 您后来怎么解决的呢?

打开debug日志呢,肯定会有明显的报错信息的

感谢提供的排查思路

birkhoff2017 avatar Jul 10 '24 07:07 birkhoff2017