forwarding in the envoy doesn't deal with `https` correctly

[GKilm]

Describe the bug When the protocol in the debugged pod is https, forwarding is not handled correctly with https.

Mention the platform you are using

• System : Windows 10
• Remote Machine : Centos 7.9
• vscode version: 1.74.3

To Reproduce Steps to reproduce the behavior:

1. to debug the pod which protocol is https
2. invoke the service

Additional context I has tried to edit the config about envoy which is /etc/envoy/envoy.yaml,it works. But I cann't revise the envoy depoyment and the envoy configmap in the cluster,they are always restored right away.How are they controlled?

Here are some of my attempts that might be helpful. envoy.yaml

static_resources:
  listeners:
  - name: listener_30888_30888
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 30888
    listener_filters:
    - name: envoy.filters.listener.http_inspector
    filter_chains:
    - filter_chain_match:
        application_protocols:
        - http/1.0
        - http/1.1
        - h2c
      filters:
      - name: envoy.http_connection_manager
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: listener_30888_30888
          route_config:
            name: listener_30888_30888_route
            virtual_hosts:
            - name: listener_30888_30888_route_default
              domains:
              - '*'
              routes:
              - match:
                  headers:
                  - name: kubernetes-route-as
                    exact_match: root-g
                  prefix: /
                route:
                  cluster: service_debug_withHeader_kubernetes-route-as_root-g_30888_30888
                  timeout: 0s
                  idle_timeout: 0s
              - match:
                  prefix: /
                route:
                  cluster: service_original_clone_30888_30888
                  timeout: 0s
                  idle_timeout: 0s
          http_filters:
          - name: envoy.filters.http.router
        # to add
        transport_socket:
          name: envoy.transport_sockets.tls
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
            common_tls_context:
              tls_certificates:
                - certificate_chain:
                    filename: "/etc/yunion/pki/service.crt"
                  private_key:
                    filename: "/etc/yunion/pki/service.key"  
clusters:
  - name: service_original_clone_30888_30888
    connect_timeout: 1.00s
    type: strict_dns
    load_assignment:
      cluster_name: service_original_clone_30888_30888
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: default-region-cloned-routing-svc.onecloud
                port_value: 30888
  - name: service_debug_withHeader_kubernetes-route-as_root-g_30888_30888
    connect_timeout: 1.00s
    type: static
    load_assignment:
      cluster_name: service_debug_withHeader_kubernetes-route-as_root-g_30888_30888
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 10.40.216.46
                port_value: 30888
    # to add
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext

The file mentioned in tls_certificates is added manually. With the transport_socket added, forwarding is ok.

reference

• envoy-secure

[hsubramanianaks]

@GKilm Thanks for raising this issue. Can you provide logs where bridge is failing to forward when protocol is https? That would be helpful for us to triage further.

[GKilm]

What logs are needed?

It is clear that Envoy does not handle https,but the service being proxied is https. The enovy pod receives https request, then returns information that is not supported.

[hsubramanianaks]

@GKilm Thanks for your response. Logs would be in your C:\Users\youruserid\AppData\Local\Temp\Bridge To Kubernetes, please share the logs from this location to further investigate.

[elenavillamil]

@GKilm, sorry for the late response, trying to catch up to all issues. I believe this file is where all the envoy configurations are placed. It seems like your comment is that when isolation is on, http is supported but https is not. Your request would be to add https support to bridge. I also understand from the issue you have tried adding transport_socket configuration and with this configuration it works for you. Can you confirm my understanding of the ask is correct? Once you do, we will add this ask to our planning. Please note we are small team. I also always like to mention we welcome contributions in case you would be interested :)

[GKilm]

@elenavillamil You are right.The restatement of the question is fantastic. I 'm also happy to contribute If I have enough ability :)

[hsubramanianaks]

@GKilm Thanks for the confirmation. are you using this service with ingress controller or just a load balancer with https ? I am looking at the your initial request, seems like you are using appProtocol as https ? Please confirm. Thanks.

[GKilm]

Yes,the service protocol as https is configured while the pod starts. To what I know,the Service only supports Layer 4 networks.

[hendrikgaffo]

Are there any news on this?

We have a similar issue. We are running a WebSocket Service (using Socket.IO) on an AKS cluster with Bridge to Kubernetes installed. We are using the NGINX Ingress Controller for our services.

Whenever we are running the debugger with isolation mode enabled, any attempt to connect to said service returns a 403: "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 403

If we turn isolation mode off, we get the proper 101 (Switching Protocols): "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 101

This has to do with the fact that in isolation mode, the request is being made to the Envoy pod instead of the original pod.

We thought about adding this config in the Envoy config under HttpConnectionManager in order to enable WebSocket connections:

upgrade_configs:
  upgrade_type: "websocket"

However, the Envoy config resets itself, as @GKilm mentioned.