scope icon indicating copy to clipboard operation
scope copied to clipboard

Don't have TCP established connection from client but still show connection on the map

Open thaonguyen-ct opened this issue 5 years ago • 2 comments

Let's say, I have two pods A and B are running on K8s. A(source) sends a TCP request to B(dest) and both have a TCP established connection, but somehow pod A is stopped suddenly and don't send FIN package to close this TCP connection, so the TCP connection is still established on pod B and on pod A has gone. But the response of weave-scope-app API of deployment B still has the inbound connection from A and the response of weave-scope-app API of deployment A still have the outbound connection to B even pod A doesn't send TCP request to B anymore.

What you expected to happen?

Don't see the connection between A and B or at least the don't have the outbound connection on deployment A

What happened?

The connection from A to B still exist on both deployment A and deployment B when call weave-scope-app api

How to reproduce it?

  1. Deploy two deployment A and B like Nginx for example
  2. From pod A telnet to pod B and you can see the TCP established connection on pod A and B by check netstat -nat
  3. From pod A install the iptables and drop outbound of FIN package iptables -I OUTPUT --protocol tcp --tcp-flags FIN FIN -j DROP
  4. Kill the telnet command from pod A or crtl-c
  5. You can see the TCP connection still established on pod B but not on pod A
  6. Call API of weave-scope-app to verify the inbound connection of pod B and outbound connection of pod A still there http:///api/topology/kube-controllers/;%3Cdeployment%3E

Versions:

Weave scope: 1.13.1 Kubernetes: 1.13.3 Docker on kubernetes node: 18.6.3

thaonguyen-ct avatar Nov 21 '20 02:11 thaonguyen-ct

It sounds like Scope is telling you what Linux is telling it.

The line on the graph goes away, right? Because one of the process endpoints is now missing.

bboreham avatar Nov 30 '20 11:11 bboreham

@bboreham No. The line on the graph is still appear

thaonguyen-ct avatar Nov 30 '20 12:11 thaonguyen-ct