Assigned IP addresses not released when container is destroyed

[kkuphal]

Summary

Container created using docker run with simple network configuration

docker run -it -rm -p 162:162/udp --name snmptrapd --network vic-container-network snmptrapd

After container is terminated, docker inspect network shows IP addresses assigned to the deleted container that cannot be reused or removed

docker inspect vic-container-network [ { "Name": "vic-container-network", "Id": "369341720b6e041e4aabb48e6871677106dff0fea080cc542572b867b4a98256", "Created": "2019-04-23T18:21:44.649754577Z", "Scope": "", "Driver": "external", "EnableIPv6": false, "IPAM": { "Driver": "", "Options": {}, "Config": [ { "Subnet": "192.168.4.0/24", "IPRange": "192.168.4.140-192.168.4.199", "Gateway": "192.168.4.254" } ] }, "Internal": false, "Attachable": false, "Containers": { "0046e74521eb4b46ec6e0b382ed7e82da91980135356653d1567882d14554213": { "Name": "traefik-mcm995-103866763972", "EndpointID": "0046e74521eb4b46ec6e0b382ed7e82da91980135356653d1567882d14554213", "MacAddress": "", "IPv4Address": "192.168.4.140/24", "IPv6Address": "" }, "45fa4915e092b157dc37587e5d0aca466ff5397e010f9a28f3fade398f000446": { "Name": "snmptrapd", "EndpointID": "45fa4915e092b157dc37587e5d0aca466ff5397e010f9a28f3fade398f000446", "MacAddress": "", "IPv4Address": "192.168.4.142/24", "IPv6Address": "" }, "a4c3431b6a3ba2551759371ecc1061666d6977660c335b3e859f7825e8f45c60": { "Name": "snmptrapd", "EndpointID": "a4c3431b6a3ba2551759371ecc1061666d6977660c335b3e859f7825e8f45c60", "MacAddress": "", "IPv4Address": "192.168.4.143/24", "IPv6Address": "" }, "d03b544ec0f58f4e2bfd0187b30fa01507f7578f3356f74a0ca44f262e38396d": { "Name": "snmptrapd", "EndpointID": "d03b544ec0f58f4e2bfd0187b30fa01507f7578f3356f74a0ca44f262e38396d", "MacAddress": "", "IPv4Address": "192.168.4.141/24", "IPv6Address": "" } }, "Options": {}, "Labels": {} } ]

docker container rm d03 Error: No such container: d03

Environment information

vSphere and vCenter Server version

vCenter 6.5.0.23000 vSphere 6.5.0, 11925212

VIC Appliance version

vic-v1.5.2-7206-92ebfaf5

Configuration

• Embedded or external PSC: Embedded
• How was the OVA deployed? (Flex client, HTML5 client, ovftool): Flex client
• Does the VIC appliance recieve configuration by DHCP? No. VIC is statically assigned
• What stage of the Appliance Lifecycle is the VIC appliance in? Running

• IP address of VIC appliance: 192.168.5.103
• Hostname of VIC appliance: vic.fhpaschen.com
• IP address of vCenter Server: 192.168.5.114
• Hostname of vCenter Server: fhp-vcbe.fhpaschen.com

Details

Steps to reproduce

Create VCH with static IP range

--container-network vic-public:vic-container-network --container-network-ip-range vic-public:192.168.4.140-192.168.4.199 --container-network-gateway vic-public:192.168.4.254/24 --container-network-dns vic-public:192.168.4.98

Actual behavior

Allocated IP addresses are not released on container removal

Expected behavior

Allocated IP addresses are released on container removal

Logs

See also

Troubleshooting attempted

• [ ] Searched GitHub for existing issues. (Mention any similar issues under "See also", above.)
• [ ] Searched the documentation for relevant troubleshooting guidance.
• [ ] Searched for a relevant VMware KB article.

[omnidepp]

In addition, it seems like manually removing containers containers from a network via docker netwocker disconnect is not implemented yet. What other options are there besides destroying and recreating the entire network?

[dafik]

I'm affected this issue also.

This was working in 1.4.3

Now for workaround i must stop all containers, restart VCH , start containers. If i use docker-compose, every time i changed eg. env variable, containers are recreated, IP doesn't return to "free" pool, restart VCH is needed :/

[yuyangbj]

could you please run 'docker ps -a' to see the output? In 1.5.2 we fixed an ip conflict issue, so we do not release the ip address until the container is deleted. Terminated container is only stop the container. Please run docker rm to delete container.

[yuyangbj]

@kkuphal @dafik I am waiting for your response. I will close it if no comment.

[dafik]

$ cat docker-compose.yml 
version: "3.4"

networks:
  sinternal:
    internal: true
    ipam:
      config:
        - subnet: 172.69.0.0/16

services:
  db1:
    image: *********
    networks:
      sinternal:
        ipv4_address: 172.69.0.10

$ docker-compose ps
   Name                 Command              State    Ports  
-------------------------------------------------------------
test_db1_1   docker-entrypoint.sh postgres   Up      5432/tcp

lets change somthing in config eg. add env variable.

$ cat docker-compose.yml 
version: "3.4"

networks:
  sinternal:
    internal: true
    ipam:
      config:
        - subnet: 172.69.0.0/16

services:
  db1:
    image: *********
    networks:
      sinternal:
        ipv4_address: 172.69.0.10
    restart: always
    environment:
      POSTGRES_PASSWORD: xxxxx

restart services

$ docker-compose up -d
Recreating test_db1_1 ... 
Recreating test_db1_1 ... error

ERROR: for test_db1_1  Cannot start service db1: Server error from portlayer: Cannot reserve IP range 172.69.0.10 - 172.69.0.10.  Already in use

ERROR: for db1  Cannot start service db1: Server error from portlayer: Cannot reserve IP range 172.69.0.10 - 172.69.0.10.  Already in use

$ docker-compose ps   
         Name                        Command              State     Ports  
---------------------------------------------------------------------------
55a23d5e2054_test_db1_1   docker-entrypoint.sh postgres   Exit 0   5432/tcp
test_db1_1                docker-entrypoint.sh postgres   Exit 0   5432/tcp

[dafik]

@yuyangbj do you need any additional data to address this issue?

[pmagn]

We are also experiencing this issue on version 1.5.2 of VCH

[samized]

Me too, on version 1.5.5

[bognerf]

Same here, v1.5.6

This renders compose up -d to recreate changed containers with static IP addresses quite useless. As soon as one uses external networks to connect different docker-compose configurations interdependently, dynamic IP addresses are not useable.

Currently, one has to destroy a docker-compose container manually to release its IP address, before a recreation via docker-compose is even possible.