unable to uninstall calico and get stuck on 'test-install-calicovpp-dev'

[sheercoding]

Environment

• Calico/VPP version: master,bff8eba4cf614b9f3591150ded186e914a0105ab
• Kubernetes version: V1.23.12
• Deployment type: VM
• Network configuration: Calico / host-local IPAM? IPIP / VXLAN enabled? Specific BGP settings?

Issue description Can't remove calico pods when 'make test-install-calicovpp-dev'.

To Reproduce Steps to reproduce the behavior:

• 1> successfully set up develop env according to 'https://github.com/projectcalico/vpp-dataplane/blob/master/docs/developper_guide.md'
• 2> change&modify VPP then reload vpp-nodes by 'make load-images'
• 3> update calico-vpp by 'make test-install-calicovpp-dev'

Expected behavior Remove the old version calico-app and successfully upgrade the new one.

Additional context Some of the pods stuck on the status of 'Terminating'. $k get pods -A -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-apiserver calico-apiserver-69fcc8ccf5-8j9qt 0/1 Terminating 0 7d19h 172.16.104.2 node2 calico-apiserver calico-apiserver-69fcc8ccf5-xv74m 0/1 Terminating 0 7d19h 172.16.166.130 node1 calico-system calico-kube-controllers-555bcb4844-zhn9k 0/1 CrashLoopBackOff 7 (44s ago) 7d19h 172.16.219.67 master calico-system calico-node-gnmkf 1/1 Running 0 7d19h 192.168.0.12 node2 calico-system calico-node-p55z9 1/1 Running 0 7d19h 192.168.0.254 master calico-system calico-node-tdvw2 1/1 Running 0 7d19h 192.168.0.11 node1 calico-system calico-typha-844764bf54-27ps6 1/1 Running 0 7d19h 192.168.0.11 node1 calico-system calico-typha-844764bf54-d8z6b 1/1 Running 0 7d19h 192.168.0.12 node2 calico-system csi-node-driver-4vrp6 2/2 Running 0 7d19h 172.16.219.68 master calico-system csi-node-driver-b5lxz 2/2 Running 0 7d19h 172.16.104.1 node2 calico-system csi-node-driver-tt9k2 2/2 Running 0 7d19h 172.16.166.129 node1 calico-vpp-dataplane calico-vpp-node-97trd 2/2 Running 1 (7d19h ago) 7d19h 192.168.0.12 node2 calico-vpp-dataplane calico-vpp-node-s87g5 2/2 Running 0 7d19h 192.168.0.254 master calico-vpp-dataplane calico-vpp-node-x4mmk 2/2 Running 1 (7d19h ago) 7d19h 192.168.0.11 node1 kube-system coredns-64897985d-b74hd 1/1 Running 0 7d19h 172.16.219.66 master kube-system coredns-64897985d-wfl6l 1/1 Running 0 7d19h 172.16.219.65 master kube-system etcd-master 1/1 Running 0 7d19h 192.168.0.254 master kube-system kube-apiserver-master 1/1 Running 0 7d19h 192.168.0.254 master kube-system kube-controller-manager-master 1/1 Running 0 7d19h 192.168.0.254 master kube-system kube-proxy-9svks 1/1 Running 0 7d19h 192.168.0.254 master kube-system kube-proxy-bpzlp 1/1 Running 0 7d19h 192.168.0.11 node1 kube-system kube-proxy-kbr6g 1/1 Running 0 7d19h 192.168.0.12 node2 kube-system kube-scheduler-master 1/1 Running 0 7d19h 192.168.0.254 master

================ make test-install-calicovpp-dev make test-install-calico CALICO_INSTALLATION=installation-test-v4 make[1]: Entering directory '/home/douchao/project/vpp-dataplane' kubectl replace --force -f https://raw.githubusercontent.com/projectcalico/calico/master/manifests/tigera-operator.yaml customresourcedefinition.apiextensions.k8s.io "installations.operator.tigera.io" deleted

error: timed out waiting for the condition make[1]: *** [Makefile:92: test-install-calico] Error 1 make[1]: Leaving directory '/home/douchao/project/vpp-dataplane' make: *** [Makefile:99: test-install-calicovpp] Error 2

[hedibouattour]

Hello, sorry for the late reply. I managed to reproduce this issue. This actually is not a calico-vpp issue as I can reproduce it with calico itself as well. It looks to be some mismatch between tigera operator job and vagrant setup. Operator is not launching the uninstallation of some components for some reason. I am still investigating that. Will keep you updated.