opnsense
VirtIO (vtnet) adapters not handling inbound/outbound traffic above L2 in link-local subnets
Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
- [x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
- [x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue
Describe the bug
VirtIO adapter handling continues to be problematic. At current revision (22.1.10), seeing several devices in our OpenStack forward traffic correctly, register ARP table entries, but not handle inbound or outbound packets at L3 or above in the local subnet (or ICMP from anywhere). Forwarding and filtering works fine for forwarded traffic, ICMP over IPSEC tunnels works fine, as do inbound routed connections going into the device through another gateway at layer 4 (management ports). This seems to have something to do with link-local traffic.
When pinging the device from an in-subnet host i can see the first ICMP packet on tcpdump come into the interface on the OPNSense VM, but no reply, and no subsequent packets coming in either. Pinging a host in-subnet from the device, i see no such packet come into the target host (its local firewall is off and its security group rules permit all IP traffic from the OPNSense instance).
Flipping the offload switches has no effect.
This "smells of filtering" but i have enabled ingress on all protocols from the local subnet (and the routed subnet in which the administrative machine connecting to the routed administrative ports is connecting successfully) and still no ICMP. Concurrently netflow and syslog from the device are not reaching host in its LAN subnet. Security group rules permit all in-subnet traffic between instances, problem persists even with completely disabling security groups for the instance (both LAN and WAN ports).
@fichtner - i think this actually a PF issue with VirtIO. Turns out that when you service pf onestop, local-net connectivity is instantly restored. I've tried disabling all of the vtnet-specific kernel tunables and stumbled on to the pf-stopping thing somewhat by accident.
@sempervictus it could be anything from NAT to scrub to antispoof, but at least it would indicate this is solvable.
