ngrep fails when using libpcap v1.8.1 and newer kernel version with warning kernel filter failed

[thimslugga]

Using ngrep with more than 6 filters on a newer kernel version such as 4.18, 5.0, 5.3 or 5.4 will result in a complete fail when libpcap v1.8.1 is used (e.g. ubuntu 18.04). ngrep with more than 6 filters with libpcap 1.9.1 will run but still returns a warning about kernel filter failed: cannot allocate memory.

The complete fail with libpcap v1.8.1 appears to have been fixed around 1.9.0. As for the warnings "Warning: Kernel filter failed: Cannot allocate memory", which persists even with libpcap 1.9.1, I suspect this is due to recent bug fixes and improvements to the bpf verifier as well as ngrep attempting to apply filters in a non-optimal way. I do not experience any of these same failures or warnings when attempting to use tcpdump with more than 6 filters.

• https://www.tcpdump.org/libpcap-changes.txt
• 1.9.0-rc1 - https://github.com/the-tcpdump-group/libpcap/commit/995849abbf59194665367f95453ef153eef4a70b Fix some Linux memory-mapped capture buffer size issues Don't fail if kernel filter can't be set on Linux (GitHub issue #549)
• https://github.com/the-tcpdump-group/libpcap/issues/549

Some bug fixes to bpf verifier:

• https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=979d63d50c0c0f7bc537bf821e056cc9fe5abd38
• https://nvd.nist.gov/vuln/detail/CVE-2019-7308
• https://blog.cloudflare.com/ebpf-cant-count/

libpcap 1.8.1 + kernel 4.18, 5.0, 5.3, 5.4 Output:

# uname -sr
Linux 5.4.0-42-generic

# /usr/bin/ngrep -V
ngrep: V1.47.1-git, libpcap version 1.8.1

# ngrep -q -t -d any -W byline '' port 123 or 6001 or 6010 or 6002 or 6003 or 6004 or 6005
interface: any
Warning: Kernel filter failed: Cannot allocate memory
Warning: Kernel filter failed: Cannot allocate memory
pcap: can't remove kernel filter: No such file or directory

libpcap 1.9.1 + kernel 4.18, 5.0, 5.3, 5.4 Output:

# uname -sr
Linux 4.18.0-1020-aws

# LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ./ngrep -V
ngrep: V1.47.1-git, libpcap version 1.9.1 (with TPACKET_V3)

# LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH /usr/local/src/ngrep*/ngrep -q -t -d any -W byline '' \(port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004\)
interface: any
Warning: Kernel filter failed: Cannot allocate memory
filter: ( (port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004) ) and (ip)
U 2020/09/01 13:46:22.778659 10.10.11.123:42044 -> 91.189.89.199:123 #1
#............................................i..

U 2020/09/01 13:46:22.852287 91.189.89.199:123 -> 10.10.11.123:42044 #2
$......U......"{...a.W.X.....i........a8......y.

libpcap 1.8.1 + kernel 4.15

# uname -sr
Linux 4.15.0-1080-aws

# ngrep -q -t -d any -W byline '' \(port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004\)
interface: any
filter: ( (port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004) ) and (ip)

U 2020/09/01 13:57:07.137353 10.10.11.123:54195 -> 91.189.94.4:123 #1
#............................................/..

U 2020/09/01 13:57:07.212640 91.189.94.4:123 -> 10.10.11.123:54195 #2
$...........^..
.....n......./......-..)....-...