[inetstack] System Calls on the Fast Path

[carvalhof]

Description

I am observing some calls (that seem to be system calls) running tcp-ping-pong example using Catnip in the dev branch (commit 5730393b274d067a5a4a1324e4d3ecd6f396d3e6). I recorded events using perf and plotted the Flame Graph (attached). As we can see, at some points, there is something that looks like some system calls, like in demikernel::scheduler::scheduler::Scheduler::insert calling __GI___libc_malloc, in demikernel::catnip::runtime::DPDKRuntime::transmit calling __memcpy_avx_unaligned, in demikernel::inetstack::Inetstack::poll_bg_work calling __clock_gettime_2, and others.

Are they system calls? Can't they affect Demikernel's performance?

How to Reproduce

Steps to reproduce the behavior:

1. Increase the number of rounds from 1024 to 512*1024;
2. Remove the sanity check;
3. Remove the println for pings (and pongs);
4. Run tcp-ping-pong as server at the node0
5. Run tcp-ping-pong as client at the node1.

Flame Graph