collector icon indicating copy to clipboard operation
collector copied to clipboard

Count the number of throttled log message and print it.

Open ovalenti opened this issue 2 years ago • 1 comments

Description

We have recently increased significantly the delay for some throttled messages.

In order to get a better idea of the amount of messages, this PR adds a counter.

Checklist

inspected CI test results for throttled messages:

  • [x] Seen a message not throttled.
  • [x] Seen a message which was throttled.

Testing

With a container generating 100 zombie processes, the collector contains:

...
[INFO    2024/07/05 19:12:09] Successfully established GRPC stream for signals.
[INFO    2024/07/05 19:12:09] Found self-check process event.
[ERROR   2024/07/05 19:12:09] Could not determine network namespace: No such file or directory
[INFO    2024/07/05 19:12:10] Found self-check connection event.
[INFO    2024/07/05 19:12:19] self-check (pid=76) exited with status: 0
[INFO    2024/07/05 19:12:39] Flushing thread table
[INFO    2024/07/05 19:12:39] Flushing container table
[ERROR   2024/07/05 19:12:39] [Throttled 99 messages] Could not determine network namespace: No such file or directory
[INFO    2024/07/05 19:13:09] Flushing container table
[ERROR   2024/07/05 19:13:09] [Throttled 99 messages] Could not determine network namespace: No such file or directory
...

ovalenti avatar Jan 26 '24 09:01 ovalenti

I will approve after merge conflicts are fixed.

JoukoVirtanen avatar Aug 01 '24 21:08 JoukoVirtanen