osquery icon indicating copy to clipboard operation
osquery copied to clipboard

Fix a memory leak in `unified_log`

Open Micah-Kolide opened this issue 1 year ago • 0 comments

Relates to #7914

I narrowed down the/a memory leak in unified_log caused from the OSLogEnumerator. Unless all objects are enumerated, they are held in memory.

Example:

Pre PR:

On startup memory consumption looks like this:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 63022 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
 25936 409002864  0.1 63022 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

After running a very heavy unified_log query such as:

select count(*) from unified_log where timestamp >= (select unix_time-100000 from time);

The memory jumps and doesn't release after the query is finished:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 63022 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
559920 410133152  1.7 63022 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

Post PR:

On startup memory consumption looks like this:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 50377 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
 25392 408852336  0.1 50377 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

After running a very heavy unified_log query such as:

select count(*) from unified_log where timestamp >= (select unix_time-100000 from time);

The memory jumps but releases as each OSLogEntryLog is iterated over until the query is finished:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 50377 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
195808 410110032  0.6 50377 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

An unfortunate con to this is of course the increase in runtime while waiting for the iterator to finish.

Micah-Kolide avatar Feb 16 '24 06:02 Micah-Kolide