client_python
client_python copied to clipboard
Use rust to improve reading .db file performance
This is an idea to try to improve multiprocess collect performance by using rust to collect the metrics from various .db files. Current state of the benchmarks shows a 2x improvement on my machine (x86 linux).
---------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_speedup_collect_performance 33.9249 (1.0) 35.3835 (1.0) 34.6150 (1.0) 0.3620 (1.0) 34.6984 (1.0) 0.5581 (1.0) 6;0 28.8892 (1.0) 20 1
test_native_collect_performance 70.0792 (2.07) 81.2032 (2.29) 72.6370 (2.10) 3.4046 (9.41) 70.7422 (2.04) 3.4498 (6.18) 2;1 13.7671 (0.48) 13 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------