spiraldb
Consolidate on Criteria or Divan for benchmarking
My 2c here - I wished I could fork Criterion multiple times, and it seems like Divan is actually maintained which is a big plus in my book
I agree with your point. Seeing criterion have 23 open prs is not a good look for a project with a well defined functionality
Only issue I see atm is that Divan doesn't support async benchmarks? Any ideas how to workaround @AdamGS ?
The first thing I would try is to create the runtime as part of the setup, and then call Runtime::block_on or it's equivalent in the actual benchmark code.
From reading the docs seems like there are some threading-related configs you might have to use, and hopefully everything will play nice together.
Hi! I plan to prioritize async benchmarks after I ship JSON output. 🙂 Follow https://github.com/nvzqz/divan/issues/39 for updates
From reading the docs seems like there are some threading-related configs you might have to use, and hopefully everything will play nice together.
That's separate and unrelated from async. Multi-threaded benchmarks are more so for measuring contention on locks and atomics.
We seem to be settling into hand-written benchmark code a la tpc-h and click bench. Should we close / re-focus this issue?
I think the difference here is big vs small benchmarks, where I would love for smaller benchmarks to be divan/criterion based, and the larger ones can be based on our own homebrewed framework (I assume that number of them is more bounded by the number of "official" benchmarks)