FAForever
Revamp benchmarking
Adds benchmarking to the profiler and debug classes for functions.
Fixes https://github.com/FAForever/fa/issues/3730
I'm completely open to UI or stats changes by the way. I'm pretty happy with how the bytecode displays (although it requires a lot of work to parse--we're one step away from our own Lua virtual machine at this point), but that's open to feedback too (e.g. if you have a different way to represent jumped-to instructions than just prefixing them with '>').
I don't know what half of the statistics that you included means in practice. I'll have to look that up 😄 ! I don't have any initial feedback on the work done, we may need to tweak some benchmarks to not last a second (have a smaller number of iterations).
Absolutely great work 👍 , I hope this changes our perception as to what is and is not a performance improvement. In particular, using upvalues to cache a table access has not shown to be as effective in practice. Running those benchmarks now reveals that too in some cases.
I already set a metadata system for the benchmarks (e.g. for display like this), so it'd be a simple thing to add an iteration count to it if we need certain benchmarks to use fewer iterations.
As for the statistics... it's looking like we might need to reevaluate what information is actually useful (e.g. Excess Kurtosis [how much more "tailedness" the data has compared to the normal distribution] is always about -1.5). I rewrote the statistics file more as a library that we could reuse elsewhere, so we shouldn't feel bad about not using everything it has to offer.
There we go--I synced benchmark progress (though now it's a little slower) and implemented tooltips and custom benchmark runs
WARNING: Error running OnMouseoverItem script in CScriptObject at 1b4a0c00: ...\jip\documents\delete-me\fa\lua\ui\game\profiler.lua(927): attempt to call method `AddressToString' (a nil value)
stack traceback:
...\jip\documents\delete-me\fa\lua\ui\game\profiler.lua(927): in function `BytecodeTooltip'
...\jip\documents\delete-me\fa\lua\ui\game\profiler.lua(677): in function <...\jip\documents\delete-me\fa\lua\ui\game\profiler.lua:674>
It appears to be related to a tooltip, in particular this line causes it:
These return invalid results:
It appears to load the file in the UI, causing this particular up value to fail:
We can change the benchmark.
@Hdt80bro do you have the intention to finish this pull request? If not, then you can close it