kostya
Binaries sizes
Adding the size in bytes for the executable binaries (or VM bytecode) generated by the compiler would be very interesting. Bonus points for also mentioning size sum of ldd library dependencies.
There's the dom96/binary_size repo showing that executables produced by Nim are much leaner than Crystal, Rust, D, Haskell, and Go for a "Hello World" example. Adding this column to this benchmark would show the difference for non-trivial examples and many more languages, with very little extra effort if automated.
The tests are not automated yet (however, some efforts are under way to do it). But I agree that binary sizes are interesting to see. There are few caveats though:
- Optimized for performance != optimized for size (as the former may use inlining and that definitely increases the size);
- Some binaries may use static vs shared libraries, therefore it may require changing build options and it can affect performance too;
- Some applications are built to be used as much conveniently as possible (for example, fat JARs). In a way, it's similar to the previous item (static vs shared libs), but related more to the process of using the applications.
Therefore, to make the fair comparison, some tweaks are required and it may diminish the original results. I guess we can modify the runner scripts to report:
- The binary size for native executables;
- Bytecode size for VM-based executables;
- Script size if no compilation is available.
But please note that it won't be the fair comparison.
