map_benchmark icon indicating copy to clipboard operation
map_benchmark copied to clipboard

CreateInsert & ClearInsert benchmarks

Open camel-cdr opened this issue 2 years ago • 1 comments

The benchmarks are kind of similar to InsertHugeInt, but stress test different sizes, some maps seem to really struggle after certain sizes:

ClearInsert

CreateInsert

(The above was just a quick average of two benchmark with the timeout set to 100s)


This was the last benchmark I had lying around on my system, but I think an erase-if benchmark is still missing.

I tried implementing one today, but I don't really see a good way of doing it, as creating the map is always way more expensive than the erase-if. Also, abseil has non-standard iterators, and I'm not sure if all the other maps confirm to the standard implementation.

camel-cdr avatar Mar 12 '23 22:03 camel-cdr

@camel-cdr I have add your new changes into my benchmark, It's a single cpp file and easy to be benchmarked and compared with each other. https://github.com/ktprime/emhash/blob/master/bench/martinus_bench.cpp

hyb@LAPTOP-8JBPPNI4:/mnt/d/emhash/bench$ make BF=1 SW=1 std=20 g++ -flto -O3 -march=native -mtune=native -I.. -I../thirdparty -DNDEBUG=1 -DEMH_ALLOC=0 -Wall -DABSL_HMAP=1 -DHAVE_BOOST=1 -std=c++20 -static -static-libstdc++ martinus_bench.cpp -o mbench hyb@LAPTOP-8JBPPNI4:/mnt/d/emhash/bench$ ./mbench b9 clang 14.0.0 on llvm/gcc 4.2.1 __cplusplus = 202002 x86-64 OS = linux, cpu = AMD Ryzen 7 5800H with Radeon Graphics

bench_knucleotide:
                    jg_dense (lf=0.50)  ans = 1967415 time = 5.61 s
                     rigtorp (lf=0.33)  ans = 1967415 time = 3.44 s
              martinus dense (lf=0.50)  ans = 1967415 time = 6.33 s
                  boost flat (lf=0.54)  ans = 1967415 time = 3.30 s
                   absl flat (lf=0.50)  ans = 1967415 time = 2.53 s
                     emhash6 (lf=0.50)  ans = 1967415 time = 4.19 s
                     emhash5 (lf=0.50)  ans = 1967415 time = 4.02 s
                     emhash7 (lf=0.50)  ans = 1967415 time = 4.33 s
                     emhash8 (lf=0.50)  ans = 1967415 time = 4.89 s
                     emilib2 (lf=0.50)  ans = 1967415 time = 4.47 s
                     emilib3 (lf=0.50)  ans = 1967415 time = 4.44 s

bench_GameOfLife:
                     emhash6 stastabilizing = 6.27, grow = 2.66 (total 8.92) time s
                     emhash5 stastabilizing = 7.17, grow = 2.98 (total 10.16) time s
                     emhash7 stastabilizing = 6.70, grow = 2.77 (total 9.47) time s
                     emhash8 stastabilizing = 5.91, grow = 2.67 (total 8.58) time s
                    jg_dense stastabilizing = 7.41, grow = 2.98 (total 10.39) time s
                     rigtorp stastabilizing = 8.08, grow = 3.19 (total 11.27) time s
              martinus dense stastabilizing = 7.52, grow = 3.52 (total 11.03) time s
                  boost flat stastabilizing = 6.08, grow = 2.33 (total 8.41) time s
                   absl flat stastabilizing = 7.28, grow = 2.62 (total 9.91) time s
                     emilib2 stastabilizing = 5.30, grow = 2.03 (total 7.33) time s
                     emilib3 stastabilizing = 5.48, grow = 2.16 (total 7.64) time s

ktprime avatar Mar 14 '23 09:03 ktprime