fast-elixir
fast-elixir copied to clipboard
devonestes
Compare membership
Branched off of update-elixir-erlang, will rebase once that's been merged.
Compares various methods for checking whether an element is a member of an enumerable.
I'll add the README section once the benchmark code gets a thumbs up.
I removed benchmarks that tested misses (i.e. element not a member) to keep the results more focused, but can revert 917386a if we think it's useful to include. Basically, MapSet.member?/2 is way more performant for large n.
Click to see full benchmark results
Operating System: macOS
CPU Information: Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
Number of Available Cores: 4
Available memory: 16 GB
Elixir 1.8.2
Erlang 22.0.1
Benchmark suite executing with the following configuration:
warmup: 2 s
time: 10 s
memory time: 0 ns
parallel: 1
inputs: Large (in range), Large (out of range), Medium (in range), Medium (out of range), Small (in range), Small (out of range)
Estimated total run time: 6 min
Benchmarking Enum.any? with input Large (in range)...
Benchmarking Enum.any? with input Large (out of range)...
Benchmarking Enum.any? with input Medium (in range)...
Benchmarking Enum.any? with input Medium (out of range)...
Benchmarking Enum.any? with input Small (in range)...
Benchmarking Enum.any? with input Small (out of range)...
Benchmarking Enum.member? with input Large (in range)...
Benchmarking Enum.member? with input Large (out of range)...
Benchmarking Enum.member? with input Medium (in range)...
Benchmarking Enum.member? with input Medium (out of range)...
Benchmarking Enum.member? with input Small (in range)...
Benchmarking Enum.member? with input Small (out of range)...
Benchmarking MapSet.member? with input Large (in range)...
Benchmarking MapSet.member? with input Large (out of range)...
Benchmarking MapSet.member? with input Medium (in range)...
Benchmarking MapSet.member? with input Medium (out of range)...
Benchmarking MapSet.member? with input Small (in range)...
Benchmarking MapSet.member? with input Small (out of range)...
Benchmarking New MapSet + MapSet.member? with input Large (in range)...
Benchmarking New MapSet + MapSet.member? with input Large (out of range)...
Benchmarking New MapSet + MapSet.member? with input Medium (in range)...
Benchmarking New MapSet + MapSet.member? with input Medium (out of range)...
Benchmarking New MapSet + MapSet.member? with input Small (in range)...
Benchmarking New MapSet + MapSet.member? with input Small (out of range)...
Benchmarking x in y with input Large (in range)...
Benchmarking x in y with input Large (out of range)...
Benchmarking x in y with input Medium (in range)...
Benchmarking x in y with input Medium (out of range)...
Benchmarking x in y with input Small (in range)...
Benchmarking x in y with input Small (out of range)...
##### With input Large (in range) #####
Name ips average deviation median 99th %
MapSet.member? 990.69 K 1.01 μs ±4443.46% 1 μs 2 μs
Enum.member? 1.44 K 695.55 μs ±59.44% 687 μs 1427.72 μs
x in y 1.32 K 756.34 μs ±66.04% 727 μs 1984 μs
Enum.any? 0.0834 K 11986.08 μs ±61.53% 12136.01 μs 29520.32 μs
New MapSet + MapSet.member? 0.00452 K 221097.33 μs ±11.90% 229234.50 μs 307566 μs
Comparison:
MapSet.member? 990.69 K
Enum.member? 1.44 K - 689.08x slower +694.54 μs
x in y 1.32 K - 749.31x slower +755.33 μs
Enum.any? 0.0834 K - 11874.55x slower +11985.08 μs
New MapSet + MapSet.member? 0.00452 K - 219039.93x slower +221096.32 μs
##### With input Large (out of range) #####
Name ips average deviation median 99th %
MapSet.member? 1084461.54 0.00092 ms ±4546.06% 0.00100 ms 0.00200 ms
Enum.member? 695.59 1.44 ms ±10.06% 1.40 ms 2.15 ms
x in y 645.85 1.55 ms ±26.14% 1.45 ms 2.73 ms
Enum.any? 42.58 23.48 ms ±14.22% 22.57 ms 37.92 ms
New MapSet + MapSet.member? 4.55 219.98 ms ±11.65% 227.22 ms 307.94 ms
Comparison:
MapSet.member? 1084461.54
Enum.member? 695.59 - 1559.06x slower +1.44 ms
x in y 645.85 - 1679.13x slower +1.55 ms
Enum.any? 42.58 - 25467.02x slower +23.48 ms
New MapSet + MapSet.member? 4.55 - 238561.55x slower +219.98 ms
##### With input Medium (in range) #####
Name ips average deviation median 99th %
MapSet.member? 1313.78 K 0.76 μs ±5227.46% 1 μs 1 μs
Enum.member? 148.53 K 6.73 μs ±155.66% 7 μs 14 μs
x in y 140.91 K 7.10 μs ±255.14% 7 μs 17 μs
Enum.any? 7.99 K 125.21 μs ±136.51% 113 μs 417.00 μs
New MapSet + MapSet.member? 0.75 K 1337.37 μs ±6.99% 1310 μs 1780.66 μs
Comparison:
MapSet.member? 1313.78 K
Enum.member? 148.53 K - 8.85x slower +5.97 μs
x in y 140.91 K - 9.32x slower +6.34 μs
Enum.any? 7.99 K - 164.50x slower +124.45 μs
New MapSet + MapSet.member? 0.75 K - 1757.01x slower +1336.61 μs
##### With input Medium (out of range) #####
Name ips average deviation median 99th %
MapSet.member? 1324.94 K 0.75 μs ±5340.65% 1 μs 1 μs
Enum.member? 77.05 K 12.98 μs ±73.50% 13 μs 19 μs
x in y 71.57 K 13.97 μs ±194.29% 13 μs 27 μs
Enum.any? 4.49 K 222.85 μs ±44.82% 200 μs 455 μs
New MapSet + MapSet.member? 0.74 K 1351.54 μs ±9.65% 1316 μs 1941.11 μs
Comparison:
MapSet.member? 1324.94 K
Enum.member? 77.05 K - 17.19x slower +12.22 μs
x in y 71.57 K - 18.51x slower +13.22 μs
Enum.any? 4.49 K - 295.27x slower +222.10 μs
New MapSet + MapSet.member? 0.74 K - 1790.71x slower +1350.78 μs
##### With input Small (in range) #####
Name ips average deviation median 99th %
MapSet.member? 1.38 M 724.38 ns ±5501.26% 1000 ns 1000 ns
Enum.member? 1.32 M 754.88 ns ±5251.67% 1000 ns 1000 ns
x in y 1.28 M 782.09 ns ±5340.69% 1000 ns 1000 ns
Enum.any? 0.59 M 1705.19 ns ±1344.36% 2000 ns 3000 ns
New MapSet + MapSet.member? 0.153 M 6532.43 ns ±221.05% 6000 ns 11000 ns
Comparison:
MapSet.member? 1.38 M
Enum.member? 1.32 M - 1.04x slower +30.50 ns
x in y 1.28 M - 1.08x slower +57.71 ns
Enum.any? 0.59 M - 2.35x slower +980.81 ns
New MapSet + MapSet.member? 0.153 M - 9.02x slower +5808.05 ns
##### With input Small (out of range) #####
Name ips average deviation median 99th %
MapSet.member? 1.39 M 721.88 ns ±5766.50% 1000 ns 1000 ns
Enum.member? 1.24 M 806.11 ns ±4965.41% 1000 ns 1000 ns
x in y 1.19 M 837.44 ns ±5384.61% 1000 ns 1000 ns
Enum.any? 0.37 M 2730.09 ns ±811.43% 3000 ns 4000 ns
New MapSet + MapSet.member? 0.140 M 7141.31 ns ±378.30% 6000 ns 20000 ns
Comparison:
MapSet.member? 1.39 M
Enum.member? 1.24 M - 1.12x slower +84.23 ns
x in y 1.19 M - 1.16x slower +115.57 ns
Enum.any? 0.37 M - 3.78x slower +2008.22 ns
New MapSet + MapSet.member? 0.140 M - 9.89x slower +6419.43 ns
