dask-benchmarks icon indicating copy to clipboard operation
dask-benchmarks copied to clipboard

Benchmark for graph construction

Open quasiben opened this issue 5 years ago • 0 comments

While exploring scheduler improvements it was recently discovered that removing redundant string concatenations has a measurable performance increase. Tom noted that there currently were no isolated benchmarks which tested graph construction

We could do something similar to what was suggested in https://github.com/dask/dask/pull/6137

In [2]: ddf_d = timeseries(start='2000-01-01', end='2002-01-01', partition_freq='1d')

In [3]: %timeit shuffle(ddf_d, "id", shuffle="tasks")
67.2 ms ± 2.54 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

quasiben avatar May 08 '20 17:05 quasiben