dask-benchmarks Automation

This PR contains a bash script and relevant asv configuration files to run benchmarking as a cron script. It is setup to run for both dask and distributed and to push to the gh-pages branch. You can already see the published benchmarks for both:

https://dask.github.io/dask-benchmarks/results/distributed https://dask.github.io/dask-benchmarks/results/dask

Distributed was a little tricky because of it's heavy dependency on dask. To work around this I set it so asv install dask github master. Unfortunately this is not supported with asv and a little hack was added to ensure a file is in the right place for html generation. I was also unable to run benchmarks for previous versions of distributed as distributed.Client did not exist yet, which the tests depend on.

Dec 21 '16 21:12 postelrich

At first glance everything here seems clear to me. However I haven't actually run through things myself yet.

@TomAugspurger and @pitrou might also have thoughts.

Dec 21 '16 23:12 mrocklin

@pitrou OK I added in the conda env creation and running new benchmarks. It always reruns benchmarks for distributed at the EXISTING step. Probably due to the pip hack. Need to investigate.

Dec 22 '16 00:12 postelrich

No comments from me, other than a thanks to @postelrich for setting this up.

Dec 22 '16 01:12 TomAugspurger