Dagger.jl

A framework for out-of-core and parallel computing

Documentation	Build Status

At the core of Dagger.jl is a scheduler heavily inspired by Dask. It can run computations represented as directed-acyclic-graphs (DAGs) efficiently on many Julia worker processes and threads, as well as GPUs via DaggerGPU.jl.

DTable has been moved out of this repository. You can find it in a standalone package format here.

Installation

You can install Dagger by typing

julia> ] add Dagger

Usage

Once installed, the Dagger package can by used like so

using Distributed; addprocs() # get us some workers
using Dagger

# do some stuff in parallel!
a = Dagger.@spawn 1+3
b = Dagger.@spawn rand(a, 4)
c = Dagger.@spawn sum(b)
fetch(c) # some number!

Acknowledgements

We thank DARPA, Intel, and the NIH for supporting this work at MIT.