Bakht Kahloon

Last few tasks stall while merging two large dataframes

8

**What happened**: 1. read in two large dataframes via (both dfs are around 40Gb of parquet file data and we select a subset of the data so in memory is...

**What happened**: Connecting to a remote EMR cluster from a Jupyter notebook (using YarnCluster for Dask Cluster creation) causes notebook cell to hang. The YarnCluster client is able to successfully...