Sheilah Kirui

@gpucibot rerun tests

@vyasr @mroeschke , need your eyes on this pr, the `codecov/patch` check keeps failing despite having added said tests cases.. is this a block to merging?

Hi @cjac , could you please update to work with latest `dask-rapids v22.12`?

@cjac ``` root@test-dataproc-rapids-dask-m:/# conda list ^cu # packages in environment at /opt/conda/miniconda3: # # Name Version Build Channel cucim 22.04.00 cuda_11_py38_g8dfed80_0 rapidsai cuda-python 11.8.1 py38h241159d_2 conda-forge cudatoolkit 11.2.72 h2bc3f7f_0 nvidia...

I've tried creating a `DaskCluster` object with out-of-range nodeport and I see this: ``` HTTP response headers: HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"simple-scheduler\" already exists","reason":"AlreadyExists","details":{"name":"simple-scheduler","kind":"pods"},"code":409} ``` cluster.yaml ``` # cluster.yaml apiVersion:...

Seeing this error: ``` cluster = YarnCluster(worker_class="dask_cuda.CUDAWorker", worker_gpus=1, worker_vcores=4, worker_memory='24GB', worker_env={"CONDA_PREFIX":"/opt/conda/default/"}) cluster.scale(4) client = Client(cluster) client --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In [3], line 1 ----> 1...

> Once you have things working could you write this up as a documentation page in rapidsai/deployment? I have opened PR https://github.com/rapidsai/deployment/pull/99 to update the Dataproc instructions

Update: PR on-hold awaiting this [issue](https://github.com/GoogleCloudDataproc/initialization-actions/issues/1039) to be resolved -- Google team needs to upgrade the dask rapids installation to `v22.12`.

@jacobtomlinson Blocked by this error, fails to load the `parquet` dataset, perhaps related to https://github.com/apache/arrow/issues/31812 OUTPUT ``` > File already exists. Ready to load at /rapids_hpo/data/airlines.parquet --------------------------------------------------------------------------- ArrowInvalid Traceback (most...

I am testing this [notebook ](https://github.com/rapidsai/cloud-ml-examples/blob/main/dask/notebooks/HPO_demo.ipynb) in a GCP Dataproc cluster using the airline parquet dataset located in my GCS [bucket](https://storage.cloud.google.com/rapids-cloud-ml-sample-data/airline_small.parquet) (i think you need to access GCP for this)