datacomp
datacomp copied to clipboard
Downloading Commonpool XLarge
Hi all thank you for the great package. I'm currently running into issues downloading Commonpool X-Large with Pyspark. I've downloaded the metadata files and uploaded them to S3, however my Spark pipelines seem to continually fail after some time. I'm using the same setup as above, but am wondering if I have setup something incorrectly.
Would it be possible to share the configuration you used to download Commonpool X-Large?