nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

Support disk image for Google Batch via `disk` directive

Open hnawar opened this issue 2 years ago • 3 comments

New feature

Support specifying a disk image for additional disk when using Google Batch. This would be useful in many cases but particularly with large reference files which can be both time consuming and expensive. Using DIsk image is faster and cheaper as the Disk is created from the image before the provisioning the VM

Usage scenario

User creates a Disk image containing all reference files User specifies the image as part of the disk directive in the process

#4534

hnawar avatar Nov 22 '23 10:11 hnawar

Deploy Preview for nextflow-docs-staging ready!

Name Link
Latest commit d6fca2b5db0aa3fa206ecad374507188adced315
Latest deploy log https://app.netlify.com/sites/nextflow-docs-staging/deploys/656dcfc713970700084c7bb0
Deploy Preview https://deploy-preview-4535--nextflow-docs-staging.netlify.app/google
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Nov 22 '23 10:11 netlify[bot]

This change by itself is fine, but the disk directive now has three purposes (resize boot disk, attach empty disk, attach disk with image) but can be specified only once. That means that e.g. you couldn't attach a disk with reference inputs and also attack a local SSD to use as scratch storage.

To solve this issue we just need to allow the disk directive to be specified multiple times. But I am fine with merging this PR as it is and implementing multiple disk directives in a separate PR.

bentsherman avatar Nov 22 '23 13:11 bentsherman

Thanks Ben :) Note, that the disk can still be used as scratch space if it is not completely full and is mounted in the same path with read/write permissions

I did think about the multiple disk directive, but did not have an answer for that

hnawar avatar Nov 22 '23 14:11 hnawar