nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

More helpful warnings when publish fails to create parent directory

Open robsyme opened this issue 2 years ago • 3 comments

At the moment, if publishing a file fails because of permissions, Nextflow will return an error indicating the Access Denied exception, but will not indicate which file was the problem.

This commit adds the file name to the error message, so that the user can determine from the logs alone which file path was the problem. Catching this exception also triggers the safeProcessFile catch block which also gives the helpful message saying "Failed to publish file". It's not entirely clear to me why this catch block does not catch the Access Denied exception being thrown inside the makeDirs method.

robsyme avatar Aug 30 '23 20:08 robsyme

Deploy Preview for nextflow-docs-staging canceled.

Name Link
Latest commit 3f6798da92041304f82b9a4928f839ca324d8d18
Latest deploy log https://app.netlify.com/sites/nextflow-docs-staging/deploys/64efa059bfd07c0008a9dc03

netlify[bot] avatar Aug 30 '23 20:08 netlify[bot]

I'd like to note that I'm very unclear why this change is necessary. From my reading of the code, safeProcessFile calls processFile which calls makeDirs, which throws the exception:

safeProcessFile
  | processFile
     | makeDirs -> throws the Exception

But I'm very unclear why this exception isn't caught by the try block

protected void safeProcessFile(Path source, Path target) {
    try {
        processFile(source, target)
    }
    catch( Throwable e ) {
        log.warn "Failed to publish file: ${source.toUriString()}; to: ${target.toUriString()} [${mode.toString().toLowerCase()}] -- See log file for details", e
        if( NF.strictMode || failOnError){
            final session = Global.session as Session
            session?.abort(e)
        }
    }
}

Ben/Paolo, can you explain why the try block in the safeProcessFile method isn't catching the Exception?

robsyme avatar Aug 31 '23 02:08 robsyme

Sorry for letting this one fall through. Did you confirm that makeDirs was called via processFile ? Because it is also called via apply0 > createPublishDirs.

If that's not it, then I don't know why the exception wouldn't be caught. But in any case it would probably be helpful to have this warning directly in makeDirs so that it is properly logged

bentsherman avatar Nov 22 '23 10:11 bentsherman