User Defined Managed Identity Authentication for Azure VM/Batch

[swampie]

New feature

Allow Azure Batch VMs to be authenticated with User Assigned Managed Identity when the pool is correctly configured. This will prevent to leak secrets into nextflow configuration and create more granular access control for batch and storage (and secrets)

Usage scenario

Will allow azure instances in batch pool to authenticate using a managed identity to batch/storage/secrets features

Suggest implementation

1. New configuration snippet for authentication on azure:

azure {
     managedIdentity {
         type: 'USER_ASSIGNED'
         clientId: <client_id>
     }
}

2. Use the clientId to instrument the SDK to create a new DefaultCredentials instance: a simple project that make use of this feature can be found here, the relevant part of the authentication happens here

[pditommaso]

What the type is meant for? also would be interesting how the workload identity client id fits here

[swampie]

There are two different types of managed identities that could be assigned to vm (system and user): this page contains a table with the differences. TLDR: system-assigned managed identities are limited to a single azure resource while the user assigned can be tailored to have access to multiple resources. I haven't seen a usecase for system assigned managed identity now but I thought that adding a type property now could be useful for extensibility later on. It can also be added in a future iteration, if needed

[pditommaso]

@swampie can you please provide a clear path how to replicate your setup and validate the proposed implementation ?

[swampie]

the README lists the steps I took to test it. I saw that @adamrtalbot took a similar approach on the PR

[pditommaso]

it doesn't tell the most imporant thing ie how to create the Entra managed identity

[swampie]

Ok here we go:

• From Azure Portal create a new Manage identity (it is pretty straight forward as it ask for name and region)
• Once it has been created click on it and from the right menu click on Azure Role Assignments
• Add the required roles: for storage access I add the following: