Want to authenticate in synapse spark pool using managed identity.

[dbeavon]

I'm looking for a way to authenticate within the synapse spark pool, using the MSI.

Managed Identity Support is already available in a variety of other Azure platforms via the nuget called Azure.Identity. Here is a list of products that support "DefaultAzureCredential": https://www.nuget.org/packages/Azure.Identity#readme-body-tab

• Azure Virtual Machines
• Azure App Service and Azure Functions
• Azure Kubernetes Service
• etc

Whenever this is supported, it becomes possible to create an MSAL access token for a list of scopes WITHOUT providing an additional service principal or secret. Here is an example of authentication to Azure Analysis Services:

using Azure.Core; using Azure.Identity; using Microsoft.Identity.Client;

private static Azure.Core.AccessToken GetAzureAccessTokenFromDefaultCredential()
{
TokenCredential p_CurrentTokenCredential = null;
string[] p_ScopesList = null;


// Retrieve the current credentials (for the app service)
p_CurrentTokenCredential = new DefaultAzureCredential(includeInteractiveCredentials: true);

 
// Special-purpose scope used for 
// application permissions 
p_ScopesList = new string[] { "https://*.asazure.windows.net/.default" };
 
 

// The context and the resulting token.
var ContextObj = new TokenRequestContext(p_ScopesList);
Azure.Core.AccessToken TokenObj = p_CurrentTokenCredential.GetToken(ContextObj, CancellationToken.None);
return TokenObj;
}

Is there any way to accomplish the same thing in synapse spark pools today? I had found the "TokenLibrary" but it seemed extremely limited and I found no way of creating an access token for an arbitrary list of scopes.

Any tips or pointers would be very much appreciated.

[mzivtins]

@dbeavon did you have any luck with this?

Are you writing application/sparkjobs/notebooks? I know MSI is fully supported from a synapse perspective but directly on the spark pool I am not sure?

Synapse will allow you to get an MSI token, although 99% of the time it is not needed as it is automatic. MSSparkUtils.Credentials.GetConnectionStringOrCreds()

Outside of setting the MSI IAM on a Synapse workspace I struggle to think of a way too directly authenticate with a spark pool as it does not represent itself in that manner inside AAD applications