Add a function to create new token without having a token

[saldroubi]

Hello,

For Azure, sometimes you want to manage a workspace by creating a token programmatically to use in Set-DatabricksEnvironment -AccessToken $tokenObj.token_value -ApiRootUrl $apiUrl

In Azure, there is a way you can do that if you have the write permissions. I wrote the function below New-DBRApiToken which can be used to create a token that can be used in Set-DatabricksEnviornment.

Also, you can get the workspace url programmatically if you know the workspace name and the resource group name. See function below called Get-WorkspaceHostname

Does it make sense to add these to your module or some variation of these functions?

Thank you

Function New-DBRApiToken { <# .SYNOPSIS Create and return a token. This call returns the error QUOTA_EXCEEDED if the caller exceeds the token quota, which is 600. .DESCRIPTION Create and return a token. This call returns the error QUOTA_EXCEEDED if the caller exceeds the token quota, which is 600. Official API Documentation: https://docs.databricks.com/api/latest/tokens.html#create .PARAMETER LifetimeSeconds The lifetime of the token, in seconds. If no lifetime is specified, the token remains valid for 14400 seconds (30 hours) to create an indefinite token pass -1 .PARAMETER Comment Optional description to attach to the token. Default value is "Generated for workspace managment" .EXAMPLE Add-DatabricksApiToken -LifetimeSeconds 360 -Comment "MyComment #>

[CmdletBinding()] Param ( [Parameter(Mandatory = $True)][string]$ResourceGroupName, [Parameter(Mandatory = $True)][string]$WorkspaceName, [Parameter(Mandatory = $False)][long] $LifetimeSeconds = 1800, [Parameter(Mandatory = $False)][string] $Comment = "Generated for workspace management" )

# Code based on these References:
#   https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token
#   https://cloudarchitected.com/2020/01/using-azure-ad-with-the-azure-databricks-api/

#$tenantID=    (Get-AzContext).Tenant.id
$workspaceID =  Get-AzResource -ResourceGroupName $ResourceGroupName | Where-Object  {$_.ResourceType -eq "Microsoft.Databricks/workspaces"} | Select-Object -ExpandProperty ResourceID

#Get a token for the global Databricks application.
$DBRGlobalUrl = "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"
$tokenObj = Get-AzAccessToken -ResourceUrl $DBRGlobalUrl
$DBRtoken = $tokenObj.Token

# Get a token for the Azure management API
$tokenObj = Get-AzAccessToken -ResourceUrl "https://management.core.windows.net"
$AMAtoken = $tokenObj.Token

# Create token DBR url
$url = "https://" + (Get-WorkspaceHostname -WorkspaceName $WorkspaceName -ResourceGroupName $ResourceGroupName) + "/api/2.0/token/create"

# set header and body for rest api call
$h = @{
    Authorization= "Bearer $($DBRtoken)"
    'X-Databricks-Azure-SP-Managment-Token' = "$($AMAtoken)"
    'X-Databricks-Azure-Workspace-Resource-Id' = "$($workspaceID)"
 }
$body = @{
    lifetime_seconds = $LifetimeSeconds 
    comment = $Comment
}
$resultObj = Invoke-RestMethod -Method Post -Headers $H -Body ($body | ConvertTo-Json -depth 20) -Uri $url
return $resultObj

}

<# .SYNOPSIS Returns the hostname for a specified databricks workspace. .EXAMPLE PS C:> Get-WorkspaceHostname -WorkspaceName "bricks-workspace-dataengineering" -ResourceGroup "rg-workspace-dataengineering" .OUTPUTS The workspace hostname #> function Get-WorkspaceHostname { [CmdletBinding()] [OutputType([string])] param ( # Databricks workspace name [Parameter(Mandatory)][string]$WorkspaceName, # Resource group name [Parameter(Mandatory)][string]$ResourceGroupName )

$workspaceResource = Get-AzResource `
    -ResourceType "Microsoft.Databricks/workspaces" `
    -Name $WorkspaceName  `
    -ResourceGroupName $ResourceGroupName `
    -ExpandProperties

return $workspaceResource.Properties.workspaceUrl

}

[gbrueckl]

Hi @saldroubi,

you can already use a user account or a service principal to authenticate without a Personal Access Token (PAT) - see https://github.com/gbrueckl/Databricks.API.PowerShell#azure-active-directory-aad-usernamepassword

instead of authenticating using Connect-AzAccount you would simply pass the same/similar parameters directly to Set-DatabricksEnvironment

The part of getting the URL programmatically is nice but introduces a dependency to the Az module which I definitely do not want as this module is supposed to also work with Databricks on AWS and GCP

[gbrueckl]

I just published v1.9.9.11 which introduces a new switch -UsingAzContext which in combination with -AzureResourceID allows you to derive all settings from an already authenticated Az module for details see https://github.com/gbrueckl/Databricks.API.PowerShell#azure-az-module-integration

[saldroubi]

I just installed the v1.9.9.11 version but that switch Is missing. Here is the error I get when I run it.

Set-DatabricksEnvironment -UsingAzContext -AzureResourceID $azureResourceId Set-DatabricksEnvironment: Cannot bind positional parameters because no names were given.

When I tab after the "-" I get these possible parameters. None of which is UsingAzContext Set-DatabricksEnvironment -
ApiRootUrl OrgID JobsAPIVersion ErrorVariable
AccessToken SubscriptionID AzureActiveDirectoryAuthorityUrl WarningVariable
UsingDatabricksCLIAuthentication ResourceGroupName AzureActiveDirectoryServiceEndpointResourceId InformationVariable
Credential WorkspaceName Verbose OutVariable
UsingAzureDevOpsServiceConnection ServicePrincipal Debug OutBuffer
ClientID DynamicParameterCacheTimeout ErrorAction PipelineVariable
TenantID ApiCallRetryCount WarningAction
AzureResourceID ApiCallRetryWait InformationAction

Thank you

[gbrueckl]

it is definitely there: make sure that you do not have any other parameters set already which do not belong to the same parameterset as this would hide the parameter

[saldroubi]

Hi Gerhard,

Honestly, I am not sure why it was not working and I had version 1.9.9.1 installed.  I uninstalled it and re-installed and I see the parameter now.  Thank you very much. This works great!

One question, so you are not generating a token to setup the environment but rather using the existing az account token? Are you then this same az token to pass to Databricks as the bearer token in the REST API request? If so, why will this token work since it is not a token generated by Databricks?

I am finding the Azure security model a bit confusing with all these different tokens, some are AAD and some are Databricks. Is there a good reference to understand this well.

Thank you so much for this module. It is very helpful. I am hope I am doing my part by giving good feedback.

Sincerely, Sam Al-Droubi

[gbrueckl]

so there are two ways how you can interact with the DAtabricks REST API on Azure:

• Personal Access Token (PAT)
• Azure AD token (usernam/password or service principal)

either of them can be used for any call you make to the Databricks REST API - so you do not necessarily need PATs anymore but can only use AAD tokens

for the DatabricksPS module the only difference is how you call the Set-DatabricksEnvironment cmdlet, all other cmdlets just work the very same way

[saldroubi]

Thank you Gerhard,

I understand the the personal Access Token (PAT). That is the one I can generate from the Databricks UI. But am confused about the username/password. Using

Set-DatabricksEnvironment -UsingAzContext -AzureResourceID $azureResourceId

I am not passing a username/password in this call and yet I am able to use functions like Get-DatabricksCluster. That is why I am confused. I hope you can explain this to me?

[gbrueckl]

In order to use -UsingAzContext you must have run Connect-AzAccount first with a username/pw The module then basically uses that information to generate a new AAD token for the Databricks API calls for you

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: saldroubi @.> Sent: Thursday, May 12, 2022 6:31:18 PM To: gbrueckl/Databricks.API.PowerShell @.> Cc: Gerhard Brueckl @.>; Comment @.> Subject: Re: [gbrueckl/Databricks.API.PowerShell] Add a function to create new token without having a token (Issue #54)

Thank you Gerhard,

I understand the the personal Access Token (PAT). That is the one I can generate from the Databricks UI. But am confused about the username/password. Using

Set-DatabricksEnvironment -UsingAzContext -AzureResourceID $azureResourceId

I am not passing a username/password in this call and yet I am able to use functions like Get-DatabricksCluster. That is why I am confused. I hope you can explain this to me?

— Reply to this email directly, view it on GitHubhttps://github.com/gbrueckl/Databricks.API.PowerShell/issues/54#issuecomment-1125198583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2VO4UWTMINQUXR76P2HATVJUW5NANCNFSM5VSEDV3Q. You are receiving this because you commented.Message ID: @.***>

[saldroubi]

This feature works. Thank you for adding it.