gbrueckl
Export databrick env fails
WARNING: This feature is EXPERIMENTAL and still UNDER DEVELOPMENT! Invoke-RestMethod : {"error_code":403,"message":"Invalid access to Org: 7127657513572111"} At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.12.0.1\Public\General.ps1:96 char:13
- ... $result = Invoke-RestMethod -Uri $apiUrl -Method $Method -Headers $ ...
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~- CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
- FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand Invoke-RestMethod : {"error_code":403,"message":"Invalid access to Org: 7127657513572111"} At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.12.0.1\Public\General.ps1:96 char:13
- ... $result = Invoke-RestMethod -Uri $apiUrl -Method $Method -Headers $ ...
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~- CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
- FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand WARNING: The following parameters are not supported in Databricks Jobs API 2.0 and will be ignored: -JobName, -ExpandTasks, -Offset, -Limit Invoke-RestMethod : {"error_code":403,"message":"Invalid access to Org: 7127657513572111"} At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.12.0.1\Public\General.ps1:96 char:13
- ... $result = Invoke-RestMethod -Uri $apiUrl -Method $Method -Headers $ ...
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~- CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
- FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
very much looks like an permission/authentication issue
is any of the cmdlets working at all, e.g. Get-DatabricksWorkspaceItem -Path "/" ?
Yes it is because of wrong PAT Token. Could you please help me with the following error or let me know if this can be ignored
WARNING: The following parameters are not supported in Databricks Jobs API 2.0 and will be ignored: -JobName, -ExpandTasks, -Offset, -Limit WARNING: It is not possible to extract secret values via the Databricks REST API. This export only exports the names of SecretScopes and their Secrets but not the values! WARNING: It is not possible to donwload the whole DBFS. This export will only download files from DBFS that already exist locally and overwrite them! Export-DatabricksEnvironment : Local DBFS path C:\Export\DBFS does not exist so the DBFS export cannot work properly! At line:1 char:1
- Export-DatabricksEnvironment -LocalPath 'C:\Export' -CleanLocalPath
-
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Export-DatabricksEnvironment
Export-DatabricksEnvironment : To export SQL objects the parameter -SQLClusterID is mandatory but was not supplied! At line:1 char:1
- Export-DatabricksEnvironment -LocalPath 'C:\Export' -CleanLocalPath
-
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Export-DatabricksEnvironment
WARNING: The export of SQL objects is still experimental! Get-DatabricksExecutionContext : Cannot bind argument to parameter 'ClusterID' because it is an empty string. At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.12.0.1\Public\ImportExport.ps1:342 char:59
- ... ext = Get-DatabricksExecutionContext -ClusterID $SQLClusterID -Langua ...
-
~~~~~~~~~~~~~- CategoryInfo : InvalidData: (:) [Get-DatabricksExecutionContext], ParameterBindingValidationException
- FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Get-DatabricksExecutionContext
Start-DatabricksCommand : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input. At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.12.0.1\Public\ImportExport.ps1:346 char:37
- ... ses = $sqlContext | Start-DatabricksCommand -Command "SHOW DATABASES"
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~- CategoryInfo : InvalidArgument: (:) [Start-DatabricksCommand], ParameterBindingException
- FullyQualifiedErrorId : InputObjectNotBound,Start-DatabricksCommand
well, one part of the Export is also exporting SQL objects. For this to work you need to specify a Databricks Cluster via -SQLClusterID where the SQL commands can be executed
in general, if it is just a Warning, you can usually ignore it if you need anything specific for SQL or DBFS you need to provide the information (e.g. SQLClusterID) otherwise those objects will/can not be epxorted
It exports partial data, if its wrong PAT then none of workspace or user data can be exported, so I don't think it is an issue with PAT. Could you also guide me to export Unity catalog data
not sure your answer relates to my last message!?
the unity catalog APIs are currently only very limited - at least there are no dedicated functions for it in this module nor in the Export-DatabricksEnvironment command
as an alternative you could use Invoke-DatabricksApiRequest and create the necessary requests on your own