azure.synapse.tools
azure.synapse.tools copied to clipboard
Unsupported Type: SparkConfigurations
Deployment throws error "ASWT0029: Unknown object type: SparkConfiguration" when deploying a notebook (example shown below) that has a reference to a custom spark configuration. I believe this may be a very similar issue to #11 where SparkConfiguration would need to be added to the allowed valid types in private/!SynapseObject.class.ps1 as well as private/Get-SynapseObjectByName.ps1 so that it can pretend that the object referenced is valid and exists. Notebook:
{
"name": "nb_example",
"properties": {
"folder": {
"name": "utility"
},
"nbformat": 4,
"nbformat_minor": 2,
"bigDataPool": {
"referenceName": "sparkSm",
"type": "BigDataPoolReference"
},
"targetSparkConfiguration": {
"referenceName": "CustomSparkConfig",
"type": "SparkConfigurationReference"
},
Failure:
**mVERBOSE: A***lyzing notebook dependencies...
**mVERBOSE: Folder: D:\a\***\drop\***\notebook
**mVERBOSE: - nb_example.json
Failure occurred while publishing artifacts to ***
##[debug]Error record:
##[debug]Exception: D:\a\_temp\a560ab36-63de-480f-888a-bc877***f23f0a.ps***:38
##[debug] | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##[debug] | ASWT0029: Unknown object type: SparkConfiguration.
azure.synapse.tools Version 0.23.000