azure-cli Exported ARM template that includes StorageAccount can't be re-deployed

Describe the bug

An ARM template generated from any resource group that includes a StorageAccount fails when redeploying with an InvalidXmlDocument error.

Command Name az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidXmlDocument\",\r\n    \"message\": \"XML specified is not syntactically valid.\\nRequestId:e878d855-f01a-005d-286e-b197b2000000\\nTime:2021-09-24T18:00:42.5358098Z\"\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

#!/bin/bash

rand=$RANDOM

az group create -g group${rand} -l westeurope
az storage account create -g group${rand} -n stor${rand}
az group export -g group${rand} --skip-all-params >group-arm.json
az group delete -y -g group${rand}

rand=$RANDOM

az group create -g group${rand} -l westeurope
az deployment group create -g group${rand} -f group-arm.json

This fails at the:

az deployment group create -g {} -f {}

Expected Behavior

Should not fail. According to internet searches this is related to "type": "Microsoft.Storage/storageAccounts/xxxServices" entries in the generated ARM template, but then either: a) they should not be generated if they are not needed b) az should be able to handle the resulting template correctly

Environment Summary

Linux-5.10.43.3-microsoft-standard-WSL2-x86_64-with-debian-10.10, Debian GNU/Linux 10 (buster)
Python 3.7.3
Installer: PIP

azure-cli 2.28.1

Additional Context

A template saved through the Azure portal has the exact same issue

Sep 24 '21 18:09 jepio

ARM

Sep 24 '21 22:09 yonzhan

This is the generated ARM template:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "resources": [
    {
      "apiVersion": "2021-04-01",
      "kind": "StorageV2",
      "location": "westeurope",
      "name": "stor32142",
      "properties": {
        "accessTier": "Hot",
        "encryption": {
          "keySource": "Microsoft.Storage",
          "services": {
            "blob": {
              "enabled": true,
              "keyType": "Account"
            },
            "file": {
              "enabled": true,
              "keyType": "Account"
            }
          }
        },
        "networkAcls": {
          "bypass": "AzureServices",
          "defaultAction": "Allow",
          "ipRules": [],
          "virtualNetworkRules": []
        },
        "supportsHttpsTrafficOnly": true
      },
      "sku": {
        "name": "Standard_RAGRS",
        "tier": "Standard"
      },
      "type": "Microsoft.Storage/storageAccounts"
    },
    {
      "apiVersion": "2021-04-01",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts', 'stor32142')]"
      ],
      "name": "stor32142/default",
      "properties": {
        "cors": {
          "corsRules": []
        },
        "deleteRetentionPolicy": {
          "enabled": false
        }
      },
      "sku": {
        "name": "Standard_RAGRS",
        "tier": "Standard"
      },
      "type": "Microsoft.Storage/storageAccounts/blobServices"
    },
    {
      "apiVersion": "2021-04-01",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts', 'stor32142')]"
      ],
      "name": "stor32142/default",
      "properties": {
        "cors": {
          "corsRules": []
        },
        "protocolSettings": {
          "smb": {}
        },
        "shareDeleteRetentionPolicy": {
          "days": 7,
          "enabled": true
        }
      },
      "sku": {
        "name": "Standard_RAGRS",
        "tier": "Standard"
      },
      "type": "Microsoft.Storage/storageAccounts/fileServices"
    },
    {
      "apiVersion": "2021-04-01",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts', 'stor32142')]"
      ],
      "name": "stor32142/default",
      "properties": {
        "cors": {
          "corsRules": []
        }
      },
      "type": "Microsoft.Storage/storageAccounts/queueServices"
    },
    {
      "apiVersion": "2021-04-01",
      "dependsOn": [
        "[resourceId('Microsoft.Storage/storageAccounts', 'stor32142')]"
      ],
      "name": "stor32142/default",
      "properties": {
        "cors": {
          "corsRules": []
        }
      },
      "type": "Microsoft.Storage/storageAccounts/tableServices"
    }
  ],
  "variables": {}
}

Sep 25 '21 17:09 jepio

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

Issue Details

Describe the bug

An ARM template generated from any resource group that includes a StorageAccount fails when redeploying with an InvalidXmlDocument error.

Command Name az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidXmlDocument\",\r\n    \"message\": \"XML specified is not syntactically valid.\\nRequestId:e878d855-f01a-005d-286e-b197b2000000\\nTime:2021-09-24T18:00:42.5358098Z\"\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

#!/bin/bash

rand=$RANDOM

az group create -g group${rand} -l westeurope
az storage account create -g group${rand} -n stor${rand}
az group export -g group${rand} --skip-all-params >group-arm.json
az group delete -y -g group${rand}

rand=$RANDOM

az group create -g group${rand} -l westeurope
az deployment group create -g group${rand} -f group-arm.json

This fails at the:

az deployment group create -g {} -f {}

Expected Behavior

Should not fail. According to internet searches this is related to "type": "Microsoft.Storage/storageAccounts/xxxServices" entries in the generated ARM template, but then either: a) they should not be generated if they are not needed b) az should be able to handle the resulting template correctly

Environment Summary

Linux-5.10.43.3-microsoft-standard-WSL2-x86_64-with-debian-10.10, Debian GNU/Linux 10 (buster)
Python 3.7.3
Installer: PIP

azure-cli 2.28.1

Additional Context

A template saved through the Azure portal has the exact same issue

Author:	jepio
Assignees:	zhoxing-ms
Labels:	`Service Attention`, `ARM`, `customer-reported`
Milestone:	Backlog

Sep 26 '21 02:09 ghost

@jepio Apologies for the late reply. I was able to reproduce the issue at my end. While looking at the backend logs, I found that the failure was occurring due to the schema validation while empty value was sent to the smb protocolSettings. Could you please remove the below line from the ARM JSON template and then run the deployment again ?

"{"protocolSettings":{"smb":{}}"

Awaiting your reply.

Mar 03 '22 10:03 navba-MSFT

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

Mar 10 '22 14:03 ghost

Hi @navba-MSFT, Indeed without that line the ARM JSON template deploys correctly. But since exporting a template with az group export -g group${rand} --skip-all-params produces the "protocolSettings" entry, could you fix the backend validation to handle this correctly?

Mar 10 '22 16:03 jepio

@jepio Thanks for getting back and confirming the workaround fixes the issue. We have created an internal workitem with the Product team, to fix the backend logic.

Mar 30 '22 07:03 navba-MSFT

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @josephkwchan, @jennyhunter-msft.

Issue Details

Describe the bug

An ARM template generated from any resource group that includes a StorageAccount fails when redeploying with an InvalidXmlDocument error.

Command Name az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidXmlDocument\",\r\n    \"message\": \"XML specified is not syntactically valid.\\nRequestId:e878d855-f01a-005d-286e-b197b2000000\\nTime:2021-09-24T18:00:42.5358098Z\"\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

#!/bin/bash

rand=$RANDOM

az group create -g group${rand} -l westeurope
az storage account create -g group${rand} -n stor${rand}
az group export -g group${rand} --skip-all-params >group-arm.json
az group delete -y -g group${rand}

rand=$RANDOM

az group create -g group${rand} -l westeurope
az deployment group create -g group${rand} -f group-arm.json

This fails at the:

az deployment group create -g {} -f {}

Expected Behavior

Should not fail. According to internet searches this is related to "type": "Microsoft.Storage/storageAccounts/xxxServices" entries in the generated ARM template, but then either: a) they should not be generated if they are not needed b) az should be able to handle the resulting template correctly

Environment Summary

Linux-5.10.43.3-microsoft-standard-WSL2-x86_64-with-debian-10.10, Debian GNU/Linux 10 (buster)
Python 3.7.3
Installer: PIP

azure-cli 2.28.1

Additional Context

A template saved through the Azure portal has the exact same issue

Author:	jepio
Assignees:	-
Labels:	`bug`, `Service Attention`, `ARM`, `customer-reported`, `needs-team-attention`
Milestone:	-

May 04 '22 11:05 ghost

@josephkwchan, @jennyhunter-msft Could you please provide an update on this once you get a chance ?

May 04 '22 11:05 navba-MSFT

@bmoore-msft - Do you have any insight into this issue?

May 04 '22 18:05 jennyhunter-msft

@jepio Thanks for getting back and confirming the workaround fixes the issue. We have created an internal workitem with the Product team, to fix the backend logic.

@navba-MSFT - can you share the internal workitem you created? Normally issues like this are addressed in the schemas (which is a public repo)

May 04 '22 21:05 bmoore-msft

@bmoore-msft I have emailed you the internal workitem details.

May 05 '22 02:05 navba-MSFT

Having the same issue, works if I removed the "smb" section. Any news for a fix in the backend?

Sep 22 '22 13:09 henkeberggren