Layout with special characters can't be retrieved with source tracking - encoding issue between MetadataAPI and real name

[ChristianMenzinger]

Summary

Layouts are one of the few metadata components which are not limited in the names they can have. Creating a Layout with special characters can't be retrieved with the Salesforce CLI with source tracking. There is an encoding happening in MetadataAPI, which is not following any known standard like URL encode

Steps To Reproduce

• Create a new DX project and a new default scratch org

• Create a new Account Layout with name: ^ ° ! " § $ % & / ( ) = ? ´ ` + @ # ' < > [ ] \ , ; . : - _ __ * ä ß 製

• Check Layout has been saved

• run command sf project retrieve preview -> will display change for the new Layout

• run command sf project retrieve start -> will fail to retrieve the new Layout

no repo needed to reproduce as it is an empty DX project and nothing has been retrieved tested against v60 and v61 with no difference

I am able to retrieve the Layout in MetadataAPI when I use the encoded name: Account-%5E %C2%B0 %21 %22 %C2%A7 %24 %25 %26 %2F %28 %29 %3D %3F %C2%B4 %60 %2B %40 %23 %27 %3C %3E %5B %5D %5C %2C %3B %2E %3A - _ %5F%5F * ä ß 製.layout

Expected result

Layout should be retrieved without any issue

Actual result

Salesforce CLI shows error: Entity of type 'Layout' named 'Account-^ ° ! " § $ % & / ( ) = ? ´ ` + @ # ' < > [ ] \ , ; . : - _ __ * ä ß 製' cannot be found

System Information

zsh 5.9 (x86_64-apple-darwin23.0)

{
  "architecture": "darwin-arm64",
  "cliVersion": "@salesforce/cli/2.40.7",
  "nodeVersion": "node-v20.12.2",
  "osVersion": "Darwin 23.4.0",
  "rootPath": "/Users/christianmenzinger/.local/share/sf/client/2.40.7-bde121f",
  "shell": "zsh",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.0.16 (core)",
    "@oclif/plugin-commands 3.3.1 (core)",
    "@oclif/plugin-help 6.0.21 (core)",
    "@oclif/plugin-not-found 3.1.7 (core)",
    "@oclif/plugin-plugins 5.0.18 (core)",
    "@oclif/plugin-search 1.0.23 (core)",
    "@oclif/plugin-update 4.2.9 (core)",
    "@oclif/plugin-version 2.0.17 (core)",
    "@oclif/plugin-warn-if-update-available 3.0.16 (core)",
    "@oclif/plugin-which 3.1.8 (core)",
    "@salesforce/cli 2.40.7 (core)",
    "apex 3.1.11 (core)",
    "auth 3.6.5 (core)",
    "data 3.3.2 (core)",
    "deploy-retrieve 3.6.8 (core)",
    "info 3.2.4 (core)",
    "limits 3.3.4 (core)",
    "marketplace 1.2.4 (core)",
    "org 4.1.5 (core)",
    "packaging 1.26.4 (user)",
    "schema 3.3.4 (core)",
    "settings 2.2.3 (core)",
    "sobject 1.3.6 (core)",
    "source 3.3.5 (core)",
    "telemetry 3.3.4 (core)",
    "templates 56.2.4 (core)",
    "trust 3.6.7 (core)",
    "user 3.5.4 (core)"
  ]
}

Additional information

It is not clear how encoding works: it is clearly to some distinct Salesforce specific: underscore ('_') is not encoded, but double-underscores ('__') are. Also asterisk ('*'), dash ('-') or empty-space (' ') are not encoded neither. Where is the encoding algorithm defined?

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[ChristianMenzinger]

I just realised when using a package.xml the encoding is different:

If I am not wrong, it follows standard HTML encoding.

Is there any documentation around how encoding is done?

[RupertBarrow]

Only non-ASCII diacritical character users (é è ç à ß ô Å etc.) are crazy enough like you @ChristianMenzinger to name a Page Layout like that ! 😂

[iowillhoit]

Hey @ChristianMenzinger, this is a known issue that is mostly resolved. See this comment. Steve wrote some code help handle the differences in encoding here

It looks like the issue is just the __. If you remove that from your Layout name it will retrieve successfully. Of if you create a Layout named Foo__Bar, you will see the same error. I am going to talk with the team to see what (if anything) we should do about this.

[iowillhoit]

Ok, I spoke with the team. This is related to how double underscores are handled in API names. If you create a new Custom Object with a __ (for example Foo__Bar) it will automatically change the API name to have a single underscore (Foo_Bar). However, the Layout that is automatically created for that Custom Object still uses the double underscore and will fail to retrieve.

This is not a CLI bug and it is not something that we are going to add code to try to work around. If you'd like doc changes, you'll need to open a Custom Support request to get the Layout docs updated (maybe here?)

[github-actions[bot]]

We have determined that the issue you reported exists in code owned by another team that uses only the official support channels. To ensure that your issue is addressed, open an official Salesforce customer support ticket with a link to this issue. We encourage anyone experiencing this issue to do the same to increase the priority. We will keep this issue open for the community to collaborate on.