Support using `Fn::Sub`/`!Sub` and `Fn::Cmd`/`!Cmd` intrinsic functions together

[mbarneyjr]

Support using Fn::Sub/!Sub and Fn::Cmd/!Cmd intrinsic functions together

I'd like to be able to do something similar to the following:

SomeStringJson:
  Fn::Sub: !Cmd generate-json-cli

Where generate-json-cli returns json that expects to be passed to the Fn::Sub/!Sub intrinsic function, like this:

{
  "some-property": "${resourcePrefix}-some-value"
}

Your environment

• version of org-foramtion (ofn --version)

$ ofn --version
0.9.15

• version of node (node --version)

$ node --version
v14.15.4

• which OS/distro OS X

$ uname -a
Darwin T10059.local 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

Steps to reproduce

Create a CloudFormation template similar to the following:

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  resourcePrefix:
    Type: String
Resources:
  Parameter:
    Type: AWS::SSM::Parameter
    Name: some-name
    Value:
      Fn::Sub: !Cmd cat file.json

where you have a file.json with the following:

{
  "some-property": "${resourcePrefix}-some-value"
}

Expected behaviour

Running print-tasks should generate a template that looks similar to the following:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Parameters": {
        "resourcePrefix": {
            "Type": "String"
        }
    },
    "Resources": {
        "Parameter": {
            "Type": "AWS::SSM::Parameter",
            "Name": "some-name",
            "Value": {
                "Fn::Sub": "{\"some-property\":\"${resourcePrefix}-some-value\"}"
            }
        }
    }
}

Actual behaviour

I get an error:

ERROR: Task DoThing print failed. reason: unable to parse !Sub expression

[zaro0508]

I believe you can accomplish this same use case by using the new ofn templating feature, https://github.com/org-formation/org-formation-cli/blob/master/docs/task-files.md#templating

[mbarneyjr]

Yeah that was my work around (may not be exact):

# task.yml
SomeTask:
  ...
  TemplatingContext:
    Value: !Cmd cat file.json
    
# template.yml
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  resourcePrefix:
    Type: String
Resources:
  Parameter:
    Type: AWS::SSM::Parameter
    Name: some-name
    Value: !Sub |
      {{ Value | indent('4') }}

But it'd be nice to have the !Cmd evaluated first so the validation of the template doesn't fail

The templating feature definitely came in handy! Really happy to have a workaround initially until this issue is resolved

[OlafConijn]

nice!

agree that functions must be better composable. i had not run into this before, but will look into fixing this for sure. one thing: !Cmd cat file.json -> not equivalent to !ReadFile file.json?

[mbarneyjr]

Right, !ReadFile would be a valid substitution (I think it still isn’t compatible with !Sub though), I just broke my problem down into something very simple, my actual use case is converting a yaml file to minified json with yq

  Policy: !Cmd cat policy.yml | yq -c

For a Community::Organizations::Policy resource, which only supports json for the Content property

[OlafConijn]

ah, ok.

!ReadFile also doesn't allow to me used to produce the template of a !Sub (for now)

[hexionas]

Hey @OlafConijn, any traction on using !ReadFile with !Sub? :)

[OlafConijn]

yes! could you check [email protected] ? if it doesn't work, please comment an example of what you try to achieve. thanks!