cdklabs
add GitHubActionStep that extends Step
Use case
We would like to add a e2e test after deploy step, this can be achieved by adding option with ShellStep . The problem for this case is that there is no way to use the checkout action or any other action. So I am asking if we could introduce preDeploySteps and postDeploySteps like what we already had preBuildSteps and postBuildSteps?
Please let me know if you have any better idea to achieve this?
Not sure how possible it would be, but it would be great to define ShellStep installCommands and commands as JobStep objects (as opposed to raw bash strings) to give us the flexibility that we'd have with raw GitHub Actions yaml
Hi @julianiag! This should be possible with the API today. Specifically, the API of addStage or addStageWithGitHubOptions.
Say you want to add pre/post at the stage level. You can do this:
const myStage = new MyStage(this, 'StageA', { env: EnvironmentUtils.parse(props.envA) });
pipeline.addStage(myStage, {
pre: [new ShellStep('Pre', {
commands: ['echo hello'],
})],
post: [new ShellStep('Post', {
envFromCfnOutputs: {
FN_NAME: myStage.fnName,
},
commands: ['echo FN_NAME equals: $FN_NAME'],
})],
});
Say you want to add pre/post at the stack level. You can do this:
pipeline.addStage(prod, {
stackSteps: [{
stack: prod.stack1,
pre: [new ShellStep('Pre-Stack Check')], // Executed before stack is prepared
post: [new ShellStep('Post-Deploy Check')], // Executed after stack is deployed
}, {
stack: prod.stack2,
post: [new ShellStep('Post-Deploy Check')], // Executed after stack is deployed
}],
});
This API is avaliable through the @aws-cdk/pipelines module that this module extends. I took this example from the readme of that module. Notably, I've changed the example to fit the confines of cdk-pipelines-github -- we don't have a concept of a ManualApprovalStep and changeSet is not supported.
Please let me know if that answers your question and/or fixes your problem. Happy to help further if I've misinterpreted what you're asking for!
@dann-iag, ShellStep is taken from the parent module and thus has to be generic. It's definitely possible to create our own GitHubActionStep and make a smoother API for users of this module though. I'll have to think about that one.
Thanks for the quick replies @kaizencc!
I have tried your suggestion to @julianiag and it has a limitation of only being able to run raw bash as suppoorted by ShellStep, although runs the steps exactly when we need them.
Because of the limitation, we're not able to use the github actions/checkout@v2 action (or any other non-shell GHA).
We're currently working around this by installing git on our self hosted runners, and running a git clone in shell, but ideally we'd follow the idioms of github actions.
I think your idea of a GitHubActionStep would fulfil our need, and provide tons of flexibility in future
I've repurposed this issue into the new ask of a GitHubActionStep. I haven't fully looked into what this entails or if it's truly feasible, but I can see how it would be helpful. Hopefully I'll have time to look into this later this week.
Hi @kaizencc , we are having more and more use cases that needs this GitHubActionStep to be passed as either pre or post steps when we call addStageWithGitHubOptions to create a stage.
For example, we have dev, test and prod stages and we only want to add an extra step that uses one github action before the Deploy job for test and prod stages but not dev.
I found it is possible to introduce such a GithubActionStep after a few attempts of modifying the code.
a GithubActionStep might look like this.
import { Step } from 'aws-cdk-lib/pipelines';
import { JobStep } from './workflows-model';
export class GithubActionStep extends Step {
constructor(id: string, private readonly jobStep: JobStep) {
super(id);
}
toJobStep(): JobStep {
return this.jobStep;
}
}
Then add a new method called jobForGithubActionStep in pipeline.ts like this
private jobForGithubActionStep(node: AGraphNode, step: GithubActionStep): Job {
return {
id: node.uniqueId,
definition: {
name: step.id,
...this.jobSettings,
permissions: {
contents: github.JobPermission.WRITE,
},
runsOn: this.runner.runsOn,
needs: this.renderDependencies(node),
env: {},
steps: [
step.toJobStep(),
],
},
};
}
Then update the switch case in the jobForNode method like
case 'step':
...
} else if (node.data.step instanceof GithubActionStep) {
return this.jobForGithubActionStep(node, node.data.step);
}
...
I also tried to add a unit test by calling
pipeline.addStageWithGitHubOptions(stage, {
pre: [new GithubActionStep('test', {
name: 'test',
uses: 'my-custom/[email protected]',
with: {
'app-id': 1234,
'key': 'value',
},
})],
gitHubEnvironment: 'test',
});
and in the final test result I got
name: deploy
on:
push:
branches:
- main
workflow_dispatch: {}
jobs:
Build-Build:
name: Synthesize
permissions:
contents: read
id-token: none
runs-on: ubuntu-latest
needs: []
env: {}
container: null
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install
run: yarn
- name: Build
run: yarn build
- name: Upload cdk.out
uses: actions/[email protected]
with:
name: cdk.out
path: cdk.out
Assets-FileAsset1:
name: Publish Assets Assets-FileAsset1
needs:
- Build-Build
permissions:
contents: read
id-token: none
runs-on: ubuntu-latest
outputs:
asset-hash: ${{ steps.Publish.outputs.asset-hash }}
steps:
- name: Download cdk.out
uses: actions/download-artifact@v2
with:
name: cdk.out
path: stage.out
- name: Install
run: npm install --no-save cdk-assets
- name: Authenticate Via GitHub Secrets
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: us-west-2
role-duration-seconds: 1800
role-skip-session-tagging: true
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- id: Publish
name: Publish Assets-FileAsset1
run: /bin/bash ./cdk.out/publish-Assets-FileAsset1-step.sh
MyStack-test:
name: test
permissions:
contents: write
runs-on: ubuntu-latest
needs:
- Build-Build
env: {}
steps:
- name: test
uses: my-custom/[email protected]
with:
app-id: 1234
key: value
MyStack-MyStack-Deploy:
name: Deploy MyStack098574E7
permissions:
contents: read
id-token: none
environment: test
needs:
- Build-Build
- Assets-FileAsset1
- MyStack-test
runs-on: ubuntu-latest
steps:
- name: Authenticate Via GitHub Secrets
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: us-east-1
role-duration-seconds: 1800
role-skip-session-tagging: true
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: arn:aws:iam::111111111111:role/cdk-hnb659fds-deploy-role-111111111111-us-east-1
role-external-id: Pipeline
- id: Deploy
uses: aws-actions/aws-cloudformation-github-deploy@v1
with:
name: MyStack-MyStack
template: https://cdk-hnb659fds-assets-111111111111-us-east-1.s3.us-east-1.amazonaws.com/${{
needs.Assets-FileAsset1.outputs.asset-hash }}.json
no-fail-on-empty-changeset: "1"
role-arn: arn:aws:iam::111111111111:role/cdk-hnb659fds-cfn-exec-role-111111111111-us-east-1
Please let me know if you like the idea or if I shall submit a PR.
Hi @julianiag, thanks for the PR, sorry I've been MIA. Will take a look next week.