data.all pipelines monitoring

[dlpzx]

From data.all I can create CICD pipelines (data.all pipelines) which initializes a DDK application. Then I work in CodeCommit and I develop and deploy DDK data pipelines, stages etc I struggle to know the status of these data pipelines. There is also no kind of link for the resources that I created.

I would like to have more visibility on what I have deployed from the CICD pipeline.

Describe alternatives you've considered We cannot know what the data pipeline will include before we deploy or at the deployment of the data.all pipeline. The data.all pipeline only includes CICD. Therefore we need a flexible way of giving visibility to the data.all pipelines from data.all.

User experience:

1. Create data.all pipeline - deploys CICD stack
2. Customize the DDK application deployed and add DDK stages, data pipelines (e.g. one Step Function, and .add_monitoring method)
3. Go back to data.all and in the pipeline, go to a "Links" tab. In this tab you can add new items to a table. Each of this items consists of a link and a data.all environment-group
4. Users that belong to those environment-groups can assume the link and be redirected to the AWS Console. For example, to the Step Function console...