Support for AWS EventBridge Pipes

[bradwood]

Is there an existing issue for this?

• [X] I have searched existing issues, it hasn't been reported yet

Use case description

AWS recently announced EventBridge Pipes

I would like to be able to add an "Enrichment Lambda" and "Target Lambda" to an AWS EventBridge Pipe using the serverless framework.

Proposed solution (optional)

No response

[medikoo]

@bradwood, thanks for reporting that. Can you propose how do you envision providing support for this in the Framework. What would be the new properties added to the configuration and hat would be the effect in generated CloudFormation stack?

[bradwood]

@medikoo -- I don't have a strong view on it -- my interest in the serverless framework is more as a PO for a team that uses the framework.

[cmygray]

@medikoo I wrote an example configuration.

input sls template

resources:
  Resources:
    MyPipe:
      Type: AWS::Pipes::Pipe
      Properties:
        RoleArn: !Ref AllowSendingToTargetRole
        Source: !Ref MyTable
        Target: !Ref MyEventBus

output CF template

{
  "Type" : "AWS::Pipes::Pipe",
  "Properties" : {
    "RoleArn": "arn:aws:iam::000000000000:role/allow-sending-to-target-role",
    "Source": "arn:aws:dynamodb:ap-northeast-2:000000000000:table/my-table",
    "Target": "arn:aws:events:ap-northeast-2:000000000000:event-bus/my-event-bus"
  }
}

This is an example template containing only minimum properties(required). Maybe someone can improve to use things later:

• enrichment
• source params
• target params

reference:

• cloudformation resource
• eventbridge pipe sources

[medikoo]

@cmygray it clearly defines what needs to end in CF template. Do you think we need to support any dedicated configuration properties for that? If so what should they be?

[bradwood]

For me the enrichment use case and binding a lambda to it is of critical importance as this is the lambda:everything_else configuration that is most important to manage inside the serverless framework.

[cmygray]

@medikoo

Here are configurations, not required but should be supported to use EventBrigePipe full features:

• SourceParameter (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourceparameters.html)
• TargetParameter (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipetargetparameters.html)
• Enrichment
• EnrichmentParameter (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipeenrichmentparameters.html)

[medikoo]

Can you propose the Serverless Framework configuration properties that should be added to ensure the best support?

[yuhongliu82]

Hi @medikoo @cmygray @bradwood, Could I have any updates on the issue? I am curious about it. Thanks

[medikoo]

@yuhongliu82 please see my previous comment

[bradwood]

Note that terraform support is pretty close to being released -- perhaps this can give some suggestions on an implementation approach in serverless?

https://github.com/hashicorp/terraform-provider-aws/issues/28153

[s1mrankaur]

+1 It's been over an year now. Are pipes still not supported?

[s1mrankaur]

Just FYI, There seems to be a plugin available https://github.com/distinction-dev/serverless-pipes, however has some existing issues and limited support for sources/targets. It'd be nice if it's available as part of the framework itself. @medikoo @bradwood