Make "building" lambdas easier

[sethtjf]

I've been searching for a way to "build" a lambda function with copilot. The addon docs and related issues like #3659 outline a few options as far as I can tell:

1. Download dependencies locally into the directory containing the lambda code so that it can be zipped and deployed
2. Using build scripts or a Makefile that builds the lambda package and then runs copilot deploy
3. As mentioned in #3659, using SAM+Copilot to create a "build" concept before deployment
4. Adding the lambda function to the primary docker container so that it's built by copilot and then change the container command

Ideally, building lambda would get first-party treatment by copilot. There may be limitations why this can't be the case, but I would love to see one of two solutions:

1. Support for build options inside the addon file so that the lambda (or even another docker container) is built and deployed as part of the regular copilot deploy and in pipeline; or
2. Support for Worker Services as lambda functions so that code can be easily packaged and built as part of deployment and pipelines

Would love to hear what the team is thinking about doing here. I think there is a huge opportunity for copilot + lambda.

[Lou1415926]

Hello @sethtjf ! Thank you very much for the feature request! Just a few follow up / clarification questions for us to kickstart a discussion:

1. Could you elaborate on what you meant by "build"? Is it mostly referring to building the image that's used by the lambda function?
2. Do you have any specific use case in mind? How do you envision your lambda to work with the other services?

Other folks with similar requests are welcome to chime in!

Thank you!

[sethtjf]

Hey @Lou1415926, thanks for following up. I'm happy to kick of a discussion:

1. I think that's right. Ideally I would be able to define a lambda function as an addon and specify a Dockerfile that is built, deployed to ECR, then used to create a lambda function.
2. Use case is to use lambda to create something similar to the Worker Service pattern linked above. I'd prefer to use lambda functions so that I can publish to SNS topics, subscribe with SQS and have my lambda workers do their thing. A reason for tighter integration with lambda is to more easily allow my lambda to talk to the the EFS and RDS services that I have copilot provision. I've spend a fair bit of time recently trying to get everything to talk to each other properly and it's not been easy (nor can I confirm that it works yet). So anything that makes that flow easier and allows me to replicate across different environments would help. Especially as I look to add more worker functions in the future.

I hope that helps to clarify!

[efekarakus]

Just sharing some interesting links to explore to make building Lambda functions easier:

• https://github.com/aws/aws-lambda-builders that is used by sam build under the hood that we can also look into leveraging.
• https://github.com/aws/aws-sam-build-images/blob/develop/build-image-src/Dockerfile-go1x to create a container to build and upload artifacts