aws
`sam sync` with typescript not updating Lambda layer code in Lambda function
Description
When using sam sync for updating a Lambda Layer written in typescript, the change is not reflected in the imported code in the lambda function.
Referencing a LambdaLayer in a Lambda function, I'm expecting the lambda function to use the updated layer during invocation. This is the observed bahaviour using python, but when using a Typescript layer together with sam sync, the change is not propagated to the live function, despite the SAM cli feedback indicating it has:
Manifest is not changed for (SharedLambdaLayer), running incremental build
Building layer 'SharedLambdaLayer'
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrcAndLockfile
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:CleanUpNpmrc
Finished syncing Layer SharedLambdaLayer.
Syncing Function Layer Reference Sync HelloWorldFunction...
Finished syncing Function Layer Reference Sync HelloWorldFunction.
SAM template:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs18.x
Architectures:
- x86_64
Layers:
- !Ref SharedLambdaLayer
....
SharedLambdaLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: shared-layer
Description: Layer containing shared code
ContentUri: shared/lib/
RetentionPolicy: Delete
CompatibleRuntimes:
- nodejs18.x
Metadata:
BuildMethod: nodejs18.x
Steps to reproduce
(If you don't want to set it up from scratch, you can clone this repo, where I've done the below steps)
-
sam initwith a typescript hello world example. - Add a lambda layer in typescript, that exports a function that logs something or manipulates the return of the lambda
-
npm installinside the hello-world function diretory to install esbuild -
sam build --build-in-source && sam deployin the root -
sam sync --code --watch --build-in-sourcein the root. - Update the hello-world and verify changes are live by curling the endpoint
- Update the layer (and observe the CLI feedback says that the referenced function has been updated, after updating the layer)
- curl the endpoint and observe that the changes have not been propagated sucessfully.
If we now make an update to the lambda function, the correct and latest layer code is used.
Observed result
Updating the Lambda layer, SAM cli syncs the layer and reports that the sync was sucessfull. Inspecting the Lambda function in the console, I also see that the layer version has been bumped. Invoking the lambda function, however, is using the old layer code.
Expected result
The latest layer code to be used.
Additional environment details
- OS: OSX
-
- If using the SAM CLI,
sam --version: 1.108.0
- If using the SAM CLI,
- AWS region: eu-west-1, eu-north-1
Thanks for creating the issue, and preparing example repo for re-producing the problem.
Due to the nature of esbuild there is no way to offload some dependencies into a shared AWS::Lambda::LayerVersion. esbuild itself bundles everything into single package and that package is deployed as lambda function. For that reason, if I remove the SharedLambdaLayer and its reference in Lambda function, I can still build and invoke the function itself, even though the layer is removed from the template.
This is my updated template.yaml file;
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
lambda-layer-sync-ts
Sample SAM Template for lambda-layer-sync-ts
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs18.x
Architectures:
- x86_64
Events:
HelloWorld:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /hello
Method: get
Metadata: # Manage esbuild properties
BuildMethod: esbuild
BuildProperties:
Minify: true
Target: "es2020"
Sourcemap: true
EntryPoints:
- app.ts
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
HelloWorldApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
HelloWorldFunction:
Description: "Hello World Lambda Function ARN"
Value: !GetAtt HelloWorldFunction.Arn
HelloWorldFunctionIamRole:
Description: "Implicit IAM Role created for Hello World function"
Value: !GetAtt HelloWorldFunctionRole.Arn
And I can get the sam local invoke working without any issues;
❯ sam local invoke
Invoking app.lambdaHandler (nodejs18.x)
Local image is out of date and will be updated to the latest runtime. To skip this, pass in the parameter --skip-pull-image
Building image...........................................................................................................................
Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.
Mounting /private/tmp/lambda-layer-sync-ts/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: 37febcd6-c1cf-4838-988d-825531bfb193 Version: $LATEST
END RequestId: 34cf177a-17b1-48e4-b526-af5ac85a44a8
REPORT RequestId: 34cf177a-17b1-48e4-b526-af5ac85a44a8 Init Duration: 0.03 ms Duration: 150.54 ms Billed Duration: 151 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode": 200, "body": "{\"message\":\"4 2\"}"}
However, when you remove the Layer, it might still not work since the layer contents and function contents are different folderes. sam sync process listens to file changes in the folder which is reffered by CodeUri property of the function. In order to get this work, you need to move your function and layer into a folder, and update your function definition to point to higher level folder.
- Inside
template.yaml,CodeUripoints to/foo - Your lambda function code is in
/foo/bar - Your layer code is in
/foo/baz
Please let us know if you have other questions or issues.
I'm not sure I understand. In the sample I shared I am in-fact offloading shared dependencies into a layer - which is working as expected. The bug is that during sam sync, the layer is updated, which I can both manually inspect, and the logs state that the lambda functions which depend on the layers are updated. If I manually inspect the Lambda function that depends on the layer in the console, I can see that the layer it references has indeed been incremented, but the imported layer in the function itself is still the previous version, until a change to the lambda function code triggers a re-deployment and new lambda runtime.
In addition, if I'm not misstaken the change you suggest would mean that every function is re-built every time a change happens in one function. With your suggestion, my file structure would be
functions/
function1/
app.ts
function2/
app.ts
shared/
lib/
utils.ts
In my template, if I set
HelloWorldFunction1:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/
Handler: function1.app.lambdaHandler
...
HelloWorldFunction2:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/
Handler: function2.app.lambdaHandler
SharedLambdaLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: shared-layer
Description: Layer containing shared code
ContentUri: functions/lib/
Wouldn't that trigger all Lambdas to update if I update the code for one function?
I just verified that is the case. I have 7 lambda function (and using the structure I mentioned in the previous message), and with the change you suggested each Lambda function is re-built during a sync whenever there's a change in one single Lambda function. I understand that all need to be re-built when the Layer code changes, but updating the code of one Lambda function then re-builds and deploys all Lambda functions.
This is not desirable and greatly impacts the development experience. The sync now takes 7x longer, since I have 7 Lambda functions.
Would appreciate some guidance here. Given the feedback from the CLI when using sync, stating it is in fact updating functions that are referencing the layer, when the layer has been updated, this is a bug. The solution you suggested is not viable in multi-lambda projects.
What I was trying to say, esbuild bundles everything into single package, therefore you don't need to use layers anymore. Can you try to remove the layer and do sam build and sam deploy? You should see that your function will work even without layers.
⚠️COMMENT VISIBILITY WARNING⚠️
Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}