aws-sam-cli
aws-sam-cli copied to clipboard
aws
Bug: Inconsistent Behavior of Streaming Response in AWS SAM CLI
Description:
AWS Lambda now supports streaming responses, but it appears that the AWS SAM CLI does not. When I try to develop my application locally, it doesn't work at all.
Steps to reproduce:
Create a SAM project with the following template.json:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "",
"Resources": {
"HelloWorld": {
"Type": "AWS::Serverless::Function",
"Properties": {
"CodeUri": "./lambda",
"Handler": "hello-word.handler",
"Runtime": "nodejs18.x",
"Events": {
"HelloWorldEvent0": {
"Type": "Api",
"Properties": {
"Path": "/hello-world",
"Method": "get"
}
}
},
"FunctionUrlConfig": {
"AuthType": "NONE",
"MODE": "RESPONSE_STREAM"
},
"Timeout": 60,
"MemorySize": 1024
}
}
}
}
Create index.mjs with the following content:
import * as awslambda from 'aws-lambda';
export const handler = awslambda.streamifyResponse(async (event, responseStream, _context) => {
responseStream = awslambda.HttpResponseStream.from(responseStream, {
statusCode: 200
});
responseStream.write("Streaming with Helper \n");
responseStream.end();
// 'responseStream.finished' does not exist in the local development environment either.
// await responseStream.finished();
});
Run the following commands:
sam local start-api --template template.json --port 8090
Observed result:
After the AWS SAM CLI is ready, visiting http://127.0.0.1:8090/hello-world yields:
Console outputs:
Invalid lambda response received: Lambda response must be valid JSON
The lambda invocation just throws the error and does nothing.
Expected result:
If I upload the code to AWS Cloud, everything works well. I would like the AWS SAM CLI to have consistent behavior between development and production environments.
2024-01-01 18:34:49,059 | Constructed Event 1.0 to invoke Lambda. Event: {'version': '1.0', 'httpMethod': 'GET', 'body': None, 'resource': '/hello-world', 'requestContext':
{'resourceId': '123456', 'apiId': '1234567890', 'resourcePath': '/hello-world', 'httpMethod': 'GET', 'requestId': '50f39c31-b40e-407e-9251-8d93cfba79d0', 'accountId':
'123456789012', 'stage': 'Prod', 'identity': {'apiKey': None, 'userArn': None, 'cognitoAuthenticationType': None, 'caller': None, 'userAgent': 'Custom User Agent String',
'user': None, 'cognitoIdentityPoolId': None, 'cognitoAuthenticationProvider': None, 'sourceIp': '127.0.0.1', 'accountId': None}, 'extendedRequestId': None, 'path':
'/hello-world', 'protocol': 'HTTP/1.1', 'domainName': '127.0.0.1:8090', 'requestTimeEpoch': 1704105259, 'requestTime': '01/Jan/2024:10:34:19 +0000'}, 'queryStringParameters':
None, 'multiValueQueryStringParameters': None, 'headers': {'Host': '127.0.0.1:8090', 'Connection': 'keep-alive', 'Cache-Control': 'max-age=0', 'Sec-Ch-Ua': '"Not_A
Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"', 'Sec-Ch-Ua-Mobile': '?0', 'Sec-Ch-Ua-Platform': '"macOS"', 'Upgrade-Insecure-Requests': '1', 'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7', 'Sec-Fetch-Site': 'none',
'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-User': '?1', 'Sec-Fetch-Dest': 'document', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language':
'zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7,ja;q=0.6,ru;q=0.5', 'X-Forwarded-Proto': 'http', 'X-Forwarded-Port': '8090'}, 'multiValueHeaders': {'Host': ['127.0.0.1:8090'],
'Connection': ['keep-alive'], 'Cache-Control': ['max-age=0'], 'Sec-Ch-Ua': ['"Not_A Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"'], 'Sec-Ch-Ua-Mobile': ['?0'],
'Sec-Ch-Ua-Platform': ['"macOS"'], 'Upgrade-Insecure-Requests': ['1'], 'User-Agent': ['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/120.0.0.0 Safari/537.36'], 'Accept':
['text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'], 'Sec-Fetch-Site': ['none'],
'Sec-Fetch-Mode': ['navigate'], 'Sec-Fetch-User': ['?1'], 'Sec-Fetch-Dest': ['document'], 'Accept-Encoding': ['gzip, deflate, br'], 'Accept-Language':
['zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7,ja;q=0.6,ru;q=0.5'], 'X-Forwarded-Proto': ['http'], 'X-Forwarded-Port': ['8090']}, 'pathParameters': None, 'stageVariables': None,
'path': '/hello-world', 'isBase64Encoded': False}
2024-01-01 18:34:49,069 | Found one Lambda function with name 'HelloWorld'
2024-01-01 18:34:49,072 | Invoking hello-word.handler (nodejs18.x)
2024-01-01 18:34:49,073 | No environment variables found for function 'HelloWorld'
2024-01-01 18:34:49,073 | Resolving code path. Cwd=/Users/tenghou/aws-sam-bug-streaming, CodeUri=/Users/tenghou/aws-sam-bug-streaming/lambda
2024-01-01 18:34:49,074 | Resolved absolute path to code is /Users/tenghou/aws-sam-bug-streaming/lambda
2024-01-01 18:34:49,075 | Code /Users/tenghou/aws-sam-bug-streaming/lambda is not a zip/jar file
2024-01-01 18:34:49,094 | Checking free port on 127.0.0.1:5588
2024-01-01 18:34:49,100 | Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.
2024-01-01 18:34:49,101 | Mounting /Users/tenghou/aws-sam-bug-streaming/lambda as /var/task:ro,delegated, inside runtime container
2024-01-01 18:34:49,388 | Starting a timer for 60 seconds for function 'HelloWorld'
START RequestId: 1f115a13-138b-43f1-b340-0ff328817e90 Version: $LATEST
END RequestId: 474c6068-753c-4d85-bf5f-5af7f1891de1
REPORT RequestId: 474c6068-753c-4d85-bf5f-5af7f1891de1 Init Duration: 0.38 ms Duration: 899.27 ms Billed Duration: 900 ms Memory Size: 1024 MB Max Memory Used: 1024 MB
2024-01-01 18:34:50,299 | Failed to deserialize response from RIE, returning the raw response as is
2024-01-01 18:34:50,407 | Cleaning all decompressed code dirs
2024-01-01 18:34:50,408 | Invalid lambda response received: Lambda response must be valid json
2024-01-01 18:34:50,408 | Lambda execution failed ()
sam --info
{
"version": "1.105.0",
"system": {
"python": "3.8.13",
"os": "macOS-13.4.1-arm64-arm-64bit"
},
"additional_dependencies": {
"docker_engine": "24.0.2",
"aws_cdk": "2.115.0 (build 58027ee)",
"terraform": "1.6.5"
},
"available_beta_feature_env_vars": [
"SAM_CLI_BETA_FEATURES",
"SAM_CLI_BETA_BUILD_PERFORMANCE",
"SAM_CLI_BETA_TERRAFORM_SUPPORT",
"SAM_CLI_BETA_RUST_CARGO_LAMBDA"
]
}
Thanks for raising this issue. At the moment, sam local commands do not support response streaming and this is a known feature gap.
In the meantime, I recommend trying to test your application with sam sync which will test in the cloud instead of using local emulation.
The issue still didn't fix? I face exactly the same problem sam sync is not an option for me.
