aws-sam-cli icon indicating copy to clipboard operation
aws-sam-cli copied to clipboard

Feature request: local invocation should apply EphemeralStorage limits and report on usage

Open davidjb opened this issue 2 years ago • 1 comments

Describe your idea/feature/enhancement

AWS Lambda functions have a set limit for ephemeral storage available at /tmp in the file system, defaulting to 512MB but able to be set up to 10GB as of 2022.

By default, Docker containers are allowed an unlimited tmpfs which allows code to succeed in a local environment when it would inevitably fail in AWS, something you can only discover on deployment. Because of the nature of out-of-space errors in tmpfs (e.g ENOSPC: no space left on device, mkdtemp '/tmp/...), they can be caused by and present in varying/unreleated ways, depending on how an application or operating system handles the situation, making it hard to debug and identify the root cause.

By comparison, if SAM is able to set limits early, it will allow developers to identify issues locally and address them immediately, and make SAM/Docker more closely match AWS Lambda.

Proposal

To assist with this, SAM can configure Docker with limits placed on tmpfs with the tmpfs-size option; this should be set at 512MB to match the Lambda defaults and be set based upon the EphemeralStorage setting for the function in the template.

In a similar vein to #6260 for memory/RAM usage, SAM should report an error if a function invocation has reached its ephemeral storage limit. It would be useful to see a warning if an invocation is close to exceeding the limit also (within 20%, perhaps), but also an accounting of the tmpfs consumed, not just in error/warning situations. Ideally these messages would be coloured, like parts of SAM's build/invocation output.

Additional Details

davidjb avatar Jan 18 '24 07:01 davidjb

Hi @davidjb this is interesting, thanks for the feature request. I will bring this up with the team and PM's.

hnnasit avatar Jan 20 '24 23:01 hnnasit