aws
Implement support for function-timeout configuration
Issue #665
Description of changes:
The templates deploy a function-timeout field which should correlate to the number of seconds until the lambda function times out and exits. This is a requirement for functions that rely on the ILambdaContext.RemainingTime as a CancellationTokenSoucre. The remaining time is always set to TimeSpan.Zero currently making it harder to test, or requiring to break and manually set the value.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
This is an interesting change for the test tool and wonder if would disrupt user's debugging in other ways. For example I can see users having function-timeout set to something small for production purposes like 10 seconds. But if we take this change as is and the value is used as the CancellationTokenSource then users would only have 10 seconds to debug.
Is the currently struggle that the value is 0 and that isn't an acceptable value for CancellationTokenSource. Wondering if instead of taking the function-timeout from the config file which is meant for deployment should the test tool set the RemainingTime to some high value so that the same code can be used for setting up the CancellationTokenSource but we would not interfere with debugging. Otherwise I think we have to come up some sort of opt-in mechanism to set the RemainingTime instead of always taking deployment config files function-timeout value.
Thanks for the reply @normj!
The current struggle is that since we're not loading in the ILambdaContext.RemainingTime which is supposed to be equivalent to the lambda's configured Timeout, the value is always 0 seconds. For lambdas where we want to cancel before AWS terminates the run, we need to know how long we have to operate. In my example, we allow the full 15-minute runtime but give 2 minutes to gracefully shut down. So any runs that go long, can stop processing, upload logs, etc, and exit gracefully without being terminated.
In my view, passing the config value into their appropriate context properties brings the test tool closer to how Lambda functions work. This is also why I default to 15 minutes when the value is not set. For anyone using the remaining time as a cancellation token, when checking cancellationToken.IsCancellationRequested it is always true. In my case, I have a loop at the start that exits right away. For anyone with short time frames, anything is better than 0. I have to hit a breakpoint and manually update the value before continuing.
Hey @normj @96malhar @philasmar, I wanted to follow up to see if there is any additional feedback on this issue?
Thanks
Is there any estimate when this could be deployed? I'm also waiting for this feature.
@normj What do you think of the idea of having an additional configuration property that can override the function-timeout during debugging sessions. This way, they can either use the configured debug timeout, the function's real timeout, or the default 15 minutes. Either way, my argument that any of these are better than the existing 0 second timeout.
Also, any other recommendations such that we could get this pushed out? Thanks
Hey @philasmar + @GarrettBeatty, I wanted to follow up and see if you two had thoughts? Thanks
The V1 version of the test tool which is what @UnholyRanger is using supports reading the config from either the JSON config file or the CloudFormation template. If we do this PR of adding a debug timeout then we have an inconsistency with the CloudFormation version. I think in that situation we would have to allow setting the debug timeout in the CloudFormation template's metadata. Then there is the getting the value read from both JSON and YAML based templates.
In the V2 version which is what we do with the .NET Aspire integration. It does default to 15 minutes for the timeout. In v2 the could at pretty easily extend it to allow setting default timeout on the emulator. Something like this:
builder.AddAWSLambdaServiceEmulator(new LambdaEmulatorOptions
{
DefaultFunctionTimeout = TimeSpan.FromMinutes(10)
});
To do it at a per function level in V2 we would need to do some rework to pass the function configuration into the emulator.
@UnholyRanger Given we really want to move V2 to get past some of V1's architecture problems would the 15 minute default it uses today and if we make the default timeout configurable would that solve your needs?
A 15-minute timeout is perfectly fine for what I need; it doesn't even need to be configurable. That may be a nice follow-up feature. When does V2 come out? This is different from the Aspire integration right? as we don't leverage that and have no plans to move in that direction.
@unholyranger-work .NET Aspire is the easier way to use the V2 test tool but you can run it without Aspire. Here are the install and run instructions. https://github.com/aws/aws-lambda-dotnet/tree/master/Tools/LambdaTestTool-v2#installing
I know the flow is rough now and we want to work on smoothing the process out. The major difference between V1 and V2 is the test tool is always run out of process and then you configure your Lambda project to point to the test tool process to get the events. So without the Aspire integration something, normally a manual process, has to start the test tool process.