lambda-perf icon indicating copy to clipboard operation
lambda-perf copied to clipboard

Feature request: p95 of cold starts

Open Munawwar opened this issue 2 years ago • 5 comments

First off, this is a useful tool. Thanks for setting it up.

Mean is a useful metric. But to get a feel of how much the cold starts can vary from the mean, it would be nice to have p95 and standard deviation (assuming normal distribution here) of the measurements.

Munawwar avatar May 24 '23 20:05 Munawwar

This would probably require quite a lot more than 10 runs to get good data, which would take usage outside of the free tier. I guess we could average this out over 30 days, maybe?

GrahamCampbell avatar Aug 04 '23 15:08 GrahamCampbell

If you have kept every run's result from past 30 days, then calculate p95 from list of all the results. That would be even more reliable.

Munawwar avatar Aug 04 '23 19:08 Munawwar

The run history going back very far is stored in git.

GrahamCampbell avatar Aug 04 '23 23:08 GrahamCampbell

Every day since Sep 4th 2022.

GrahamCampbell avatar Aug 04 '23 23:08 GrahamCampbell

I am removing stddev from issue, on second thought it isnt too useful. e.g. Cloud Run shows p95 and p99 response times. It turns out to be more accurate and useful.

Munawwar avatar Aug 20 '23 08:08 Munawwar