getsentry
Reduce the chance of last-minute regressions making it into a release
Reticketed from https://github.com/getsentry/relay/pull/912.
Problem
For SaaS we practice CD, deploying multiple times per day. If there's a bug, we fix forward, we never(?) roll back. Since we have a monthly cadence for CalVer self-hosted releases, operators of self-hosted Sentry expect—and insofar as they expect this we would like to provide—a greater level of stability and quality. If we ship self-hosted on Friday and discover a "significant" bug on Saturday that we fix on Monday, we would rather not have to amplify the firedrill by also reshipping self-hosted.
Possible Solutions
- Code freeze globally for N days before each CalVer release.
- Emphasize nightlies over monthly releases; use CalVer for nightlies and let consumers decide which version to use.
- Run publish N days before prepare; maintain a "short long-running release branch" that we can backport fixes to if needed prior to release.
- ???
I personally far prefer option 3 over 1. I think 1 can end up hurting productivity across all of our repositories that are shipped as part of self hosted (which is most core parts of Sentry), and should be done with caution. I personally can't think of major downsides to onpremise being a couple of days behind. Agree with your observation that most people tend to already avoid shipping code late on Fridays so it may be the case that the snapshot at that time has a tendency to be a bit more stable.
Closing as won't do after 18+ months and little additional pain around this.