st2-packages icon indicating copy to clipboard operation
st2-packages copied to clipboard

Package revision race

Open enykeev opened this issue 9 years ago • 4 comments

Between fetching a revision number as a part of checkout and uploading a new package with this revision during deploy, every parallel package build will end up with the same revision number and will fail on deploy stage due to package already being present.

enykeev avatar Dec 19 '16 05:12 enykeev

That's what I thought about when implemented existing revision number logic. I wanted to rely on external service like Redis with locking and releasing to avoid any possible race conditions around revision number like this.

But with time, I must admit that it was over-engineering, rather then real case. Race never happened, because one thing is PR where release number doesn't matter and another thing is Merge. What's important for us is post-merge build, which happens pretty rarely (a couple of times/month in st2-packages and a couple of times/day for st2 in average).

arm4b avatar Dec 20 '16 13:12 arm4b

Happened for the first time now with @Kami during the year or something :+1: https://circleci.com/gh/StackStorm/st2/4522 If there is less than 10mins between merging 2 PRs into master, - they'll race on package revision.

arm4b avatar Jul 20 '17 13:07 arm4b

I still think by implementing some locking and relying on external service would be overengineering.

Solution for now: don't merge 2 PRs in one go.

If that issue will happen again and again, - then yes worth adding some external Consul or Redis or whatever for revision locking.

arm4b avatar Jul 20 '17 13:07 arm4b

Update: the race happened one more time during the 2.4 release when we trigger the build for both st2-packages + st2 repos at the same time.

arm4b avatar Aug 23 '17 19:08 arm4b