Icinga
Run only a single job per workflow at a time
This limits each run of these workflows to a single action runner. This means, for each PR, merge or release only five runners run concurrently.
It also cancels pending jobs automatically, scoped to the workflow and pull request. Merge and release runs cannot cancel each other though.
We are limited to 20 runners concurrently. This means four PRs can still lock them up. But at least it prevents a single PR from locking up all resources.
At the moment it's only a draft, so feel free to ignore this at the moment. But I'd really like to merge this in the next days/weeks. (If it works as I thought, I'll cancel the runs after creating this PR so I can't tell if it does until all releases are finished)
I just clicked re-run for a single workflow to have a look at how this works and all jobs started running immediatly: 
So I'm not yet convinced that it works.
I doubt that’s acceptable.
Is it acceptable that one repo uses up the full organization runner limit for quite some time blocking other repos? I mean probably probably not great, but acceptable if that's the best what we can do with GH Actions (if we get it to actually work somehow).
I doubt that’s acceptable.
I could have proposed to limit it to just a single runner at a time. For web repositories this wouldn't be a problem, but I figured it would be one here ;)
So I'm not yet convinced that it works.
Seems to work now.
Runs also get cancelled mostly fine. There were only two cases with deb.yml and raspbian.yml which weren't cancelled at all. Though, neither the previous cancel action was successful nor my own attempts. These workflows, once running, seem to be not cancellable at all.
Both #9636 and this PR have a lot of overlapping changes. How will they interact with each other? One uses group: deb-${{ github.ref }}, the other group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}. How would the combination of both look? Or is there a general conflict between the PRs (i.e. not only a conflict preventing automatic merging of both)?
group: deb-${{ github.ref }} will be the final definition. Mine is then obsolete. It has the only change that master / tag runs don't get cancelled.
github.ref will also be a tag name, so subsequent pushes onto the same tag cancel the previous runs like on the master
Only in this case they get cancelled? Then I consider that behaviour desired.
So I'll have a closer look at #9636 and if it gets merged as-is, this PR will reduce down to adding max-parallel: 1?
LGTM, but... only 4/5?
Docker has no matrix and runs only one job anyways.
Wait! One job per PR! #9636 already does this, but AFAIK OP's goals are beyond that.
#9636 limits to one workflow, this PR additionally limits to one job per workflow.