cortex
cortex copied to clipboard
Introduced lock file to shuffle sharding grouper
What this PR does:
- Introduced lock file for every blocks are under compaction while using shuffle sharding grouper. The content of lock file is simply a timestamp when the lock file is updated by heart beat process while under compaction. The lock file will make sure blocks are under compaction will not be picked up by other compactor at the same time which fixed https://github.com/cortexproject/cortex/pull/4433#issuecomment-991414965
- Grouper will return the first group with all blocks not having a lock file or timestamp inside lock file is more than 5 minutes old. This replaced the logic using group hash to determine which compactor should own which group. After this change, every compactor can compact groups that belong to the tenant assigned to the compactor and the blocks in the group are not locked.
- Incoming groups are still sorted in the same way that smallest range, oldest min time first. The reason only first group would return is that big tenant compaction would not block compaction for other tenant for a long period of time.
Checklist
- [x] Tests updated
- [ ] Documentation added
- [x]
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]