rocketmq icon indicating copy to clipboard operation
rocketmq copied to clipboard

Tiered storage on slave

Open yuz10 opened this issue 2 years ago • 3 comments

Hi, teams! Currently the we have only local file tiered storage implementation, if we implement OSS/S3 tiered storage, the master and slave write to the same file on OSS/S3, I think only master should write to OSS/S3, Is there any design about this?

yuz10 avatar Feb 15 '23 03:02 yuz10

Tiered storage should be only enabled in the master. But the compatibility of tiered storage and HA needs more discussion, such as tiered storage should not upload data beyond the confirmed offset and how to quickly enable tiered storage when the slave broker promotes to master.

ShadowySpirits avatar Feb 15 '23 04:02 ShadowySpirits

Tiered storage should be only enabled in the master. But the compatibility of tiered storage and HA needs more discussion, such as tiered storage should not upload data beyond the confirmed offset and how to quickly enable tiered storage when the slave broker promotes to master.

I think this can be considered. The new version of HA has the concept of confirm offset. In addition, we can complete enable tiered storage during role switching, just as we handle timing messages.

RongtongJin avatar Feb 16 '23 01:02 RongtongJin

This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.

github-actions[bot] avatar Feb 17 '24 00:02 github-actions[bot]

This issue was closed because it has been inactive for 3 days since being marked as stale.

github-actions[bot] avatar Feb 20 '24 00:02 github-actions[bot]