skywalking icon indicating copy to clipboard operation
skywalking copied to clipboard

[Feature] Deprecate the logic sharding concept in the OAP

Open wu-sheng opened this issue 2 years ago • 3 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

    # Enable shard metrics and records indices into multi-physical indices, one index template per metric/meter aggregation function or record.
    logicSharding: ${SW_STORAGE_ES_LOGIC_SHARDING:false}

Currently, we have logicSharding concept in the OAP's booting parameter list. This could be manually controlled by the users. But still, I feel this is a little exposing too much, as actually, this is an expired feature already.

I want to propose an automatic sharding mechanism to replace this.

  1. OAP always booting with no-logic-sharding in the clean installation.
  2. OAP could automatically pick up the logic sharding value when the configuration is absent or logicSharding=false.
  3. Flag logicSharding setting deprecated as we would never recommend the user to set it manually again.
  4. We remove this configuration from YAML and only keep the system env reading(SW_STORAGE_ES_LOGIC_SHARDING) in the code in case of force setting to resolve bugs in urgent.

Use case

As a result of this enhancement, we could always have the latest and recommended index pattern in the new installation, and would not break users' old deployment.

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

wu-sheng avatar May 10 '23 13:05 wu-sheng

@wankai123 Could you take a look and provide feedback about this proposal?

wu-sheng avatar May 10 '23 13:05 wu-sheng

Make sense. As the merge mode has been set by default since 9.2.0. I think it is time to remove it from the YAML and only keep it in system env in case the old env doesn't want to wipe the storage when upgrading.

wankai123 avatar May 10 '23 13:05 wankai123

Another option is we permanently remove this feature in a milestone. Don't have to be 9.5.0. I am considering two iterations(9.7.0) later.

wu-sheng avatar May 10 '23 14:05 wu-sheng