checkHistogram metrics are over-expanded, recommended to downgrade or have a disable option

[potterhe]

Is your feature request related to a problem? Please describe.

When meter otlp is enabled, the number of check_amount_buckets in the metrics pushed to otlp-collector (such as prometheus 3.x) is particularly large. The reason is that it has a label "subject_id" with user granularity. When the user reaches the thousand level, the combination with the label "le" will cause the metrics to exceed the ten thousand level. Add check_amount_sum and check_amount_count, and there will be more. From prometheus, the metrics look like this

check_amount_bucket{job="permify", le="7500", subject_id="xxx", subject_type="user"}
check_amount_sum{job="permify", subject_id="xxx", subject_type="user"}
check_amount_count{job="permify", subject_id="xxx", subject_type="user"}

Describe the solution you'd like

• a option to disable

[ucatbas]

hi @potterhe, we currently provide only a flag in the config file to enable or disable the meter. unfortunately, this request is not part of our short-term roadmap. if I understand correctly, to prevent this in prometheus or any other observability tool you use, I’d recommend adding a filter in your otlp-collector for specific metrics. also, if you’re interested in contributing by implementing this feature in our config file, we’d be happy to review and merge it 🚀🚀🚀