client_python icon indicating copy to clipboard operation
client_python copied to clipboard

supporting custom multiprocess metric path

Open iostream96 opened this issue 10 months ago • 1 comments

This pr is about supporting a custom multiprocess metric path, not changing current usage about env PROMETHEUS_MULTIPROC_DIR.

I found that MultiProcessCollector supports a path param https://github.com/prometheus/client_python/blob/master/prometheus_client/multiprocess.py#L22, though metrics don't support it. According to the discussion here https://github.com/prometheus/client_python/pull/395#issuecomment-481289392 and some more, path is fixed to PROMETHEUS_MULTIPROC_DIR and MultiProcessCollector path param don't actually function afaics. So this pr keeps PROMETHEUS_MULTIPROC_DIR usage, adds support for custom path if multiprocess_dir is specified by metric.

Now MultiProcessCollector path works fine and allows us to serve different metrics exporters for scrapers to scrape in multiprocess mode. These changes have been applied in our production environment for months and look good.

@csmarchbanks ptal :) happy to know if any suggestions.

iostream96 avatar Feb 26 '25 03:02 iostream96

I'm not opposed to this feature, as you say, it seems like it could be helpful for cases where you want to have multiple /metrics endpoints in one server?

That said, how are you dealing with value class? Just setting the env var to a default then using these as overrides? Something else?

Yes it's used like an override. My case is our PaaS platform runs some platform metrics, and we want to provide prom metrics usage for app devs. So it will be better to expose 2 different endpoints in one server to separate platform and app metrics. In practice, PROMETHEUS_MULTIPROC_DIR is set to enable multiprocess mode and is used for platform metrics, and there is an api for app devs like

def new_app_counter(name, documentation, labelnames=None):
    namespace = os.environ.get('APPNAME', '')
    multiprocess_dir = os.environ.get('PROMETHEUS_MULTIPROC_DIR_APP')
    return Counter(
        name=name,
        documentation=documentation,
        namespace=namespace,
        labelnames=labelnames if labelnames else (),
        multiprocess_dir=multiprocess_dir
    )

It sets multiprocess_dir to PROMETHEUS_MULTIPROC_DIR_APP for app metrics.

iostream96 avatar Mar 04 '25 06:03 iostream96