Metric help text is ignored in multiprocess mode

[haizaar]

When using prometheus client in multiprocess mode as in the docs example, metric description always says Multiprocess metric:

# HELP inprogress_requests Multiprocess metric
# TYPE inprogress_requests gauge
inprogress_requests 0.0
# HELP num_requests Multiprocess metric
# TYPE num_requests counter
num_requests 81634.0

[rszalski]

What version are you on @haizaar ? I'm using multiprocess mode on version ==0.1.1 and all the help messages look ok.

OS: macOS 10.13.3 Python: 3.6.4 (through pipenv)

[haizaar]

The bug report is for 0.0.22. Now tried with the latest 0.1.1 and python 3.4.6 on Ubuntu Linux 16.04 and it's still the same issue:

# HELP gunicron_workers Multiprocess metric
# TYPE gunicron_workers gauge
gunicron_workers 0.0

On 12 February 2018 at 20:53, Radosław Szalski [email protected] wrote:

What version are you on @haizaar https://github.com/haizaar ? I'm using multiprocess mode on version ==0.1.1 and all the help messages look ok.

OS: macOS 10.13.3 Python: 3.6.4 (through pipenv)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/prometheus/client_python/issues/211#issuecomment-364874441, or mute the thread https://github.com/notifications/unsubscribe-auth/AADjWRpIKnbmMjp3li8uxKtqni60c4QZks5tUAoBgaJpZM4QboY7 .

-- Zaar

[rszalski]

Paste the code you use to define and generate those metrics, maybe we can figure something out or reproduce this.

[brian-brazil]

This is the way it works currently, multiprocess mode does not preserve the help text.

[rszalski]

This is the way it works currently, multiprocess mode does not preserve the help text.

Gotta check my config then, might have botched something when enabling multiprocess mode 😕

[HannesMvW]

Is this perhaps in the pipe, to preserve help texts in multiprocess mode?

[tsibley]

@brian-brazil Is there a reason why multiprocess mode does not preserve the help text? It seems like it could be passed along and serialized alongside the metric name and labels and then used when instantiating the Metric() again by the MultiProcessCollector.

[csmarchbanks]

I don't know of any blocking reason this could not be implemented, if someone wants to submit a pull request I would be happy to review it. There might be some edge cases in the implementation such as what to do if the help text across the multiple processes does not match for some reason.

[tsibley]

@csmarchbanks Great, thanks! I expect I'll make a PR sometime in the next few weeks unless someone beats me to it (but given this issue was opened over 3 years ago, it seems unlikely! :upside_down_face:)

In the case of conflicting help text, two options I see are arbitrarily keeping one of them or falling back to the existing placeholder text. The first seems more helpful to me since it'll still be more specific than the placeholder, but I'd be curious to hear your thoughts.

[csmarchbanks]

Arbitrarily keeping one, but then only using that one for future scrapes would be ok with me (basically first one wins). It is an edge case that most users hopefully will not run into, but avoiding metadata churn in the Prometheus server would be nice. I don't know if there will be other similar edge cases or not, but I am sure you will find out when you start working on it :).

[sandipb]

Any update on this? Since the metric directory is supposed to be wiped on every gunicorn restart, I am ok with reusing any previously set placeholder text, in case of conflicts.

[harpener]

Hi guys, is it okay that the change is not backwards-compatible? There is an exception raised now in version 1.16.0 in case the registry tries to collect data from database files that existed before:

from prometheus_client import CollectorRegistry, generate_latest, multiprocess

registry = CollectorRegistry()
multiprocess.MultiProcessCollector(registry, path="metrics")
data = generate_latest(registry)

Traceback (most recent call last):
  File "path-to-app", line X, in Y
    data = generate_latest(registry)
  File "path-to-env\prometheus_client\exposition.py", line 198, in generate_latest
    for metric in registry.collect():
  File "path-to-env\prometheus_client\registry.py", line 97, in collect
    yield from collector.collect()
  File "path-to-env\prometheus_client\multiprocess.py", line 151, in collect
    return self.merge(files, accumulate=True)
  File "path-to-env\prometheus_client\multiprocess.py", line 43, in merge
    metrics = MultiProcessCollector._read_metrics(files)
  File "path-to-env\prometheus_client\multiprocess.py", line 72, in _read_metrics
    metric_name, name, labels, labels_key, help_text = _parse_key(key)
  File "path-to-env\prometheus_client\multiprocess.py", line 54, in _parse_key
    metric_name, name, labels, help_text = json.loads(key)
ValueError: not enough values to unpack (expected 4, got 3)

[csmarchbanks]

I probably should have put a note in release about that, but it is expected and from the docs:

This directory must be wiped between process/Gunicorn runs (before startup is recommended).

So old versions should not be processed by the new code (and vice versa).

Also, closing the issue as it has been implemented!