Multiprocess application sometimes throwing UnicodeDecodeError in prometheus_client/mmap_dict.py

[Velan987]

Hi I have multiprocess enabled application with flask exporter, sometimes it is throwing below error File "usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 44, in _read_all_values yield encoded_key.decode('utf-8'), value, pos UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 18: invalid continuation byte.

After this error my application is crashed. Can someone please help me on this?

[csmarchbanks]

Hmm, I haven't seen this one before, is this something that can be reproduced? Some version and environment information may also be helpful.

[Velan987]

Hi @csmarchbanks, we are using prometheus-client 0.15.0 in docker environment Below is the full error:

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2051, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in full_dispatch_request return self.finalize_request(rv) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1523, in finalize_request response = self.process_response(response) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1865, in process_response response = self.ensure_sync(handler)(response) File "/usr/local/lib/python3.8/site-packages/prometheus_flask_exporter/init.py", line 482, in after_request request_duration_metric.labels(**request_duration_labels).observe(total_time) File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 190, in labels self._metrics[labelvalues] = self.class( File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 558, in init super().init( File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 138, in init self._metric_init() File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 586, in _metric_init self._sum = values.ValueClass(self._type, self._name, self._name + '_sum', self._labelnames, self._labelvalues) File "/usr/local/lib/python3.8/site-packages/prometheus_client/values.py", line 68, in init self.__reset() File "/usr/local/lib/python3.8/site-packages/prometheus_client/values.py", line 82, in __reset files[file_prefix] = MmapedDict(filename) File "/usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 78, in init for key, _, pos in self._read_all_values(): File "/usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 44, in _read_all_values yield encoded_key.decode('utf-8'), value, pos UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 18: invalid continuation byte

[Velan987]

I have configured my application like below metrics = MultiprocessInternalPrometheusMetrics(app, path="/api/metrics") values.ValueClass = values.MultiProcessValue(os.getppid)

If I do not add "values.ValueClass = values.MultiProcessValue(os.getppid)" facing slowness in metrics endpoint