[BUG] metrics with utf8 strings in path are not displayed

[mcikosos]

Describe the bug metrics with utf8 strings in path are not displayed in graphs

To Reproduce just send any metric with national chars in it

ex. echo "local.random.gówno 5 date +%s" | nc ${SERVER} ${PORT}

Expected behavior metrics show in graphs

Screenshots no point

Environment (please complete the following information):

• OS flavor: Ubuntu
• Graphite-web version: latest docker - graphiteapp/graphite-statsd
• Django/Python version: latest docker - graphiteapp/graphite-statsd
• Setup type latest docker - graphiteapp/graphite-statsd

[mcikosos]

When I change metric path in graphite web - using edit to change it to 'local.random.g?wno' - the data are shown

[deniszh]

Hi @mcikosos ,

IIRC UTF-8 characters can work in tags names/values (because tags are hashed internally), but UTF-8 support in metrics itself is depending on your OS / filesystem setup. I mean, I'm not saying that's not a bug - and if someone can solve it in generic and backward compatible way - I would be very glad.

[piotr1212]

I found a few issues with unicode chars in Graphite-web, carbon on the other hand does work.

This fixes two https://github.com/piotr1212/graphite-web/commit/17e23efecb6cb40059264fabd3ea36718065176e but I never got around of making a PR. I think there was also one in the carbonlink code. And last, the images rendering code cannot handle unicode (metricnames in the legenda). The last one is the hardest to fix but probably the least important one. There is probably some more, potential places to look are str() calls

[mcikosos]

IIRC UTF-8 characters can work in tags names/values (because tags are hashed internally), but UTF-8 support in metrics itself is depending on your OS / filesystem setup. I mean, I'm not saying that's not a bug - and if someone can solve it in generic and backward compatible way - I would be very glad.

Well I thought it is a BUG because carbon/whisper etc. works ok but if it was not supported from the start I can change it to "feature" ;)

[mcikosos]

@piotr1212 can You make a PR? So some day it will be merged to master.

[piotr1212]

Sorry, I don't expect to anytime soon.

[deniszh]

Oh, cool, I didn't know that, @piotr1212 ! PR is welcome ofc, I will pick this up if nobody else.

[deniszh]

@mcikosos : Fix is merged in master. UTF8_METRICS should be enabled in config (disabled by default because of possible negative performance consequences).

[mcikosos]

I will check this ASAP

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mcikosos]

Issue still exists I have graphiteapp/graphite-statsd:master GRAPHITE_UTF8_METRICS=true in env and still no data for metrics with utf8 chars :(

[deniszh]

Hello,

I’m afraid we will need exact metrics names to check, because simple tests (built in and test run) shows that setting works.

On Wed, 30 Dec 2020 at 17:20, mcikosos [email protected] wrote:

Issue still exists I have graphiteapp/graphite-statsd:master GRAPHITE_UTF8_METRICS=true in env and still no data for metrics with utf8 chars :(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/graphite-project/graphite-web/issues/2641#issuecomment-752678729, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJLTVTEF32K2B3AVR4J65TSXNHL7ANCNFSM4SVG5YOQ .

[mcikosos]

example which does not work: home.Piętro.Suszarnia.Pralka.energy when I change to home.Pi?tro.Suszarnia.Pralka.energy data are shown I will try the example from the description to see if it works on my instance

[mcikosos]

I tested with test.random.pokój and no data is shown for that exact metric

[deniszh]

@mcikosos : Yes, you're right. Looks like some part still missing. I see data coming but rendering part is not working, not in curl nor in browser. :(

%curl 'http://localhost/render/?target=test.random.pokój&format=json&from=-1min'
[]

but

% curl 'http://localhost/render/?target=test.random.pok?j&format=json&from=-1min'
[{"target": "test.random.pok\u00f3j", "tags": {"name": "test.random.pok\u00f3j"}, "datapoints": [[null, 1609532390], [null, 1609532400], [null, 1609532410], [null, 1609532420], [null, 1609532430], [null, 1609532440]]}]

Encoding URL not helping either (I think curl and browser dong that anyway):

[dzhdanov:~] % curl 'http://localhost/render/?target=test.random.pok%C3%B3j&format=json&from=-1min'
[]%