textfile collector won't send any metric to prometheus server

[IdeoG]

Host operating system: output of uname -a

Linux ... 4.14.287-215.504.amzn2.x86_64 #1 SMP Wed Jul 13 21:34:43 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

v1.5.0

node_exporter command line flags

version: "3"

services:
  node_exporter:
    image: quay.io/prometheus/node-exporter:v1.5.0
    volumes:
      - "/proc:/host/proc:ro"
      - "/sys:/host/sys:ro"
      - "/:/rootfs:ro"
    command:
      - "--path.procfs=/host/proc"
      - "--path.rootfs=/rootfs"
      - "--path.sysfs=/host/sys"
      - "--collector.disable-defaults"
      - "--web.disable-exporter-metrics"
      # - "--collector.loadavg"
      # - "--collector.meminfo"
      # - "--collector.filesystem"
      # - "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc|mnt|run)($$|/)"
      - "--collector.textfile"
      - "--collector.textfile.directory=/rootfs/prom_files"
    ports:
      - 9010:9100
    deploy:
      mode: global

node_exporter log output

ts=2023-06-01T06:41:38.332Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)"
ts=2023-06-01T06:41:38.332Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)"
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:186 level=debug msg="Go MAXPROCS" procs=1
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:117 level=info collector=textfile
ts=2023-06-01T06:41:38.333Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100
ts=2023-06-01T06:41:38.333Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100
ts=2023-06-01T06:41:45.478Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:41:45.479Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000507488
ts=2023-06-01T06:42:00.476Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:42:00.477Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000264999
ts=2023-06-01T06:42:15.476Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:42:15.476Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000256308

Are you running node_exporter in Docker?

Yes, especially in docker swarm installation. I have deployed node exporter only on manager nodes.

What did you do that produced an error?

I just added textfile collector metrics from a file that is updated by cronjob every minute. The file is correct and updated correctly every 1 minute. The data weren't appeared in prometheus, but after several reloads it is. So, we live with that bug for weeks, but didn't find any solution to make node exporter stable to send metrics from textfile collector.

What did you expect to see?

Textfile collector sends data from the file correctly after any node exporter reload.

What did you see instead?

After node exporter reload, node exporter won't send any textfile collector metric to prometheus server, only after several more reloads.

[discordianfish]

Please provide

• content of /rootfs/prom_files/*
• output of curl node-exporter-host:9100/metrics