netbox-plugin-prometheus-sd icon indicating copy to clipboard operation
netbox-plugin-prometheus-sd copied to clipboard

Intended use / Add more usage examples

Open ekrichbaum opened this issue 3 years ago • 3 comments

I can't seem to wrap my head around how this works in general. Netbox api call (with slight mod to change id to string from other thread) returns a boatload of items out of my netbox.

Those 758 objects all show as "down" because prometheus then tries to contact each of them on port 80 (of which most do not respond). I assume that I should be adding something to the netbox side so that prometheus then knows what to look for in an object? i.e. use snmp, snmp community, address to use instead of the name?

ekrichbaum avatar Feb 24 '23 15:02 ekrichbaum

Here is an example how you can use it with node-exporterfor example. You have to set the proper port on the __address__ label. In this example this is hard coded, but could also be fetched from a regex.

To filter valid targets use the keep and drop relable configs.

- job_name: node_exporter
  honor_timestamps: true
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  follow_redirects: true
  enable_http2: true
  relabel_configs:
  - source_labels: [__meta_netbox_services]
    separator: ;
    regex: .*(node-exporter|node_exporter).*
    replacement: $1
    action: keep
  - source_labels: [__address__]
    separator: ;
    regex: ([^:]+)(:[0-9]+)?
    target_label: __address__
    replacement: ${1}:9100
    action: replace

FlxPeters avatar Feb 27 '23 10:02 FlxPeters

so, in that case, an element comes in from netbox, then uses anything listed as services. That is a useful then (should be shown like that in some form of documentation but I think still needs more explanation.

I brute forced this and figured out that if I create a custom field icmp (boolean) and built my sd query on that, I can get a proper subset, sent to the blackbox-exporter for an icmp check. This would seem to be the simplest form of a use case but not visible in any sort of documentation anywhere. I'll work on a second version for an snmp check against a router, then try to build a document to be included here so that noone else has to go through all of this. Most of the items googled assumed too much familiarity with prometheus as a base rather than using a good plugin with a good data source to add prometheus to the stack.

Thanks for the work here. Don't get me wrong, this was a critical piece of the puzzle and great work.

ekrichbaum avatar Feb 27 '23 15:02 ekrichbaum

Yes, the trick is to filter the targets as good as possible with Netbox filters and than use relabel_configs for further processing. This is a quite common pattern for other SD modules too. That's the reason why there is not too many docs on this repo, because it's more a Prometheus usage pattern.

I will let this issue open for now as a reminder to provide some more meaningful examples.

FlxPeters avatar Feb 28 '23 08:02 FlxPeters

This issue has been automatically closed because it has been inactive for more than 60 days. Please reopen if you still intend to submit this pull request.

github-actions[bot] avatar May 25 '24 00:05 github-actions[bot]