fritzinfluxdb icon indicating copy to clipboard operation
fritzinfluxdb copied to clipboard

Data extrapolation for connected devices

Open muenzpraeger opened this issue 3 years ago • 12 comments

I was wondering if you're open to add some more data extrapolation capabilities for active devices to the project.

Specifically:

  • Is part of Mesh
  • Wifi frequency
  • Upstream / Downstream speed
  • Parent device

Alternatively maybe an option to use a config file with additional services that would then be merged at runtime into the built-in config.

Happy to provide a PR for any of that.

muenzpraeger avatar Oct 24 '22 17:10 muenzpraeger

Hi @muenzpraeger,

I'm open to additions to the current collected data. If you know where to pull the data from you are more then welcome to create a PR. Please fork from next-release as there are already a lot of changes prepared for the next version.

bb-Ricardo avatar Oct 24 '22 18:10 bb-Ricardo

Hi @muenzpraeger,

Just pushed 0faba89 which reworks your PR a bit. Can you review and test it let me know what you think?

I removed the separation between powerline and wifi upstream and downstream values. The regex should now match all cases.

bb-Ricardo avatar Oct 28 '22 14:10 bb-Ricardo

Also a dashboard is still missing.

We should add another "Wifi" Dashboard.

bb-Ricardo avatar Oct 29 '22 09:10 bb-Ricardo

Change looks good. I can add my dashboard. That's in InfluxQL, but if I recall correctly you're moving to Flux, no?

muenzpraeger avatar Oct 31 '22 09:10 muenzpraeger

Hi,

Yes I try to limit maintenance to one type of Dahboards. A Flux Dashboard would be great.

bb-Ricardo avatar Oct 31 '22 13:10 bb-Ricardo

Hey,

I would like to get version 1.1.0 out of the door. Is it ok if I release this version and add the dashboard later?

bb-Ricardo avatar Nov 01 '22 20:11 bb-Ricardo

Feel free to release. Still reading through Flux documentation...

muenzpraeger avatar Nov 02 '22 08:11 muenzpraeger

Great. Or just post the influx version here (or a PR) and I'll try to convert it to Flux queries

bb-Ricardo avatar Nov 02 '22 08:11 bb-Ricardo

Actually I've got it working now. 😄 Feel free to adjust as you see fit.

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 4,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "influxdb",
        "uid": "p2cAWORRz"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.2.1",
      "targets": [
        {
          "alias": "$col",
          "datasource": {
            "type": "influxdb",
            "uid": "p2cAWORRz"
          },
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "name"
              ],
              "type": "tag"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "measurement": "fritzbox",
          "orderByTime": "ASC",
          "policy": "default",
          "query": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => (r[\"_value\"]>0 and (r._field == \"active_hosts_upstream\" or r._field == \"active_hosts_downstream\")))\n  |> filter(fn: (r) => (r._field == \"active_hosts_upstream\" or r._field == \"active_hosts_downstream\"))\n  |> filter(fn: (r) => contains(value: r[\"name\"], set: ${device:json}))\n  |> group(columns: [\"_measurement\", \"_field\"])\n  |> map(fn: (r) => ({ r with _field: \n      if r._field == \"active_hosts_upstream\" then r.name + \" Upstream\" else r.name + \" Downstream\"\n  }))\n  |> map(fn: (r) => ({\n    _time: r._time,\n    _field: r._field,\n    _value: r._value\n  }))\n\n\n\n",
          "rawQuery": true,
          "refId": "A",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "active_hosts_name"
                ],
                "type": "field"
              },
              {
                "params": [
                  "alias"
                ],
                "type": "alias"
              }
            ]
          ],
          "tags": []
        }
      ],
      "title": "Connection Speed",
      "type": "timeseries"
    }
  ],
  "refresh": false,
  "schemaVersion": 37,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {
          "selected": true,
          "text": [
            "Ella-iPhone",
            "Badezimmer"
          ],
          "value": [
            "Ella-iPhone",
            "Badezimmer"
          ]
        },
        "datasource": {
          "type": "influxdb",
          "uid": "p2cAWORRz"
        },
        "definition": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => (r[\"_value\"]>0 and (r._field == \"active_hosts_upstream\" or r._field == \"active_hosts_downstream\")))\n  |> map(fn: (r) => ({\n    label: r.name\n  }))\n\n\n\n",
        "hide": 0,
        "includeAll": true,
        "label": "Device",
        "multi": true,
        "name": "device",
        "options": [],
        "query": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => (r[\"_value\"]>0 and (r._field == \"active_hosts_upstream\" or r._field == \"active_hosts_downstream\")))\n  |> map(fn: (r) => ({\n    label: r.name\n  }))\n\n\n\n",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-7d",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "Connection over time",
  "uid": "FlGAWdggk",
  "version": 25,
  "weekStart": ""
}

muenzpraeger avatar Nov 02 '22 10:11 muenzpraeger

Thank you for the dashboard. I get the idea but need to rework it a bit. Also couldn't import the dashboard because the uuid for the datasource was not found. The issue is, you have to set the export to "extern" so it can be imported by other people. I think this is a silly option which should be the default, but is isn't.

I will have a look into a Wifi dashboard the next few days.

bb-Ricardo avatar Nov 02 '22 18:11 bb-Ricardo

Hello, I also tried a dashboard, I hope I interpreted the requirements correctly.

Dashboard Connection over time:

{
  "__inputs": [
    {
      "name": "DS_INFLUXDB2-FRITZINFLUXDB",
      "label": "InfluxDB2-fritzinfluxdb",
      "description": "",
      "type": "datasource",
      "pluginId": "influxdb",
      "pluginName": "InfluxDB"
    }
  ],
  "__elements": {},
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "9.2.4"
    },
    {
      "type": "datasource",
      "id": "influxdb",
      "name": "InfluxDB",
      "version": "1.0.0"
    },
    {
      "type": "panel",
      "id": "timeseries",
      "name": "Time series",
      "version": ""
    }
  ],
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": null,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "influxdb",
        "uid": "${DS_INFLUXDB2-FRITZINFLUXDB}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "Mbit/s"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 22,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "options": {
        "legend": {
          "calcs": [
            "lastNotNull",
            "min",
            "max"
          ],
          "displayMode": "table",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.2.4",
      "targets": [
        {
          "alias": "$col",
          "datasource": {
            "type": "influxdb",
            "uid": "${DS_INFLUXDB2-FRITZINFLUXDB}"
          },
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "name"
              ],
              "type": "tag"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "measurement": "fritzbox",
          "orderByTime": "ASC",
          "policy": "default",
          "query": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => (r._field == \"active_hosts_upstream\" or r._field == \"active_hosts_downstream\"))\n  |> filter(fn: (r) => r[\"name\"] =~ /^${device:regex}$/)\n  |> group(columns: [\"_measurement\", \"_field\"])\n  |> map(fn: (r) => ({ r with _field: \n      if r._field == \"active_hosts_upstream\" then r.name + \" Upstream\" else r.name + \" Downstream\"\n  }))\n  |> map(fn: (r) => ({\n    _time: r._time,\n    _field: r._field,\n    _value: r._value\n  }))\n\n\n\n",
          "rawQuery": true,
          "refId": "A",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "active_hosts_name"
                ],
                "type": "field"
              },
              {
                "params": [
                  "alias"
                ],
                "type": "alias"
              }
            ]
          ],
          "tags": []
        }
      ],
      "title": "Connection Speed",
      "type": "timeseries"
    }
  ],
  "refresh": false,
  "schemaVersion": 37,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "allValue": "",
        "current": {},
        "datasource": {
          "type": "influxdb",
          "uid": "${DS_INFLUXDB2-FRITZINFLUXDB}"
        },
        "definition": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r[\"_measurement\"] == \"fritzbox\" and (r[\"_field\"] == \"active_hosts_downstream\" and r[\"_value\"]>0) or (r[\"_field\"] == \"active_hosts_upstream\" and  r[\"_value\"]>0) and r[\"box\"] == \"fritz.box\")\n  |> distinct(column: \"uid\")\n  |> map(fn: (r) => ({\n     label: r.name\n  }))",
        "hide": 0,
        "includeAll": true,
        "label": "Device",
        "multi": false,
        "name": "device",
        "options": [],
        "query": "from(bucket: v.defaultBucket)\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r[\"_measurement\"] == \"fritzbox\" and (r[\"_field\"] == \"active_hosts_downstream\" and r[\"_value\"]>0) or (r[\"_field\"] == \"active_hosts_upstream\" and  r[\"_value\"]>0) and r[\"box\"] == \"fritz.box\")\n  |> distinct(column: \"uid\")\n  |> map(fn: (r) => ({\n     label: r.name\n  }))",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-24h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "Connection over time Copy",
  "uid": "mu2hGBDVz",
  "version": 2,
  "weekStart": ""
}

But I noticed something. I use a FritzBox 7590 and a FRITZ!WLAN Repeater 1160. Now I noticed that all the devices that are connected to the repeater are not visible in my dashboard because some values ​​are missing.

Question, new problem or can this be managed by configuration?

dp20eic avatar Nov 10 '22 20:11 dp20eic

Sorry for not working on this. It is currently blocked by this issue https://github.com/bb-Ricardo/fritzinfluxdb/issues/114. I need to find a good way forward with influxDB as I don't like to repeat everything once again.

Any ideas from your side?

bb-Ricardo avatar Jul 13 '24 08:07 bb-Ricardo