Traefik Logs Parser Fails on Non-JSON Logs

[qymab]

The crowdsecurity/traefik-logs parser fails when encountering non-JSON log lines, such as those generated by GeoBlock. These logs are plain text and not in JSON format, causing the UnmarshalJSON function to throw an error. This results in the following error message:

level=warning msg="failed to run filter : invalid character 'I' looking for beginning of value (1:1)
| UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]
| ^" id=blue-grass name=child-crowdsecurity/traefik-logs stage=s01-parse

This happens because the parser attempts to process all log lines, including non-JSON lines, which are incompatible with the UnmarshalJSON function.

---

Steps to Reproduce:

1. Configure Traefik to log both access logs and GeoBlock logs in the same file.
2. Enable the crowdsecurity/traefik-logs parser in CrowdSec.
3. Feed the combined log file to CrowdSec.
4. Observe the parser failing on non-JSON log lines (e.g., INFO: GeoBlock).

---

Expected Behavior:

The crowdsecurity/traefik-logs parser should:

• Ignore non-JSON log lines (e.g., GeoBlock logs) by default.
• Process only valid JSON log lines from Traefik.

---

Actual Behavior:

The parser fails when encountering non-JSON log lines, causing errors like:

invalid character 'I' looking for beginning of value

---

Environment:

• CrowdSec Version: v1.6.4
• GeoBlock Version: v0.2.8
• Traefik Version: v3
• Operating System: Docker/Debian

---

Additional Context:

GeoBlock logs are plain text logs generated by Traefik, such as:

INFO: GeoBlock: 2024/12/26 11:36:01 allow local IPs: true
INFO: GeoBlock: 2024/12/26 11:36:01 log local requests: false

These logs are not in JSON format and should not be processed by the crowdsecurity/traefik-logs parser. Including a filter to exclude non-JSON lines would resolve this issue and prevent unnecessary errors.

---

[LaurenceJJones]

Hey 👋🏻

Thank you opening an issue.

So there an issue with trying to guess if the log line is a not a JSON log line, since traefik supports logging "plain text" known as "CLF" also so what we could do is test if evt.Parsed.message[0] == '{' which would test if the first byte is an opening bracket but we may fall into the trap of whitespace.

However, an easier solution I seen is making sure general traefik log messages and access logs go to different files (which you configure crowdsec to only monitor the access logs) or if both go to stdout then divert general traefik messages to a file instead.

[qymab]

This is my acqui.yaml, which I believe already only logs from access logs.

---
filenames:
  - /var/log/auth.log
  - /var/log/syslog
  - /var/log/kern.log
  - /var/log/ufw.log
labels:
  type: syslog

---
filenames:
  - /logs/web/traefik/access.log
labels:
  type: traefik

---
filenames:
  - /logs/vaultwarden/vaultwarden.log
labels:
  type: vaultwarden

---
filenames:
  - /logs/authelia/authelia.log
labels:
  type: authelia

---
filenames:
  - /logs/nextcloud/nextcloud.log
labels:
  type: nextcloud

---
source: journalctl
journalctl_filter:
  - "_TRANSPORT=syslog"
labels:
  type: syslog

---
source: journalctl
journalctl_filter:
  - "_TRANSPORT=journal"
labels:
  type: syslog

---
source: journalctl
journalctl_filter:
  - "_TRANSPORT=stdout"
labels:
  type: syslog

---
source: journalctl
journalctl_filter:
  - "_TRANSPORT=kernel"
labels:
  type: syslog

---
listen_addr: 0.0.0.0:7422
appsec_config: crowdsecurity/virtual-patching
name: myAppSecComponent
source: appsec
labels:
  type: appsec

[LaurenceJJones]

OP opened another issue #1343 closing this old one