fluent
bug: Configuring fluent-watcher
Describe the issue
HI Team, I deployed fluent-bit using fluent operator v3.3.0. In case of any configuration change, the changes are not effective unless if we restart the fluent-bit pods manually. Tried using fluent-watcher with the below changes in values.yaml.
Fluent-watcher
command: ["/fluent-bit/bin/fluent-bit-watcher"]
# custom command line arguments.
args:
- "-c=/fluent-bit/etc/fluent-bit.conf"
- "-watch-path=/fluent-bit/etc
With these chagnes, the configurational changes are not effective unless with a manual restart. Can you please let me know if I am missing anything?
To Reproduce
Incorporated fluent-watcher with the below changes:
Fluent-watcher
command: ["/fluent-bit/bin/fluent-bit-watcher"]
# custom command line arguments.
args:
- "-c=/fluent-bit/etc/fluent-bit.conf"
- "-watch-path=/fluent-bit/etc"
The configurational changes should be effective without a manual restart of fluent-bit pods.
Expected behavior
The configurational changes should be effective without a manual restart of fluent-bit pods with fluent-watcher
Your Environment
- Fluent Operator version: v3.3.0 on a k8s cluster.
How did you install fluent operator?
No response
Additional context
No response
from my understanding, you don't need to configure command and args, the fluent-bit-watcher will automatically enable the file watcher feature.
I have a similar issue with Fluent Operator: v3.4.0, Fluent Bit: v4.0.1. It looks like a race condition:
- Update a ClusterFilter, sync an ArgoCD application.
- Wait few minutes and check OpenSearch: some Fluent Bit's pods use the new filter, others — are not, though in logs I see the messages about
level=info time=2025-07-07T12:58:12Z msg="Config file changed, reloading..."
[2025/07/07 12:58:12] [engine] caught signal (SIGHUP)
…
- Update a ClusterFilter, sync the application, and wait again.
- See the old (from step1) filter is applied everywhere, but new one only on few nodes.
An only idea I have to describe that behavior: Fluent Bit always restarts, but sometimes it rereads an old version of its config.
