output kafka plugin cannot configure 'compression.type' property with zstd compression

[jiachinzhao]

Bug Report

Describe the bug cannot configure 'compression.type' property

To Reproduce running in K8s container my config

output-kafka.conf: |
    [OUTPUT]
        Name           kafka
        Match          access-log
        Brokers        xxx
        Topics         access-log
        rdkafka.compression.type zstd

Logs

Fluent Bit v1.8.11
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2022/01/04 07:06:07] [ info] [engine] started (pid=1)
[2022/01/04 07:06:07] [ info] [storage] version=1.1.5, initializing...
[2022/01/04 07:06:07] [ info] [storage] in-memory
[2022/01/04 07:06:07] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/01/04 07:06:07] [ info] [cmetrics] version=0.2.2
[2022/01/04 07:06:07] [ info] [input:tcp:tcp.0] listening on 0.0.0.0:5170
[2022/01/04 07:06:07] [error] [output:kafka:kafka.0] cannot configure 'compression.type' property

Expected behavior

as the document mentioned, kafka should output plugin support compression zstd

[yackushevas]

in version 1.9.1 problem still exists

[hafkaM]

I have the same problem. I'm using version 1.9.4,

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.