[OUTPUT] s3 plugin crashes when multiple workers are enabled

[cdancy]

Hey All,

We're using latest fluentbit 4.0.1 and tried enabling multiple workers on the s3 output plugin and hit the below:

#0  0x56030b04abc0      in  chunk_state_sync() at lib/chunkio/src/cio_chunk.c:548
#1  0x56030b04abc0      in  chunk_state_sync() at lib/chunkio/src/cio_chunk.c:538
#2  0x56030b04abc0      in  cio_chunk_down() at lib/chunkio/src/cio_chunk.c:566
#3  0x56030ad9ab68      in  flb_fstore_file_meta_set() at src/flb_fstore.c:104
#4  0x56030ad57cbb      in  s3_store_buffer_put() at plugins/out_s3/s3_store.c:161
#5  0x56030ad56c0d      in  buffer_chunk() at plugins/out_s3/s3.c:1676
#6  0x56030ad56c0d      in  cb_s3_flush() at plugins/out_s3/s3.c:2243
#7  0x56030b267e66      in  co_init() at lib/monkey/deps/flb_libco/amd64.c:117

here is the config:

[OUTPUT]
    Name s3
    Match kube.*
    region out-region
    bucket out-bucket
    store_dir /tmp/fluent-bit/s3-kube
    tls on
    tls.verify  off
    retry_limit 10
    total_file_size 3M
    upload_timeout 30s
    store_dir_limit_size 100M
    send_content_md5 true
    s3_key_format /our/special/path
    s3_key_format_tag_delimiters .
    compression gzip
    workers 2

I remember once upon a time this was not supported for the s3 plugin but that seems to be removed in the latest 4.x docs.

[cdancy]

Any updates here?

[aslobodskoy]

Have the same issue