Daniel Hillman
Daniel Hillman
@m1cha3lf @jszwedko @zamazan4ik we are experiencing this, and other erratic behavior with `disk` buffers at scale. We also notice that sometimes when a buffer gets corrupted, new pods that spin...
@zamazan4ik we have a pretty large volume compared to you I suppose... We process petabytes of data per month and we're seeing these errors way more frequently. I think this...
@m1cha3lf what data do you feel would be helpful? I can share our buffer configuration, etc. but there's not much I can share that isn't private. I'm a bit confused...
Another potentially related issue: https://github.com/vectordotdev/vector/issues/19379
> That limitation is speaking only about the maximum active size of a disk buffer @tobz just to clarify, none of our disk buffers are configured to 8TB, it's way...
@tobz thanks for the clarification. What is the priority of addressing this and other `disk` buffer related bugs in Vector?
@jszwedko can you confirm that sinks try to drain disk buffers when SIGTERM is sent? Can you direct me to the code responsible for that?
@jszwedko opened this ticket up as promised, do you have any insights? Thanks!
@jszwedko, yes. The credentials were not loaded automatically and we had "ExpiredToken" errors that would occur with every event until we issued a `SIGTERM` signal. After that the new pod...
Another potential related issue: https://github.com/vectordotdev/vector/issues/7013