Shomil Jain
Shomil Jain
Hi! Are there updates on the prioritization of this feature? Handling the termination case is a critical requirement of our workflow.
Does anyone know if there are any hacks we can take in the meantime to ensure chunks are flushed to S3?
I'm also having a similar issue and can't seem to resolve it despite setting all there parameters (timeout_secs, max_bytes, max_events) to very large numbers. Vector created 136 chunk files for...
We're seeing similar numbers to @amanmahajan26 -- does anyone know if there's a stable version that we can downgrade to that proper batch sizing worked on?
I'm still seeing a S3 sink time out on SIGTERM on Vector 0.32.1 – here is the schema of my sink configuration (w/ some fake values): ``` customer_bucket_application_logs: type: aws_s3...
@cjackal - did you make any progress on investigating this? We're seeing this in a different context and are unable to make any progress debugging it (https://github.com/ray-project/ray/issues/41137).
> @shomilj can you take a look approve this PR? it's adding a 1s timeout when GCS asks each driver core worker about is_running_tasks. If it timed out, GCS returns...