raccoon
raccoon copied to clipboard
Feat: Kinesis sink
This PR adds support for AWS Kinesis sink(#78).
Configuration
This change introduces the following new configurations
- a new
PUBLISHER_TYPEvalue calledkinesis -
PUBLISHER_KINESIS_AWS_REGIONregion with the target kinesis stream resides (fallbacks back toAWS_REGION) -
PUBLISHER_KINESIS_CREDENTIALSpath to AWS Credentials file (defaults to `$HOME/.aws/credentials) -
PUBLISHER_KINESIS_STREAM_PROBE_INTERVAL_MStime interval for stream readiness check. -
PUBLISHER_KINESIS_STREAM_AUTOCREATEwhether Raccoon should create streams that don't exist. -
PUBLISHER_KINESIS_STREAM_MODEwhen creating streams, what mode to set. (ON_DEMANDorPROVISIONED) -
PUBLISHER_KINESIS_STREAM_SHARDSwhen creating streams, how many shards to configure. -
PUBLISHER_KINESIS_PUBLISH_TIMEOUT_MShow long to wait for before aborting a publish operation (per batch)
Metrics
This change also introduces the following new metrics
-
kinesis_messages_delivered_totaltotal number of events processed (includes false negatives) -
kinesis_messages_undelivered_totaltotal number of events that failed -
kinesis_unknown_stream_failure_totalevents that failed to publish to an unknown stream -
kinesis_producebulk_tt_msbatch production time.