Dead letter queue sink that writes failed events to S3.

Dlq🔗

Dead letter queue sink that writes failed events to S3.

Configuration🔗

FieldTypeRequiredDefaultDescription
inputs[string]yesUpstream component names that feed events into this sink.
error_policyErrorPolicynodrop_on_errorPolicy for handling errors during event processing.
encodingEncoderConfigEnum (optional)noOptional encoder configuration for output events.
bufferBufferConfignoBuffer configuration for event batching.
bucketstringyesS3 bucket for DLQ storage.
regionstringyesAWS region for the DLQ bucket.
prefixstringyesKey prefix for DLQ objects in S3.
batch_sizeunsigned integerno20Number of events per batch write.
timeoutstringno1mMaximum time before flushing a batch.
encodingstringnojsonEncoding format for DLQ objects.
retry_delaystringno5sInitial retry delay for failed writes.
retry_max_delaystringno300sMaximum retry delay (exponential backoff cap).
retry_max_attemptsunsigned integerno5Maximum number of retry attempts before giving up.
max_pending_writesunsigned integerno1024Maximum number of concurrent pending writes.

Telemetry🔗

This component emits the following standard telemetry metrics and events.

Metrics🔗

NameTypeDescription
component_sent_events_totalCounterTotal number of Kinetic events successfully sent by this sink.
component_sent_event_bytes_totalCounterTotal estimated byte size of events sent.
component_sent_network_bytes_totalCounterTotal number of raw network bytes sent.
component_errors_totalCounterTotal number of errors encountered during processing.

Events🔗

NameDescription
ComponentErrorEmitted when an error occurs during component execution.
ComponentEventsDroppedEmitted when events are intentionally or unintentionally dropped.

Example🔗

sinks:
  my_dlq:
    type: dlq
    inputs: ["my_source"]
    bucket: my-kinetic-dlq
    region: us-east-1
    prefix: dlq/alerts/