Dead letter queue sink that writes failed events to S3.
Dlq🔗
Dead letter queue sink that writes failed events to S3.
Configuration🔗
| Field | Type | Required | Default | Description |
inputs | [string] | yes | — | Upstream component names that feed events into this sink. |
error_policy | ErrorPolicy | no | drop_on_error | Policy for handling errors during event processing. |
encoding | EncoderConfigEnum (optional) | no | — | Optional encoder configuration for output events. |
buffer | BufferConfig | no | — | Buffer configuration for event batching. |
bucket | string | yes | — | S3 bucket for DLQ storage. |
region | string | yes | — | AWS region for the DLQ bucket. |
prefix | string | yes | — | Key prefix for DLQ objects in S3. |
batch_size | unsigned integer | no | 20 | Number of events per batch write. |
timeout | string | no | 1m | Maximum time before flushing a batch. |
encoding | string | no | json | Encoding format for DLQ objects. |
retry_delay | string | no | 5s | Initial retry delay for failed writes. |
retry_max_delay | string | no | 300s | Maximum retry delay (exponential backoff cap). |
retry_max_attempts | unsigned integer | no | 5 | Maximum number of retry attempts before giving up. |
max_pending_writes | unsigned integer | no | 1024 | Maximum number of concurrent pending writes. |
Telemetry🔗
This component emits the following standard telemetry metrics and events.
Metrics🔗
| Name | Type | Description |
component_sent_events_total | Counter | Total number of Kinetic events successfully sent by this sink. |
component_sent_event_bytes_total | Counter | Total estimated byte size of events sent. |
component_sent_network_bytes_total | Counter | Total number of raw network bytes sent. |
component_errors_total | Counter | Total number of errors encountered during processing. |
Events🔗
| Name | Description |
ComponentError | Emitted when an error occurs during component execution. |
ComponentEventsDropped | Emitted when events are intentionally or unintentionally dropped. |
Example🔗
sinks:
my_dlq:
type: dlq
inputs: ["my_source"]
bucket: my-kinetic-dlq
region: us-east-1
prefix: dlq/alerts/