Amazon Kinesis stream configuration.
Aws Kinesis🔗
Amazon Kinesis stream configuration.
Configuration🔗
| Field | Type | Required | Default | Description |
|---|
inputs | [string] | no | — | Upstream component names that feed events into this sink (when used as sink). |
error_policy | ErrorPolicy | no | drop_on_error | Policy for handling errors during event processing (when used as sink). |
decoding | DecoderConfigEnum (optional) | no | — | Optional decoder/encoder configuration for input/output events. |
buffer | BufferConfig | no | — | Buffer configuration for event batching (when used as sink). |
auth | AwsConfig (optional) | no | — | AWS authentication and region settings. |
region | string (optional) | no | — | AWS region to operate in. |
endpoint | string (optional) | no | — | Custom AWS endpoint URL (for LocalStack, MinIO, etc.). |
access_key_id | string (optional) | no | — | AWS access key ID for static credentials. |
secret_access_key | string (optional) | no | — | AWS secret access key for static credentials. (sensitive) |
session_token | string (optional) | no | — | AWS session token for temporary credentials. (sensitive) |
assume_role | string (optional) | no | — | IAM role ARN to assume. |
session_name | string (optional) | no | — | Session name when assuming a role. |
external_id | string (optional) | no | — | External ID for cross-account role assumption. |
profile | string (optional) | no | — | AWS profile name from shared credentials file. |
credentials_file | string (optional) | no | — | Path to an AWS credentials file. |
imds | ImdsConfig (optional) | no | — | Instance Metadata Service (IMDS) configuration. |
connect_timeout_seconds | unsigned integer (optional) | no | 1 | Timeout in seconds for IMDS connection. |
max_attempts | unsigned integer (optional) | no | 3 | Maximum number of IMDS retry attempts. |
read_timeout_seconds | unsigned integer (optional) | no | 1 | Timeout in seconds for IMDS read operations. |
load_timeout_secs | unsigned integer (optional) | no | 10 | Timeout in seconds for loading AWS credentials. |
stream_name | string | yes | — | Kinesis stream name. |
Telemetry🔗
This component emits the following standard telemetry metrics and events.
Metrics🔗
| Name | Type | Description |
|---|
component_received_events_total | Counter | Total number of Kinetic events created by this source. |
component_received_event_bytes_total | Counter | Total estimated byte size of events created. |
component_received_network_bytes_total | Counter | Total number of raw network bytes received. |
component_errors_total | Counter | Total number of errors encountered during processing. |
Events🔗
| Name | Description |
|---|
ComponentError | Emitted when an error occurs during component execution. |
ComponentEventsDropped | Emitted when events are intentionally or unintentionally dropped. |
Example🔗
sources:
my_aws_kinesis:
type: aws_kinesis
inputs: ["my_transform"]
auth:
region: us-west-2
endpoint: http://localhost:4566
access_key_id: AKIAIOSFODNN7EXAMPLE
secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
assume_role: arn:aws:iam::123456789012:role/kinetic-ingest
session_name: kinetic-session
profile: default
credentials_file: /home/kinetic/.aws/credentials
imds:
connect_timeout_seconds: 5
max_attempts: 5
read_timeout_seconds: 5
stream_name: my-stream