Expand description
Configuration models for Kinetic pipelines.
Structs§
- Aggregate
Config - Configuration for generic SQL aggregate transform
- Aggregate
Config Wrapped - Wrapper for generic SQL aggregate transform with common pipeline fields.
- Archive
Config - Archive configuration for preserving raw input.
- Archive
Storage Config - Configuration for the automated source-level archival storage.
- Config
- Root configuration for a Kinetic pipeline.
- Config
Diff - Difference between two pipeline configurations.
- Delta
Sink Config Wrapped - Writes events to Delta Lake tables.
- DlqSink
Config - Dead letter queue sink that writes failed events to S3.
- DlqSink
Config Wrapped - Dead letter queue sink that writes failed events to S3.
- Filter
Condition - Condition expression for the filter transform.
- Filter
Config - Filters events based on a condition, either passing or dropping matches.
- Flight
SqlConfig Wrapped - Produces events to an Apache Arrow Flight SQL server.
- GcsSink
Config Wrapped - Writes events to Google Cloud Storage as batched objects.
- GcsSource
Config Wrapped - Reads events from Google Cloud Storage objects. Supports both list-based polling and Pub/Sub notifications.
- Iceberg
Sink Config Wrapped - Writes events to Apache Iceberg tables.
- Internal
Logs Config - Exposes Kinetic’s own internal log events as a source.
- Internal
Metrics Config - Exposes Kinetic’s own internal metrics as a source.
- Kafka
Consumer Config Wrapped - Reads events from one or more Apache Kafka topics using a consumer group.
- Kafka
Producer Config Wrapped - Produces events to an Apache Kafka topic.
- Kinesis
Config Wrapped - Amazon Kinesis stream configuration.
- MapTransform
Config - Configuration for WASM-based map transform.
- MapTransform
Config Wrapped - Wrapper for WASM map transform with common pipeline fields.
- Otel
Aggregate Config - Configuration for OTel-aware aggregate transform
- Otel
Aggregate Config Wrapped - Wrapper for OTel-aware aggregate transform with common pipeline fields.
- Otel
Source Config Wrapped - Receives logs, metrics, and traces via OpenTelemetry gRPC/HTTP.
- PubSub
Sink Config Wrapped - Produces events to a Google Cloud Pub/Sub topic.
- PubSub
Source Config Wrapped - Reads events from a Google Cloud Pub/Sub subscription.
- S3Sink
Config Wrapped - Writes events to Amazon S3 as batched objects.
- S3Source
Config Wrapped - Reads events from Amazon S3 objects. Supports both list-based polling and SQS event notifications.
- Sampling
Config Wrapped - Sampling transform with common pipeline envelope fields.
- SqlMap
Config - Configuration for generic SQL map transform
- SqlMap
Config Wrapped - Wrapper for SQL map transform with common pipeline fields.
- SqsConfig
Wrapped - Amazon SQS (Simple Queue Service) source configuration.
- Star
Rocks Sink Config Wrapped - Writes events to StarRocks OLAP tables.
- Tail
Rule Config - A single rule for tail sampling.
Enums§
- Aggregation
Type - Type of aggregation for OTel metrics
- Buffer
Config - Buffer configuration for event batching between transforms and sinks.
- Decoder
Config Enum - Available decoder codecs for input deserialization.
- Encoder
Config Enum - Available encoder codecs for output serialization.
- Error
Policy - Policy for handling errors during event processing.
- Filter
Type - Type of filter operation.
- Naming
Convention - Naming convention for output metric names.
- Otel
Aggregation - Type of Otel aggregation to perform.
- Sampling
Mode - Sampling mode configuration.
- Sink
Config Enum - Discriminated union of all sink component types.
- Source
Config Enum - Discriminated union of all source component types.
- Transform
Config Enum - Discriminated union of all transform component types.