Arrow IPC Decoder
Decodes Apache Arrow IPC Stream format into Arrow RecordBatches.
Configuration
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
codec | string | yes | — | Must be "arrow" |
max_size | integer | no | 10485760 | Maximum message size in bytes (10MB default) |
Example
sources:
my_kafka:
type: kafka
bootstrap_servers: localhost:9092
group_id: my-group
topics: [arrow-events]
decoding:
codec: arrowWhen to Use
- Kinetic-to-Kinetic pipelines - Efficient internal communication
- Analytics ingestion - Zero-copy deserialization
- High-throughput sources - Minimal parsing overhead