Decodes Apache Arrow IPC format into events.

Arrow IPC Decoder

Decodes Apache Arrow IPC Stream format into Arrow RecordBatches.

Configuration

FieldTypeRequiredDefaultDescription
codecstringyesMust be "arrow"
max_sizeintegerno10485760Maximum message size in bytes (10MB default)

Example

sources:
  my_kafka:
    type: kafka
    bootstrap_servers: localhost:9092
    group_id: my-group
    topics: [arrow-events]
    decoding:
      codec: arrow

When to Use

  • Kinetic-to-Kinetic pipelines - Efficient internal communication
  • Analytics ingestion - Zero-copy deserialization
  • High-throughput sources - Minimal parsing overhead