Arrow IPC Encoder

codec

Encodes events to Apache Arrow IPC format.

Arrow IPC Encoder

Encodes Arrow RecordBatches to Apache Arrow IPC Stream format.

Configuration

FieldTypeRequiredDefaultDescription
codecstringyes—Must be "arrow"

Example

sinks:
  my_kafka:
    type: kafka
    bootstrap_servers: localhost:9092
    topic: events
    encoding:
      codec: arrow

When to Use

  • Kinetic-to-Kinetic pipelines - Efficient internal communication
  • Analytics workloads - Zero-copy deserialization
  • Columnar data processing - Preserve Arrow format end-to-end