Decodes Apache Avro binary format into events.

Avro Decoder

Decodes Apache Avro binary format into Arrow RecordBatches.

Configuration

FieldTypeRequiredDefaultDescription
codecstringyesMust be "avro"
max_sizeintegerno10485760Maximum message size in bytes (10MB default)

Example

sources:
  my_kafka:
    type: kafka
    bootstrap_servers: localhost:9092
    group_id: my-group
    topics: [avro-events]
    decoding:
      codec: avro

When to Use

  • Schema registry integration - Confluent Schema Registry
  • Compact binary ingestion - Smaller than JSON
  • Hadoop ecosystem - Hive, Spark data sources