Avro Decoder
Decodes Apache Avro binary format into Arrow RecordBatches.
Configuration
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
codec | string | yes | — | Must be "avro" |
max_size | integer | no | 10485760 | Maximum message size in bytes (10MB default) |
Example
sources:
my_kafka:
type: kafka
bootstrap_servers: localhost:9092
group_id: my-group
topics: [avro-events]
decoding:
codec: avroWhen to Use
- Schema registry integration - Confluent Schema Registry
- Compact binary ingestion - Smaller than JSON
- Hadoop ecosystem - Hive, Spark data sources