Parquet Decoder

codec

Decodes Apache Parquet columnar format into events.

Parquet Decoder

Decodes Apache Parquet columnar format into Arrow RecordBatches.

Configuration

FieldTypeRequiredDefaultDescription
codecstringyes—Must be "parquet"
max_sizeintegerno10485760Maximum message size in bytes (10MB default)

Example

sources:
  my_s3:
    type: aws_s3
    bucket: data-lake
    prefix: events/
    decoding:
      codec: parquet

When to Use

  • Data lake ingestion - S3, GCS parquet files
  • Batch analytics - Columnar query efficiency
  • Historical data - Compressed columnar storage