Writes events to Apache Iceberg tables.

Iceberg🔗

Writes events to Apache Iceberg tables.

Configuration🔗

FieldTypeRequiredDefaultDescription
inputs[string]noUpstream component names that feed events into this sink.
error_policyErrorPolicynoPolicy for handling errors during event processing.
encodingEncoderConfigEnum (optional)noOptional encoder configuration for output events.
bufferBufferConfignoBuffer configuration for event batching.
catalog_endpointstringnoIceberg catalog endpoint URL.
tablestringnoFully qualified Iceberg table name (e.g., 'db.table').
warehousestringnoBase URI for the Iceberg warehouse storage.
max_rowsunsigned integernoMaximum number of rows per batch before flushing.
max_bytesunsigned integernoMaximum byte size of a batch before flushing.
max_age_msunsigned integernoMaximum age of a batch in milliseconds before flushing.

Telemetry🔗

This component emits the following standard telemetry metrics and events.

Metrics🔗

NameTypeDescription
component_sent_events_totalCounterTotal number of Kinetic events successfully sent by this sink.
component_sent_event_bytes_totalCounterTotal estimated byte size of events sent.
component_sent_network_bytes_totalCounterTotal number of raw network bytes sent.
component_errors_totalCounterTotal number of errors encountered during processing.

Events🔗

NameDescription
ComponentErrorEmitted when an error occurs during component execution.
ComponentEventsDroppedEmitted when events are intentionally or unintentionally dropped.

Example🔗

sinks:
  my_iceberg:
    type: iceberg