Skip to main content

CatalogProvider

catalog_common

Trait CatalogProvider 

Source 
pub trait CatalogProvider: Send + Sync {
    type Error: Error + Send + Sync + 'static;

    // Required methods
    fn get_table_spec<'life0, 'life1, 'async_trait>(
        &'life0 self,
        table: &'life1 TableIdent,
    ) -> Pin<Box<dyn Future<Output = Result<TableSpec, Self::Error>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn write_batch<'life0, 'life1, 'async_trait>(
        &'life0 self,
        table: &'life1 TableIdent,
        batch: RecordBatch,
    ) -> Pin<Box<dyn Future<Output = Result<WriteReceipt, Self::Error>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn commit<'life0, 'life1, 'async_trait>(
        &'life0 self,
        table: &'life1 TableIdent,
        receipts: Vec<WriteReceipt>,
    ) -> Pin<Box<dyn Future<Output = Result<CommitOutcome, Self::Error>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn get_security_context<'life0, 'life1, 'async_trait>(
        &'life0 self,
        table: &'life1 TableIdent,
    ) -> Pin<Box<dyn Future<Output = Result<SecurityContext, Self::Error>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
}
Expand description

The core abstraction for data lake catalog backends.

Implementations MUST be Send + Sync and MUST NOT panic on invalid input. Errors MUST be returned via the backend-specific snafu error enum exposed through the associated Error type.

Required Associated Types§

Source

type Error: Error + Send + Sync + 'static

Backend-specific error type (snafu enum).

Required Methods§

Source

fn get_table_spec<'life0, 'life1, 'async_trait>( &'life0 self, table: &'life1 TableIdent, ) -> Pin<Box<dyn Future<Output = Result<TableSpec, Self::Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Load a table’s current spec (schema, partition spec, properties, current snapshot pointer). MUST be idempotent and safe to call concurrently.

Source

fn write_batch<'life0, 'life1, 'async_trait>( &'life0 self, table: &'life1 TableIdent, batch: RecordBatch, ) -> Pin<Box<dyn Future<Output = Result<WriteReceipt, Self::Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Write a batch of Arrow data to the backend’s storage layer. MUST NOT visibly commit until commit is called — for open table formats, this produces staged data files; for StarRocks Stream Load and Flight SQL, this is part of the commit itself.

Source

fn commit<'life0, 'life1, 'async_trait>( &'life0 self, table: &'life1 TableIdent, receipts: Vec<WriteReceipt>, ) -> Pin<Box<dyn Future<Output = Result<CommitOutcome, Self::Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Commit previously written batches. For CAS-based backends (Iceberg, Delta, Paimon), MUST be implemented with a retry loop. For non-transactional backends (StarRocks), MAY be a no-op.

Source

fn get_security_context<'life0, 'life1, 'async_trait>( &'life0 self, table: &'life1 TableIdent, ) -> Pin<Box<dyn Future<Output = Result<SecurityContext, Self::Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Return the security context (credentials, encryption metadata) for the table. Implementations MUST fetch short-lived credentials and MUST NOT cache beyond the credential’s expiry.

Implementors§