Struct DiskBuffer 

pub struct DiskBuffer {
    pub max_size_bytes: Option<u64>,
    pub current_size_bytes: u64,
    /* private fields */
}

A persistent buffer that stores Arrow batches on disk.

Fields

max_size_bytes: Option<u64>

current_size_bytes: u64

Implementations

impl DiskBuffer

pub fn new<P: AsRef<Path>>( data_dir: P, component_id: String, max_size_bytes: Option<u64>, ) -> Result<Self>

Creates a new DiskBuffer or loads existing state from disk.

pub fn enqueue(&mut self, batch: &EventBatch) -> Result<()>

Enqueues a batch to disk.

pub fn dequeue(&mut self) -> Result<Option<EventBatch>>

Dequeues the next batch from disk.

pub fn len(&self) -> usize

Returns the number of pending batches on disk.

pub fn is_empty(&self) -> bool