Trait polars_lazy::frame::LazyFileListReader

pub trait LazyFileListReader: Clone {
    // Required methods
    fn finish_no_glob(self) -> PolarsResult<LazyFrame>;
    fn path(&self) -> &Path;
    fn paths(&self) -> &[PathBuf];
    fn with_path(self, path: PathBuf) -> Self;
    fn with_paths(self, paths: Arc<[PathBuf]>) -> Self;
    fn with_n_rows(self, n_rows: impl Into<Option<usize>>) -> Self;
    fn with_row_index(self, row_index: impl Into<Option<RowIndex>>) -> Self;
    fn rechunk(&self) -> bool;
    fn with_rechunk(self, toggle: bool) -> Self;
    fn n_rows(&self) -> Option<usize>;
    fn row_index(&self) -> Option<&RowIndex>;

    // Provided methods
    fn finish(self) -> PolarsResult<LazyFrame> { ... }
    fn concat_impl(&self, lfs: Vec<LazyFrame>) -> PolarsResult<LazyFrame> { ... }
    fn glob(&self) -> bool { ... }
    fn cloud_options(&self) -> Option<&CloudOptions> { ... }
    fn iter_paths(&self) -> PolarsResult<Option<PathIterator>> { ... }
}

Reads LazyFrame from a filesystem or a cloud storage. Supports glob patterns.

Use LazyFileListReader::finish to get the final LazyFrame.

Required Methods

fn finish_no_glob(self) -> PolarsResult<LazyFrame>

Get the final LazyFrame. This method assumes, that path is not a glob.

It is recommended to always use LazyFileListReader::finish method.

fn path(&self) -> &Path

Path of the scanned file. It can be potentially a glob pattern.

fn paths(&self) -> &[PathBuf]

fn with_path(self, path: PathBuf) -> Self

Set path of the scanned file. Support glob patterns.

fn with_paths(self, paths: Arc<[PathBuf]>) -> Self

Set paths of the scanned files. Doesn’t glob patterns.

fn with_n_rows(self, n_rows: impl Into<Option<usize>>) -> Self

Configure the row limit.

fn with_row_index(self, row_index: impl Into<Option<RowIndex>>) -> Self

Configure the row index.

fn rechunk(&self) -> bool

Rechunk the memory to contiguous chunks when parsing is done.

fn with_rechunk(self, toggle: bool) -> Self

Rechunk the memory to contiguous chunks when parsing is done.

fn n_rows(&self) -> Option<usize>

Try to stop parsing when n rows are parsed. During multithreaded parsing the upper bound n cannot be guaranteed.

fn row_index(&self) -> Option<&RowIndex>

Add a row index column.

Provided Methods

fn finish(self) -> PolarsResult<LazyFrame>

Get the final LazyFrame.

fn concat_impl(&self, lfs: Vec<LazyFrame>) -> PolarsResult<LazyFrame>

Recommended concatenation of LazyFrames from many input files.

This method should not take into consideration LazyFileListReader::n_rows nor LazyFileListReader::row_index.

fn glob(&self) -> bool

fn cloud_options(&self) -> Option<&CloudOptions>

CloudOptions used to list files.

fn iter_paths(&self) -> PolarsResult<Option<PathIterator>>

Get list of files referenced by this reader.

Returns None if path is not a glob pattern.

Object Safety

This trait is not object safe.

Implementors

impl LazyFileListReader for LazyCsvReader

Available on crate feature csv only.

impl LazyFileListReader for LazyJsonLineReader

Available on crate feature json only.