Loading or writing Parquet files is lightning fast. Pandas uses PyArrow -Python bindings exposed by Arrow- to load Parquet files into memory, but it has to copy that data into Pandas memory. With Polars there is no extra cost due to copying as we read Parquet directly into Arrow memory and keep it there.


df = pl.read_parquet("path.parquet")

let mut file = std::fs::File::open("path.parquet").unwrap();

let df = ParquetReader::new(&mut file).finish().unwrap();


df = pl.DataFrame({"foo": [1, 2, 3], "bar": [None, "bak", "baz"]})
let mut df = df!(
    "foo" => &[1, 2, 3],
    "bar" => &[None, Some("bak"), Some("baz")],

let mut file = std::fs::File::create("path.parquet").unwrap();
ParquetWriter::new(&mut file).finish(&mut df).unwrap();


Polars allows you to scan a Parquet input. Scanning delays the actual parsing of the file and instead returns a lazy computation holder called a LazyFrame.

df = pl.scan_parquet("path.parquet")
use polars::prelude::*;

let args = ScanArgsParquet::default();
let df = LazyFrame::scan_parquet("./file.parquet",args).unwrap();

If you want to know why this is desirable, you can read more about those Polars optimizations here.

Note about Rust usage

Parquet functionality is not enabled by default. It must be added as an additional feature. This can be enabled via cargo add polars --features parquet or by directly adding it to your Cargo.toml

polars = { version = "0.24.3", features = ["parquet"] }

Additionally, scanning of parquet files requires the lazy feature

polars = { version = "0.24.3", features = ["parquet", "lazy"] }