polars.LazyFrame.profile#

LazyFrame.profile(type_coercion: bool = True, predicate_pushdown: bool = True, projection_pushdown: bool = True, simplify_expression: bool = True, no_optimization: bool = False, slice_pushdown: bool = True, common_subplan_elimination: bool = True, show_plot: bool = False, truncate_nodes: int = 0, figsize: tuple[int, int] = (18, 8), allow_streaming: bool = False) tuple[DataFrame, DataFrame][source]#

Profile a LazyFrame.

This will run the query and return a tuple containing the materialized DataFrame and a DataFrame that contains profiling information of each node that is executed.

The units of the timings are microseconds.

Parameters:
type_coercion

Do type coercion optimization.

predicate_pushdown

Do predicate pushdown optimization.

projection_pushdown

Do projection pushdown optimization.

simplify_expression

Run simplify expressions optimization.

no_optimization

Turn off (certain) optimizations.

slice_pushdown

Slice pushdown optimization.

common_subplan_elimination

Will try to cache branching subplans that occur on self-joins or unions.

show_plot

Show a gantt chart of the profiling result

truncate_nodes

Truncate the label lengths in the gantt chart to this number of characters.

figsize

matplotlib figsize of the profiling plot

allow_streaming

Run parts of the query in a streaming fashion (this is in an alpha state)

Returns:
DataFrame

Examples

>>> df = pl.DataFrame(
...     {
...         "a": ["a", "b", "a", "b", "b", "c"],
...         "b": [1, 2, 3, 4, 5, 6],
...         "c": [6, 5, 4, 3, 2, 1],
...     }
... ).lazy()
>>> df.groupby("a", maintain_order=True).agg(pl.all().sum()).sort(
...     "a"
... ).profile()  
(shape: (3, 3)
 ┌─────┬─────┬─────┐
 │ a   ┆ b   ┆ c   │
 │ --- ┆ --- ┆ --- │
 │ str ┆ i64 ┆ i64 │
 ╞═════╪═════╪═════╡
 │ a   ┆ 4   ┆ 10  │
 ├╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌┤
 │ b   ┆ 11  ┆ 10  │
 ├╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌┤
 │ c   ┆ 6   ┆ 1   │
 └─────┴─────┴─────┘,
 shape: (3, 3)
 ┌────────────────────────┬───────┬──────┐
 │ node                   ┆ start ┆ end  │
 │ ---                    ┆ ---   ┆ ---  │
 │ str                    ┆ u64   ┆ u64  │
 ╞════════════════════════╪═══════╪══════╡
 │ optimization           ┆ 0     ┆ 5    │
 ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌┤
 │ groupby_partitioned(a) ┆ 5     ┆ 470  │
 ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌┤
 │ sort(a)                ┆ 475   ┆ 1964 │
 └────────────────────────┴───────┴──────┘)