Skip to content

Combining DataFrames

There are two ways DataFrames can be combined depending on the use case: join and concat.

Join

Polars supports all types of join (e.g. left, right, inner, outer). Let's have a closer look on how to join two DataFrames into a single DataFrame. Our two DataFrames both have an 'id'-like column: a and x. We can use those columns to join the DataFrames in this example.

join

df = pl.DataFrame(
    {
        "a": np.arange(0, 8),
        "b": np.random.rand(8),
        "d": [1, 2.0, np.NaN, np.NaN, 0, -5, -42, None],
    }
)

df2 = pl.DataFrame(
    {
        "x": np.arange(0, 8),
        "y": ["A", "A", "A", "B", "B", "C", "X", "X"],
    }
)
joined = df.join(df2, left_on="a", right_on="x")
print(joined)

join

use rand::Rng;
let mut rng = rand::thread_rng();

let df: DataFrame = df!("a" => 0..8,
                        "b"=> (0..8).map(|_| rng.gen::<f64>()).collect::<Vec<f64>>(),
                        "d"=> [Some(1.0), Some(2.0), None, None, Some(0.0), Some(-5.0), Some(-42.), None]
                    ).expect("should not fail");
let df2: DataFrame = df!("x" => 0..8,
                        "y"=> &["A", "A", "A", "B", "B", "C", "X", "X"],
                    ).expect("should not fail");
let joined = df.join(&df2,["a"],["x"],JoinType::Left,None)?;
println!("{}",joined);

join

df = pl.DataFrame({
  a: [...Array(8).keys()],
  b: Array.from({ length: 8 }, () => Math.random()),
  d: [1, 2.0, null, null, 0, -5, -42, null],
});

df2 = pl.DataFrame({
  x: [...Array(8).keys()],
  y: ["A", "A", "A", "B", "B", "C", "X", "X"],
});
joined = df.join(df2, { leftOn: "a", rightOn: "x" });
console.log(joined);

shape: (8, 4)
┌─────┬──────────┬───────┬─────┐
│ a   ┆ b        ┆ d     ┆ y   │
│ --- ┆ ---      ┆ ---   ┆ --- │
│ i64 ┆ f64      ┆ f64   ┆ str │
╞═════╪══════════╪═══════╪═════╡
│ 0   ┆ 0.940413 ┆ 1.0   ┆ A   │
│ 1   ┆ 0.791234 ┆ 2.0   ┆ A   │
│ 2   ┆ 0.634507 ┆ NaN   ┆ A   │
│ 3   ┆ 0.879961 ┆ NaN   ┆ B   │
│ 4   ┆ 0.973144 ┆ 0.0   ┆ B   │
│ 5   ┆ 0.205376 ┆ -5.0  ┆ C   │
│ 6   ┆ 0.131481 ┆ -42.0 ┆ X   │
│ 7   ┆ 0.385729 ┆ null  ┆ X   │
└─────┴──────────┴───────┴─────┘

To see more examples with other types of joins, go the User Guide.

Concat

We can also concatenate two DataFrames. Vertical concatenation will make the DataFrame longer. Horizontal concatenation will make the DataFrame wider. Below you can see the result of an horizontal concatenation of our two DataFrames.

hstack

stacked = df.hstack(df2)
print(stacked)

hstack

let stacked = df.hstack(df2.get_columns())?;
println!("{}",stacked);

hstack

stacked = df.hstack(df2);
console.log(stacked);

shape: (8, 5)
┌─────┬──────────┬───────┬─────┬─────┐
│ a   ┆ b        ┆ d     ┆ x   ┆ y   │
│ --- ┆ ---      ┆ ---   ┆ --- ┆ --- │
│ i64 ┆ f64      ┆ f64   ┆ i64 ┆ str │
╞═════╪══════════╪═══════╪═════╪═════╡
│ 0   ┆ 0.940413 ┆ 1.0   ┆ 0   ┆ A   │
│ 1   ┆ 0.791234 ┆ 2.0   ┆ 1   ┆ A   │
│ 2   ┆ 0.634507 ┆ NaN   ┆ 2   ┆ A   │
│ 3   ┆ 0.879961 ┆ NaN   ┆ 3   ┆ B   │
│ 4   ┆ 0.973144 ┆ 0.0   ┆ 4   ┆ B   │
│ 5   ┆ 0.205376 ┆ -5.0  ┆ 5   ┆ C   │
│ 6   ┆ 0.131481 ┆ -42.0 ┆ 6   ┆ X   │
│ 7   ┆ 0.385729 ┆ null  ┆ 7   ┆ X   │
└─────┴──────────┴───────┴─────┴─────┘