Input/output

CSV

read_csv(file[, has_header, columns, ...])

Read a CSV file into a Dataframe.

scan_csv(file[, has_header, sep, ...])

Lazily read from a CSV file or multiple files via glob patterns.

DataFrame.write_csv([file, has_header, sep, ...])

Write Dataframe to comma-separated values file (csv).

Feather/ IPC

read_ipc(file[, columns, n_rows, ...])

Read into a DataFrame from Arrow IPC (Feather v2) file.

scan_ipc(file[, n_rows, cache, rechunk, ...])

Lazily read from an Arrow IPC (Feather v2) file or multiple files via glob patterns.

read_ipc_schema(file)

Get a schema of the IPC file without reading data.

DataFrame.write_ipc(file[, compression])

Write to Arrow IPC binary stream, or a feather file.

Parquet

read_parquet(source[, columns, n_rows, ...])

Read into a DataFrame from a parquet file.

scan_parquet(file[, n_rows, cache, ...])

Lazily read from a parquet file or multiple files via glob patterns.

DataFrame.write_parquet(file[, compression, ...])

Write the DataFrame disk in parquet format.

SQL

read_sql(sql, connection_uri[, ...])

Read a SQL query into a DataFrame.

JSON

read_json(source[, json_lines])

Read into a DataFrame from JSON format.

DataFrame.write_json()

Serialize to JSON representation.

AVRO

read_avro(file[, columns, n_rows])

Read into a DataFrame from Apache Avro format.

DataFrame.write_avro(file[, compression])

Write to Apache Avro file.

Datasets

Connect to pyarrow datasets.

scan_ds(ds)