polars.DataFrame.write_delta#
- DataFrame.write_delta(target: str | Path | deltalake.DeltaTable, *, mode: Literal['error', 'append', 'overwrite', 'ignore'] = 'error', overwrite_schema: bool = False, storage_options: dict[str, str] | None = None, delta_write_options: dict[str, Any] | None = None) None [source]#
Write DataFrame as delta table.
Note: Some polars data types like Null, Categorical and Time are not supported by the delta protocol specification.
- Parameters:
- target
URI of a table or a DeltaTable object.
- mode{‘error’, ‘append’, ‘overwrite’, ‘ignore’}
How to handle existing data.
If ‘error’, throw an error if the table already exists (default).
If ‘append’, will add new data.
If ‘overwrite’, will replace table with new data.
If ‘ignore’, will not write anything if table already exists.
- overwrite_schema
If True, allows updating the schema of the table.
- storage_options
Extra options for the storage backends supported by deltalake. For cloud storages, this may include configurations for authentication etc.
- delta_write_options
Additional keyword arguments while writing a Delta lake Table. See a list of supported write options here.
Examples
Instantiate a basic dataframe:
>>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3, 4, 5], ... "bar": [6, 7, 8, 9, 10], ... "ham": ["a", "b", "c", "d", "e"], ... } ... )
Write DataFrame as a Delta Lake table on local filesystem.
>>> table_path = "/path/to/delta-table/" >>> df.write_delta(table_path)
Append data to an existing Delta Lake table on local filesystem. Note: This will fail if schema of the new data does not match the schema of existing table.
>>> df.write_delta(table_path, mode="append")
Overwrite a Delta Lake table as a new version. Note: If the schema of the new and old data is same, then setting overwrite_schema is not required.
>>> existing_table_path = "/path/to/delta-table/" >>> df.write_delta( ... existing_table_path, mode="overwrite", overwrite_schema=True ... )
Write DataFrame as a Delta Lake table on cloud object store like S3.
>>> table_path = "s3://bucket/prefix/to/delta-table/" >>> df.write_delta( ... table_path, ... storage_options={ ... "AWS_REGION": "THE_AWS_REGION", ... "AWS_ACCESS_KEY_ID": "THE_AWS_ACCESS_KEY_ID", ... "AWS_SECRET_ACCESS_KEY": "THE_AWS_SECRET_ACCESS_KEY", ... }, ... )