Skip to content

Construct a LazyFrame which scans lines into a string column from a file

Source code

Description

[Experimental]

Usage

pl$scan_lines(
  source,
  ...,
  name = "lines",
  n_rows = NULL,
  row_index_name = NULL,
  row_index_offset = 0L,
  glob = TRUE,
  storage_options = NULL,
  include_file_paths = NULL
)

Arguments

source Path(s) to a file or directory. When needing to authenticate for scanning cloud locations, see the storage_options parameter.
… These dots are for future extensions and must be empty.
name Name to use for the output column.
n_rows Stop reading from the source after reading n_rows.
row_index_name If not NULL, this will insert a row index column with the given name.
row_index_offset Offset to start the row index column (only used if the name is set by row_index_name).
glob Expand path given via globbing rules.
storage_options Named vector containing options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here:
  • aws
  • gcp
  • azure
  • Hugging Face (hf://): Accepts an API key under the token parameter c(token = YOUR_TOKEN) or by setting the HF_TOKEN environment variable.
If storage_options is not provided, Polars will try to infer the information from environment variables.
include_file_paths Include the path of the source file(s) as a column with this name.

Value

A polars LazyFrame

Examples

library("polars")


dest <- withr::local_tempfile()
writeLines("Hello\nworld", dest)
pl$scan_lines(dest)$collect()

#> shape: (2, 1)
#> ┌───────┐
#> │ lines │
#> │ ---   │
#> │ str   │
#> ╞═══════╡
#> │ Hello │
#> │ world │
#> └───────┘