Function scanCSV

Parameters

• path: string

  path to a file

• Optionaloptions: Partial<ScanCsvOptions>

  • hasHeader

    Indicate if first row of dataset is header or not. If set to False first row will be set to column_x, x being an enumeration over every column in the dataset.

  • sep

    Character to use as delimiter in the file.

  • commentPrefix

    character that indicates the start of a comment line, for instance '#'.

  • quoteChar

    character that is used for csv quoting. Default: '"'. Set to null to turn special handling and escaping of quotes off.

  • skipRows

    Start reading after skipRows position.

  • skipLines

    Start reading after skipLines lines. The header will be parsed at this offset. Note that CSV records may span multiple lines, so this skips lines rather than records.

  • nullValues

    Values to interpret as null values. You can provide a - string -> all values encountered equal to this string will be null - Array<string> -> A null value per column. - Record<string,string> -> An object or map that maps column name to a null value string.Ex. {"column_1": 0}

  • ignoreErrors

    Try to keep reading lines if some lines yield errors.

  • cache

    Cache the result after reading.

  • inferSchemaLength

    Maximum number of lines to read to infer schema. If set to 0, all columns will be read as pl.Utf8. If set to null, a full table scan will be done (slow).

  • nRows

    After n rows are read from the CSV, it stops reading. During multi-threaded parsing, an upper bound of n rows cannot be guaranteed.

  • nThreads

    Number of threads to use in csv parsing. Defaults to the number of physical cpu's of your system.

  • rechunk

    Make sure that all columns are contiguous in memory by aggregating the chunks into a single array.

  • lowMemory

    Reduce memory usage in expense of performance.

  • rowIndexName

    Insert a row index column with this name.

  • rowIndexOffset

    Start the row index at this offset. Only used when rowIndexName is set.

  • decimalComma

    Parse floats using a comma as the decimal separator instead of a period.

  • glob

    Expand path given via globbing rules.

  • schema

    Set the CSV file's schema. This only accepts datatypes that are implemented in the csv parser and expects a complete Schema.

  • cloudOptions

    Options that indicate how to connect to a cloud provider. Also accepts the retry keys max_retries, retry_timeout_ms, retry_init_backoff_ms, retry_max_backoff_ms, retry_base_multiplier, and file_cache_ttl.

  • includeFilePaths

    Include the path of the source file(s) as a column with this name.

  • missingColumns

    How to handle columns of the schema that are missing from a file: "insert" fills them with nulls, "raise" (default) raises an error.

    ---