read_file_duckdb: Read files using DuckDB
Description
read_file_duckdb() uses arbitrary readers to read data.
See https://duckdb.org/docs/data/overview for a documentation
of the available functions and their options.
To read multiple files with the same schema,
pass a wildcard or a character vector to the path argument,
Usage
read_file_duckdb(
path,
table_function,
...,
prudence = c("thrifty", "lavish", "stingy"),
options = list()
)Value
A duckplyr frame, see as_duckdb_tibble() for details.
Arguments
- path
Path to files, glob patterns
*and?are supported.- table_function
The name of a table-valued DuckDB function such as
"read_parquet","read_csv","read_csv_auto"or"read_json".- ...
These dots are for future extensions and must be empty.
- prudence
Memory protection, controls if DuckDB may convert intermediate results in DuckDB-managed memory to data frames in R memory.
"thrifty": up to a maximum size of 1 million cells,"lavish": regardless of size,"stingy": never.
- options
Arguments to the DuckDB function indicated by
table_function.
The default is "thrifty" for the ingestion functions,
and may be different for other functions.
See vignette("prudence") for more information.
Fine-tuning prudence
![[Experimental]](figures/lifecycle-experimental.svg?package=duckplyr&version=1.0.1)
The prudence argument can also be a named numeric vector
with at least one of cells or rows
to limit the cells (values) and rows in the resulting data frame
after automatic materialization.
If both limits are specified, both are enforced.
The equivalent of "thrifty" is c(cells = 1e6).
See Also
read_csv_duckdb(), read_parquet_duckdb(), read_json_duckdb()