R.AlphA.Home (version 2.0.2)

importAll: Function to Import and Concatenate Multiple Data Files

Description

Imports multiple files into a list, concatenates them into a single table, and adds an `fName` variable. The function automatically handles type harmonization when different file types are mixed and supports various formats including CSV, Excel, and RDS files.

The files can be selected either by giving a file list (character vector), or by specifying a pattern. The function also supports column renaming and file exclusion patterns. When type conflicts are detected across files, the function automatically harmonizes column types using a priority system (character > numeric > Date > integer).

Usage

importAll(
  path = ".",
  pattern = "",
  ignore.case = FALSE,
  importFunction = NULL,
  fill = FALSE,
  fileList = NULL,
  renameTable = data.frame(oldName = character(), newName = character()),
  excludePattern = NULL
)

Value

A data.table containing the concatenated table with the fName column indicating the source file for each row. All imported data is converted to data.table format with automatic type harmonization when necessary.

Arguments

path

Character. Path to the directory, passed to `list.files`. Default is current directory (".").

pattern

Character. Pattern to match file names, passed to `list.files`. Default is empty string (all files).

ignore.case

Logical. If `TRUE`, ignores case when matching file names. Passed to `list.files`. Default behavior is case-sensitive (`FALSE`).

importFunction

Function. A custom function for importing files. If not set, the function selects an import method based on the file extension.

fill

Logical. Passed to `rbind` to allow filling missing columns with NA values. Default is `FALSE`.

fileList

Character vector. A vector of file names to import (used instead of `pattern`). Can contain absolute or relative paths.

renameTable

Data.frame. A data.frame with 2 columns (oldName/newName). importAll will rename the columns of each file following this table. Default is empty data.frame.

excludePattern

Character. Pattern to exclude files from import, applied after initial file selection. Default is `NULL` (no exclusion).

Examples

# Directory containing test files
test_path <- tempdir()

# Create test files
write.csv(data.frame(a = 1:3, b = 4:6), file.path(test_path, "file1.csv"))
write.csv(data.frame(a = 7:9, b = 10:12), file.path(test_path, "file2.csv"))
write.csv(data.frame(a = 3:5, b = 8:10), file.path(test_path, "file3.csv"))
saveRDS(data.frame(a = 1:5, b = 6:10), file.path(test_path, "file1.rds"))
saveRDS(data.frame(a = 11:15, b = 16:20), file.path(test_path, "file2.rds"))

# Example 1: Import all csv files
result <- importAll(path = test_path, pattern = "\\.csv$")
print(result)

# Example 2: Import only selected files
file_list <- c("file1.csv", "file2.csv")
result <- importAll(path = test_path, fileList = file_list)
print(result)

# Example 3: Import all .rds files
result <- importAll(path = test_path, pattern = "\\.rds$")
print(result)

# Example 4: Use a custom import function
custom_import <- function(file) {
  data <- read.csv(file, stringsAsFactors = FALSE)
  return(data)
}
result <- importAll(path = test_path, pattern = "\\.csv$", importFunction = custom_import)
print(result)