mc_read_files: Reading files or directories

Description

This function read one or more CSV/TXT files or directories of identical, pre-defined logger type (format) see mc_DataFormat and mc_data_formats. This function does not support loading locality or sensor metadata while reading. Metadata can be loaded through mc_read_data() or can be provided later with function mc_prep_meta_locality()

Usage

mc_read_files(
  paths,
  dataformat_name,
  logger_type = NA_character_,
  recursive = TRUE,
  date_format = NA_character_,
  tz_offset = NA_integer_,
  step = NA_integer_,
  clean = TRUE,
  silent = FALSE,
  user_data_formats = NULL
)

Value

myClim object in Raw-format see myClim-package

Arguments

paths

vector of paths to files or directories

dataformat_name

data format of logger; one of names(mc_data_formats)

logger_type

type of logger (default NA), can be one of pre-defined see mc_read_data() or any custom string

recursive

recursive search in sub-directories (default TRUE)

date_format

format of date in your hobo files e.g. "%d.%m.%y %H:%M:%S" (default NA). TOMST TMS files used to have stable date format, therefore this parameter may be omitted for TMS files because myClim will try to detect one of formerly stable formats, but nowadays user can adjust any date format also for TMS. For other loggers this parameter is required. You can provide multiple formats to by tried, multiple formats can be combined for reading single file. e.g. c("%d.%m.%Y %H:%M:%S", "%Y.%m.%d %H:%M", "%d.%m.%Y")

tz_offset

timezone offset in minutes; It is required only for non-UTC data (custom settings in HOBO). Not used in TMS (default NA)

step

time step of microclimatic time-series in seconds. When provided, then is used in mc_prep_clean instead of automatic step detection. See details. If not provided (NA), is automatically detected in mc_prep_clean. (default NA)

clean

if TRUE, then mc_prep_clean is called automatically while reading (default TRUE)

silent

if TRUE, then any information is not printed in console (default FALSE)

user_data_formats

custom data formats; use in case you have your own logger files not pre-defined in myClim - list(key=mc_DataFormat) mc_DataFormat (default NULL)

If custom data format is defined the key can be used in data_format parameter in mc_read_files() and mc_read_data(). Custom data format must be defined first, and then an be used for reading.

Details

If file is not in expected format, then file is skipped and warning printed in console. CSV/TXT files (loggers raw data) are in resulting myClim object placed to separate localities with empty metadata. Localities are named after serial_number of logger. Pre-defined logger types are ("Dendro","HOBO","Thermo","TMS","TMS_L45")

By default, data are cleaned with the function mc_prep_clean see function description. mc_prep_clean detects gaps in time-series data, duplicated records, or records in the wrong order. Importantly, mc_prep_clean also applies a step parameter if provided. The step parameter can be used either instead of automatic step detection which can sometime failed, or to prune microclimatic data. For example, if you have a 15-minute time series but you wish to keep only one record per hour (without aggregating), you can use step parameter. However, if a step is provided and clean = FALSE, then the step is only stored in the metadata of myClim, and the time-series data is not cleaned, and the step is not applied.

It is good to specify date_formatas this can often be the reason why reading have failed (see warnings after reading).

Examples

Run this code

files <- c(system.file("extdata", "data_91184101_0.csv", package = "myClim"),
           system.file("extdata", "data_94184102_0.csv", package = "myClim"))
tomst_data <- mc_read_files(files, "TOMST", 
                            date_format = c("%d.%m.%Y %H:%M:%S", 
                                            "%Y.%m.%d %H:%M", 
                                            "%d.%m.%Y"))

# user_data_formats
files <- system.file("extdata", "TMS94184102.csv", package = "myClim")
user_data_formats <- list(my_logger=new("mc_DataFormat"))
user_data_formats$my_logger@date_column <- 2
user_data_formats$my_logger@date_format <- "%Y-%m-%d %H:%M:%S"
user_data_formats$my_logger@tz_offset <- 0
user_data_formats$my_logger@columns[[mc_const_SENSOR_T_C]] <- c(3, 4, 5)
user_data_formats$my_logger@columns[[mc_const_SENSOR_real]] <- 6
my_data <- mc_read_files(files, "my_logger", silent=TRUE, user_data_formats=user_data_formats)