s2dverification (version 2.5.0)

.LoadDataFile: Load Data From File Into Environment

Description

This function receives a 'work piece', a named list which contains information on a data file to be loaded. It can be run in 'explore' mode or in 'load' mode. When running in 'explore' mode, the metadata of the file is read and the sizes of the dimensions in the file are returned in a named list:

• 'member': Number of members
• 'time': Number of lead-times
• 'lon': Longitudes in the file
• 'lat': Latitudes in the file

When running in 'load' mode it loads and performs any requested computations in additional parameters in the 'work piece', such as interpolating, slicing, ..., and finally stores it in a shared memory matrix pointed by the parameter 'out_pointer' in the 'work piece'.

Usage

.LoadDataFile(work_piece, explore_dims = FALSE, silent = FALSE)

Arguments

work_piece

Named list with information on the file to load or explore and additional parameters. The needed variables in the work piece are:

• In explore mode:
  • dataset_type: 'exp'/'obs'
  • filename: full path to the data file
  • dimnames: named list with names 'lon', 'lat' and 'member' and with values the associated actual dimension names inside the NetCDF files.
  • namevar: name of the variable in the nc file
  • grid: common grid into which interpolate or NULL if no interpolation
  • remap: interpolation method ('remapbil'/'remapdis'/'remapcon'/'remapbic')
  • lon_limits: c(lon_min, lon_max)
  • lat_limits: c(lat_min, lat_max)
  • is_file_per_member: TRUE/FALSE
  • is_file_per_dataset: TRUE/FALSE
  • is_single_dataset: TRUE/FALSE

• In load mode:
  • dataset_type: 'exp'/'obs'
  • filename: full path to the data file
  • dimnames: named list with names 'lon', 'lat' and 'member' and with values the associated actual dimension names inside the NetCDF files.
  • namevar: name of the variable in the nc file
  • is_2d_var: TRUE/FALSE
  • grid: common grid into which interpolate or NULL if no interpolation
  • remap: interpolation method ('remapbil'/'remapdis'/'remapcon'/'remapbic')
  • lon_limits: c(lon_min, lon_max)
  • lat_limits: c(lat_min, lat_max)
  • is_file_per_dataset: TRUE/FALSE
  • startdates: in the case that filename points to a whole dataset file, the list of starting dates to load must be specified. c('sdate1', 'sdate2', ...)
  • out_pointer: big.matrix descriptor pointing to the array (transformed into a matrix) where to keep the data
  • dims: named list with dimension sizes of the original array into which the data is kept. Names must be c(['lon', ]['lat', ]'time', 'member', 'sdates', 'dataset').
  • indices: vector of initial positions corresponding to each dimension in 'dims' where to store data in the original array. First two indices ('lon', 'lat') can be missing.
  • nmember: number of members expected to be loaded
  • mask: complete (untrimmed + interpolated if needed) mask to activate/deactivate data points, with dimensions c('lon', 'lat'), see ?Load for more details.
  • leadtimes: vector of time indices to be loaded from the file
  • var_limits: c(var_min, var_max)
  • is_single_dataset: TRUE/FALSE (whether the user only asked for data of one single dataset. If so, then data won't be re-interpolated when the first longitude of its grid is != 0. Otherwise it must be re-interpolated to ensure all data will be properly aligned over longitudes.

explore_dims

Run in dimension explore mode (TRUE) or in load and calculation mode (FALSE). Takes by default the value FALSE (calculation mode).

silent

Parameter to allow (FALSE) or deactivate (TRUE) printing of explanatory messages. When deactivated any warning messages will still be displayed. Takes by default the value FALSE (verbose mode).

remapcells

Width in number of cells of the surrounding area of the requested subset to be taken into account for the interpolation. See parameter remapcells of Load().

Value

When called in 'explore' mode, a named list is returned with:

• dims List with the found lengths for members, leadtimes and the latitudes and longitudes already trimmed and reordered if needed. The names are 'member', 'time', 'lon', 'lat'. If it is a file from a file-per-member dataset, the number of files that match the filename replacing the $MEMBER_NUMBER$ part by an asterisk is returned (which is the supposed number of members). There are known issues with this method of detection. See documentation on parameter 'nmember' and 'nmemberobs' in Load() function. When the specified file path is a URL, the returned number of members is NULL.
• is_2d_var Boolean indicating whether the found variable is 2-dimensional (TRUE) or a global mean (FALSE).
• grid Character string with the name of the grid of the file, following the cdo grid naming conventions.
• var_long_name A character string with the variable long name. If not available, the short name is returned.
• units A character string with the units of the variable.

When called in 'calculation' mode, the found file path or URL is returned if the file was found and NULL is returned otherwise.

Examples

  ## Not run: 
# data <- s2dverification:::.LoadDataFile(list(dataset_type = 'exp', 
#         filename = system.file('sample_data/model/experiment/monthly_mean', 
#                                'tos_3hourly/tos_19901101.nc', 
#                                package = 's2dverification'),
#         namevar = 'tos', lon_limits = c(-12, 40), 
#         lat_limits = c(27, 48), is_file_per_member = TRUE, 
#         dimnames = list(lon = 'longitude', lat = 'latitude', 
#         member = 'ensemble')), explore_dims = TRUE, silent = FALSE)
#   ## End(Not run)