bigleaf (version 0.7.0)

filter.data: Basic Eddy Covariance Data Filtering

Description

Filters time series of EC data for high-quality values and specified meteorological conditions.

Usage

filter.data(data, quality.control = TRUE, filter.growseas = FALSE,
  filter.precip = FALSE, filter.vars = NULL, filter.vals.min,
  filter.vals.max, NA.as.invalid = TRUE, vars.qc = NULL,
  quality.ext = "_qc", good.quality = c(0, 1),
  missing.qc.as.bad = TRUE, GPP = "GPP", doy = "doy",
  year = "year", tGPP = 0.5, ws = 15, min.int = 5,
  precip = "precip", tprecip = 0.01, precip.hours = 24,
  records.per.hour = 2, filtered.data.to.NA = TRUE,
  constants = bigleaf.constants())

Arguments

data

Data.frame or matrix containing all required input variables in half-hourly or hourly resolution. Including year, month, day information

quality.control

Should quality control be applied? Defaults to TRUE.

filter.growseas

Should data be filtered for growing season? Defaults to FALSE.

filter.precip

Should precipitation filtering be applied? Defaults to FALSE.

filter.vars

Additional variables to be filtered. Vector of type character.

filter.vals.min

Minimum values of the variables to be filtered. Numeric vector of the same length than filter.vars. Set to NA to be ignored.

filter.vals.max

Maximum values of the variables to be filtered. Numeric vector of the same length than filter.vars. Set to NA to be ignored.

NA.as.invalid

If TRUE (the default) missing data are filtered out (applies to all variables).

vars.qc

Character vector indicating the variables for which quality filter should be applied. Ignored if quality.control = FALSE.

quality.ext

The extension to the variables' names that marks them as quality control variables. Ignored if quality.control = FALSE.

good.quality

Which values indicate good quality (i.e. not to be filtered) in the quality control (qc) variables? Ignored if quality.control = FALSE.

missing.qc.as.bad

If quality control variable is NA, should the corresponding data point be treated as bad quality? Defaults to TRUE. Ignored if quality.control = FALSE.

GPP

Gross primary productivity (umol m-2 s-1); Ignored if filter.growseas = FALSE.

doy

Day of year; Ignored if filter.growseas = FALSE.

year

Year; Ignored if filter.growseas = FALSE.

tGPP

GPP threshold (fraction of 95th percentile of the GPP time series). Must be between 0 and 1. Ignored if filter.growseas is FALSE.

ws

Window size used for GPP time series smoothing. Ignored if filter.growseas = FALSE.

min.int

Minimum time interval in days for a given state of growing season. Ignored if filter.growseas = FALSE.

precip

Precipitation (mm time-1)

tprecip

Precipitation threshold used to identify a precipitation event (mm). Ignored if filter.precip = FALSE.

precip.hours

Number of hours removed following a precipitation event (h). Ignored if filter.precip = FALSE.

records.per.hour

Number of observations per hour. I.e. 2 for half-hourly data.

filtered.data.to.NA

Logical. If TRUE (the default), all variables in the input data.frame/matrix are set to NA for the time step where ANY of the filter.vars were beyond their acceptable range (as determined by filter.vals.min and filter.vals.max). If FALSE, values are not filtered, and an additional column 'valid' is added to the data.frame/matrix, indicating if any value of a row did (1) or did not fulfill the filter criteria (0).

constants

frac2percent - conversion between fraction and percent

Value

If filtered.data.to.NA = TRUE (default), the input data.frame/matrix with observations which did not fulfill the filter criteria set to NA. If filtered.data.to.NA = FALSE, the input data.frame/matrix with an additional column "valid", which indicates whether all the data of a time step fulfill the filtering criteria (1) or not (0).

Details

This routine consists of two parts:

1) Quality control: All variables included in vars.qc are filtered for good quality data. For these variables a corresponding quality variable with the same name as the variable plus the extension as specified in quality.ext must be provided. For time steps where the value of the quality indicator is not included in the argument good.quality, i.e. the quality is not considered as 'good', its value is set to NA.

2) Meteorological filtering. Under certain conditions (e.g. low ustar), the assumptions of the EC method are not fulfilled. Further, some data analysis require certain meteorological conditions, such as periods without rainfall, or active vegetation (growing season, daytime). The filter applied in this second step serves to exclude time periods that do not fulfill the criteria specified in the arguments. More specifically, time periods where one of the variables is higher or lower than the specified thresholds (filter.vals.min and filter.vals.max) are set to NA for all variables. If a threshold is set to NA, it will be ignored.

Examples

# NOT RUN {
# Example of data filtering; data are for a month within the growing season,
# hence growing season is not filtered.
# If filtered.data.to.NA=TRUE, all values of a row are set to NA if one filter
# variable is beyond its bounds. 
DE_Tha_Jun_2014_2 <- filter.data(DE_Tha_Jun_2014,quality.control=FALSE,
                                 vars.qc=c("Tair","precip","H","LE"),
                                 filter.growseas=FALSE,filter.precip=TRUE,
                                 filter.vars=c("Tair","PPFD","ustar"),
                                 filter.vals.min=c(5,200,0.2),
                                 filter.vals.max=c(NA,NA,NA),NA.as.invalid=TRUE,
                                 quality.ext="_qc",good.quality=c(0,1),
                                 missing.qc.as.bad=TRUE,GPP="GPP",doy="doy",
                                 year="year",tGPP=0.5,ws=15,min.int=5,precip="precip",
                                 tprecip=0.1,precip.hours=24,records.per.hour=2,
                                 filtered.data.to.NA=TRUE)

 ## same, but with filtered.data.to.NA=FALSE
 DE_Tha_Jun_2014_3 <- filter.data(DE_Tha_Jun_2014,quality.control=FALSE,
                                 vars.qc=c("Tair","precip","H","LE"),
                                 filter.growseas=FALSE,filter.precip=TRUE,
                                 filter.vars=c("Tair","PPFD","ustar"),
                                 filter.vals.min=c(5,200,0.2),
                                 filter.vals.max=c(NA,NA,NA),NA.as.invalid=TRUE,
                                 quality.ext="_qc",good.quality=c(0,1),
                                 missing.qc.as.bad=TRUE,GPP="GPP",doy="doy",
                                 year="year",tGPP=0.5,ws=15,min.int=5,precip="precip",
                                 tprecip=0.1,precip.hours=24,records.per.hour=2,
                                 filtered.data.to.NA=FALSE)
                                 
 # note the additional column 'valid' in DE_Tha_Jun_2014_3.
 # To remove time steps marked as filtered out (i.e. 0 values in column 'valid'):
 DE_Tha_Jun_2014_3[DE_Tha_Jun_2014_3["valid"] == 0,] <- NA
  
  
# }