Partial_DT_GDL_Feature_Engineering: A version of the DT_GDL function for creating the GDL features for a new set of records

Description

For scoring models in production that have > 1 grouping variables and for when you need > 1 record (or records per grouping variables) returned. This function is for generating lags and moving averages (along with lags and moving averages off of time between records), for a partial set of records in your data set, typical new records that become available for model scoring. Column names and ordering will be identical to the output from the corresponding DT_GDL_Feature_Engineering() function, which most likely was used to create features for model training.

Usage

Partial_DT_GDL_Feature_Engineering(data, lags = c(seq(1, 5, 1)),
  periods = c(3, 5, 10, 15, 20, 25), statsNames = c("MA"),
  targets = c("Target"), groupingVars = NULL,
  sortDateName = c("DateTime"), timeDiffTarget = c("Time_Gap"),
  timeAgg = "days", WindowingLag = 1, Type = "Lag", Timer = TRUE,
  SimpleImpute = TRUE, AscRowByGroup = "temp", RecordsKeep = 1,
  AscRowRemove = TRUE)

Arguments

data

A data.table you want to run the function on

lags

A numeric vector of the specific lags you want to have generated. You must include 1 if WindowingLag = 1.

periods

A numeric vector of the specific rolling statistics window sizes you want to utilize in the calculations.

statsNames

A character vector of the corresponding names to create for the rollings stats variables.

targets

A character vector of the column names for the reference column in which you will build your lags and rolling stats

groupingVars

A character vector of categorical variable names you will build your lags and rolling stats by

sortDateName

The column name of your date column used to sort events over time

timeDiffTarget

Specify a desired name for features created for time between events. Set to NULL if you don't want time between events features created.

timeAgg

List the time aggregation level for the time between events features, such as "hour", "day", "week", "month", "quarter", or "year"

WindowingLag

Set to 0 to build rolling stats off of target columns directly or set to 1 to build the rolling stats off of the lag-1 target

Type

List either "Lag" if you want features built on historical values or "Lead" if you want features built on future values

Timer

Set to TRUE if you percentage complete tracker printout

SimpleImpute

Set to TRUE for factor level imputation of "0" and numeric imputation of -1

AscRowByGroup

Required to have a column with a Row Number by group (if grouping) with the smallest numbers being the records for scoring (typically the most current in time).

RecordsKeep

List the number of records to retain (1 for last record, 2 for last 2 records, etc.)

AscRowRemove

Set to TRUE to remove the AscRowByGroup column upon returning data.

Value

data.table of original data plus created lags, rolling stats, and time between event lags and rolling stats

Examples

Run this code

# NOT RUN {
N = 25116
data <- data.table::data.table(DateTime = as.Date(Sys.time()),
  Target = stats::filter(rnorm(N,
                               mean = 50,
                               sd = 20),
                         filter=rep(1,10),
                         circular=TRUE))
data[, temp := seq(1:N)][, DateTime := DateTime - temp]
data <- data[order(DateTime)]
data <- Partial_DT_GDL_Feature_Engineering(data,
                                           lags           = c(1:5),
                                           periods        = c(seq(10,50,10)),
                                           statsNames     = "mean",
                                           targets        = c("Target"),
                                           groupingVars   = NULL,
                                           sortDateName   = "DateTime",
                                           timeDiffTarget = c("Time_Gap"),
                                           timeAgg        = "days",
                                           WindowingLag   = 1,
                                           Type           = "Lag",
                                           Timer          = TRUE,
                                           SimpleImpute   = TRUE,
                                           AscRowByGroup  = "temp",
                                           RecordsKeep    = 1,
                                           AscRowRemove   = TRUE)
# }

Run the code above in your browser using DataLab

Last chance! 50% off unlimited learning