policy_data: Create Policy Data Object

Description

policy_data() creates a policy data object which is used as input to policy_eval() and policy_learn() for policy evaluation and data adaptive policy learning.

Usage

policy_data(
  data,
  baseline_data,
  type = "wide",
  action,
  covariates,
  utility,
  baseline = NULL,
  deterministic_rewards = NULL,
  id = NULL,
  stage = NULL,
  event = NULL,
  time = NULL,
  time2 = NULL,
  action_set = NULL,
  verbose = FALSE
)
# S3 method for policy_data
print(x, digits = 2, ...)
# S3 method for policy_data
summary(object, probs = seq(0, 1, 0.25), ...)

Value

policy_data() returns an object of class "policy_data". The object is a list containing the following elements:

stage_data: data.table::data.table containing the id, stage number, event indicator, action ($A_k$), state covariates ($X_k$), reward ($U_k$), and the deterministic rewards.
baseline_data: data.table::data.table containing the id and baseline covariates ($B$).
colnames: List containing the state covariate names, baseline covariate names, and the deterministic reward variable names.
action_set: Sorted character vector describing the action set, i.e., the possible actions at all stages.
stage_action_sets: List of sorted character vectors describing the observed actions at each stage.
dim: List containing the number of observations (n) and the number of stages (K).

Arguments

data

data.frame or data.table::data.table; see Examples.

baseline_data

data.frame or data.table::data.table; see Examples.

type

Character string. If "wide", data is considered to be on wide format. If "long", data is considered to be on long format; see Examples.

action

Action variable name(s). Character vector or character string.

A vector is valid for wide data. The length of the vector determines the number of stages (K).
A string is valid for single stage wide data or long data.

covariates

Stage specific covariate name(s). Character vector or named list of character vectors.

A vector is valid for single stage wide data or long data.
A named list is valid for multiple stages wide data. Each element must be a character vector with length K. Each vector can contain NA elements, if a covariate is not available for the given stage(s).

utility

Utility/Reward variable name(s). Character string or vector.

A string is valid for long data and wide data with a single final utility.
A vector is valid for wide data with incremental rewards. Must have length K+1; see Examples.

baseline

Baseline covariate name(s). Character vector.

deterministic_rewards

Deterministic reward variable name(s). Named list of character vectors of length K. The name of each element must be on the form "U_Aa" where "a" corresponds to an action in the action set.

id

ID variable name. Character string.

stage

Stage number variable name.

event

Event indicator name.

time

Character string

time2

Character string

action_set

Character string. Action set across all stages.

verbose

Logical. If TRUE, formatting comments are printed to the console.

x

Object to be printed.

digits

Minimum number of digits to be printed.

...

Additional arguments passed to print.

object

Object of class policy_data

probs

numeric vector (probabilities)

S3 generics

The following S3 generic functions are available for an object of class policy_data:

partial(): Trim the maximum number of stages in a policy_data object.
subset_id(): Subset a a policy_data object on ID.
get_history(): Summarize the history and action at a given stage.
get_history_names(): Get history variable names.
get_actions(): Get the action at every stage.
get_utility(): Get the utility.
plot(): Plot method.

Details

Each observation has the sequential form $$O= {B, U_1, X_1, A_1, ..., U_K, X_K, A_K, U_{K+1}},$$ for a possibly stochastic number of stages K.

$B$ is a vector of baseline covariates.
$U_k$ is the reward at stage k (not influenced by the action $A_k$).
$X_k$ is a vector of state covariates summarizing the state at stage k.
$A_k$ is the categorical action at stage k.

The utility is given by the sum of the rewards, i.e., $U = \sum_{k = 1}^{K+1} U_k$.

Examples

Run this code

library("polle")
### Single stage: Wide data
d1 <- sim_single_stage(n = 5e2, seed=1)
head(d1, 5)
# constructing policy_data object:
pd1 <- policy_data(d1,
                   action="A",
                   covariates=c("Z", "B", "L"),
                   utility="U")
pd1
# associated S3 methods:
methods(class = "policy_data")
head(get_actions(pd1), 5)
head(get_utility(pd1), 5)
head(get_history(pd1)$H, 5)

### Two stage: Wide data
d2 <- sim_two_stage(5e2, seed=1)
head(d2, 5)
# constructing policy_data object:
pd2 <- policy_data(d2,
                  action = c("A_1", "A_2"),
                  baseline = c("B"),
                  covariates = list(L = c("L_1", "L_2"),
                                    C = c("C_1", "C_2")),
                  utility = c("U_1", "U_2", "U_3"))
pd2
head(get_history(pd2, stage = 2)$H, 5) # state/Markov type history and action, (H_k,A_k).
head(get_history(pd2, stage = 2, full_history = TRUE)$H, 5) # Full history and action, (H_k,A_k).

### Multiple stages: Long data
d3 <- sim_multi_stage(5e2, seed = 1)
head(d3$stage_data, 10)
# constructing policy_data object:
pd3 <- policy_data(data = d3$stage_data,
                   baseline_data = d3$baseline_data,
                   type = "long",
                   id = "id",
                   stage = "stage",
                   event = "event",
                   action = "A",
                   utility = "U")
pd3
head(get_history(pd3, stage = 3)$H, 5) # state/Markov type history and action, (H_k,A_k).
head(get_history(pd3, stage = 2, full_history = TRUE)$H, 5) # Full history and action, (H_k,A_k).

Run the code above in your browser using DataLab