polle (version 1.6.2)

history: Get History Object

Description

get_history summarizes the history and action at a given stage from a policy_data object.

Usage

get_history(object, stage = NULL, full_history = FALSE, event_set = c(0))

Value

Object of class history. The object is a list containing the following elements:

H

data.table::data.table with keys id and stage and with variables {\(B\), \(X_k\)} (state history) or {\(B\), \(X_1\), \(A_1\), ..., \(X_k\)} (full history), see details.

A

data.table::data.table with keys id and stage and variable \(A_k\), see details.

action_name

Name of the action variable in A.

action_set

Sorted character vector defining the action set.

U

(If stage is not NULL) data.table::data.table with keys id and stage and with variables U_bar and U_Aa for every a in the actions set. U_bar is the accumulated rewards up till and including the given stage, i.e., \(\sum_{j=1}^k U_j\). U_Aa is the deterministic reward of action a.

Arguments

object

Object of class policy_data.

stage

Stage number. If NULL, the state/Markov-type history across all stages is returned.

full_history

Logical. If TRUE, the full history is returned If FALSE, only the state/Markov-type history is returned.

event_set

Integer vector. Subset of c(0,1,2).

Details

Each observation has the sequential form $$O= {B, U_1, X_1, A_1, ..., U_K, X_K, A_K, U_{K+1}},$$ for a possibly stochastic number of stages K.

• \(B\) is a vector of baseline covariates.

• \(U_k\) is the reward at stage k (not influenced by the action \(A_k\)).

• \(X_k\) is a vector of state covariates summarizing the state at stage k.

• \(A_k\) is the categorical action at stage k.

Examples

library("polle")
### Single stage:
d1 <- sim_single_stage(5e2, seed=1)
# constructing policy_data object:
pd1 <- policy_data(d1, action="A", covariates=list("Z", "B", "L"), utility="U")
pd1

# In the single stage case, set stage = NULL
h1 <- get_history(pd1)
head(h1$H)
head(h1$A)

### Two stages:
d2 <- sim_two_stage(5e2, seed=1)
# constructing policy_data object:
pd2 <- policy_data(d2,
                  action = c("A_1", "A_2"),
                  baseline = c("B"),
                  covariates = list(L = c("L_1", "L_2"),
                                    C = c("C_1", "C_2")),
                  utility = c("U_1", "U_2", "U_3"))
pd2
# getting the state/Markov-type history across all stages:
h2 <- get_history(pd2)
head(h2$H)
head(h2$A)

# getting the full history at stage 2:
h2 <- get_history(pd2, stage = 2, full_history = TRUE)
head(h2$H)
head(h2$A)
head(h2$U)

# getting the state/Markov-type history at stage 2:
h2 <- get_history(pd2, stage = 2, full_history = FALSE)
head(h2$H)
head(h2$A)

### Multiple stages
d3 <- sim_multi_stage(5e2, seed = 1)
# constructing policy_data object:
pd3 <- policy_data(data = d3$stage_data,
                   baseline_data = d3$baseline_data,
                   type = "long",
                   id = "id",
                   stage = "stage",
                   event = "event",
                   action = "A",
                   utility = "U")
pd3

# getting the full history at stage 2:
h3 <- get_history(pd3, stage = 2, full_history = TRUE)
head(h3$H)
# note that not all observations have two stages:
nrow(h3$H) # number of observations with two stages.
get_n(pd3) # number of observations in total.