pomdp (version 1.0.2)

transition_matrix: Extract the Transition, Observation or Reward Information from a POMDP

Description

Converts the description of transition probabilities and observation probabilities in a POMDP into a list of matrices. Individual values or parts of the matrices can be more efficiently retrieved using the functions ending _prob and _val.

Usage

transition_matrix(x, episode = 1, action = NULL)
transition_prob(x, action, start.state, end.state, episode = 1)
observation_matrix(x, episode = 1, action = NULL)
observation_prob(x, action, end.state, observation, episode = 1)
reward_matrix(x, episode = 1, action = NULL, start.state = NULL)
reward_val(x, action, start.state, end.state, observation, episode = 1)

Arguments

x

A POMDP object.

episode

Episode used for time-dependent POMDPs (POMDP).

action

only return the matrix/value for a given action.

start.state, end.state, observation

name of the state or observation.

Value

A list or a list of lists of matrices.

Details

See Details section in POMDP for details.

See Also

Other POMDP: POMDP(), plot_belief_space(), sample_belief_space(), simulate_POMDP(), solve_POMDP(), solve_SARSOP(), update_belief(), write_POMDP()

Examples

# NOT RUN {
data("Tiger")

# List of |A| transition matrices. One per action in the from states x states
Tiger$transition_prob
transition_matrix(Tiger)
transition_prob(Tiger, action = "listen", start.state = "tiger-left")

# List of |A| observation matrices. One per action in the from states x observations
Tiger$observation_prob
observation_matrix(Tiger)
observation_prob(Tiger, action = "listen", end.state = "tiger-left")

# List of list of reward matrices. 1st level is action and second level is the
#  start state in the form end state x observation
Tiger$reward
reward_matrix(Tiger)
reward_val(Tiger, action = "listen", start.state = "tiger")

# Visualize transition matrix for action 'open-left'
library("igraph")
g <- graph_from_adjacency_matrix(transition_matrix(Tiger)$"open-left", weighted = TRUE)
edge_attr(g, "label") <- edge_attr(g, "weight")

igraph.options("edge.curved" = TRUE)
plot(g, layout = layout_on_grid, main = "Transitions for action 'open=left'")

## Use a function for the Tiger transition model
trans <- function(action, end.state, start.state) {
  ## listen has an identity matrix
  if(action == 'listen')
    if(end.state == start.state) return(1)
    else return(0)

  # other actions have a uniform distribution
  return(1/2)
}

Tiger$transition_prob <- trans
transition_matrix(Tiger)
# }