pomdp (version 1.0.2)

plot_belief_space: Plot a 2D or 3D Projection of the Belief Space

Description

Plots the optimal action, the node in the policy graph or the reward for a given set of belief points on a line (2D) or on a ternary plot (3D). If no points are given, points are sampled using a regular arrangement or randomly from the (projected) belief space.

Usage

plot_belief_space(
  model,
  projection = NULL,
  epoch = 1,
  sample = "regular",
  n = 100,
  what = c("action", "pg_node", "reward"),
  legend = TRUE,
  pch = 20,
  col = NULL,
  jitter = 0,
  ...
)

Arguments

model

a solved POMDP.

projection

a vector with state IDs or names to project on. Allowed are projections on two or three states. NULL uses the first two or three states. All other states are held at a belief of 0 (seesample_belief_space())

epoch

display this epoch.

sample

a matrix with belief points as rows or a character string specifying the method used for sample_belief_space().

n

number of points sampled.

what

what to plot.

legend

logical; add a legend? If the legend is covered by the plot then you need to increase the plotting region of the plotting device.

pch

plotting symbols.

col

plotting colors.

jitter

y jitter amount for 2D belief spaces (good values are between 0 and 4).

...

additional arguments are passed on to plot for 2D or TerneryPlot for 3D.

Value

Returns invisibly the sampled points.

See Also

Other POMDP: POMDP(), sample_belief_space(), simulate_POMDP(), solve_POMDP(), solve_SARSOP(), transition_matrix(), update_belief(), write_POMDP()

Examples

# NOT RUN {
# two-state POMDP
data("Tiger")
sol <- solve_POMDP(Tiger)

plot_belief_space(sol)
plot_belief_space(sol, n = 10)
plot_belief_space(sol, n = 10, sample = "random")

# plot the belief points used by the grid-based solver
plot_belief_space(sol, sample = sol$solution$belief_states)

# plot different measures
plot_belief_space(sol, what = "pg_node")
plot_belief_space(sol, what = "reward")

# three-state POMDP
# Note: If the plotting region is too small then the legend might run into the plot
data("Three_doors")
sol <- solve_POMDP(Three_doors)
sol

plot_belief_space(sol)
plot_belief_space(sol, sample = "random", n = 1000)
plot_belief_space(sol, what = "pg_node")
plot_belief_space(sol, what = "reward", sample = "random", n = 1000)

# plot the belief points used by the grid-based solver
plot_belief_space(sol, sample = sol$solution$belief_states)

# plot the belief points obtained using simulated trajectories with an epsilon-greedy policy.
# Note that we only use n = 50 to save time.
plot_belief_space(sol, sample = simulate_POMDP(sol, n = 50, horizon = 100,
  epsilon = 0.1, visited_beliefs = TRUE))
  
# plot a 3-state belief space using ggtern (ggplot2)
# library(ggtern)
# samp <- sample_belief_space(sol, n = 1000)
# df <- cbind(as.data.frame(samp), reward = reward(sol, belief = samp))
#
# ggtern(df, aes(x = `tiger-left`, y = `tiger-center`, z = `tiger-right`)) + 
#   geom_point(aes(color = reward))
# }