pomdp (version 0.9.2)

solve_POMDP: Solve a POMDP Problem

Description

This function utilizes the 'pomdp-solve' program (written in C) to use different solution methods [2] to solve problems that are formulated as partially observable Markov decision processes (POMDPs) [1]. The result is a (close to) optimal policy.

Usage

solve_POMDP(model, horizon = NULL, method = "grid", parameter= NULL, verbose = FALSE)
solve_POMDP_parameter()

Arguments

model

a POMDP problem specification created with POMDP. Alternatively, a POMDP file or the URL for a POMDP file can be specified.

method

string; one of the following solution methods: "grid", "enum", "twopass", "witness", or "incprune". Details can be found in [1].

horizon

an integer with the number of iterations for finite horizon problems. If set to NULL, the algorithm continues running iterations till it converges to the infinite horizon solution.

parameter

a list with parameters passed on to the pomdp-solve program.

verbose

logical, if set to TRUE, the function provides the output of the pomdp solver in the R console.

Value

The solver returns an object of class POMDP which is a list with the model specifications (model), the solution (solution), and the solver output (solver_output). The elements can be extracted with the functions model, solution, and solver_output.

Details

solve_POMDP_parameter() displays available solver parameter options.

Note: The parser for POMDP files is experimental. Please report problems here: https://github.com/farzad/pomdp/issues.

References

[1] For further details on how the POMDP solver utilized in this R package works check the following website: http://www.pomdp.org

[2] Cassandra, A. Rocco, Exact and approximate algorithms for partially observable Markov decision processes, (1998). https://dl.acm.org/citation.cfm?id=926710

Examples

# NOT RUN {
data("TigerProblem")
TigerProblem

tiger_solved <- solve_POMDP(model = TigerProblem, parameter = list(fg_points = 10))
tiger_solved

## look at the model
model(tiger_solved)

## look at the solution
solution(tiger_solved)

## look at solver output
solver_output(tiger_solved)

## plot the policy graph
plot(tiger_solved)

## display available solver options which can be passed on to the solver as parameter.
solve_POMDP_parameter()

## solve a POMDP from http://www.pomdp.org/examples
sol <- solve_POMDP("http://www.pomdp.org/examples/cheese.95.POMDP")
sol
plot(sol)
# }