Learn R Programming

MDPtoolbox (version 4.0.4)

Markov Decision Processes Toolbox

Description

The Markov Decision Processes (MDP) toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: finite horizon, value iteration, policy iteration, linear programming algorithms with some variants and also proposes some functions related to Reinforcement Learning.

Copy Link

Version

Install

install.packages('MDPtoolbox')

Monthly Downloads

350

Version

4.0.4

License

BSD_3_clause + file LICENSE

Maintainer

Guillaume Chapron

Last Published

August 31st, 2026

Functions in MDPtoolbox (4.0.4)

mdp_eval_policy_optimality

Computes sets of 'near optimal' actions for each state
mdp_value_iterationGS

Solves discounted MDP using Gauss-Seidel's value iteration algorithm
mdp_value_iteration_bound_iter

Computes a bound for the number of iterations for the value iteration algorithm
mdp_check

Checks the validity of a MDP
mdp_eval_policy_TD_0

Evaluates a policy using the TD(0) algorithm
mdp_Q_learning

Solves discounted MDP using the Q-learning algorithm (Reinforcement Learning)
MDPtoolbox-package

Markov Decision Processes Toolbox
mdp_check_square_stochastic

Checks if a matrix is square and stochastic
mdp_bellman_operator

Applies the Bellman operator
mdp_LP

Solves discounted MDP using linear programming algorithm
mdp_computePR

Computes a reward matrix for any form of transition and reward functions
mdp_eval_policy_iterative

Evaluates a policy using an iterative method
mdp_computePpolicyPRpolicy

Computes the transition matrix and the reward matrix for a fixed policy
mdp_example_rand

Generates a random MDP problem
mdp_relative_value_iteration

Solves MDP with average reward using relative value iteration algorithm
mdp_finite_horizon

Solves finite-horizon MDP using backwards induction algorithm
mdp_example_forest

Generates a MDP for a simple forest management problem
mdp_policy_iteration

Solves discounted MDP using policy iteration algorithm
mdp_policy_iteration_modified

Solves discounted MDP using modified policy iteration algorithm
mdp_value_iteration

Solves discounted MDP using value iteration algorithm
mdp_span

Evaluates the span of a vector
mdp_eval_policy_matrix

Evaluates a policy using matrix inversion and product