  This tutorial describes recently developed general optimality conditions for
Markov Decision Processes that have significant applications to inventory
control. In particular, these conditions imply the validity of optimality
equations and inequalities. They also imply the convergence of value iteration
algorithms. For total discounted-cost problems only two mild conditions on the
continuity of transition probabilities and lower semi-continuity of one-step
costs are needed. For average-cost problems, a single additional assumption on
the finiteness of relative values is required. The general results are applied
to periodic-review inventory control problems with discounted and average-cost
criteria without any assumptions on demand distributions. The case of partially
observable states is also discussed.

Optimization and Control