Dynamic Probabilistic Decision Processes
ABSTRACT
This paper presents a general model for the formulation and solution of the risk-sensitive dynamic decision problem that maximizes the certain equivalent of the discounted rewards of a time-varying Markov decision process. The problem is solved by applying the principle of optimality and stochastic dynamic programming to the immediate rewards and the certain equivalent associated with the remaining transitions of a time-varying Markov process over a finite or infinite time horizon, under the assumptions of constant risk aversion and discounting of future cash flows. The solution provides transient and stationary optimal decision policies which depend on the presence or absence of discounting. The construction equipment replacement problem serves as an example application of the model to illustrate the solution methodology, and the sensitivity of the optimal policy to the discount factor and the degree of risk aversion.
AUTHORS
Photios G. Ioannou
Civil & Environmental Engineering Department
University of Michigan
Ann Arbor, Michigan 48109-2125, U.S.A.
e-mail: photios@umich.edu
KEYWORDS
Dynamic programming, optimal policies, Markov process, decision analysis, uncertainty, discounting, risk aversion, engineering economy, equipment replacement.
AVAILABILITY