Search USGS⌕ Search
Home/Archive/USGS

USGS · 5221948

MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes

B. Kenneth Williams
View original source ↗Read source document ↗Export metadata ↓

Abstract

Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value-improvement and policy-improvement techniques are used in the algorithms. Computing procedures are also described. The algorithms are appropriate for processes that are either finite or infinite, deterministic or stochastic, discounted or undiscounted, in any meaningful combination of these features. Computing procedures are described in terms of initial data processing, bound improvements, process reduction, and testing and solution. Application of the methodology is illustrated with an example involving natural resource management. Management implications of certain hypothesized relationships between mallard survival and harvest rates are addressed by applying the optimality procedures to mallard population models.

Explore related subjects

Applied Stochastic Models and Data AnalysisArticle

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

B. Kenneth Williams. 2006-08-31. MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes. https://doi.org/10.1002/asm.3150040405

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related USGS reports

USGS2006

The use of analysis of variance procedures in biological studies

The analysis of variance (ANOVA) is widely used in biological studies, yet there remains considerable confusion among researchers about the interpretation of hypotheses being tested. Ambiguities arise when statistical designs are unbalanced, and in particular when not all combinations of design factors are represented in the data. This paper clarifies the relationship among hypothesis testing, statistical modelling and computing procedures in ANOVA for unbalanced data. A simple two-factor fixed effects design is used to illustrate three common parametrizations for ANOVA models, and some associations among these parametrizations are developed. Biologically meaningful hypotheses for main effects and interactions are given in terms of each parametrization, and procedures for testing the hypotheses are described. The standard statistical computing procedures in ANOVA are given along with their corresponding hypotheses. Throughout the development unbalanced designs are assumed and attention is given to problems that arise with missing cells.

Applied Stochastic Models and Data Analysis↗