Thumbnail
Access Restriction
Subscribed

Author Cogill, R. ♦ Vargo, E.
Source IEEE Xplore Digital Library
Content type Text
Publisher Institute of Electrical and Electronics Engineers, Inc. (IEEE)
File Format PDF
Copyright Year ©2012
Language English
Subject Domain (in DDC) Technology ♦ Engineering & allied operations ♦ Other branches of engineering
Subject Keyword Markov processes ♦ Poisson equations ♦ Mathematical model ♦ Approximation methods ♦ Steady-state ♦ Equations ♦ Accuracy
Abstract The steady-state behavior a finite-state Markov chain can be evaluated by solving a Poisson equation, a special case of the optimality equation arising in average cost Markov decision processes. In practice, solving the Poisson equation is typically no easier than evaluating steady state behavior directly using the invariant probability mass function of the Markov chain. However, it is known that approximate solutions to the Poisson equation can be used to produce bounds on steady state performance and to accelerate simulations for estimating steady state behavior. In this paper we study the special structure taken on by the Poisson equation when the associated Markov chain is reversible. In particular, we show that reversible Markov chains have a special form of the Poisson equation that admits a closed form solution. As an application of the reversible Poisson equation we consider the construction of control variates that can be used in Markov chain-based samplers. We show one class of control variates that are obtained by approximating the known solution to the reversible Poisson equation, and demonstrate the application of these control variates on an example involving the Metropolis-Hastings algorithm.
Description Author affiliation: Dept. of Systems and Information Engineering, University of Virginia, USA (Cogill, R.; Vargo, E.)
ISBN 9781467320658
ISSN 07431546
Educational Role Student ♦ Teacher
Age Range above 22 year
Educational Use Research ♦ Reading
Education Level UG and PG
Learning Resource Type Article
Publisher Date 2012-12-10
Publisher Place USA
Rights Holder Institute of Electrical and Electronics Engineers, Inc. (IEEE)
e-ISBN 9781467320665
Size (in Bytes) 208.37 kB
Page Count 7
Starting Page 6676
Ending Page 6682


Source: IEEE Xplore Digital Library