| Astrophysics (Index) | About |
Markov chain Monte Carlo (MCMC) is a technique in calculating an approximation of a multi-dimensional definite integral, based upon tests of individual values that are chosen randomly but using a particular strategy. One use of the technique has been determining approximations of probabilities that are impractical to calculate directly. The development of the technique (which began in the 1950s) and its use with computers is one of the keys to the increased popularity of Bayesian statistics.
A Markov chain is a process that goes from state to state according to given probabilities that do not depend upon history, i.e., the probabilities regarding which state will follow each state are fixed, and, for example, if it happens to come back to the current state at some time in the future, the probabilities then will be the same as at present. (Note that some processes that seemingly don't fit this criteria can be modeled as a Markov chain through inclusion of more states.)
In computational mathematics/science, a Monte Carlo method that produces an approximate solution to some equation or inequality (or set of these) through many randomly-selected trials. The quality of the result depends upon selection of test-values relevant to the solution, and appropriate randomness in their selection avoids some kinds of systematic errors in the procedure. The method's name is after Monaco's Monte Carlo district, which has an association with gambling, thus chance.
MCMC uses a Markov chain process to provide an especially effective means of producing test values for the Monte Carlo method, together forming in a practical method for (relatively) efficiently approximating the solutions of multi-dimensional integrals, effectively solving sets of differential equations too complex for more straight-forward numerical integration techniques. For statistics, the general idea is that a Markov chain can be devised that generates simulated samples of a probability distribution, generating these sample instances with this distribution's frequency. An approximation of the distribution is gathered by counting the samples generated, e.g., binning them to make a histogram. The advantage is that such a Markov chain can at times be devised in cases where determining the distribution through solving equations analytically or through (other) numerical methods is impossible or impractical. In determining such a distribution, it is effectively evaluating a complicated integral, thus can be adapted to other applications requiring similar integral evaluations.