Markov chains and Markov chain Monte Carlo methods

Abstract

[en] The aim of this project is to thoroughly study the main properties of discretetime Markov chains with finite state spaces and one of its applications that finds greatest usage, Markov chain Monte Carlo (MCMC) methods, which are simulation tools to estimate integrals and sample from distributions. A brief description of regular Monte Carlo is included to introduce and understand MCMC. Aside from the theoretical description and algorithms, practical considerations to take into account when implementing MCMC, such as the thermalization of chains and determining the number of iterations, are included as well. A simple example of the calculation of $\Gamma(3 / 2)$ is executed so as to illustrate the functioning and performance of MCMC.