Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
43,706
result(s) for
"Markov processes"
Sort by:
Ergodicity of Markov Processes via Nonstandard Analysis
The Markov chain ergodic theorem is well-understood if either the time-line or the state space is discrete. However, there does not
exist a very clear result for general state space continuous-time Markov processes. Using methods from mathematical logic and
nonstandard analysis, we introduce a class of hyperfinite Markov processes-namely, general Markov processes which behave like finite
state space discrete-time Markov processes. We show that, under moderate conditions, the transition probability of hyperfinite Markov
processes align with the transition probability of standard Markov processes. The Markov chain ergodic theorem for hyperfinite Markov
processes will then imply the Markov chain ergodic theorem for general state space continuous-time Markov processes.
eBook
Dynamics of the Box-Ball System with Random Initial Conditions via Pitman’s Transformation
The box-ball system (BBS), introduced by Takahashi and Satsuma in 1990, is a cellular automaton that exhibits solitonic behaviour. In
this article, we study the BBS when started from a random two-sided infinite particle configuration. For such a model, Ferrari et al.
recently showed the invariance in distribution of Bernoulli product measures with density strictly less than
eBook
Stability of heat kernel estimates for symmetric non-local Dirichlet forms
In this paper, we consider symmetric jump processes of mixed-type on metric measure spaces under general volume doubling condition,
and establish stability of two-sided heat kernel estimates and heat kernel upper bounds. We obtain their stable equivalent
characterizations in terms of the jumping kernels, variants of cut-off Sobolev inequalities, and the Faber-Krahn inequalities. In
particular, we establish stability of heat kernel estimates for
eBook
THE ZIG-ZAG PROCESS AND SUPER-EFFICIENT SAMPLING FOR BAYESIAN ANALYSIS OF BIG DATA
Standard MCMC methods can scale poorly to big data settings due to the need to evaluate the likelihood at each iteration. There have been a number of approximate MCMC algorithms that use sub-sampling ideas to reduce this computational burden, but with the drawback that these algorithms no longer target the true posterior distribution. We introduce a new family of Monte Carlo methods based upon a multidimensional version of the Zig-Zag process of [Ann. Appl. Probab. 27 (2017) 846–882], a continuous-time piecewise deterministic Markov process. While traditional MCMC methods are reversible by construction (a property which is known to inhibit rapid convergence) the Zig-Zag process offers a flexible nonreversible alternative which we observe to often have favourable convergence properties. We show how the Zig-Zag process can be simulated without discretisation error, and give conditions for the process to be ergodic. Most importantly, we introduce a sub-sampling version of the Zig-Zag process that is an example of an exact approximate scheme, that is, the resulting approximate process still has the posterior as its stationary distribution. Furthermore, if we use a control-variate idea to reduce the variance of our unbiased estimator, then the Zig-Zag process can be super-efficient: after an initial preprocessing step, essentially independent samples from the posterior distribution are obtained at a computational cost which does not depend on the size of the data.
Journal Article
Markov chains and decision processes for engineers and managers
\"This book presents an introduction to finite Markov chains and Markov decision processes, with applications in engineering and management. It introduces discrete-time, finite-state Markov chains, and Markov decision processes. The text describes both algorithms and applications, enabling students to understand the logical basis for the algorithms and be able to apply them. The applications address problems in government, business, and nonprofit sectors. The author uses Markov models to approximate the random behavior of complex systems in diverse areas, such as management, production, science, education, health services, finance, and marketing\"-- Provided by publisher.
Book
Conformal Graph Directed Markov Systems on Carnot Groups
We develop a comprehensive theory of conformal graph directed Markov systems in the non-Riemannian setting of Carnot groups equipped
with a sub-Riemannian metric. In particular, we develop the thermodynamic formalism and show that, under natural hypotheses, the limit
set of an Carnot conformal GDMS has Hausdorff dimension given by Bowen’s parameter. We illustrate our results for a variety of examples
of both linear and nonlinear iterated function systems and graph directed Markov systems in such sub-Riemannian spaces. These include
the Heisenberg continued fractions introduced by Lukyanenko and Vandehey as well as Kleinian and Schottky groups associated to the
non-real classical rank one hyperbolic spaces.
eBook