Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
4,020
result(s) for
"Stochastic games"
Sort by:
ALGORITHMS FOR STOCHASTIC GAMES WITH PERFECT MONITORING
We study the pure-strategy subgame-perfect Nash equilibria of stochastic games with perfect monitoring, geometric discounting, and public randomization. We develop novel algorithms for computing equilibrium payoffs, in which we combine policy iteration when incentive constraints are slack with value iteration when incentive constraints bind. We also provide software implementations of our algorithms. Preliminary simulations indicate that they are significantly more efficient than existing methods. The theoretical results that underlie the algorithms also imply bounds on the computational complexity of equilibrium payoffs when there are two players. When there are more than two players, we show by example that the number of extreme equilibrium payoffs may be countably infinite.
Journal Article
The Blockchain Folk Theorem
Blockchains are distributed ledgers, operated within peer-to-peer networks. We model the proof-of-work blockchain protocol as a stochastic game and analyze the equilibrium strategies of rational, strategic miners. Mining the longest chain is a Markov perfect equilibrium, without forking, in line with Nakamoto (2008). The blockchain protocol, however, is a coordination game, with multiple equilibria. There exist equilibria with forks, leading to orphaned blocks and persistent divergence between chains. We also show how forks can be generated by information delays and software upgrades. Last we identify negative externalities implying that equilibrium investment in computing capacity is excessive.
Journal Article
Uncertainty-driven symmetry-breaking and stochastic stability in a generic differential game of lobbying
We study a 2-player stochastic differential game of lobbying. Players invest in lobbying activities to alter the legislation in her own benefit. The payoffs are quadratic and uncertainty is driven by a Wiener process. We consider the Nash symmetric game where players face the same cost and extract symmetric payoffs, and we solve for Markov Perfect Equilibria (MPE) in the class of affine functions. First, we prove a general sufficient (catching up) optimality condition for two-player stochastic games with uncertainty driven by Wiener processes. Second, we prove that the number and nature of MPE depend on the extent of uncertainty (i.e. the variance of the Wiener processes). In particular, we prove that while a symmetric MPE always exists, two asymmetric MPE emerge if and only if uncertainty is large enough. Third, we study the stochastic stability of all the equilibria. We notably find, that the state converges to a stationary invariant distribution under asymmetric MPE. Fourth, we study the implications for rent dissipation asymptotically and compare the outcomes of symmetric vs asymmetric MPE in this respect, ultimately enhancing again the role of uncertainty.
Journal Article
DISCOUNTED STOCHASTIC GAMES WITH NO STATIONARY NASH EQUILIBRIUM: TWO EXAMPLES
We present two examples of discounted stochastic games, each with a continuum of states, finitely many players, and actions, that possess no stationary equilibria. The first example has deterministic transitions—an assumption undertaken in most of the early applications of dynamics games in economics—and perfect information, and does not possess even stationary approximate equilibria or Markovian equilibria. The second example satisfies, in addition to stronger regularity assumptions, that all transitions are absolutely continuous with respect to a fixed measure—an assumption that has been widely used in more recent economic applications. This assumption has been undertaken in several positive results on the existence of stationary equilibria in special cases, and in particular, guarantees the existence of stationary approximate equilibria.
Journal Article
A stochastic game model for infectious disease management decisions in schools
Infectious diseases pose a significant threat to the health and well-being of young children. Schools are often at risk of outbreaks owing to close interactions among students; therefore, they can implement measures such as enhancing hygiene practices or temporarily canceling classes to manage these infectious diseases during outbreaks. Meanwhile, parents face the difficult decisions of keeping their children at home, sending them to school unprotected or wearing masks, weighing the risks of exposure to potential disruptions in education and work. This study developed a stochastic game model to address this issue and analyzed the strategic interactions between schools and parents during outbreaks. The results highlight the significance of the estimated utilities and transition probabilities in shaping the decisions of schools and parents to manage school closures effectively, minimize disruption to education, and protect the health of students and communities.
Journal Article
Stationary, completely mixed and symmetric optimal and equilibrium strategies in stochastic games
In this paper, we address various types of two-person stochastic games—both zero-sum and nonzero-sum, discounted and undiscounted. In particular, we address different aspects of stochastic games, namely: (1) When is a two-person stochastic game completely mixed? (2) Can we identify classes of undiscounted zero-sum stochastic games that have stationary optimal strategies? (3) When does a two-person stochastic game possess symmetric optimal/equilibrium strategies? Firstly, we provide some necessary and some sufficient conditions under which certain classes of discounted and undiscounted stochastic games are completely mixed. In particular, we show that, if a discounted zero-sum switching control stochastic game with symmetric payoff matrices has a completely mixed stationary optimal strategy, then the stochastic game is completely mixed if and only if the matrix games restricted to states are all completely mixed. Secondly, we identify certain classes of undiscounted zero-sum stochastic games that have stationary optima under specific conditions for individual payoff matrices and transition probabilities. Thirdly, we provide sufficient conditions for discounted as well as certain classes of undiscounted stochastic games to have symmetric optimal/equilibrium strategies—namely, transitions are symmetric and the payoff matrices of one player are the transpose of those of the other. We also provide a sufficient condition for the stochastic game to have a symmetric pure strategy equilibrium. We also provide examples to show the sharpness of our results.
Journal Article
Stationary Bayesian-Markov equilibria in Bayesian stochastic games with periodic revelation
I consider a class of dynamic Bayesian games in which types evolve stochastically according to a first-order Markov process on a continuous type space. Types are privately informed, but they become public together with actions when payoffs are obtained, resulting in a delayed information revelation. In this environment, I show that there exists a stationary Bayesian-Markov equilibrium in which a player's strategy maps a tuple of the previous type and action profiles and the player's current type to a mixed action. The existence can be extended to K-periodic revelation. I also offer a computational algorithm to find an equilibrium.
Journal Article
Dynamic Pricing of Perishable Assets Under Competition
We study dynamic price competition in an oligopolistic market with a
mix
of substitutable and complementary perishable assets. Each firm has a fixed initial stock of items and competes in setting prices to sell them over a finite sales horizon. Customers sequentially arrive at the market, make a purchase choice, and then leave immediately with some likelihood of no purchase. The purchase likelihood depends on the time of purchase, product attributes, and current prices. The demand structure includes time-variant linear and multinomial logit demand models as special cases. Assuming deterministic customer arrival rates, we show that any equilibrium strategy has a simple structure, involving a finite set of
shadow prices
measuring capacity externalities that firms exert on each other: equilibrium prices can be solved from a one-shot price competition game under the
current-time
demand structure, taking into account capacity externalities through the
time-invariant
shadow prices. The former reflects the transient demand side at every moment, and the latter captures the aggregate supply constraints over the sales horizon. This simple structure sheds light on dynamic revenue management problems under competition, which helps capture the essence of the problems under demand uncertainty. We show that the equilibrium solutions from the deterministic game provide precommitted and contingent heuristic policies that are asymptotic equilibria for its stochastic counterpart, when demand and supply are sufficiently large.
This paper was accepted by Yossi Aviv, operations management
.
Journal Article
The Duel Discounted Stochastic Game
A
duel
involves two
stationary
players who shoot at each other until at least one of them dies; a
truel
is similar but involves three players. In the past, the duel has been studied mainly as a component of the truel, which has received considerably more attention. However we believe that the duel is interesting in itself. In this paper we formulate the duel (with either simultaneous or sequential shooting) as a
discounted stochastic game
. We show that this game has a
unique
Nash equilibrium in
stationary strategies
; however, it also possesses
cooperation-promoting
Nash equilibria in
nonstationary strategies
. We show that these are also subgame perfect equilibria. Finally, we argue that the nature of the game and its equilibria is similar to that of the
repeated Prisoner’s dilemma
.
Journal Article