Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
63,579
result(s) for
"decision process"
Sort by:
Decision-making under uncertainty: beyond probabilities
This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty. The paper features an overview of Markov decision processes (MDPs) and extensions to account for partial observability and adversarial behavior. These models sufficiently capture aleatoric uncertainty, but fail to account for epistemic uncertainty robustly. Consequently, we present a thorough overview of so-called uncertainty models that exhibit uncertainty in a more robust interpretation. We show several solution techniques for both discrete and continuous models, ranging from formal verification, over control-based abstractions, to reinforcement learning. As an integral part of this paper, we list and discuss several key challenges that arise when dealing with rich types of uncertainty in a model-based fashion.
Journal Article
Analysis of a time–cost trade-off in a resource-constrained GERT project scheduling problem using the Markov decision process
Nowadays the advent of new types of projects such as startups, maintenance, and education make a revolution in project management, so that, classical project scheduling methods are incapable in analyzing of these stochastic projects. This study considers a time–cost trade-off project scheduling problem, where the structure of the project is uncertain. To deal with the uncertainties, we implemented Graphical Evaluation and Review Technique (GERT). The main aim of the study is to balance time and the amount of a non-renewable resource allocated to each activity considering the finite-time horizon and resource limitations. To preserve the generality of the model, we considered both discrete and continuous distribution functions for the activity’s duration. From a methodological standpoint, we proposed an analytical approach based on the Markov Decision Process (MDP) and Semi-Markov Decision Process (SMDP) to find the probability distribution of project makespan. These models are solved using the value iteration and a finite-horizon Linear Programming (LP) model. Two randomly generated examples explain the value iteration for models in detail. Furthermore, seven example groups each with five instances are adopted from a well-known data set, PSPLIB, to validate the efficiency of the proposed models in contrast to the two extensively-studied methods, Genetic algorithm (GA) and Monte-Carlo simulation. The convergence of the GA and simulation results to those of MDP and SMDP represent the efficiency of the proposed models. Besides, conducting a sensitivity analysis on the project completion probability with respect to the available resource, gives a good insight to managers to plan their resources.
Journal Article
Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems: Part 1—Fundamentals and Applications in Games, Robotics and Natural Language Processing
The first part of a two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) applications for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approach to simulate the human’s natural learning process, whose key is to let the agent learn by interacting with the stochastic environment. The fact that the agent has limited access to the information of the environment enables AI to be applied efficiently in most fields that require self-learning. Although efficient algorithms are being widely used, it seems essential to have an organized investigation—we can make good comparisons and choose the best structures or algorithms when applying DRL in various applications. In this overview, we introduce Markov Decision Processes (MDP) problems and Reinforcement Learning and applications of DRL for solving POMDP problems in games, robotics, and natural language processing. A follow-up paper will cover applications in transportation, communications and networking, and industries.
Journal Article
Decision making by the modern Supreme Court
\"There are three general models of Supreme Court decision making: the legal model, the attitudinal model and the strategic model. But each is somewhat incomplete. This book advances an integrated model of Supreme Court decision making that incorporates variables from each of the three models. In examining the modern Supreme Court, since Brown v. Board of Education, the book argues that decisions are a function of the sincere preferences of the justices, the nature of precedent, and the development of the particular issue, as well as separation of powers and the potential constraints posed by the president and Congress. To test this model, the authors examine all full, signed civil liberties and economic cases decisions in the 1953-2000 period. Decision Making by the Modern Supreme Court argues, and the results confirm, that judicial decision making is more nuanced than the attitudinal or legal models have argued in the past\"--Provided by publisher.
BOOK
Hierarchical reinforcement learning via dynamic subspace search for multi-agent planning
We consider scenarios where a swarm of unmanned vehicles (UxVs) seek to satisfy a number of diverse, spatially distributed objectives. The UxVs strive to determine an efficient plan to service the objectives while operating in a coordinated fashion. We focus on developing autonomous high-level planning, where low-level controls are leveraged from previous work in distributed motion, target tracking, localization, and communication. We rely on the use of state and action abstractions in a Markov decision processes framework to introduce a hierarchical algorithm, Dynamic Domain Reduction for Multi-Agent Planning, that enables multi-agent planning for large multi-objective environments. Our analysis establishes the correctness of our search procedure within specific subsets of the environments, termed ‘sub-environment’ and characterizes the algorithm performance with respect to the optimal trajectories in single-agent and sequential multi-agent deployment scenarios using tools from submodularity. Simulated results show significant improvement over using a standard Monte Carlo tree search in an environment with large state and action spaces.
Journal Article
A Fast Approach for Reoptimization of Railway Train Platforming in Case of Train Delays
Train platforming is critical for ensuring the safety and efficiency of train operations within the stations, especially when unexpected train delays occur. This paper studies the problem of reoptimization of train platforming in case of train delays, where the train station is modeled using the discretization of the platform track time-space resources. To solve the reoptimization problem, we propose a mixed integer linear programming (MILP) model, which minimizes the weighted sum of total train delays and the platform track assignment costs, subject to constraints defined by operational requirements. Moreover, we design an efficient heuristic algorithm to solve the MILP model such that it can speed up the reoptimization process with good solution precision. Furthermore, a real-world case is taken as an example to show the efficiency and effectiveness of the proposed model and algorithm. The computational results show that the MILP model established in this paper can describe the reoptimization of train platforming accurately, and it can be solved quickly by the proposed heuristic algorithm. In addition, the model and algorithm developed in this paper can provide an effective computer-aided decision-making tool for the train dispatchers in case of train delays.
Journal Article