Explore the vast range of titles available.

A goal of runtime software-fault monitoring is to observe software behavior to determine whether it complies with its intended behavior. Monitoring allows one to analyze and recover from detected faults, providing additional defense against catastrophic failure. Although runtime monitoring has been in use for over 30 years, there is renewed interest in its application to fault detection and recovery, largely because of the increasing complexity and ubiquitous nature of software systems. We present taxonomy that developers and researchers can use to analyze and differentiate recent developments in runtime software fault-monitoring approaches. The taxonomy categorizes the various runtime monitoring research by classifying the elements that are considered essential for building a monitoring system, i.e., the specification language used to define properties; the monitoring mechanism that oversees the program's execution; and the event handler that captures and communicates monitoring results. After describing the taxonomy, the paper presents the classification of the software-fault monitoring systems described in the literature.

The ISO 26262 Road vehicles Functional Safety Standard is intended to guide the derivation of appropriate requirements and processes for avoiding systematic and/or random failures in automotive electrical/electronic equipment. Functional safety statements can be captured in the requirements specifications for automotive embedded control units and systems. However, the process of verifying the behaviour of resulting products continues to be incomplete; because embedded programme verification is unsolvable in general. This study shows that it is possible to monitor some proof obligations in the testing phase, or even in the actual operating phase of a system by the use of an on-chip, real-time runtime verification monitor. In this work, the ISO 26262 standard for functional safety is used to guide the definition of the functional safety requirements for a product, and the specific requirements are mapped to logic formulae, such that the actual runtime behaviour of the system for selected properties can be formally verified throughout the lifetime of a product. A case study example for an automotive gearbox control system is presented to demonstrate the feasibility of the scheme. The monitor is constructed as a permanent feature within an integrated circuit that can continuously observe the system's runtime behaviour.

In this paper, we study the problem of runtime verification of real-time event streams; in particular, we propose a language to describe monitors for real-time event streams that can manipulate data from rich domains. We propose a solution based on stream runtime verification (SRV), where monitors are specified by describing how output streams of data are computed from input streams of data. SRV enables a clean separation between the temporal dependencies among incoming events and the concrete operations that are performed during the monitoring. Most SRV specification languages assume that all streams share a global synchronous clock and divide time in discrete instants. At each instant every input has a reading, and for every instant the monitor computes an output. In this paper, we generalize the time assumption to cover real-time event streams, but keep the explicit time offsets present in some synchronous SRV languages like Lola. The language we introduce, called Striver, shares with SRV the simplicity and economy of operators, and the separation between the reasoning about time and the computation of data values. The version of Striver in this paper allows expressing future and past dependencies. Striver is a general language that allows expressing for certain time domains other real-time monitoring languages, like TeSSLa, and temporal logics, like STL. We show in this paper translations from other formalisms for (piecewise-constant) real-time signals and timed event streams. Finally, we report an empirical evaluation of an implementation of Striver.

This article gives an overview of the, monitoring oriented programming framework (MOP). In MOP, runtime monitoring is supported and encouraged as a fundamental principle for building reliable systems. Monitors are automatically synthesized from specified properties and are used in conjunction with the original system to check its dynamic behaviors. When a specification is violated or validated at runtime, user-defined actions will be triggered, which can be any code, such as information logging or runtime recovery. Two instances of MOP are presented: JavaMOP (for Java programs) and BusMOP (for monitoring PCI bus traffic). The architecture of MOP is discussed, and an explanation of parametric trace monitoring and its implementation is given. A comprehensive evaluation of JavaMOP attests to its efficiency, especially in comparison with similar systems. The implementation of BusMOP is discussed in detail. In general, BusMOP imposes no runtime overhead on the system it is monitoring.

In this paper, we present the Real-Time Analog Monitoring Tool (RTAMT), a tool for quantitative monitoring of Signal Temporal Logic (STL) specifications. The library implements a flexible architecture that supports: (1) various environments connected by an Application Programming Interface (API) in Python, (2) various flavors of temporal logic specification and robustness notion such as STL, including an interface-aware variant that distinguishes between input and output variables, and (3) discrete-time and dense-time interpretation of STL with generation of online and offline monitors. We specifically focus on robotics and Cyber-Physical System (CPS) applications, showing how to integrate RTAMT into (1) the Robot Operating System (ROS) and (2) MATLAB/Simulink ® environments. We evaluate the tool by demonstrating several use scenarios involving service robotic and avionic applications.

Runtime enforcement serves as a mechanism to enforce expected properties upon a system, often considered a black box. This is achieved by utilizing an enforcement monitor/enforcer, which transforms an (untrusted) sequence of events into one that conforms to the desired property. Typically, existing frameworks are effective at enforcing a single property. However, when the need arises to enforce multiple properties compositionally, the common practice involves combining these properties by taking their intersection and creating a single enforcement monitor capable of enforcing all these combined properties. Although this approach, known as the monolithic approach, is functional, it does possess certain drawbacks, such as the lack of modularity. Hence, there is a pressing need to construct individual enforcement monitors for each property instead of a monolithic enforcer. These individual enforcement monitors can then be composed accordingly, e.g. serially (one EM after other EM and so on, where the output of one EM serves as the input to the next/succeeding EM in a sequential manner), parallelly (EMs run concurrently, receiving the same input; their individual outputs are then merged using specific methods). Additionally, timed properties offer a more precise means of specifying desired system behaviours by explicitly defining the time intervals between events. Therefore, our research delves into compositional (serial and parallel) monitor composition schemes tailored to safety and co-safety timed properties (modelled as timed automata). We demonstrate that, in general, enforcement monitors for these timed properties do not readily lend themselves to compositional enforcement. We investigate whether, in cases where particular syntactic conditions align with the corresponding timed automata of the properties, their enforcement monitors can be employed in a compositional approach to leverage modular compositional techniques. To provide empirical evidence, we conduct performance evaluations of our framework through a prototype implementation.

The underlying property, its definition, and representation play a major role when monitoring a system. Having a suitable and convenient framework to express properties is thus a concern for runtime analysis. It is desirable to delineate in this framework the sets of properties for which runtime analysis approaches can be applied to. This paper presents a unified view of runtime verification and enforcement of properties in the Safety-Progress classification. First, we extend the Safety-Progress classification of properties in a runtime context. Second, we characterize the set of properties which can be verified (monitorable properties) and enforced (enforceable properties) at runtime. We propose in particular an alternative definition of “property monitoring” to the one classically used in this context. Finally, for the delineated sets of properties, we define specialized verification and enforcement monitors.

Monitorability underpins the technique of runtime verification because it delineates what properties can be verified at runtime. Although many monitorability definitions exist, few are defined explicitly in terms of the operational guarantees provided by monitors, i.e. the computational entities carrying out the verification. We view monitorability as a spectrum, where the fewer guarantees that are required of monitors, the more properties become monitorable. Accordingly, we present a monitorability hierarchy based on this trade-off. For regular specifications, we give syntactic characterisations in Hennessy–Milner logic with recursion for its levels. Finally, we map existing monitorability definitions into our hierarchy. Hence, our work gives a unified framework that makes the operational assumptions and guarantees of each definition explicit. This provides a rigorous foundation that can inform design choices and correctness claims for runtime verification tools.

The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Runtime monitors for example check that the data at runtime is compatible with the data used to train the model. In a first step towards identifying challenges when specifying requirements for training data and runtime monitors, we conducted and thematically analysed ten interviews with practitioners who develop ML models for critical applications in the automotive industry. We identified 17 themes describing the challenges and classified them in six challenge groups. In a second step, we found interconnection between the challenge themes through an additional semantic analysis of the interviews. We explored how the identified challenge themes and their interconnections can be mapped to different architecture views. This step involved identifying relevant architecture views such as data, context, hardware, AI model, and functional safety views that can address the identified challenges. The article presents a list of the identified underlying challenges, identified relations between the challenges and a mapping to architecture views. The intention of this work is to highlight once more that requirement specifications and system architecture are interlinked, even for AI-specific specification challenges such as specifying requirements for training data and runtime monitoring.

This paper presents the open-source runtime verification tool MESA (MEssage-based System Analysis), implemented in Scala, which supports concurrent monitors using the Actor model. Furthermore, the tool supports indexing (slicing) on the data values occurring in data-carrying events, for each individual monitor. The tool is generic in the sense that any monitoring system can be used for creating monitors. In this paper, we use the internal Scala DSL Daut for programming such data-parameterized state machines and temporal logic. To illustrate MESA/Daut, we present a case study that monitors flights from live U.S. airspace data streams, verifying that they conform to planned routes. With base in the case study, we then perform an extensive empirical study of the potential benefits from monitoring slices of a single property in concurrently executing actors. Due to the overhead of scheduling “small” actors (one for each slice or a small number of slices), it is not obvious that concurrent execution of such is beneficial. However, as a main result, we demonstrate that concurrent monitoring of slices to handle data-carrying events can provide considerable speed gains.