Explore the vast range of titles available.

Studies have shown that combinatorial testing (CT) can be effective for detecting faults in software systems. By focusing on the interactions between different factors of a system, CT shows its potential for detecting faults, especially those that can be revealed only by the specific combinations of values of multiple factors (multi-factor faults). However, is CT practical enough to be applied in the industry? Can it be more effective than other industry-favored techniques? Are there any challenges when applying CT in practice? These research questions remain in the context of industrial settings. In this paper, we present an empirical study of CT on five industrial systems with real faults. The details of the input space model (ISM) construction, such as factor identification and value assignment, are included. We compared the faults detected by CT with those detected by the in-house testing teams using other methods, and the results suggest that despite some challenges, CT is an effective technique to detect real faults, especially multi-factor faults, of software systems in industrial settings. Observations and lessons learned are provided to further improve the fault detection effectiveness and overcome various challenges.

Training exercises are an important tool in crisis management, as they can assist in a multitude of tasks, such as planning pre-crisis resource requirements and allocation, response planning and help train emergency personnel for actual crises. To be effective, the exercises have to utilize well constructed scenarios and be able to replicate certain characteristics of a crisis situation. In this paper, we propose a conceptual mathematical modeling approach for the automated generation of scenarios for disaster exercises via certain combinatorial sequence structures. The derived scenarios within an exercise collectively fulfill different notions of combinatorial sequence coverage, thereby providing the means to test existing response strategies for deficiencies as well as to train emergency personnel for their ability to handle different arrangements of events. This guaranteed diversity by construction can be used as a basis to obtain quantitative assurance statements when these scenarios have been successfully mastered by participants in exercises. We illustrate our proposed approach utilizing two different combinatorial structures for two example disasters.

Combinatorial interaction testing, which is a technique to verify a system with numerous input parameters, employs a mathematical object called a covering array as a test input. This technique generates a limited number of test cases while guaranteeing a given combinatorial coverage. Although this area has been studied extensively, handling constraints among input parameters remains a major challenge, which may significantly increase the cost to generate covering arrays. In this work, we propose a mathematical operation, called “weaken-product based combinatorial join”, which constructs a new covering array from two existing covering arrays. The operation reuses existing covering arrays to save computational resource by increasing parallelism during generation without losing combinatorial coverage of the original arrays. Our proposed method significantly reduce the covering array generation time by 13–96% depending on use case scenarios.

Testing is an important activity to detect faults before software deployment. We focus on black-box combinatorial testing, where fault detection is one of the main objectives. In this paper, we argue that input model abstraction notably impacts the fault detection capability of a combinatorial test suite. First, we present experiments from previous work that support this argument. We then perform new experiments on a more diverse set of programs. These experiments use mutation testing to estimate fault detection capability, but we also include structural coverage measures in the new experiments. Finally, we elaborate on two possible improvements to obtain an optimal input abstraction strategy for not just continuous but all input domains. Both experiments suggest that input abstraction affects the fault detection capability. We claim that the improvements will produce a better input abstraction with which we can achieve better fault detection capability without increasing the test suite size.

Combinatorial Interaction Testing is a widely used method for testing intricate systems. In most cases, the test suites are generated from scratch. However, there are cases when testers may want to reuse existing tests, in order to include them in a new test suite, both for enhancing the performance of the generation process or because those tests are valuable for checking the functioning of the system under test in critical conditions. In this paper, we propose a general framework for dealing with existing test suites using combinatorial test generators. We also discuss the definition of partial tests and test suites, and the scenarios in which partial tests should or could be reused. Finally, we compare the most common tools for completing test suites, namely ACTS, PICT, and pMEDICI+, using different incompleteness levels in the seeds. ACTS with seeds generally performed the best in terms of test suite size and generation time. The other two tools, namely PICT and pMEDICI+, were slower and produced larger test suites on average. We have found that using seeds could sometimes come with a cost, especially in the scenario where test cases are partial and completing them is not always cost-effective in terms of generation time. The choice of re-using or throwing away existing tests must be based on use case-specific requirements. We do not recommend using seeds when they are composed of partial test cases, providing that they are not required for some other reason. On the contrary, we envision the use of partial test suites when a test suite with higher strength is needed.

We present a Satisfiability (SAT)-based approach for building Mixed Covering Arrays with Constraints of minimum length, referred to as the Covering Array Number problem. This problem is central in Combinatorial Testing for the detection of system failures. In particular, we show how to apply Maximum Satisfiability (MaxSAT) technology by describing efficient encodings for different classes of complete and incomplete MaxSAT solvers to compute optimal and suboptimal solutions, respectively. Similarly, we show how to solve through MaxSAT technology a closely related problem, the Tuple Number problem, which we extend to incorporate constraints. For this problem, we additionally provide a new MaxSAT-based incomplete algorithm. The extensive experimental evaluation we carry out on the available Mixed Covering Arrays with Constraints benchmarks and the comparison with state-of-the-art tools confirm the good performance of our approaches.

With the technology of electronic chassis control systems of automobile is widely used, the functional interaction between brake system and the other electronic systems may lead to brake boost degradation. Therefore, it is necessary to find out brake boost degradation events in the quite large number of driving scenarios. To solve the difficulty of thoroughly and quickly searching for brake boost degradation conditions in the large number of driving scenarios, based on Mechatronic-Hardware-In-the-Loop (M-HIL) technology, this paper constructs an electrical chassis system M-HIL bench to verify the function and performance of the electronic brake control system under actual chassis system conditions. To search and locate the brake boost degradation conditions rapidly and enhance the searching efficiency of levels boundary of affecting factors for brake boost degradation, firstly, based on pair-wise coverage combinatorial testing, brake boost degradation occurrence rate is estimated and initial level combinations including brake boost degradation events are calculated by conducting the brake boost degradation single test of all test cases automatically. Secondly, if the brake boost degradation occurrence rate exceeds the threshold of occurrence rate, based on particle swarm optimization (PSO) algorithm and islet cases expansion, levels boundary of affecting factors for brake boost degradation is searched by execution of automatic testing program. The optimizing search testing method proposed in this paper, which combines pair-wise coverage combinatorial testing theory with PSO, can enhance testing efficiency for electronic brake control system. In addition, the scenario coverage of brake boost degradation events can also be increased by expanding islet cases.

Software testing is one of important phase in software development. The capabilities of t-way testing to cater bugs due to interactions while reducing the test suite size compare to exhaustive testing has been proven in past decades. However, the execution’s time of the t-way strategy also should be given attention as it could increase the productivity of the testing phase. Thus, this paper proposed a tune version of ant colony optimization algorithm (TACO). TACO is metaheuristic strategy where it adopts ant colony optimization in generating test suites. As further improvement, TACO also integrated with fuzzy logic to dynamically select amount of ant in the algorithm. TACO able to supports uniform strength t-way testing. Experiment result shows that TACO produce a remarkable result of test suite size and execution’s time compared to other strategy for uniform strength t-way testing.

Providing a reusable methodology for the evaluation of covering array generation utilities and apply it to a corpus of such tools, obtaining an overview of supported features and constraints, performance, output size, file formats, and practical considerations that may ease or hinder adoption. Analysis of supported capabilities, input and output formats, and constraints, followed by an experimental evaluation of eight covering array generation tools, two of which were provided in updated versions, against a corpus of 295 publicly available benchmark models in six categories, producing arrays of strength two to six. Capabilities, particularly constraint support, vary widely amongst competitors. Metaheuristic algorithms, commonly focused on postoptimization, tend to produce small arrays at the cost of performance. Approaches based on the In-Parameter-Order paradigm offer a good balance between speed and output size that may prove conducive to real-world adoption. The choice of a covering array generation utility should be based on specific requirements related to the use case. Nevertheless, our evaluation identifies some candidates – CAgen, ACTS, and APPTS – which lead the field in terms of overall score. Others, such as PICT, offer unique features; however, a lack of standardization may lead to vendor lock-in.

This paper introduces a new variant of a metaheuristic algorithm based on the whale optimization algorithm (WOA), the Q -learning algorithm and the Exponential Monte Carlo Acceptance Probability called (QWOA-EMC). Unlike WOA, QWOA-EMC permits just-in-time adaptive selection of its operators (i.e., between shrinking mechanism, spiral shape mechanism, and random generation) based on their historical performances as well as exploits the Monte Carlo Acceptance probability to further strengthen its exploration capabilities by allowing a poor performing operator to be reselected with probability in the early part of the iteration. Experimental results for constraints combinatorial test generation demonstrate that the proposed QWOA-EMC outperforms WOA and performs competitively against other metaheuristic algorithms.