Explore the vast range of titles available.

The evaluation and assessment of conversational interfaces is a complex task since such software products are challenging to validate through traditional testing approaches. We conducted a systematic Multivocal Literature Review (MLR), on five different literature sources, to provide a view on quality attributes, evaluation frameworks, and evaluation datasets proposed to provide aid to the researchers and practitioners of the field. We came up with a final pool of 118 contributions, including grey (35) and white literature (83). We categorized 123 different quality attributes and metrics under ten different categories and four macro-categories: Relational, Conversational, User-Centered and Quantitative attributes. While Relational and Conversational attributes are most commonly explored by the scientific literature, we testified a predominance of User-Centered Attributes in industrial literature. We also identified five different academic frameworks/tools to automatically compute sets of metrics, and 28 datasets (subdivided into seven different categories based on the type of data contained) that can produce conversations for the evaluation of conversational interfaces. Our analysis of literature highlights that a high number of qualitative and quantitative attributes are available in the literature to evaluate the performance of conversational interfaces. Our categorization can serve as a valid entry point for researchers and practitioners to select the proper functional and non-functional aspects to be evaluated for their products.

Designing a software architecture that satisfies all quality requirements is a difficult task. To determine whether the requirements are achieved, it is necessary to quantitatively evaluate quality attributes on the architecture model. A good evaluation process should have proper answers for these questions: (1) how to feedback the evaluation results to the architecture model (i.e., improve the architecture based on the evaluation results), (2) how to analyze uncertainties in calculations, and (3) how to handle conflicts that may exist between the quality preferences of stakeholders. In this paper, we introduce SQME as a framework for automatic evaluation of software architecture models. The framework uses evolutionary algorithms for architecture improvement, evidence theory for uncertainty handling, and EV/TOPSIS for making trade-off decisions. To validate the applicability of the framework, a case study is performed, and a software tool is developed to support the evaluation process.

The cost of software maintenance is always increasing. The companies are often confronted to failures and software errors. The quality of software to use is so required. In this paper, the authors propose a new formal approach for assessing the quality of object-oriented system design according to the quality assessment model. This approach consists in modeling the input software system by an automaton based on object-oriented design metrics and their relationship with the quality attributes. The model exhibits the importance of metrics through their links with the attributes of software quality. In addition, it is very practical and flexible for all changes. It allows the quality estimation and its validation. For the verification of proposed probabilistic model (automaton), they use the model-checking and the prism tool. The model-checking is very interesting for the evaluation and validation of the probabilistic automaton. They use it to approve the software quality of the three experimental projects. The obtained results are very interesting and of great importance.

This article describes how software quality engineering is an inevitable activity, which must be accomplished during software development process in order to avoid software failures and ensuring its quality. Embedded systems are computer platforms, which require high quality software. Many researchers interested in embedded systems have demonstrated that the quality of the embedded software has a significant effect on the performances of the entire system. In the literature, several works have been emerged from this line of research. The aim of this article is to present a survey of the most important works, which deal with embedded software quality engineering. A comparative study is also given in order to show strengths and weaknesses of each work.

Maintainability is an essential software quality attribute as software maintenance is a costly process. ISO 9126 characterised maintainability with five sub-characteristics, one of which is stability. Unstable software may lead to high maintenance cost and effort. Classes in object-oriented systems form the basic elements of the software architecture; hence, stable classes may contribute to reducing the software maintenance cost and effort. In this study, the author conducts an empirical study to evaluate the relationship between class stability and maintainability. The author correlates class stability with maintainability effort measured by the number of hours spent on maintenance activities and by the line of code changes. Results show that classes with higher values of stability measured by the class stability metric (CSM) are associated with a lower value of perfective maintenance effort measured by hours. CSM also correlated with all types of maintenance (corrective, adaptive and perfective) if measured for the cumulatively combined system classes in all iterations rather than per iteration. The author also found that none of the stability metrics show a relationship with maintainability when measured by number of line of code changes.

A software product line consists of a family of software systems. Most of quality attributes are defined for single systems. When we are facing a family of products instead of a single system, some aspects of architecture evaluation, such as cost, time, and reusability of available assets, become more highlighted. In this paper a new quality attribute for software product line, which we called it extractability, is introduced. Also extractability measuring method and relationship between extractability with some quality attributes is presented. At the end, Extractability Effectiveness on Software Product Line is evaluated in practice.

Most external software quality attributes are conceptually subjective. For example, maintainability is an external software quality attribute, and it is subjective because interpersonally agreed definitions for the attribute include the phrase ‘the ease with which maintenance tasks can be performed’. Subjectivity clearly makes measurement of the attributes and validation of prediction systems for the attributes problematic. In fact, in spite of the definitions, few statistically valid attempts at determining the predictive capability of prediction systems for external quality attributes have been published. When validations have been attempted, one approach used is to ask experts to indicate if the values provided by the prediction system informally agree with the experts’ intuition. These attempts are undertaken without determining, independently of the prediction system, whether the experts are capable of direct consistent measurement of the attribute. Hence, a statistically valid and unbiased estimate of the predictive capability of the prediction system cannot be obtained (because the experts’ measurement process is not independent of the prediction system’s values). In this paper, it is argued that the problem of subjective measurement of quality attributes should not be ignored if quality is to be introduced into software in a controlled way. Further, it is argued that direct measurement of quality attributes should be encouraged and that in fact such measurement can be quantified to establish consistency using an existing approach. However, the approach needs to be made more accessible to promote its use. In so doing, it would be possible to decide whether consistent independent estimates of the true values of software quality attributes can be assigned and prediction systems for quality attributes developed.

One reason that researchers may wish to demonstrate that an external software quality attribute can be measured consistently is so that they can validate a prediction system for the attribute. However, attempts at validating prediction systems for external subjective quality attributes have tended to rely on experts indicating that the values provided by the prediction systems informally agree with the experts’ intuition about the attribute. These attempts are undertaken without a pre-defined scale on which it is known that the attribute can be measured consistently. Consequently, a valid unbiased estimate of the predictive capability of the prediction system cannot be given because the experts’ measurement process is not independent of the prediction system’s values. Usually, no justification is given for not checking to see if the experts can measure the attribute consistently. It seems to be assumed that: subjective measurement isn’t proper measurement or subjective measurement cannot be quantified or no one knows the true values of the attributes anyway and they cannot be estimated . However, even though the classification of software systems’ or software artefacts’ quality attributes is subjective, it is possible to quantify experts’ measurements in terms of conditional probabilities. It is then possible, using a statistical approach, to assess formally whether the experts’ measurements can be considered consistent. If the measurements are consistent, it is also possible to identify estimates of the true values, which are independent of the prediction system. These values can then be used to assess the predictive capability of the prediction system. In this paper we use Bayesian inference, Markov chain Monte Carlo simulation and missing data imputation to develop statistical tests for consistent measurement of subjective ordinal scale attributes.

Software maintenance is the dominant factor contributing to the high cost of software. In this paper, the software maintenance process and the important software quality attributes that affect the maintenance effort are discussed. One of the most important quality attributes of software maintainability is the stability of a program, which indicates the resistance to the potential ripple effect that the program would have when it is modified. Measures for estimating the stability of a program and the modules of which the program is composed are presented, and an algorithm for computing these stability measures is given. An algorithm for normalizing these measures is also given. Applications of these measures during the maintenance phase are discussed along with an example. An indirect validation of these stability measures is also given. Future research efforts involving application of these measures during the design phase, program restructuring based on these measures, and the development of an overall maintainability measure are also discussed.

Good data quality is crucial for any data-driven system’s effective and safe operation. For critical safety systems, the significance of data quality is even higher since incorrect or low-quality data may cause fatal faults. However, there are challenges in identifying and managing data quality. In particular, there is no accepted process to define and continuously test data quality concerning what is necessary for operating the system. This lack is problematic because even safety-critical systems become increasingly dependent on data. Here, we propose a Candidate Framework for Data Quality Assessment and Maintenance (CaFDaQAM) to systematically manage data quality and related requirements based on design science research. The framework is constructed based on an advanced driver assistance system (ADAS) case study. The study is based on empirical data from a literature review, focus groups, and design workshops. The proposed framework consists of four components: a Data Quality Workflow, a List of Data Quality Challenges, a List of Data Quality Attributes, and Solution Candidates. Together, the components act as tools for data quality assessment and maintenance. The candidate framework and its components were validated in a focus group.