Explore the vast range of titles available.

The aim of this study was to suggest a methodological evaluation framework for assessing the employment impacts of transport infrastructure construction. The applicability and usability of different ex-ante employment impact assessment methods were evaluated. Commonly, the employment impacts during construction are used as a justification for investment decisions. In this study, we tested three commonly used methods to estimate the employment impacts during the construction of three real-life case studies and compared the results to the known impacts of these projects. The results indicate that transport infrastructure construction is not an effective means of employment policy nationwide. This is partly due to insufficient labor supply in the infrastructure engineering and construction industries. A higher employment rate on a national level would require an increase in labor supply instead of an increase in labor demand. However, even though the national net impact on employment was close to zero, the gross regional impact on employment would still be useful information in project planning. The methodological framework, presented in this paper, helps to manage the employment impacts of transport infrastructure construction in a proper context.

Software product-line engineering is arguably one of the most successful methods for establishing large portfolios of software variants in an application domain. However, despite the benefits, establishing a product line requires substantial upfront investments into a software platform with a proper product-line architecture, into new software-engineering processes (domain engineering and application engineering), into business strategies with commercially successful product-line visions and financial planning, as well as into re-organization of development teams. Moreover, establishing a full-fledged product line is not always possible or desired, and thus organizations often adopt product-line engineering only to an extent that deemed necessary or was possible. However, understanding the current state of adoption, namely, the maturity or performance of product-line engineering in an organization, is challenging, while being crucial to steer investments. To this end, several measurement methods have been proposed in the literature, with the most prominent one being the Family Evaluation Framework (FEF), introduced almost two decades ago. Unfortunately, applying it is not straightforward, and the benefits of using it have not been assessed so far. We present an experience report of applying the FEF to nine medium- to large-scale product lines in the avionics domain. We discuss how we tailored and executed the FEF, together with the relevant adaptations and extensions we needed to perform. Specifically, we elicited the data for the FEF assessment with 27 interviews over a period of 11 months. We discuss experiences and assess the benefits of using the FEF, aiming at helping other organizations assessing their practices for engineering their portfolios of software variants.

Health recommender systems (HRSs) are digital platforms designed to deliver personalized health information, resources, and interventions tailored to users' specific needs. However, existing evaluations of HRSs largely focus on algorithmic performance, with limited scientific evidence supporting user-centered assessment approaches and insufficiently defined evaluation metrics. Moreover, no unified or scientifically validated framework currently exists for evaluating these systems, resulting in limited cross-study comparability and constraining regulatory and implementation decision-making. This study aimed to develop a comprehensive, consensus-based evaluation index system for HRSs grounded in the health technology assessment (HTA) framework. This cross-sectional study used a 2-round Delphi process conducted with 18 experts comprising clinicians, digital health researchers, and policymakers who possessed relevant professional experience and domain knowledge in HRSs. The age range of the experts was between 30 and 58 years, with 67% (n=12) of them possessing over 10 years of professional experience. On the basis of literature analysis and HTA principles, a preliminary indicator set comprising 5 primary and 16 secondary indicators was constructed. Experts rated the importance of each indicator using a 5-point Likert scale and provided qualitative suggestions for refinement. After the Delphi process, the analytic hierarchy process was applied to determine indicator weights and assess consistency. The Delphi survey reached full participation in the first round (18/18, 100%) and maintained an 88.9% (16/18) response rate in the second round. The final evaluation index system of HRSs contained 5 first-level indicators (performance, effectiveness, safety, economy, and social appropriateness) and 18 second-level indicators. The mean importance scores of the second-level indicators ranged from 4.25 (SD 0.45) to 5.00 (SD 0.00), with coefficients of variation between 0.000 and 0.220. Among the first-level indicators, safety received the highest weight (0.289), followed by social appropriateness (0.251), effectiveness (0.193), performance (0.136), and economy (0.132). This study presents an evaluation index system for HRSs grounded in the HTA framework and validated through expert consensus. The resulting framework not only provides actionable guidance for the design, optimization, and implementation of HRSs but also fills a methodological gap in the field by offering quantifiable, hierarchical evaluation indicators with validated weighting. Future research will involve iterative refinement and empirical validation of the system in real-world deployment settings, thereby enabling continuous improvement and facilitating the establishment of unified evaluation standards for HRS research and practice.

Background/Objectives: The integration of AI chatbots in nursing education, particularly in simulation-based learning, is advancing rapidly. However, there is a lack of structured evaluation models, especially to assess AI-generated simulations. This article introduces the AI-Integrated Method for Simulation (AIMS) evaluation framework, a dual-phase evaluation framework adapted from the FAITA model, designed to evaluate both prompt design and chatbot performance in the context of nursing education. Methods: This simulation-based study explored the application of an AI chatbot in an emergency planning course. The AIMS framework was developed and applied, consisting of six prompt-level domains (Phase 1) and eight performance criteria (Phase 2). These domains were selected based on current best practices in instructional design, simulation fidelity, and emerging AI evaluation literature. To assess the chatbots educational utility, the study employed a scoring rubric for each phase and incorporated a structured feedback loop to refine both prompt design and chatbox interaction. To demonstrate the framework’s practical application, the researchers configured an AI tool referred to in this study as “Eval-Bot v1”, built using OpenAI’s GPT-4.0, to apply Phase 1 scoring criteria to a real simulation prompt. Insights from this analysis were then used to anticipate Phase 2 performance and identify areas for improvement. Participants (three individuals)—all experienced healthcare educators and advanced practice nurses with expertise in clinical decision-making and simulation-based teaching—reviewed the prompt and Eval-Bot’s score to triangulate findings. Results: Simulated evaluations revealed clear strengths in the prompt alignment with course objectives and its capacity to foster interactive learning. Participants noted that the AI chatbot supported engagement and maintained appropriate pacing, particularly in scenarios involving emergency planning decision-making. However, challenges emerged in areas related to personalization and inclusivity. While the chatbot responded consistently to general queries, it struggled to adapt tone, complexity and content to reflect diverse learner needs or cultural nuances. To support replication and refinement, a sample scoring rubric and simulation prompt template are provided. When evaluated using the Eval-Bot tool, moderate concerns were flagged regarding safety prompts and inclusive language, particularly in how the chatbot navigated sensitive decision points. These gaps were linked to predicted performance issues in Phase 2 domains such as dialog control, equity, and user reassurance. Based on these findings, revised prompt strategies were developed to improve contextual sensitivity, promote inclusivity, and strengthen ethical guidance within chatbot-led simulations. Conclusions: The AIMS evaluation framework provides a practical and replicable approach for evaluating the use of AI chatbots in simulation-based education. By offering structured criteria for both prompt design and chatbot performance, the model supports instructional designers, simulation specialists, and developers in identifying areas of strength and improvement. The findings underscore the importance of intentional design, safety monitoring, and inclusive language when integrating AI into nursing and health education. As AI tools become more embedded in learning environments, this framework offers a thoughtful starting point for ensuring they are applied ethically, effectively, and with learner diversity in mind.

This paper presents a comprehensive review of the use of Artificial Intelligence (AI) in Systematic Literature Reviews (SLRs). A SLR is a rigorous and organised methodology that assesses and integrates prior research on a given topic. Numerous tools have been developed to assist and partially automate the SLR process. The increasing role of AI in this field shows great potential in providing more effective support for researchers, moving towards the semi-automatic creation of literature reviews. Our study focuses on how AI techniques are applied in the semi-automation of SLRs, specifically in the screening and extraction phases. We examine 21 leading SLR tools using a framework that combines 23 traditional features with 11 AI features. We also analyse 11 recent tools that leverage large language models for searching the literature and assisting academic writing. Finally, the paper discusses current trends in the field, outlines key research challenges, and suggests directions for future research. We highlight three primary research challenges: integrating advanced AI solutions, such as large language models and knowledge graphs, improving usability, and developing a standardised evaluation framework. We also propose best practices to ensure more robust evaluations in terms of performance, usability, and transparency. Overall, this review offers a detailed overview of AI-enhanced SLR tools for researchers and practitioners, providing a foundation for the development of next-generation AI solutions in this field.

In this paper we present a case of a structured, facilitated group process with a climate action group engaged in a local Transition initiative. We explore how the interacting contexts between action researchers and the group acted as a constraint for the trajectory of the group process, by looking at the mismatches between the group’s and the researchers’ purposes and differences in expectations about methods of engagement. A methodological framework was used for evaluating the outcomes. The primary aim of this article was to investigate and point out dynamics that may be a hindrance to the effectiveness of a facilitated local climate initiative, with the view to inform facilitation practices and improve future action research processes.

Getting a good feature representation of data is paramount for Human Activity Recognition (HAR) using wearable sensors. An increasing number of feature learning approaches—in particular deep-learning based—have been proposed to extract an effective feature representation by analyzing large amounts of data. However, getting an objective interpretation of their performances faces two problems: the lack of a baseline evaluation setup, which makes a strict comparison between them impossible, and the insufficiency of implementation details, which can hinder their use. In this paper, we attempt to address both issues: we firstly propose an evaluation framework allowing a rigorous comparison of features extracted by different methods, and use it to carry out extensive experiments with state-of-the-art feature learning approaches. We then provide all the codes and implementation details to make both the reproduction of the results reported in this paper and the re-use of our framework easier for other researchers. Our studies carried out on the OPPORTUNITY and UniMiB-SHAR datasets highlight the effectiveness of hybrid deep-learning architectures involving convolutional and Long-Short-Term-Memory (LSTM) to obtain features characterising both short- and long-term time dependencies in the data.

Processing IoT applications directly in the cloud may not be the most efficient solution for each IoT scenario, especially for time-sensitive applications. A promising alternative is to use fog and edge computing, which address the issue of managing the large data bandwidth needed by end devices. These paradigms impose to process the large amounts of generated data close to the data sources rather than in the cloud. One of the considerations of cloud-based IoT environments is resource management, which typically revolves around resource allocation, workload balance, resource provisioning, task scheduling, and QoS to achieve performance improvements. In this paper, we review resource management techniques that can be applied for cloud, fog, and edge computing. The goal of this review is to provide an evaluation framework of metrics for resource management algorithms aiming at the cloud/fog and edge environments. To this end, we first address research challenges on resource management techniques in that domain. Consequently, we classify current research contributions to support in conducting an evaluation framework. One of the main contributions is an overview and analysis of research papers addressing resource management techniques. Concluding, this review highlights opportunities of using resource management techniques within the cloud/fog/edge paradigm. This practice is still at early development and barriers need to be overcome.

Bridges are critical components of transportation systems and are susceptible to various natural and man-made disasters throughout their lifecycle. With the rapid development of the transportation industry, the frequency of vehicle-induced disasters has been steadily increasing. These incidents not only result in structural damage to bridges but also have the potential to cause traffic interruptions, weaken social service functions, and impose significant economic losses. In recent years, research on resilience has become a new focus in civil engineering disaster prevention and mitigation. This study proposes a concept of generalized bridge resilience and presents an evaluation framework for cable-stayed bridges under disasters. The framework includes a resilience evaluation indicator system from multiple dimensions, including safety, society, environment, and economy, which facilitates the dynamic and comprehensive control of bridge resilience throughout its entire lifecycle with the ultimate goals of enhancing structural safety and economic efficiency while promoting the development of environmentally friendly structural ecosystems. Furthermore, considering the influence of recovery speed, the study evaluates various repair strategies through resilience assessment, revealing the applicable environments and conditions for different repair strategies. This methodology offers significant implications for enhancing the safety, efficiency, and environmental sustainability of infrastructure systems, providing valuable guidance for future research in this field.

Purpose To address the gap in evidence‐based information required to support the development of advanced practice nursing (APN) roles in Switzerland, stakeholders identified the need for guidance to generate strategic evaluation data. This article describes an evaluation framework developed to inform decisions about the effective utilization of APN roles across the country. Approach A participatory approach was used by an international group of stakeholders. Published literature and an evidenced‐based framework for introducing APN roles were analyzed and applied to define the purpose, target audiences, and essential elements of the evaluation framework. Through subsequent meetings and review by an expert panel, the framework was developed and refined. Findings A framework to evaluate different types of APN roles as they evolve to meet dynamic population health, practice setting, and health system needs was created. It includes a matrix of key concepts to guide evaluations across three stages of APN role development: introduction, implementation, and long‐term sustainability. For each stage, evaluation objectives and questions examining APN role structures, processes, and outcomes from different perspectives (e.g., patients, providers, managers, policy‐makers) were identified. Conclusions A practical, robust framework based on well‐established evaluation concepts and current understanding of APN roles can be used to conduct systematic evaluations. Clinical Relevance The evaluation framework is sufficiently generic to allow application in developed countries globally, both for evaluation as well as research purposes.