Catalogue Search | MBRL
Search Results Heading
Explore the vast range of titles available.
MBRLSearchResults
-
DisciplineDiscipline
-
Is Peer ReviewedIs Peer Reviewed
-
Item TypeItem Type
-
SubjectSubject
-
YearFrom:-To:
-
More FiltersMore FiltersSourceLanguage
Done
Filters
Reset
187
result(s) for
"Development and Evaluation of Research Methods, Instruments and Tools"
Sort by:
The Adult Inpatient eHealth Literacy Scale (AIPeHLS): Development and Validation Study
2025
The rapid evolution of digital health technologies, particularly within the Web 3.0 framework, has underscored eHealth literacy (eHL) as a critical competency for patients engaging with digital health care platforms. Patients in sustained hospital stays, often in vulnerable conditions, face unique challenges in using eHealth tools effectively. However, existing eHL assessment tools are insufficient to address the intricate and dynamic demands of contemporary health care systems, especially for individuals under continuous hospital care.
This study aimed to develop the Adult Inpatient eHealth Literacy Scale (AIPeHLS), a comprehensive, multidimensional tool grounded in the Lily Model, to evaluate eHL among adult inpatients within the context of digital health care innovations.
The development of the AIPeHLS followed a systematic, multiphase process. Initial item pool generation was informed by a literature review and then refined using the Delphi method, resulting in a preliminary set of 53 items spanning 6 dimensions of the Lily Model. The scale was refined through a pilot survey among 100 individuals requiring inpatient care, followed by item analysis and exploratory factor analysis (EFA). Validation was achieved via a cross-sectional study with 532 participants, using confirmatory factor analysis (CFA) to verify the scale structure, alongside evaluations of convergent, discriminant, criterion-related, and content validity. Reliability was assessed using Cronbach α, Omega, and split-half reliability.
The finalized AIPeHLS comprised 44 items across 6 dimensions: traditional literacy, information literacy, media literacy, health literacy, computer literacy, and scientific literacy, reflecting the skills necessary in the Web 3.0 context. Both EFA and CFA confirmed the 6-factor structure, demonstrating acceptable model fit indices (χ²=1974.654 (df=887), root mean square error of approximation=0.048, comparative fit index=0.957, normed fit index=0.925, and incremental fit index=0.957). The scale exhibited robust content validity, convergent and discriminant validity, criterion-related validity, and high internal consistency, with a Cronbach α of .965, Omega coefficient of 0.962, and a split-half reliability of 0.791 for the entire scale.
The 44-item AIPeHLS was found to be a reliable and valid instrument for assessing eHL in adult inpatients in the evolving Web 3.0 context. Its comprehensive framework and strong psychometric properties make it an effective tool for health care providers to understand patients' digital health competencies and tailor interventions accordingly. For researchers, our findings provided opportunities to explore the relationship between eHL and health outcomes, while offering valuable insights into the development of more effective eHealth interventions and policies.
Journal Article
Beyond Missingness: Systematizing Methods for Comprehensive Data Fitness Assessment in Clinical Research
by
Forrest, Christopher B
,
Weiskopf, Nicole G
,
Wieand, Kaleigh
in
Advanced Data Analytics in eHealth
,
Biomedical Research
,
Care and treatment
2026
Secondary use of clinical data offers unprecedented opportunities to rapidly conduct large-scale research and improve patient care. However, incomplete understanding of data quality requirements for a study often causes significant delays in executing analyses and validating results. Current practice has largely followed 2 paths. First, multi-institutional networks have developed general data quality programs, but these are typically tied to unique network characteristics and do not address study-specific requirements well. Second, models have been proposed to formalize the requirements for data fitness analyses without extending to the methods needed to meet these requirements. More recently, tools have been developed to conduct cohort-centric screening, focusing on generally applicable structural checks such as missingness or facial implausibility. These provide a first level of information but incompletely capture the fitness requirements of an analysis. In turn, investigators conduct per-study exploratory analyses, but these efforts are typically ad hoc and partially reported, which can hinder reproducible science and delay advances in patient care. Analogously to advances over the past decade in data modeling and reproducible analytics, there is a need for a more systematic, capable approach to study-specific data quality assessment (SSDQA). We discuss such a model, which guides improved SSDQA design and implementation, including metadata for consistent annotation and reporting of data quality assessment results. The model integrates theoretical principles of data quality testing with pragmatic considerations of application to clinical data, providing a consistent approach to specifying data quality assessment checks. Additionally, it proposes to regularize check application through a standard set of options. The SSDQA model builds on current practice, providing a path toward more complete, sound, and reproducible assessments. These characteristics foster multidisciplinary collaboration to identify data quality issues that, in turn, inform decisions about study design and provide important context that has a bearing on adoption of results.
Journal Article
The Impact of Individual Factors on Careless Responding Across Different Mental Disorder Screenings: Cross-Sectional Study
2025
Online questionnaires are widely used for large-scale screening. However, careless responding (CR) from participants can compromise the reliability of screening outcomes. Prior studies have focused on the effects of individual and environmental factors on CR, but the effect of questionnaire type remains underexplored.
This study investigates the individual factors influencing CR in online mental health screening and assesses how the effect of these factors varies across different psychological questionnaires.
This study analyzed data from 24,367 participants across 4 questionnaires (PHQ-9 [Patient Health Questionnaire-9], PSS [Perceived Stress Scale], ISI [Insomnia Severity Index], and GAD-7 [Generalized Anxiety Disorder-7 Scale]). CR was defined as the proportion of items completed in less than 2 seconds per item. We used a multiple linear regression model to examine the effect of individual factors (sex, age, education, smoking, and drinking) on CR across 4 questionnaires. In addition, response times were visualized to identify patterns between careless and careful responders.
Females demonstrate lower levels of CR than males when completing the PHQ-9 (β=-.172, 95% CI -0.104 to -0.089; P<.001), PSS (β=-.234, 95% CI -0.162 to -0.14; P<.001), ISI (β=-.207, 95% CI -0.13 to -0.114; P<.001), and GAD-7 (β=-.177, 95% CI -0.108 to -0.093; P<.001). Older participants demonstrated lower levels of CR on the PHQ-9 (β=-.036, 95% CI -0.007 to -0.003; P<.001), ISI (β=-.036, 95% CI -0.007 to -0.003; P<.001), and GAD-7 (β=-.053, 95% CI -0.009 to -0.005; P<.001), but their age was unrelated to CR on the PSS. Interestingly, compared with participants with an associate-level education, those with a high education (bachelor's, master's, or doctoral degree) demonstrated higher levels of CR, especially those with a master's degree (PHQ-9: β=.098, 95% CI 0.136 to 0.188; P<.001 and GAD-7: β=.091, 95% CI 0.125 to 0.178; P<.001). Smokers exhibited varied patterns, with current smokers demonstrating lower levels of CR on the PHQ-9 (β=-.022, 95% CI -0.064 to -0.016; P=.001) and GAD-7 (β=-.014, 95% CI -0.051 to -0.002; P=.03), whereas occasional smokers demonstrated higher levels of CR on the PSS (β=.019, 95% CI 0.010 to 0.050; P=.003) than nonsmokers. Drinkers demonstrated lower levels of CR than nondrinkers, with the strongest effect among occasional drinkers on the PHQ-9 (β=-.163, 95% CI -0.103 to -0.087; P<.001). Analysis of response times revealed that participants tended to spend less time on PHQ-9 and GAD-7 surveys, and CR on PSS and ISI surveys was characterized by skipping questions.
The effect of individual factors on CR varies across questionnaire types. These findings offer valuable insights for questionnaire designers and administrators, highlighting the need for targeted intervention.
Journal Article
Validation of the Updated Digital Health Literacy Instrument and Development of a Short Form: Online Survey Study of the General Population
2026
The digital health literacy instrument (DHLI) was developed in 2017 to measure individuals' ability to access, understand, evaluate, and apply online health information. Since that time, digital health has shifted from desktop-based internet use to mobile devices, and there has been a rapidly expanding range of health apps. Additionally, heightened privacy and data security requirements have increased the complexity of user competencies needed to engage with digital health tools. These developments underscore the need to update the original DHLI.
This study aimed to create an updated version of the DHLI (DHLI 2.0) that reflects current digital health practices and to examine its reliability and validity by exploring associations with user characteristics. Additionally, we aimed to develop a short-form version to facilitate broader use in research and practice.
The instrument was iteratively updated and pilot-tested to retain the original theoretical framework while reflecting current digital health practices, devices, and emerging challenges such as mobile use and data security. Several items were reworded and a new 2-item subscale on digital safety was added. The full DHLI 2.0 comprises 24 items across 8 skill domains. A 16-item short form was developed by iteratively removing 1 or 2 items per subscale based on the \"α if item deleted\" criterion, while retaining the same subscale structure as the full form. Data to validate the new version of the instrument were collected in June 2024 through an online survey among members of a representative citizen panel in Friesland, a province in the Netherlands (N=2728). Sociodemographics, internet and health-related internet use, general health literacy (measured with the Single Item Literacy Screener), self-reported health, and health care use were assessed. Internal consistency was evaluated using Cronbach α, and construct validity was assessed via Spearman ρ correlations with related constructs.
Internal consistency was high for both the full (α=0.94) and short-form (α=0.90) scales. Most subscales showed satisfactory to excellent reliability (α=0.71-0.93), while \"Securing privacy\" and \"Using security measures\" demonstrated moderate reliability (α=0.65-0.66). The DHLI 2.0 total scores were approximately normally distributed (skewness -0.5; kurtosis 0.4). As expected, digital health literacy was negatively correlated with age (ρ=-0.39, P<.001) and positively correlated with education (ρ=0.22, P<.001), income (ρ=0.27, P<.001), time spent online (ρ=0.32, P<.001), and general health literacy (ρ=-0.42, P<.001).
The DHLI 2.0 provides an updated, reliable, and valid measure of digital health literacy covering 8 key domains, including data security. The 16-item short form offers a concise alternative suitable for research and possibly practical applications in health and eHealth contexts.
Journal Article
Developing a Framework for Online Review-Based Health Care Service Quality Assessment: Text-Mining Study
by
Sun, Jianshan
,
Zhang, Xue
,
Li, Chenwei
in
Data mining
,
Data Mining - methods
,
Development and Evaluation of Research Methods, Instruments and Tools
2025
With the development of online health care platforms, patient reviews have become an important source for assessing medical service quality. However, the critical aspects of quality dimensions in textual reviews remain largely unexplored.
This study aims to establish a comprehensive medical service quality assessment framework by leveraging online review data. Such a framework would support large service providers, such as online platforms, to assess the quality of many doctors efficiently.
We adopted a text-mining approach with theory-driven topic extraction from online reviews to develop a service quality assessment framework. The framework is based on topic and sentiment classification methods. We conducted an empirical analysis to assess the validity of the framework. Specifically, we examined if patients' sentiments regarding our extracted dimensions affect demand (number of consultation requests) due to quality signals reflected in these dimensions.
We develop a 5-dimensional health care service quality framework (HSQ-5D model). In the empirical study, patient demand is affected by these dimensions, including expertise (coefficient=1.12; P<.001), service delivery process (coefficient=5.60; P<.001), attitude (coefficient=0.82; P<.001), empathy (coefficient=2.65; P<.001), and outcome (coefficient=0.26; P<.001; through patients' perceived quality from reviews). The 5 dimensions can explain 85.52% of the variance in patient demand, while all information from online reviews can explain 85.67%. The results show the validity and the potential practical value of the proposed HSQ-5D model.
This study explores how online reviews can be used to evaluate health care services, offering significant implications for health care management. Theoretically, we extend existing service quality frameworks by integrating text-mining analysis of online reviews, thereby enhancing the understanding of service quality assessment in the digital health context. Practically, the framework can allow health care platforms to identify and reveal doctors' service quality to reduce patients' information asymmetry and strengthen patient-provider relationships, ultimately contributing to a more effective and patient-centered health care system.
Journal Article
Spanish Version of the mHealth App Usability Questionnaire (S-MAUQ): Translation, Adaptation, and Validation Study (Preprint)
by
Díaz Mohedo, Esther
,
Carrillo León, Antonio L.
,
Falgueras Cano, Juan
in
Development and Evaluation of Research Methods, Instruments and Tools
,
Development of Instruments and Surveys
,
Evaluation and Research Methodology for mHealth
2026
In the current digital landscape, ensuring optimal usability is one of the most crucial factors determining the success of any mobile app. Questionnaire-based usability evaluations represent a highly prevalent methodology for this purpose. To date, questionnaires have been developed to assess the general system usability; however, there are hardly any questionnaires specifically designed to assess the usability of mobile health (mHealth) apps. The most widespread, the mHealth App Usability Questionnaire (MAUQ), has been developed in 4 versions according to the type of app (interactive or standalone) and the target user (patient or provider).
The objective of this study was to translate and validate the English version of the MAUQ (standalone, for patients) into a Spanish version (S-MAUQ).
The methodology used here follows that proposed by Sousa and Rojjanasrirat, which comprises 4 stages. The initial stage of the process entails a translation, harmonization, and adaptation procedure. The second and third entailed content validation (by 10 experts) and face validation (by 12 target users), respectively, which were conducted to evaluate the relevance and clarity of the questionnaire items. The item-level content validity index, scale content validity index (S-CVI), item-level face validity index, and scale face validity index (S-FVI), as well as the modified kappa statistic (κ) were used to evaluate interrater agreement among the raters, considering the probability of agreement by chance (Pc). The fourth and final stage of the process involved the assessment of the questionnaire's reliability. A sample of 61 young adult participants installed an mHealth app (the Yazio app), used it, and responded to the S-MAUQ. The Cronbach α value for the entire questionnaire and its subscales were then calculated.
For the second stage, the S-CVI was initially 0.778. We removed items #14 and #15 from the Spanish version as they were unclear and not relevant. The S-CVI changed to 0.881. The third stage had an S-FVI of 0.927, indicating that the items are clear and straightforward for the nonexpert target user to understand. Furthermore, with each κ value >0.74, the validity of the instrument is supported. The fourth stage demonstrated the reliability of the S-MAUQ with a Cronbach α value of 0.87.
The final version of the S-MAUQ met the validation criteria, demonstrating reliability and validity that are comparable with those of the original version. Consequently, the S-MAUQ is suitable for evaluating the usability of mHealth apps for young Spanish adults. Further research involving larger and more diverse samples is recommended.
Journal Article
Beyond Benchmarks: Evaluating Generalist Medical Artificial Intelligence With Psychometrics
by
Jiang, Liming
,
Sun, Luning
,
Wang, Xiting
in
Accuracy
,
Alternative approaches
,
Applications of AI
2025
Rigorous evaluation of generalist medical artificial intelligence (GMAI) is imperative to ensure their utility and safety before implementation in health care. Current evaluation strategies rely heavily on benchmarks, which can suffer from issues with data contamination and cannot explain how GMAI might fail (lacking explanatory power) or in what circumstances (lacking predictive power). To address these limitations, we propose a new methodology to improve the quality of GMAI evaluation using construct-oriented processes. Drawing on modern psychometric techniques, we introduce approaches to construct identification and present alternative assessment formats for different domains of professional skills, knowledge, and behaviors that are essential for safe practice. We also discuss the need for human oversight in future GMAI adoption.
Journal Article
Developing Requirements for a Digital Self-Care Intervention for Adults With Heart Failure: Qualitative Workshop Study
2025
Heart failure is a complex syndrome that requires long-term management, including self-care, to prevent decompensation and hospitalization. Although a range of interventions exists, evidence supporting their effectiveness remains limited, prompting calls for more theory-informed and user-centered approaches. The rapid advancement of mobile and digital technologies offers new opportunities to improve self-care, particularly when interventions are grounded in behavioral theory and shaped by user input.
This study aimed to define user-centered, theory-informed requirements for a digital intervention to support self-care among people with heart failure. We combined the Behavior Change Wheel (BCW) with user-centered design (UCD) to explore self-care barriers and generate actionable intervention requirements.
A qualitative study was conducted, involving 4 workshops with people with heart failure (n=16) and informal caregivers (n=4) across metropolitan and regional Australia. Guided by UCD principles, the workshops explored self-care barriers and elicited ideas for a digital intervention. Barriers were coded using the capability, opportunity, motivation, and behavior (COM-B) model and the Theoretical Domains Framework to identify behavioral determinants and user needs. Ideas and preferences for the intervention were analyzed using requirements analysis and affinity mapping to generate themes describing intervention components (\"what\" the system should do) and user requirements (\"how\" it should operate). Intervention components were then mapped to relevant BCW intervention functions.
Participants identified self-care barriers across all 3 COM-B components and 11 of 14 Theoretical Domains Framework domains, including barriers related to capability (eg, lack of knowledge and forgetfulness), opportunity (eg, busy lifestyle and limited access to resources), and motivation (eg, emotional burden and lack of confidence). These were translated into 28 distinct user needs. From participants' ideas, 6 themes relating to intervention components were identified: education, monitoring and feedback, social connection and support, psychological and emotional support, planning and preparing, and health care support. These components mapped to 7 BCW functions: education, persuasion, incentivization, training, environmental restructuring, modeling, and enablement. Additionally, 6 user requirement themes were developed: physical design, accessibility and usability, personalization and control, engagement and user experience, support and implementation, and integration and system organization.
This study demonstrates the value of integrating UCD with the BCW to develop intervention requirements that are both user-centered and theoretically grounded. By exploring both what the intervention should do and how it should do it, we identified actionable requirements that bridge the gap between understanding behavior and developing effective solutions. Future work can focus on translating these requirements into prototype interventions and evaluating their feasibility, acceptability, and effectiveness.
Journal Article
Developing an Evaluation Index System for Service Capability of Internet Hospitals in China: Mixed Methods Study
by
Xia, Mingge
,
Li, Min
,
Ma, Li
in
Adoption and Change Management of eHealth Systems
,
China
,
Contract manufacturing
2025
Rapid advancements in web-based technology have significantly transformed the health care landscape. In China, internet hospitals have emerged as vital components of the health care system. This rapid growth highlights the necessity for a thorough evaluation of internet hospitals within the health care system, as they operate under models distinct from traditional health care settings.
This study aimed to identify critical indicators that reflect the service capabilities of internet hospitals and to establish a comprehensive evaluation index system for their assessment.
This study initially compiled a pool of indicators through literature review and expert consultation. The final evaluation index system was established using the Delphi method, involving 2 rounds of expert consultation, and the index weights were determined using the Analytic Hierarchy Process.
In total, 21 experts from relevant fields, such as hospital management, clinical services, and health information management, were enrolled in the consultation. After 2 rounds of Delphi consultation, the experts' positive coefficients were 95.45% and 100%, and the authoritative coefficients were both >0.7. The final evaluation index system for the service capabilities of internet hospitals contained 3 first-level indicators, 9 second-level indicators, and 29 third-level indicators. The first-level indicators were categorized into 3 dimensions: \"Internet hospital infrastructure,\" \"Internet hospital services,\" and \"Internet hospital management.\" \"Internet hospital infrastructure\" encompasses the essential conditions for service delivery, such as hardware and software resources, human capital, information security, and payment systems. \"Internet hospital services\" focuses on the scope and depth of services offered, such as \"online medical services,\" \"online pharmaceutical services,\" and \"collaborative services.\" Finally, \"Internet hospital management\" is divided into \"medical administration\" and \"general management.\" Weights were assigned to each indicator using the Analytic Hierarchy Process, revealing that \"Internet hospital services\" held the highest importance (0.573) among 3 first-level indicators, followed by \"Internet hospital infrastructure\" (0.239) and \"Internet hospital management\" (0.188). Among the second-level indicators, \"Online medical service\" emerged as the most critical (0.344), followed by \"Medical administration\" (0.140), \"Online pharmaceutical service\" (0.119), \"Collaboration service\" (0.110), and \"Information security\" (0.087). Among the third-level indicators, \"Online health consultation\" (0.092) had the highest weight, followed by \"Online chronic disease management\" (0.080), \"Online pharmaceutical consultation\" (0.076), \"Consistency between online medical service and offline medical service\" (0.071), and \"Medical quality management\" (0.071).
This study identified and established a comprehensive evaluation index system for assessing the service capabilities of internet hospitals in China. The resulting index system not only provides a valuable tool for evaluating and improving service delivery in internet hospitals but also serves as a foundation for future studies in this rapidly evolving field.
Journal Article
Validity and Reliability of the Psychodynamic Organizational Diagnostic Instrument SyMOA: Protocol for a Mixed Methods Study
by
Lahmann, Claas
,
Sachs, Sophia
,
Rieder, Yannik
in
Design, Usability, and Evaluation of Research Instruments, Scales, and Measures
,
Development and Evaluation of Research Methods, Instruments and Tools
,
Humans
2026
A comprehensive understanding of organizations is fundamental for implementing successful change measures. However, to date, there is no empirically testable, operationalized systems-psychodynamic organizational diagnostic method that can capture the deeper, more complex dynamics that are crucial for sustainable transformation. To address this gap, we developed the Systematic Multidimensional Organizational Assessment (SyMOA), a qualitative instrument based on an evidence-based clinical diagnostic framework, the Operationalized Psychodynamic Diagnostics III. SyMOA integrates clinical, organizational, and systemic psychodynamic theory and analyzes an organization's challenges based on invisible and unconscious aspects, that is, those lurking beneath the surface. It hypothesizes 3 organizational dimensions: (1) current challenges based on the sociotechnical integration and organizational internal functioning level, (2) internal relationship dynamics, and (3) unconscious organizational conflicts. The SyMOA dimensions are operationalized into a semistructured interview guide and coding protocols for the analysis of the content. By capturing the underlying dynamics, SyMOA aims to provide a deeper understanding of an organization's challenges and establish a solid foundation for targeted interventions.
This study aims to evaluate the validity and intercoder reliability of the SyMOA instrument.
For this purpose, semistructured interviews will be conducted with employees of at least 3 different companies in Germany. The evaluation will be carried out by calculating Krippendorff α to determine intercoder reliability. In addition, construct validity, content validity, and external validity will also be analyzed.
Recruitment and training commenced in May 2025. Data collection is planned for the second half of 2025, with analysis to follow thereafter. As this is a study protocol, no results are available yet. At the time of submission, 46 participants have been recruited.
This study will give methodological insights into the validity, reliability, feasibility, and acceptability of the SyMOA instrument. The findings are expected to help further instrument refinement and inform the application of systems-psychodynamic approaches in organizational diagnostics.
Journal Article