Catalogue Search | MBRL
Search Results Heading
Explore the vast range of titles available.
MBRLSearchResults
-
DisciplineDiscipline
-
Is Peer ReviewedIs Peer Reviewed
-
Item TypeItem Type
-
SubjectSubject
-
YearFrom:-To:
-
More FiltersMore FiltersSourceLanguage
Done
Filters
Reset
1,517
result(s) for
"Assessors"
Sort by:
Professional power, job security, and decision making in project funding: The assessors' perspective
by
Li, Yanbin
,
Hu, Xiaoting
,
Zhang, Chuqing
in
Assessors
,
Assessors' Professional Power
,
Assessors' Social Connections
2020
We examined the influence of assessors' professional power and social connections on their decision making regarding evaluation for funding of new business projects, as well as the moderating effect of the assessors' own job security. Using 1,517 observations from 2017 Beijing Innofund
data, we found that the assessors' professional power and social connections were both positively related to their evaluation scores. The job security of the assessors significantly positively moderated the effect of their social connections, but the moderating effect of job security on their
professional power was nonsignificant. Theoretical and practical implications are discussed.
Journal Article
Improving the assessment of physical performance in older adults with dementia using a progressive cueing system
2025
Background Older adults with dementia have difficulties following and completing common standardized physical performance measures. Cues are shown to be potentially effective in enhancing their completion rate of certain physical performance measures (e.g., walk tests). Nevertheless, the effects of cues on the completion rate, reliability, and validity of common physical performance measures in this population have not been thoroughly investigated. This study aims to evaluate the effects of a progressive cueing system on the completion rate, test‐retest reliability, and inter‐rater reliability of physical performance measures in older adults with dementia. Method Older adults with dementia who can walk 10 m with or without a walking aid independently were recruited. The participants completed the Berg Balance Scale, the 2‐minute walk test, the 10‐meter walk test – single task and dual task, the 3‐meter backward walk test, and the 5‐time sit‐to‐stand test with or without the support of the progressive cueing system. The progressive cueing system consists of five levels: (0) no cue, (1) verbal cue, (2) modeling, (3) one‐off physical cue, and (4) repeated physical cues. Cues were provided if the participants failed to follow any procedures of the measures (e.g., a sudden discontinuation of movement due to being distracted by the surroundings). Two assessors conducted the physical performance measures on six separate testing occasions within three weeks. The mean percentages of the physical performance measures completed by the participants and the intra‐class correlation coefficients (ICC) of the physical performance measures were evaluated. Result Seventy‐two older adults with dementia were recruited (Table 1). The participants’ completion rate (with cueing = 83 – 90%; without cueing = 46 – 88%), test‐retest reliability (with cueing ICC = 0.69 – 0.91; without cueing ICC = 0.73 – 0.96), and inter‐rater reliability (with cueing = 0.62 – 0.96; without cueing = 0.56 – 0.90) of the physical performance measures were generally higher when the progressive cueing system supported the participants during the measurement (Tables 2 and 3). Conclusion Our findings show that the progressive cueing system is potentially effective in improving the completion rate, test‐retest, and inter‐rater reliability of physical performance measures in older adults with dementia.
Journal Article
AI for IMPACTS Framework for Evaluating the Long-Term Real-World Impacts of AI-Powered Clinician Tools: Systematic Review and Narrative Synthesis
by
Jacob, Christine
,
Laurenzi, Emanuele
,
Heuss, Sabina
in
Acceptability
,
Accountability
,
Accuracy
2025
Artificial intelligence (AI) has the potential to revolutionize health care by enhancing both clinical outcomes and operational efficiency. However, its clinical adoption has been slower than anticipated, largely due to the absence of comprehensive evaluation frameworks. Existing frameworks remain insufficient and tend to emphasize technical metrics such as accuracy and validation, while overlooking critical real-world factors such as clinical impact, integration, and economic sustainability. This narrow focus prevents AI tools from being effectively implemented, limiting their broader impact and long-term viability in clinical practice.
This study aimed to create a framework for assessing AI in health care, extending beyond technical metrics to incorporate social and organizational dimensions. The framework was developed by systematically reviewing, analyzing, and synthesizing the evaluation criteria necessary for successful implementation, focusing on the long-term real-world impact of AI in clinical practice.
A search was performed in July 2024 across the PubMed, Cochrane, Scopus, and IEEE Xplore databases to identify relevant studies published in English between January 2019 and mid-July 2024, yielding 3528 results, among which 44 studies met the inclusion criteria. The systematic review followed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews. Data were analyzed using NVivo through thematic analysis and narrative synthesis to identify key emergent themes in the studies.
By synthesizing the included studies, we developed a framework that goes beyond the traditional focus on technical metrics or study-level methodologies. It integrates clinical context and real-world implementation factors, offering a more comprehensive approach to evaluating AI tools. With our focus on assessing the long-term real-world impact of AI technologies in health care, we named the framework AI for IMPACTS. The criteria are organized into seven key clusters, each corresponding to a letter in the acronym: (1) I-integration, interoperability, and workflow; (2) M-monitoring, governance, and accountability; (3) P-performance and quality metrics; (4) A-acceptability, trust, and training; (5) C-cost and economic evaluation; (6) T-technological safety and transparency; and (7) S-scalability and impact. These are further broken down into 28 specific subcriteria.
The AI for IMPACTS framework offers a holistic approach to evaluate the long-term real-world impact of AI tools in the heterogeneous and challenging health care context and lays the groundwork for further validation through expert consensus and testing of the framework in real-world health care settings. It is important to emphasize that multidisciplinary expertise is essential for assessment, yet many assessors lack the necessary training. In addition, traditional evaluation methods struggle to keep pace with AI's rapid development. To ensure successful AI integration, flexible, fast-tracked assessment processes and proper assessor training are needed to maintain rigorous standards while adapting to AI's dynamic evolution.
reviewregistry1859; https://tinyurl.com/ysn2d7sh.
Journal Article
More than doubling the clinical benefit of each hour of therapist time: a randomised controlled trial of internet cognitive therapy for social anxiety disorder
2023
Cognitive therapy for social anxiety disorder (CT-SAD) is recommended by NICE (2013) as a first-line intervention. Take up in routine services is limited by the need for up to 14 ninety-min face-to-face sessions, some of which are out of the office. An internet-based version of the treatment (iCT-SAD) with remote therapist support may achieve similar outcomes with less therapist time.
102 patients with social anxiety disorder were randomised to iCT-SAD, CT-SAD, or waitlist (WAIT) control, each for 14 weeks. WAIT patients were randomised to the treatments after wait. Assessments were at pre-treatment/wait, midtreatment/wait, posttreatment/wait, and follow-ups 3 & 12 months after treatment. The pre-registered (ISRCTN 95 458 747) primary outcome was the social anxiety disorder composite, which combines 6 independent assessor and patient self-report scales of social anxiety. Secondary outcomes included disability, general anxiety, depression and a behaviour test.
CT-SAD and iCT-SAD were both superior to WAIT on all measures. iCT-SAD did not differ from CT-SAD on the primary outcome at post-treatment or follow-up. Total therapist time in iCT-SAD was 6.45 h. CT-SAD required 15.8 h for the same reduction in social anxiety. Mediation analysis indicated that change in process variables specified in cognitive models accounted for 60% of the improvements associated with either treatment. Unlike the primary outcome, there was a significant but small difference in favour of CT-SAD on the behaviour test.
When compared to conventional face-to-face therapy, iCT-SAD can more than double the amount of symptom change associated with each therapist hour.
Journal Article
Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors
by
Ravaud, Philippe
,
Boutron, Isabelle
,
Tendal, Britta
in
Arthritis
,
Clinical trials
,
Double-Blind Method
2012
Objective To evaluate the impact of non-blinded outcome assessment on estimated treatment effects in randomised clinical trials with binary outcomes.Design Systematic review of trials with both blinded and non-blinded assessment of the same binary outcome. For each trial we calculated the ratio of the odds ratios—the odds ratio from non-blinded assessments relative to the corresponding odds ratio from blinded assessments. A ratio of odds ratios <1 indicated that non-blinded assessors generated more optimistic effect estimates than blinded assessors. We pooled the individual ratios of odds ratios with inverse variance random effects meta-analysis and explored reasons for variation in ratios of odds ratios with meta-regression. We also analysed rates of agreement between blinded and non-blinded assessors and calculated the number of patients needed to be reclassified to neutralise any bias.Data Sources PubMed, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press, and Google Scholar.Eligibility criteria for selecting studies Randomised clinical trials with blinded and non-blinded assessment of the same binary outcome.Results We included 21 trials in the main analysis (with 4391 patients); eight trials provided individual patient data. Outcomes in most trials were subjective—for example, qualitative assessment of the patient’s function. The ratio of the odds ratios ranged from 0.02 to 14.4. The pooled ratio of odds ratios was 0.64 (95% confidence interval 0.43 to 0.96), indicating an average exaggeration of the non-blinded odds ratio by 36%. We found no significant association between low ratios of odds ratios and scores for outcome subjectivity (P=0.27); non-blinded assessor’s overall involvement in the trial (P=0.60); or outcome vulnerability to non-blinded patients (P=0.52). Blinded and non-blinded assessors agreed in a median of 78% of assessments (interquartile range 64-90%) in the 12 trials with available data. The exaggeration of treatment effects associated with non-blinded assessors was induced by the misclassification of a median of 3% of the assessed patients per trial (1-7%).Conclusions On average, non-blinded assessors of subjective binary outcomes generated substantially biased effect estimates in randomised clinical trials, exaggerating odds ratios by 36%. This bias was compatible with a high rate of agreement between blinded and non-blinded outcome assessors and driven by the misclassification of few patients.
Journal Article
A Review of Mercury Bioavailability in Humans and Fish
2017
To estimate human exposure to methylmercury (MeHg), risk assessors often assume 95%–100% bioavailability in their models. However, recent research suggests that assuming all, or most, of the ingested mercury (Hg) is absorbed into systemic circulation may be erroneous. The objective of this paper is to review and discuss the available state of knowledge concerning the assimilation or bioavailability of Hg in fish and humans. In fish, this meant reviewing studies on assimilation efficiency, that is the difference between ingested and excreted Hg over a given period of time. In humans, this meant reviewing studies that mostly investigated bioaccessibility (digestive processes) rather than bioavailability (cumulative digestive + absorptive processes), although studies incorporating absorption for a fuller picture of bioavailability were also included where possible. The outcome of this review shows that in a variety of organisms and experimental models that Hg bioavailability and assimilation is less than 100%. Specifically, 25 studies on fish were reviewed, and assimilation efficiencies ranged from 10% to 100% for MeHg and from 2% to 51% for Hg(II). For humans, 20 studies were reviewed with bioaccessibility estimates ranging from 2% to 100% for MeHg and 0.2% to 94% for Hg(II). The overall absorption estimates ranged from 12% to 79% for MeHg and 49% to 69% for Hg(II), and were consistently less than 100%. For both fish and humans, a number of cases are discussed in which factors (e.g., Hg source, cooking methods, nutrients) are shown to affect Hg bioavailability. The summaries presented here challenge a widely-held assumption in the Hg risk assessment field, and the paper discusses possible ways forward for the field.
Journal Article
Moving beyond tinkering round the edges: A systematic review of the role of the mentor in the assessment of nursing students in practice
by
Lindsay, Iain
,
Timba-Emmanuel, Tabeth
,
Drew, Karen
in
Assessors
,
Check Lists
,
Clinical Competence - standards
2026
To analyse the literature from the introduction of Project 2000, providing insight into the evolution of the role of the mentor and the understanding of assessment in practice, highlighting areas for development.
The challenges of assessment in practice have been the subject of much research, but none taking a historical perspective of the UK experience.
The following databases were searched from 1990 to 2024; CINAHL+ , PubMed, APA PsycArticles, SCOPUS, Web of Science and Google Scholar.
First level screening of titles and abstracts was undertaken by 3 reviewers (n = 7888). Second level screening on full text articles (n = 145) identified 33 articles for inclusion. A rerun from 2022 to 2024 produced 3 more articles for inclusion. Quality appraisal excluded 6 articles, leaving 30 articles for inclusion in the review.
Articles were categorised into pre- and post-introduction of the NMC standards and codes identified from findings. Themes and sub-themes were identified, with four factors emerging as integral to effective assessment in practice. Lack of a clear, shared definition of competence is reflected in confusing documentation. Time to undertake assessments and trusted support were also highlighted.
While standards (NMC, 2008) brought about important changes, this review has highlighted the need for an agreed definition of competence to guide the development of appropriate assessment documentation. Research exploring the benefits in building trusting relationships through the tripartite assessment process is also needed.
Only views of UK assessors were considered, while timespan and heterogeneity of data may influence the quality and generalisability of findings.
Journal Article
Physician Versus Large Language Model Chatbot Responses to Web-Based Questions From Autistic Patients in Chinese: Cross-Sectional Comparative Analysis
2024
There is a dearth of feasibility assessments regarding using large language models (LLMs) for responding to inquiries from autistic patients within a Chinese-language context. Despite Chinese being one of the most widely spoken languages globally, the predominant research focus on applying these models in the medical field has been on English-speaking populations.
This study aims to assess the effectiveness of LLM chatbots, specifically ChatGPT-4 (OpenAI) and ERNIE Bot (version 2.2.3; Baidu, Inc), one of the most advanced LLMs in China, in addressing inquiries from autistic individuals in a Chinese setting.
For this study, we gathered data from DXY-a widely acknowledged, web-based, medical consultation platform in China with a user base of over 100 million individuals. A total of 100 patient consultation samples were rigorously selected from January 2018 to August 2023, amounting to 239 questions extracted from publicly available autism-related documents on the platform. To maintain objectivity, both the original questions and responses were anonymized and randomized. An evaluation team of 3 chief physicians assessed the responses across 4 dimensions: relevance, accuracy, usefulness, and empathy. The team completed 717 evaluations. The team initially identified the best response and then used a Likert scale with 5 response categories to gauge the responses, each representing a distinct level of quality. Finally, we compared the responses collected from different sources.
Among the 717 evaluations conducted, 46.86% (95% CI 43.21%-50.51%) of assessors displayed varying preferences for responses from physicians, with 34.87% (95% CI 31.38%-38.36%) of assessors favoring ChatGPT and 18.27% (95% CI 15.44%-21.10%) of assessors favoring ERNIE Bot. The average relevance scores for physicians, ChatGPT, and ERNIE Bot were 3.75 (95% CI 3.69-3.82), 3.69 (95% CI 3.63-3.74), and 3.41 (95% CI 3.35-3.46), respectively. Physicians (3.66, 95% CI 3.60-3.73) and ChatGPT (3.73, 95% CI 3.69-3.77) demonstrated higher accuracy ratings compared to ERNIE Bot (3.52, 95% CI 3.47-3.57). In terms of usefulness scores, physicians (3.54, 95% CI 3.47-3.62) received higher ratings than ChatGPT (3.40, 95% CI 3.34-3.47) and ERNIE Bot (3.05, 95% CI 2.99-3.12). Finally, concerning the empathy dimension, ChatGPT (3.64, 95% CI 3.57-3.71) outperformed physicians (3.13, 95% CI 3.04-3.21) and ERNIE Bot (3.11, 95% CI 3.04-3.18).
In this cross-sectional study, physicians' responses exhibited superiority in the present Chinese-language context. Nonetheless, LLMs can provide valuable medical guidance to autistic patients and may even surpass physicians in demonstrating empathy. However, it is crucial to acknowledge that further optimization and research are imperative prerequisites before the effective integration of LLMs in clinical settings across diverse linguistic environments can be realized.
Chinese Clinical Trial Registry ChiCTR2300074655; https://www.chictr.org.cn/bin/project/edit?pid=199432.
Journal Article
Effects of Virtual Reality Motor-Cognitive Training for Older People With Cognitive Frailty: Multicentered Randomized Controlled Trial
2024
Cognitive frailty refers to a clinical syndrome in which physical frailty and mild cognitive impairment coexist. Motor-cognitive training and virtual reality (VR) have been used to launch various therapeutic modalities to promote health in older people. The literature advocates that motor-cognitive training and VR are effective in promoting the cognitive and physical function of older people. However, the effects on older people with cognitive frailty are unclear.
This study examined the effects of VR motor-cognitive training (VRMCT) on global cognitive function, physical frailty, walking speed, visual short-term memory, inhibition of cognitive interference, and executive function in older people with cognitive frailty.
This study used a multicentered, assessor-blinded, 2-parallel-group randomized controlled trial design. Participants were recruited face-to-face in 8 older adult community centers. Eligible participants were aged ≥60 years, were community dwelling, lived with cognitive frailty, had no dementia, and were not mobility restricted. In the intervention group, participants received VRMCT led by interventionists with 16 one-hour training sessions delivered twice per week for 8 weeks. In the control group, participants received the usual care provided by the older adult community centers that the investigators did not interfere with. The primary outcome was global cognitive function. The secondary outcomes included physical frailty, walking speed, verbal short-term memory, inhibition of cognitive interference, and executive function. Data were collected at baseline (T0) and the week after the intervention (T1). Generalized estimating equations were used to examine the group, time, and interaction (time × group) effects on the outcomes.
In total, 293 eligible participants enrolled in the study. The mean age of the participants was 74.5 (SD 6.8) years. Most participants were female (229/293, 78.2%), had completed primary education (152/293, 52.1%), were married (167/293, 57.2%), lived with friends (127/293, 43.3%), and had no VR experience (232/293, 79.5%). In the intervention group, 81.6% (119/146) of participants attended >80% (13/16, 81%) of the total number of sessions. A negligible number of participants experienced VR sickness symptoms (1/146, 0.7% to 5/146, 3%). VRMCT was effective in promoting global cognitive function (interaction effect: P=.03), marginally promoting executive function (interaction effect: P=.07), and reducing frailty (interaction effect: P=.03). The effects were not statistically significant on other outcomes.
VRMCT is effective in promoting cognitive functions and reducing physical frailty and is well tolerated and accepted by older people with cognitive frailty, as evidenced by its high attendance rate and negligible VR sickness symptoms. Further studies should examine the efficacy of the intervention components (eg, VR vs non-VR or dual task vs single task) on health outcomes, the effect of using technology on intervention adherence, and the long-term effects of the intervention on older people with cognitive frailty at the level of daily living.
ClinicalTrials.gov NCT04730817; https://clinicaltrials.gov/study/NCT04730817.
Journal Article
The Effects of Immersive Virtual Reality Applications on Enhancing the Learning Outcomes of Undergraduate Health Care Students: Systematic Review With Meta-synthesis
2023
Immersive virtual reality (IVR) applications are gaining popularity in health care education. They provide an uninterrupted, scaled environment capable of simulating the full magnitude of sensory stimuli present in busy health care settings and increase students' competence and confidence by providing them with accessible and repeatable learning opportunities in a fail-safe environment.
This systematic review aimed to evaluate the effects of IVR teaching on the learning outcomes and experiences of undergraduate health care students compared with other teaching methods.
MEDLINE, Embase, PubMed, and Scopus were searched (last search on May 2022) for randomized controlled trials (RCTs) or quasi-experimental studies published in English between January 2000 and March 2022. The inclusion criteria were studies involving undergraduate students majoring in health care, IVR teaching, and evaluations of students' learning outcomes and experiences. The methodological validity of the studies was examined using the Joanna Briggs Institute standard critical appraisal instruments for RCTs or quasi-experimental studies. The findings were synthesized without a meta-analysis using vote counting as the synthesis metric. A binomial test with P<.05 was used to test for statistical significance using SPSS (version 28; IBM Corp). The overall quality of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation tool.
A total of 17 articles from 16 studies totaling 1787 participants conducted between 2007 and 2021 were included. The undergraduate students in the studies majored in medicine, nursing, rehabilitation, pharmacy, biomedicine, radiography, audiology, or stomatology. The IVR teaching domains included procedural training (13/16, 81%), anatomical knowledge (2/16, 12%), and orientation to the operating room setting (1/16, 6%). The quality of the 75% (12/16) of RCT studies was poor, with unclear descriptions of randomization, allocation concealment, and outcome assessor blinding procedures. The overall risk of bias was relatively low in the 25% (4/16) of quasi-experimental studies. A vote count showed that 60% (9/15; 95% CI 16.3%-67.7%; P=.61) of the studies identified similar learning outcomes between IVR teaching and other teaching approaches regardless of teaching domains. The vote count showed that 62% (8/13) of the studies favored using IVR as a teaching medium. The results of the binomial test (95% CI 34.9%-90%; P=.59) did not show a statistically significant difference. Low-level evidence was identified based on the Grading of Recommendations Assessment, Development, and Evaluation tool.
This review found that undergraduate students had positive learning outcomes and experiences after engaging with IVR teaching, although the effects may be similar to those of other forms of virtual reality or conventional teaching methods. Given the identification of risk of bias and low level of the overall evidence, more studies with a larger sample size and robust study design are required to evaluate the effects of IVR teaching.
International prospective register of systematic reviews (PROSPERO) CRD42022313706; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=313706.
Journal Article