Explore the vast range of titles available.

Background Artificial intelligence (AI) chatbots are emerging educational tools for students in healthcare science. However, assessing their accuracy is essential prior to adoption in educational settings. This study aimed to assess the accuracy of predicting the correct answers from three AI chatbots (ChatGPT-4, Microsoft Copilot and Google Gemini) in the Italian entrance standardized examination test of healthcare science degrees (CINECA test). Secondarily, we assessed the narrative coherence of the AI chatbots’ responses (i.e., text output) based on three qualitative metrics: the logical rationale behind the chosen answer, the presence of information internal to the question, and presence of information external to the question. Methods An observational cross-sectional design was performed in September of 2023. Accuracy of the three chatbots was evaluated for the CINECA test, where questions were formatted using a multiple-choice structure with a single best answer. The outcome is binary (correct or incorrect). Chi-squared test and a post hoc analysis with Bonferroni correction assessed differences among chatbots performance in accuracy. A p -value of < 0.05 was considered statistically significant. A sensitivity analysis was performed, excluding answers that were not applicable (e.g., images). Narrative coherence was analyzed by absolute and relative frequencies of correct answers and errors. Results Overall, of the 820 CINECA multiple-choice questions inputted into all chatbots, 20 questions were not imported in ChatGPT-4 ( n  = 808) and Google Gemini ( n  = 808) due to technical limitations. We found statistically significant differences in the ChatGPT-4 vs Google Gemini and Microsoft Copilot vs Google Gemini comparisons ( p -value < 0.001). The narrative coherence of AI chatbots revealed “Logical reasoning” as the prevalent correct answer ( n  = 622, 81.5%) and “Logical error” as the prevalent incorrect answer ( n  = 40, 88.9%). Conclusions Our main findings reveal that: (A) AI chatbots performed well; (B) ChatGPT-4 and Microsoft Copilot performed better than Google Gemini; and (C) their narrative coherence is primarily logical. Although AI chatbots showed promising accuracy in predicting the correct answer in the Italian entrance university standardized examination test, we encourage candidates to cautiously incorporate this new technology to supplement their learning rather than a primary resource. Trial registration Not required.

Background Artificial Intelligence (AI) Chatbots (e.g., ChatGPT, Microsoft Bing, and Google Bard) can emulate human interaction and may support physiotherapy education. Despite growing interest, physiotherapy students’ perspectives remain unexplored. This study investigated Italian physiotherapy students’ knowledge, use, and perception of the benefits and limitations of AI Chatbots. Methods A cross-sectional study was conducted through Survey Monkey from February to June 2024. One thousand five hundred and thirty-one physiotherapy students from 10 universities were involved. The survey consisted of 23 questions investigating: (a) respondent characteristics, (b) AI Chatbot knowledge and use, (c) perceived benefits, and (d) limitations. Multiple-choice and Likert-scale-based questions were adopted. Factors associated with knowledge, use, and perceptions of AI were explored using logistic regression models. Results Of 589 students (38%) that completed the survey, most were male ( n  = 317; 53.8%) with a mean age of 22 years (SD = 3.88). Nearly all ( n  = 561; 95.3%) had heard of AI Chatbots, but 53.7% ( n  = 316) never used these tools for academic purposes. Among users, learning support was the most common purpose ( n  = 187; 31.8%), while only 9.9% ( n  = 58) declared Chatbot use during internships. Students agreed that Chatbots have limitations in performing complex tasks and may generate inaccurate results (median = 3 out of 4). However, they neither agreed nor disagreed about Chatbots’ impact on academic performance, emotional intelligence, bias, and fairness (median = 2 out of 4). The students agreed to identify the risk of misinformation as a primary barrier (median = 3 out of 4). In contrast, they neither agreed nor disagreed on content validity, plagiarism, privacy, and impacts on critical thinking and creativity (median = 2 out of 4). Young students had 11% more odds of being familiar with Chatbots than older students (OR = 0.89; 95%CI 0.84–0.95; p  = < 0.01), whereas female students had 39% lesser odds than males to have used Chatbots for academic purposes (OR = 0.61; 95%CI 0.44–0.85; p  = < 0.01). Conclusions While most students recognize the potential of AI Chatbots, they express caution about their use in academia. Targeted training for students and faculty, supported by institutional and national guidelines, could guarantee a responsible integration of these technologies into physiotherapy education. Trial registration Not applicable.

Artificial Intelligence (AI) chatbots, which generate human-like responses based on extensive data, are becoming important tools in healthcare by providing information on health conditions, treatments, and preventive measures, acting as virtual assistants. However, their performance in aligning with clinical practice guidelines (CPGs) for providing answers to complex clinical questions on lumbosacral radicular pain is still unclear. We aim to evaluate AI chatbots' performance against CPG recommendations for diagnosing and treating lumbosacral radicular pain. We performed a cross-sectional study to assess AI chatbots' responses against CPGs recommendations for diagnosing and treating lumbosacral radicular pain. Clinical questions based on these CPGs were posed to the latest versions (updated in 2024) of six AI chatbots: ChatGPT-3.5, ChatGPT-4o, Microsoft Copilot, Google Gemini, Claude, and Perplexity. The chatbots' responses were evaluated for (a) consistency of text responses using Plagiarism Checker X, (b) intra- and inter-rater reliability using Fleiss' Kappa, and (c) match rate with CPGs. Statistical analyses were performed with STATA/MP 16.1. We found high variability in the text consistency of AI chatbot responses (median range 26%-68%). Intra-rater reliability ranged from \"almost perfect\" to \"substantial,\" while inter-rater reliability varied from \"almost perfect\" to \"moderate.\" Perplexity had the highest match rate at 67%, followed by Google Gemini at 63%, and Microsoft Copilot at 44%. ChatGPT-3.5, ChatGPT-4o, and Claude showed the lowest performance, each with a 33% match rate. Despite the variability in internal consistency and good intra- and inter-rater reliability, the AI Chatbots' recommendations often did not align with CPGs recommendations for diagnosing and treating lumbosacral radicular pain. Clinicians and patients should exercise caution when relying on these AI models, since one to two-thirds of the recommendations provided may be inappropriate or misleading according to specific chatbots.

Artificial Intelligence (AI) Chatbots are increasingly being integrated into healthcare, but little is known about their role in physiotherapy. This study investigated the knowledge and use, perceived benefits, limits, and barriers of AI Chatbots in the Italian physiotherapy community. A cross-sectional survey was conducted between March and July 2024. Italian physiotherapists, members of the Associazione Italiana di Fisioterapia (AIFI), were invited through mailing lists and social media. Inclusion criteria: AIFI membership, current employment as a physiotherapist, Italian language proficiency, and willingness to participate. A total of 415 out of 2,773 physiotherapists responded (15% response rate); 50.6% were women, and 50.4% had more than 10 years of experience. The survey comprised four sections: (a) respondent characteristics; (b) knowledge and use of AI Chatbots; (c) perception of benefits; and (d) perception of limits and barriers. Descriptive statistics and multivariable logistic regression analyses were performed. Overall, 93.3% of physiotherapists had heard of AI Chatbots, but 66.9% had never used them in clinical practice. Among those who had, 11.3% reported a \"positive\" and 14.5% a \"very positive\" experience. Despite limited use, 78% expressed a positive attitude towards future adoption, and 50% considered AI Chatbots potentially helpful in clinical practice. Reported risks included patient self-diagnosis (84.4%), spread of false information (72.1%), and reduced human interaction (64%). Having more than 21 years of experience was significantly associated with a higher frequency of AI Chatbot use (OR: 5.93,  = 0.013). Age was also a significant predictor of use frequency (OR: 1.05,  = 0.013), with older physiotherapists reporting more frequent AI use. Italian physiotherapists acknowledged both opportunities and risks in implementing AI Chatbots. Although current adoption is limited, the overall positive attitude suggests a likely increase in future use. Targeted strategies, including guidelines and educational initiatives, are needed to ensure safe and effective integration into clinical practice.