Journal Article

ChatGPT's competence in responding to urological emergencies/Urolojik acil durumlarda ChatGPT'nin yanit yetkinligi

Ergul, Rifat Burak, Sarilar, Omer, Ozervarli, Muhammet Firat, Ozgor, Faruk, Yazili, Huseyin Burak, Ortac, Mazhar, Tonyali, Senol

2025

Overview

BACKGROUND: In recent years, artificial intelligence (AI) applications have been increasingly used as sources of medical information, alongside their applications in many other fields. This study is the first to evaluate ChatGPT's performance in addressing urological emergencies (UE). METHODS: The study included frequently asked questions (FAQs) by the public regarding UE, as well as UE-related questions formulated based on the European Association of Urology (EAU) guidelines. The FAQs were selected from questions posed by patients to doctors and hospital accounts on social media platforms (Facebook, Instagram, and X) and on websites. All questions were presented to ChatGPT 4 (premium version) in English, and the responses were recorded. Two urologists assessed the quality of the responses using a Global Quality Score (GQS) on a scale of 1 to 5. RESULTS: Of the 73 total FAQs, 53 (72.6%) received a GQS score of 5, while only two (2.7%) received a GQS score of 1. The questions with a GQS score of 1 pertained to priapism and urosepsis. The topic with the highest proportion of responses receiving a GQS score of 5 was urosepsis (82.3%), whereas the lowest scores were observed in questions related to renal trauma (66.7%) and postrenal acute kidney injury (66.7%). A total of 42 questions were formulated based on the EAU guidelines, of which 23 (54.8%) received a GQS score of 5 from the physicians. The mean GQS score for FAQs was 4.38 [+ or -] 1.14, which was significantly higher (p=0.009) than the mean GQS score for EAU guideline-based questions (3.88 [+ or -] 1.47). CONCLUSION: This study demonstrated for the first time that nearly three out of four FAQs were answered accurately and satisfactorily by ChatGPT However, the accuracy and proficiency of ChatGPT's responses significantly decreased when addressing guideline-based questions on UE. Keywords: Artificial intelligence; ChatGPT; urological emergencies. AMAC: Son yillarda, yapay zeka (AI) uygulamalari tipta ve bircok diger alanda bir bilgi kaynagi olarak kullanilmaktadir. Bu calisma, ChatGPT'nin urolojik aciller (UA) konusunda gosterdigi performansi degerlendiren ilk calismadir. GEREC VE YONTEM: Calisma, halk tarafindan urolojik acillerle ilgili sikca sorulan sorulari (SSS) ve Avrupa Uroloji Dernegi (EAU) kilavuzlarini incelenerek olusturulan urolojik acillerle ilgili sorulari icermektedir. SSS, sosyal medya (Facebook, Instagram ve X) veya doktor / hastane web sayfalarinda halk tarafindan sorulan sorular arasindan secilmistir. Tum sorular ingilizce olarak ChatGPT 4 (Premium versiyonu) ile sorulmus ve cevaplar kaydedilmistir. iki urolog, yanitlari global kalite puani (GQS) skalasina gore 1-5 puan arasinda degerlendirmistir. BULGULAR: Toplam 73 yanitin 53'u (%72.6) 5 GQS puanina sahipti ve yalnizca 2 yanit (%2.7) 1 GQS puanina sahipti. 1 GQS puanina sahip yanitlar priapizm ve urosepsis ile ilgiliydi. En yuksek GQS puanina (%82.3) sahip konu urosepsis iken, en dusuk puanlar renal travma (%66.7) ve postrenal akut bobrek 15 hasari konularindaydi (%66.7). EAU kilavuzuna dayali olarak olusturulan soru sayisi 42 idi. Bu sorulara olusturulan yanitlarin 23'u (%54.8) hekimlerden 5 GQS puani aldi. SSS'ye yonelik yanitlar icin GQS ortalama puani 4.38 [+ or -] I.I4 idi ve bu, EAU kilavuzuna dayali sorular icin ortalama GQS puanindan (3.88 [+ or -] 1.47) istatistiksel olarak daha yuksekti (p=0.009). SONUC: Bu calisma, ilk kez ChatGPT'nin SSS'lerin yaklasik dortte ucunu dogru ve tatmin edici bir sekilde yanitladigini gostermistir. Buna karsilik, UA hakkinda kilavuz temelli sorulari yanitlarken ChatGPT'nin dogrulugu ve yetkinligi onemli olcude azalmistir. Anahtar sozcukler: ChatGPT; urolojik acil durumlar; yapay zeka.

• Publisher: KARE Publishing
• Subject: Medical advice systems / Patient compliance / Social media