Explore the vast range of titles available.

Large language models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM, an extension of the TRIPOD + artificial intelligence statement, addressing the unique challenges of LLMs in biomedical applications. TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories. Developed through an expedited Delphi process and expert consensus, TRIPOD-LLM emphasizes transparency, human oversight and task-specific performance reporting. We also introduce an interactive website ( https://tripod-llm.vercel.app/ ) facilitating easy guideline completion and PDF generation for submission. As a living document, TRIPOD-LLM will evolve with the field, aiming to enhance the quality, reproducibility and clinical applicability of LLM research in healthcare through comprehensive reporting. TRIPOD-LLM (transparent reporting of a multivariable model for individual prognosis or diagnosis–large language model) is a checklist of items considered essential for good reporting of studies that are developing or evaluating an LLM for use in healthcare settings. It is a ‘living guideline’ that emphasizes transparency, human oversight and task-specific performance reporting.

With emerging innovations in artificial intelligence (AI) poised to substantially impact medical practice, interest in training current and future physicians about the technology is growing. Alongside comes the question of what, precisely, should medical students be taught. While competencies for the clinical usage of AI are broadly similar to those for any other novel technology, there are qualitative differences of critical importance to concerns regarding explainability, health equity, and data security. Drawing on experiences at the University of Toronto Faculty of Medicine and MIT Critical Data’s “datathons”, the authors advocate for a dual-focused approach: combining robust data science-focused additions to baseline health research curricula and extracurricular programs to cultivate leadership in this space.

The study provides a comprehensive review of OpenAI’s Generative Pre-trained Transformer 4 (GPT-4) technical report, with an emphasis on applications in high-risk settings like healthcare. A diverse team, including experts in artificial intelligence (AI), natural language processing, public health, law, policy, social science, healthcare research, and bioethics, analyzed the report against established peer review guidelines. The GPT-4 report shows a significant commitment to transparent AI research, particularly in creating a systems card for risk assessment and mitigation. However, it reveals limitations such as restricted access to training data, inadequate confidence and uncertainty estimations, and concerns over privacy and intellectual property rights. Key strengths identified include the considerable time and economic investment in transparent AI research and the creation of a comprehensive systems card. On the other hand, the lack of clarity in training processes and data raises concerns about encoded biases and interests in GPT-4. The report also lacks confidence and uncertainty estimations, crucial in high-risk areas like healthcare, and fails to address potential privacy and intellectual property issues. Furthermore, this study emphasizes the need for diverse, global involvement in developing and evaluating large language models (LLMs) to ensure broad societal benefits and mitigate risks. The paper presents recommendations such as improving data transparency, developing accountability frameworks, establishing confidence standards for LLM outputs in high-risk settings, and enhancing industry research review processes. It concludes that while GPT-4’s report is a step towards open discussions on LLMs, more extensive interdisciplinary reviews are essential for addressing bias, harm, and risk concerns, especially in high-risk domains. The review aims to expand the understanding of LLMs in general and highlights the need for new reflection forms on how LLMs are reviewed, the data required for effective evaluation, and addressing critical issues like bias and risk.

Reports of Large Language Models (LLMs) passing board examinations have spurred medical enthusiasm for their clinical integration. Through a narrative review, we reflect upon the skill shifts necessary for clinicians to succeed in an LLM-enabled world, achieving benefits while minimizing risks. We suggest how medical education must evolve to prepare clinicians capable of navigating human-AI systems.

Large Language Models and the Medical RecordInstead of facilitating communication and transparency, the insertion of LLM-generated text directly into the medical record risks diminishing the quality, efficiency, and humanity of health care.

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper “Machine Learning that Matters”, which highlighted such structural issues in the ML community at large, and offered a series of clearly defined “Impact Challenges” to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare —the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

ObjectivesHealth research that significantly impacts global clinical practice and policy is often published in high-impact factor (IF) medical journals. These outlets play a pivotal role in the worldwide dissemination of novel medical knowledge. However, researchers identifying as women and those affiliated with institutions in low- and middle-income countries (LMICs) have been largely under-represented in high-IF journals across multiple fields of medicine. To evaluate disparities in gender and geographical representation among authors who have published in any of five top general medical journals, we conducted scientometric analyses using a large-scale dataset extracted from the New England Journal of Medicine, Journal of the American Medical Association, The BMJ, The Lancet and Nature Medicine.MethodsAuthor metadata from all articles published in the selected journals between 2007 and 2022 were collected using the DimensionsAI platform. The Genderize.io Application Programming Interface was then used to infer each author’s likely gender based on their extracted first name. The World Bank country classification was used to map countries associated with researcher affiliations to the LMIC or the high-income country (HIC) category. We characterised the overall gender and country income category representation across the five medical journals. In addition, we computed article-level diversity metrics and contrasted their distributions across the journals.ResultsWe studied 151 536 authors across 49 764 articles published in five top medical journals, over a period spanning 15 years. On average, approximately one-third (33.1%) of the authors of a given paper were inferred to be women; this result was consistent across the journals we studied. Further, 86.6% of the teams were exclusively composed of HIC authors; in contrast, only 3.9% were exclusively composed of LMIC authors. The probability of serving as the first or last author was significantly higher if the author was inferred to be a man (18.1% vs 16.8%, p<0.01) or was affiliated with an institution in a HIC (16.9% vs 15.5%, p<0.01). Our primary finding reveals that having a diverse team promotes further diversity, within the same dimension (ie, gender or geography) and across dimensions. Notably, papers with at least one woman among the authors were more likely to also involve at least two LMIC authors (11.7% vs 10.4% in baseline, p<0.001; based on inferred gender); conversely, papers with at least one LMIC author were more likely to also involve at least two women (49.4% vs 37.6%, p<0.001; based on inferred gender).ConclusionWe provide a scientometric framework to assess authorship diversity. Our research suggests that the inclusiveness of high-impact medical journals is limited in terms of both gender and geography. We advocate for medical journals to adopt policies and practices that promote greater diversity and collaborative research. In addition, our findings offer a first step towards understanding the composition of teams conducting medical research globally and an opportunity for individual authors to reflect on their own collaborative research practices and possibilities to cultivate more diverse partnerships in their work.

Background The expectation of pandemic-induced severe resource shortages has prompted authorities to draft and update frameworks to guide clinical decision-making and patient triage. While these documents differ in scope, they share a utilitarian focus on the maximization of benefit. This utilitarian view necessarily marginalizes certain groups, in particular individuals with increased medical needs. Main body Here, we posit that engagement with the disability critique demands that we broaden our understandings of justice and fairness in clinical decision-making and patient triage. We propose the capabilities theory, which recognizes that justice requires a range of positive capabilities/freedoms conducive to the achievement of meaningful life goals, as a means to do so. Informed by a disability rights critique of the clinical response to the pandemic, we offer direction for the construction of future clinical triage protocols which will avoid ableist biases by incorporating a broader apprehension of what it means to be human. Conclusion The clinical pandemic response, codified across triage protocols, should embrace a form of justice which incorporates a vision of pluralistic human capabilities and a valuing of positive freedoms.

The development of artificial intelligence (AI) applications in healthcare is often positioned as a solution to the greatest challenges facing global health. Advocates propose that AI can bridge gaps in care delivery and access, improving healthcare quality and reducing inequity, including in resource-constrained settings. A broad base of critical scholarship has highlighted important issues with healthcare AI, including algorithmic bias and inequitable and inaccurate model outputs. While such criticisms are valid, there exists a much more fundamental challenge that is often overlooked in global health policy debates: the dangerous mismatch between AI’s imagined benefits and the material realities of healthcare systems globally. AI cannot be deployed effectively or ethically in contexts lacking sufficient social and material infrastructure and resources to provide effective healthcare services. Continued investments in AI within unprepared, under-resourced contexts risk misallocating resources and potentially causing more harm than good. The article concludes by providing concrete questions to assess AI systemic capacity and socio-technical readiness in global health.