Explore the vast range of titles available.

Recent advancements in artificial intelligence have witnessed human-level performance; however, AI-enabled cognitive assistance for therapeutic procedures has not been fully explored nor pre-clinically validated. Here we propose AI-Endo, an intelligent surgical workflow recognition suit, for endoscopic submucosal dissection (ESD). Our AI-Endo is trained on high-quality ESD cases from an expert endoscopist, covering a decade time expansion and consisting of 201,026 labeled frames. The learned model demonstrates outstanding performance on validation data, including cases from relatively junior endoscopists with various skill levels, procedures conducted with different endoscopy systems and therapeutic skills, and cohorts from international multi-centers. Furthermore, we integrate our AI-Endo with the Olympus endoscopic system and validate the AI-enabled cognitive assistance system with animal studies in live ESD training sessions. Dedicated data analysis from surgical phase recognition results is summarized in an automatically generated report for skill assessment. AI-enabled cognitive assistance for therapeutic procedures has rarely been pre-clinically validated. Here, the authors propose an intelligent surgical workflow recognition suit AI-Endo for endoscopic submucosal dissection, extensively validated on external and animal trial datasets.

Modern endoscopy relies on digital technology, from high-resolution imaging sensors and displays to electronics connecting configurable illumination and actuation systems for robotic articulation. In addition to enabling more effective diagnostic and therapeutic interventions, the digitization of the procedural toolset enables video data capture of the internal human anatomy at unprecedented levels. Interventional video data encapsulate functional and structural information about a patient’s anatomy as well as events, activity and action logs about the surgical process. This detailed but difficult-to-interpret record from endoscopic procedures can be linked to preoperative and postoperative records or patient imaging information. Rapid advances in artificial intelligence, especially in supervised deep learning, can utilize data from endoscopic procedures to develop systems for assisting procedures leading to computer-assisted interventions that can enable better navigation during procedures, automation of image interpretation and robotically assisted tool manipulation. In this Perspective, we summarize state-of-the-art artificial intelligence for computer-assisted interventions in gastroenterology and surgery.Advances in artificial intelligence (AI) are changing endoscopy and gastrointestinal surgery, including computer-assisted detection and diagnosis, computer-aided navigation, robot-assisted intervention and automated reporting. This Perspective introduces the role of AI in computer-assisted interventions in gastroenterology with insights on regulatory aspects and the challenges ahead.

AI is becoming ubiquitous, revolutionizing many aspects of our lives. In surgery, it is still a promise. AI has the potential to improve surgeon performance and impact patient care, from post-operative debrief to real-time decision support. But, how much data is needed by an AI-based system to learn surgical context with high fidelity? To answer this question, we leveraged a large-scale, diverse, cholecystectomy video dataset. We assessed surgical workflow recognition and report a deep learning system, that not only detects surgical phases, but does so with high accuracy and is able to generalize to new settings and unseen medical centers. Our findings provide a solid foundation for translating AI applications from research to practice, ushering in a new era of surgical intelligence.

Computer-assisted systems are becoming broadly used in medicine. In endoscopy, most research focuses on the automatic detection of polyps or other pathologies, but localization and navigation of the endoscope are completely performed manually by physicians. To broaden this research and bring spatial Artificial Intelligence to endoscopies, data from complete procedures is needed. This paper introduces the Endomapper dataset, the first collection of complete endoscopy sequences acquired during regular medical practice, making secondary use of medical data. Its main purpose is to facilitate the development and evaluation of Visual Simultaneous Localization and Mapping (VSLAM) methods in real endoscopy data. The dataset contains more than 24 hours of video. It is the first endoscopic dataset that includes endoscope calibration as well as the original calibration videos. Meta-data and annotations associated with the dataset vary from the anatomical landmarks, procedure labeling, segmentations, reconstructions, simulated sequences with ground truth and same patient procedures. The software used in this paper is publicly available.

Endoscopy, a widely used medical procedure for examining the gastrointestinal (GI) tract to detect potential disorders, poses challenges in manual diagnosis due to non-specific symptoms and difficulties in accessing affected areas. While supervised machine learning models have proven effective in assisting clinical diagnosis of GI disorders, the scarcity of image-label pairs created by medical experts limits their availability. To address these limitations, we propose a curriculum self-supervised learning framework inspired by human curriculum learning. Our approach leverages the HyperKvasir dataset, which comprises 100k unlabeled GI images for pre-training and 10k labeled GI images for fine-tuning. By adopting our proposed method, we achieved an impressive top-1 accuracy of 88.92% and an F1 score of 73.39%. This represents a 2.1% increase over vanilla SimSiam for the top-1 accuracy and a 1.9% increase for the F1 score. The combination of self-supervised learning and a curriculum-based approach demonstrates the efficacy of our framework in advancing the diagnosis of GI disorders. Our study highlights the potential of curriculum self-supervised learning in utilizing unlabeled GI tract images to improve the diagnosis of GI disorders, paving the way for more accurate and efficient diagnosis in GI endoscopy.

Oesophageal cancer is the sixth most common cause of cancer-related death worldwide and is therefore a major global health challenge. The two major subtypes of oesophageal cancer are oesophageal squamous cell carcinoma (OSCC) and oesophageal adenocarcinoma (OAC), which are epidemiologically and biologically distinct. OSCC accounts for 90% of all cases of oesophageal cancer globally and is highly prevalent in the East, East Africa and South America. OAC is more common in developed countries than in developing countries. Preneoplastic lesions are identifiable for both OSCC and OAC; these are frequently amenable to endoscopic ablative therapies. Most patients with oesophageal cancer require extensive treatment, including chemotherapy, chemoradiotherapy and/or surgical resection. Patients with advanced or metastatic oesophageal cancer are treated with palliative chemotherapy; those who are human epidermal growth factor receptor 2 (HER2)-positive may also benefit from trastuzumab treatment. Immuno-oncology therapies have also shown promising early results in OSCC and OAC. In this Primer, we review state-of-the-art knowledge on the biology and treatment of oesophageal cancer, including screening, endoscopic ablative therapies and emerging molecular targets, and we discuss best practices in chemotherapy, chemoradiotherapy, surgery and the maintenance of patient quality of life. Oesophageal cancer is the sixth most common cause of cancer-related death worldwide and comprises two major subtypes — oesophageal squamous cell carcinoma and oesophageal adenocarcinoma — which are epidemiologically and biologically distinct. In this Primer, Cunningham and colleagues describe the epidemiology, pathophysiology and management of oesophageal cancer.

We present a comprehensive analysis of the submissions to the first edition of the Endoscopy Artefact Detection challenge (EAD). Using crowd-sourcing, this initiative is a step towards understanding the limitations of existing state-of-the-art computer vision methods applied to endoscopy and promoting the development of new approaches suitable for clinical translation. Endoscopy is a routine imaging technique for the detection, diagnosis and treatment of diseases in hollow-organs; the esophagus, stomach, colon, uterus and the bladder. However the nature of these organs prevent imaged tissues to be free of imaging artefacts such as bubbles, pixel saturation, organ specularity and debris, all of which pose substantial challenges for any quantitative analysis. Consequently, the potential for improved clinical outcomes through quantitative assessment of abnormal mucosal surface observed in endoscopy videos is presently not realized accurately. The EAD challenge promotes awareness of and addresses this key bottleneck problem by investigating methods that can accurately classify, localize and segment artefacts in endoscopy frames as critical prerequisite tasks. Using a diverse curated multi-institutional, multi-modality, multi-organ dataset of video frames, the accuracy and performance of 23 algorithms were objectively ranked for artefact detection and segmentation. The ability of methods to generalize to unseen datasets was also evaluated. The best performing methods (top 15%) propose deep learning strategies to reconcile variabilities in artefact appearance with respect to size, modality, occurrence and organ type. However, no single method outperformed across all tasks. Detailed analyses reveal the shortcomings of current training strategies and highlight the need for developing new optimal metrics to accurately quantify the clinical applicability of methods.

Michalina Gora and her colleagues have developed a tethered capsule endoscope in the form of a swallowable pill that does not require sedation and is the size of a one-cent coin. Once swallowed, the device was well tolerated and used to capture three-dimensional microstructural images of the digestive tract, particularly the esophagus, using optical frequency domain imaging. Feasibility was demonstrated in patients with Barrett’s esophagus, including high-grade dysplasia. Here we introduce tethered capsule endomicroscopy, which involves swallowing an optomechanically engineered pill that captures cross-sectional microscopic images of the gut wall at 30 μm (lateral) × 7 μm (axial) resolution as it travels through the digestive tract. Results in human subjects show that this technique rapidly provides three-dimensional, microstructural images of the upper gastrointestinal tract in a simple and painless procedure, opening up new opportunities for screening for internal diseases.

Image-based tracking of medical instruments is an integral part of surgical data science applications. Previous research has addressed the tasks of detecting, segmenting and tracking medical instruments based on laparoscopic video data. However, the proposed methods still tend to fail when applied to challenging images and do not generalize well to data they have not been trained on. This paper introduces the Heidelberg Colorectal (HeiCo) data set - the first publicly available data set enabling comprehensive benchmarking of medical instrument detection and segmentation algorithms with a specific emphasis on method robustness and generalization capabilities. Our data set comprises 30 laparoscopic videos and corresponding sensor data from medical devices in the operating room for three different types of laparoscopic surgery. Annotations include surgical phase labels for all video frames as well as information on instrument presence and corresponding instance-wise segmentation masks for surgical instruments (if any) in more than 10,000 individual frames. The data has successfully been used to organize international competitions within the Endoscopic Vision Challenges 2017 and 2019. Measurement(s) colorectum Technology Type(s) Laparoscopy Factor Type(s) surgery type Sample Characteristic - Organism Homo sapiens Machine-accessible metadata file describing the reported data: https://doi.org/10.6084/m9.figshare.14178773

Bladder cancer is a highly prevalent disease and is associated with substantial morbidity, mortality and cost. Environmental or occupational exposures to carcinogens, especially tobacco, are the main risk factors for bladder cancer. Most bladder cancers are diagnosed after patients present with macroscopic haematuria, and cases are confirmed after transurethral resection of bladder tumour (TURBT), which also serves as the first stage of treatment. Bladder cancer develops via two distinct pathways, giving rise to non-muscle-invasive papillary tumours and non-papillary (solid) muscle-invasive tumours. The two subtypes have unique pathological features and different molecular characteristics. Indeed, The Cancer Genome Atlas project identified genetic drivers of muscle-invasive bladder cancer (MIBC) as well as subtypes of MIBC with distinct characteristics and therapeutic responses. For non-muscle-invasive bladder cancer (NMIBC), intravesical therapies (primarily Bacillus Calmette–Guérin (BCG)) with maintenance are the main treatments to prevent recurrence and progression after initial TURBT; additional therapies are needed for those who do not respond to BCG. For localized MIBC, optimizing care and reducing morbidity following cystectomy are important goals. In metastatic disease, advances in our genetic understanding of bladder cancer and in immunotherapy are being translated into new therapies. Urothelial bladder cancer is one of the most common, and most deadly, malignant diseases worldwide. This Primer summarizes the current epidemiological and outcome data of patients with this disease, as well as describing how new molecular subtyping strategies might improve patient care in the future.