Explore the vast range of titles available.

PurposeReal-time surgical tool tracking is a core component of the future intelligent operating room (OR), because it is highly instrumental to analyze and understand the surgical activities. Current methods for surgical tool tracking in videos need to be trained on data in which the spatial positions of the tools are manually annotated. Generating such training data is difficult and time-consuming. Instead, we propose to use solely binary presence annotations to train a tool tracker for laparoscopic videos.MethodsThe proposed approach is composed of a CNN + Convolutional LSTM (ConvLSTM) neural network trained end to end, but weakly supervised on tool binary presence labels only. We use the ConvLSTM to model the temporal dependencies in the motion of the surgical tools and leverage its spatiotemporal ability to smooth the class peak activations in the localization heat maps (Lh-maps).ResultsWe build a baseline tracker on top of the CNN model and demonstrate that our approach based on the ConvLSTM outperforms the baseline in tool presence detection, spatial localization, and motion tracking by over 5.0% , 13.9% , and 12.6% , respectively.ConclusionsIn this paper, we demonstrate that binary presence labels are sufficient for training a deep learning tracking model using our proposed method. We also show that the ConvLSTM can leverage the spatiotemporal coherence of consecutive image frames across a surgical video to improve tool presence detection, spatial localization, and motion tracking.

BackgroundFluorescence-based enhanced reality (FLER) is a computer-based quantification method of fluorescence angiographies to evaluate bowel perfusion. The aim of this prospective trial was to assess the clinical feasibility and to correlate FLER with metabolic markers of perfusion, during colorectal resections.MethodsFLER analysis and visualization was performed in 22 patients (diverticulitis n = 17; colorectal cancer n = 5) intra- and extra-abdominally during distal and proximal resection, respectively. The fluorescence signal of indocyanine green (0.2 mg/kg) was captured using a near-infrared camera and computed to create a virtual color-coded cartography. This was overlaid onto the bowel (enhanced reality). It helped to identify regions of interest (ROIs) where samples were subsequently obtained. Resections were performed strictly guided according to clinical decision. On the surgical specimen, samplings were made at different ROIs to measure intestinal lactates (mmol/L) and mitochondria efficiency as acceptor control ratio (ACR).ResultsThe native (unquantified) fluorescent signal diffused to obvious ischemic areas during the distal appreciation. Proximally, a lower diffusion of ICG was observed. Five anastomotic complications occurred. The expected values of local capillary lactates were correlated with the measured values both proximally (3.62 ± 2.48 expected vs. 3.17 ± 2.8 actual; rho 0.89; p = 0.0006) and distally (4.5 ± 3 expected vs. 4 ± 2.5 actual; rho 0.73; p = 0.0021). FLER values correlated with ACR at the proximal site (rho 0.76; p = 0.04) and at the ischemic zone (rho 0.71; p = 0.01). In complicated cases, lactates at the proximal resection site were higher (5.8 ± 4.5) as opposed to uncomplicated cases (2.45 ± 1.5; p = 0.008). ACR was reduced proximally in complicated (1.3 ± 0.18) vs. uncomplicated cases (1.68 ± 0.3; p = 0.023).ConclusionsFLER allows to image the quantified fluorescence signal in augmented reality and provides a reproducible estimation of bowel perfusion (NCT02626091).

BackgroundClinical evaluation of the demarcation line separating ischemic from non-ischemic liver parenchyma may be challenging. Hyperspectral imaging (HSI) is a noninvasive imaging modality, which combines a camera with a spectroscope and allows quantitative imaging of tissue oxygenation. Our group developed a software to overlay HSI images onto the operative field, obtaining HSI-based enhanced reality (HYPER). The aim of the present study was to evaluate the accuracy of HYPER to identify the demarcation line after a left vascular inflow occlusion during an anatomical left hepatectomy.Materials and methodsIn the porcine model (n = 3), the left branches of the hepatic pedicle were ligated. Before and after vascular occlusion, HSI images based on tissue oxygenation (StO2), obtained through the Near-Infrared index (NIR index), were regularly acquired and superimposed onto RGB video. The demarcation line was marked on the liver surface with electrocautery according to HYPER. Local lactates were measured on blood samples from the liver surface in both ischemic and perfused segments using a strip-based device. At the same areas, confocal endomicroscopy was performed.ResultsAfter ligation, HSI demonstrated a significantly lower oxygenation (NIR index) in the left medial lobe (LML) (0.27% ± 0.21) when compared to the right medial lobe (RML) (58.60% ± 12.08; p = 0.0015). Capillary lactates were significantly higher (3.07 mmol/L ± 0.84 vs. 1.33 ± 0.71 mmol/L; p = 0.0356) in the LML versus RML, respectively. Concordantly, confocal videos demonstrated the absence of blood flow in the LML and normal perfusion in the RML.ConclusionsHYPER has made it possible to correctly identify the demarcation line and quantify surface liver oxygenation. HYPER could be an intraoperative tool to guide perfusion-based demarcation line assessment and segmentation.

PurposeAutomatic segmentation and classification of surgical activity is crucial for providing advanced support in computer-assisted interventions and autonomous functionalities in robot-assisted surgeries. Prior works have focused on recognizing either coarse activities, such as phases, or fine-grained activities, such as gestures. This work aims at jointly recognizing two complementary levels of granularity directly from videos, namely phases and steps.MethodsWe introduce two correlated surgical activities, phases and steps, for the laparoscopic gastric bypass procedure. We propose a multi-task multi-stage temporal convolutional network (MTMS-TCN) along with a multi-task convolutional neural network (CNN) training setup to jointly predict the phases and steps and benefit from their complementarity to better evaluate the execution of the procedure. We evaluate the proposed method on a large video dataset consisting of 40 surgical procedures (Bypass40).ResultsWe present experimental results from several baseline models for both phase and step recognition on the Bypass40. The proposed MTMS-TCN method outperforms single-task methods in both phase and step recognition by 1-2% in accuracy, precision and recall. Furthermore, for step recognition, MTMS-TCN achieves a superior performance of 3-6% compared to LSTM-based models on all metrics.ConclusionIn this work, we present a multi-task multi-stage temporal convolutional network for surgical activity recognition, which shows improved results compared to single-task models on a gastric bypass dataset with multi-level annotations. The proposed method shows that the joint modeling of phases and steps is beneficial to improve the overall recognition of each type of activity.

Background Modern chemotherapy achieves the shrinking of colorectal cancer liver metastases (CRLM) to such extent that they may disappear from radiological imaging. Disappearing CRLM rarely represents a complete pathological remission and have an important risk of recurrence. Augmented reality (AR) consists in the fusion of real-time patient images with a computer-generated 3D virtual patient model created from pre-operative medical imaging. The aim of this prospective pilot study is to investigate the potential of AR navigation as a tool to help locate and surgically resect missing CRLM. Methods A 3D virtual anatomical model was created from thoracoabdominal CT-scans using customary software (VR RENDER ® , IRCAD). The virtual model was superimposed to the operative field using an Exoscope (VITOM ® , Karl Storz, Tüttlingen, Germany). Virtual and real images were manually registered in real-time using a video mixer, based on external anatomical landmarks with an estimated accuracy of 5 mm. This modality was tested in three patients, with four missing CRLM that had sizes from 12 to 24 mm, undergoing laparotomy after receiving pre-operative oxaliplatin-based chemotherapy. Results AR display and fine registration was performed within 6 min. AR helped detect all four missing CRLM, and guided their resection. In all cases the planned security margin of 1 cm was clear and resections were confirmed to be R0 by pathology. There was no postoperative major morbidity or mortality. No local recurrence occurred in the follow-up period of 6–22 months. Conclusions This initial experience suggests that AR may be a helpful navigation tool for the resection of missing CRLM.

BackgroundLaparoscopic sleeve gastrectomy (LSG) is currently the most commonly performed bariatric procedure. Endoscopic sleeve gastroplasty (ESG) is a promising new bariatric technique which is less invasive in its approach. To date no study has compared quality of life (QoL) outcomes between LSG and ESG. The aim of this study is to compare QoL after ESG and LSG using a propensity score analysis.MethodsQoL was evaluated by means of Gastrointestinal Quality of Life Index (GIQLI) questionnaire before and 6 months after the procedure. Patients were matched for age, sex, preoperative weight, and comorbidities.ResultsPropensity score matching resulted in 23 pairs of patients homogeneous for age (p = 0.3), preoperative BMI (p = 0.3), sex (p = 0.74), and comorbidities (p = 0.9). Post-ESG patients, despite a less important %EWL (39.9 (17.5–58.9)vs 54.9 (46.2–65); p = 0.01) and %TWL (13.4 (7.8–20.9) vs 18.8 (17.6–21.8); p = 0.03), presented better QoL (14 [3–24] vs 13 (− 1–23) ΔGIQLI score; p = 0.79) with clear advantage for the gastrointestinal symptoms subdomain (66.5 (61–70.5) vs 59 (55–63); p = 0.001), while post-LSG patients presented a worsening of GERD symptoms (30.7% vs 0%) and an increased use of PPI therapy (p = 0.004). Resolution or improvement of comorbidities was similar (ESG 53% vs LSG 45.8%; p = 0.79) in both groups.ConclusionLSG may significantly affect QoL and results in worsening of gastrointestinal symptoms including GERD. ESG is a promising less invasive bariatric endoscopic procedure that demonstrated a positive impact on both QoL and comorbidities, which could lead to greater patient acceptance earlier in their disease or at a younger age.

Purpose Long-term results on sleeve gastrectomy (SG) with more than 10 years report patients needing sleeve revision for weight loss failure, de novo gastroesophageal reflux (GERD), or sleeve complications. The aim of this study was to analyze the results of laparoscopic conversion of failed SG to Roux-en-Y gastric bypass (RYGB). Materials and Methods Retrospective review of a prospectively institutional maintained database to identify patients who underwent conversion of SG to RYGB between 2012 and June 2020. Results Sixty patients(50 females) underwent conversion to RYGB. Average time to conversion was 5.6 years (2–11). Mean %WL and TWL after SG were respectively 26±8.8% and 33.2±14.1kg. Mean BMI at the time of RYGB was 38.1±7.1 kg/m 2 . Mean follow-up was 30.4±16.8 months (6–84). Available patients at each time of follow-up: 1 year 59 (98.3%); 2 years 47 (78.3%); 3 years 39 (71.6%); and 5 years 33 (55%). Patients were divided according to indication for revision in weight regain/insufficient weight loss (30 patients) group 1 and GERD/complications (25 patients) group 2. Percentage of excess weight loss at 1, 3, and 5 years follow-up after bypass was for group 1 40.3±17.6, 34.3±19.5, and 23.2±19.4 and for group 2 90.4±37, 62.6±28.2, and 56±35.02. Total weight loss at last follow-up since sleeve was respectively 31kg in group 1 and 46.7kg in group 2 ( p =0.002). No mortality was observed. Thirty-day complication rate was 3.3%. Conclusion RYGB after SG is a safe and effective revisional procedure to manage weight regain and de novo GERD, to address complications, and to improve comorbidities.

BackgroundAugmented reality (AR) in surgery consists in the fusion of synthetic computer-generated images (3D virtual model) obtained from medical imaging preoperative workup and real-time patient images in order to visualize unapparent anatomical details. The 3D model could be used for a preoperative planning of the procedure. The potential of AR navigation as a tool to improve safety of the surgical dissection is outlined for robotic hepatectomy.Materials and methodsThree patients underwent a fully robotic and AR-assisted hepatic segmentectomy. The 3D virtual anatomical model was obtained using a thoracoabdominal CT scan with a customary software (VR-RENDER®, IRCAD). The model was then processed using a VR-RENDER® plug-in application, the Virtual Surgical Planning (VSP®, IRCAD), to delineate surgical resection planes including the elective ligature of vascular structures. Deformations associated with pneumoperitoneum were also simulated. The virtual model was superimposed to the operative field. A computer scientist manually registered virtual and real images using a video mixer (MX 70; Panasonic, Secaucus, NJ) in real time.ResultsTwo totally robotic AR segmentectomy V and one segmentectomy VI were performed. AR allowed for the precise and safe recognition of all major vascular structures during the procedure. Total time required to obtain AR was 8 min (range 6–10 min). Each registration (alignment of the vascular anatomy) required a few seconds. Hepatic pedicle clamping was never performed. At the end of the procedure, the remnant liver was correctly vascularized. Resection margins were negative in all cases. The postoperative period was uneventful without perioperative transfusion.ConclusionsAR is a valuable navigation tool which may enhance the ability to achieve safe surgical resection during robotic hepatectomy.

BackgroundIn laparoscopic cholecystectomy (LC), achievement of the Critical View of Safety (CVS) is commonly advocated to prevent bile duct injuries (BDI). However, BDI rates remain stable, probably due to inconsistent application or a poor understanding of CVS as well as unreliable reporting. Objective video reporting could serve for quality auditing and help generate consistent datasets for deep learning models aimed at intraoperative assistance. In this study, we develop and test a method to report CVS using videos.MethodLC videos performed at our institution were retrieved and the video segments starting 60 s prior to the division of cystic structures were edited. Two independent reviewers assessed CVS using an adaptation of the doublet view 6-point scale and a novel binary method in which each criterion is considered either achieved or not. Feasibility to assess CVS in the edited video clips and inter-rater agreements were evaluated.ResultsCVS was attempted in 78 out of the 100 LC videos retrieved. CVS was assessable in 100% of the 60-s video clips. After mediation, CVS was achieved in 32/78(41.03%). Kappa scores of inter-rater agreements using the doublet view versus the binary assessment were as follows: 0.54 versus 0.75 for CVS achievement, 0.45 versus 0.62 for the dissection of the hepatocystic triangle, 0.36 versus 0.77 for the exposure of the lower part of the cystic plate, and 0.48 versus 0.79 for the 2 structures connected to the gallbladder.ConclusionsThe present study is the first to formalize a reproducible method for objective video reporting of CVS in LC. Minute-long video clips provide information on CVS and binary assessment yields a higher inter-rater agreement than previously used methods. These results offer an easy-to-implement strategy for objective video reporting of CVS, which could be used for quality auditing, scientific communication, and development of deep learning models for intraoperative guidance.

Surgical video analysis facilitates education and research. However, video recordings of endoscopic surgeries can contain privacy-sensitive information, especially if the endoscopic camera is moved out of the body of patients and out-of-body scenes are recorded. Therefore, identification of out-of-body scenes in endoscopic videos is of major importance to preserve the privacy of patients and operating room staff. This study developed and validated a deep learning model for the identification of out-of-body images in endoscopic videos. The model was trained and evaluated on an internal dataset of 12 different types of laparoscopic and robotic surgeries and was externally validated on two independent multicentric test datasets of laparoscopic gastric bypass and cholecystectomy surgeries. Model performance was evaluated compared to human ground truth annotations measuring the receiver operating characteristic area under the curve (ROC AUC). The internal dataset consisting of 356,267 images from 48 videos and the two multicentric test datasets consisting of 54,385 and 58,349 images from 10 and 20 videos, respectively, were annotated. The model identified out-of-body images with 99.97% ROC AUC on the internal test dataset. Mean ± standard deviation ROC AUC on the multicentric gastric bypass dataset was 99.94 ± 0.07% and 99.71 ± 0.40% on the multicentric cholecystectomy dataset, respectively. The model can reliably identify out-of-body images in endoscopic videos and is publicly shared. This facilitates privacy preservation in surgical video analysis.