Journal Article

Real-time surgical instrument detection in robot-assisted surgery using a convolutional neural network cascade

Cai, Tongbiao, Chang, Faliang, Zhao, Zijian, Cheng, Xiaolin

2019

Overview

Surgical instrument detection in robot-assisted surgery videos is an import vision component for these systems. Most of the current deep learning methods focus on single-tool detection and suffer from low detection speed. To address this, the authors propose a novel frame-by-frame detection method using a cascading convolutional neural network (CNN) which consists of two different CNNs for real-time multi-tool detection. An hourglass network and a modified visual geometry group (VGG) network are applied to jointly predict the localisation. The former CNN outputs detection heatmaps representing the location of tool tip areas, and the latter performs bounding-box regression for tool tip areas on these heatmaps stacked with input RGB image frames. The authors’ method is tested on the publicly available EndoVis Challenge dataset and the ATLAS Dione dataset. The experimental results show that their method achieves better performance than mainstream detection methods in terms of detection accuracy and speed.

• Publisher: The Institution of Engineering and Technology,John Wiley & Sons, Inc,Wiley
• Subject: Accuracy / ATLAS Dione dataset / authors / bounding-box regression / cascading convolutional neural network / CNN / convolutional neural nets / convolutional neural network cascade / Datasets / deep learning methods / detection heatmaps / EndoVis Challenge dataset / frame-by-frame detection method / hourglass network / image colour analysis / learning (artificial intelligence) / mainstream detection methods / medical image processing / medical robotics / Methods / modified VGG network / Neural networks / object detection / Performance evaluation / Real time / real-time multi-tool detection / real-time multitool detection / real-time surgical instrument detection / regression analysis / RGB image frames / robot vision / robot-assisted surgery videos / Robotic surgery / single-tool detection / Special Issue: Papers from the 13th Workshop on Augmented Environments for Computer Assisted Interventions / Surgeons / surgery / Surgical apparatus & instruments / tool tip areas / vision component