Journal Article

Evaluating the impact of reinforcement learning on automatic deep brain stimulation planning

Essert, Caroline, Pantovic, Anja

2024

Overview

Purpose Traditional techniques for automating the planning of brain electrode placement based on multi-objective optimization involving many parameters are subject to limitations, especially in terms of sensitivity to local optima, and tend to be replaced by machine learning approaches. This paper explores the feasibility of using deep reinforcement learning (DRL) in this context, starting with the single-electrode use-case of deep brain stimulation (DBS). Methods We propose a DRL approach based on deep Q-learning where the states represent the electrode trajectory and associated information, and actions are the possible motions. Deep neural networks allow to navigate the complex state space derived from MRI data. The chosen reward function emphasizes safety and accuracy in reaching the target structure. The results were compared with a reference (segmented electrode) and a conventional technique. Results The DRL approach excelled in navigating the complex anatomy, consistently providing safer and more precise electrode placements than the reference. Compared to conventional techniques, it showed an improvement in accuracy of 2.3% in average proximity to obstacles and 19.4% in average orientation angle. Expectedly, computation times rose significantly, from 2 to 18 min. Conclusion Our investigation into DRL for DBS electrode trajectory planning has showcased its promising potential. Despite only delivering modest accuracy gains compared to traditional methods in the single-electrode case, its relevance for problems with high-dimensional state and action spaces and its resilience against local optima highlight its promising role for complex scenarios. This preliminary study constitutes a first step toward the more challenging problem of multiple-electrodes planning.

• Publisher: Springer International Publishing,Springer Nature B.V,Springer Verlag
• Subject: Algorithms / Artificial neural networks / Automation / Brain / Computer Imaging / Computer Science / Datasets / Deep brain stimulation / Deep Brain Stimulation - methods / Deep Learning / Electrodes / Electrodes, Implanted / Feasibility Studies / Health Informatics / Humans / Image Processing / Imaging / Machine learning / Magnetic Resonance Imaging - methods / Medicine / Medicine & Public Health / Methods / Multiple objective analysis / Neurosurgery / Optimization techniques / Original Article / Parameter sensitivity / Pattern Recognition and Graphics / Radiology / Reinforcement, Psychology / Stimulation / Surgery / Trajectory planning / Vision