Explore the vast range of titles available.

Three-dimensional (3D) display technology has continuously been attracting public attention with the progress in today's 3D television and mature display technologies. The primary characteristics of conventional glasses-free autostereoscopic displays, such as spatial resolution, image depths and viewing angle, are often limited due to the use of optical lenses or optical gratings. We present a 3D display using MEMS-scanning-mechanism-based light homogeneous emitting (LHE) approach and demonstrate that the display can directly generate an autostereoscopic 3D image without the need for optical lenses or gratings. The generated 3D image has the advantages of non-aberration and a high-definition spatial resolution, making it the first to exhibit animated 3D images with image depth of six meters. Our LHE 3D display approach can be used to generate a natural flat-panel 3D display with super long viewing distance and alternative real-time image update.

PurposeAutonomous ultrasound imaging by robotic ultrasound scanning systems in complex soft uncertain clinical environments is important and challenging to assist in therapy. To cope with the complex environment faced by the ultrasound probe during the scanning process, we propose an autonomous robotic ultrasound (US) control method based on reinforcement learning (RL) model to build the relationship between the environment and the system. The proposed method requires only contact force as input information to achieve robot control of the posture and contact force of the probe without any a priori information about the target and the environment.MethodsFirst, an RL agent is proposed and trained by a policy gradient theorem-based RL model with the 6-degree-of-freedom (DOF) contact force of the US probe to learn the relationship between contact force and output force directly. Then, a force control strategy based on the admittance controller is proposed for synchronous force, orientation and position control by defining the desired contact force as the action space.ResultsThe proposed method was evaluated via collected US images, contact force and scan trajectories by scanning an unknown soft phantom. The experimental results indicated that the proposed method differs from the free-hand scanned approach in the US images within 3 ± 0.4%. The analysis results of contact forces and trajectories indicated that our method could make stable scanning processes on a soft uncertain skin surface and obtained US images.ConclusionWe propose a concise and efficient force-guided US robot scanning control method for soft uncertain environment based on reinforcement learning. Experimental results validated our method's feasibility and validity for complex skin surface scanning, and the volunteer experiments indicated the potential application value in the complex clinical environment of robotic US imaging system especially with limited visual information.

PurposeWe present a novel augmented reality (AR) surgical navigation method with ultrasound-assisted point cloud registration for percutaneous ablation of liver tumors. A preliminary study is carried out to verify its feasibility.MethodsTwo three-dimensional (3D) point clouds of the liver surface are derived from the preoperative images and intraoperative tracked US images, respectively. To compensate for the soft tissue deformation, the point cloud registration between the preoperative images and the liver is performed using the non-rigid iterative closest point (ICP) algorithm. A 3D AR device based on integral videography technology is designed to accurately display naked-eye 3D images for surgical navigation. Based on the above registration, naked-eye 3D images of the liver surface, planning path, entry points, and tumor can be overlaid in situ through our 3D AR device. Finally, the AR-guided targeting accuracy is evaluated through entry point positioning.ResultsExperiments on both the liver phantom and in vitro pork liver were conducted. Several entry points on the liver surface were used to evaluate the targeting accuracy. The preliminary validation on the liver phantom showed average entry-point errors (EPEs) of 2.34 ± 0.45 mm, 2.25 ± 0.72 mm, 2.71 ± 0.82 mm, and 2.50 ± 1.11 mm at distinct US point cloud coverage rates of 100%, 75%, 50%, and 25%, respectively. The average EPEs of the deformed pork liver were 4.49 ± 1.88 mm and 5.02 ± 2.03 mm at the coverage rates of 100% and 75%, and the average covered-entry-point errors (CEPEs) were 4.96 ± 2.05 mm and 2.97 ± 1.37 mm at 50% and 25%, respectively.ConclusionExperimental outcomes demonstrate that the proposed AR navigation method based on US-assisted point cloud registration has achieved an acceptable targeting accuracy on the liver surface even in the case of liver deformation.

Hepatic echinococcosis (HE) is a life-threatening liver disease caused by parasites that requires a precise diagnosis and proper treatments. To assess HE lesions accurately, we propose a novel automatic HE lesion segmentation and classification network that contains lesion region positioning (LRP) and lesion region segmenting (LRS) modules. First, we used the LRP module to obtain the probability map of the lesion distribution and the position of the lesion. Then, based on the result of the LRP module, we used the LRS module to precisely segment the HE lesions within the high-probability region. Finally, we classified the HE lesions and identified the lesion types by a convolutional neural network (CNN). The entire dataset was delineated by the hospital’s senior radiologist. We collected CT slices of 160 patients from Qinghai Provincial People’s Hospital. The Dice score of the final segmentation result reached 89.89%. The Dice scores, indicating the classification accuracy, for cystic vs. alveolar echinococcosis and calcified vs. noncalcified lesions were 80.32% and 82.45%, the sensitivities were 72.41% and 75.17%, the specificities were 83.72% and 86.04%, the NPVs were 80.01% and 86.96%, the PPVs were 80.45% and 81.74%, and the areas under the ROC curves were 0.8128 and 0.8205, respectively.

ObjectivesThe aim of this study was to propose and validate an automatic approach based on iterative closest point algorithm for virtual complement and reconstruction for maxillofacial bone defects.Materials and methodsA 3D craniomaxillofacial database of normal Chinese people including 500 skull models was established. Modified iterative closest point (ICP) algorithm was developed to complete bone defects automatically. The performances were evaluated by two approaches: (1) model experiment, virtual bony defects were created on 30 intact normal skull models not included in the database. For each defect model, the algorithm was applied to select the reference skull model from the database. 3-Dimensional and 2-dimensional comparison were conducted to evaluate the error between reference skull model with original intact model. Root mean square error (RMSE) and processing time were calculated. (2) Clinical application, the algorithm was utilized to assist reconstruction of 5 patients with maxillofacial bone defects. The symmetry of post-operative skull model was evaluated by comparing with its mirrored model.ResultsThe algorithm was tested on an CPU with 1.80 GHz and average processing time was 493.5 s. (1) Model experiment, the average root-mean-square deviation of defect area was less than 2 mm. (2) Clinical application, the RMSE of post-operative skull and its mirrored model was 1.72 mm.ConclusionIt is feasible using iterative closest point algorithm based on normal people database to automatically predict the reference data of missing maxillofacial bone.Clinical relevanceAn automated approach based on ICP algorithm and normal people database for maxillofacial bone defect reconstruction has been proposed and validated.

PurposeFor tumor resections near critical structures, accurate identification of tumor boundaries and maximum removal are the keys to improve surgical outcome and patient survival rate, especially in neurosurgery. In this paper, we propose an intelligent optical diagnosis and treatment system for tumor removal, with automated lesion localization and laser ablation.MethodsThe proposed system contains a laser ablation module, an optical coherence tomography (OCT) unit, and a robotic arm along with a stereo camera. The robotic arm can move the OCT sample arm and the laser ablation front-end to the suspected lesion area. The corresponding diagnosis and treatment procedures include computer-aided lesion segmentation using OCT, automated ablation planning, and laser control. The ablation process is controlled by a deflectable mirror, and a non-common-path ablation planning algorithm based on the transformation from lesion positions to mirror deflection angles is presented.ResultsPhantom and animal experiments are carried out for system verification. The robot could reach the planned position with high precision, which is approximately 1.16 mm. Tissue classification with OCT images achieves 91.7% accuracy. The error of OCT-guided automated laser ablation is approximately 0.74 mm. Experiments on mouse brain tumors show that the proposed system is capable of clearing lesions efficiently and precisely. We also conducted an ex vivo porcine brain experiment to verify the whole process of the system.ConclusionAn intelligent optical diagnosis and treatment system is proposed for tumor removal. Experimental results show that the proposed system and method are promising for precise and intelligent theranostics. Compared to conventional cancer diagnosis and treatment, the proposed system allows for automated operations monitored in real-time, with higher precision and efficiency.

Breast cancer is the leading killer of Chinese women. Immunohistochemistry index has great significance in the treatment strategy selection and prognosis analysis for breast cancer patients. Currently, histopathological examination of the tumor tissue through surgical biopsy is the gold standard to determine immunohistochemistry index. However, this examination is invasive and commonly causes discomfort in patients. There has been a lack of noninvasive method capable of predicting immunohistochemistry index for breast cancer patients. This paper proposes a machine learning method to predict the immunohistochemical index of breast cancer patients by using noninvasive contrast-enhanced ultrasound. A total of 119 breast cancer patients were included in this retrospective study. Each patient implemented the pathological examination of immunohistochemical expression and underwent contrast-enhanced ultrasound imaging of breast tumor. The multi-dimension features including 266 three-dimension features and 837 two-dimension dynamic features were extracted from the contrast-enhanced ultrasound sequences. Using the machine learning prediction method, 21 selected multi-dimension features were integrated to generate a model for predicting the immunohistochemistry index noninvasively. The immunohistochemical index of human epidermal growth factor receptor-2 (HER2) was predicted based on multi-dimension features in contrast-enhanced ultrasound sequence with the sensitivity of 71%, and the specificity of 79% in the testing cohort. Therefore, the noninvasive contrast-enhanced ultrasound can be used to predict the immunohistochemical index. To our best knowledge, no studies have been reported about predicting immunohistochemical index by using contrast-enhanced ultrasound sequences for breast cancer patients. Our proposed method is noninvasive and can predict immunohistochemical index by using contrast-enhanced ultrasound in several minutes, instead of relying totally on the invasive and biopsy-based histopathological examination.

PurposeThe visualization of remote surgical scenes is the key to realizing the remote operation of surgical robots. However, current non-endoscopic surgical robot systems lack an effective visualization tool to offer sufficient surgical scene information and depth perception.MethodsWe propose a novel autostereoscopic surgical visualization system integrating 3D intraoperative scene reconstruction, autostereoscopic 3D display, and augmented reality-based image fusion. The preoperative organ structure and the intraoperative surface point cloud are obtained from medical imaging and the RGB-D camera, respectively, and aligned by an automatic marker-free intraoperative registration algorithm. After registration, preoperative meshes with precalculated illumination and intraoperative textured point cloud are blended in real time. Finally, the fused image is shown on a 3D autostereoscopic display device to achieve depth perception.ResultsA prototype of the autostereoscopic surgical visualization system was built. The system had a horizontal image resolution of 1.31 mm, a vertical image resolution of 0.82 mm, an average rendering rate of 33.1 FPS, an average registration rate of 20.5 FPS, and average registration errors of approximately 3 mm. A telesurgical robot prototype based on 3D autostereoscopic display was built. The quantitative evaluation experiments showed that our system achieved similar operational accuracy (1.79 ± 0.87 mm) as the conventional system (1.95 ± 0.71 mm), while having advantages in terms of completion time (with 34.11% reduction) and path length (with 35.87% reduction). Post-experimental questionnaires indicated that the system was user-friendly for novices and experts.ConclusionWe propose a 3D surgical visualization system with augmented instruction and depth perception for telesurgery. The qualitative and quantitative evaluation results illustrate the accuracy and efficiency of the proposed system. Therefore, it shows great prospects in robotic surgery and telesurgery.

Background This study evaluated the use of an augmented reality navigation system that provides a markerless registration system using stereo vision in oral and maxillofacial surgery. Method A feasibility study was performed on a subject, wherein a stereo camera was used for tracking and markerless registration. The computed tomography data obtained from the volunteer was used to create an integral videography image and a 3-dimensional rapid prototype model of the jaw. The overlay of the subject’s anatomic site and its 3D-IV image were displayed in real space using a 3D-AR display. Extraction of characteristic points and teeth matching were done using parallax images from two stereo cameras for patient-image registration. Results Accurate registration of the volunteer’s anatomy with IV stereoscopic images via image matching was done using the fully automated markerless system, which recognized the incisal edges of the teeth and captured information pertaining to their position with an average target registration error of < 1 mm. These 3D-CT images were then displayed in real space with high accuracy using AR. Even when the viewing position was changed, the 3D images could be observed as if they were floating in real space without using special glasses. Conclusion Teeth were successfully used for registration via 3D image (contour) matching. This system, without using references or fiducial markers, displayed 3D-CT images in real space with high accuracy. The system provided real-time markerless registration and 3D image matching via stereo vision, which, combined with AR, could have significant clinical applications.