Intensity-Based Visual Servoing for Instrument and Tissue Tracking in 3D Ultrasound Volumes

This paper presents a three dimensional ultrasound (3DUS)-based visual servoing technique for intraoperative tracking of the motion of both surgical instruments and tissue targets. In the proposed approach, visual servoing techniques are used to control the position of a virtual ultrasound probe so as to keep a target centered within the virtual probe's field-of-view. Multiple virtual probes can be servoed in parallel to provide simultaneous tracking of instruments and tissue. The technique is developed in the context of robotic beating-heart intracardiac surgery in which the goal of tracking is to both provide guidance to the operator as well as to provide the means to automate the surgical procedure. To deal with the low signal-to-noise ratio (SNR) of the 3DUS volumes, an intensity-based method is proposed that requires no primitive extraction or image segmentation since it directly utilizes the image intensity information as a visual feature. This approach is computationally efficient and can be applied to a wide range of tissue types and medical instruments. This paper presents the first validation of these techniques through offline robot and tissue tracking using actual in vivo cardiac volume sequences from a robotic beating-heart surgery. Note to Practitioners-This paper is motivated by the practical navigation problem in 3DUS-guided beating-heart robotic surgery. A real-time and robust robot guidance system is of great significance in computer-assisted and computer-navigated robotic surgeries. The current position-based ultrasound guidance systems typically involve sequential feature extraction followed by position estimation, while the proposed intensity-based ultrasound virtual servoing can directly track both tissue and instruments from the raw 3D images. Without necessitating segmentation or feature extraction, the proposed approach is efficient enough to be used in guiding the surgical robot to the target using real-time 3DUS. The implementation is based on virtual probe servoing and can be extended to other similar volumetric image-based servoing problems in industrial applications. The proposed algorithm can be implemented as a plug-in module, which could be easily integrated into existing image-guided robotic surgery platforms.