Constant Visual and Haptic Time Delays in Simulated Bilateral Teleoperation: Quantifying the Human Operator Performance

Visual feedback and force feedback (haptics) are the two streams of information in a robotic bilateral teleoperation where the operator manipulates a robot in a remote location. Delivering the visual and the haptic information depends in part on the characteristics of the communication network and results in a nonsynchronized delay. The goal is to study the effect of constant nonsynchronized and synchronized time delay of visual and haptic information on the human teleoperation performance. The experimental setup included a virtual reality environment, which allows the operator to manipulate the virtual objects in a simulated remote environment through a haptic device that renders the force feedback. The visual and the haptic information were delayed independently in the range of 0-500 ms, creating 121 different scenarios of synchronized and nonsynchronized delays. Selecting specific parameters of the remote virtual environment guaranteed stable teleportation, given the time delays under study. The experimental tasks included tracing predefined geometrical shapes and a pick-and-place task, which simulates both structured and unstructured interactions under the influence of guiding forces. Eight subjects (n = 8) participated in the experiment performing three repetitions of three different teleoperation tasks with 121 combinations of visual and haptic time delays. The measured parameters that were used to assess the human performance were the task completion time and the position errors expressed as a function of the visual and the haptic time delay. Then, regression and ANOVA analyses were performed. The results indicated that the human performance is a function of the sum of the two delays. As the sum of the two delays increases, the human performance degrades and is expressed with an increase in completion time and position errors. The performance degradation is more pronounced in the pick-and-place task compared to the tracing task. In scenarios where the visual and the haptics information were out of synchronization, the human performance was better than intentionally delaying one source of information in an attempt to synchronize and unify the two delays. The results of this study may be applied to any teleoperation tasks over a network with inherent time delays and more specifically to telesurgery in which performance degradation due to time delay has a profound effect on the quality of the healthcare delivered, patient safety, and ultimately the outcomes of the surgical procedure itself.