The importance of image viewing conditions for timing and precision in minimally invasive surgery

Minimally invasive medical procedures such as laparoscopic surgery are challenging because they involve navigating the instruments in use via image views from tiny cameras attached to them. Surgeons need to interpret visual information correctly to guide their instruments with precision. Understanding how the brain adapts to these procedures will help improve fine surgical skills. Dr Birgitta Dresp-Langley, Research Director at the French National Centre for Scientific Research (CNRS) has designed well-tailored experiments with to-the-single-pixel precise measurement of hand-tool movement trajectories in task time at the millisecond level. Her work highlights the drawbacks of virtual reality headset viewing and reveals critical factors that affect the timing and precision of image-guided tasks.

Image-guided processes have revolutionised surgical procedures and are crucial in minimally invasive surgery, i.e. laparoscopy. The surgeon uses specifically designed tools equipped with tiny cameras to look inside your body via a small incision in the abdomen or pelvis. Laparoscopy is used to diagnose or treat a range of conditions, including appendicitis, endometriosis, and certain types of cancer.

Performing laparoscopy can be challenging because humans are accustomed to interacting with objects at the real 3D scale rather than responding to virtual images of the environment. Dr Birgitta Dresp-Langley, research director at the French National Centre for Scientific Research (CNRS), has run studies [1, 2, 3] to identify the factors that determine the timing and precision of hand-tool movements and their trajectories in image-guided tasks. Her findings can help us understand how surgeons may improve their performance in critical medical procedures such as minimally invasive surgery.

Virtual navigation challenges

In virtual navigation, the image representations need to be processed and interpreted by the brain to ensure the correct positioning of hands and tools while surgeons are looking at a two-dimensional (2D) camera image or a three-dimensional (3D) reconstruction of the camera image by means of software piloting a virtual reality (VR) headset. Experienced surgeons become proficient with training and practice and, finally, adapt to the constraints of virtual navigation. The cognitive mechanisms of this adaptation are still not well understood.

A major problem in virtual navigation is that depth information needs to be reconstructed by the brain because reliable information about depth and relative distances in the task space the surgeon has to act upon (a patient’s organ) is missing from the image representation. Time-to-contact anticipation, movement planning, and eye-hand coordination are challenged compared with direct ‘natural’ viewing conditions. Also, the image views change in computational time as the tool moves through the body in real time. Research by Dresp-Langley has investigated performance characteristics of healthy volunteers with no specific previous experience in such procedures.

Study design

An equal number of men and women took part in the study. The task described in [1] involved executing a series of moves in a particular order to place a small object as swiftly and precisely as possible on the indicated spots on a task board. The participants were tested under different conditions, either using their hands or a surgical tool. They were given an unconstrained direct view of their hand-tool movements or were guided by 2D fisheye camera images displayed on a monitor, positioned either straight ahead or sideways, or a reconstructed 3D view from a virtual reality (VR) headset. Before starting the trial, the participants had the chance to look at a printout of the trajectory steps they had to follow during the task.

Each participant completed two rounds of every experimental condition described above. For a round to be counted as complete, they had to execute the series of moves in the correct order and without dropping the object. Specific software allowed measuring both the precision (in image pixels) and time (in milliseconds) of each participant in a fully complete task set. Individual completion time corresponded to the duration from the moment the object was picked up to the time it was placed back to its initial position on the task board.

2D versus 3D image

Image conditions directly affect task performance. Direct ‘natural’ viewing of the task space generates the fastest and most precise hand-tool movement trajectories compared with any of the 2D image viewing conditions. The reconstructed 3D view of the task space conveyed by the VR headset produced the worst performance in terms of precision and completion times. This drawback is further highlighted by hand movement trajectories recovered by processing of the video images of task execution by the different participants [2], displayed in the trajectory reconstructions in Figure 1.

Speed versus precision

Participants consistently performed faster in the second session compared to the first, suggesting there was an immediate training effect on task time. However, no training effect was observed for movement or trajectory precision, which did not improve in the second round. Compared to an image-guided setting, subjects performed faster and more precisely when they had a direct view of the task-space and their hand movements and did not have to use the tool to move the object across the task board. Perception of the so-called near-body space and the task space within arm’s reach directly determines performance depending on the viewing condition. An extra effort is needed to further adjust hand movements to the constraints of the tool.

The researchers also noticed that performance was significantly affected by the monitor position in the 2D viewing conditions. Although subjects were faster when the monitor was positioned straight ahead, they were more precise when the monitor was placed sideways. This finding suggests effects of viewing angle and/or attention on the recovery of efficient image cues for movement planning. When looking straight ahead, there is less strain on the neck and participants feel more comfortable, which results in faster moves. Sideways viewing may generate more effective cues to relative distances in the task field and, although the participants are in a less comfortable position, this also prompts them to pay more attention, which contributes to more precise moves.

Another advantage of sideways viewing may be better control of hand and tool movements due to changes in lighting conditions, allowing for better discrimination of objects and their trajectories in the sideways viewing condition. Trade-off between speed and precision is always a critical aspect of training for image-guided tasks and depends on the specific task conditions.

The road ahead

Safe and effective performance of tasks guided by virtual information, such as minimally invasive surgery, is of paramount importance in modern medicine. However, processes that involve image-guided tasks entail serious challenges due to the risk of misperception of relative distances and time-to-contact of moving objects.

Dresp-Langley’s work highlights the need for precise measurement criteria to establish objective gold-standard benchmarks for training novices in image-guided precision tasks [3]. This will help ensure safety and precision in virtually guided surgery in the future.