Engineering & Technology
July 30, 2018

Using technology to improve needle procedures

Dr Gabor Fichtinger is Chair of Cancer Care Ontario Research and Professor at Queen’s University School of Computing in Kingston, Ontario. His interests are in computer-assisted surgery and interventions. Dr Jan Fritz is the Director of Interventional MR Imaging and an Assistant Professor of Radiology and Radiological Science at Johns Hopkins University School of Medicine, Baltimore, Maryland. The two have been collaborating to create a computerised system that overlays 2D magnetic resonance (MR) images onto a patient during surgical procedures, to improve needle placement.

Percutaneous procedures, which take place beneath the skin, are common in medicine. Using needles for such procedures requires considerable clinical acumen, and can take several attempts, leading to discomfort for the patient and extending the duration of the procedure.

Image-guided systems making use of computed tomography (CT) or magnetic resonance (MR) imaging can help doctors by providing tracking of the location of the needle, allowing them to ‘see’ beneath the skin more accurately (with what Fichtinger and co-authors call ‘X-ray vision’). However, new images must be taken for every insertion of the needle, so if it is not done perfectly the first time, this can greatly extend the length of a procedure. The aim is to be able to insert a needle in one go, thereby eliminating potential complications from multiple reinsertions. This would minimise the amount of human inaccuracy involved, and reduce the time and effort required to take new CT/MR images to guide insertion.


Students are testing the accuracy of electromagnetic tracking against optical tracking. Accurate real-time position tracking is an essential component of the image overlay and other navigation systems for medical interventions.

Overlay imagery
Overlay images can help guide the placement of the needle. Overlay imagery involves acquiring an image slice, adjusting the magnification and alignment of the image, and then displaying it where the physician can see it. This allows in situ visualisation. Overlay imagery can be used for a wide range of treatments such as biopsies or musculoskeletal interventions for the shoulder, hip, and spine, as well as in procedures such as the removal of kidney stones (nephrolithotomy).

Static Image Overlay Systems (SIOS) were developed to improve upon manual or CT/MR-based methods of needle placement. These systems use mirrors and monitors that are configured in a ‘viewbox’, so that the clinician can see a virtual image that appears to float at the correct position inside the patient.

The problem with SIOS
A weighty, counterbalanced mechanical arm, which holds the viewbox, is required for previous SIOS configurations, which are fixed either to the floor or the CT/MR scanner. This limits the range of motion possible and gets in the way of the practitioner. Additionally, the equipment takes up a lot of room, occupying valuable space in treatment rooms, and limiting precise movement. It can easily be knocked and requires careful calibration, which can limit its usefulness and ease of use.


The Mobile Image Overlay System in use as part of a training exercise.

MIOS to the rescue
Mobile Image Overlay Systems (MIOS) can be used with a tablet, so have a greatly reduced system weight, smaller dimensions, and improved tracking accuracy. The weight of an MIOS is eight times less than an SIOS, meaning the heavy counterbalanced arm is not needed, and its tracking is five times more precise. A table-mounted version is also available, which professionals found easier to use.

How it works
Previous incarnations of the MIOS were less easily calibrated and less accurate, but Drs Fichtinger and Fritz have subsequently improved the angle of the display, developed a self-calibrating model, and validated the tool to show that the error in precision is minimal.

The set-up
The viewbox is an integral part of MIOS. This is a handheld device that attaches the tablet to a mirror and the optical markers that are used to guide the image and calibrate the system. Optical markers on the side of the viewbox position the needle in the virtual image accurately, and markers on the patient track its position in the real world. These markers allow the position of the needle to be tracked precisely in three dimensions. The system uses open-source software to create ‘slices’ from these marker positions that are visualised on the tablet.

Optimisation and validation
Image overlay systems have been tested throughout their development. Drs Fichtinger and Fritz have tested various versions of the technology using phantoms (inanimate subjects, such as a melon or a full-size replica of the human body), cadavers, and real-life human studies. They have also performed usability tests, where practitioners have made use of the technology and evaluated its application to their work. From this latter study, they concluded that doctors found the technology useful, though the table-mounted version was easier to use than the hand-held tablet device.

Image-guided systems can help doctors by providing tracking of the location of the needle, allowing them to ‘see’ beneath the skin more accurately

The team has found that the optimum angle between the mirror and monitor is 90°, rather than the 60° that had previously been used. They have also improved the type of laser used and its alignment with the virtual image overlay.

Further, they have developed a calibration method. Two-step calibration takes place away from the patient, and is only required once. The first step is the system calibration, which requires three optical markers. The system is moved through space, and the changes, or transforms, that are registered are recorded using these trackers. These transforms can then be compared and the system can be calibrated accordingly.

The second step is to calibrate the system to each individual patient by overlapping known points on the patient’s body with the same points in the scanned image. The software also self-calibrates using trackers attached to the patient and viewbox, which reports the position of the patient in relation to a visualised virtual image plane.

Overall, the team has validated MIOS and shown that it is accurate enough to be a clinically useful tool.

Use with CT and MR
More experiments have shown that the use of overlay imagery with CT or MR can provide accurate needle placement by reducing the number of steps and rounds of imaging required, and offering a visual guide that improves the chances of achieving accurate needle placement in the first attempt.

Use in training
Further, a validation system called ‘Perk Station’ has been developed that is designed for training and evaluation. Studies show that medical students who were trained using the Perk Station software performed needle insertions more accurately than those who performed freehand insertions.

The team has made the code and blueprints freely available and open source, so that others can easily replicate and adapt the software and hardware. This is especially useful for training medical students and professionals who need to perfect their technique for these incredibly difficult and highly skilled procedures.

This feature article was created with the approval of the research team featured. This is a collaborative production, supported by those featured to aid free of charge, global distribution.

Want to read more articles like this?

Sign up to our mailing list and read about the topics that matter to you the most.
Sign Up!

Leave a Reply

Your email address will not be published.

Thank you for expressing interest in joining our mailing list and community. Below you can select how you’d like us to interact with you and we’ll keep you updated with our latest content.

Would you like to learn more about our services?


If you have any questions about how we handle your data, please review our privacy notice.


You can unsubscribe at any time by contacting us at


We use MailChimp and Salesforce Marketing Cloud as our marketing automation platforms. By submitting this form, you acknowledge that the information you provide will be transferred to MailChimp for processing in accordance with their Privacy notice and Terms, and to Salesforce Marketing Cloud in accordance with their Privacy notice and Terms.

Subscribe to our mailing list