Background: Technological advances have transformed vascular intervention in recent decades. In particular, improvements in imaging and data processing have allowed for the development of increasingly complex endovascular and hybrid interventions. Augmented reality (AR) is a subject of growing interest in surgery, with the potential to improve clinicians’ understanding of 3D anatomy and aid in the processing of real-time information. This study hopes to elucidate the potential impact of AR technology in the rapidly evolving fields of vascular and endovascular surgery.
Objective: The aim of this review is to summarize the fundamental concepts of AR technologies and conduct a scoping review of the impact of AR and mixed reality in vascular and endovascular surgery.
Methods: A systematic search of MEDLINE, Scopus, and Embase was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. All studies written in English from inception until January 8, 2021, were included in the search. Combinations of the following keywords were used in the systematic search string: (“augmented reality” OR “hololens” OR “image overlay” OR “daqri” OR “magic leap” OR “immersive reality” OR “extended reality” OR “mixed reality” OR “head mounted display”) AND (“vascular surgery” OR “endovascular”). Studies were selected through a blinded process between 2 investigators (JE and AS) and assessed using data quality tools.
Results: AR technologies have had a number of applications in vascular and endovascular surgery. Most studies (22/32, 69%) used 3D imaging of computed tomography angiogram–derived images of vascular anatomy to augment clinicians’ anatomical understanding during procedures. A wide range of AR technologies were used, with heads up fusion imaging and AR head-mounted displays being the most commonly applied clinically. AR applications included guiding open, robotic, and endovascular surgery while minimizing dissection, improving procedural times, and reducing radiation and contrast exposure.
Conclusions: AR has shown promising developments in the field of vascular and endovascular surgery, with potential benefits to surgeons and patients alike. These include reductions in patient risk and operating times as well as in contrast and radiation exposure for radiological interventions. Further technological advances are required to overcome current limitations, including processing capacity and vascular deformation by instrumentation.
Emerging technologies are transforming vascular surgery at a rapid pace. In particular, the introduction of endovascular techniques has opened the way for a Cambrian explosion of technological evolution in terms of both hardware  and software [ ].
One such technology is augmented reality (AR), which aims to enhance clinicians’ ability by offering intuitive augmentation of the real environment with computer-generated real-time input of virtual information. AR lies on the continuum from virtual reality (VR), in which the user is immersed in a completely virtual setting, to real life. AR allows for minimal interaction between the virtual and real worlds, whereas mixed reality (MR) involves a combination of the real and virtual worlds where both elements are able to interact .
Complications in vascular surgery such as wound infections are associated with the extent of dissection, size of the wound, and duration of the surgery . Anatomical localization with AR may help in reducing complications and improving overall outcomes. Although the “Getting It Right First Time” program used a principle of standardization across vascular surgery to improve outcomes in the United Kingdom [ ], AR has promise in allowing for personalization taking into account individual patient anatomy and pathology while standardizing the technical approach by providing intraprocedural guidance.
Surgical AR uses a range of technologies, including everyday smartphone devices  and commercial products, including AR head-mounted displays (HMDs), which offer developers flexibility, allowing clinicians to experience virtual content that is overlaid directly onto the present reality [ ].
Across a number of surgical specialties and subspecialties, the potential of AR as an important training tool has been identified . With increasingly specialized surgical practice, more patients with comorbidities, and prevalent ethical challenges for surgical training, trainees have become increasingly reliant on simulation. Simulation has been demonstrated to effectively reduce training risks and costs [ ]. Both VR and AR have been applied to surgical simulation successfully [ ]; however, the superimposition of real-time information onto the real world and the flexibility offered by AR could make it a more realistic and adaptable simulation tool [ ].
Literature on the applicability of AR in vascular surgery is extremely limited . A recent review on the applicability of HMDs and smart glasses in vascular surgery highlighted the application of AR HMDs in a number of surgical specialties. However, only 4 papers in this review reported applications relevant to vascular surgery in particular [ ]. To date, no review has been conducted on the application of the spectrum of AR technologies in the fields of vascular and endovascular surgery.
This scoping review aimed to systematically search the literature, identify the current applications of AR in vascular and endovascular surgery, and summarize key results and learning points while identifying avenues and gaps for future research in this evolving field.
A scoping review was chosen after advice from PROSPERO (international prospective register of systematic reviews), and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews)  was used to ensure validity.
The inclusion criteria were (1) studies in the English language, (2) a minimum of level V evidence using the Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence, (3) use of AR in vascular or endovascular surgery, and (4) applicability to clinical practice or training reported.
The exclusion criteria were (1) review articles or conference abstracts, (2) non–English-language articles, (3) articles lacking an available full text, (4) use of AR outside of vascular or endovascular surgery, and (5) use only of VR.
Search and Information Sources
A systematic search of the MEDLINE, Scopus, and Embase databases was performed, allowing for access to a range of global clinical, scientific, and engineering research. All studies written in English from inception until January 8, 2021, were included in the search.
The following keywords were used in the systematic search string for all 3 database searches: (“augmented reality” OR “hololens” OR “image overlay” OR “daqri” OR “magic leap” OR “immersive reality” OR “extended reality” OR “mixed reality” OR “head mounted display”) AND (“vascular surgery” OR “endovascular”). No limits were applied.
Selection of Sources of Evidence
First, a blinded and independent process of selection based on titles and abstracts was performed without collusion by 2 authors (AS and JE), with a third author (JS) consulted with regard to discrepancies. Next, a selection of eligible studies was conducted by analyzing the full texts.
Each study was evaluated individually, and the following variables were sought for data collection: year of study, category (endovascular surgery [aortic, peripheral, venous, or visceral], open surgery, and training), applied study or concept and design, number of patients (if the study was clinically applied), risk of bias (see the following section for the tools used), characteristics of the study group and operators, methods and outcomes, imaging type, and type of display (if applicable).
Critical Appraisal of Individual Sources of Evidence
The quality of the data was evaluated using the Cochrane tools Risk of Bias 2 for randomized trials and Risk of Bias In Non-randomized Studies of Interventions for nonrandomized trials [, ]. For those studies that included human participants but did not meet the criteria for the aforementioned scoring tools, the Johanna Briggs Institute Critical Appraisal Checklist was used, with a score derived based on the checklist. A score of 1 was assigned for Yes, a score of 0.5 was assigned for unclear, and a score of 0 was assigned for no. The results were presented as percentages. A higher percentage represented a reduced risk of bias.
Data Charting Process and Synthesis
Data were extracted from eligible studies into evidence tables to summarize the following: year of publication, type of study design, number of patients, method, and outcome. A second table presents more technical aspects of the research, including the type of imaging, methods for tracking or registration, and display. The data collected in the evidence tables were used to define the main themes of discussion. Any data related to the application of AR in vascular and endovascular surgery could be synthesized.
Selection of Sources of Evidence
A total of 726 articles were identified from the initial search across the 3 databases, with 32 (4.4%) meeting the inclusion and exclusion criteria and being included in the final results. Please seefor detailed information, including the reasons for article exclusion.
Characteristics of the Sources of Evidence
A total of 32 articles applicable to vascular and endovascular surgery met the criteria for inclusion. The included articles were sorted into three categories: open vascular surgery, endovascular surgery (subdivided into aortic, visceral, peripheral, and venous, where applicable), and training ([ , - ]). Many studies were preclinical in application, with 44% (14/32) including human study populations with limited numbers. There were no human studies with equivalent outcome measures for comparative quantitative data analysis.
Quality assessment could only be performed in cases where human study populations were evaluated, which was in 44% (14/32) of the studies. The Cochrane risk of bias assessment was not used as no randomized trials were included in our review. Among the clinical studies, 6% (2/32) were cohort studies [, ], which were found to have a moderate risk of bias according to the Cochrane risk of bias tool for nonrandomized trials [ ]. Of the 32 studies, 1 (3%) was a case-control study [ ] with a score of 55% according to the Joanna Briggs Institute Critical Appraisal Checklist, 7 (22%) were case series studies [ , , - ] with SD (16.1%) an average score of 70% (SD 16.1%; range 50%-80%), and 3 (9%) were case reports [ , , ] an average score of 65% (range 40%-95%).
The articles were further subdivided into studies that focused on concept and design to reflect the early stage of this technology in vascular surgery and those that conducted research directly applied to surgery, training, or simulation.
Synthesis of Results
The proportion of studies related to concept development of AR was 38% (12/32), whereas the rest were applied to clinical and simulation contexts. Of the 32 studies, 29 (91%) involved deriving digital structures of blood vessels from clinical imaging ([ , - ]), including noncontrast computed tomography (CT), CT angiography, cone-beam CT (CBCT), magnetic resonance angiography, conventional ultrasound (US), intravascular US, and 3D US.
Virtual data are processed into a virtual object and displayed to the clinician on a screen, projector, monitor, or AR HMD. Several steps are required to complete processing, including segmentation to reduce the model to the area of interest (eg, aorta), often by thresholding via a Hounsfield scale, for example, and then using a 3D computer-generated surface mesh, refining the mesh by filling holes or defects into a model and sometimes using a slicer to break up the model into its different anatomical sections .
Registration and Tracking
Registration and tracking are essential components of AR, allowing for the alignment of virtual and real data in a usable way. Registration is commonly performed using markers, where a particular pattern or anatomical landmark in the real world is used as a reference corresponding to a virtual marker derived from medical imaging . This can be performed manually or using trackers. Marker-less tracking is now possible owing to technological advances that correlate patterns in real and virtual data in real time [ , ]. Integrated or external optical sensors (infrared or color) can be used to track markers or recognize patterns from a patient’s anatomy. In recent years, Red, Green, Blue-Depth cameras that track both depth and color simultaneously have allowed for the contemporaneous tracking of the real world [ ]. Electromagnetic tracking is a useful method for tracking instruments deep below the surface [ ]. In many cases, such as with Microsoft HoloLens, a variety of sensor inputs are used for hybrid tracking [ ].
Challenges with these techniques include the deformability of the anatomy; for example, when stiff endovascular devices are inserted into blood vessels. These changes can cause registration errors, which can be clinically significant, for example, with complex visceral anatomy in endovascular aneurysm repair (EVAR). Groher et al  proposed computational algorithms to allow for some accurate deformability of 3D models for registration with 2D fluoroscopy. Another innovative tracking solution combating this problem involves electromagnetic trackers on the tips of angiography catheters, which can provide accurate information on the 3D position and orientation of the catheter in space, such as those used by Garcia-Vasquez et al [ ].
Alternatively, intravascular US on the tip of catheters can be used to create a 3D model of blood vessels during endovascular procedures with magnetic trackers to orientate the catheter in real time, as proposed by Shi et al . Parrini et al [ ] tested the use of freehand external magnets as a means of guiding magnetic endovascular devices to their targets using AR vessel models on a phantom.
Given the inherent reliance of endovascular procedures on imaging, the potential benefits of AR were identified early, in particular to improve clinicians’ understanding of 3D anatomy during critical steps as well as to reduce time and radiation and contrast exposure . Image overlay techniques are used for complex endovascular work worldwide and usually rely on merging live fluoroscopy with 2D or 3D images from x-ray, CT, or magnetic resonance imaging.
EVAR and Aortic Disease
EVAR is increasingly being performed for the repair of both complex and conventional infrarenal abdominal aortic aneurysms in the elective and emergency settings.
AR visualization has now been used in planning and EVAR navigation, with a case report by Rynio et al  describing the use of an AR HMD to project an aortic aneurysm and bones (vertebral column lying posterior to the aneurysm) as a 3D hologram that responds to gestures and voice commands.
A “3D road map” published in 2013 was developed by Fukuda et al  using preoperative CT imaging, where bone marrow subtraction and the use of the iliac crest and lumbar vertebrae as landmarks resulted in the creation of an image overlay to guide aortic endografting. This reduced the need for digital subtraction angiography (DSA) and its associated risks, ultimately giving rise to fusion imaging.
Fusion imaging has been used for endovascular navigation in hybrid vascular operating theaters, particularly in complex fenestrated EVARs. This uses preoperative 3D CT imaging to overlay vascular structures onto perioperative 2D fluoroscopy images with the aim of reducing fluoroscopy time, contrast agent dosing, and overall operating time. This can also be used in a conventional operating theater with a mobile C-arm with good results for conventional aortic endografting, as reported by Kaladji et al . Koutouzi et al [ ] modified this modality to require carbon dioxide DSA to confirm accurate registration of the CT-derived 3D vascular overlay to avoid iodinated contrast in patients with poor renal function or contrast allergies.
Kaladji et al  went on to successfully demonstrate the use of fusion imaging to perform EVAR without the use of any iodinated contrast in the pre- or perioperative phase in a case series of 6 patients. Patients with severe chronic kidney disease underwent unenhanced CT imaging. Centerlines for EVAR planning were manually drawn, and key anatomical points or landing zones were marked following segmentation and processing. A preoperative 3D image overlay reconstruction with these markers was then projected onto 2D fluoroscopy imaging to guide the placement of the aortic endograft without the use of a contrast agent ( ). No endoleaks were noted during postoperative duplex surveillance, with minimal error in the positioning of the aortic endograft on postoperative CT imaging and no significant reduction in renal function.
A series of 101 patients by Schulz et al  in 2016 tested the use of image overlay alone in patients undergoing conventional infrarenal EVAR. Fusion image overlay using images from preoperative CT angiograms with images obtained from an intraoperative CBCT was compared with intraoperative DSA. Although <5-mm accuracy was observed in most patients (68% of the patient cohort), significant deviations were noted in some patients, including significant caudal deviation in 9 patients (9% of the patient cohort), which would have resulted in coverage of the lowest-lying renal artery, resulting in significant operative morbidity. Therefore, a contrast injection study for cannulation of the lowest-lying visceral vessel was recommended.
Koutouzi et al  described a technique that aims to use image overlay to reduce the risk of covering intercostal arteries during thoracic EVAR (TEVAR), which may affect spinal cord perfusion, causing paraparesis. Intraoperative CBCT was used to perform 3D-3D registration with a preoperative CT angiogram to enhance accuracy. Once calibrated onto live fluoroscopy, this overlaid 3D model was used to guide the TEVAR and avoid preplanned intercostal arteries. A case series of 7 was reported without spinal cord injury.
A common problem with image overlay techniques is the nondeformability of a rigid 3D model created from a preoperative CT scan. Garcia-Vasquez et al  attempted to overcome this problem using 3D US to create a real-time model of the patients’ anatomy that was visible to a clinician wearing the AR HMD. Coupled with the electromagnetic catheter tip tracking system used by von Haxthausen et al [ ], this concept has been proposed to perform EVAR completely free of radiation and intravenous contrast and tested on a phantom model with a 3D model of an aortic aneurysm [ ]. The concept showed promising progress ( ); however, issues around registration accuracy remain.
Endovascular—Peripheral Arterial Disease
Endovascular techniques are being applied to increasingly complex surgical problems in patients with comorbidities. The potential to reduce radiation and contrast exposure while navigating tortuous vessels makes the application of AR technology more attractive.
Lu et al  described successful retrograde peroneal access through an AR system with CT angiogram image–derived overlay of fluoroscopic images. AR glasses were worn by the operator to guide the needle trajectory. The case report highlighted technical difficulties, including the need for manual registration and having to align virtual dots on their AR HMD with fiducial markers on the patient’s leg. In addition, aligning the trajectory of the needle path on the AR HMD required the acquisition of new technical skills.
The work of Goudeketting et al  focused on the accuracy of image fusion based on preprocedural contrast-enhanced magnetic resonance angiography during percutaneous angioplasty of iliac lesions. They found that guidewires and endovascular catheters did not cause significant vessel displacement to influence image fusion according to angiographic experts.
Swerdlow et al  retrospectively compared carotid stenting with 2D-3D image fusion (46 patients) and without (70 patients). Magnetic resonance imaging or CT angiography images were overlaid onto the real-time 2D fluoroscopy. They observed significantly improved cannulation times and reduced radiation exposure using this technique.
Endovascular venous intervention, compared with arterial intervention, can be more challenging because of the lack of landmarks from vessel calcification and reduced vessel wall thickness. Schwein et al  used image fusion techniques with magnetic resonance venography to successfully recanalize 4 patients with central venous occlusion. It was felt that magnetic resonance venography image fusion improved clinician confidence and the safety of difficult venous endovascular navigation.
“Traditional” open vascular surgery has increasingly been replaced by novel endovascular techniques, which, in combination with problems with tissue deformation, as previously described, may explain why AR is less well researched in this field.
The use of a new image overlay system has been demonstrated successfully in lower limb revascularization surgery by Mochazuki et al . This system used preoperative CT imaging to help locate the target distal anastomotic site and plan a limited incision accordingly using a branch artery as a reference point ( ).
Modern-day mobile phones have multiple sensors that may be used for AR-assisted surgery. The benefits of easy accessibility (most operators have access to a mobile phone) and cost make them an attractive option. An AR-assisted surgery system was developed by Aly  to provide 3D guidance during vascular procedures. CT angiogram–derived images were used by a smartphone to produce a 3D guidance model fused with the patient’s anatomy, and rotational and positional tracking were compared for planning an iliofemoral bypass as well as for an endovascular sheath placement in the common femoral artery ( ).
The use of AR with robot-assisted surgery is of increasing academic interest worldwide. Pietrabissa et al  successfully used an AR HMD to superimpose a surgical anatomy from CT angiography onto the patient’s body for preoperative planning to guide trocar placement and dissection so as to minimize disruption using the da Vinci surgical system. Although robot-assisted surgery is becoming increasingly common for vascular interventions, the use of AR is in the early stages of adoption and investigation.
Training and Simulation
Published studies on AR vascular and endovascular training were limited in our review to 16% (5/32) of the papers. Of these 5 papers, 1 (20%) applied this to training vascular surgeons, whereas 4 (80%) focused on design concepts.
Mangina et al  tested the concept of creating a 3D model of the aorta from CT angiography images using a number of programming tools and platforms, including Unity, ARToolKit, and Pro/ENGINEER. A model aorta was created with the potential for merging with VR and AR tracking technologies to create an accurate training and educational tool for application to both open and endovascular training.
Bartesaghi et al  used similar techniques to create a 3D model of the aorta projected onto a mannequin to allow for 3D AR simulation of an EVAR. The original model was derived from CT angiography images, and segmentation with meshing was performed to create the model. The software was optimized for the interplay between the simulation and a hardware mannequin, allowing surgical trainees to learn the steps of an EVAR.
Similarly, Anderson et al  described the development of a PC-based AR simulation system for interventional radiology ( ). This system allows for real-time manipulation of catheters and guidewires through a tactile user interface device that creates hand-eye coordinated realism of the movement of instruments over 3D-generated models of vessels. Extra enhancements have been built in, including the administration and subsequent washout of contrast and control of the C-arm. The paper reports good initial feedback from neurological and peripheral vascular interventionalists who felt it had 75% of the desired functionality and was useful for educational simulation, with further work required to achieve sufficient realism for preoperative planning.
Crowley et al [, ] built on the work of Mangina et al [ ], further exploring the concept of EVAR MR simulation. They found that the creation of a solid 3D-printed model of an aortic aneurysm through which real instruments (guidewires, catheters, and delivery devices) can be placed and manipulated gave the simulation tactile realism. A mobile device provides the user with fluoroscopic data through an AR software (ARToolKit or Daqri) while haptic feedback is provided by the 3D model. Limitations of 3D-printed models include the lack of vascular wall deformability and the realism of feedback from catheter tips against the model wall. The system’s usability was tested by medical students and university lecturers.
An AR system for US-guided endovascular surgical training was devised by Cheng et al . Live US images were transferred to a prescanned phantom model of the aorta and iliac arteries, which was extended to a computer-assisted remote endovascular surgery model. The user is able to navigate an endovascular catheter using a robotic device with computer-generated images to augment the 3D understanding of the catheter in space. The phantom provides both haptic and visual feedback aiming to be as realistic as possible.
Rudarakanchana et al  applied the concept of simulation to the pressured setting of ruptured EVAR. This was achieved through VR simulators being integrated into a simulated angio suite with the whole team present. A total of 10 teams were tested: 5 led by interventional specialists and 5 led by trainees. This study measured the time to achieve proximal control and total procedure and fluoroscopy times. Experts were significantly faster than trainees using reduced fluoroscopy time, suggesting that the simulation model had good applicability to real-life experience. In particular, the value of the simulation for improving team communication and human factor skills was highlighted through feedback, suggesting that AR and MR simulation could be useful adjuncts to improving ruptured EVAR outcomes.
Our review demonstrates that the most used modality for obtaining data for display is CT angiography (), with CBCT often used in the registration process with the patient in the operating suite. The type of display will depend on the application, and a range of display types were included, with image fusion studies making monitors the most prevalent display ( ).
|Data source||Studies, n (%)|
|Noncontrast CTa||1 (3)|
|CT angiography||22 (69)|
|Cone-beam CT||11 (34)|
|MRb angiography or venography||3 (9)|
|Conventional USc||1 (3)|
|Intravascular US||1 (3)|
|3D US||2 (6)|
|Rotational XRd||1 (3)|
|Computer-generated model||3 (9)|
aCT: computed tomography.
bMR: magnetic resonance.
|Types of AR display||Studies, n (%)|
|AR HMDa (including HoloLens)||9 (28)|
|Mobile phone||2 (6)|
aHMD: head-mounted display.
Summary of Evidence
The reviewed studies demonstrate that AR in surgery often relies on the registration of a virtual image or object onto the real patient using a tracking method, which can trace the real environment and place virtual objects in the correct position and orientation. Our study revealed a wide range of applications of AR in vascular surgery. Image overlay technology is increasingly used, with its value being reported in endovascular treatment of peripheral arterial disease , deep venous disease [ ], carotid stenting [ ], and aortic disease (EVAR [ , ], TEVAR [ ], and complex EVAR cases [ , ]). There is evidence that it can improve technical success while reducing procedural times, radiation dose, and contrast volume.
However, concerns remain regarding the accuracy of image registration, especially with more complex anatomies . A common problem is the deformation of vascular structures with respiration, surgical manipulation, or stretching out of vessels (such as iliac and target visceral arteries for EVAR) with rigid stent delivery systems. Emerging technologies offer novel solutions for improving the real-time accuracy of overlaid data. Robust deformable registration algorithms or intraoperative real-time 3D scanners are potential solutions to assist AR-guided surgery on soft or deformable structures. In particular, 3D and intravascular US offer radiation- and contrast-free modalities to create real-time images for intraoperative use [ ], whereas electromagnetic tracking provides contemporaneous spatial information about catheter devices [ ]. The integration of these technologies demands high levels of data transmission and processing, which must be met by technology to prevent lag [ ].
AR HMDs allow the wearer hands-free access to virtual data overlaid directly onto the patient. Several studies with EVAR in particular have shown early promise that they can reduce the need for radiation and contrast exposure while improving accuracy [, , ]. Studies on open and robotic surgery corroborate this [ , ]. AR HMDs such as HoloLens still have a few technical concerns that could restrict their usefulness. Their weight (500-645 g depending on the device and manufacturer) can cause discomfort and fatigue during long procedures. AR HMD processing capacity and memory are limited, which can restrict certain applications. The immersive experience can be limited by a restricted field of view and projection size, whereas image quality and computational time will improve as the technology evolves. There is no consensus yet on the optimal methods for merging and displaying virtual and real information so that depth perception or focus and visual clutter do not distract the wearer [ , ].
Most of the included studies (28/32, 88%) described small-scale implementations of AR with limited study participants and a wide range of AR hardware and software platforms in their research. Only 44% (14/32) of the studies included human study populations. As AR becomes more ubiquitous for vascular surgery, more evidence will become available, and we believe that these limitations will pose less of an issue.
A significant proportion of the studies were preclinical in their applications and demonstrated proof-of-concept findings. Nevertheless, AR solutions are applicable to a wide spectrum of fields in vascular surgery, including those highlighted in this scoping review.
Three databases (MEDLINE, Embase, and Scopus) were searched for this review, allowing for a broad assessment of the current impact of AR clinically and educationally while also scoping potential future applications. However, in this fast-moving field, AR applications under development by private enterprises were not included in the contents of our review. Although our selection criteria focused on vascular and endovascular surgery, there is significant crossover with work applied to other medical and surgical specialties, and wider development within interventional radiology and allied surgical specialties will further inform applications within vascular and endovascular surgery.
AR has shown potential to enhance accuracy and reduce procedural time, radiation exposure, and contrast dose in a range of vascular surgery applications by digitally augmenting the clinicians’ procedural ability, reducing surgical risk, and improving patient outcomes. Clinicians, who demand high levels of accuracy and patient safety, should be aware of potential technical pitfalls when using AR technology on patients but should also be aware of the potential benefits. AR has also been shown to be an increasingly valuable tool in surgical simulation and education. As technology improves, it can be expected that AR will become increasingly relied upon as an arrow in the surgical quiver and that more applications will be found for AR, which could benefit clinicians. Future development and studies should assess whether the use of AR affords improvements in patient experience and in clinical effectiveness as objective measures of improvement in outcomes and cost-effectiveness of this technology.
Conflicts of Interest
JS received grant funding from the UK National Institute for Health Research and the British Heart Foundation, as well as consulting with Oxford HealthTech Ltd. All the funding was paid to the institution and is not related to this publication. DA is a clinical advisor for Medical iSight Ltd.
Studies applicable to augmented reality in vascular and endovascular surgery.DOCX File , 28 KB
Imaging, registration, and display types within review.DOCX File , 21 KB
- Cheng S. Novel endovascular procedures and new developments in aortic surgery. Br J Anaesth 2016 Sep;117 Suppl 2:ii3-i12 [FREE Full text] [CrossRef] [Medline]
- Bedi VS, Sharma N. Game changers in vascular and endovascular surgery. Indian J Vasc Endovasc Surg 2020;7:13-17. [CrossRef]
- Rahman R, Wood ME, Qian L, Price CL, Johnson AA, Osgood GM. Head-mounted display use in surgery: a systematic review. Surg Innov 2020 Mar 12;27(1):88-100. [CrossRef] [Medline]
- Gwilym BL, Saratzis A, Benson R, Forsythe R, Dovell G, Dattani N, et al. Groin wound infection after vascular exposure (GIVE) multicentre cohort study. Int Wound J 2020 Nov 25;18(2):164-175. [CrossRef]
- Gray WK, Day J, Horrocks M. Editor's choice - volume-outcome relationships in elective abdominal aortic aneurysm surgery: analysis of the uk hospital episodes statistics database for the getting it right first time (GIRFT) programme. Eur J Vasc Endovasc Surg 2020 Oct;60(4):509-517 [FREE Full text] [CrossRef] [Medline]
- Aly O. Assisting vascular surgery with smartphone augmented reality. Cureus 2020 May 08;12(5):e8020 [FREE Full text] [CrossRef] [Medline]
- Loukas C. Surgical simulation training systems: box trainers, virtual reality and augmented reality simulators. Int J Advanced Robot Auto 2016 Jul 11;1(2):1-9. [CrossRef]
- Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, et al. Training and simulation for patient safety. Qual Saf Health Care 2010 Aug 06;19 Suppl 2(Suppl 2):i34-i43 [FREE Full text] [CrossRef] [Medline]
- McKnight RR, Pean CA, Buck JS, Hwang JS, Hsu JR, Pierrie SN. Virtual reality and augmented reality-translating surgical training into surgical technique. Curr Rev Musculoskelet Med 2020 Dec 11;13(6):663-674 [FREE Full text] [CrossRef] [Medline]
- Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X. A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices 2021 Jan 16;18(1):47-62. [CrossRef] [Medline]
- Lareyre F, Chaudhuri A, Adam C, Carrier M, Mialhe C, Raffort J. Applications of head-mounted displays and smart glasses in vascular surgery. Ann Vasc Surg 2021 Aug;75:497-512. [CrossRef] [Medline]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372:n71 [FREE Full text] [CrossRef] [Medline]
- Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016 Oct 12;355:i4919 [FREE Full text] [CrossRef] [Medline]
- Mayhew AD, Kabir M, Ansari MT. Considerations from the risk of bias perspective for updating Cochrane reviews. Syst Rev 2015 Oct 06;4(1):136 [FREE Full text] [CrossRef] [Medline]
- Swerdlow NJ, Jones DW, Pothof AB, O'Donnell TF, Liang P, Li C, et al. Three-dimensional image fusion is associated with lower radiation exposure and shorter time to carotid cannulation during carotid artery stenting. J Vasc Surg 2019 Apr;69(4):1111-1120 [FREE Full text] [CrossRef] [Medline]
- Rynio P, Witowski J, Kamiński J, Serafin J, Kazimierczak A, Gutowski P. Holographically-guided endovascular aneurysm repair. J Endovasc Ther 2019 Aug 13;26(4):544-547. [CrossRef] [Medline]
- Grinshpoon A, Sadri S, Loeb GJ, Elvezio C, Feiner SK. Hands-free interaction for augmented reality in vascular interventions. In: Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 2018 Presented at: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR); Mar 18-22, 2018; Tuebingen/Reutlingen, Germany.
- Kilian-Meneghin J, Xiong Z, Guo C, Rudin S, Bednarek DR. Evaluation of methods of displaying the real-time scattered radiation distribution during fluoroscopically-guided interventions for staff dose reduction. Proc SPIE Int Soc Opt Eng 2018 Feb;10573:1057366 [FREE Full text] [CrossRef] [Medline]
- Schulz CJ, Schmitt M, Böckler D, Geisbüsch P. Fusion imaging to support endovascular aneurysm repair using 3D-3D registration. J Endovasc Ther 2016 Oct 28;23(5):791-799. [CrossRef] [Medline]
- Cheng I, Shen R, Moreau R, Brizzi V, Rossol N, Basu A. An augmented reality framework for optimization of computer assisted navigation in endovascular surgery. In: Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014 Presented at: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; Aug 26-30, 2014; Chicago, IL, USA. [CrossRef]
- Parrini S, Cutolo F, Freschi C, Ferrari M, Ferrari V. Augmented reality system for freehand guide of magnetic endovascular devices. In: Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014 Presented at: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; Aug 26-30, 2014; Chicago, IL, USA. [CrossRef]
- van den Berg JC. Update on new tools for three-dimensional navigation in endovascular procedures. Aorta (Stamford) 2014 Dec;2(6):279-285 [FREE Full text] [CrossRef] [Medline]
- Wang J, Fallavollita P, Wang L, Kreiser M, Navab N. Augmented reality during angiography: integration of a virtual mirror for improved 2D/3D visualization. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 2012 Presented at: IEEE International Symposium on Mixed and Augmented Reality (ISMAR); Nov 05-08, 2012; Atlanta, GA, USA. [CrossRef]
- Groher M, Zikic D, Navab N. Deformable 2D-3D registration of vascular structures in a one view scenario. IEEE Trans Med Imaging 2009 Jun;28(6):847-860. [CrossRef]
- Kaladji A, Villena A, Pascot R, Lalys F, Daoudal A, Clochard E, et al. Fusion imaging for EVAR with mobile C-arm. Ann Vasc Surg 2019 Mar;55:166-174. [CrossRef] [Medline]
- von Haxthausen F, Jäckle S, Strehlow J, Ernst F, García-Vázquez V. Catheter pose-dependent virtual angioscopy images visualized on augmented reality glasses. Curr Direction Biomed Eng 2019;5(1):289-291. [CrossRef]
- García-Vázquez V, von Haxthausen F, Jäckle S, Schumann C, Kuhlemann I, Bouchagiar J, et al. Navigation and visualisation with HoloLens in endovascular aortic repair. Innov Surg Sci 2018 Sep;3(3):167-177 [FREE Full text] [CrossRef] [Medline]
- Koutouzi G, Sandström C, Skoog P, Roos H, Falkenberg M. 3D image fusion to localise intercostal arteries during TEVAR. EJVES Short Rep 2017;35:7-10 [FREE Full text] [CrossRef] [Medline]
- Shi C, Tercero C, Ikeda S, Ooe K, Fukuda T, Komori K, et al. In vitro three-dimensional aortic vasculature modeling based on sensor fusion between intravascular ultrasound and magnetic tracker. Int J Med Robot 2012 Sep 01;8(3):291-299. [CrossRef] [Medline]
- Rolls AE, Maurel B, Davis M, Constantinou J, Hamilton G, Mastracci TM. A comparison of accuracy of image- versus hardware-based tracking technologies in 3D fusion in aortic endografting. Eur J Vasc Endovasc Surg 2016 Sep;52(3):323-331 [FREE Full text] [CrossRef] [Medline]
- Duménil A, Kaladji A, Castro M, Göksu C, Lucas A, Haigron P. A versatile intensity-based 3D/2D rigid registration compatible with mobile C-arm for endovascular treatment of abdominal aortic aneurysm. Int J Comput Assist Radiol Surg 2016 Sep 26;11(9):1713-1729. [CrossRef] [Medline]
- Koutouzi G, Henrikson O, Roos H, Zachrisson K, Falkenberg M. EVAR guided by 3D image fusion and CO2 DSA: a new imaging combination for patients with renal insufficiency. J Endovasc Ther 2015 Dec 17;22(6):912-917. [CrossRef] [Medline]
- Kaladji A, Dumenil A, Mahé G, Castro M, Cardon A, Lucas A, et al. Safety and accuracy of endovascular aneurysm repair without pre-operative and intra-operative contrast agent. Eur J Vasc Endovasc Surg 2015 Mar;49(3):255-261 [FREE Full text] [CrossRef] [Medline]
- Fukuda T, Matsuda H, Doi S, Sugiyama M, Morita Y, Yamada M, et al. Evaluation of automated 2D-3D image overlay system utilizing subtraction of bone marrow image for EVAR: feasibility study. Eur J Vasc Endovasc Surg 2013 Jul;46(1):75-81 [FREE Full text] [CrossRef] [Medline]
- Carrell TW, Modarai B, Brown JR, Penney GP. Feasibility and limitations of an automated 2D-3D rigid image registration system for complex endovascular aortic procedures. J Endovasc Ther 2010 Aug;17(4):527-533. [CrossRef] [Medline]
- Lu W, Wang L, Zhou W, Fu W. Augmented reality navigation to assist retrograde peroneal access for the endovascular treatment of critical limb ischemia. J Vasc Surg Cases Innov Tech 2019 Dec;5(4):518-520 [FREE Full text] [CrossRef] [Medline]
- Goudeketting SR, Heinen SG, van den Heuvel DA, van Strijen MJ, de Haan MW, Slump CH, et al. The use of 3D image fusion for percutaneous transluminal angioplasty and stenting of iliac artery obstructions: validation of the technique and systematic review of literature. J Cardiovasc Surg 2018 Feb;59(1):26-36. [CrossRef]
- Schwein A, Lu T, Chinnadurai P, Kitkungvan D, Shah D, Chakfe N, et al. Magnetic resonance venography and three-dimensional image fusion guidance provide a novel paradigm for endovascular recanalization of chronic central venous occlusion. J Vasc Surg Venous Lymphat Disord 2017 Jan;5(1):60-69. [CrossRef] [Medline]
- Pietrabissa A, Morelli L, Ferrari M, Peri A, Ferrari V, Moglia A, et al. Mixed reality for robotic treatment of a splenic artery aneurysm. Surg Endosc 2010 May 14;24(5):1204. [CrossRef] [Medline]
- Mochizuki Y, Hosaka A, Kamiuchi H, Nie JX, Masamune K, Hoshina K, et al. New simple image overlay system using a tablet PC for pinpoint identification of the appropriate site for anastomosis in peripheral arterial reconstruction. Surg Today 2016 Dec 17;46(12):1387-1393. [CrossRef] [Medline]
- Mangina E, Almaksy A, Campbell A. 3D modeling for augmented reality systems in novel vascular models. In: Proceedings of the 10th EAI International Conference on Simulation Tools and Techniques. 2017 Presented at: SIMUTOOLS '17: 10th EAI International Conference on Simulation Tools and Techniques; Sep 11 - 13, 2017; Hong Kong China. [CrossRef]
- 42 BE, Felle P, Crowley C, Jones J, Mangina E, Campbell A. Augmented reality EVAR training in mixed reality educational space. IEEE Global Engineering Education Conference, EDUCON. IEEE Computer Society. pp. 1571– 2017:9. [CrossRef]
- Bartesaghi S, Colombo G, Morone S. Spatial augmented reality and simulations to improve abdominal aortic aneurysm diagnosis and monitoring. Comput Aided Des Appl 2015 Jun 19;12(6):803-810. [CrossRef]
- Rudarakanchana N, Van Herzeele I, Bicknell CD, Riga CV, Rolls A, Cheshire NJ, et al. Endovascular repair of ruptured abdominal aortic aneurysm: technical and team training in an immersive virtual reality environment. Cardiovasc Intervent Radiol 2014 Aug 7;37(4):920-927. [CrossRef] [Medline]
- Anderson J, Chui C, Cai Y, Wang Y, Li Z, Ma X, et al. Virtual reality training in interventional radiology: the Johns Hopkins and Kent Ridge digital laboratory experience. Semin intervent Radiol 2002;19(2):179-186. [CrossRef]
- Sun Q, Mai Y, Yang R, Ji T, Jiang X, Chen X. Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int J Comput Assist Radiol Surg 2020 Nov 18;15(11):1907-1919. [CrossRef] [Medline]
- Andrews CM, Henry AB, Soriano IM, Southworth MK, Silva JR. Registration techniques for clinical applications of three-dimensional augmented reality devices. IEEE J Transl Eng Health Med 2021;9:1-14. [CrossRef]
- Wang X, Habert S, Zu Berge CS, Fallavollita P, Navab N. Inverse visualization concept for RGB-D augmented C-arms. Comput Biol Med 2016 Oct 01;77:135-147. [CrossRef] [Medline]
- Horn M, Ernst F, Kuhlemann I, Stahlberg E, Goltz JP, Wiedner M, et al. Experimental visualization of vascular structures with Microsoft Hololens – set-up for navigated contrast agent and radiation-sparing endovascular procedures (NAV-CARS EVAR). Eur J Vasc Endovasc Surg 2019 Dec;58(6):e104. [CrossRef]
- van den Berg JC. Three-dimensional image overlay to assist endovascular procedures. Vascular Disease Manag 2013;10(9):E179-E184 [FREE Full text]
- Burke E, Mangina E, Campbell A, Crowley C, Felle A, Jones J. Augmented reality EVAR training in mixed reality educational space. In: Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON). 2017 Presented at: 2017 IEEE Global Engineering Education Conference (EDUCON); Apr 25-28, 2017; Athens, Greece. [CrossRef]
- Cheng SWK. Novel endovascular procedures and new developments in aortic surgery. Br J Anaesth 2016 Sep;117 Suppl 2:ii3-ii12 [FREE Full text] [CrossRef] [Medline]
- Ferrari V, Carbone M, Condino S, Cutolo F. Are augmented reality headsets in surgery a dead end? Expert Rev Med Devices 2019 Dec 22;16(12):999-1001. [CrossRef] [Medline]
- Pratt P, Ives M, Lawton G, Simmons J, Radev N, Spyropoulou L, et al. Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur Radiol Exp 2018;2(1):2 [FREE Full text] [CrossRef] [Medline]
- Park BJ, Hunt SJ, Martin C, Nadolski GJ, Wood BJ, Gade TP. Augmented and mixed reality: technologies for enhancing the future of IR. J Vasc Interv Radiol 2020 Jul;31(7):1074-1082 [FREE Full text] [CrossRef] [Medline]
|AR: augmented reality|
|CBCT: cone-beam computed tomography|
|CT: computed tomography|
|DSA: digital subtraction angiography|
|EVAR: endovascular aneurysm repair|
|HMD: head-mounted display|
|MR: mixed reality|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews|
|PROSPERO: international prospective register of systematic reviews|
|TEVAR: thoracic endovascular aneurysm repair|
|VR: virtual reality|
Edited by N Zary; submitted 26.10.21; peer-reviewed by F Velayati, R de Leeuw, H Ayatollahi, J Egger; comments to author 16.01.22; revised version received 22.04.22; accepted 23.06.22; published 23.09.22Copyright
©Joshua Eves, Abhilash Sudarsanam, Joseph Shalhoub, Dimitri Amiras. Originally published in JMIR Serious Games (https://games.jmir.org), 23.09.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.