This paper is in the following e-collection/theme issue:

Games for Medical Education and Training (88) e-Learning and Medical Education (645) Nursing Education and Training (107) Augmented Reality (incl. Google Glass) Applications (88) Virtual Reality and Augmented Reality in Medical Education (66) New Methods and Approaches in Medical Education (192) Training and Education in XR and Spatial Computing (65)

Published on in Vol 10, No 3 (2022): Jul-Sep

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38433, first published .
An Augmented Reality–Based Guide for Mechanical Ventilator Setup: Prospective Randomized Pilot Trial

An Augmented Reality–Based Guide for Mechanical Ventilator Setup: Prospective Randomized Pilot Trial

An Augmented Reality–Based Guide for Mechanical Ventilator Setup: Prospective Randomized Pilot Trial

Authors of this article:

Sejin Heo1, 2 Author Orcid Image ;   Suhyeon Moon3 Author Orcid Image ;   Minha Kim1 Author Orcid Image ;   Minsu Park4 Author Orcid Image ;   Won Chul Cha1, 2 Author Orcid Image ;   Meong Hi Son2, 5 Author Orcid Image
Article Authors Cited by (7) Tweetations (5) Metrics

    Original Paper

    1Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea

    2Department of Digital Health, Samsung Advanced Institute for Health Sciences and Technology, Sungkyunkwan University, Seoul, Republic of Korea

    3Research Institute for Future Medicine, Samsung Medical Center, Seoul, Republic of Korea

    4Department of Information and Statistics, Chungnam National University, Daejeon, Republic of Korea

    5Department of Pediatrics, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea

    Corresponding Author:

    Meong Hi Son, MD

    Department of Pediatrics

    Samsung Medical Center

    Sungkyunkwan University School of Medicine

    81, Irwon-ro, Gangnam-gu

    Seoul, 06351

    Republic of Korea

    Phone: 82 34103539

    Email: meonghi.son@samsung.com


    Background: Recently, the demand for mechanical ventilation (MV) has increased with the COVID-19 pandemic; however, the conventional approaches to MV training are resource intensive and require on-site training. Consequently, the need for independent learning platforms with remote assistance in institutions without resources has surged.

    Objective: This study aimed to determine the feasibility and effectiveness of an augmented reality (AR)–based self-learning platform for novices to set up a ventilator without on-site assistance.

    Methods: This prospective randomized controlled pilot study was conducted at Samsung Medical Center, Korea, from January to February 2022. Nurses with no prior experience of MV or AR were enrolled. We randomized the participants into 2 groups: manual and AR groups. Participants in the manual group used a printed manual and made a phone call for assistance, whereas participants in the AR group were guided by AR-based instructions and requested assistance with the head-mounted display. We compared the overall score of the procedure, required level of assistance, and user experience between the groups.

    Results: In total, 30 participants completed the entire procedure with or without remote assistance. Fewer participants requested assistance in the AR group compared to the manual group (7/15, 47.7% vs 14/15, 93.3%; P=.02). The number of steps that required assistance was also lower in the AR group compared to the manual group (n=13 vs n=33; P=.004). The AR group had a higher rating in predeveloped questions for confidence (median 3, IQR 2.50-4.00 vs median 2, IQR 2.00-3.00; P=.01), suitability of method (median 4, IQR 4.00-5.00 vs median 3, IQR 3.00-3.50; P=.01), and whether they intended to recommend AR systems to others (median 4, IQR 3.00-5.00 vs median 3, IQR 2.00-3.00; P=.002).

    Conclusions: AR-based instructions to set up a mechanical ventilator were feasible for novices who had no prior experience with MV or AR. Additionally, participants in the AR group required less assistance compared with those in the manual group, resulting in higher confidence after training.

    Trial Registration: ClinicalTrials.gov NCT05446896; https://beta.clinicaltrials.gov/study/NCT05446896

    JMIR Serious Games 2022;10(3):e38433

    doi:10.2196/38433

    Keywords

    augmented reality; mechanical ventilation education; medical education; critical care; medical training; virtual reality; medical education; virtual education; nurse; nursing education; nursing; health care professional; learning platform; digital learning; digital health 


    Mechanical ventilation (MV) is a lifesaving treatment that reduces the difficulty of breathing in patients and reverses acute life-threatening respiratory failure [1]. During the COVID-19 pandemic, the incidence of acute respiratory failure increased, leading to an increase in the demand for not only physical resources, such as ventilators and intensive care unit (ICU) beds, but also the ability to provide MV care expertise [2-4]. Effective and continuous MV education is important because adequate MV support improves clinical outcomes [5-8]. Regarding MV education, ICU nurses considered ventilator setup as an important topic; In ventilator setup, hands-on training is the most beneficial, suggesting that workshops or self-learning packages are not sufficient for novices to learn how to set up a ventilator [9]. However, conventional education usually focuses on theoretical knowledge (eg, prevention of infection and mode settings), and the type of hands-on training or bedside training that is required is human resource and time intensive, which limits educating several essential trainees [10-12].

    Recently, augmented reality (AR) systems have been widely applied in medical education and training [13-16]. The AR system enables virtual objects to be overlaid onto a real-world environment by visualizing the physiological anatomy or enhancing the operator’s view [17,18]. A few AR-guided medical procedure training regimes have been reported in the emergency department and intensive care environments [13,15,19-21]. They suggested that AR systems are effective in step-by-step procedures; however, the studies were limited to procedures lasting less than 10 minutes or to simple steps that did not reflect the usual complexity of procedures performed in the ICU or emergency department [22,23]. Additionally, limited research has been conducted on the independence or accuracy of the step-by-step procedures in AR systems [19].

    In this study, we aimed to determine the effectiveness and feasibility of AR-based learning for novices to set up a ventilator by focusing on independently completing the procedures and assessing the degree of assistance required. Additionally, we evaluated the step characteristics in terms of the precision and assistance required.


    This was a prospective randomized controlled pilot study conducted at Samsung Medical Center, Korea, from January to February 2022. We compared 2 modes of training, namely, the conventional method (via the printed manual) and the AR-based instructions. This study followed the CONSORT reporting guidelines (Multimedia Appendix 1).

    Ethical Considerations

    The research design was approved by the institutional review board of Samsung Medical Center (2021-12-112). Prior to inclusion in the study, all participants provided written informed consent.

    Participants

    We recruited nurses from the Samsung Medical Center who were interested in AR and ventilator education using a web-based hospital bulletin board. We enrolled nurses who had no prior experience with ventilator setup or AR systems, regardless of their work department or age. We excluded nurses who had already experienced setting up a ventilator or who had trouble wearing or using a head-mounted display (HMD). As this was a preliminary study, we were unable to determine the sample size. However, we referenced to past research on the step-by-step procedures with AR [19,24]. We set a target of 30 participants for recruitment.

    Study Design

    Using a lottery method, we randomly assigned the participants to 2 groups. One group (the manual group) used a printed manual to set up a ventilator and the other group (the AR group) used AR-based instructions through an HMD—HoloLens 2 (Microsoft Corporation). The participants in the AR group were provided with 15 minutes of learning and practice time with HoloLens 2. If they needed assistance, participants in the manual group made a phone call, and those in the AR group requested it remotely with HoloLens 2; subsequently, both groups were assisted by the same ICU nurse. In the AR group, the participants shared the same view as the nurse using Microsoft Dynamics 365 Remote Assist, which allowed the ICU nurse to guide the participants through voice commands and drawing marks on their view. Both groups were surveyed immediately after the task.

    Instructions for Ventilator Setup

    The instructions to set up the Servo-i mechanical ventilator (Maquet) were developed by researchers, including emergency physicians, pediatricians, and ICU nurses. The instructions detailed the entire process, from plugging in a socket to turning on the power, by performing initial ventilator mode setting with 35 steps. The AR instructions were developed as a step-by-step guide with the same text and images as in the printed manual, using the Microsoft Dynamics 365 Guide. The AR instructions were delivered using Microsoft HoloLens 2. The device allows users to go back and forth through the entire procedure by gazing at the screen when required. Some steps had a guide with a hologram of the 3D objects to indicate the location of the steps and direct the action of the connecting parts (Figure 1).

    ‎
    Figure 1. Scenes from the study: (A) trainee following the instructions with the printed manual; (B) trainee following the augmented reality (AR) instruction with head mounted display (HMD); (C) hologram instruction shown to trainee via HMD; (D) AR direction with hologram (3D object) overlaid on the mechanical ventilator.
    View this figure

    Evaluation of Outcomes

    The primary outcome is the overall score of the procedure, which is a 100-point scale converted from the original score. Participants scored 1 point for each step if they successfully finished the step within 5 minutes and obtained a maximum score of 35. The secondary outcome was the required level of assistance (ie, the number of steps and the number of participants who required assistance, assistance frequency, and assistance time). We also evaluated the user’s experience with short questions on 3 themes: confidence, suitability, and whether they intended to recommend AR system to others [25,26]. All the participants were asked to respond to general questions on a 5-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). The usability of the HMD in AR-based training was determined using previously validated system usability scale (SUS) standards [27,28].

    Statistical Analysis

    All continuous variables are described as mean (SD) and median (IQR), and categorical variables are described as numbers and percentages. For continuous variables, we used the Wilcoxon rank-sum test; for categorical values, we used the chi-square test or Fisher exact test. A proportion test was performed to compare the proportions between the two groups. For all statistical analyses, P<.05 was considered statistically significant. The statistical analysis was performed using the R software (version 4.1.2; R Foundation for Statistical Computing).


    Characteristics of the Participants

    A total of 31 nurses with no prior experience in setting up ventilators were enrolled, with the exception of 1 participant who had an HMD-related technical issue. The remaining 30 nurses completed the entire procedure and the surveys afterward. Each participant was randomly assigned to either the manual or the AR group. The participants ranged in age from 24-53 years and came from a variety of departments: 4 participants from outpatient nursing, 9 participants from inpatient nursing, 11 participants from specialized nursing, and 6 participants from administrative and educational departments. There were no significant differences in sex, age, work experience, prior observation experience, or department of work between the two groups (Table 1).

    Table 1. Demographics of the study participants (N=30).
    CharacteristicsManual group (n=15)Augmented reality group (n=15)P value
    Gender, n (%).48

    		Female15 (100)13 (86.7)

    		Male0 (0)2 (13.3)
    Age (years), n (%)>.99

    		20-309 (60)8 (53.3)

    		30-403 (20)4 (26.7)

    		≥403 (20)3 (20)
    Age, median (Q1, Q3)28 (26, 34.5)28 (26.5, 35).84
    Work (years), n (%).89

    		<57 (46.7)7 (46.7)

    		5-105 (33.3)3 (20)

    		>103 (20)5 (33.3)
    Previous experience of observing, n (%)0 (0)1 (6.7)>.99
    Department of work, n (%).84

    		Outpatient nursing3 (20)1 (6.7)

    		Inpatient nursing4 (26.7)5 (33.3)

    		Specialized nursinga5 (33.3)6 (40)

    		Administration and educational part3 (20)3 (20)

    aIn operating room, emergency department, radiology department, intensive care unit, and imaging center.

    Overall Performance

    All 30 participants completed the entire procedure, with or without remote assistance. One participant in the AR group completed the procedure successfully without any assistance. Table 2 summarizes the overall results for the manual and AR groups. There was no significant difference in the overall score between the two groups, regardless of assistance. When only the steps without assistance were considered successfully passed, the median score was 88.57 (IQR 82.86-91.43) in the manual group and 91.43 (IQR 88.57-97.14; P=.10) in the AR group. However, if assisted steps were also considered as successful, the median score was 94.29 (IQR 91.43-94.29) in the manual group and 94.29 (IQR 92.86-97.14; P=.20) in the AR group.

    The duration of the procedure between the two groups was not statistically significant. Without assistance, the median procedure time was 22.95 (IQR 19.37-24.69) minutes in the manual group and 23.95 (IQR 20.83-26.95; P=.60) minutes in the AR group. With assistance, the median procedure time was 25.32 (IQR 22.41-29.02) minutes in the manual group and 24.18 (IQR 22.37-28.41; P=.97) minutes in the AR group.

    Multimedia Appendix 2 presents findings in terms of the step characteristics. We discovered that when following the directions of the ventilator or connecting and disconnecting materials such as a tube, circuit, or line, the manual group had a greater tendency to fail the steps or require assistance compared to the AR group. However, we did not identify any significant difference between the groups.

    Table 2. Overall outcomes of augmented reality (AR)–based instructions (N=30).
    CharacteristicsManual group (n=15)AR group (n=15)P value
    Score, median (IQR)

    		without assistance88.57 (82.86-91.43)91.43 (88.57-97.14).10

    		with assistance94.29 (91.43-94.29)94.29 (92.86-97.14).20
    Procedure time (min), median (IQR)

    		without assistance22.95 (19.37-24.69)23.95 (20.83-26.95).60

    		with assistance25.32 (22.41-29.02)24.18 (22.37-28.41).97
    Assistance

    		Steps that needed assistance, n3313.004

    		Steps that needed assistance per participant, median (IQR)2 (1-2)0 (0-1.5).03

    		Participants who requested assistance, n (%)14 (93)7 (48).02

    		Assistance time (min), median (IQR)1.53 (0.78-2.98)0 (0-3.02).12

    Need of Assistance

    The manual group required considerably more assistance than the AR group. The median number of steps that required assistance per participant was greater in the manual group compared to the AR group (median 2, IQR 1-2 vs median 0, IQR 0-1.5; P=.03; Table 2).

    The manual group had a greater proportion of participants who requested assistance compared to the AR group (14/15, 93.3% vs 7/15, 47.7%; P=.02; Figure 2A). Additionally, in the manual group, 33 requests for assistance were recorded, whereas only 13 requests were made in the AR group (P=.004; Figure 2B). There were no statistically significant differences in the time spent on assisting; the median was 1.53 (IQR 0.78-2.98) minutes for the manual group and 0 (IQR 0-3.02) minutes for the AR group (P=.12).

    ‎
    Figure 2. The effectiveness of augmented reality (AR)–based instruction for ventilator set up; (A) independent learning—the number of participants who requested assistance; (B) independent learning—number of stages that required assistance; (C) user experience.
    View this figure

    Survey Outcomes

    All the participants answered 3 general questions. Notably, only the AR group answered the SUS questions. Figure 2C shows the responses to the general questions on a 5-point Likert scale (from 1=strongly disagree to 5=strongly agree). The AR-based instructions received higher ratings for confidence (median 3, IQR 2.50-4.00 vs median 2, IQR 2.00-3.00; P=.01), suitability of method (median 4, IQR 4.00-5.00 vs median 3, IQR 3.00-3.50; P=.01), and whether they intended to recommend AR system to others (median 4, IQR 3.00-5.00 vs median 3, IQR 2.00-3.00; P=.002). The median of SUS score was 55 (IQR 47.5-67.5). Table 3 shows the details of each statement. Of all the statements, “well-integrated AR systems” received the best evaluation from the users, with the highest mean score and the lowest SD. Other statements such as “the simplicity of the system” (mean 3.7, SD 1.2), “ease of use” (mean 3.7, SD 1.1), and “technical assistance requirement” (mean 3.7, SD 1.1) obtained relatively high ratings.

    Table 3. Evaluation of augmented reality (AR)–based instructions using the standardized system usability scale (SUS).
    SUS questionsAR group (n=15), mean (SD)AR group (n=15), median (IQR)
    I think that I would like to use this system frequently.3.5 (1.0)4 (3.0-4.0)
    I found the system unnecessarily complex.3.7 (1.2) 4 (3.0-5.0)
    I thought the system was easy to use.3.7 (1.1)4 (3.0-4.5)
    I think that I would need the support of a technical person to be able to use this system.3.7 (1.1)4 (3.0-4.0)
    I found that the various functions in this system were well integrated.4.0 (0.6)4 (3.0-5.0)
    I thought there was too much inconsistency in this system.1.9 (0.9)2 (1.0-2.0)
    I would imagine that most people would learn to use this system very quickly.3.8 (1.0)4 (3.0-5.0)
    I found the system cumbersome to use.2.4 (1.1)2 (2.0-3.5)
    I felt very confident using the system.3.3 (1.0)3 (2.5-4.0)
    I needed to learn a lot of things before I could get going with this system.3.3 (1.0)4 (2.5-4.0)

    Principal Results

    In this study, the participants had no prior experience with the ventilator setup or the HMD; additionally, all participants completed the entire procedure, from preparing materials to setting up the initial ventilator mode, prior to connecting to the patient. Moreover, the AR group was able to complete all the procedures following AR-based instructions in the planned design of the study, including a brief HMD practice and self-learning session. They required significantly less assistance compared to the manual group, and all assistance could be provided properly through a remote AR system. There were no technical issues or dropouts in either group.

    Generally, hands-on training is required and beneficial when training trainees on complex procedures [29-31]. In addition, in a step-by-step procedure, failures in one step affect the subsequent steps, preventing the trainee from completing the process and requiring real-time guidance. However, experts in critical care cannot stay all day, and novices are required to use a systemic remote assistance when they face difficulties [32,33]. In addition, trainees do not have sufficient time to repeat the procedures, and when they practice alone, it is difficult to assist them in a proper manner.

    AR-based training benefits both sides, as discussed above. The instructors are not required to be on-site, as the remote assistance software enables them to monitor and guide the trainee’s view as well as draw and transfer data, voice, and so on. Additionally, without prior technical knowledge or experience with computer programming, developing instructions for a procedure using HoloLens 2 software was possible in 2-3 hours. From the trainee’s perspective, they can learn frequently without visiting an education center or engaging in on-site instructions. Additionally, the desire to develop a contactless education platform has increased to protect health care workers and save on personal protection equipment [34].

    The difference between the median number of assistance requests was not significant between the two groups; however, from a practical view, the difference in the workflow interruption between the two groups was more significant. A request for assistance resulted in procedure interruption by phone calls in the manual group and was difficult to support because they were held on to the phone. However, with an AR-based system, they could request help by speaking and connecting to the supervisor and be supported while continuing procedures.

    Comparison With Prior Work

    Our results provide new evidence for the feasibility of AR-based independent learning in complex advanced procedures with 35 steps lasting more than 20 minutes. In earlier research, studies also discovered that participants were more satisfied with AR-based instructions than with conventional instructions [35]. They were more confident and felt that they had received adequate training for the procedure, and they intended to recommend AR systems to others. It is important for continuous learning to attain competency [36]. We expect that strong confidence and user satisfaction would result in greater willingness and self-practice for learning to set up a ventilator independently.

    Limitations

    As a pilot study, there was no specific guideline regarding how to deal with technology issues, such as time for battery charging, overheating of the device without break time, and network instability. These issues were observed in a few cases but were solved without affecting the study; however, these issues will be addressed and planned in a larger-scale study.

    Additionally, in the step-by-step procedures, the content of the errors is important; however, this was not addressed in this study. To extend AR-based training in other step-by-step advanced procedures and explore additional outcomes, considering the characteristics of steps and designing a training platform for suitable technology integration would be required.

    Conclusions

    AR-based instructions to set up mechanical ventilator were feasible for novices with no prior experience with MV and AR. In addition, participants using AR required less assistance, resulting in higher confidence after training.

    Acknowledgments

    We thank Minji Park and Kyung mi Min from the Research Institute for Future Medicine, Samsung Medical Center, Seoul, Korea for their assistance in conducting the study and for advising on the development of AR-based instructions.

    Authors' Contributions

    MHS contributed to the study’s ideas and design and supervised the overall study progress. SH contributed to the enrollment of participants, prepared and conducted the study, analyzed the results, and wrote the draft. MSH and MP performed data analysis and interpreted the data. MK developed a short session of practice with AR systems and the survey questionnaire. WCC reviewed the study and the survey questionnaire. All the authors read and approved the final draft of the study.

    Conflicts of Interest

    None declared.

    Editorial notice: This randomized study was only retrospectively registered, as the authors did not believe registration to be necessary. The editor granted an exception from ICMJE rules mandating prospective registration of randomized trials, because the risk of bias appears low. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness, as retrospective registration does not prevent authors from changing their outcome measures retrospectively.

    ‎
    Multimedia Appendix 1

    CONSORT-eHealth checklist (V 1.6.1).

    PDF File (Adobe PDF File), 95 KB
    ‎
    Multimedia Appendix 2

    Overall results of the study in a step-by-step manner.

    DOCX File , 21 KB
    1. Tobin MJ. Mechanical ventilation. N Engl J Med 1994 Apr 14;330(15):1056-1061. [CrossRef]
    2. Wells CR, Fitzpatrick MC, Sah P, Shoukat A, Pandey A, El-Sayed AM, et al. Projecting the demand for ventilators at the peak of the COVID-19 outbreak in the USA. Lancet Infect Dis 2020 Oct;20(10):1123-1125. [CrossRef]
    3. Kwon AH, Slocum AH, Varelmann D, Nabzdyk CGS, MIT E-Vent Team. Rapidly scalable mechanical ventilator for the COVID-19 pandemic. Intensive Care Med 2020 Aug 25;46(8):1642-1644. [CrossRef] [Medline]
    4. Andellini M, De Santis S, Nocchi F, Bassanelli E, Pecchia L, Ritrovato M. Clinical needs and technical requirements for ventilators for COVID-19 treatment critical patients: an evidence-based comparison for adult and pediatric age. Health Technol 2020 Jul 30;10(6):1403-1411. [CrossRef]
    5. Acute Respiratory Distress Syndrome Network, Brower RG, Matthay MA, Morris A, Schoenfeld D, Thompson BT, et al. Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med 2000 Dec 04;342(18):1301-1308. [CrossRef] [Medline]
    6. Adiyeke E, Koc BI, Coskun N, Bakan N, Sen E, Turan A, et al. The impact of a multidisciplinary education program for intensive care unit staff regarding ventilator care bundle on the frequency of ventilator-associated events. Dimens Crit Care Nurs 2021;40(4):210-216. [CrossRef]
    7. Mireles-Cabodevila E, Dugar S. On the need for standard definitions and education to optimize patient-ventilator interactions. Respir Care 2017 Feb 20;62(2):248-249 [FREE Full text] [CrossRef] [Medline]
    8. Mogyoródi B, Dunai E, Gál J, Iványi Z. Ventilator-associated pneumonia and the importance of education of ICU nurses on prevention - preliminary results. Interv Med Appl Sci 2016 Dec;8(4):147-151 [FREE Full text] [CrossRef] [Medline]
    9. Guilhermino MC, Inder KJ, Sundin D, Kuzmiuk L. Education of ICU nurses regarding invasive mechanical ventilation: findings from a cross-sectional survey. Aust Crit Care 2014 Aug;27(3):126-132. [CrossRef] [Medline]
    10. Guilhermino MC, Inder KJ, Sundin D. Education on invasive mechanical ventilation involving intensive care nurses: a systematic review. Nurs Crit Care 2018 Sep 26;23(5):245-255. [CrossRef] [Medline]
    11. Pedigo R. Ventilator management of adult patients in the emergency department. Emerg Med Pract 2020 Jul;22(7):1-20. [Medline]
    12. Haisley KR, Dunst CM, Kizziar P, Swanström LL. Impact of focused hands-on training course on practice adoption of advanced endoscopic techniques and per-oral endoscopic myotomy. J Laparoendosc Adv Surg Tech A 2022 Mar 01;32(3):251-255. [CrossRef] [Medline]
    13. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 2016 Dec;30(10):4174-4183 [FREE Full text] [CrossRef] [Medline]
    14. Barteit S, Lanfermann L, Bärnighausen T, Neuhann F, Beiersmann C. Augmented, mixed, and virtual reality-based head-mounted devices for medical education: systematic review. JMIR Serious Games 2021 Jul 08;9(3):e29080 [FREE Full text] [CrossRef] [Medline]
    15. Munzer BW, Khan MM, Shipman B, Mahajan P. Augmented reality in emergency medicine: a scoping review. J Med Internet Res 2019 Apr 17;21(4):e12368 [FREE Full text] [CrossRef] [Medline]
    16. Williams MA, McVeigh J, Handa AI, Lee R. Augmented reality in surgical training: a systematic review. Postgrad Med J 2020 Sep 30;96(1139):537-542. [CrossRef] [Medline]
    17. Iqbal H, Tatti F, Rodriguez Y Baena F. Augmented reality in robotic assisted orthopaedic surgery: a pilot study. J Biomed Inform 2021 Aug;120:103841. [CrossRef] [Medline]
    18. Cofano F, Di Perna G, Bozzaro M, Longo A, Marengo N, Zenga F, et al. Augmented reality in medical practice: from spine surgery to remote assistance. Front Surg 2021 Mar 30;8:657901 [FREE Full text] [CrossRef] [Medline]
    19. Wolf J, Wolfer V, Halbe M, Maisano F, Lohmeyer Q, Meboldt M. Comparing the effectiveness of augmented reality-based and conventional instructions during single ECMO cannulation training. Int J Comput Assist Radiol Surg 2021 Jul 23;16(7):1171-1180 [FREE Full text] [CrossRef] [Medline]
    20. Ingrassia PL, Mormando G, Giudici E, Strada F, Carfagna F, Lamberti F, et al. Augmented reality learning environment for basic life support and defibrillation training: usability study. J Med Internet Res 2020 May 12;22(5):e14910 [FREE Full text] [CrossRef] [Medline]
    21. Azimi E, Winkler A, Tucker E, Qian L, Doswell J, Navab N, et al. Can mixed-reality improve the training of medical procedures? Annu Int Conf IEEE Eng Med Biol Soc 2018 Jul;2018:4065-4068. [CrossRef] [Medline]
    22. Leary M, McGovern SK, Balian S, Abella BS, Blewer AL. A pilot study of CPR quality comparing an augmented reality application vs a standard audio-visual feedback manikin. Front Digit Health 2020 Feb 28;2:1 [FREE Full text] [CrossRef] [Medline]
    23. Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc 2017 Feb;12(1):57-62 [FREE Full text] [CrossRef] [Medline]
    24. Azimi E, Winkler A, Tucker E, Qian L, Sharma M, Doswell J, et al. Evaluation of optical see-through head-mounted displays in training for critical care and trauma. 2018 Mar 22 Presented at: IEEE Conference on Virtual Reality and 3D User Interfaces; 18-22 March; Reutlingen, Germany. [CrossRef]
    25. Schrepp M, Hinderks A, Thomaschewski J. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). IJIMAI 2017;4(6):103. [CrossRef]
    26. Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H. Augmented reality learning experiences: survey of prototype design and evaluation. IEEE Trans Learning Technol 2014 Jan;7(1):38-56. [CrossRef]
    27. Helin K, Kuula T, Vizzi C, Karjalainen J, Vovk A. User experience of augmented reality system for astronaut's manual work support. Front Robot AI 2018 Sep 12;5:106 [FREE Full text] [CrossRef] [Medline]
    28. Wang C, Tsai N, Lu J, Wang MJ. Usability evaluation of an instructional application based on Google Glass for mobile phone disassembly tasks. Appl Ergon 2019 May;77:58-69. [CrossRef] [Medline]
    29. Sedlack RE, Petersen BT, Kolars JC. The impact of a hands-on ERCP workshop on clinical practice. Gastrointest Endosc 2005 Jan;61(1):67-71. [CrossRef]
    30. DiMusto PD, Chen H, Kent KC. Hands-on training in the operating room as a method of continuing education for surgeons in practice. JAMA Surg 2016 Sep 01;151(9):869-870. [CrossRef] [Medline]
    31. Stovel RG, Gabarin N, Cavalcanti RB, Abrams H. Curricular needs for training telemedicine physicians: a scoping review. Med Teach 2020 Nov;42(11):1234-1242. [CrossRef] [Medline]
    32. Khurrum M, Asmar S, Joseph B. Telemedicine in the ICU: innovation in the critical care process. J Intensive Care Med 2021 Dec;36(12):1377-1384. [CrossRef] [Medline]
    33. du Toit M, Malau-Aduli B, Vangaveti V, Sabesan S, Ray RA. Use of telehealth in the management of non-critical emergencies in rural or remote emergency departments: a systematic review. J Telemed Telecare 2017 Oct 05;25(1):3-16. [CrossRef]
    34. Greenberg JA, Schwarz E, Paige J, Dort J, Bachman S. At-home hands-on surgical training during COVID19: proof of concept using a virtual telementoring platform. Surg Endosc 2021 May 06;35(5):1963-1969 [FREE Full text] [CrossRef] [Medline]
    35. Siff L, Mehta N. An interactive holographic curriculum for urogynecologic surgery. Obstet Gynecol 2018 Oct;132 Suppl 1:27S-32S. [CrossRef] [Medline]
    36. Liang J, Wu S, Tsai C. Nurses' internet self-efficacy and attitudes toward web-based continuing learning. Nurse Educ Today 2011 Nov 31;31(8):768-773. [CrossRef] [Medline]

    ‎
    AR: augmented reality
    HMD: head-mounted display
    ICU: intensive care unit
    MV: mechanical ventilation
    SUS: system usability scale

    Edited by N Zary; submitted 01.04.22; peer-reviewed by A Alkhaqani; comments to author 25.04.22; revised version received 01.05.22; accepted 12.06.22; published 22.07.22

    Copyright

    ©Sejin Heo, Suhyeon Moon, Minha Kim, Minsu Park, Won Chul Cha, Meong Hi Son. Originally published in JMIR Serious Games (https://games.jmir.org), 22.07.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.