This paper is in the following e-collection/theme issue:

Serious Games for Education (149) e-Learning and Medical Education (691) Virtual Reality and Augmented Reality in Medical Education (76) Augmented Reality (incl. Google Glass) Applications (90) Games for Medical Education and Training (95) Nursing Education and Training (129) Serious Games for Health and Medicine (683) mHealth in Medical Education and Training (77)

Published on in Vol 12 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/54188, first published .
Adoption of Augmented Reality in Educational Programs for Nurses in Intensive Care Units of Tertiary Academic Hospitals: Mixed Methods Study

Adoption of Augmented Reality in Educational Programs for Nurses in Intensive Care Units of Tertiary Academic Hospitals: Mixed Methods Study

Adoption of Augmented Reality in Educational Programs for Nurses in Intensive Care Units of Tertiary Academic Hospitals: Mixed Methods Study

Authors of this article:

Suyoung Yoo1 Author Orcid Image ;   Sejin Heo2 Author Orcid Image ;   Soojin Song3 Author Orcid Image ;   Aeyoung Park3 Author Orcid Image ;   Hyunchung Cho1 Author Orcid Image ;   Yuna Kim3 Author Orcid Image ;   Won Chul Cha1, 2 Author Orcid Image ;   Kyeongsug Kim4 Author Orcid Image ;   Meong Hi Son1, 2 Author Orcid Image
Article Authors Cited by (1) Tweetations (4) Metrics

    Original Paper

    1Department of Digital Health, Samsung Advanced Institute for Health Science & Technology, Sungkyunkwan University, Seoul, Republic of Korea

    2Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Gangnam-gu, Seoul, Republic of Korea

    3Department of Nursing Education, Samsung Medical Center, Seoul, Republic of Korea

    4Graduate School of Clinical Nursing Science, Sungkyunkwan University, Seoul, Republic of Korea

    *these authors contributed equally

    Corresponding Author:

    Meong Hi Son, MD

    Department of Emergency Medicine

    Samsung Medical Center

    Sungkyunkwan University School of Medicine

    115 Irwon-ro

    Gangnam-gu, Seoul, 06355

    Republic of Korea

    Phone: 82 2 3410 2053

    Fax:82 2 3414 2832

    Email: meonghison@gmail.com


    Background: In the wake of challenges brought by the COVID-19 pandemic to conventional medical education, the demand for innovative teaching methods has surged. Nurse training, with its focus on hands-on practice and self-directed learning, encountered significant hurdles with conventional approaches. Augmented reality (AR) offers a potential solution to addressing this issue.

    Objective: The aim of this study was to develop, introduce, and evaluate an AR-based educational program designed for nurses, focusing on its potential to facilitate hands-on practice and self-directed learning.

    Methods: An AR-based educational program for nursing was developed anchored by the Kern six-step framework. First, we identified challenges in conventional teaching methods through interviews and literature reviews. Interviews highlighted the need for hands-on practice and on-site self-directed learning with feedback from a remote site. The training goals of the platform were established by expert trainers and researchers, focusing on the utilization of a ventilator and extracorporeal membrane oxygenation system. Intensive care nurses were enrolled to evaluate AR education. We then assessed usability and acceptability of the AR training using the System Usability Scale and Technology Acceptance Model with intensive care nurses who agreed to test the new platform. Additionally, selected participants provided deeper insights through semistructured interviews.

    Results: This study highlights feasibility and key considerations for implementing an AR-based educational program for intensive care unit nurses, focusing on training objectives of the platform. Implemented over 2 months using Microsoft Dynamics 365 Guides and HoloLens 2, 28 participants were trained. Feedback gathered through interviews with the trainers and trainees indicated a positive reception. In particular, the trainees mentioned finding AR particularly useful for hands-on learning, appreciating its realism and the ability for repetitive practice. However, some challenges such as difficulty in adapting to the new technology were expressed. Overall, AR exhibits potential as a supplementary tool in nurse education.

    Conclusions: To our knowledge, this is the first study to substitute conventional methods with AR in this specific area of critical care nursing. These results indicate the multiple principal factors to take into consideration when adopting AR education in hospitals. AR is effective in promoting self-directed learning and hands-on practice, with participants displaying active engagement and enhanced skill acquisition.

    Trial Registration: ClinicalTrials.gov NCT05629663; https://clinicaltrials.gov/study/NCT05629663.

    JMIR Serious Games 2024;12:e54188

    doi:10.2196/54188

    Keywords

    augmented reality; AR; clinical skills education; nurse education; technology-based education; education; nurse; nursing; allied health; technology-enhanced learning; interview; training; usability; acceptability; educational; teaching; ICU; intensive care unit; self-guided; self-directed; hands-on; adoption; TAM; Technology Acceptance Model; skill; acquisition 


    In recent years, conventional education, especially in the medical field, has been challenged by the introduction of new technologies [1]. The COVID-19 pandemic further highlighted the limitations of conventional teaching methods [2]. Nurse training, with its emphasis on hands-on practice and self-directed learning, was particularly affected by the pandemic, making it evident that conventional training methods could not sustain the demands of the situation [3,4]. Given these constraints, the search for alternative, technology-driven educational methods intensified, aiming to address both physical resource and time challenges without compromising education quality [5,6]. In this context, an immersive learning environment, based on a computer-generated environment enabling real-time user interactions [7], has emerged as a promising solution.

    Such an immersive platform merges augmented reality (AR) and virtual reality (VR), offering a dynamic 3D space for learners. This integration not only enhances the learning experience by providing a rich, immersive environment [8,9] but also reshapes the boundaries between reality and the virtual realm, paving the way for innovative learning avenues [10,11]. Exploring AR’s potential reveals that its uses surpass merely aiding in remote education. AR also introduces real-time feedback mechanisms, empowering trainees to obtain immediate insights about their actions and performance through virtual aids [12,13]. This immediacy in feedback is invaluable, as it allows errors to be addressed promptly, fostering continuous improvement in learning [14,15].

    In the field of critical care, there has been an exploration of the use of AR and VR in educational applications [16]. VR-enhanced training for tracheostomy care in the intensive care unit (ICU) setting has demonstrated the potential of education in an immersive learning environment [17,18]. Studies have been conducted for training mechanical ventilator settings and central line insertion, showing improvements in self-efficacy, increased familiarity, confidence, and reduced anxiety compared to conventional methods [13,17,19]. However, while the advantages are evident, existing research into AR and VR remains limited. These studies are usually one-time or short-term investigations, mainly focusing on the effectiveness of the immersive learning environment [20]. Moreover, integrating these technologies into a nursing curriculum represents an area yet to be fully explored [21].

    In this study, we aimed to derive key considerations for each phase of implementation based on our experience of introducing an AR nursing program within an ICU in a tertiary hospital setting.


    Experimental Design

    Overview

    Our methodology was refined based on the Kern six-step approach to transition nursing education into an immersive AR-based format [22], focusing on the following key stages: (1) problem identification, (2) needs assessment, (3) setting goals and objectives, (4) choosing educational strategies, (5) implementation, and (6) evaluation [23] (Figure 1).

    Our initial steps involved conducting interviews with trainer nurses to discern existing issues and identify a procedure amenable to transition into an AR format [24]. We then developed and implemented an AR-based educational program encapsulating two distinct procedures. We surveyed and interviewed trainees, focusing on technology acceptance and usability.

    ‎
    Figure 1. Overall process of the adoption of augmented reality into nursing education based on the Kern 6-step framework [22]. AR: augmented reality; ECMO: extracorporeal mechanical oxygenation.
    Stage 1: Problem Identification

    In this foundational stage, we endeavored to identify core problems through a general needs assessment, employing two primary approaches: engaging in interviews with training nurses and reviewing the extant literature [24-27].

    The interviews, conducted with nurses from the nursing education department, were based on a semistructured format and were held online or offline depending on participant preference. Voice recordings ensured the precise capture of data shared during these interactions.

    In addition to the interviews, we explored previous studies with the aim of harvesting insights and identifying common issues found within the existing research landscape, thereby anchoring our findings in a robust context of existing knowledge [28].

    Stage 2: Needs Assessment

    Upon conducting the interviews in stage 1, we recognized the challenges endemic to conventional educational methods as identified by educators. We investigated possible solutions to these challenges, and found the need for feedback through remote supervision, especially regarding certain devices. We further identified key elements that should be considered in the development of the educational program. This stage also served to validate our problem identification process.

    Stage 3: Goals And Objectives

    We defined our goals through collaboration between trainers and researchers, focusing on improving access and proficiency with complex medical equipment.

    Stage 4: Educational Strategies
    Step 1: Selecting Appropriate Technology

    With the imperative for hands-on practice in nursing education, AR was chosen to enable nurses to interact with virtual medical devices within a realistic clinical setting [29]. AR’s ability to superimpose digital models onto the physical world allows for a highly interactive and immersive learning experience without the traditional constraints of location, time, or physical resources [30]. This aligns with our goal to empower self-directed learning, permitting nurses to engage in practical education at their own pace and convenience [31].

    Step 2: Developing Educational Material

    Navigating through the lens of self-directed learning and hands-on practice, we considered AR options that could facilitate tangible interaction with 3D objects. The development of materials required a detailed comprehension of the unique needs of nurses and the incorporation of AR content to support self-directed learning.

    Step 3: Preparation of the Educational Environment

    To facilitate a high-fidelity learning experience, our AR-based educational program was set within the hospital’s simulation laboratory. A designated area within the lab was prepared, encompassing a minimum of 3×3 meters to provide trainees with sufficient room to maneuver and interact with the virtual elements without spatial constraints. To ensure uninterrupted delivery of our AR-based educational program, we utilized five HoloLens 2 devices. This approach was adopted to mitigate against battery and overheating issues that could disrupt the learning process. This setup was optimized to allow multiple nurses to receive training at the same time, promoting efficient learning throughout while maintaining an individualized, hands-on experience.

    Stage 5: Implementation

    We recruited the participants through an advertisement posted on the hospital’s internal internet network. Our goal was to enroll a minimum of 10 participants for each session to ensure a dynamic and interactive learning environment while still allowing for personalized instruction. We designed the sessions to accommodate up to five nurses at a time, which was determined as the optimal number for both effective learning and space utilization within our AR setup. This small-group approach not only facilitated focused attention from the instructors but also ensured that each participant could engage deeply with the AR modules.

    To maintain a high standard of education and safety, we appointed two experienced supervisors for each training session. These supervisors were selected based on their expertise in intensive care procedures and their familiarity with AR technology. Their role was to provide immediate assistance and feedback, ensuring that any technical issues could be addressed without disrupting the learning process. They were also tasked with observing the sessions to gather informal feedback, contributing to the continuous improvement of the program.

    Stage 6: Evaluation

    This study was conducted at a large academic tertiary hospital in Seoul, Korea, which accommodates more than 3100 nurses and 2000 inpatient beds. The research provided education to intensive care nurses and included a postsession survey and semistructured interview.

    Ethical Considerations

    The Institutional Review Board of Samsung Medical Center approved the study design (SMC-2022-08-058 and SMC-2022-08-079), and all trainers and trainees provided written informed consent before participating in the study, ensuring ethical adherence throughout the research. To protect the participants’ privacy and confidentiality, all data collected during this study were anonymized or deidentified. Stringent data protection measures are in place, including the use of secure, encrypted data storage systems accessible only by authorized personnel. These precautions are designed to safeguard sensitive information and maintain the integrity of the data. Participants were compensated for their time and contribution. Each participant received 30,000 KRW (~US $22) upon completion of their involvement in the study. This compensation was intended to acknowledge their valuable time and effort and to offset any inconvenience associated with participation. The compensation structure was clearly communicated to all participants prior to their enrollment and was administered transparently to ensure fairness.

    Outcome Measures of the Survey

    Upon completion of the educational program, participants were asked to fill out a questionnaire evaluating their user experience. This evaluation was based on the theories of self-directed learning and hands-on practice, including questions on personal characteristics, job satisfaction, and appropriateness. To evaluate the AR program’s acceptability and usability, we employed the System Usability Scale (SUS) and the Technology Acceptance Model (TAM) [20,32,33].

    We chose the SUS for its proven reliability and efficiency across various technologies. The SUS is widely used across various domains, including software, websites, and medical devices, to assess overall user experience. Moreover, it has been validated in hospital environments and shown effectiveness with small sample sizes. The SUS consists of 10 simple questions presented in a 5-point Likert-scale format, assessing both positive and negative aspects of the system, with total scores ranging from 0 to 100 [34,35]. The TAM was selected for its emphasis on understanding user acceptance of information technology [16,36,37]. We adapted the TAM-based survey questions to fit the context of AR nurse education, informed by various relevant studies [24,26,27,38]. In contrast, the SUS was employed in its unmodified form. In addition, we conducted a correlation analysis between the TAM and SUS elements [37,39].

    Outcome Measures of Interviews

    Nurses who responded to the questionnaire were selectively screened for their willingness to participate in further interviews. These interviews were semistructured and guided by nursing education theories from previous studies. The format allowed flexibility, permitting up to two additional questions based on the responses of the interviewees.

    Statistical Analysis

    The statistical analysis was performed using R software (version 4.3.2). Continuous variables are expressed as either mean (SD) or median (IQR), depending on their distribution, while nominal variables are expressed as counts (n) and percentages (%). We performed a correlation analysis to examine the relationship between the TAM and SUS using survey data.


    Stage 1: Problem Identification

    Overview

    We obtained interview results from four nurses who are trainers and operators in the nursing education department. The insights garnered from the interviews are summarized below and detailed in Table 1.

    Table 1. Key considerations in augmented reality (AR) education development as expressed during trainer interviews.
    CategoryKey detailsCore implications
    Educational needs and challengesTraining requirements necessary for handling advanced medical equipment; aligning AR educational content to complement the features of specific medical devices; adapting training modules to meet the unique demands precipitated by the COVID-19 pandemic; implementing streamlined training processes for the rapid acclimatization of new nurses; addressing the limitations inherent in conventional training techniques; tackling the deficit of hands-on training equipment in nursing trainingImperative for AR solutions in bridging training disparities and responding to progressive requirements
    Program development and strategiesTailoring educational programs to align with the diverse experience levels of nursing professionals; standardizing the phases of training to ensure uniformity and consistency in educational outcomes; formulating well-structured and strategic plans for nursing training; ensuring efficient and effective distribution and management of training equipment and resourcesCriticality of a holistic design and meticulous implementation in AR training for optimal efficacy
    Challenges and future concernsAddressing trainees’ physical challenges, such as the necessity to wear glasses or masks, in the training environment; guaranteeing the safety and appropriateness of both the devices and venues utilized for training; modifying AR training methods to be inclusive and effective for older nursing personnel; integrating strategies within training programs to manage and reduce trainee fatigue effectivelyRecognition of and addressing present and prospective hurdles for continuous advancement in AR training
    Requirement of Education in Difficult-to-Use Devices

    Using difficult-to-use medical devices in health care can pose a significant challenge for medical staff due to the increased risk of errors, negatively impacting patient outcomes. Proper education on these devices is essential to ensure that medical staff can use them safely and effectively.

    Lack of Resources: Space, Instructor, Time, and Cost

    The lack of education resources in health care can be a significant challenge for health care organizations and medical staff [15]. Education is essential to ensure that health care professionals possess the necessary knowledge, skills, and abilities to provide safe and effective care to patients. However, the interviewees mentioned that many health care organizations face barriers in providing adequate education resources to their staff, which can negatively impact patient outcomes.

    Another challenge is the lack of adequate education time. Health care professionals are often required to work long hours, and finding time to attend education sessions and complete the necessary course work can be difficult. This lack of time can make it difficult for health care organizations to provide education tailored to the specific needs of their staff.

    Stage 2: Needs Assessment

    Following the interviews, we identified crucial factors to consider when selecting educational topics. Educators highlighted the significant challenges of limited access to educational devices and instructors. Additionally, they emphasized the necessity for education in technically demanding skills. Trainers expressed a preference for educational topics that required hands-on practice. Their reasoning is grounded in the knowledge that complex devices are frequently used in treating patients with critical illnesses. The competence of nurses in operating these devices directly impacts patient outcomes. The responses from trainers and operators related to needs are summarized in Table 1.

    Stage 3: Goals and Objectives

    Selected Objectives

    Based on the interview results, extracorporeal membrane oxygenation (ECMO) machines and ventilators were selected as the objectives for training owing to their complexity and difficulty of use. Ventilators were selected as important yet challenging devices to master. The complexity of ventilators, compounded by the multitude of lines and connections involved, can pose challenges for nurses with limited experience.

    By contrast, an ECMO machine is a high-risk medical device that is essential for patients with COVID-19. When the alarm of an ECMO machine sounds, nurses must promptly find a solution. However, given its rarity, even experienced nurses may not have encountered this situation. Nevertheless, as this could pose a risk to the patient, appropriate education was deemed necessary, and therefore use of the ECMO machine was selected as the problem scenario for this evaluation.

    Selected Goals

    Our aim was to develop a sustainable AR-based educational program that could offer numerous benefits to trainers and trainees. These benefits include enhanced engagement and motivation, interactive and immersive learning experiences, and the facilitation of personalized learning. Key considerations for developing such a sustainable AR-based educational program encompass designing for scalability and accessibility, and incorporating user feedback to enhance the program in its maintenance and operation over time.

    Stage 4: Educational Strategies

    Step 1: Development of an AR-Based Educational Program

    The ECMO machine educational program comprises 45 slides divided into four parts, each detailing the operation of the machine, managing machine disruptions, responding to “low battery” alarms, and addressing the loss of flow signal (“SIG”) alarm. Each part includes approximately 8-10 steps, guiding trainees on how to manage each situation effectively.

    The ventilator program is composed of three parts encompassing a total of 46 slides. Each part involves 26 steps related to ventilator settings and preuse inspections, as well as seven steps for application, alarm configuration, and educational content evaluation. The education process involves checking supplies, power sources, wall oxygen and medical gas connections, and exhalation cassette connections; powering on the device; performing preuse checks; connecting test tubes; selecting the target and application method; turning on the humidifier; configuring the mode and parameters; connecting the patient to the system; monitoring after patient application; and setting alarms.

    Step 2: Adoption of Innovative Technology

    The AR-based education was performed with a Maquet Servo-I mechanical ventilator and the RotaFlow II System Permanent Life Support ECMO machine from Getinge. We attempted to incorporate a 3D guide for hands-on practice and used videos to enhance understanding. The AR-based educational program was developed using the Microsoft Dynamics 365 Guides program. The program’s content was delivered to users through a Microsoft HoloLens 2 device.

    Step 3: Operation Plan

    The previously designated simulation laboratory was successfully used during the AR-based educational program. The allocated space for the program proved sufficient, with each trainee having access to the minimum 3×3 meters space as planned [40]. This spatial arrangement allowed for unimpeded movement and interaction with the AR components, which was critical for the immersive learning experience.

    In practice, the ventilator and ECMO machine simulations were conducted without any spatial constraints, enabling a total of 28 trainees to complete the training per the session schedules. The effective use of space was evidenced by the trainees’ ability to perform the necessary tasks and their reported comfort level during the training sessions.

    Stage 5: Implementation

    The AR-based educational program platform was operational for a period of 2 months, with education sessions scheduled from 9 AM to 5 PM. This schedule allowed nurses to select their preferred date and time within this interval. To facilitate the program’s implementation, we used five HoloLens 2 devices, along with two laptops for supervisor screen connections and two large screens for the research environment. Throughout the research, a total of 22 trainees actively engaged in the education sessions. The trainees’ screens, as viewed through the HoloLens 2 devices, were immediately visible to the trainer, enabling real-time progress monitoring. Additionally, trainees were encouraged to request assistance if they encountered any difficulties during the session.

    Stage 6: Evaluation

    Participants

    Training sessions were conducted by two trainers and two operators for the 28 nurses in the ICU from January 1 to February 3, 2022. Twenty-four nurses participated in the survey, 11 of whom took part in an in-depth interview. They were trained either in ventilator or ECMO machine usage with an even distribution across both groups. The participants’ baseline characteristics are presented in Table 2. The median work experience was 3 (IQR 0-6.25) years with a mean of 3.75 (SD 3.90) years. There was a predominance of female participants (17/24, 71%). All participants belonged to the general nursing field with a slight majority working in the medical ICU compared to the surgical ICU (Table 2).

    Additionally, all participants (24/24, 100%) owned smartphones and the majority (23/24, 95.83%) possessed either a tablet PC or laptop. Prior to the instruction, 13 (54%) nurses had previous experience with a head-mounted display.

    Table 2. Demographic and clinical characteristics of the surveyed nurse trainees (N=24).
    CharacteristicsTrainees, n (%)
    Method trained on

    		ECMOa12 (50)

    		Ventilator12 (50)
    Sex

    		Male7 (29)

    		Female17 (71)
    Medicine specialty

    		Internal medicine15 (63)

    		Surgical department9 (38)
    Experience (years)

    		<15 (25)

    		1-23 (15)

    		3-42 (10)

    		5-62 (10)

    		≥73 (15)

    aECMO: extracorporeal membrane oxygenation.

    Comparison of SUS and TAM Scores

    In the usability test, the items “I think that I would like to use this system frequently” and “I don’t think the system is unnecessarily complex” received the highest rating of 4.38 out of a possible 5, while the lowest-rated item, “I thought there was too much inconsistency in this system,” received an average score of 1.83. The responses concerning technology acceptance were categorized into four areas according to the TAM: perceived usefulness (PU), perceived ease of use (PEU), perceived enjoyment (PE), and intention to use (IU). The survey included 15 questions scored on a 7-point scale. The item with the highest score was “It is fun to use,” scoring 6.71, while the lowest-rated item was “It is easy to use,” scoring 5.17. In further survey results, factors such as age, sex, department of work, and years of work did not impact satisfaction with the education or usability. All response results for the survey are provided in Multimedia Appendix 1.

    Correlation of Usability and Acceptance

    Our correlation analysis revealed varying degrees of association between SUS and TAM factors. For instance, there was a strong correlation between PU and IU and a moderate correlation between PE and PU. However, the correlation between PEU and IU was not significant (Table 3).

    Table 3. Correlation between the Technology Acceptance Model–based survey items and System Usability Scale (SUS).
    VariableUEa (SUS)PUbPEUcPEdIUe
    UE (SUS)

    		r10.20.340.150.22

    		P value—f.50.12.45.30
    PU

    		r0.210.20.690.76

    		P value.50—.50<.001<.001
    PEU

    		r0.340.201
    		0.31

    		P value.12.50—.34.33
    PE

    		r0.150.690.3210.72

    		P value.45<.001.34—<.001
    IU

    		r0.220.760.310.721

    		P value.30<.0010.33<.001—

    aUE: user experience.

    bPU: perceived usefulness.

    cPEU: perceived ease of use.

    dPE: perceived enjoyment.

    eIU: intention to use.

    fNot applicable.

    Insights From Participant Interviews

    Four participants completed interviews related to their experiences with the AR-based educational program. Key insights from these interviews have been collated and are summarized in Table 4. We present a curated selection of interview responses that most effectively capture the key insights. These selections were thoroughly chosen for their relevance and ability to represent the broader findings of the study.

    Overall, the evaluation of the AR-based education was positive, with participants indicating that AR could enhance their actual clinical performance. AR technology is particularly well-suited for individuals interested in self-directed or hands-on learning theories. Nurses were found to be open to education using innovative technology. When asked if they needed assistance with the curriculum, no participant responded negatively regarding the content. However, some participants did express a need for help in adapting to new devices and technologies.

    Table 4. Trainees’ feedback after augmented reality (AR) implementation in the training program.
    Category and subcategoryDetails
    Motivation

    		Intrinsic motivationInterest in the integration of AR into educational settings; desire among learners for practical experience with medical devices; expectations that AR technology will significantly improve learning efficacy

    		Reasons for participationInfluence of colleague recommendations; curiosity about AR teaching methods; specific needs related to own job

    		Existing issuesNoticing varying standards in educational quality; underscoring the need for improvement; necessity to establish standardized training procedures and protocols; need for training programs to be customized to individual learning styles and needs
    Learning preferences

    		Preferred learning methodBalancing traditional (54.5%) and self-directed (45.5%) learning approaches; valuing feedback and interaction in traditional learning; preference for flexibility and pace in self-directed learning

    		Face-to-face versus nonface-to-faceEquilibrium between face-to-face (54.5%) and remote learning (45.5%); diverse preferences shaped by feedback, comfort, and flexibility
    Feedback

    		Practical useMajority opinion holding that AR technology is beneficial for skill development; mixed opinions regarding the real-world applicability of AR in professional settings; varied levels of expectation regarding the use of AR devices in educational contexts

    		Training experienceRecognizing the benefits of AR in providing realistic scenarios, allowing for self-directed learning, and enabling repeated practice; challenges include unfamiliarity with AR, focus on operation over content, and limited interaction

    		Content and supportGeneral satisfaction with AR content amid comparisons to traditional methods; requirement for technical support and assistance in AR training

    		Comparative analysis and outlookAR’s superiority in learning pace, error identification, and training repetition over conventional methods; challenges in mastering AR operation and content depth; mixed perspectives on AR replacing traditional methods (viewed as supplementary); AR’s efficacy in specific scenarios; considered resource-intensive for broad implementation; potential for enhancing self-directed and iterative learning
    Future considerations

    		Target demographics for AR trainingTarget new nurses, individuals lacking device experience, and department transfers; however, limited relevance for experienced nurses

    		Benefits of self-learning with ARReduced pressure, time efficiency, review flexibility; utility in learning uncommon scenarios and repeatable sessions

    		Concerns with self-learningLimitations of AR training in actual clinical settings; lack of communal learning opportunities in AR environments; concerns over system errors and device quantity limitations

    Principal Results

    The principal findings of this study provide valuable insights into the strategic translation of conventional critical nursing education to AR-based education platforms in the use of difficult-to-use medical devices [41]. Through interviews conducted with trainers before program development, the study successfully identified the specific needs and requirements of trainers in critical care nursing education. The study employed AR-based educational technology to enhance self-directed learning and hands-on practice.

    In the educational strategies employed, the study leveraged the unique features of AR to facilitate self-directed learning. By offering interactive and self-controlled learning experiences, AR empowered trainees to take ownership of their learning process. The program incorporated instructional materials and modules that allowed learners to explore and acquire knowledge at their own pace, fostering a sense of autonomy and self-guided learning. The use of AR also enabled real-time feedback and assessment, allowing learners to track their progress and identify areas for improvement [29].

    Through the overlay of 3D objects and virtual models onto real-world settings, trainees engaged in simulated scenarios closely resembling authentic ICU environments. This hands-on component of the program enabled learners to apply their knowledge and skills in realistic contexts, promoting an understanding of the subject matter and the development of critical thinking and problem-solving abilities.

    This study highlights how AR technology significantly contributes to the success of self-directed learning and hands-on practice. The utilization of AR technology facilitates active engagement, learner-centeredness, and skill development, thereby enhancing the overall effectiveness of critical care nursing education. Moreover, we provide useful insights based on the perspectives of trainers and operators of the platform. The inherent nature of education often necessitates a lower number of educators compared to learners. The main strength of our study thus lies in presenting an infrequent perspective of educators, a viewpoint seldom encountered within the large-scale hospital setting.

    Comparison With Prior Work

    AR technology has been extensively explored in areas related to nursing education [42] such as surgical simulation [43], anatomy education [20], and patient safety education [44]. However, it is worth noting that this study represents the first investigation into the use of AR for replacing a conventional educational program in the use of difficult-to-use devices such as an ECMO machine and mechanical ventilator specifically within the critical care nursing field. By incorporating AR into these fields, this study pioneers the integration of innovative approaches in nursing education.

    Limitations

    This study, being characteristic of a pilot study to identify and apply new educational methods, has the limitation of a restricted number of participants. In further research, a larger sample size could be recruited to identify factors influencing user acceptability and to enhance usability, leveraging insights for more effective implementation.

    Implications

    This study highlights the implications of AR in future research and practice. The findings suggest the need for longitudinal studies to assess AR’s long-term impact on clinical performance and patient outcomes, and to explore its scalability and cost-effectiveness compared to traditional training. Practically, the results of our study indicate that institutions adopting AR should invest in technical support and training and consider integrating AR as a supplementary tool in curricula for a blended learning approach.

    Conclusions

    This study provides insights on the development, launch, and operation of an AR-based medical educational program. The study suggests that an AR-based educational program can be an alternative to compensate for insufficient resources for conventional critical care nursing education. Further research can be conducted to compare the effectiveness and feasibility of this program with other AR-based educational programs and traditional nursing educational programs.

    Acknowledgments

    This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI23C038700). We would like to express our sincere gratitude to all the individuals who contributed to the successful completion of this study. Primarily, we would like to thank the nursing education chiefs for their valuable insights and participation in the interviews. We are also grateful to the nurses who willingly participated in the study, sharing their experiences and providing valuable feedback. Additionally, we extend our appreciation to the operators who assisted in the implementation of the augmented reality–based educational program. The authors would like to thank Seungheon Choo for his invaluable assistance with the English proofreading of this manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    All response results for the System Usability Scale and survey based on the Technology Acceptance Model.

    DOCX File , 16 KB
    1. Lockee BB. Online education in the post-COVID era. Nat Electron. Jan 25, 2021;4(1):5-6. [CrossRef]
    2. Gause G, Mokgaola IO, Rakhudu MA. Technology usage for teaching and learning in nursing education: an integrative review. Curationis. Jun 15, 2022;45(1):e1-e9. [FREE Full text] [CrossRef] [Medline]
    3. Robbins DE. The effect of hands-on practice on the skills proficiency of nursing home nurses. Int J Stud Nurs. 2020;5(1):39. [CrossRef]
    4. Örs M. The self-directed learning readiness level of the undergraduate students of midwife and nurse in terms of sustainability in nursing and midwifery education. Sustainability. Oct 07, 2018;10(10):3574. [CrossRef]
    5. Johnsen HM, Briseid HS, Brodtkorb K, Slettebø Å, Fossum M. Nursing students' perceptions of combining hands-on simulation with simulated patients and a serious game in preparing for clinical placement in home healthcare: a qualitative study. Nurse Educ Today. Feb 2021;97:104675. [FREE Full text] [CrossRef] [Medline]
    6. Noble C, Billett S, Hilder J, Teodorczuk A, Ajjawi R. Enriching medical trainees' learning through practice: a video reflexive ethnography study protocol. BMJ Open. Aug 22, 2019;9(8):e031577. [FREE Full text] [CrossRef] [Medline]
    7. Moro C. Utilizing the metaverse in anatomy and physiology. Anat Sci Educ. Jan 05, 2023;16(4):574-581. [CrossRef] [Medline]
    8. Patel ST, Shah S, Sood RP, Siddiqui Z, McKay-Davies I. The implementation of virtual clinical skills teaching in improving procedural confidence in ENT trainees. Adv Med Educ Pract. 2021;12:965-969. [FREE Full text] [CrossRef] [Medline]
    9. Ordov K, Madiyarova A, Ermilov V, Tovma N, Murzagulova M. New trends in education as the aspect of digital technologies. Int J Mech Eng Technol. 2019;10(2):1319-1330. [FREE Full text]
    10. Dwivedi YK, Hughes L, Baabdullah AM, Ribeiro-Navarrete S, Giannakis M, Al-Debei MM, et al. Metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int J Inf Manage. Oct 2022;66:102542. [CrossRef]
    11. Zhang X, Chen Y, Hu L, Wang Y. The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol. Oct 11, 2022;13:1016300. [FREE Full text] [CrossRef] [Medline]
    12. Dinh A, Tseng E, Yin AL, Estrin D, Greenwald P, Fortenko A. Perceptions about augmented reality in remote medical care: interview study of emergency telemedicine providers. JMIR Form Res. Mar 28, 2023;7:e45211. [FREE Full text] [CrossRef] [Medline]
    13. Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc. Feb 2017;12(1):57-62. [FREE Full text] [CrossRef] [Medline]
    14. Lateef F. Simulation-based learning: just like the real thing. J Emerg Trauma Shock. Oct 2010;3(4):348-352. [FREE Full text] [CrossRef] [Medline]
    15. Real FJ, Whitehead M, Rosen BL, Meisman A, Crosby LE, Klein MD, et al. Assessment of competency-based behavioral health anticipatory guidance skills among pediatric residents: the role of virtual reality. J Technol Behav Sci. Mar 18, 2021;7(1):115-124. [CrossRef]
    16. Samadbeik M, Aslani N, Maleki M, Garavand A. Acceptance of mobile health in medical sciences students: applying technology acceptance model. Inform Med Unlocked. 2023;40:101290. [CrossRef]
    17. Chiang D, Huang C, Cheng S, Cheng J, Wu C, Huang S, et al. Immersive virtual reality (VR) training increases the self-efficacy of in-hospital healthcare providers and patient families regarding tracheostomy-related knowledge and care skills: a prospective pre-post study. Medicine. Jan 14, 2022;101(2):e28570. [FREE Full text] [CrossRef] [Medline]
    18. Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ. Apr 2020;147:103778. [CrossRef]
    19. Heo S, Moon S, Kim M, Park M, Cha WC, Son MH. An augmented reality-based guide for mechanical ventilator setup: prospective randomized pilot trial. JMIR Serious Games. Jul 22, 2022;10(3):e38433. [FREE Full text] [CrossRef] [Medline]
    20. Baashar Y, Alkawsi G, Ahmad WNW, Alhussian H, Alwadain A, Capretz LF, et al. Effectiveness of using augmented reality for training in the medical professions: meta-analysis. JMIR Serious Games. Jul 05, 2022;10(3):e32715. [FREE Full text] [CrossRef] [Medline]
    21. Aebersold M, Dunbar D. Virtual and augmented realities in nursing education: state of the science. Annu Rev Nurs Res. Dec 01, 2020;39(1):225-242. [CrossRef] [Medline]
    22. Heidari H, Beni ZHM, Deris F. Using Kern model to design, implement, and evaluate an infection control program for improving knowledge and performance among undergraduate nursing students: a mixed methods study. BMC Med Educ. Oct 25, 2023;23(1):795. [FREE Full text] [CrossRef] [Medline]
    23. Sweet LR, Palazzi DL. Application of Kern's six-step approach to curriculum development by global health residents. Educ Health. 2015;28(2):138-141. [CrossRef] [Medline]
    24. Uymaz P, Uymaz AO. Assessing acceptance of augmented reality in nursing education. PLoS One. Feb 17, 2022;17(2):e0263937. [FREE Full text] [CrossRef] [Medline]
    25. Elo S, Kääriäinen M, Kanste O, Pölkki T, Utriainen K, Kyngäs H. Qualitative content analysis. SAGE Open. Feb 11, 2014;4(1):215824401452263. [CrossRef]
    26. Dumpit DZ, Fernandez CJ. Analysis of the use of social media in Higher Education Institutions (HEIs) using the Technology Acceptance Model. Int J Educ Technol High Educ. Mar 1, 2017;14(1):5. [CrossRef]
    27. Papakostas C, Troussas C, Krouska A, Sgouropoulou C. Exploring users’ behavioral intention to adopt mobile augmented reality in education through an extended Technology Acceptance Model. Int J Hum Comput Interact. Apr 17, 2022;39(6):1294-1302. [CrossRef]
    28. Carroll JK, Yancey AK, Spring B, Figueroa-Moseley C, Mohr DC, Mustian KM, et al. What are successful recruitment and retention strategies for underserved populations? Examining physical activity interventions in primary care and community settings. Transl Behav Med. Jun 12, 2011;1(2):234-251. [FREE Full text] [CrossRef] [Medline]
    29. Chaghari M, Saffari M, Ebadi A, Ameryoun A. Empowering education: a new model for in-service training of nursing staff. J Adv Med Educ Prof. Jan 2017;5(1):26-32. [FREE Full text] [Medline]
    30. Al-Ansi AM, Jaboob M, Garad A, Al-Ansi A. Analyzing augmented reality (AR) and virtual reality (VR) recent development in education. Soc Sci Humanit Open. 2023;8(1):100532. [CrossRef]
    31. Lau ST, Siah CJR, Loh WL, Rusli KDB, Schmidt LT, Lim FP, et al. Enhancing professional competency in clinical procedures using head-mounted display virtual reality - a mixed method study. Med Educ Online. Dec 05, 2023;28(1):2232134. [FREE Full text] [CrossRef] [Medline]
    32. Davis FD. A technology acceptance model for empirically testing new end-user information systems: theory and results. Doctoral dissertation. MIT Sloan School of Management, MIT Libraries. 1986. URL: https://dspace.mit.edu/handle/1721.1/15192 [accessed 2024-04-17]
    33. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart. Sep 1989;13(3):319. [CrossRef]
    34. Brooke J. SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability Evaluation In Industry. Boca Raton, FL. CRC Press, Taylor & Francis Group; 1996:189-194.
    35. Cheah W, Mat Jusoh N, Aung MMT, Ab Ghani A, Mohd Amin Rebuan H. Mobile technology in medicine: development and validation of an adapted System Usability Scale (SUS) questionnaire and modified Technology Acceptance Model (TAM) to evaluate user experience and acceptability of a mobile application in MRI safety screening. Indian J Radiol Imaging. Jan 07, 2023;33(1):36-45. [FREE Full text] [CrossRef] [Medline]
    36. Wahdain EA, Ahmad MN. User acceptance of information technology: factors, theories and applications. J Inf Syst Res Innov. Jan 2014;6(1):17-25. [FREE Full text]
    37. Kim S, Garrison G. Investigating mobile wireless technology adoption: an extension of the technology acceptance model. Inf Syst Front. Apr 9, 2008;11(3):323-333. [CrossRef]
    38. Martínez-Caro E, Cegarra-Navarro J, Solano-Lorente M. An extension of the technology acceptance model in hospital-in-the-home units. In: Cruz-Cunha MM, Miranda IM, Gonçalves P, editors. Handbook of Research on ICTs and Management Systems for Improving Efficiency in Healthcare and Social Care. Hershey, PA. IGI Global; 2013:1191-1207.
    39. Chuenyindee T, Montenegro LD, Ong AKS, Prasetyo YT, Nadlifatin R, Ayuwati ID, et al. The perceived usability of the learning management system during the COVID-19 pandemic: integrating system usability scale, technology acceptance model, and task-technology fit. Work. 2022;73(1):41-58. [CrossRef] [Medline]
    40. Zorzal ER, Campos Gomes JM, Sousa M, Belchior P, da Silva PG, Figueiredo N, et al. Laparoscopy with augmented reality adaptations. J Biomed Inform. Jul 2020;107:103463. [FREE Full text] [CrossRef] [Medline]
    41. AlGerafi MAM, Zhou Y, Oubibi M, Wijaya TT. Unlocking the potential: a comprehensive evaluation of augmented reality and virtual reality in education. Electronics. Sep 20, 2023;12(18):3953. [CrossRef]
    42. Tang YM, Chau KY, Kwok APK, Zhu T, Ma X. A systematic review of immersive technology applications for medical practice and education - Trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev. Feb 2022;35:100429. [CrossRef]
    43. Williams MA, McVeigh J, Handa AI, Lee R. Augmented reality in surgical training: a systematic review. Postgrad Med J. Sep 30, 2020;96(1139):537-542. [CrossRef] [Medline]
    44. Chen P, Liou W. The effects of an augmented reality application developed for paediatric first aid training on the knowledge and skill levels of nursing students: An experimental controlled study. Nurse Educ Today. Jan 2023;120:105629. [CrossRef] [Medline]

    ‎
    AR: augmented reality
    ECMO: extracorporeal membrane oxygenation
    ICU: intensive care unit
    IU: intention to use
    PE: perceived enjoyment
    PEU: perceived ease of use
    PU: perceived usefulness
    SUS: System Usability Scale
    TAM: Technology Acceptance Model
    VR: virtual reality

    Edited by A McDaniel; submitted 01.11.23; peer-reviewed by G Alkawsi, G Evangelinos; comments to author 01.12.23; revised version received 22.12.23; accepted 31.03.24; published 23.05.24.

    Copyright

    ©Suyoung Yoo, Sejin Heo, Soojin Song, Aeyoung Park, Hyunchung Cho, Yuna Kim, Won Chul Cha, Kyeongsug Kim, Meong Hi Son. Originally published in JMIR Serious Games (https://games.jmir.org), 23.05.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.