This study included 67 volunteer students from Acibadem Mehmet Ali Aydinlar University Vocational School for Anesthesiology. The participants, aged between 20-22 years, were in their fourth semester (spring semester 2023-2024) of the anesthesiology program. Five students declined to participate in this study. The remaining participants were randomly assigned to 2 groups: 31 students using VR controllers for game interaction (VR controller group) and 31 students using AI-supported voice recognition for game interaction (voice command group). The exclusion criteria for the study included prior ACLS training, a history of VR-induced motion sickness, and medical conditions such as vertigo attacks or using medications causing vertigo-like symptoms. Three participants from the voice command group were excluded for not meeting the inclusion criteria. The randomization was based on the participants’ university ID numbers, with odd numbers assigned to one group and even numbers to the other. The clinical trial registration identifier of the study is ClinicalTrials.gov NCT06458452.

This study was conducted in compliance with ethical guidelines for human subject research. The research protocol was reviewed and approved by the Scientific Ethical Committee of Acibadem Mehmet Ali Aydinlar University (ATADEK 2024-6). For the primary data collection, informed consent was obtained from all participants in accordance with the guidelines set forth by the scientific ethical committee of Acibadem Mehmet Ali Aydinlar University. The informed consent obtained for the primary study explicitly allowed for the deidentified data to be used in subsequent research, including secondary analyses, without requiring additional consent. This study was conducted with strict adherence to privacy and confidentiality standards to protect the data of all participants. The study data were deidentified prior to analysis, ensuring that no personally identifiable information was accessible during the research process. Any potential identifiers were removed or coded to maintain confidentiality throughout the study. Participation in the study was entirely voluntary, and no compensation was provided to the participants. All images included in the study and supplementary materials do not feature individual participant images or any identifying clues that could reveal the identity of the users.

All the participants completed a pretest consisting of 10 multiple-choice questions about ACLS, which served to assess the participants’ initial knowledge level and whether the content was compatible with the latest version of ACLS (Multimedia Appendix 1) [21].

Following the pretest, participants were randomly divided into 2 groups: the voice command group (n=31) and the VR controller group (n=31). The voice command group members completed the VR-based ACLS serious game in training mode twice, using an AI-supported voice command as the game interface. The VR controller group members also completed the VR-based ACLS serious game in training mode twice, but they used VR controllers as the game interface. Although having volunteered for the study, 3 participants from the voice command group did not attend the study.

The participants from both groups completed the Turkish version of the Presence Questionnaire (PQ), a 7-point Likert scale, to estimate the level of presence during gameplay [22]. The participants rated their experience on a scale from 1=not at all to 7=completely by selecting the option that best reflects their experience. The PQ was first developed by Witmer and Singer [23] to subjectively measure the level of presence in 3D virtual environments. The scale was revised in a subsequent study conducted in 2005, which identified its factor structure as consisting of 4 factors [24]. The adaptation of the PQ to Turkish was conducted by Gokoglu and Cakiroglu [22] in 2019. The factor analysis of the Turkish version of the PQ found a 5-factor structure—involvement, sensory fidelity, adaptation or immersion, interaction, and interface quality—which was confirmed through a confirmatory factor analysis. The Turkish version of the questionnaire used in this study, along with its translated English version, are provided in Multimedia Appendices 2 and 3.

In the involvement factor of the scale, there are items for evaluating a situation obtained because of focusing the individual’s attention and mental potential consistently or meaningfully on relevant stimuli, activities, or events. For the sensory fidelity factor, items related to perceiving the VR scenario visually, auditorily, and haptically were included. The adaptation or immersion factor includes items related to being included in and interacting with a continuous flow of experiences and stimuli with a sense of being surrounded, while the interface quality factor includes items to evaluate the effect of visual and control interfaces in the VR experience. Finally, in the interaction factor, items related to interaction between individuals and the virtual environment were included. The adapted scale was found to be valid and reliable with a Cronbach α=.84 and applicable with its 29 items under 5 factors [22]. Participants’ presence score was calculated as the mean of their responses to all items in the scale.

Following the survey, participants were asked to indicate how they believed they would perform in the upcoming exam on a scale of 1 to 5, where 1=very bad and 5=very good, to assess confidence in future performance. After confidence assessments, participants completed the exam module of the VR-based serious gaming module, which evaluates the technical and nontechnical of the participants. The voice command group used a voice interface, while the VR controller group used VR controllers to complete the exam. At the final stage of the study, participants completed a posttest, which had the same content as the pretest.

The software used in this study is a VR-based ALS serious game that has been developed in accordance with the ACLS guidelines European Resuscitation Council and American Heart Association [21,25]. The development team worked closely with clinicians to ensure compliance with these guidelines as well as crisis resource management criteria [26-28].

This serious game works with a learning management system and keeps user credentials as well as game results in a shared database with the help of a Learning Record Store [29-31]. The interactions of the players with the virtual world are tracked using a 3D visualization engine and experience application programming interface calls are created accordingly [30,32]. These calls are sent to the Learning Record Store servers with the help of a unity extension library that uses HTTP protocol [33]. This library also includes security measures and authentication methods included with HTTP protocols required for the safekeeping of user data.

The VR-based ALS serious game includes an AI-driven voice recognition feature that uses NLP algorithms of a service called Wit.ai [34]. Wit.ai is an open-source service that has existing Unity plug-ins that facilitate the implementation process. Through this service, the sentences of the user are sent to wit.ai servers, processed into game commands according to predefined keywords, and returned to determine the action taken by the user.

The serious gaming module offers 2 language options: English and Turkish. Participants were given the choice between the 2 options. As all volunteers preferred Turkish during gameplay, both the VR controller group and voice command groups selected Turkish as the game language. Since the voice command group also chose Turkish, the AI-driven voice recognition was based on the Turkish model. There are 2 stages in this serious game: beginner training and VR-based exam. These stages can be played by either voice command or using the controller of the VR headset. The simulated environment consists of a virtual hospital room and equipment to immerse the user as much as possible in this experience.

The beginner training stage is directed toward users who are willing to familiarize themselves with the ACLS algorithm and the VR environment. In this stage, the user is completely guided from the beginning to the end using visual and audio cues. Which objects to interact with or which voice commands to say are given to the users? Examples of guidance can be seen in Figure 1 and Figure 2 respectively. There are also no timing constraints in this mode which removes the stress element and allows an inexperienced user to focus on learning the algorithm [8].

The exam stage is designed for users who are confident in their abilities after spending some time in the basic training stage. Each action of the user is scored according to its accuracy, order, and timing. Each action taken by the user is graded based on its timing, order, and accuracy. The final score of the VR-based exam, out of 100 points, is derived from 2 categories: 70% (70/100) from ACLS assessments and 30% (30/100) from crisis resource management performance. The scores of the exam mode were stored and could be accessed afterward.

Data from the comparison of pretest and posttest scores, VR-based exam scores, confidence ratings, presence ratings, and time spent in training mode were evaluated to compare the 2 groups. Additionally, the relationship between VR-based exam scores and time spent in training was assessed. Given the presence of multiple variables, a variety of statistical tests were used for data analyses based on the nature of the addressed question. To compare the performance change from pretest to posttest across the voice command and VR controller groups, a 2×2 mixed ANOVA was used. VR-based exam scores of voice command and VR controller groups were compared using 2-tailed, Welch independent-sample t test as the homogeneity of variances assumption was not met. Confidence ratings of the voice command and VR controller groups were compared using a Shapiro-Wilk test as the data were not normally distributed. Confidence bias was assessed using a 2-tailed, 1-sample t test. Presence scores of the groups and time spent in training mode across groups were compared using a 2-tailed, Student independent-sample t test.

Prior to analysis, the dataset was screened for outliers across all variables. No outliers were detected; thus, no data points were removed. The central tendency of the data was described using the mean. Effect sizes were reported as Cohen d (0.2=small, 0.5=medium, and 0.8=large) for t tests and η²P (0.01=small, 0.06=medium, and 0.14=large) for ANOVA.

Each reported analyses were deemed appropriate based on data distribution and variance homogeneity. Data distribution was checked using the Shapiro-Wilk test of normality and homogeneity of variances was assessed by performing the Levene test of equality of variances. Details regarding the selection of the statistical analyses used can be found in Multimedia Appendix 4.

Data was analyzed using JASP (Version 0.19.3, JASP Team), an open-source software for statistical analysis. The figures were created using JASP.

The following analysis was conducted with 59 participants, as shown in Figure 3.

To reveal whether performance change from the pretest to the posttest is different between groups, a 2 (group: voice command group vs VR controller group) × 2 (test: pretest vs posttest) mixed ANOVA where the first factor is the between-subjects variable and the second is the within-subjects variable was conducted. Levene test showed that the assumption of homogeneity of variances across groups was met for both pretest (P=.85) and posttest scores (P=.54). Furthermore, a Shapiro-Wilk test for normality showed that pretest scores of the voice command group (P=.14) and VR controller group (P=.15) were normally distributed. Similarly, posttest scores of both groups were also normally distributed (P_{voice command group}=.09 and P_{VR controller group} =.43). Considering that the within-subjects variable has only 2 levels, a test for sphericity assumption was not required. Hence, the analysis was conducted without any correction as all assumptions were met. The results showed that the voice command group performed better than the VR controller group in general, (F_1,57=6.43; P=.01; n²p=0.10). Moreover, performance was higher in the posttest than in the pretest for both groups (F_1,57=15.69; P<.001; n²p=0.22). This increase in performance from pretest to posttest, however, was not different for voice command and VR controller groups, as the interaction between group and test was not significant (F_1,57= 0.05; P=.83; n²p=8.18×10^–4). These findings indicate that while the voice command group had a higher performance in general, both groups showed a performance increment from pretest to posttest, the increase in performance was not higher in one group than the other (Figure 4).

Crucial to the purpose of the study, an independent-sample, 2-tailed Welch t test showed that the VR controller group’s performance in the VR-based exam (mean 80.47, SD 13.12) was higher than the voice command group’s performance (mean 66.70, SD 21.65; t_45.83=2.94; P=.005; d=0.77, 95% CI 0.23-1.30; Figure 5). Moreover, both groups spent a similar amount of time in the exam (t₅₇=–0.95; P=.35; d=–0.25, 95% CI –0.76 to 0.27; voice command group: mean 18.59_, SD 5.28 and VR controller group: mean 17.33, SD 4.83; Figure 6).

Results revealed that there was no significant difference between groups’ confidence ratings (W=392.00; P=.49, r_b=.15, 95% CI –0.38 to 0.20; voice command group: mean 3.79, SD 0.77; VR controller group: mean 3.60, SD 0.72).

In general, there is a positive relationship between performance and confidence levels, such that higher performance is accompanied by higher confidence levels [18,35]. Therefore, given the lower performance of the voice command group, it can be expected that their confidence ratings would also be lower. The absence of this expected difference points to a confidence bias in groups voice command group, that is, a mismatch between confidence and performance [36]. To see if this is the case, confidence bias for each participant was calculated by scaling their confidence ratings and advanced training performance over 1 and subtracting the scaled performance from the scaled confidence rating, a commonly used method for confidence bias quantification [37]. A 2-tailed, independent-sample t test showed a significant difference, with the voice command group’s performance and confidence difference (mean 0.09, SD 0.24) being bigger than the VR controller group’s performance and confidence difference (mean –0.09, SD 0.18; t₅₇=–3.21; P=.002; d=–0.83, 95 CI –1.37 to –0.30; Figure 7). To clarify the possible source of the group differences in confidence bias, a 2-tailed, 1-sample t test was conducted for each group. The results showed that the mean confidence bias of the VR controller group was significantly lower than 0, indicating an underconfidence bias (t₂₉=–2.56; P=.02; d=–0.47), and the mean confidence bias of the voice command group was significantly higher than 0, indicating an overconfidence bias (t₂₈=2.08; P=.047; d=0.39).

The 2-tailed, independent-sample t test showed that the presence ratings did not differ across groups (t₅₇=1.16; P=.25; d=0.30, 95% CI –0.21 to 0.81; voice command group: mean 5.18, SD 0.83; VR controller group: mean 5.42, SD 0.75).

This study explored the potential of AI-driven voice control in VR-based ACLS training. The findings revealed that both groups improved from pretest to posttest and the VR controller group achieved higher scores in the VR-based exam compared to the voice command group. This suggests that traditional controllers may currently provide a more reliable interaction method in VR environments. Analyses confirmed that the data followed a normal distribution and satisfied the assumption of equal variances, with the exceptions being exam scores and confidence ratings, for which appropriate corrections were applied to address the assumption violations. These corrections, along with the reported effect sizes demonstrating medium to large effects across the majority of significant findings, underscored the robustness of the results and the adequacy of the sample size.

Despite the potential advantages, using voice command systems in VR gaming has also its limitations in speech recognition accuracy, latency, the system’s ability to understand context-specific commands, problems caused by environmental noise, and the user’s accent or speech patterns. Voice command systems in VR gaming present both promising advantages and notable limitations [17,38-41]. While these systems offer an intuitive, hands-free interaction method, their effectiveness can be hindered by factors like speech recognition accuracy, latency issues, context-specific command understanding, environmental noise, user’s accent, and speech patterns. Errors in recognizing words or phrases can disrupt gameplay and may lead to reduced immersion. Delays in system response to voice commands can significantly impact the flow of VR experiences, particularly in fast-paced gaming scenarios where real-time interaction is crucial. Research shows that even minor delays can break the sense of presence and reduce overall engagement. The ability of a system to interpret context-specific commands remains a challenge. For instance, players might use commands with implied meanings or colloquial expressions that the system struggles to process, affecting functionality and the seamlessness of gameplay. Research highlights that background noise significantly impacts voice recognition performance. In noisy environments, players might need to repeat commands, which can be both disruptive and exhausting, diminishing the immersive qualities of VR gaming. Studies also show that accents, dialects, and individual speech patterns affect recognition accuracy. Systems trained on limited datasets may struggle with diverse user inputs, leading to unequal performance across different user demographics [17,38-41].

These literature data underscore the potential problems of AI-guided voice recognition in terms of accuracy and performance in different languages [42,43]. Since Turkish was selected as the language for AI-driven voice recognition in this study, there were similar difficulties due to the challenges of voice recognition, which caused participants in the voice command group to lose time and achieve lower scores compared to the VR controller group.

The level of presence did not significantly differ between the 2 groups, which contrasts with prior expectations that voice control would enhance immersion [18,19]. The level of presence may be hindered by problems in word recognition and delays in system response [17,38-41].

Confidence ratings were comparable across groups; however, an interesting finding was the presence of an overconfidence bias in the voice command group and an underconfidence bias in the VR controller group. This suggests that the mode of interaction can influence learners’ self-perception of their performance. Aligning perceived competence with actual performance remains a critical goal for optimizing educational outcomes. While they are related, confidence bias and task performance are dissociable [36], making it important to examine both to better understand the learning process. Although both groups reported comparable confidence levels, the voice command group exhibited a significant overconfidence bias while the VR controller group exhibited a significant under-confidence bias. This discrepancy highlights the importance of aligning perceived competence with actual performance outcomes in educational settings [44] and indicates that the means of education can differentially affect this alignment.

Although current limitations in language recognition pose challenges, ongoing advancements in AI and NLP technologies may close these gaps and provide more effective solutions in the future. These limitations indicate that while voice command systems present innovative possibilities in VR gaming, developers need to address these issues to enhance the user experience and ensure accessibility for a wider audience.

The first limitation of the study is the potential for selection bias, as participants were recruited from a single institution, which limits the generalizability of the results to other educational settings or demographics. The second limitation is the use of AI-driven voice recognition, which is prone to issues such as speech accuracy, environmental noise, and user-specific factors like accent, all of which could impact user experience and performance. A third limitation is the lack of longitudinal follow-up assessments to evaluate the long-term retention of skills, which is essential for assessing the effectiveness of VR-based training methods.

The results of this study highlight the effectiveness of VR-based training for ACLS. Although voice commands did not enhance the exam performance, level of presence, and confidence level of the participants in this study, voice commands showed a potential promise as an innovative interaction method. Addressing challenges such as language-specific recognition accuracy could significantly improve their application in VR-based educational modules.

Future research should focus on optimizing AI-driven interfaces and exploring their role in fostering learner immersion and confidence. Enhancing these aspects could lead to improved educational outcomes and better preparation for real-world scenarios in health care.