Self-injurious behavior and aggression manifest in as many as 30% of individuals with intellectual and developmental disabilities (IDD) [1]. Beyond physical harm, these behaviors produce chronic stress for families, economic strain on service systems, and overall disruptions to stakeholder well-being [2]. Restrictive placement, chemical/physical restraints, and punishment often used to suppress severe challenging behaviors are not long-term solutions and are associated with negative outcomes [3]. Educators and other personnel in service settings require training in nonpunitive, effective methods for reducing challenging behaviors and achieving optimal outcomes for people with IDD [4].

Challenging behaviors have many potential causes [5]. Functional communication training (FCT) is one of the most widely validated interventions for reducing aggression and self-injury associated with social causes in children with IDD by teaching appropriate communication alternatives (ie, replacement behaviors, [6,7]). Instead of hitting adults to escape from difficult tasks, for example, children can be taught to request assistance (eg, “I don’t understand,” [8]). FCT has been shown to be effective at reducing challenging behaviors, often yielding improvements in behavior and communication [6,9].

Despite its demonstrated effectiveness, the dissemination of FCT may be hindered by the complexity of the procedure and intensive nature of staff preparation [10]. Effective training typically involves didactic instruction, modeling, rehearsal with feedback, and in-situ performance checks, all of which require expert trainers and access to clients or trained confederates [10,11]. These logistical and resource-intensive demands pose challenges to scaling FCT, especially when opportunities for rehearsal are limited. Recent reviews note that, although effective training improves implementation of FCT, studies vary widely in their use of role-play, simulations, and direct client application [10]. Such constraints can impede skill fluency and may slow adoption in schools and settings facing shortages in personnel or other resources.

Advancements in digital training have increasingly positioned immersive virtual reality (VR), or interfaces (eg, head-mounted displays, motion sensors) that situate participants within virtual space [12], as a promising modality for training educators and service providers (eg, [13,14]). For the purposes of training individuals to use FCT, VR places trainees inside realistic, 3-dimensional scenarios populated by avatars that emit challenging behavior, with feedback adjusted based on the trainee’s performance. Clay and colleagues [13] demonstrated the feasibility of VR-based opportunities for practice among small cohorts (n=6‐13) of novice graduate students and clinicians [15]. Training consisted of lecture, modeling the technique within VR, prompting, and feedback. Trainees demonstrated mastery of the procedure by engaging with a scripted avatar within the simulation. Across studies, some trainees acquired the procedure without proceeding through the entire training [13]; others required additional training from clinicians following VR rehearsal [15]. Gains from VR training transferred to brief live role-play simulations [15].

One barrier to the use of VR simulations for training is the need for professional oversight; VR provides opportunities to rehearse a wide range of skills but, on its own, does not alleviate shortages of qualified supervisors in disability service fields. Artificial intelligence (AI), broadly defined as software capable of acquiring, processing, and acting upon information [16], has the potential to reduce the need for human assistance for tasks such as assessment and feedback in VR simulations [17]. Generative AI and natural language processing can create speech for avatars and assess written or spoken trainee responses; however, much of this work involves speech or text-based interaction, with limited emphasis on the behaviors often addressed through FCT [10].

King and colleagues [18] evaluated an AI-augmented FCT simulation in VR using an automated training model incorporating speech-to-text, text-to-speech, text classification, and motion sensing. In 2 small, single-case design studies, participants received prerecorded instruction, video models, and automated systematic prompting, with performance measured through in-simulation speech and motion tracking. Trainees exhibited improved FCT skills within the simulation following training. A subsequent randomized controlled trial [19] evaluated the simulation with 49 general education undergraduates using a waitlisted design, randomly assigning participants to either an intervention group receiving lecture, modeling, and VR practice or a control group receiving lecture and modeling only. Results indicated that the VR group demonstrated significantly more correct FCT steps at posttest (d=2.45) and maintenance (d=1.58) compared with controls, with correspondence between AI-administered and human-observed assessments averaging approximately 90%, though groups did not differ on content knowledge tests or self-reported confidence and assessments were conducted exclusively within the simulation environment.

Despite these promising findings, VR remains less attractive as an instructional platform due to concerns regarding the cost of hardware and its acceptability across diverse learners [19]. Trainee motions that the system acknowledges, as well as characteristics of the avatar and virtual world, must be programmed into the lesson in advance. Thus, opportunities to provide trainees with multiple exemplars of practical situations are limited. The cumbersome nature of VR, which can be difficult to implement in crowded spaces and difficult to use for individuals without previous experience, represents an additional impediment to its application in many training settings. Additionally, there are critical issues that undermine positive findings of many VR training studies [20]. One of the most persistent involves the frequent misalignment between intervention conditions featuring VR and control conditions, which often differ in terms of both opportunities for practice and other crucial elements of instruction [21]. The imbalance impedes attempts to isolate the unique advantages of immersive training. However, comparing AI-augmented VR to high-quality training featuring multiple opportunities to rehearse practice may also be inappropriate, as these systems are often intended to compensate for resource constraints that prevent optimal training [19].

Chatbots—or computer-based applications that can engage humans in conversations through spoken or written text—represent an increasingly popular AI tool in higher education settings [22]. Uses for chatbots range from the generation of learning exercises, planning documents, conversation partners, and assistance in writing assignments [23,24]. Role-play activities designed to assist in the acquisition of procedural skills, such as those facilitated by VR, are relatively scarce in the chatbot literature [22]. Nonetheless, the studies that have examined this topic suggest that chatbots designed to engage students in role-play might improve procedural accuracy and fluency (eg, [25]). Compared with VR, chatbots may be able to provide a wider range of learning scenarios and, when coupled with large language models (LLMs; eg, ChatGPT), may be able to provide rich feedback in relation to targeted skills. On the other hand, King and colleagues [26] suggested that immersion and opportunities to rehearse relevant physical activities might make VR more suited to learning complex procedures such as FCT. Given the relative ease at which chatbots may be deployed, the extent to which skills learned in a purely textual format generalize to physical activities as well as the utility of chatbots in learning steps in the procedure more generally warrant further examination.

Chatbots can process learner input, provide adaptive feedback, and simulate interactive dialogue in a manner similar to supports featured in previous instructional VR studies (eg, [19]). In educational contexts, chatbots are increasingly regarded as scalable tools to support personalized learning and address the diverse needs of classrooms [23]. However, text-based formats do not simulate aural and visual stimuli encountered in authentic environments or require the learner to engage in physical or spoken behaviors associated with appropriate practice. These features are associated with effective training in behavioral service fields [27] and represent one of the purported advantages of the VR format [28], particularly in the context of behavior analytic theories of learning emphasizing correspondence between the modalities engaged during learning and performance [29-31]. More specifically, instruction and assessment can be characterized as learning channels that represent both how stimuli are perceived (eg, see) and how a response is produced (eg, say) [29,30]. The primary implication is that instruction must explicitly support this relationship, particularly if the learner does not have experience with the relevant channels [31].

Cognitive load theory, which proposes that humans have limited mental capacity for processing information and storing it in long-term memory [32], has further implications for training simulation. Intrinsic cognitive load refers to the mental demands associated with the complexity of the information and interactions relevant to instructional goals (eg, breadth of information required for adequate performance); extraneous load refers to additional mental resources expended in processing stimuli irrelevant to instructional goals (eg, reconciliation of various media used in instruction) [33]. Although this would appear to support the deployment of chatbot simulations or simulations that do not require management of controllers or other materials [34], VR may be better positioned to fill the gap between text and the physical and mental demands of practice [13,15,18]. Recent research further suggests greater task complexity can be tolerated by trainees [35] and may promote deeper learning [36].

Research on chatbot-based training for preservice teachers, and especially for teaching FCT, remains limited. No published studies have directly compared VR-based FCT training simulations to text-based alternatives, leaving questions about their relative effectiveness and generalization. Additionally, a primarily text-based medium may not be suitable for teaching skills comprising both speech and physical action, which has motivated much of the VR scholarship in this area [13,15,18]. This gap highlights the need for systematic evaluation of chatbot training alongside VR in the context of FCT.

This exploratory study addresses questions regarding both the relative advantages of VR-based training compared with aligned alternatives and the viability of chatbots as tools for teaching instructional procedures. Building on two recent works that established the technological infrastructure and preliminary empirical support for automated VR–based behavioral skills training in FCT [18,19], we examined performance within and across VR and chatbot training modalities to examine whether skills acquired in one modality generalize effectively to the other. By investigating these aspects, this study explored the potential of various instructional modalities for training individuals in evidence-based behavioral interventions. The study was guided by the following primary question: To what extent does improved performance from pre- to posttraining differ by training modality and probe format? A secondary question pertained to participant perceptions of learning following the intervention: To what extent do participants’ perceived knowledge, confidence, and utility of the procedure vary based on training modality?

All study procedures and consent forms were approved by the institutional review board (IRB) of a midwestern public university in the United States as exempt prior to the commencement of participant recruitment (2022409364). Although all procedures were disclosed to the IRB, this study was not registered. The first author met with interested students via teleconferencing software to describe the study in greater detail and confirm participation. Written consent was obtained prior to the initiation of procedures. Study exclusion criteria were developed to prevent discomfort or exacerbation of conditions associated with the use of VR. Participants were advised to terminate sessions if they experienced discomfort; no attrition or adverse events occurred, however. All initial data were confidential, with numbers used in the place of names; all codes linking individuals to data were destroyed after the conclusion of the study. Text transmitted to OpenAI during chatbot sessions consisted of typed statements, described actions, and other procedurally constrained responses directed at a simulated student within partially scripted scenarios. No names, codes, or identifying information were disclosed. Data-sharing options in the OpenAI platform were disabled across all settings for model feedback, evaluation, inputs, and outputs. Consent procedures therefore addressed confidentiality and chatbot participation without specifically itemizing data transmission. No figures or supplementary materials permit the identification of participants. Secure digital media were used for data storage; paper forms were secured in a locked office. A US $10 gift card was offered as an incentive for participation. All participants accepted payment upon conclusion of the study. This study is reported in accordance with CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) [37].

Eligibility criteria included being at least 18 years old, a currently enrolled student at the university in which the study was conducted, proficient in English, having no history of epilepsy or discomfort associated with virtual reality usage, and availability for on-campus meetings scheduled in March 2025 and April 2025.

A total of 23 participants voluntarily enrolled in the study. Participants ranged in age from 19 years to 42 years (mean 24.0, SD 5.3 years). Of the participants, 15 identified as female, 7 identified as male, and 1 did not disclose. Most (21/23, 91%) were undergraduates, with an average of 2.8 (SD 1; range=1‐5) years in their respective programs including education, special education, and fields outside of education. Demographic surveys indicated there were no significant differences in participant characteristics or familiarity with VR or AI assistants prior to the study. We assigned participants to either the VR training (n=12) or chatbot training (n=11) groups.

Recruitment, which extended from January 2025 to March 2025, was conducted through convenience sampling with a snowball component. Specific methods included (1) in-class announcements by the instructor of a classroom management course with a total enrollment of 90 and (2) peer referrals among students enrolled in computer engineering courses. Participation through both channels was determined via self-selection. Ten students from the classroom management course participated, resulting in a participation rate of 11% (10/90). Given the nature of snowball sampling, it was not possible to determine a recruitment rate for students from engineering courses.

The project was conducted over 2 weeks in April 2025. Each participant completed 1 session per week across 2 consecutive weeks, with sessions scheduled to ensure approximately 7 days elapsed between sessions. For each session, the first author scheduled individual, face-to-face meetings at a time of convenience for participants. Sessions were conducted individually in an office area equipped with reliable internet access and the necessary equipment.

No a priori power analysis was conducted. To our knowledge, no published study had compared chatbot and VR training in the current instructional model. Consequently, no effect size was available to derive a target sample. No target sample size was set in advance, and no interim analyses nor stopping rules were used. Enrollment was constrained by the limited number of people who responded through recruitment channels, in addition to restricted availability of personnel and funds beyond the study window. All recruited participants successfully completed the study. Precision of parameter estimates of the exploratory analyses was determined using 95% confidence intervals.

Primary outcomes included the observed mastery probe (OMP) and the text-based probe (TBP). Secondary outcomes related to participants’ perceptions and experiences were obtained through responses to the SKIL Survey [38]. No covariates were included. A social validity survey assessing acceptability of the intervention was also administered. Participants completed a demographic survey used for describing the sample and assessing equivalence between groups.

OMP data were collected through direct observation of participants’ use of VR at baseline and training sessions. All OMPs were terminated by observers as soon as participants exhibited an incorrect response or failed to respond within 4 seconds (ie, single-opportunity method; [42]). Steps participants did not complete due to termination were counted as incorrect. Feedback was not provided following OMPs.

As with the OMP, researchers observing typed responses from a secondary screen assessed responses during the baseline and training sessions. TBPs were terminated immediately following an incorrect response. Participants who did not initiate typing after 4 seconds were reminded that they could opt out at any time. A time-based discontinuation rule was not observed due to unpredictable duration of typed responses relative to actions or brief statements required during OMPs. However, the interval between each simulated student message and participants’ submission of responses—which includes the duration needed to read, compose, and type the response—was logged by the system. Participants did not receive feedback during TBPs.

Participants completed the SKIL Survey during baseline and the training sessions. Participants accessed a digital version of the SKIL Survey using a Qualtrics link. A digital social validity survey was completed at the end of the training session.

Interobserver agreement (IOA) was collected for OMP and TBP across multiple phases of the project. The first author trained observers prior to the project by scoring videos depicting nonparticipants. Observers were required to master the training criteria of scoring 3 consecutive videos with greater than 90% agreement with the first author before finishing training. Agreement was defined as 2 observers independently recording an identical participant response for the same step of the procedure. Scheduling allowed IOA collection for 87% (20/23) of all probes administered within baseline and 61% (14/23) of all intervention session probes across VR and chatbot groups (ie, VR=58.33% and chatbot=63.64%).

The OMP was created by a multidisciplinary team including engineers, behavior analysts, and school psychologists and subsequently evaluated by a panel of experts in behavior intervention [18,19]. The TBP was directly modeled after the OMP by the original development team. The SKIL Survey is commonly featured in personnel preparation studies involving behavior intervention [43,44]. Adapted SKIL Surveys have demonstrated acceptable Cronbach α for knowledge (0.91), confidence (0.88), and usefulness (0.92) at baseline (eg, [43]). The social validity survey was adapted from previous VR studies [19,41].

IOA was calculated by dividing the number of agreements by the total number of steps observed. Average IOA for baseline sessions was 100% for OMP and 98.75% (SD 5.59%; range=75%‐100%) for TBP. The average training session IOA for the VR group was 92.21% (SD 12.01%; range=66.67%‐100%) for the OMP and 94.29% (SD 12.42%; range=66.67%‐100%) for the TBP. For the chatbot group, the average IOA for the OMP and TBP was 100%. Disagreements were reconciled via consensus, and reconciled scores were subsequently analyzed. For the SKIL Survey, Cronbach α for all subscales among the current sample was acceptable (knowledge, 0.95; confidence, 0.90; usefulness, 0.92) at baseline.

Conditions were experimentally manipulated. We analyzed the effectiveness of the 2 multicomponent training packages using an independent measures pretest-posttest design with 2 conditions: the VR condition (ie, lecture, model, and VR practice) or a chatbot condition (ie, lecture, model, and chatbot practice). All participants completed both the OMP and TBP at baseline and following training regardless of assigned condition.

A commercially available desktop, connected to 2 external monitors, was used to project the two 4-minute instructional videos. The monitor view was mirrored to the researchers’ computer, allowing them to confirm playback quality and timing. Chatbot simulations were run on a high-performance Windows laptop connected to a fast campus network, which enabled the experimenters to mirror participants’ screens in real time and verify the automated scoring routines. The laptop was connected to a portable secondary monitor and keyboard, which allowed the principal investigator to monitor trainee performance and maintain control of the chatbot during training sessions.

For the VR condition, participants interacted via standalone Meta Oculus Quest 2 head-mounted displays equipped with native motion controllers. The headsets’ inside-out tracking obviated the need for external sensors, and the built-in microphones and speakers enabled natural voice exchanges with the simulated students. After completing the VR or chatbot sequences, participants used standard Windows desktops to complete project assessments.

The first author, a faculty member with a PhD in special education and certification as an applied behavior analyst, led all sessions. Four graduate students with backgrounds in special education and school psychology assisted with observations, with no more than one additional observer attending each session. An undergraduate student pursuing a BA in computer science provided technical assistance as needed. All sessions were conducted in English.

Following baseline assessments, the 23 participants were randomly assigned to groups who received either the chatbot (n=11) or VR (n=12) training. No participants withdrew or switched conditions; both groups completed the OMP, TBP, and all secondary measures at follow-up without losses; and all were included in the analysis. A diagram depicting participant flow appears in Figure 3.

Groups did not differ significantly on demographic characteristics (Table 1). Descriptive statistics for baseline outcomes, including mean TBP response times, appear in Table 2. Differences between VR and chatbot groups at baseline were not significant for either the OMP (OR 0.89, 95% CI 0.40‐2.01; P=.78) or TBP (OR 0.80, 95% CI 0.36‐1.76; P=.57).

Checklists corresponding with appropriate steps for each session were created prior to the beginning of the project and used to assess protocols across all sessions. Specific steps pertained to the use of correct instructions, administration of required assessments, and management of the simulation. All observers were trained by the first author to assess the procedural fidelity prior to the project. Fidelity observers were required to reach the mastery criterion (>90% agreement with the first author) before coding independently. Fidelity observers were required to observe the first author administer 3 sessions and administer a session in the presence of the first author before administering sessions independently. Administrators consulted fidelity checklists during sessions. The research team collected fidelity in baseline (13/23, 57%), VR training (5/12, 42%), and chatbot training (3/11, 27%) sessions. Fidelity was calculated by dividing the number of items on the observation checklist correctly performed by the session administrator by the total number of checklist items. Mean fidelity was 99.70% (SD 1.07%; range 96.15%‐100%) in baseline sessions. Mean fidelity for both VR training and chatbot training sessions was 100%.

Participants rated the intervention favorably across all domains (Table 2). Mann-Whitney U tests indicated no significant differences between the chatbot and VR groups on any domain (U=55.50‐62.50, all P>.05). On the social validity survey, 10 participants (10/23, 44%) left open‐ended comments. Representative comments, though not formally analyzed, illustrate participant impressions of the training. Several participants noted positive impressions of the VR sessions (eg, “VR session [was] more helpful than chatbot”) but also offered constructive feedback on its mechanics and design (eg, “Eyes in the VR creep me out,” “I had some issues getting the VR system to detect my voice”). A chatbot participant noted that “It was difficult to tell which parts of the screen I should be looking at, or when I needed to try again.” Others praised the video training components (eg, “I liked the diagrams and the video summary”), although some felt the videos were overly dense. One participant expressed reservations about the behavioral strategies, and another felt uneasy rushing through the practice sessions. Despite these concerns, several participants described the project as interesting or beneficial (eg, “This was wild”).

No adverse effects were observed or reported in either condition.

This exploratory study compared the effectiveness of 2 aligned multicomponent training packages featuring either VR or chatbot simulations for teaching steps in FCT. Our first question pertained to the improvement participants in each condition exhibited across the OMP and TBP. Although participant performance improved following both training packages, results suggest the pattern of transfer differed by training condition. Whereas participants who trained in the chatbot simulation outperformed VR-trained participants on the TBP, there was no significant difference between the 2 groups on the OMP—yielding a significant 3-way interaction. Notwithstanding the limited power of this study, these findings indicate that the chatbot group’s learning was maintained across both text-based and physical assessments. Our second question pertained to changes in participants’ self-perceived knowledge, confidence, or intervention utility. Across groups, no differences were identified. Participants reported that both modalities were generally acceptable and easy to use. Findings provide tentative support for the use of chatbot training for skills involving speech and movement while underscoring the importance of considering the alignment of training and assessment modality in training evaluation.

This study differs from earlier scholarship concerning VR simulations and chatbots. Few studies have compared immersive simulations with comparison conditions aligned in terms of content and instructional methods [20,21]. The disparity between control conditions in VR studies—which frequently consist of didactic instruction—and immersive learning conditions has previously been justified by the absence of scalable methods of role-play and other activities facilitated through VR [19,57]. That even the modest comparative advantages associated with immersive VR instruction [58] did not materialize in this study is therefore notable given the large advantages of VR simulations identified in our previous scholarship [19]. This likely reflects the nature of the comparison: VR previously outperformed didactic instruction because the latter lacks opportunities for rehearsal, whereas the chatbot condition provided aligned rehearsal of the same procedure.

Previous chatbot instructional literature generally does not pertain to acquisition of procedural skills such as the use of FCT [23]. The within-subject effects demonstrated here suggest that chatbots may be a viable format for procedural rehearsal. As both VR and the chatbot were embedded within the same instructional approach [18], the 2 technologies should be viewed as supplemental to effective instruction rather than the active ingredient [19,20]. The apparent asymmetry in transfer effects between the two modalities, or the extent to which chatbot-trained participants’ performance on OMP did not differ significantly from that of the VR group while outperforming them on the TBP, represents a potential advantage of the chatbot medium that merits closer consideration.

Our findings, which support an advantage of chatbot training for text-based assessment and nonsignificant differences on a VR-based assessment when compared with a group trained using VR, must be interpreted cautiously given the small sample. Leaving this aside, there are additional explanations of the asymmetry in transfer effects. The tutorial included many of the motions needed in the OMP (eg, manipulating objects, grabbing the avatar’s hands). Combined with their text-based training of FCT procedures, this motor experience may have assisted OMP performance. As both groups completed identical VR and chatbot tutorials, their effects were likely constant across conditions. Neither group performed more than a few steps correctly on baseline probes completed after the VR and chatbot tutorials. Results of prior studies similarly suggest the tutorial, in the absence of rehearsal, was not equivalent to the VR training condition [18], even when accompanied by a video explaining the FCT procedure [19]. Given that the tutorial, on its own, was insufficient to produce comparable gains, the chatbot group’s OMP performance in this study is better attributed to the combination of tutorial exposure and text-based FCT rehearsal than to incidental motor practice. The possibility that motor practice divorced from knowledge of the goal—such as the VR tutorial—may support future goal-directed performance has been considered, and rejected, in theoretical work (see [59]). This phenomenon also did not emerge in our previous work [19]. Regardless, tutorials are necessary to separate the challenges of engaging with new technology from effective training [46]. Acclimating learners to any supplemental technology is a critical aspect of equity in service delivery [60]. The extent to which preparing learners for a novel instructional format (ie, VR) unavoidably overlaps with the content is worth considering in future simulation research.

The asymmetry in transfer may also reflect the relative familiarity of the learning channel or the combination of learners’ sensory engagement with content (eg, reading, hearing) and the requirements of any subsequent response (eg, act, write; [61]). These transfer relations do not emerge uniformly and develop as a result of an individual’s instructional history [29-31]. For the chatbot group, transferring skills to the OMP meant generalizing steps acquired through reading and typing (ie, read/type) to a format in which the observation of simulated events—previously rendered in textual terms during the chatbot training—occasioned specific statements or actions (ie, see/say, see/do). This form of transfer is consistent with typical instructional practices [31]. In contrast, VR participants learned to speak or act in response to simulated events and were required by the TBP to read descriptions of events then type a description of the appropriate action. Translating action into description represents a channel most learners have little occasion to develop [31]. In terms of cognitive load, an unfamiliar channel requires learners to allocate resources to deciphering the stimulus–response relation rather than the task itself [35]. Conceptualizing the alignment between instructional and assessment channels in these terms may help developers streamline the demands that instruction and assessment place on learners.

Although responses on the SKIL survey did not differ across conditions, participant perceptions of their knowledge and confidence, as well as the utility of FCT, improved following training. Apart from the asymmetry in skill transfer across modalities, these findings are consistent with the observed improvement in performance and previous research (eg, [18]). Given the dissociation between participant perception and performance, we recommend future work in this area incorporate multiple outcomes [62]. Social validity measures indicated that several aspects of VR (eg, grasping virtual objects via controllers) and the chatbot (eg, orientation of feedback) presented challenges for students. Although technological advances—such as controller-free VR interfaces—may address some of these challenges, an emphasis on user experiences across modalities and diverse user populations is warranted [46].

A discussion of AI-enhanced instructional simulations would be remiss without reference to ethical concerns, including privacy [63] and effective treatment [46]. Reserving technologically advanced, yet functionally inferior, methods for underserved areas could perpetuate poor outcomes for certain groups [64]. Future research must focus on efficacy and the equitability of underlying assessment [46]. Automated systems (eg, speech recognition) typically score performance against local criteria [19]. Standardized procedures for establishing the validity of automated assessment would permit meaningful comparisons across studies and increase the generality of findings [65].

This study has several limitations. The small sample in this underpowered exploratory study reduced our ability to detect small to moderate effects. Thus, the absence of significant effects between VR and chatbot groups on the OMP may be a statistical artifact rather than an indication of the equivalence of the two formats. Significant effects were estimated with wide confidence intervals that warrant replication. Future studies should analyze multiple factors in the context of adequately powered comparisons of modally distinct but fundamentally aligned instruction [66]. Observers responsible for scoring probes were aware of participant training conditions, which introduces a risk of expectancy bias. Additionally, the sample consisted of a relatively homogeneous group of college students, some of whom studied topics with little relation to FCT. Future studies should incorporate broader and more diverse participant samples, including in-service professionals and individuals from varied educational or clinical settings, as well as those from culturally and linguistically diverse backgrounds whose experience or attitudes toward technology may differentially affect results.

Although probes were functionally aligned, there were differences in procedures. The most salient of these is the lack of a time-based discontinuation rule for the TBP. This difference was a property of the probe rather than training group: All participants, regardless of group, completed both the OMP and TBP. The advantage of chatbot training on the TBP cannot be attributed to the timing asymmetry exclusively, as both groups completed the probe under identical conditions. Mean response time on the posttest TBP (Table 2), which indicated that the VR group took more time to complete responses despite their lower scores, suggests the VR group did not benefit from the absence of the time constraint on the TBP. The chatbot training sessions, including assessments and tutorials, were approximately 10 minutes longer than the VR sessions. This may have resulted in a difference in engagement time, yet this does not appear to have led the chatbot group to outperform the VR group on both probes. Research interested in standalone measures of performance should align administration parameters to the greatest extent possible and validate the results against applied performance. This study did not evaluate in vivo generalization. Future research should examine how skills acquired through VR or chatbot interactions transfer to naturalistic or live settings, such as classrooms, community-based environments, or homes.

This study compared the effectiveness of two emerging approaches to procedural training. Results are tentative and may be attributed to methodological factors. Future work might also manipulate the alignment between training and assessment channels directly, testing whether the transfer asymmetry observed here reflects the relative familiarity of the response channels each modality requires. The extent to which aligned instruction produces differential outcomes across specific training modalities such as high-fidelity or virtual simulations [66]—and efforts to facilitate the development and alignment of evaluation platforms—represents an important area for further study [21]. A related area concerns the use of technologies such as AI-augmented VR and chatbots within more authentic contexts (eg, coursework), as opposed to isolated simulations [67]. At minimum, the study provides support for the use of chatbots in the acquisition of procedural skills, which may serve to support ongoing work examining the incorporation of chatbots into various instructional settings as well as their interplay with other technology platforms [68].

The ultimate utility of instructional simulations is determined by the extent to which they influence practice. The same features that necessitate automated simulations for skills such as FCT—risk for injury and fluidity of clinical situations [13,19]—also impede efforts to evaluate effects of instruction on professional practice and client outcomes. Validating the instructional model on lower-risk skills amenable to this approach may be warranted.

This study—to our best knowledge, the first comparison of aligned chatbot- and VR-based FCT training simulations—suggests that multicomponent trainings featuring either chatbot- or VR-delivered rehearsal improved participants’ acquisition of a complex educational procedure, with similar improvements in participant knowledge, confidence, and perceived usefulness. Whereas many studies compare VR to didactic instruction or assess performance within a single modality, this study compared VR with a chatbot capable of providing comparable opportunities for rehearsal. Findings contribute to the field by supporting the viability of chatbots for procedural training and by increasing the salience of alignment between modalities emphasized in training and assessment. Positive findings for chatbots are particularly encouraging given the potentially lower costs of development and deployment relative to VR, which could increase access to effective practices in behavior intervention and related fields. Additional work with larger samples is needed to confirm the observed transfer asymmetry and to examine the effects of these approaches on practitioners in applied settings.