TACTICS VR Stroke Telehealth Virtual Reality Training for Health Care Professionals Involved in Stroke Management at Telestroke Spoke Hospitals: Module Design and Implementation Study

Background: Stroke management in rural areas is more variable and there is less access to reperfusion therapies, when compared with metropolitan areas. Delays in treatment contribute to worse patient outcomes. To improve stroke management in rural areas, health districts are implementing telestroke networks. The New South Wales Telestroke Service provides neurologist-led telehealth to 23 rural spoke hospitals aiming to improve treatment delivery and patient outcomes. The training of clinical staff was identified as a critical aspect for the successful implementation of this service


Background and Rationale
Ischemic stroke is a leading (and increasing) cause of adult-acquired disability worldwide [1].Longer time to reperfusion treatment has severe and negative impacts for patients with ischemic stroke [2], with a progressively increased likelihood of poor patient outcome with every hour of delay before reperfusion treatment is started [3].Variability in clinical care is a major contributor to delayed treatment [4], particularly in rural areas, and there are numerous challenges in delivering best practice stroke management in rural contexts in Australia [5] and worldwide [6].One significant contributor to delays at rural sites is the nature of the workforce, which is typically more junior, nonspecialist, and transient than the workforce at metropolitan hospitals, as well as having less access to continuing professional development training [7][8][9].There is extensive clinical trial evidence to inform best practice stroke workflow to deliver reperfusion therapies (eg, thrombolysis and endovascular thrombectomy).However, additional research is needed to inform the implementation of effective workflow processes and training in real-world clinical settings, particularly in rural areas.
To address variability in stroke care in rural settings, telestroke networks have been developed in Australia [10,11] and worldwide [12,13], where rural spoke hospitals are supported by comprehensive centers.Locally, in New South Wales (NSW) in Australia, we first initiated a telestroke network in 2013 with a single rural hospital, which expanded over subsequent years to include 5 rural sites linked to a single comprehensive stroke center.The telestroke service aims to provide urgent access to a stroke neurologist, improving access to reperfusion therapies as well as outcomes at rural sites.To date, the service has documented increased thrombolysis treatment rates [14] and highlighted the value of multimodal computed tomography (CT) imaging for stroke diagnosis in rural areas [15].The successful initial implementation of the local telestroke service provided pilot data [14] and is a cornerstone of the current statewide NSW Telestroke Service, which has undertaken a progressive rollout providing services to 23 rural hospitals [16,17].The implementation of the statewide service started in March 2020, with staggered rollout across the 23 sites through June 2022.We proposed that virtual reality (VR) training to support telestroke service implementation would be useful at both existing and new sites as a complement to support existing training, new staff inductions, training refreshers for existing staff, and initial site onboarding.Acute stroke management workflow is complex, and efficient team coordination and the sequencing of processes are challenging.The effective delivery of timely reperfusion treatment in acute ischemic stroke requires structured stroke workflow processes.Intensive quality improvement processes aimed at optimizing stroke workflow pathways by enhancing team communication and implementing site-specific processes can markedly improve reperfusion treatment delivery times [18][19][20], for example, the implementation of multifaceted implementation strategies via targeted quality improvement at 2 Australian comprehensive hospitals effectively reduced time to treatment [18,19].However, site-specific stroke workflow pathways are often not applied at rural hospitals where there are gaps in local training and resourcing, transient workforces, and limited effective application of quality improvement activities.In this context, stroke telehealth services can complement local processes by providing access to expert neurologist support and implementing a structured routine for stroke workflow.Importantly, the successful integration of telestroke services requires training local staff to increase awareness, confidence, and competence in telestroke processes.
Staff education and training are critical for the delivery of evidence-based stroke management, but this is limited in rural settings.Less than two-thirds of Australian physicians and nurses involved in stroke management report having received any interactive or competency-based training, including approximately 60% of nurses and 40% of physicians [21].Stroke management staff in regional areas, at sites with fewer annual stroke admissions (<75/year), and at sites without a dedicated stroke unit were less likely to have opportunities for professional development (63%, 62%, and 64%, respectively) [22].Stroke workflow training is typically delivered by neurology specialists via traditional face-to-face workshops.This training happens infrequently, has high logistical complexity and costs, and cannot be readily accessed by all staff at a given rural hospital.The delivery of training via this traditional model was further impaired during the COVID-19 pandemic, when travel between sites was restricted [23].On the basis of our previous work developing and implementing VR-based workflow training for XSL • FO RenderX health care staff in hyperacute stroke management in rural settings [24], we proposed VR-based training specific for stroke telehealth to support the implementation of the NSW Telestroke Service.
In medical fields, VR-based training consistently increases student engagement and motivation [25,26] and is effective for practical skills acquisition [27,28] and knowledge development [29][30][31].VR technology has previously been applied in the context of both stroke management and medical education [32][33][34].To date, VR applications for stroke have primarily targeted patients rather than health care professionals (eg, in the context of limb rehabilitation [35,36]).In the broader medical education context beyond stroke, most VR training applications for health care professionals have only been assessed in small pilot trials or in the context of medical school or residency training (systematic reviews and meta-analyses [37][38][39]).
In a nontelehealth context, we previously developed and implemented a novel fit-for-purpose VR-based stroke workflow training application called TACTICS VR Hyperacute Stroke Management [24].During initial implementation in the first trial cluster of 7 hospitals, TACTICS VR users reported a high degree of usability, acceptability, utility, and feasibility [24].Trainees self-reported increased confidence in their ability to make improvements in stroke management after training [24].The application is currently actively being assessed in the Trial of Advanced CT Imaging and Combined Education Support for Drip and Ship, which is implementing a package intervention to support advanced CT imaging and streamlined workflow training at rural sites in 3 Australian states and will also report effects on clinical behavior and patient outcomes [40].On the basis of the initial success of implementing TACTICS VR training in rural hospitals within a clinical trial context, we proposed the development of a second TACTICS VR module to support the implementation of the NSW Telestroke Service in this study.

Study Objectives
This paper describes the development and implementation of TACTICS VR Stroke Telehealth, a VR training module for spoke hospital workflow processes.The platform was conceptualized to support the NSW Telestroke Service rollout and provide improved training and awareness on effective stroke management to health care professionals at spoke hospitals involved in stroke management (eg, physicians, nurses, and radiographers).The study aims to assess TACTICS VR Stroke Telehealth implementation against critical elements that typically predict the future adoption and uptake of new technologies in educational practice.This includes the feasibility of the platform as a training resource in a real-world clinical setting, specifically in rural areas.Furthermore, end user acceptance, including perceived usefulness and usability, and perceived training impacts are assessed via survey feedback.

Project Conception and Scoping
The NSW Telestroke Service was jointly funded by the NSW and Australian federal governments in 2019, with service delivery at the first rural sites starting in March 2020 [16,17].It is a collaboration between the Prince of Wales Hospital, eHealth NSW, the NSW Agency for Clinical Innovation (ACI), and the NSW Ministry of Health in partnership with the Australian Stroke Foundation.The NSW Telestroke Service was rolled out progressively to its total coverage, with the final sites activated in June 2022, providing services to 23 rural hospitals across NSW [41].
TACTICS VR Stroke Telehealth was initially conceived through discussions among the Centre for Advanced Training Systems (ATS) at The University of Newcastle, the ACI, and the NSW Telestroke Service in 2019, informed by the development and implementation of the first TACTICS VR Hyperacute Stroke Management module within the TACTICS trial [24,40].

Overview
A descriptive survey study approach was applied to the creation and deployment of TACTICS VR Stroke Telehealth.Quantitative and qualitative data were collected during the development (from May 1, 2020, through April 14, 2021) and implementation of the training module across 23 hospitals in NSW (from April 14, 2021, through August 20, 2023).A particular focus was placed on assessing elements that typically predict technology adoption, including end user acceptance (perceived usability and usefulness), training impacts, and the practical challenges of technology delivery within a real-world health care setting.

TACTICS VR Stroke Telehealth Module
TACTICS VR Stroke Telehealth was developed for the stand-alone Oculus Quest 2 enterprise-enabled headset (Oculus or Meta) using the Unity platform (Unity Technologies).A detailed overview of included scenes, key features, objectives, and user interactions is provided in Table 1, with representative images presented in Figure 1.Users initially enter a 3D VR XSL • FO RenderX infinity space containing the application menu and brief overview text about the application.Upon initiating training, users are requested to provide brief demographic information (eg, hospital location and position title) and are provided a video overview of the training module and instructions on how to use the VR technology.Users then proceed through a single real-world stroke case from the prenotification and initial assessment phase through multimodal CT imaging, consenting, treatment with thrombolysis, and patient transfer for endovascular thrombectomy.Specific emphasis is placed on 3 telehealth consults for assessment, imaging interpretation, and consenting and treatment, with all other aspects of the workflow streamlined to minimize the total length of training.Users interact with elements in the VR environment, including avatars (eg, patient and nurse), equipment (eg, telephone and computer), and question and answer prompts to gather case information and prompt active decision-making.Suboptimal decisions result in immediate feedback (via text or audio) and time penalties, providing an element of gamification.For each user interaction, the accuracy was calculated as the number of suboptimal decisions selected (including both suboptimal decisions made and optimal decisions missed) as a percentage of total possible decisions.Time penalties for each suboptimal decision (or missed optimal decision) were arbitrarily determined through context expert feedback based on the perceived priority and impact that the specific decision would have on real-world workflow times (ie, suboptimal decisions deemed more serious were assigned larger time penalties).A stroke clock and an estimate of neuron loss are displayed at the top of the screen to emphasize the importance of accurate and timely workflow and model the real-world consequences of delayed or inaccurate decision-making.At the end of the module, users are provided with feedback on their training, including time spent in training, errors, and time penalties.TACTICS VR Stroke Telehealth was integrated into the existing TACTICS VR training platform, which includes a public-facing supporting website [42] and backend automated database and password-protected reporting environment (Figure 2).A detailed overview of the TACTICS VR training platform has been published previously [24].In this study, the public website was updated with a streamlined design to include information about the VR training module, the development process, funding sources, and contributors and stakeholders [42].The database and reporting environment were duplicated from the first TACTICS VR module and updated to reflect locations, user interactions, and scenes from the TACTICS VR Stroke Telehealth module to support the automated tracking of training implementation.The developed training module was endorsed by the Australasian Stroke Academy.

Study Participants and Recruitment
Stroke coordinators at spoke hospitals were requested to encourage any relevant health care staff involved in stroke management at their site to use TACTICS VR Stroke Telehealth training.All health care staff who tested the application were invited to participate in the data collection regarding its utility and training impact.Health care professionals from any background (eg, physicians, nurses, and radiographers) involved in stroke care and as identified by the stroke telehealth coordinator, were encouraged to explore TACTICS VR and provide automated anonymous feedback within the headset.Additional structured pre-post survey data were collected from this population at 7 spoke sites, with corresponding ethics approvals.

Measures: Self-Report and Survey Data
Self-report data (qualitative and quantitative in nature) were collected both automatically within the headset for all users and The full lists of questions and responses are provided in Tables S1 and S2 in Multimedia Appendix 1.

Data Analysis
Self-report data were summarized and presented as mean (SD) or absolute number of responses, using Prism (version 8.0; GraphPad), and every response was included.

Development of TACTICS VR Stroke Telehealth
The overall concept and philosophy for TACTICS VR Stroke Telehealth built on the preexisting TACTICS VR module and training platform [24].TACTICS VR Stroke Telehealth was developed using a structured stepwise approach throughout (an overview schematic is presented in Figure 3).Most importantly, the development involved and actively included subject matter experts (SMEs) and stakeholders.Specifically, members from the ATS, the ACI, and the NSW Telestroke Service as well as experts, including specialist neurologists, nurses, and radiographers, were consulted and were active creators of the platform from concept to software development.

TACTICS VR Stroke Telehealth Implementation
TACTICS VR was deployed as part of the NSW Telestroke Service, which embeds stroke coordinators at telehealth spoke sites and includes structured education and training, communications (eg, communities of practice), the deployment of technology (eg, workstation on wheels and web conferencing), and ongoing support.
Headsets were shipped out to telestroke site coordinators at 23 participating locations.Using enterprise-enabled headsets allowed the remote monitoring of activations, battery status, and the update deployment of the software, removing the management burden from the telestroke coordinators and making it the responsibility of a dedicated centralized position.Telestroke site coordinators were initially briefed and trained with the VR headset on a site-by-site basis via web conferencing and provided supporting documents (eg, a quick-access guide and an instruction manual).Site coordinators were then requested to encourage any health care staff involved in stroke management at their spoke hospital to use
Respondents had minimal previous experience with VR technology in any context (Table 3), and 3 (14%) of the 21 respondents had previously used any TACTICS VR training module.Before completing training, respondents indicated that they were likely to use telehealth in the future (19/21, 90% answered: mean 4.2, SD 1.0; 1=strongly disagree and 5=strongly agree) and were somewhat confident in their ability to effectively assess and treat patients with stroke (21/21, 100% answered: mean 3.5, SD 0.9; Table 3).Before the training, respondents were somewhat confident in managing the technical aspects of VR training (21/21, 100%: mean 3.6, SD 0.8; Table 3) and considered VR an effective method to teach or transfer knowledge about stroke telehealth workflow (21/21, 100% answered: mean 3.8, SD 0.8; Table 3).
When asked an open-ended question on which elements of the TACTICS VR Stroke Telehealth training module were most beneficial, respondents primarily noted the VR virtual environment, interactions or interactivity, content presentation, and the overview of the complete workflow process.Selected comments are provided herein (all responses are available in Table S2 in Multimedia Appendix 1): • Visual environment and scans, feedback on selections made, and step-by-step process

Easy to follow
• Instant feedback: "awesome" • Explanations provided at each decision point

Feedback or interaction
Respondents were also asked which elements of TACTICS VR Stroke Telehealth could be improved, and they primarily identified aspects relating to VR technology familiarity, site-specific differences in processes, and areas for future development or expansion (eg, role-specific workflow modules).Comments included the following:    Self-reported stroke management confidence <.001 4.2 (0.6) 3.2 (0.9) I am confident in my understanding of telehealth workflow processes .0014.1 (0.6) 3.1 (1.0)I am confident in my knowledge around accessing telehealth for acute stroke assessment or treatment .034.1 (0.6) 3.4 (0.9) I am aware of local stroke telehealth services and processes .024.2 (0.6) 3.7 (0.9) I am confident in my ability to optimally communicate with my colleagues to enable effective treatment of acute stroke patients .034.0 (0.6) 3.5 (0.9) I am confident in my ability to make improvements to how acute stroke care is provided to patients presenting to this hospital .154.3 (0.5) 3.9 (0.6) I am confident in my understanding of workflow practices to manage acute stroke patients a Survey responses collected on 5-point Likert scale (1=strongly disagree and 5=strongly agree; n=17-21).

Principal Findings
This paper describes the development and pilot implementation of TACTICS VR Stroke Telehealth, a VR-based training application specifically tailored for stroke telehealth workflow training.The TACTICS VR module was developed to address an identified training gap to support the rollout of the NSW Telestroke Service by, and with, involved stakeholders.In this study, we evaluated the acceptability, usability, and perceived training impacts of TACTICS VR Stroke Telehealth training across 20 rural NSW hospitals, including survey response data from a subset of 7 sites within a single local health district.The feedback indicated a high level of end user acceptability (including perceived usability and usefulness) among health care professionals involved in stroke management at telehealth spoke hospitals.Furthermore, the feasibility of deploying VR training in a real-world clinical setting outside of a formal clinical trial context was demonstrated, while noting that additional work is required to optimize training uptake and integration into existing training processes.TACTICS VR users provided positive feedback on application delivery and design.Survey respondents reported perceived improvements in key aspects of stroke telehealth management, including an increased understanding and awareness of telehealth workflow, knowledge XSL • FO RenderX around accessing stroke telehealth, an improved ability to optimally communicate with colleagues, and an increased ability to make improvements to stroke care.
Extensive participant feedback was very positive, and implementation was overall deemed feasible, but we believe that there is room for improvement in terms of overall training uptake.To encourage use and simplify deployment, we specifically developed training sessions that were relatively short (<20 minutes), included brief talking head videos emphasizing key learning objectives, and provided supporting documentation as well as a user-facing website so that training could be deployed on site without trainer support.
Although this study was not structured to objectively determine factors influencing site-specific implementation, the captured data and discussions did provide some useful insights to assess in future implementation interventions.Rural telestroke hospitals varied widely across a range of factors, including overall hospital size (from 30 to 520 beds), stroke team composition (eg, specific stroke teams vs emergency department-based or locum management, staffing numbers and positions, skill sets, and on-site vs off-site radiographers), resourcing and NSW Telestroke Service site coordinator demographics (eg, on site vs off site, preexisting members of the local team vs new appointments, clinical vs administrative background, and single site vs multisite coverage).Discussions with study investigators and site coordinators highlighted barriers relating to staffing (both trainer and trainee availability), existing attitudes and training frameworks, and competing priorities.Mapping these factors onto the theoretical domains framework highlights relevant aspects within the domains of knowledge, skills, social or professional role and identity, beliefs about capabilities, reinforcement, environmental context and resources, and social influences [48].By applying the Behavior Change Wheel [49], we propose a range of strategies that could address these domains within local contexts, including education (eg, promoting awareness of, and practical skills in delivering, VR training; and tailored training documentation), persuasion (eg, making VR training a mandatory component of local staff orientation), incentivization, environmental restructuring (eg, local staffing and resourcing), enablement (eg, upskilling training facilitators and providing for dedicated training time), communication or marketing, and service provision (eg, facilitated training by an external service provider).In the long term, if VR training is demonstrated to be effective and sustainable in supporting clinical behavior change and improving patient outcomes, it may be appropriate to consider its integration into consensus guidelines and clinical pathways.We anticipate that a combination of different implementation approaches and strategies, which applies a codevelopment approach to optimize training within each organization, will be useful to increase adoption across a range of rural sites with different needs.
As part of the development process, we also documented associated costs, which may be relevant to readers who are considering developing and implementing similar VR training in their specific context.Note that during the study period conversion rates ranged from approximately Aus $1 (US $0.6-$0.8).Direct costs associated with the development of TACTICS VR Stroke Telehealth training included VR application coding and updating of the existing automated analytics portal (approximately Aus $37,000; ie, ~US $24,200), expansion and redesign of the user-facing website (approximately Aus $4000; ie, ~US $2600), and hardware (VR headsets and Wi-Fi routers; approximately Aus $1800 [~US $1200] per site).Additional costs associated with project management, internal application testing, research assessment, ongoing support and maintenance, and implementation within the broader NSW Telestroke Service rollout were not specifically captured or reported.

Comparison With Prior Work
Telestroke services improve the rates of reperfusion treatment and functional patient outcomes without increasing the rates of treatment-induced hemorrhage, although it has been noted that evidence for impacts on resource use and cost-effectiveness is limited [50,51].As a result, patients treated through telestroke networks achieve outcomes that are similar to those managed at comprehensive stroke centers [10,12,13].Of note, in Australia, the importance of telehealth-supported stroke care was recently recognized through its inclusion in the Australia Stroke Foundation guidelines as best practice to improve outcomes in rural settings [52].However, the effectiveness of stroke telehealth delivery depends upon a number of factors, including trained staff, infrastructure to support CT image transfer, videoconferencing capabilities, established protocols and processes, and funding.Furthermore, effective deployment requires the coordinated training of health care staff in telestroke workflows across diverse clinical settings.TACTICS VR Stroke Telehealth was developed to address this need.
VR technology is increasingly used to support training in a range of workplace settings, including health care.VR has advantages over traditional 2D training delivery approaches in terms of presence, privacy, and immersion [53], but the relative benefit of VR regarding training outcomes depends upon the specific application as well as the delivery context.Several recent systematic reviews have collated findings on mixed reality or VR and health care staff training, with specific focuses on head-mounted devices (HMDs) [37] or augmented reality in medical education [39] and VR-based training in nursing [38].The majority of reported studies demonstrated the effectiveness of training (or noninferiority to traditional approaches), with greater trainee enthusiasm and enjoyment when using HMD-based technology in medical education [37]; XSL • FO RenderX improved performance time, confidence, and satisfaction using AR for medical education [39]; and improved knowledge development using VR for nursing education [38].However, the existing literature remains limited, with a low number of total studies (11-27 studies each across 3 systematic reviews), low total participant numbers (a total of 654-956 participants across all studies), and application to training in a limited range of medical fields (eg, 48% of HMD-based training studies in surgery and 15% in anatomy [37]).Notably, there has been limited application of VR technology for the continuing education of health care professionals, the target population in this study; for example, across all studies of HMD-based training in medical education, 60% of the study participants were medical students, and 30% were medical residents (ie, 90% combined) [37].On the basis of the reported 956 total participants across all 27 studies, this indicates that only approximately 95 (approximately 10%) nontrainee participants were included [37].Thus, the 177 VR sessions included in this study represent almost twice the total number of reported HMD-based VR training participants in the existing literature within a continuing medical education context.Across the 3 systematic reviews, the authors noted many limitations, including most of the studies being small-scale short-term pilot studies, often conducted in a single nonclinical setting with limited participant diversity in medical field or training stage and often poor-quality study design.Our study addressed several of these areas and provides insights into the application of VR training for the continuing education of health care professionals in a clinical context, which is a setting that is underrepresented-in terms of both population and training-in the existing literature.None of the studies identified in the systematic reviews specifically focused on the continuing education of health care professionals in stroke management.To our knowledge, the only VR applications that have been previously studied for stroke (other than our previous report of the first TACTICS VR module [24]) have targeted patient users to support poststroke rehabilitation (eg, limb motor function recovery) [35,36].
This study builds on our previous experience of developing the first TACTICS VR training module, which provided hyperacute stroke workflow training without telehealth integration [24].The first TACTICS VR Hyperacute Stroke Management module continues to be assessed in a clustered stepped-wedge clinical trial across 3 Australian states, with data collection conducted through the end of 2022 [40].This study differs from the previous work by upgrading content into a 6-degrees-of-freedom VR headset, developing training content specifically for stroke telehealth workflow, and assessing implementation outside of a formal structured clinical trial context.The specific elements incorporated into training include 3 telehealth consultation calls, a detailed voice-over walkthrough of CT imaging interpretation, the incorporation of an assessment support tool (termed the Acute Stroke Assessment Protocol Triage Tool), and a simulated workstation on wheels.Despite more limited infrastructure and support during implementation in this study, we noted very similar participant feedback on acceptability, feasibility, and utility, which was very encouraging.This study also builds on our previous work deploying VR training in workplaces outside of health care, which has highlighted the importance of considering broad institutional factors and developing frameworks that support training implementation, uptake, and ongoing sustainment [54][55][56][57].Consistently across sectors (ie, in health, tertiary education [55,56], and defense [54]), these aspects must be considered and addressed at the organizational level to optimize training implementation and maximize effects.
Training programs for health care professionals in stroke management are typically delivered face-to-face by an experienced trainer or clinical expert with relevant expertise via traditional teaching modalities (eg, printed materials and slide-based presentations), informal mentor or mentee learning, and the passive distribution of guidelines and training materials.These approaches support the delivery of evidence-based clinically relevant educational content but are expensive to either deliver or scale across rural settings and are passive and limited in utility.Group education approaches contribute to relatively small improvements in professional practice (eg, <10%) [58] but are better than the passive delivery of clinical guidelines and recommendations.Limited studies have assessed simulation-based approaches to support stroke workflow training [59].We propose that TACTICS VR training specifically and stand-alone VR-based training more broadly are a feasible and useful approach to supplement existing training modalities, particularly in rural contexts.

Limitations
We note several limitations, which should be considered when interpreting data from this study.First, study recruitment was led by on-site staff at spoke hospitals who were requested to target health care professionals involved in stroke management; as such, it was nonrandom.This was partially mitigated by encouraging site coordinators to recruit all staff involved in stroke management.Study participants primarily included physicians and nurses in emergency or acute stroke care.We note that the demographic mix captured from TACTICS VR users in the headset was similar to that of the survey respondents in terms of department, position title, and relative stroke experience.Although this was the intended target population for this training, their feedback on the utility of VR training to support workplace training more broadly may not be representative of all health care professional groups.It may be useful to specifically target additional training populations in future studies (eg, radiographers).Second, implementation was limited to hospitals in NSW, and detailed survey collection was limited to a subset of 7 sites, with corresponding ethics approvals.This was due to the complexities in obtaining specific ethics approvals for survey distribution across multiple local health districts and hospitals.Although it would be useful to collect additional data in specific local health districts or contexts to inform the location-specific integration of training in the future, we believe that the responses collected at the 4 sites are representative of the general feedback overall because the sites included are reflective of regional Australian spoke hospitals.Third, TACTICS VR Stroke Telehealth training was deployed as part of the NSW Telestroke Service, which included additional education materials, site visits, and follow-up support, as well as staffing and quality improvement activities.VR training was intended to complement these training approaches and resources as an add-on that can be deployed on site in rural XSL • FO RenderX settings.However, this approach does limit the ability to directly assess and attribute the effects of TACTICS VR training on clinical behavior or patient outcomes in isolation.Metrics on clinical behavior and patient outcomes are being captured as part of the NSW Telestroke Service implementation, which will provide insights into the effects of the overall program but will not allow an assessment of the direct effect of TACTICS VR Stroke Telehealth training specifically.Fourth, given the nature of this feasibility study, no control group was included; as such, we were unable to compare the effects of TACTICS VR Stroke Telehealth training with those of alternative approaches.Finally, the survey was intentionally kept short, noting the time constraints of study participants, and limited to study-specific questions.Additional assessments, including questions relating to the implementation approach as well as the application of validated instruments, may be useful in future studies (eg, assessing presence via the Presence Questionnaire [47], usability via the System Usability Scale [46], and engagement via the User Engagement Scale [60]).

Conclusions
User acceptance among those who used TACTICS VR Stroke Telehealth training was very high and included suggestions for future implementation and expansion.The feedback provided has directly led to the subsequent scoping and development of a dedicated stroke nursing module (TACTICS VR Stroke Nursing), and additional modules are in development to support paramedic-led stroke assessment and transport decision-making in a prehospital phase (TACTICS VR Stroke Paramedic).The platform is deemed feasible for deployment within a real-world clinical setting, supported by high ratings on perceived usability and usefulness.Further research is now required to document the effects of TACTICS VR training on clinical practice and patient outcomes as well as to assess cost-effectiveness.Further research will also be useful to optimize training uptake and identify approaches to support the integration of training into existing education pathways.

Figure 1 .
Figure 1.Screenshots and key features of TACTICS VR Stroke Telehealth.(A) Starting scene.(B) Representative scene, including avatars and user interactions.(C) Representative talking head video and tablet view for patient information.(D) User feedback scene.

Figure 2 .
Figure 2. TACTICS VR training platform.The preexisting TACTICS VR platform was updated to integrate (A) a publicly accessible website for user support and educational resources as well as (B) an automated backend analytics portal for the collection of training uptake and user performance data.

Figure 3 .
Figure 3. TACTICS VR Stroke Telehealth development pathway.Development occurred via a structured process from initiation through scoping and design, content development, iterative rounds of internal testing, implementation, and monitoring.NSW: New South Wales; SME: subject matter expert; VR: virtual reality.

••
Mock scenario at beginning or instructions on how to navigate through scenarios • Perhaps the selections could be more clear regarding only selecting the first step in parallel (ie, "checklist that you tick") • Role oriented (not nurse specific; "seems more medical officer") • Further information regarding interpreting scans, including information regarding contraindications to tissue plasminogen activator or endovascular clot retrieval (stroke treatments) Respondents who indicated that the additional TACTICS VR training modules should be developed (13/18, 72%) were asked what additional modules would be useful.Responses included the following: • Allied health team involvement (or physiotherapist, speech therapist, and dietitian to demonstrate the whole of team care) Trauma survey and cardiac thrombolysis • Procedural skills training Not applicable to my practiceWhat is your experience with virtual reality technology?n (%) to use telehealth services in the future 3.5 (0.9) I am confident in my ability to effectively assess or treat acute stroke patients 3.6 (0.8) I feel confident in regards to managing the technical aspect of the VR training tool 3.8 (0.8) I believe that VR can be an effective method to teach or transfer knowledge about stroke telehealth workflow a Pretraining survey responses collected on 5-point Likert scale (1=strongly disagree and 5=strongly agree; n=19-21).

Table 4 .Table 5 .
Survey respondents provided positive feedback on TACTICS VR Stroke Telehealth training and reported perceived training benefits.Posttraining survey response a , mean (SD) Feedback on TACTICS VR stroke telehealth 4.1 (0.9)The VR hardware (eg, headset and controller) were comfortable and easy to use 4.4 (0.7)The TACTICS VR Telehealth user interface was straight-forward and easy to use (eg, menu system, buttons, etc) 4.2 (0.6) I enjoyed TACTICS VR Telehealth training 4.2 (0.4) TACTICS VR Telehealth provided useful information to be useful for trainees and staff in the management of acute stroke 4.2 (0.7) TACTICS VR Telehealth provided accurate information 4.4 (0.5) TACTICS VR Telehealth was an effective tool for transferring knowledge about stroke telehealth practices 3.9 (1.0)The simulation was of sufficient realism to communicate the critical aspects of workflow 4.0 (0.6)The feedback provided at the end of the VR training module was constructive and useful Perceived training impact 4.3 (0.6) TACTICS VR Telehealth improved my awareness of stroke telehealth approaches 4.2 (0.8) TACTICS VR Telehealth improved my understanding of stroke telehealth approaches 3.8 (0.8) TACTICS VR Telehealth increased my confidence in stroke telehealth workflow 4.0 (0.8)As a result of TACTICS VR Telehealth training I am more likely to use stroke telehealth services in the future a Survey responses collected on 5-point Likert scale (1=strongly disagree and 5=strongly agree; n=17-20).Survey respondents provide positive feedback on perceived training effects regarding confidence, via comparison of posttraining versus pretraining survey responses.P value Posttraining survey response a , mean (SD) Pretraining survey response a , mean (SD)

Future
studies should seek to document enablers and barriers to VR training uptake and explore additional implementation approaches.To this end, we are currently assessing the effects of a range of approaches to TACTICS VR training uptake and sustainment, including making training mandatory, integrating training into existing continuing professional development credit systems, deploying VR headsets via roaming educators who travel among sites, providing time-limited access to hardware at individual sites on a rolling basis, delivering supported face-to-face education sessions or workshops where multiple staff members complete training in parallel (eg, 10 trainees/1-hour session), and embedding VR training within a broader staff training program with expert-supported debriefing.
Ethics approval for the overall NSW Telestroke Service research protocol was granted by the Prince of Wales Hospital Human Research Ethics Committee.Because of the practical constraints of ethics approval being governed at a local health district level, the collection of user survey feedback was limited to a subset of sites within a single local health district (n=7) via ethics approval from the Hunter New England Human Research Ethics Committee (REGIS Ref 2019/ETH01238 and HNEHREC Ref 18/09/19/4.13)lodged with The University of Newcastle Human Research Ethics Committee (H-2019-0343).Before the deployment of TACTICS VR training, all study sites provided consent to participate.Individual user participation was voluntary, and users were able to use TACTICS VR without participating in the study.Participants were advised that their consent was implied by the completion of the surveys.

Table 1 .
TACTICS VR Stroke Telehealth scenes, key features, objectives, and user interactions.
a VR: virtual reality.bQ&A: question and answer.c ED: emergency department.d CT: computed tomography.e EVT: endovascular thrombectomy.
Within the TACTICS VR Stroke Telehealth module, basic demographics information and user feedback (location, position title, relative stroke management experience, previous use of TACTICS VR, and user ratings) is automatically collected via user prompts within the headset and automatically transferred via Wi-Fi to the password-protected reporting environment.These data are presented as TACTICS VR user data throughout this manuscript.
[46,47]][45]erX via separate structured pre-post surveys deployed at specific locations, with corresponding ethics approvals.The administered surveys were generated by the research team and adapted from previous studies to accommodate the limited availability and time constraints of health care staff.Informed by the technology acceptance model[43][44][45], the survey items and domains focused on perceived acceptability, usability, usefulness, and training impacts and included individual items from validated scales, where appropriate[46,47].Items within the pretraining survey aimed to collect relevant demographic information (eg, specialty area, position title, previous stroke telehealth experience, and VR experience), general expectations of VR-based training, and anticipated learnings.Items within the posttraining survey aimed to collect feedback on TACTICS VR Stroke Telehealth training (eg, hardware comfort, design, user interactions, and areas of improvement), usability, and usefulness and perceived value.Items relating to stroke workflow awareness, knowledge, and skill confidence aligned with TACTICS VR learning objectives and were included in both pretraining and posttraining surveys to assess training impacts via pretraining survey versus posttraining survey statistical comparisons.
Because of the limited number of open-ended survey responses, no formal analysis of qualitative data was performed.Points that were repeatedly mentioned in the open-ended responses were identified by the study investigators, and examples are provided in the Results section (a full list of responses is provided in Table S2 in Multimedia Appendix 1).

Table 2 .
TACTICS VR users have a diverse range of stroke experience and report positive training impacts of the TACTICS VR Stroke Telehealth module in the headset.For both stroke management queries, n=177; N values are included for the TACTICS VR feedback fields (n=126-127).

Table 3 .
Pretraining survey respondents have a range of stroke management experience, limited virtual reality (VR) experience, and existing attitudes to VR training.