Recruitment took place from May 16, 2023, to October 31, 2023. Using convenience sampling, our study involved 2 user groups: patients (PEUs) and therapists (SEUs, participants), from rehabilitation centers in Switzerland (Reha Rheinfelden) and Austria (Clinic for Rehabilitation Muenster).

Patients from inpatient, outpatient, and Neuro Day Care units were considered at 1 of the 2 centers. Recruitment involved screening entry lists, reviewing treated patients, distributing flyers, emailing outpatient physiotherapists, and engaging therapists. Medical records were screened for basic eligibility, and eligible patients received verbal and written study information. They had at least 24 hours to decide and sign the consent form if they were interested. See Textbox 1 for eligibility criteria. For more details on the Mini Mental State Examination (MMSE) and Berg Balance Scale, see Multimedia Appendix 3.

SEU recruitment included physiotherapists, occupational therapists, training therapists, and sports scientists from the rehabilitation centers and external clinics. This diverse sampling approach ensured a comprehensive evaluation of the ExerG across various settings and professional perspectives. SEUs were required to be at least aged 21 years and hold a therapist qualification or a relevant bachelor’s degree.

This usability study sample included 15 PEUs and 20 SEUs. The SEU group emphasized diverse recruitment, engaging staff from various therapeutic disciplines with distinct specializations. Further, 5 therapists tested the ExerG in real-world scenarios with inpatients and outpatients. To balance sample sizes, an additional 10 PEUs were enrolled. Saturation was evaluated using code meaning, which entailed reviewing each interview to document identified codes and checking subsequent interviews for any new aspects, dimensions, or nuances of those codes until saturation was reached with no new information [27].

PEUs participated in an initial eligibility evaluation and baseline data collection, followed by a single session with 2 exercising rounds to test the ExerG and safety harness. SEUs, who were already familiar with technology due to their daily work, received training on system usage and technical issue management. This training included detailed explanations of their responsibilities while treating patients or mock patients during the evaluation session, with real individuals acting as stand-ins for actual patients (see the figure in the Results section for a flow chart).

PEUs attended a single study visit with 2 training rounds. At the start of each training round, participants received an overview of the training content and procedures. Researchers collected demographic and disease-specific data and conducted baseline assessments. Patients were then briefed on the gaming activities and fitted with safety harnesses and wrist and ankle motion sensors. Once secured, the first training round began, lasting approximately 10‐15 minutes, depending on the patient’s physical and cognitive capabilities.

SEUs from both centers participated in a study visit that included ExerG usage training and 2 observation rounds. Before the training, SEUs received the system manual and were advised to read it thoroughly. During the session, they attended an introduction and training on system usage, following a standardized protocol for explanation and demonstration, and practiced procedures for memorization.

In the usage observation rounds 1 and 2, SEUs set up the system, designed a physical-cognitive training session using the software’s exergames, prepared a patient with training materials, and guided them through the session. This study’s team observed and documented the process. At the conclusion, SEUs completed rating scales and questionnaires, with the key difference being that 1 center used mock patients during the observation rounds.

Data were collected during and after the PEU and SEU study visits using an observation protocol, questionnaires, and interviews. Quantitative data on usability, user experience, and acceptance were gathered with standardized rating scales. Qualitative measures, including semistructured interviews and specific questions about ExerG training scenarios, provided a more comprehensive assessment.

This usability study focused on system interaction and safety, encompassing both software and hardware as the primary outcome. Adverse events were monitored throughout this study. To identify difficulties, problems, or errors during use, the team followed a detailed observation protocol with predefined categories. Both end user groups provided objective measurements (scale ratings) and subjective feedback as part of the protocol. Performance was assessed by calculating the proportion of participants who completed or met the criteria for each exercise. The test moderator evaluated exercise performance using an observation sheet with the following ratings: very good, good, half-half, poor, and very poor. Performance was categorized as “very good” if the sum of “very good” and “good” ratings was ≥85%. Tasks were aligned with the training program flow, assessing instruction clarity, exercise performance, task completion, and exercise intensity. The system’s acceptability was evaluated using specific quantitative thresholds based on PEU performance and feedback. The system was deemed acceptable by meeting the following criteria: (1) training completion: at least 85% of PEUs successfully completed the first training round; (2) exercise skipping: fewer than 20% of exercises were skipped by PEUs; (3) instruction comprehensibility: at least 85% of PEUs rated their understanding of the training instructions as “well” or “very well”; and (4) performance: at least 85% of PEUs performed the physical and cognitive activities with “good” to “very good” performance ratings.

For SEUs, an observation protocol recorded correct use, errors, hesitations, task completion time, and needed assistance across 8 scenarios: hardware preparation, safety system preparation or attachment, fall simulation, initiating or supporting training, compiling training, ending training, removing safety system, and shutting down hardware. “Hesitation” implied uncertainty or a lack of confidence, while “difficulties” represented objective barriers (Multimedia Appendices 4 and 5).

Following each training round, PEUs evaluated perceived physical exertion and mental effort using the Borg CR10 (Borg Category Ratio Scale) [28] and the Paas Mental Effort Rating Scale [29]. They assessed player experience using the Player Experience Inventory (PXI) [30,31], focusing on both functional and psychosocial aspects of the exergame (Multimedia Appendix 3). Semistructured interviews and observations were also conducted for deeper insights into exergame functionality.

At the end of their study visit, PEUs participated in a brief semistructured interview featuring both structured and open-ended questions to gather insights into their perceptions of game control and comfort. Throughout the exergaming training, this study’s team used an observation sheet with predetermined categories for flow, motivation, and motion sickness, documenting all training activities in sequence. This sheet included checkboxes for ratings on:

Additionally, a study team member evaluated the perceived helpfulness of the exergame feedback using a rating scale (very good to very poor). A trained member also systematically observed and noted the patient’s emotions during gameplay [32]. Any emotions and difficulties observed or reported by the patient during or after the training were recorded and discussed to analyze their causes and potential solutions [32] (Multimedia Appendix 3).

SEUs evaluated the ExerG using the System Usability Scale (SUS) [33], a common questionnaire for assessing technology usability. The SUS includes 10 items with alternating positive and negative statements, each rated on a 5-point Likert scale, and is widely used in human-computer interaction research. According to Bangor et al [34], SUS scores of 70‐79 indicate good or acceptable usability, while scores of 80‐89 indicate excellent usability.

After completing all use scenarios, any difficulties encountered were discussed with SEUs. A semistructured interview, featuring both structured and open-ended questions, was conducted after the second ExerG usage observation round to evaluate user-friendliness and comprehensively assess SEUs’ experience with the device, focusing on exercise content, relevance to everyday life, and usability.

Demographic and usability metrics were analyzed using descriptive statistics. Absolute and relative frequencies were calculated for count and nominal data. Ordinal data (MMSE, Borg CR10, Paas Mental Effort Rating Scale, and SUS) are presented with medians and IQRs, which reflect the range between the 25th and 75th percentiles of the data. Continuous data are reported with means and SDs. The usability problem discovery rate was calculated by dividing the number of identified problems by the total number of possible problems and then multiplying by 100 to obtain a percentage. Statistical analysis was performed using IBM SPSS software (version 28.0; IBM Corp) and GraphPad Prism 9 (GraphPad Software).

The patients’ responses from the brief interview were recorded and subsequently entered into an Excel (Microsoft Corp) spreadsheet for further analysis. Aligning with the principles of qualitative content analysis as outlined by Mayring [35,36], and following familiarization with the material, a coding scheme was developed inductively, based on the data and the research question. This process resulted in categories that were unidimensional, mutually exclusive, and exhaustive. Clear coding rules were established to represent recurring themes, ideas, and concepts within the texts. Each condensed phrase was assigned to one or more codes from the coding scheme, ensuring a thorough classification of segments. Through an iterative process, the codes and their assignments were reviewed and discussed by 2 researchers to enhance intersubjective conformability and ensure reliability [37]. The codes were applied to relevant segments of text, ensuring that each segment was accurately classified according to the established categories [37]. The frequencies of the codes were tallied, and the coded data were synthesized to create overarching themes that captured the key insights from the analysis. Themes were then described and illustrated with quotations accompanied by corresponding participant IDs.

In total, 15 patients, with neurologist-assessed motor and balance impairment, exhibiting a mean age of 57.8 (SD 12.25) years, completed this study (Table 1). This study’s population comprised 3 patients using walking aids and 5 with a history of falls. Among the PEUs, 11 (73%) had prior experience with technology-based training, primarily familiar with exergames and stationary weight-relieving systems. Only 1 (7%) PEU was acquainted with both mobile body-weight-supporting systems and virtual reality.

A total of 20 therapists and sports scientists participated in and completed this study (Table 2).

No adverse events were reported during this study. Table 3 summarizes the activities per exercising round, detailing the clarity of instructions, PEUs’ exercise performance, and completion rates, including activity intensity for ExerG exercising round 2. PEUs became familiar with the tasks in round 1, leading to improved performance in round 2. However, newly introduced activities continued to pose challenges, with some usability issues noted. Overall, 23 of 29 (79%) instructions were rated acceptable, good-to-very-good exercise performance was observed in 19 of 29 (65%) tasks, and 28 of 29 (97%) tasks were completed.

Figure S2 in Multimedia Appendix 4 illustrates the emotions observed during PEU exercise activities. Enjoyment, curiosity, attention, and amazement were categorized as positive emotions, accounting for 79 of 90 (88%) of the expressed emotions during training. Some PEUs exhibited negative emotions, such as nervousness, boredom, and confusion, but none expressed anger or resistance. Perceived physical and mental effort increased with higher exercise levels, with physical exertion generally viewed as greater than mental effort (Figure S3 in Multimedia Appendix 4).

Analysis of the observation protocol revealed that most SEUs successfully executed the use scenarios. Minor difficulties or hesitation were noted in 14 of 41 tasks and difficulties in 2 of 41 tasks among the 20 therapists accounting for 1.7% and 0.2% of the total 820 task cases, respectively. Assistance from this study’s personnel was required in 9 instances across all scenarios (1.1%), with no use errors detected. Independent, correct, and effective use was observed for all tasks in fall simulation (3), supporting training (4), compiling training (5), removing the safety system (7), and shutting down hardware (8). The mean time to complete tasks was 2.6 (SD 3.8) minutes, indicating the clarity and effectiveness of the ExerG user handbook and ExerG usage training (Tables 4 and 5).

SEUs demonstrated mastery of the knowledge tasks, reflecting clear user handbook instructions. These tasks included summarizing the product’s purpose, adjusting the video game volume, troubleshooting software issues (eg, user interface not opening or incidents during training), and managing ongoing activities (eg, skipping activities or restarting programs). Hardware tasks involved resolving issues with HTC Vive (HTC Corporation) cameras and trackers, ensuring proper recognition at startup, and correcting screen projections. Backup system tasks required knowledge of troubleshooting rope positioning and resistance during the end-stop test.

Ratings of patient player experience using the PXI indicated a positive or very positive experience. Functional consequences, such as audiovisual appeal or ease-of-control, were generally rated high. In contrast, related to psychosocial consequences, “autonomy” received the lowest score and showed a large IQR, suggesting that PEUs felt they lacked control over their engagement in the game. Conversely, perceived mastery was rated highly (Figure 3 and Multimedia Appendix 4), closely linked to the aspect of autonomy.

Figure 3 shows a boxplot of player experience across patients as assessed using the PXI, grouped by functional and psychosocial consequences. The boxes represent the IQR, with a median line, while whiskers extend to the minimum and maximum within 1.5 times the IQR, and outliers are indicated as points beyond this range.

Figure S4 in Multimedia Appendix 4 presents findings from the structured portion of the semistructured interview with PEUs. Most patients expressed enjoyment of the ExerG training, with positive feedback on the software and hardware. For the first question, PEUs rated their overall satisfaction at a median of 9 (IQR 7.5‐10) on a 1‐10 scale.

Saturation was achieved after 15 patient interviews, which revealed three themes through content analysis of the qualitative data: (1) enjoyment and acceptance, (2) live interaction, and (3) safety and comfort. Most patients enjoyed the training and would continue using the device (each n=12, 80%) if certain improvements were made (n=13, 87%), such as better graphic design (n=5, 33%), upgraded tracking systems (n=3, 20%), refined swivel arms (n=8, 53%), and enhanced projector quality (n=3, 20%). More variety in game activities and difficulty levels would also enrich the experience (n=2, 13%). While most users could understand the activity instructions (n=8, 53%) and feedback texts (n=9, 60%), they preferred shorter, more concise instructions with clear task-related wording and adequate reading time (n=12, 80%). Patients felt safe (n=15, 100%) and comfortable using the safety harness (n=13, 87%) but recommended improving the safety system, particularly the swivel arm’s movements, for smoother balance control (n=5, 33%). Many also noted limitations with exploring the game environment (n=11, 73%) and discomfort with the rope pulley during specific activities (n=3, 20%). Table S1 in Multimedia Appendix 6 presents theme descriptions and quotes.

In SEUs, a median SUS score of 82.5 (IQR 18.7) was reported, indicating excellent usability of the ExerG [34]. The structured interview analysis showed that therapists were satisfied with the system (median score of 8, IQR 7‐8 on a scale of 1‐10), were willing to use it (n=20, 100%), and found it highly user-friendly (n=20, 100%). Less than half of the therapists found the feedback helpful (n=8, 40%), and one-fifth considered the video feedback to be comprehensive (n=4, 20%). SEUs indicated that the training content does not or only partly resembles everyday tasks (Figure S5 in Multimedia Appendix 4).

Saturation was achieved after 19 therapist interviews—content analysis of the qualitative data identified five themes: (1) acceptance and motivation, (2) meaningful activities, (3) training feedback, (4) individualization, and (5) safety and autonomy. Therapists noted that video-game based training could boost patient motivation through its engaging design and surprising elements, making it more enjoyable than standard therapy (n=12, 60%). They found the activities beneficial for physical and cognitive functions (n=20, 100%) but recommended incorporating more exercises that simulate daily living tasks for improved relevance (n=13, 65%). Audiovisual feedback and star awards were seen as helpful (n=8, 40%), with a preference for more specific motivational feedback (n=9, 45%) and clearer performance summaries (n=8, 40%). Performance ratings should be simple and understandable (n=9, 45%), with varying levels to sustain motivation (n=6, 30%). The system’s innovative approach to addressing cognitive and physical functions was well-received (n=9, 45%), and therapists requested a wider range of customizable exercises to cater to individual patient needs (n=10, 50%). The safety harness was valued for creating secure training environments (n=4, 20%). For independent exercising, therapists suggested clearer, shorter activity instructions that remain visible until acknowledged (n=6, 30%), underscoring the importance of considering the patient’s cognitive and physical capabilities for independent use (n=16, 80%). Theme descriptions and quotes are available in Table S1 in Multimedia Appendix 7.

PEUs and SEUs have identified opportunities to enhance system functionality and user experience. Key improvements include reducing inertia and swing of the swivel arm for better training comfort, making the safety system operable for therapists of all heights, and ensuring accurate tracking. Adjustments for color-blind users and projector light issues will enhance visual clarity. Clear instructional texts and synchronized movements will improve gameplay fluidity. Additionally, automatic screen arrangement correction, seamless activity transitions, and reliable HTC Vive tracker recognition will boost overall system performance.