Published on in Vol 11 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40651, first published .
Negami: An Augmented Reality App for the Treatment of Spatial Neglect After Stroke

Negami: An Augmented Reality App for the Treatment of Spatial Neglect After Stroke

Negami: An Augmented Reality App for the Treatment of Spatial Neglect After Stroke

Original Paper

1Center of Neurology, Division of Neuropsychology, University of Tübingen, Tübingen, Germany

2Flammer & Gläser UXplain GbR, Karlsruhe, Germany

3XPACE GmbH, Pforzheim, Germany

Corresponding Author:

Hans-Otto Karnath

Center of Neurology

Division of Neuropsychology

University of Tübingen

Hoppe-Seyler-Strasse 3

Tübingen, 72076

Germany

Phone: 49 7071 29 80467

Email: Karnath@uni-tuebingen.de


Background: A widely applied and effective rehabilitation method for patients experiencing spatial neglect after a stroke is “visual exploration training.” Patients improve their ipsilesional bias of attention and orientation by training exploration movements and search strategies toward the contralesional side of space. In this context, gamification can have a positive influence on motivation for treatment and thus on the success of treatment. In contrast to virtual reality applications, treatment enhancements through augmented reality (AR) have not yet been investigated, although they offer some advantages over virtual reality.

Objective: This study aimed to develop an AR-based app (Negami) for the treatment of spatial neglect that combines visual exploration training with active, contralesionally oriented rotation of the eyes, head, and trunk.

Methods: The app inserts a virtual element (origami bird) into the real space surrounding the patient, which the patient explores with the camera of a tablet. Subjective reports from healthy elderly participants (n=10) and patients with spatial neglect after stroke (n=10) who trained with the new Negami app were analyzed. Usability, side effects, and game experience were assessed by various questionnaires.

Results: Training at the highest defined difficulty level was perceived as differently challenging but not as frustrating by the group of healthy elderly participants. The app was rated with high usability, hardly any side effects, high motivation, and entertainment. The group of patients with spatial neglect after stroke consistently evaluated the app positively on the dimensions of motivation, satisfaction, and fun.

Conclusions: The Negami app represents a promising extension by adding AR to traditional exploration training for spatial neglect. Through participants’ natural interaction with the physical surrounding environment during playful tasks, side effects as symptoms of cybersickness are minimized and patients’ motivation appeared to markedly increase. The use of AR in cognitive rehabilitation programs and the treatment of spatial neglect seems promising and should receive further investigation.

JMIR Serious Games 2023;11:e40651

doi:10.2196/40651

Keywords



Spatial neglect is the dominant disorder after a brain lesion of the human right hemisphere [1,2]. The most common cause of spatial neglect is a large stroke in the right middle cerebral artery area that affects a perisylvian network consisting of the superior/middle temporal cortex and the parietal and ventrolateral frontal cortexes [3]. Affected patients behave as though the left side of their surrounding space no longer exists. The patient’s eyes and head are sustainedly oriented to the side of the brain lesion (ie, mostly to the right side) [4,5]. When searching for objects, the patient’s visual and tactile exploration is shifted to the right side. Accordingly, objects or people located on the left side are ignored and not found. Many therapeutic approaches to spatial neglect thus entail exercises (eg, search training on large projection screens) and activities (eg, reading and copying tasks, picture descriptions) that require increased and active turning of the patients toward the neglected contralesional side [6-9]. In this training, termed “visual exploration training” or “visual scanning training,” exploration movements are improved, and compensatory search strategies are practiced, leading to improvements in neglect behavior in important everyday situations.

A critical component of neurorehabilitation and its success is related to patients’ motivation to train [10]. Improving the latter is one of the main aspects of recent rehabilitation efforts known as “gamification” [11,12]. Gamification describes the application of game design elements in a nongame context to motivate desired behavior. Treatment motivation in stroke rehabilitation can additionally be increased by gamified feedback (ie, giving feedback about the performance in the corresponding task by awarding points) [13].

The principle of gamification is further emphasized when combined with virtual reality (VR) methods. The principle of active exploration training in spatial neglect has already been extended by several VR approaches. Nonimmersive VR procedures, in which a virtual environment is displayed on a monitor with which the patient interacts without being immersed in it, have been successfully used in group studies for therapeutic purposes [14-16]. On the other hand, immersive VR procedures, in which the patient becomes part of a virtual 3D environment surrounding them by means of VR goggles, have also been developed, gearing toward active exploration scenarios. Recent proof-of-concept studies have shown that healthy participants and patients who have had a stroke perceive this technique as highly motivating and enjoyable [17-19]. Moreover, immersive VR applications have already been tested with individual patients with spatial neglect for treatment purposes [20,21].

Another promising way to increase training motivation in poststroke rehabilitation is through augmented reality (AR). AR describes the computer-generated addition of virtual elements to reality. The visual, real world is augmented by virtual images and figures via a video camera of an electronic device such as a tablet. The virtual addition takes place in real time in the sense of a live transmission and thus combines the real and the virtual [22]. Bakker and colleagues [23] developed the first game combining AR elements with visual scanning training in the context of neglect rehabilitation. It consisted of participants searching for virtual images projected onto a wall in the environment of a museum. Design decisions were made based on expert opinion as well as the opinion of patients with spatial neglect. Features to increase extrinsic motivation were rated as the most important design decision.

In this development and feasibility study, the application of an AR technique for the treatment of spatial neglect is further developed. It is based on the observation that visual exploration training seems to be particularly effective when combined with an active rotation of the trunk contralesionally [24]. An attractive extension of this treatment principle is the innovative app named Negami. This app is based on the principle that patients are playfully motivated to orient themselves to their neglected side of the real room by (1) following and (2) searching for a virtual element (in this case an origami bird), actively exploring space by turning their gaze, head, and trunk. In developing the game, special consideration was given to the age group to ensure high usability. Our goal was to develop a task that would motivate the patients and that they would enjoy. By eliciting the subjective views of both healthy elderly participants (aged 57 to 89 years) and patients with spatial neglect after stroke regarding our AR-assisted exploratory therapy app, we also aimed to uncover barriers to its use and make recommendations to improve the future development of therapies with AR.


Participants

Ten elderly neurologically healthy individuals with a mean age of 66 (SD 11.9) years, including 5 (50%) females, participated in this study. In addition, we included 10 patients, including 2 (20%) females, who were experiencing spatial neglect and left-sided hemiparesis after a stroke in the neurological right hemisphere —5 (50%) with infarction and 5 (50%) with hemorrhages. They had a mean age of 61.3 (SD 15.2) years and a poststroke interval of 138.4 (SD 192.1) days when they were tested for spatial neglect. More details of the patients are provided in Multimedia Appendix 1.

The inclusion criterion for the patients was the presence of spatial neglect after a stroke. In addition to clinical behavioral observations, diagnostic criteria had to be met in at least 2 of the following 4 neglect tests: the Letter Cancellation Test [25], the Bells Test [26], a Copying Task [27], and a Line Bisection Task [28]. All 4 neglect tests were performed on a Samsung S7+ tablet with screen dimensions of 285 × 185 mm. The severity of spatial neglect in the cancellation tasks was determined by calculating the center of gravity of the target stimuli marked in the search fields, known as the Center of Cancellation (CoC). A CoC value ≥0.08 indicated left-sided spatial neglect [3,29]. The Copying Task consisted of a complex scene consisting of 4 objects (fence, car, house, and tree), wherein points were assigned based on missing details or whole objects. One point was given for a missing detail and 2 for a whole object. The maximum number of points was therefore 8. A score higher than 1 (ie, >12.5% omissions) indicated neglect [27]. In the Line Bisection Task [28], patients were presented with 4 different line lengths eight times each (ie, 32 lines in total). The cut-off value for spatial neglect was an end point weightings bias (EWB) value ≥0.07 [30].

In the group of patients, the mean CoC on the Letter Cancellation Test was 0.42 (SD 0.28), the mean CoC on the Bells Cancellation Task was 0.38 (SD 0.29), the score on the copying test was 4.4 (SD 1.51), and the EWB score on the Line Bisection Task was 0.33 (SD 2.5).

Ethics Approval

All participants gave their informed consent to participate in the study, which was conducted following the ethical standards of the 1964 Declaration of Helsinki. This study was also approved by the ethics committee of the University Clinic of Tübingen (373/2021B02).

The Negami App

Overview

Negami [31] is a mobile app that enables the treatment of spatial neglect using gamified AR. It is optimized for use on tablets but also works on smartphones. In this study, a 3rd generation Apple iPad Pro 12.9” was used to run the app and conduct the therapy sessions. Negami provides 2 modes (Task A and Task B), which are detailed in the subsequent section. In both tasks, the live video stream from the devices’ back cameras is displayed on the screen. Additionally, a virtual element (origami bird) is added (augmented) as a 3D object that must be followed (Task A) or searched for (Task B) by moving the tablet in 3D space. An overlaying crosshair user interface (UI) element provides visual feedback and guidance for the patient.

Technical Background

The Negami app can be achieved via the Negami website. The app was implemented in C# programming language using the open-source cross-platform framework Xamarin (Microsoft Corp) [32]. It allows the creation of native Android, iOS, and Windows apps by sharing UI and code across platforms. The Xamarin platform consists of Xamarin.iOS and Xamarin.Android projects, which are implementations of the Mono framework for iOS and Android, respectively. Mono is a cross-platform open-source .NET compatible software framework. Additionally, Xamarin includes the Xamarin.Forms library for the creation of cross-platform user interfaces using XAML (Extensible Application Markup Language). A UI created in Xamarin.Forms is automatically mapped to native controls on each platform during the build process. Xamarin was chosen for the development of the Negami app because it provides native runtime performance, which is important for game-based apps, as well as an abstraction layer over many platform-specific application programming interfaces (APIs), thus increasing development speed. Additionally, Xamarin offers a set of bindings for full access to all device-specific APIs (Figure 1). Apps built with Xamarin are compiled into native packages for each platform. For the Android platform, the written code is compiled down to Intermediate Language (IL), which is then just-in-time compiled to native assembly on app launch. On iOS, the app is fully ahead-of-time compiled into native ARM (Advanced Reduced Instruction Set Computer Machine) assembly code (Figure 1 [33]).

Figure 1. Architecture of an application developed using Xamarin.iOS (left) and Xamarin.Android (right) [33]. AOT: ahead of time; API: application programming interface.

Utilizing Xamarin, the implementation of the Negami app’s entire business logic and user interface could be shared across platforms. The development followed the Model-View-View-Model design pattern and applied other best practices such as the use of an inversion of control container for dependency injection, commands, and event aggregators. Additionally, the app’s functionality was verified using both unit tests and manual testing. To facilitate the data collection process, a data synchronization feature was implemented that regularly syncs the local SQLite database (which enables the app’s offline usage) containing metadata about each patient and app settings with a remote Microsoft SQL (Structured Query Language) database utilizing the Dotmim.Sync library. As the actual data for each completed exercise session were too large to be stored efficiently in a database, a file-based synchronization was implemented using the Azure Blob Storage (Microsoft Corp) service.

Unlike the business logic and UI elements, the AR aspects of the app had to be implemented in a platform-specific way because Xamarin provides no abstraction layer for these APIs. The implementation was only completed for the iOS platform. For this, the Apple ARKit platform [34] was used (through native bindings provided by Xamarin.iOS) to implement all AR-related logic, such as world tracking, placing and rendering of the objects in 3D space, and retrieving the current screen position of the bird. For Android, the Google ARCore library was utilized in the future to implement the same logic.

Design

Since the target group consists mainly of elderly people, the design of the app was tailored to their needs to facilitate the most positive user experience. The app was intended to reduce the distance and strangeness that this user group often feels from digital devices. We did not want it to feel like a monotonous, mandatory clinical treatment. Instead, we wanted patients to playfully find the motivation to work on their recovery.

A further aim was to ensure that patients—after discharge from inpatient treatment—can safely use the app to practice on their own or with family members to make further progress. This is achieved through short distances in operation and simple instructions and assistance. The same applies to large control buttons that are easy to reach even with shaky or stiff fingers. We also took into account that patients with spatial neglect ignore information on the contralesional side. For example, this was considered when placing and labeling the exercise buttons. Simple letters like “A” and “B” presented on the right tablet side were used instead of complex icons or text. Moreover, we took into consideration that patients with spatial neglect are often hemiparetic. Instead of holding the tablet from the right and left with both hands, hemiparetic patients can hold it with the unaffected hand from the bottom/center. However, if the tablet is too heavy for the patient to hold in this single-handed position, a loop with Velcro can be attached to the patient’s wrist, which, in turn, is attached to the middle of the back of the tablet to help the patient carry the weight.

In addition to the patients, there is another user group: the therapists. This group of users has different requirements for the app than the patients. Since therapists are often responsible for several patients concurrently, they need to treat patients as efficiently as possible. For this purpose, the app offers the therapist the possibility to easily adjust the corresponding performance level of each patient to their current treatment status. The development of each patient's skills is easily visible to the therapist. Further, the app offers the possibility to create a plan for independent therapy at home. The patient's progress thus is recorded and can be viewed by the therapist at any time, allowing them to make the exercises more difficult or easier depending on the patient's level of ability.

Tasks Provided by the Negami App

During the typical use of Negami, the participant sits in a chair or a wheelchair. If possible, the tablet is held with both the left and right hands. If the participant is hemiparetic, the tablet can be held from below with one hand positioned in the middle. Using arm movements in combination with trunk rotations, the tablet can then be moved to the left or right through real space (Figure 2). The participants perform 2 different tasks in succession.

Figure 2. Patient performing tasks of the Negami app. A virtual element (origami bird) was added (augmented) to the video stream of the surroundings produced by the camera of a tablet.

The patient’s first task (Task A: “follow the bird”) is to follow the virtual origami bird through real space. The patient's straight-ahead eye/head/body orientation is set as 0. Beginning at set point 0, the bird flies with random sinusoidal movements toward one side of space (in neglect patients toward the contralesional, neglected side). While doing so, the patient sees an orange circle in the center of the screen (Figure 3). The bird should be held in this circle while performing the task. As soon as the patient fails to successfully follow the bird and keep it in the circle during its flight, they are provided with an additional orientation aid. A blue compass needle appears showing the patient in which direction the bird is located (see Figure 3). The task is successfully completed as soon as the bird has finished its trajectory and the bird has been positioned centrally in the orange circle. During the performance, the patient receives auditory feedback. Every time the patient manages to get the bird into the orange circle, a short, bright tone is presented. If the patient manages to keep the bird continuously in the orange circle, the tone is presented every 2 seconds. When the task has been successfully completed, the patient receives feedback auditorily again through a different tone and visually through the change of the color of the circle to green.

At the end of a successful trial, the options “change angle” and “lead to point 0” are provided on the screen. If the patient/therapist wants to repeat the task, they can select “lead to point 0,” and if the difficulty of the task must be varied, the patient/therapist can select “change angle.” The patient/therapist has 3 different difficulty levels to choose from: easy, medium, or difficult (Table 1). The difficulty levels are saved as templates.

Figure 3. Task A: “follow the bird." The patient has the task to follow the flying origami bird and to keep the bird within the orange/blue circle (A). If the task is successfully solved the circle turns green (B).
Table 1. Difficulty levels provided with Task A (“follow the bird”)a.
LevelMaximum distance of the bird’s pathSpeed of bird movementAmplitude of bird movement
Easyup to ±35°2.3°/secb6.8°
Mediumup to ±55°4.1°/sec8.0°
Difficultup to ±85°8.1°/sec11.3°

aAngular degrees are participants’ space coordinates, starting from their straight-ahead eye/head/body orientation, which is set as 0.

bsec: second.

In the second task of the Negami app (Task B: “find the bird”), the therapist hides the virtual origami bird in the room surrounding the patient. For this purpose, the patient's straight-ahead eye/head/body orientation again is set as 0. Starting from this location, the therapist hides the bird somewhere on the left/right side of the patient without the patient seeing it. The best way to do this is for the therapist to step behind the patient and hold the tablet above the patient's head in such a way that they cannot see where exactly the tablet is pointing. The area within the bird that can be hidden by the therapist is predefined by the app depending on the chosen difficulty level of the task (Table 2). Once the therapist has positioned the bird in space by clicking on the center of the screen, they return the tablet to the patient's hand. The patient is then instructed to find the bird, and once found, to position it centrally into the orange circle. No time limit is given. Should the patient show difficulties in finding the bird, it is possible to provide the patient with orientation assistance by turning on the blue compass needle (Figure 4). The task is successfully solved when the patient finds the bird and places it in the center of the circle. As with Task A, the circle then turns green (Figure 3), and the patient hears a bright tone signaling the successfully solved task. After successful completion of the task, options “change angle” and “lead to point 0” are provided (see Figure 3) so that the task can be repeated or changed in difficulty.

Table 2. Difficulty levels provided with Task B (“find the bird”)a.
LevelExtension of the area (along the horizontal dimension of the surroundings) within which the bird can be hidden
Easy0° to −40° (or 0° to +40°)
Medium−40° to −75° (or +40° to +75°)
Difficult−75° to −90° (or +75° to +90°)

aAngular degrees are the participants’ space coordinates, starting from their straight-ahead eye/head/body orientation, which is set as 0. Negative values indicate positions to the participant’s left, while positive values indicate positions to the participant’s right.

Figure 4. Task B: “find the bird." The patient has to search for the bird that has been hidden by the therapist somewhere in the surrounding room (in this image, at the corner located at the foot of the stairs) and has to transfer it into the orange/blue circle.

Procedure

All neurological patients received training with Negami in their room in the ward of the respective rehabilitation facility. The healthy participants also received the training indoors for comparability. The healthy participants had 1 training session with Negami that lasted 20 to 25 minutes. They completed Task A in the first half and Task B in the second half, each at the highest difficulty level. If patients were not able to use both hands to hold the tablet due to hemiparesis, they were instructed to hold the tablet with the unimpaired hand only by grasping the tablet in the middle from the top or bottom. A video example of a patient working with Negami is shown in Multimedia Appendix 2. To assess feasibility, usability, and game experience, the System Usability Scale (SUS) [35], Simulator Sickness Questionnaire (SSQ) [36], and Perception of Game Training Questionnaire (PGTQ) [37] were administered to the participants directly after the training session.

To assess usability, the participants answered the 10 questions of the SUS through a 5-point Likert scale ranging from 1 (fully disagree) to 5 (fully agree). The mean score was calculated for each participant across all questions. To capture the presence of side effects of simulation technologies, the SSQ was applied. For this purpose, the presence of a series of symptoms grouped under “cybersickness” was tested by a 4-point Likert scale ranging from 0 to 3 (not at all, slightly, moderately, and strongly). The following 15 symptoms were recorded in total: general discomfort, fatigue, headache, eyestrain, difficulty focusing, increased salivation, sweating, nausea, difficulty concentrating, fullness of the head, blurred vision, dizziness, vertigo, stomach awareness, and burping. Again, the mean score was calculated across all items for each patient. Finally, the patients were asked via the PGTQ, which uses a 7-point Likert scale ranging from 1 (fully disagree) to 7 (fully agree), about how motivating, frustrating, challenging, and enjoyable they found the training with Negami to be. Since each of the 4 questions covers a different aspect of the perception of Negami, no mean value was calculated.

The patients with spatial neglect received a 2-week application with 5 training sessions per week. Each session lasted 20 to 25 minutes. As the healthy participants, they completed Task A in the first half and Task B in the second half. The tasks were always initiated for the patients with the easiest difficulty level. The criterion for success in advancing to the next difficulty level was that the task was successfully completed 3 times in a row. A return to a lower difficulty level was indicated when the task was not solved 2 times in a row. Upon completion of the treatment, subjective reports about the use of the Negami app were collected from the patients by using the following three questions: (1) How satisfied were you with the treatment? (2) How motivating did you find the treatment? (3) How much did you enjoy the treatment? The answer options varied from 1 to 6, with 1 being the best result. The patients were further asked whether they would recommend the treatment to others (yes/no) and in an open response format, whether they had any suggestions for improvement.


Healthy Participants

The results of the healthy elderly participants from the SUS on usability resulted in a mean value of 4.3 (SD 0.6), indicating that the usability of the Negami app was rated on average as high to very high. Moreover, there were hardly any side effects detected, as assessed by the SSQ and indicated by a mean score of 0.8 (SD 1.32). More specifically, only 1 patient (10%) reported mild symptoms of fatigue, headache, difficulty focusing, and blurred vision. In addition, 2 (20%) other participants reported mild problems with strained eyes and focusing. None reported moderate or strong side effects. Finally, the PGTQ captured the dimensions of motivation, frustration, challenge, and entertainment during training with Negami. The healthy participants’ evaluations on these dimensions are shown in Figure 5. The participants found Negami to be motivating (mean 6.1, SD 0.7) and highly entertaining (mean 6.5, SD 0.9). As indicated by a mean score of 1.5 (SD 0.7), the training was not perceived as frustrating. However, some participants found the training challenging, while others did not (mean 4.1, SD 1.7).

Figure 5. Percentage of healthy, older adults (n=10) evaluating training with the Negami app on the dimensions “motivation,” “frustration,” “challenge,” and “entertainment,” on a 7-point scale (1 = "I fully disagree"; 7 = "I fully agree").

Neglect Patients

An analysis of the reports by the patients regarding the use of the app revealed that all 10 patients recommended treatment with Negami. The patients’ evaluation of the app treatment on the dimensions of fun, satisfaction, and motivation are shown in Figure 6. The results demonstrate that all patients evaluated the app treatment on all 3 dimensions with the best 2 scores. Only 1 (10%) of the patients chose a score of 4 on the dimension of satisfaction. Consistent with these very positive ratings by the patients with spatial neglect, the Spearman rank correlation coefficients between the dimensions of satisfaction, motivation, and fun and the severity of spatial neglect (expressed as the mean value of the CoC from the 2 cancellation tasks) were not significant (satisfaction: r=.21, motivation: r=.57, fun: r=.42; all P≥.086). Regarding suggestions for improvement in the open response format, 3 (30%) of the 10 patients with hemiparetic neglect reported that they had trouble holding the tablet due to the weight. They had to hold the tablet with 1 hand instead of both. One (10%) patient noted that older patients might be digitally overwhelmed with the treatment. All other patients had no suggestions for improvement.

Figure 6. Percentage of stroke patients (n=10) with spatial neglect evaluating training with the Negami app on the dimensions “satisfaction,” “motivation,” and “fun,” on a 6-point scale (1 is the best score; 6 is the lowest score).

Principal Findings

This study aimed to develop and assess the usability, side effects, and game experience of a new rehabilitation app (Negami) for patients with spatial neglect. By using AR, the app was designed to playfully encourage patients to actively explore space by turning their gaze, head, and trunk. Our investigation showed that Negami was perceived as motivating and entertaining by the healthy elderly controls. Training at the highest defined difficulty level was perceived as differently challenging but not as frustrating. The survey further revealed high usability and no to very low side effects such as headache, strained eyes, or blurred vision. Additionally, the second group evaluated in this study (ie, the patients experiencing spatial neglect after a stroke) consistently evaluated the app positively on the dimensions of motivation, satisfaction, and fun. Furthermore, we observed that the app and the tablet could also be used by patients with hemiparesis (by holding the tablet with the unimpaired hand only, grasping the tablet in the middle from the top or bottom). However, 3 (30%) of the 10 patients with hemiparesis complained about the heaviness of the tablet, so the use of even lighter devices seems beneficial.

Gamification and the Negami App

These positive results give us reason to believe that the new app could become a very useful tool in the treatment of patients with spatial neglect. Ogourtsova and colleagues [38] have shown that patient expectation of performance is the most important factor for acceptance of the use of VR technologies in rehabilitation. Therefore, when defining the 3 different difficulty levels of training with Negami, our focus was on ensuring that the easiest task could be solved relatively quickly even for patients with more severe neurological impairment. This could also be the reason why Negami was so well received by the tested patients who had a stroke. The use of different difficulty levels, as well as auditory and visual feedback on performance during training, are elements that supported the gamification of the Negami app. Gamification for the purpose of rehabilitation has been shown to have positive effects on the patients’ engagement and motivation for their treatment [39]. Another reason for the positive feedback from both patients and healthy participants could be due to our app design in that we paid special attention to the age group, as 75% of patients who have had a stroke are aged over 65 years [40] and are typically not “digital natives” [41]. In implementing short distances in operation, simple instructions, and assistance, we aimed to ensure that this age group feels comfortable in dealing with an electronic device such as a tablet when using the Negami app.

AR Versus VR Techniques and Telerehabilitation

A third aspect that might have contributed to the positive ratings by the 2 groups tested could be the choice of the AR over VR technique. It is a frequent observation that the use of VR may evoke symptoms that are summarized under the term “cybersickness” [42]. These include symptoms such as drowsiness, headache, balance and coordination problems, nausea, and blurred vision. We observed that such symptoms were absent during the training with Negami. The reason for this could be that the risk of sensory mismatch concerning visual, vestibular, and proprioceptive information, which is considered one of the causes of cybersickness [43], is reduced to a maximum—if not completely avoided—when working with AR. The AR-based Negami app prompts people to move in the real world by actually moving their heads and bodies. The use of the tablet merely adds a virtual object to the real world. In this respect, manipulation is kept to a minimum.

Negami can also offer an advantage over VR games in terms of cost. The development of a VR system is often associated with high costs due to the need for dedicated hardware [42]. Since Negami is available online (see the Negami website), it can be easily downloaded to any available tablet or smartphone. This provides wide availability and ensures that additional costs are avoided. The use of already existing technical hardware also offers immense opportunities regarding remote/telerehabilitation. This is important because, in one-third of patients experiencing spatial neglect in the acute phase of the stroke, the disorder becomes chronic [44]. During their inpatient stay, patients are well cared for in terms of their recovery, but what happens after discharge from inpatient rehabilitation? An answer to that could be remote/telerehabilitation. Easy-to-understand instructions for the tasks in Negami can allow patients to practice independently from home. With online data storage, the therapist has external access to patients’ treatment data and can monitor them remotely.

Limitations

One limitation of this study is the small sample size. We only could include 10 elderly neurologically healthy individuals and 10 patients experiencing spatial neglect and left-sided hemiparesis after a stroke in the right brain hemisphere. Therefore, testing the Negami app with a larger number of participants in a future study will be useful. It would also be desirable to evaluate the use of the app on the part of the therapists in the future. It should be further mentioned that Negami thus far can only be used on Apple devices. However, it is being further developed with the aim of being able to use on Android devices as well. The further development of all smart devices with different operating systems would also ensure the use of lighter devices, as 3 (30%) of the patients complained about the weight of the tablet. At this point, it should also be mentioned that very severely affected patients, who are not yet able to hold a tablet in the acute phase after a stroke, may not benefit from Negami initially, and other therapeutic approaches should be considered first for them.

Conclusion

AR represents a promising extension to the traditional “visual exploration training” for spatial neglect. The natural interaction with the environment during playful tasks appears to markedly increase patient motivation. An advantage of AR, as implemented in Negami, over immersive VR-based treatments may be that it minimizes the risk of sensory mismatch concerning visual, vestibular, and proprioceptive information, which is considered one of the causes of cybersickness.

Acknowledgments

This work was supported by Hector Stiftungen, Weinheim (M2107).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Demographic and clinical data of all 10 patients with right brain damage and spatial neglect.

DOCX File , 19 KB

Multimedia Appendix 2

Patient performing Task B on Negami.

MP4 File (MP4 Video), 21397 KB

  1. Buxbaum LJ, Ferraro MK, Veramonti T, Farne A, Whyte J, Ladavas E, et al. Hemispatial neglect: Subtypes, neuroanatomy, and disability. Neurology 2004 Mar 09;62(5):749-756. [CrossRef] [Medline]
  2. Becker E, Karnath HO. Incidence of visual extinction after left versus right hemisphere stroke. Stroke 2007 Dec;38(12):3172-3174. [CrossRef]
  3. Karnath HO, Rorden C. The anatomy of spatial neglect. Neuropsychologia 2012 May;50(6):1010-1017 [FREE Full text] [CrossRef] [Medline]
  4. Fruhmann Berger M, Pross RD, Ilg U, Karnath HO. Deviation of eyes and head in acute cerebral stroke. BMC Neurol 2006 Jun 26;6(1):23 [FREE Full text] [CrossRef] [Medline]
  5. Becker E, Karnath HO. Neuroimaging of eye position reveals spatial neglect. Brain 2010 Mar 15;133(Pt 3):909-914. [CrossRef] [Medline]
  6. Pizzamiglio L, Antonucci G, Judica A, Montenero P, Razzano C, Zoccolotti P. Cognitive rehabilitation of the hemineglect disorder in chronic patients with unilateral right brain damage. J Clin Exp Neuropsychol 1992 Nov 04;14(6):901-923. [CrossRef] [Medline]
  7. Antonucci G, Guariglia C, Judica A, Magnotti L, Paolucci S, Pizzamiglio L, et al. Effectiveness of neglect rehabilitation in a randomized group study. J Clin Exp Neuropsychol 1995 May;17(3):383-389. [CrossRef] [Medline]
  8. Kerkhoff G, Münßinger U, Eberle-strauss G, Stögerer E. Rehabilitation of hemianopic alexia in patients with postgeniculate visual field disorders. Neuropsychol Rehabil 1992 Jan;2(1):21-42. [CrossRef]
  9. van Kessel ME, Geurts ACH, Brouwer WH, Fasotti L. Visual scanning training for neglect after stroke with and without a computerized lane tracking dual task. Front Hum Neurosci 2013;7:358 [FREE Full text] [CrossRef] [Medline]
  10. Maclean N, Pound P, Wolfe C, Rudd A. Qualitative analysis of stroke patients' motivation for rehabilitation. BMJ 2000;321(7268):1051-1054. [CrossRef] [Medline]
  11. Darina D, Christo D, Gennady A, Galia A. Gamification in education: a systematic mapping study. J Educ Techno Soc 2015;18(3):75-88.
  12. Sokolov AA, Collignon A, Bieler-Aeschlimann M. Serious video games and virtual reality for prevention and neurorehabilitation of cognitive decline because of aging and neurodegeneration. Curr Opin Neurol 2020 Apr;33(2):239-248 [FREE Full text] [CrossRef] [Medline]
  13. Popović MD, Kostić MD, Rodić SZ, Konstantinović LM. Feedback-mediated upper extremities exercise: increasing patient motivation in poststroke rehabilitation. Biomed Res Int 2014;2014:520374 [FREE Full text] [CrossRef] [Medline]
  14. Fordell H, Bodin K, Eklund A, Malm J. RehAtt - scanning training for neglect enhanced by multi-sensory stimulation in virtual reality. Top Stroke Rehabil 2016 Jun;23(3):191-199. [CrossRef] [Medline]
  15. Kim YM, Chun MH, Yun GJ, Song YJ, Young HE. The effect of virtual reality training on unilateral spatial neglect in stroke patients. Ann Rehabil Med 2011 Jun;35(3):309-315 [FREE Full text] [CrossRef] [Medline]
  16. Navarro M, Lloréns R, Noé E, Ferri J, Alcañiz M. Validation of a low-cost virtual reality system for training street-crossing. A comparative study in healthy, neglected and non-neglected stroke individuals. Neuropsychol Rehabil 2013 Aug;23(4):597-618. [CrossRef] [Medline]
  17. Huygelier H, Schraepen B, Lafosse C, Vaes N, Schillebeeckx F, Michiels K, et al. An immersive virtual reality game to train spatial attention orientation after stroke: A feasibility study. Appl Neuropsychol Adult 2022 Sep 18;29(5):915-935. [CrossRef] [Medline]
  18. Knobel SEJ, Kaufmann BC, Gerber SM, Urwyler P, Cazzoli D, Müri RM, et al. Development of a search task using immersive virtual reality: proof-of-concept study. JMIR Serious Games 2021 Jul 02;9(3):e29182 [FREE Full text] [CrossRef] [Medline]
  19. Morse H, Biggart L, Pomeroy V, Rossit S. Exploring perspectives from stroke survivors, carers and clinicians on virtual reality as a precursor to using telerehabilitation for spatial neglect post-stroke. Neuropsychol Rehabil 2022 Jun;32(5):707-731. [CrossRef] [Medline]
  20. Yasuda K, Muroi D, Ohira M, Iwata H. Validation of an immersive virtual reality system for training near and far space neglect in individuals with stroke: a pilot study. Top Stroke Rehabil 2017 Oct 12;24(7):533-538. [CrossRef] [Medline]
  21. Numao T, Amimoto K, Shimada T. Examination and treatment of unilateral spatial neglect using virtual reality in three-dimensional space. Neurocase 2021 Dec 20;27(6):447-451. [CrossRef] [Medline]
  22. Plechatá A, Nekovářová T, Fajnerová I. What is the future for immersive virtual reality in memory rehabilitation? A systematic review. NRE 2021 Jun 16;48(4):389-412. [CrossRef]
  23. Bakker M, Boonstra N, Nijboer T, Holstege M, Achterberg W, Chavannes N. The design choices for the development of an augmented reality game for people with visuospatial neglect. Clinical eHealth 2020;3:82-88. [CrossRef]
  24. Wiart L, Côme ABS, Debelleix X, Petit H, Joseph PA, Mazaux JM, et al. Unilateral neglect syndrome rehabilitation by trunk rotation and scanning training. Arch Phys Med Rehabil 1997 Apr;78(4):424-429. [CrossRef]
  25. Weintraub S, Mesulam MM. Right cerebral dominance in spatial attention. Further evidence based on ipsilateral neglect. Arch Neurol 1987 Jun 01;44(6):621-625. [CrossRef] [Medline]
  26. Gauthier L, Dehaut F, Joanette Y. The Bells Test: A quantitative and qualitative test for visual neglect. Int J Clin Neuropsychol 1989;11(2):49-54. [CrossRef]
  27. Johannsen L, Karnath HO. How efficient is a simple copying task to diagnose spatial neglect in its chronic phase? J Clin Exp Neuropsychol 2004 Apr 09;26(2):251-256. [CrossRef] [Medline]
  28. McIntosh RD, Schindler I, Birchall D, Milner AD. Weights and measures: a new look at bisection behaviour in neglect. Brain Res Cogn Brain Res 2005 Dec;25(3):833-850. [CrossRef] [Medline]
  29. Rorden C, Karnath HO. A simple measure of neglect severity. Neuropsychologia 2010 Jul;48(9):2758-2763 [FREE Full text] [CrossRef] [Medline]
  30. McIntosh RD, Ietswaart M, Milner AD. Weight and see: Line bisection in neglect reliably measures the allocation of attention, but not the perception of length. Neuropsychologia 2017 Nov;106:146-158 [FREE Full text] [CrossRef] [Medline]
  31. Negami.   URL: https://www.negami.de/en/ [accessed 2023-02-04]
  32. What is Xamarin? Microsoft.   URL: https://docs.microsoft.com/en-us/xamarin/get-started/what-is-xamarin [accessed 2023-02-02]
  33. iOS app architecture. Microsoft.   URL: https://learn.microsoft.com/en-us/xamarin/ios/internals/architecture [accessed 2023-02-02]
  34. More to explore with ARKit 6. Apple Developer.   URL: https://developer.apple.com/augmented-reality/arkit/ [accessed 2023-02-02]
  35. Brooke J. Sus: A 'Quick and Dirty' Usability Scale. Boca Raton, FL: CRC Press; 1996.
  36. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator Sickness Questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 1993 Jul;3(3):203-220. [CrossRef]
  37. Boot WR, Champion M, Blakely DP, Wright T, Souders DJ, Charness N. Video games as a means to reduce age-related cognitive decline: attitudes, compliance, and effectiveness. Front Psychol 2013;4:31 [FREE Full text] [CrossRef] [Medline]
  38. Ogourtsova T, Souza SW, Archambault PS, Lamontagne A. Virtual reality treatment and assessments for post-stroke unilateral spatial neglect: A systematic literature review. Neuropsychol Rehabil 2017 Apr;27(3):409-454. [CrossRef] [Medline]
  39. Mubin O, Alnajjar F, Jishtu N, Alsinglawi B, Al Mahmud A. Exoskeletons with virtual reality, augmented reality, and gamification for stroke patients' rehabilitation: systematic review. JMIR Rehabil Assist Technol 2019 Sep 08;6(2):e12010 [FREE Full text] [CrossRef] [Medline]
  40. Rothwell P, Coull A, Silver L, Fairhead J, Giles M, Lovelock C, et al. Population-based study of event-rate, incidence, case fatality, and mortality for all acute vascular events in all arterial territories (Oxford Vascular Study). Lancet 2005 Nov;366(9499):1773-1783. [CrossRef]
  41. Prensky M. Digital natives, digital immigrants part 2: do they really think differently? Horiz 2001 Nov;9(6):1-6. [CrossRef]
  42. Pedroli E, Serino S, Cipresso P, Pallavicini F, Riva G. Assessment and rehabilitation of neglect using virtual reality: a systematic review. Front Behav Neurosci 2015 Aug 25;9:226 [FREE Full text] [CrossRef] [Medline]
  43. Caserman P, Garcia-Agundez A, Gámez Zerban A, Göbel S. Cybersickness in current-generation virtual reality head-mounted displays: systematic review and outlook. Virtual Real 2021 Apr 10;25(4):1153-1170. [CrossRef]
  44. Karnath HO, Rennig J, Johannsen L, Rorden C. The anatomy underlying acute versus chronic spatial neglect: a longitudinal study. Brain 2011 Mar 13;134(Pt 3):903-912 [FREE Full text] [CrossRef] [Medline]


API: application programming interface
AR: augmented reality
ARM: Advanced Reduced Instruction Set Computer Machine
CoC: Center of Cancellation PGTQ: Perception of Game Training Questionnaire
EWB: end point weightings bias
IL: Intermediate Language
SQL: Structured Query Language
SSQ: Simulator Sickness Questionnaire
SUS: System Usability Scale
UI: user interface
VR: virtual reality
XAML: Extensible Application Markup Language


Edited by G Eysenbach, N Zary; submitted 30.06.22; peer-reviewed by B Puladi, M Kapsetaki, J Heo, D Costa; comments to author 18.11.22; revised version received 06.12.22; accepted 11.01.23; published 27.02.23

Copyright

©Britta Stammler, Kathrin Flammer, Thomas Schuster, Marian Lambert, Hans-Otto Karnath. Originally published in JMIR Serious Games (https://games.jmir.org), 27.02.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.