This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.
Serious gaming has increasingly gained attention as a potential new component in clinical practice. Specifically, its use in the rehabilitation of motor dysfunctions has been intensively researched during the past three decades.
The aim of this scoping review was to evaluate the current role of serious games in upper extremity rehabilitation, and to identify common methods and practice as well as technology patterns. This objective was approached via the exploration of published research efforts over time.
The literature search, using the PubMed and Scopus databases, included articles published from 1999 to 2019. The eligibility criteria were (i) any form of game-based arm rehabilitation; (ii) published in a peer-reviewed journal or conference; (iii) introduce a game in an electronic format; (iv) published in English; and (v) not a review, meta-analysis, or conference abstract. The search strategy identified 169 relevant articles.
The results indicated an increasing research trend in the domain of serious gaming deployment in upper extremity rehabilitation. Furthermore, differences regarding the number of publications and the game approach were noted between studies that used commercial devices in their rehabilitation systems and those that proposed a custom-made robotic arm, glove, or other devices for the connection and interaction with the game platform. A particularly relevant observation concerns the evaluation of the introduced systems. Although one-third of the studies evaluated their implementations with patients, in most cases, there is the need for a larger number of participants and better testing of the rehabilitation scheme efficiency over time. Most of the studies that included some form of assessment for the introduced rehabilitation game mentioned user experience as one of the factors considered for evaluation of the system. Besides user experience assessment, the most common evaluation method involving patients was the use of standard medical tests. Finally, a few studies attempted to extract game features to introduce quantitative measurements for the evaluation of patient improvement.
This paper presents an overview of a significant research topic and highlights the current state of the field. Despite extensive attempts for the development of gamified rehabilitation systems, there is no definite answer as to whether a serious game is a favorable means for upper extremity functionality improvement; however, this certainly constitutes a supplementary means for motivation. The development of a unified performance quantification framework and more extensive experiments could generate richer evidence and contribute toward this direction.
Motor rehabilitation in various parts of the body such as the upper or lower limbs aims to help patients restore dysfunctions that affect their mobility. In this scoping review, we focus on motor disabilities related to the upper extremities. The motivation behind this review was first introduced within one of our group’s research projects related to upper limb rehabilitation, termed “Modern Interface Platform for Motor Control and Learning on People With Motor Disorders” [
Therapists have developed several clinical methods to indicate motor ability, such as range of motion (ROM) or range of force. In addition, specialized evaluation tests such as Fugl-Meyer Motor Function Assessment (FMA), Action Research Arm Test (ARAT), and Melbourne Assessment of Unilateral Upper Limb Function (MAUULF) aim to estimate the improvement of a patient’s motion condition. The usual rehabilitation scheme consists of repeated motion exercises for a specific body part, with the aim of restoring ability as close to the normal condition as possible.
The idea to introduce gamification to the therapeutic protocol of upper limb rehabilitation was born as a means to motivate patients during the rehabilitation schemes but also represents a new method for monitoring the upper limb motion for further analysis. The first attempts of the introduction of gamification in upper limb rehabilitation appeared in 1999 by a team at Rutgers University [
These rapid technological developments led to more elaborate devices regarding motion capture, challenging researchers in this field to investigate this type of rehabilitation.
Over the last few decades, there has been an increasing amount of studies regarding the enhancement of rehabilitation with the introduction of new technologies. A systematic review on the implementation of serious games and wearable technology in rehabilitation practices for patients recovering from traumatic bone and soft tissue injuries was published by Meijer et al [
This effort will contribute to the detection of gaps or limitations in this area, and may lead to new research paths and ideas.
The rest of the paper is organized as follows. The Methods section depicts the procedure that was followed regarding the literature search, data management, and eligibility criteria of this review. The Results section presents the statistical results, including figures, after reviewing the included studies. Finally, the Discussion section comments on the results and delineates possible limitations of this study, along with highlighting the importance of this review for further development of this research area.
In this scoping review, we followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) [
This review included articles published from 1999 to June 2019. The PubMed and Scopus databases were used for the literature search. The keywords utilized in the literature search were: “rehabilitation,” “hand,” “upper limb,” “upper extremity,” “upper arm,” “game,” “serious gaming,” and “serious game,” which were investigated in titles and abstracts of articles published in the English language. The following search query was used: rehabilitation AND (hand OR upper limb OR upper-limb OR upper extremity OR upper-extremity OR upper arm OR upper-arm) AND (game OR serious gaming OR serious game). Subsequently, duplicated articles were removed, and the remaining studies were screened for eligibility.
Two individual researchers (EK and IL) conducted the literature search and the removal of duplicates, and one author (IL) screened the titles and abstracts for eligibility under advisement by IC. The remaining studies were reviewed by EK, IL, and DF, guided by a set of inclusion and exclusion criteria to extract information from the selected articles. The extracted information followed a structure defined by IC, EK, and IL, as follows: (i) year of publication, (ii) purpose of the study, (iii) part of the upper limb for rehabilitation, (iv) sensors used, (v) disease that led to the patients’ condition, (vi) game type, (vii) game scenario, (viii) game target, (ix) clinic- or home-based application, (x) supervised or unsupervised, (xi) software used for creation of the rehabilitation game, (xii) hardware development, (xiii) system limitation, (xiv) use of a pilot study or not, (xv) number of patients in the pilot study, (xvi) evaluation methodology, and (xvii) features extracted from the game.
The literature search was conducted in July 2019 with the requirements described above, and a total of 682 studies were identified, including 151 from the PubMed database and 531 from the Scopus database. After removal of duplicates, 557 studies were screened with the criteria set, resulting in a total of 244 articles. In addition, 75 studies were excluded due to meeting one or more exclusion criteria, and 169 studies were finally included in the scoping review about upper limb rehabilitation based on serious gaming technology. The most common reasons for a study to be excluded were the absence of a serious game from the rehabilitation procedure and the development of a system that did not focus on upper extremity rehabilitation.
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) [
Inclusion criteria for eligibility of the selected articles were: (i) any form of game-based arm rehabilitation (interactive computer-based game, mobile/table app, or platform game software) and (ii) published in a peer-reviewed or conference journal. Exclusion criteria for this review were: (i) a nonserious game–based scheme of rehabilitation with a sensor (only using a sensor or robotic arm, without the accompanying serious game); (ii) a trial of a serious game rehabilitation scheme without any technical description of the game; (iii) medical article based on a health care professional’s perspective for arm rehabilitation without any technical description of a game; (iv) not published in English; or (v) a review, meta-analysis, or conference abstract.
Based on the extracted information, we created 11 factors of categorization for the data. The extracted information is presented in the Data Management subsection above. The factors were determined based on the combination of the extracted information. All analysis factors were categorical, except for
The results extracted from the included studies are presented according to the following structure: (i) statistics depicted in charts, (ii) descriptive statistics that provide information regarding the tendencies of research efforts, and (iii) conclusions extracted not only from the statistics but also from the general picture formed from the analysis of all included studies.
Based on our literature search, the first study was published in 1999; however, only a few relevant papers were published in this field up to 2006. In 2007, researchers showed greater interest in upper extremity rehabilitation using new technologies based on serious games, and the number of publications has continued to rise up to the present day.
Distribution of the publications over time. The “Commercial Sensors” category refers to studies using commercial sensors or any combination of commercial devices for the rehabilitation scenario, and the “Hardware Development” category refers to studies that created any type of robotic arm, glove, or other device for the connection with the game platform.
Factors analyzed in the review.
Extracted information | Description |
Medical condition | Underlying categories for the upper limb motion problems: stroke, general motion deficits, cerebral palsy, hemiparesis, neurological motor deficits, Parkinson disease, burn contractures, brain impairment, general cognitive deficits, shoulder injury, and wrist injury. |
Upper extremity part | The upper extremity part targeted for rehabilitation: upper extremity/limb, fingers, palm, wrist, elbow, forearm, shoulder, and hand muscles. |
Device development | The differentiation between hardware development and commercial sensors. The “commercial sensors” category includes studies that used commercial sensors/devices or any combination of commercial devices for the rehabilitation scenario, whereas the “hardware development” category includes studies that created any type of robotic arm, glove, or other device for the connection with the game platform |
Game type | The type of serious game developed: virtual reality, augmented reality, video game, electronic board game, and mobile health apps. |
Game target | The game scenario. Task completion (specific scenarios regarding the case, such as follow the line, daily activities, create shapes, collect a number of items), time (complete the level of the game in time intervals), score (increase the level score), force (studied the patient’s force in controlling the sensor). |
Sensors | The preferred device for the upper extremity exercise in conjunction with the game. |
Hardware use limitations | Limitations regarding the hardware used in the included studies. |
Supervision level | The rehabilitation scheme takes place at home or in a clinic while the patient is supervised by an expert or not. |
System testing with users: pilots and trials | The testing part of the proposed scenario. Some studies conducted a trial or a pilot trial for testing the rehabilitation system, while others did not. In the latter studies, the trials were conducted with patients, control subjects, or both. |
System evaluation | In cases in which the proposed system was tested, there were several means of evaluation: questionnaires, interviews, clinical tests, and scores before and after the rehabilitation scheme. |
Extracted game features | The extracted characteristics, using the game, for further analysis: time, game performance, kinematic indicators, range of motion. |
To specify the
Most of the included studies (48.5%, 83/169) referred to rehabilitation of the upper extremity/limb in general, whereas others (40.2%, 68/169) focused on a specific part such as the fingers, palm, wrist, elbow, forearm, and shoulder.
The results of our search indicated that researchers in this field are showing more interest in commercial sensors that continue to evolve. Less interest is placed on the development of new devices designed for motion of a specific upper extremity part. This may be due to the more costly and time-consuming development of such specific devices. Studies on hardware development accounted for 37.3% (63/169) of the total studies, whereas there was double the number of studies related commercial sensors, representing 62.7% (106/169 studies) of the total. Despite the fewer attempts to address hardware development, there seems to be continuous interest in this research area. However, it is evident that there are more fluctuations of publications over time in the case of hardware development due to the difficulty of the task (ie, the time and knowledge required to create a device), whereas commercial sensor–related studies showed a consistent increasing trend over time (
Between the two
Based on these definitions, the majority of studies included in the review (74.6%, 126/169) approached the rehabilitation problem by developing VR games. This
The next most common
Comparing the two categories of
Comparison between the two
Over the years, technological development has led to increased incorporation of smartphones and tablets in our daily routines. Naturally, researchers have begun testing this new technology in many fields, including upper limb rehabilitation based on serious games by creating mHealth apps. Seven studies [
The
A variety of commercial or noncommercial devices have been proposed for rehabilitation of the upper extremities combined with serious gaming based on the researchers’ ideas and accessible technologies at the time of publication. The most commonly used
With respect to commercial gaming accessories, a few studies focused on individual sensors such as VR headsets [
Additionally, some studies have attempted to create their own tracking system using
Overall, 17.8%(30/169) of the studies included webcams and cameras in their systems. Half of them [
Besides interest in developing robotic devices identified in the hardware development category, some research teams have also focused on robotic devices that are already available on the market. Several studies used haptic devices such as Phantom Omni [
In addition, some studies attempted to either control or monitor patients’ movements using medical devices such as EMG [
Overview of the devices used in the included studies. The green nodes indicate the sensors included in the
Several different ideas of
Despite the use of advanced technology, many studies that used commercial sensor devices mentioned limitations regarding the used hardware. For example, various studies [
Limitations were also mentioned with respect to the
In general, the
All of the included studies described a system that has been developed by the corresponding research team. Some of them included small pilot or limited-range trials, while others were complemented by subsequent studies reporting the results of pilot surveys. It is notable that the most frequent limitation in cases with no pilot study was the absence of clinical trials and the deficient testing of the system. Overall, 33.7%(57/169) of the studies tested their system with healthy subjects and therapists, whereas 27.1%(46/169) did not test the system at all. Although the remaining one-third of the research teams conducted clinical trials with patients, the majority of them included a very small number of patients, and also included healthy subjects in some cases to enlarge the sample. Among the 169 papers included in this scoping review, there were 2291 participants in the pilot and clinical trials. However, from the total number of participants, 689 were patients and 1602 were healthy volunteers, clinicians, therapists, and researchers.
The
Furthermore, 29.6%(50/169) of the studies used metrics regarding functional recovery via standard tests such as FMA, ARAT, and MAUULF; scores such as ROM for the elbow and forearm; and the Jamar strength test for strength of the hands. By using standard tests and scores as evaluation methods, researchers can measure the progress of a patient regarding motion dysfunction before and after the rehabilitation scheme. In addition, several studies tried to extract game features to introduce quantitative measurements for evaluation of patient improvement. Among these studies, 65%(49/75) did not analyze the evaluation methods, referring to them more generally as “data analysis” and providing descriptive statistics or as “monitoring data” in which the sessions were recorded using several sensors. In the next section, we discuss an extended analysis regarding the
Quantitative measurements of the treatment outcome are critical for clinical rehabilitation practice, which constitute an objective method for evaluating the patient’s medical progress. With these measurements, physicians can closely monitor the therapy process and adjust the treatment protocol individually. Among the studies included in this review, only a few described an assessment process of the patient’s recovery status based on extracted features.
Some of these studies [
Game performance was another consistent feature among the studies. In some cases, performance was associated with the score, and in other cases it was associated with task completion of the game target. Several studies [
It is worth noting that several studies monitored other kinematic indicators of the patient’s health condition that are not included in the
ROM and data regarding the angles of the hand during its motion constitute another significant group of features that have been commonly used by researchers and health professionals to quantify therapy progress. Several studies [
Summary of the extracted game features.
Categories, Features | Number of studies | |
|
|
|
|
Time-related | 10 |
|
|
|
|
Score/task completion | 6 |
|
Golden standard comparison | 4 |
|
|
|
|
Hand jerk | 5 |
|
Trajectory curvature | 2 |
|
Fractionation | 2 |
|
Muscle activation | 1 |
|
|
|
|
Range of motion | 7 |
It should be noted that this scoping review does not report every metric for each study, as our purpose was not to elaborate on how each study implemented the assessment of the patient’s improvement but rather to outline and categorize the features extracted from the motion analysis process, excluding metrics of patients’ engagement and motivation. Furthermore, in this attempt of feature extraction categorization, no distinction was made between studies that evaluated these features with a group of patients and those that conducted trials with healthy subjects.
With this scoping review, we aimed to explore the trends associated with deploying technologies for functional rehabilitation of the upper extremities. The results indicate that there has been increasing interest in these applications over time. The rapid evolution of technology contributes to new approaches concerning clinical practice and personalization of therapies. There is currently a wide range of sensors available for capturing motion (eg, Kinect, Leap Motion, IMUs) along with attempts to translate these technologies into an environment projected on computer screens, in VR headsets, or in AR image processing. The capabilities of various sensors related to motion, in conjunction with serious gaming, are used by research teams to develop contemporary systems for both doctors and patients.
One of the main advantages of this study is the overview of the current state of the field of upper extremity rehabilitation using serious games. As part of our research interest, we tried to investigate this topic to better understand the various approaches used to date. Based on this summary, we present a set of characteristics that depict a common direction and provide a complete picture of the sensors and technologies utilized to achieve the therapy purpose in terms of standard clinical practice.
The results indicate increasing research interest in the domain of serious gaming deployment in upper extremity rehabilitation. Based on the descriptive analysis, we can examine different aspects of this field of research. In particular, stroke seems to be a common
With respect to the factor
While reviewing the surveys included in our literature search, it is a safe assumption that computer graphic development in the last few decades has led to generalization of the term “VR.” In 1992, Coates [
With respect to the
Additionally, there is a broad selection of
In this study, we classified the used devices according to the
Summary of devices reported in the literature over time. Groups of sensors used for studies of upper limb rehabilitation are presented in a timeline. The groups include devices from both the
Based on our results, one-third of the introduced systems conducted clinical trials with patients to test their implementation. Although one-third of the included studies constitutes a sufficient number of attempts, in most of these cases, there is a need for a higher number of subjects and better testing of the rehabilitation scheme efficiency over time.
Concerning the
The broadness of the field of upper limb rehabilitation using serious games constitutes a limitation leading to many potentially included studies for this review. There was a significant number of studies, each suggesting different ways to approach the rehabilitation scheme but with poor sources or minimum attempts. Many conference papers have been published over the years introducing thoughts and preliminary results, but with no further analysis and implementation of their idea for rehabilitation. Although our exclusion criteria limited the range of the existing literature to some extent, this review includes several uncompleted attempts. Moreover, since we used specific keywords such as “upper-extremity,” “rehabilitation,” and “serious game” in different combinations, in an attempt to focus on the area of interest, we concede that some surveys in the field may have been excluded. Nevertheless, we are confident that the remaining studies that met all of the inclusion criteria can reflect the state of the field of upper extremity rehabilitation employing serious games, thereby assuring the reliability of our conclusions.
In addition, a limitation of this study is the lack of categorization based on gross motor vs fine motor or testing usability vs testing effectiveness. The many differently structured papers in combination with the heterogeneity in the provided information made such categorization very complicated and led us to the decision not to include these categories. It may be worth analyzing these categories separately to obtain an overview of this field in a different study.
Finally, due to the rapid technological progress, we consider another limitation to be the fact that this review includes studies only published up to June 2019.
Upper extremity motor dysfunction is a common problem that requires rehabilitation. Researchers studying the engagement of patients to the rehabilitation schemes have established several ways to develop more amusing training sets to better motivate patients. Technological progress constitutes an ally of these attempts, allowing for the combination of a traditional rehabilitation routine with serious games. In the last two decades, there has been a significant number of publications regarding upper limb rehabilitation using serious games, which is a field that continues to evolve based on user experience. Our goal regarding this review was to provide a complete overview of the field based on published studies over the years. Overall, this scoping review highlights several facts that point to the usefulness of serious games in rehabilitation in future medical procedures, as well as several weaknesses and challenges that have to be addressed. Despite the numerous attempts for establishing and evaluating game-based rehabilitation systems, more evidence is needed considering such systems not only as a means for patient motivation but also as an actual means for achieving upper extremity functionality improvement. In this vein, despite the challenges in the generalization and comparability of specific game decisions and implementations, it is important to support the efforts for the creation of quantitative measurements of game-based treatment, performance and outcome, and build evidence of its clinical value. In this direction, it would be important to work toward creating a framework for the therapeutic use of such gamified approaches, including the optimal dosage, personalization means, adaptations over time, session performance assessment, and therapeutic outcome. Such a therapeutic framework could enable the synthesis of more solid clinical evidence around game-based treatment, and eventually its incorporation in the clinical routine.
augmented reality
Action Research Arm Test
electromyogram
Fugl-Meyer Motor Function Assessment
inertial measurement unit
Melbourne Assessment of Unilateral Upper Limb Function
mobile health
range of motion
virtual reality
This research has been cofinanced by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship, and Innovation program, under the call RESEARCH – CREATE – INNOVATE (project code: T1EDK-02488).
IC, EK, and IL conceived of the study. All authors participated in the design of the study and drafted the manuscript. Data management was conducted by EK, IL, and DF, and advised by IC. All authors edited the manuscript, and read and approved the final manuscript.
None declared.