Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 05.03.20 in Vol 8, No 1 (2020): Jan-Mar

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13459, first published Jan 22, 2019.

This paper is in the following e-collection/theme issue:

    Review

    Health Education Serious Games Targeting Health Care Providers, Patients, and Public Health Users: Scoping Review

    1Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

    2Department of Health Policy and Management, Johns Hopkins School of Public Health, Baltimore, MD, United States

    3Department of Medical Informatics, Faculty of Medicine, Tehran University of Medical Sciences, Tehran, Iran

    Corresponding Author:

    Mahmood Tara, MD, PhD

    Department of Medical Informatics

    Faculty of Medicine

    Mashhad University of Medical Sciences

    Azadi Square

    Mashhad

    Iran

    Phone: 98 9158061511

    Email: taram@mums.ac.ir


    ABSTRACT

    Background: Serious educational games have shown effectiveness in improving various health outcomes. Previous reviews of health education games have focused on specific diseases, certain medical subjects, fixed target groups, or limited outcomes of interest. Given the recent surge in health game studies, a scoping review of health education games is needed to provide an updated overview of various aspects of such serious games.

    Objective: This study aimed to conduct a scoping review of the design and evaluation of serious educational games for health targeting health care providers, patients, and public (health) users.

    Methods: We identified 2313 studies using a unique combination of keywords in the PubMed and ScienceDirect databases. A total of 161 studies were included in this review after removing duplicates (n=55) and excluding studies not meeting our inclusion criteria (1917 based on title and abstract and 180 after reviewing the full text). The results were stratified based on games targeting health care providers, patients, and public users.

    Results: Most health education games were developed and evaluated in America (82/161, 50.9%) and Europe (64/161, 39.8%), with a considerable number of studies published after 2012. We discovered 58.4% (94/161) of studies aiming to improve knowledge learning and 41.6% (67/161) to enhance skill development. The studies targeted various categories of end users: health care providers (42/161, 26.1%), patients (38/161, 23.6%), public users (75/161, 46.6%), and a mix of users (6/161, 3.7%). Among games targeting patients, only 13% (6/44) targeted a specific disease, whereas a growing majority targeted lifestyle behaviors, social interactions, cognition, and generic health issues (eg, safety and nutrition). Among 101 studies reporting gameplay specifications, the most common gameplay duration was 30 to 45 min. Of the 61 studies reporting game repetition, only 14% (9/61) of the games allowed the users to play the game with unlimited repetitions. From 32 studies that measured follow-up duration after the game intervention, only 1 study reported a 2-year postintervention follow-up. More than 57.7% (93/161) of the games did not have a multidisciplinary team to design, develop, or assess the game.

    Conclusions: Serious games are increasingly used for health education targeting a variety of end users. This study offers an updated scoping review of the studies assessing the value of serious games in improving health education. The results showed a promising trend in diversifying the application of health education games that go beyond a specific medical condition. However, our findings indicate the need for health education game development and adoption in developing countries and the need to focus on multidisciplinary teamwork in designing effective health education games. Furthermore, future health games should expand the duration and repetition of games and increase the length of the follow-up assessments to provide evidence on long-term effectiveness.

    JMIR Serious Games 2020;8(1):e13459

    doi:10.2196/13459

    KEYWORDS



    Introduction

    Background

    Serious games have emerged as a promising educational technique across various domains [1,2]. Previous studies, including a survey study, have identified health care as one of the main targets of educational serious games [3-5]. In contrast to traditional educational techniques, the focus of serious games on health is partly derived from the fact that they provide individuals with a risk-free environment to practice high-stake tasks and experience unpredictable outcomes. Serious games also provided a unique educational platform to increase patient safety and reduce cost, which, in turn, has propelled the rapid development of new health education games [6,7].

    User acceptance is key to the successful impact of educational serious games. Previous studies have assessed various user acceptance challenges of educational games [8,9]. These studies revealed that a wide range of users, including health care providers and medical students, accept serious games as a substantial and useful educational technique [10,11]. These studies also showed that clinical instructors consider serious games as an attractive and engaging educational tool [8,12]. Higher engagement is partly explained by the active learning tasks experienced by the users while interacting with an educational game [13,14].

    Similar to other educational techniques, serious games require goal-relevant design, and their effectiveness should be methodologically evaluated [15]. Designing educational serious games requires multiple stages to ensure the engagement of potential end users in all phases of development, ranging from flowcharts and wireframes to multidimensional design and repeated user experience tests [16]. In addition, to increase the generalizability of educational serious games in improving learning objectives, they need to be rigorously evaluated across different user groups using various methods ranging from user studies to focus groups and clinical trials [17].

    Several review studies have evaluated the design, development, and outcomes of serious games; however, only a few have focused on health education games [7,15,16]. One study conducted a systematic review of educational serious games for medical students and concluded that serious games should be evaluated before use in medical school curricula [17]. Another study conducted a meta-analysis of sex education serious games and concluded that serious games can be used effectively for promoting sexual health [18]. Moreover, another study evaluated the frequency and progression of health serious games across various domains, including clinical training, rehabilitation, and health education [5]. Considering the current rapid development of serious games, an updated scoping review focusing on health education serious games is lacking.

    Study Objectives

    This study offers an updated scoping review of health education games. The study reviews various aspects of the recent developments of educational health games designed for various user groups and provides a comprehensive review of the design characteristics and evaluation of such serious games. The study also addresses the gaps and weaknesses of the recent developments of health education games.


    Methods

    Overall Framework

    The York framework was used to develop the general framework of this study [19]. We followed the following stages to guide the methodology of our search and analysis: (1) identifying the research question; (2) identifying relevant studies; (3) search strategy and study selection; (4) extracting information from the studies; and (5) collating, summarizing, and reporting the results.

    Research Questions

    Systematic reviews often focus on specific questions; however, scoping reviews explore questions with a broader scope [20]. In this review, the overarching aims targeted 3 aspects of health education serious games: general information, design specifications, and evaluation outcomes. Specific questions targeting each aspect of these health education games were discussed by a group of medical informatics experts (2 faculty members and 4 graduate students) and selected through a consensus process (Table 1).

    Table 1. Study questions and corresponding health game aspects.
    View this table
    Identifying Relevant Studies

    The research team used an established framework [20] to develop an overall guideline used to identify if a study is considered relevant to the topic of the scoping review. The guide specified additional details about the articles on 3 perspectives of population, concept, and context (Table 2). The research team then applied this guide to develop a detailed inclusion and exclusion criteria for the review process.

    Table 2. Overall guideline for the inclusion and exclusion of articles.
    View this table
    Search and Screening Strategy

    The search strategy was guided by the overall inclusion and exclusion framework (Table 2). After reviewing a handful of serious games articles, including review articles, and consulting informatics professionals, the study team developed a set of potential keywords to match the overall inclusion and exclusion criteria of the articles (Table 3). To accommodate a wider scope of articles, 3 sets of potential keywords were developed to cover gaming, health, and education domains and later were assessed for comprehensiveness while being converted into a unified and single keyword phrase.

    Table 3. Concepts and potential keywords.
    View this table
    Study Selection and Inclusion

    As depicted in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram in Figure 1 [21], the initial search retrieved 2313 articles from PubMed (n=1978) and ScienceDirect (n=335). After removing duplicates (n=55), 2258 articles were retained for screening. The search team used the inclusion and exclusion guidelines (Table 2) to filter the articles. Screening the title and abstract of the articles resulted in the exclusion of 1917 articles. To harmonize the results and exclude exercise games, health games using hardware accessories (eg, sensors) or adopting commercial games that are specifically designed for exercise (with minimal educational content) were removed during the title and abstract screening. The remaining 341 articles were furthered screened after reading their full text, which resulted in 180 articles being excluded from the study. We specifically excluded studies assessing the negative aspects of violent games, as they are not considered health games. The risk of bias was not assessed, as this scoping review did not intend to systematically review and evaluate all effective interventions. In the end, 161 articles were determined to meet all inclusion and exclusion criteria and were fully reviewed in the data extraction phase (which further split the studies based on their target groups of health care providers, patients, and public users).

    Figure 1. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram of the search methodology.
    View this figure
    Data Extraction

    A data extraction form was developed by the research team after the determination of the conforming variables with the research questions and the study’s goal (Table 4). The variables were categorized based on the study’s 3 major aspects (ie, general information, design, and evaluation; Table 1). The data extraction form was shared with all study team members and was finalized after addressing all remaining questions and comments.

    In total, 2 reviewers used the data extraction form to independently mine 9.9% (16/161) of the articles. The kappa coefficient score between the 2 reviewers was calculated at 76%. Both reviewers further discussed discrepancies internally and were trained once more to reach a high consensus rate (ie, reaching a kappa score >90% in the second round of coding). Two reviewers then extracted data from the rest of the articles. In the case of nonconformation between the reviewers, a third reviewer was consulted to reach consensus.

    Table 4. Data extraction form.
    View this table
    Data Collation and Analysis

    Data extraction results were collected in 2 Microsoft Excel sheets managed by each of the reviewers. The Excel sheets were then merged to generate the final set of results. Excel functions were used to populate the summary statistics and perform a frequency analysis. To analyze the captured data, we applied a frequency analysis for all variables of interest and presented the results in various chart formats. To improve the interpretability of the results, we stratified all findings into the 3 user groups of health care providers, patients, and public users.


    Results

    Overall Findings

    The findings of this review were categorized based on health education serious game aspects and specific questions identified earlier in the review (Table 1). Significant study findings were grouped into the geographic distribution of the studies, publication year, type and goal of the studies, target groups, gameplay duration and repetitions, intervention specs, length of the follow-up period, and the use of multidisciplinary teams.

    Publication by Geography

    Figure 2 depicts the geographical distribution (categorized by continent) of published health education serious game articles. Most of the published articles (n=82) originated from institutions in the American continent (≥90% in North America). After America, Europe (n=64), Asia (n=13), and Oceania (n=2) had the highest number of studies assessing health education games. None of the articles were published by an author affiliated with an institution in Africa.

    Figure 2. Studies based on geographical locations and stratified by user groups.
    View this figure
    Publication by Year

    Figure 3 demonstrated the number of articles on health education games per 5-year intervals. The first article was published in 1989. Since 2011, the number of articles has grown considerably, with a notable peak in articles published after 2015 (n=107).

    Figure 3. Studies by year and stratified by user groups.
    View this figure
    Type of Study

    In this review, articles were categorized into 2 classes of interventional (77.6%) and observational (22.4%) studies. Among the interventional studies, 44 were randomized clinical trials, whereas 81 of the studies were quasi-experimental (Figure 4). A few studies have assessed the educational health games using a qualitative approach (eg, design protocols and surveys).

    Figure 4. Publications by study design and stratified by user groups. RCT: randomized controlled trial.
    View this figure
    Study Outcomes

    Studies were categorized into 2 classes of knowledge improvement (94/161, 58.4%) and skill improvement (67/161, 41.6%). The knowledge improvement category included multiple subcategories, such as knowledge of diseases (6.8%), general health (6/161, 3.7%), health care management (1.9%), medications (1.2%), mental health (4.3%), nutrition (8.7%), pedagogical content (eg, higher education curriculum; 21.1%), safety and prevention (6.2%), and sexuality (5.0%; Figure 5). The skill improvement category also included multiple subcategories of skills: behavioral and emotional (1.9%), clinical competency (6.2%), cognition (11.2%), decision making (1.9%), language (1.9%), mathematics (1.9%), memory (1.9%), motor movement (1.2%), perceptual (1.2%), reading writing (3.1%), self-control (2.5%), self-efficacy (1.2%), social (5.0%), and visual-auditory (0.6%). Games designed for health care providers were mainly targeting pedagogical knowledge, clinical competencies, and decision-making skills. Games designed for the patients mostly focused on cognition, specific diseases, mental health, social challenges, and a growing number of specific skill sets (eg, self-control and language). Games targeting public users covered a variety of topics, such as nutrition, safety and prevention, and general health items (Figure 5).

    Figure 5. Studies by target outcome and stratified by user groups.
    View this figure
    Target User Groups

    We studied 3 specifications of the target groups: age groups, user groups (eg, health care provider, patients, and public users), and medical condition (specific to patients).

    Age Groups

    We used educational age ranges [22] to analyze age groups (Table 5). Some studies only included one of the age categories (78/161, 48.4%), whereas some included more than 1 age category (39/161, 24.2%). Approximately 27.3% (44/161) of studies did not specify the age groups. The studies that were limited to 1 age category included children (36/78, 46%), adolescents (10/78, 12%), adults (29/78, 37%), and elders (3/78, 3%) but did not include neonates or infants or toddlers target groups. Among studies that targeted more than 1 age category (39/161, 24.2%) of the articles, age categories included neonates and infants and toddlers (1/39, 2%), toddlers and children (1/39, 2%), children and adolescents (17/39, 43%), children and adolescents and adults (3/39, 7%), children and adults (1/39, 2%), adolescents and adults (5/39, 12%), adolescents and adults and elders (1/39, 2%), and adults and elders (10/39, 25%). There were no studies to evaluate neonate, infant, and toddler target groups independently.

    Table 5. Age-specific considerations in patient care and health education.
    View this table

    After combining the age groups into larger age bins for games targeting patients and public users, the age bin of 0 to 18 years showed the highest number of studies (n=65), compared with 18 to 65 years (n=49) and 65 years and older (n=3, Figure 6). Logically, all games designed for health care providers fell within the age range of 18 to 65 years (not shown in the figure).

    Figure 6. Studies by age ranges of health games designed for patients and public users. Educational health games designed for healthcare providers are not shown (n=42). N/A: not applicable.
    View this figure
    User Groups

    Overall, the user groups were divided into 3 general categories (Figure 1): (1) health care providers, such as physicians and nurses (42/161, 26.1%), (2) patients (38/161, 23.6%), and (3) public users (75/161, 46.6%). A total of 6 studies (6/161, 3.7%) included both patients and public users.

    Medical Conditions

    Of all reviewed studies, 42 (42/161, 26.1%) studies targeted various medical conditions (Table 6). These conditions were either meant to improve individual health outcomes or prevent specific diseases.

    Table 6. Targeted medical conditions among different age groups of patients and public users.
    View this table
    Duration of Gameplay

    Duration of gameplay (ie, time spent on interacting with the game) has been reported in 101 studies (101/161, 62.7%). Figure 7 depicts the distribution of gameplay duration among these 101 studies. The most common gameplay duration was 30 to 45 min (n=31) and the least was less than 15 min (n=13).

    Figure 7. Duration of gameplay in minutes and stratified by user groups. Studies with missing duration of gameplay are not shown (n=60).
    View this figure
    Number of Game Repetitions

    Of all 161 studies, 61 health games (37.9%) mentioned the number of times the game can be repeatedly played (Figure 8). Among these studies, 2 general categories of repetitiveness were observed: (1) articles that limited the number of times a game can be played (n=58) and (2) articles that set no limitations for the number of repeats and users were allowed to have an interaction with the game with unlimited repetitions during the intervention time (n=3; a subset of >35 bar in Figure 8).

    Among the articles of the first category, games with less than 5 repetitions had the highest number of articles among health care provider and public user groups. Games designed for patients had 20 to 25 as the highest number of repetitions (Figure 8).

    Figure 8. Number of game repetitions across the studies (stratified by user groups). Studies with missing game repetitions are not shown (n=100).
    View this figure
    Duration of Intervention

    The duration of intervention was mentioned in 94 of the reviewed studies (Figure 9). The duration of intervention varied between 1 week and 8 years. Most studies (34/94, 36%) had a time range of less than 1 month.

    Figure 9. Duration of intervention in months (stratified by user groups). Studies with missing duration of intervention are not shown (n=67).
    View this figure
    Time Between Intervention and Posttest

    The time between intervention and posttest was reported in 59 (59/161, 36%) of the reviewed studies (Figure 10). Two categories of time between intervention and posttest were seen: (1) conducting a test right after the intervention (48/59, 81%) and (2) conducting a test after at least 1 day is passed from the intervention (11/59, 18%). The time ranged between 1 and 12 weeks among the latter group of studies.

    Figure 10. Time between intervention and posttest (stratified by user groups). Studies with missing information are not shown (n=102).
    View this figure
    Follow-Up Duration

    Follow-up duration was collected by a few of the studies (32/161, 19%; Figure 11). Among these studies, 97% had a follow-up duration of 1 week to 6 months. Only 1 study reported a 2-year follow-up period.

    Figure 11. Duration of follow-up in months (stratified by user groups). Studies with missing follow-up duration are not shown (n=129).
    View this figure
    Multidisciplinary Teams

    Although the use of multidisciplinary teams to design health games is strongly recommended, only 42% (68/161) of the reviewed games either explicitly mentioned the use of such teams or implicitly mentioned the involvement of such experts (eg, instructional, clinical, and user experience) in the development and assessment of the games (Figure 12). Analyzing the use of multidisciplinary teams over the years of publications did not show any significant trends.

    Figure 12. Development of studies by multidisciplinary teams (stratified by user groups).
    View this figure

    Discussion

    Principal Findings

    Serious games are increasingly recommended as effective techniques to improve health education [23-25]. Multiple studies have assessed educational serious games in different fields of health, ranging from preventative screening to management of chronic diseases [9,26-28]. Over the last decade, a growing number of these studies have measured the efficiency and effectiveness of serious health education games using randomized trials of patients and clinicians [29-35]. Our review provides an updated scoping review of the underlying patterns and gaps of studies assessing the value of serious games in improving health education for health care providers, patients, and public users.

    Serious Game Development, Target Groups, and Topics

    Our results confirm the concentration of educational health game development in developed countries of North America, Europe, and Asia, thus lacking the opportunity to target educational needs of low-income countries by adopting contextualized or localized serious games. Given the increased availability of smart electronic devices and the penetration of the internet in developing countries [36], more research is needed to develop and assess the impact of serious games targeting specific health educational needs of such populations (eg, infectious diseases) [37]. Given the peak of health game development in recent years (between 2015 and 2018) and the potential higher commercialization of such solutions, focusing on emerging topics of developing countries can benefit larger populations in need [38].

    Our results show that more than one-fourth (26.1%) of the health education games focused on higher education needs and targeted medical students and staff. The fact that most of these studies are using health games as an intervention and a considerable number of them have assessed their effectiveness using experimental designs (including randomized trials) provides a growing opportunity to assess the effectiveness of such serious games. It can be anticipated that medical schools (especially in North America and developed countries of Europe and Asia) will gradually incorporate such interactive solutions (ie, serious games) in their common educational curriculum in the near future. In addition, given the low cost of technology needed to use such games in medical and nursing schools, open source or free versions of such serious games can tremendously help reducing educational disparities in clinical sciences among low-resource countries.

    An interesting trend revealed by our results is the gradual move from developing disease-specific serious educational games (eg, educational games for diabetes) to targeting broader public health topics (eg, safety and nutrition). Given the impact of health topics such as safety and nutrition on population-level outcomes, the future of educational health games may entail a larger coverage of the general healthy population rather than patients with specific diseases (ie, helping to bridge pubic and population health outcomes [39,40]). The wider target groups of end users can potentially translate into increased market opportunities for educational health games as well as sustainable commercialization over the long term.

    Game Design and Learning Outcomes

    Educational games primarily aim to increase awareness and knowledge among the players. Nonetheless, the ultimate goal of educational health games should include a behavioral change in the end users, thus producing a lasting effect. Developing such complex health education serious games, however, is challenging as it requires the participation of multidisciplinary team members to address various game play perspectives, ranging from principles of design to psychology of behavioral change. In this review, the evaluation of the studies revealed that the use of multidisciplinary teams to design health education games is strongly recommended, but it is accomplished occasionally by game developers. Indeed, according to our evaluation, almost half of the studies were deprived from a multidisciplinary team. Continuous collaborations among the members of the multidisciplinary expert team in addressing various aspects of game design and development are strongly recommended to improve the educational experience of the users and potentially improve the impact of health education games.

    Most studies reviewed in this study had a short duration of game play with minimal repetitions and limited follow-up periods. In addition, 6 studies did not report an effective intervention and could not achieve desired educational outcomes. These limitations can be attributed partly to the lack of expert user experience designers participating in the study teams [41]. Furthermore, although factors, such as time of intervention, time between intervention and posttest, and duration of follow-up, are critical in achieving long-term knowledge gains, generalized evidence on what factors with what frequency and length works for what age-range is still lacking. More work is needed to established common design guidelines on how to engage different user groups with best game specifications for health education for both patients and medical staff groups.

    Comparison With Prior Work

    Previous scoping reviews of health games included all types of serious games, ranging from educational to behavioral change and exercise-focused games [4,5]. Several reviews have predominantly focused on chronic diseases [42-44], whereas other reviews have focused on specific subgroups of educational games, such as games targeting clinical staff and students [10,12,14,45] or patients with distinct medical conditions or diseases (eg, diabetes, asthma, or obesity) [46-53]. Owing to either inclusivity (all health games) or exclusivity (specific educational games) of the previous reviews, these reviews do not reveal the overall trends in educational health games across various target users (ie, health care providers, patients, and public health users). This study provides a fresh review on the latest developments and trends in serious games targeting health education, regardless of the target population or observed clinical context.

    Limitations

    Our work had multiple limitations. First, we specifically narrowed our search on educational serious games for health. Our review should not be considered a systematic review of all types of health games. Second, our review did not conduct a statistical analysis to measure significant differences among various target groups (eg, measuring significant differences between patients and public health users). Third, we did not perform a meta-analysis, as the studies did not report enough details about their population to perform such analysis. In addition, the outcomes of interest varied greatly, and a meta-analysis was not appropriate for our review. Fourth, we only included English publications, thus unavoidably excluded studies that might be conducted and published in other developed countries not using English as their primary scientific language. Fifth, the variability of reviewed studies limited our review to encode generic specifications, such as publication date, study location, and target groups. More research is needed to tease out the details reported by various subgroups of these studies (eg, some studies reported exact outcomes of interest). Finally, we used PubMed and ScienceDirect as our search engines; however, other search engines might provide additional studies not indexed in the aforementioned databases.

    Future Work

    Future work should include a larger systematic review, including additional search engines and, perhaps, focusing on the educational games designed for either patients or clinical providers. Additional work on reviewing educational outcomes of interest among these studies and the effectiveness of the health games in achieving them is also needed.

    Conclusions

    Serious games are increasingly used for health education. This study offers an updated scoping review of the studies assessing the value of serious games in improving health education. Most educational health games are still developed in high-income countries, but a surge of new games focusing on healthy behaviors (eg, nutrition and safety) has been observed in the last 5 years. Game developers need to use multidisciplinary teams to improve the design of the serious games in keeping the end users engaged for a longer interaction and potentially more effective educational health outcomes. Investment in serious games, as a low-cost educational tool for both patients and medical providers, can potentially help to fill the gap for health education in developing countries.

    Conflicts of Interest

    None declared.

    References

    1. Johnson L, Becker S, Cummins M, Estrada V, Freeman A, Hall C. The NMC Horizon Report: 2016 Higher Education Edition. Austin, Texas: The New Media Consortium; 2016.
    2. Vlachopoulos D, Makri A. The effect of games and simulations on higher education: a systematic literature review. Int J Educ Technol High Educ 2017;14(1):22. [CrossRef]
    3. Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertain Comput 2013;4(4):231-247. [CrossRef]
    4. Lu AS, Kharrazi H. A state-of-the-art systematic content analysis of games for health. Games Health J 2018 Feb;7(1):1-15 [FREE Full text] [CrossRef] [Medline]
    5. Kharrazi H, Lu AS, Gharghabi F, Coleman W. A scoping review of health game research: Past, present, and future. Games Health J 2012 Apr 18;1(2) [FREE Full text] [CrossRef] [Medline]
    6. Fry H. A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice. New York, NY: Routledge; 2008.
    7. Pannese L, Prilla M, Ascolese A, Morosini D. Serious games for reflective learning: experiences from the MIRROR project. In: Baek Y, Whitton N, editors. Cases on Digital Game-Based Learning: Methods, Models, and Strategies. Hershey, Pennsylvania: IGI Global; 2013:452-474.
    8. Backlund P, Hendrix M. Educational Games - Are They Worth the Effort? A Literature Survey of the Effectiveness of Serious Games. In: Proceedings of the 2013 5th International Conference on Games and Virtual Worlds for Serious Applications. 2013 Presented at: VS-GAMES'13; September 11-13, 2013; Poole, UK. [CrossRef]
    9. Kaufman D, Sauvé L. Educational Gameplay and Simulation Environments: Case Studies and Lessons Learned. Hershey, PA: IGI Global; 2010.
    10. Akl EA, Pretorius RW, Sackett K, Erdley WS, Bhoopathi PS, Alfarah Z, et al. The effect of educational games on medical students' learning outcomes: a systematic review: BEME Guide No 14. Med Teach 2010 Jan;32(1):16-27. [CrossRef] [Medline]
    11. Kron FW, Gjerde CL, Sen A, Fetters MD. Medical student attitudes toward video games and related new media technologies in medical education. BMC Med Educ 2010 Jun 24;10:50 [FREE Full text] [CrossRef] [Medline]
    12. Bhoopathi PS, Sheoran R, Adams CE. Educational games for mental health professionals: a Cochrane review. Int J Psychiatr Nurs Res 2007 May;12(3):1497-1502. [Medline]
    13. Gorbanev I, Agudelo-Londoño S, González RA, Cortes A, Pomares A, Delgadillo V, et al. A systematic review of serious games in medical education: quality of evidence and pedagogical strategy. Med Educ Online 2018 Dec;23(1):1438718 [FREE Full text] [CrossRef] [Medline]
    14. Sipiyaruk K, Gallagher JE, Hatzipanagos S, Reynolds PA. A rapid review of serious games: from healthcare education to dental education. Eur J Dent Educ 2018 Nov;22(4):243-257. [CrossRef] [Medline]
    15. Emmerich K, Bockholt M. Serious games evaluation: processes, models, and concepts. In: Dörner R, Göbel S, Kickmeier-Rust M, Masuch M, Zweig K, editors. Entertainment Computing and Serious Games. Cham: Springer; 2016:265-283.
    16. Fullerton T. Game Design Workshop: A Playcentric Approach to Creating Innovative Games. Boca Raton, FL: CRC Press; 2014.
    17. Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012 Oct;99(10):1322-1330. [CrossRef] [Medline]
    18. DeSmet A, Shegog R, van Ryckeghem D, Crombez G, de Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J 2015 Apr;4(2):78-90. [CrossRef] [Medline]
    19. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8(1):19-32. [CrossRef]
    20. Peters M, Godfrey C, McInerney P, Soares CB, Khalil H, Parker D. Joanna Briggs Institute Reviewer's Manual.: The Joanna Briggs Institute; 2017. Chapter 11: Scoping reviews   URL: https://wiki.joannabriggs.org/display/MANUAL/Chapter+11%3A+Scoping+reviews [accessed 2019-10-10]
    21. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009 Aug 18;151(4):264-9, W64. [CrossRef] [Medline]
    22. RN Continuing Education. RN.com. 2004. Age-Specific Considerations in Patient Care   URL: https://lms.rn.com/getpdf.php/2019.pdf [accessed 2019-10-11]
    23. Hanisch SE, Birner UW, Oberhauser C, Nowak D, Sabariego C. Development and evaluation of digital game-based training for managers to promote employee mental health and reduce mental illness stigma at work: quasi-experimental study of program effectiveness. JMIR Ment Health 2017 Aug 4;4(3):e31 [FREE Full text] [CrossRef] [Medline]
    24. Johnsen HM, Fossum M, Vivekananda-Schmidt P, Fruhling A, Slettebø Å. Teaching clinical reasoning and decision-making skills to nursing students: design, development, and usability evaluation of a serious game. Int J Med Inform 2016 Oct;94:39-48. [CrossRef] [Medline]
    25. Lin H, Chiu Y, Chen YJ, Wuang Y, Chen C, Wang C, et al. Continued use of an interactive computer game-based visual perception learning system in children with developmental delay. Int J Med Inform 2017 Nov;107:76-87. [CrossRef] [Medline]
    26. Faiola A, Kharrazi H. Diabetes education and serious gaming: teaching adolescents to cope with diabetes. In: Hayes BM, Aspray W, editors. Health Informatics: A Patient-Centered Approach to Diabetes. Cambridge: MIT Press; 2010:151-177.
    27. Kharrazi H, Faiola A, Defazio J. Healthcare game design: behavioral modeling of serious gaming design for children with chronic diseases. In: Jacko JA, editor. Human-Computer Interaction. Interacting in Various Application Domains. Berlin, Heidelberg: Springer; 2009:335-344.
    28. Parisod H, Pakarinen A, Axelin A, Danielsson-Ojala R, Smed J, Salanterä S. Designing a health-game intervention supporting health literacy and a tobacco-free life in early adolescence. Games Health J 2017 Aug;6(4):187-199. [CrossRef] [Medline]
    29. Dias JD, Domingues AN, Tibes CM, Zem-Mascarenhas SH, Fonseca LM. Serious games as an educational strategy to control childhood obesity: a systematic literature review. Rev Lat Am Enfermagem 2018 Sep 3;26:e3036 [FREE Full text] [CrossRef] [Medline]
    30. Diehl LA, Souza RM, Gordan PA, Esteves RZ, Coelho IC. InsuOnline, an electronic game for medical education on insulin therapy: a randomized controlled trial with primary care physicians. J Med Internet Res 2017 Mar 9;19(3):e72 [FREE Full text] [CrossRef] [Medline]
    31. Graafland M, Bemelman WA, Schijven MP. Game-based training improves the surgeon's situational awareness in the operation room: a randomized controlled trial. Surg Endosc 2017 Oct;31(10):4093-4101 [FREE Full text] [CrossRef] [Medline]
    32. Kharrazi H. Improving healthy behaviors in type 1 diabetic patients by interactive frameworks. AMIA Annu Symp Proc 2009 Nov 14;2009:322-326 [FREE Full text] [Medline]
    33. Tan AJ, Lee CC, Lin PY, Cooper S, Lau LS, Chua WL, et al. Designing and evaluating the effectiveness of a serious game for safe administration of blood transfusion: A randomized controlled trial. Nurse Educ Today 2017 Aug;55:38-44. [CrossRef] [Medline]
    34. Primack BA, Carroll MV, McNamara M, Klem ML, King B, Rich M, et al. Role of video games in improving health-related outcomes: a systematic review. Am J Prev Med 2012 Jun;42(6):630-638 [FREE Full text] [CrossRef] [Medline]
    35. Papastergiou M. Exploring the potential of computer and video games for health and physical education: a literature review. Comput Educ 2009;53(3):603-622. [CrossRef]
    36. ITU: Committed to connecting the world. 2019. Measuring Digital Development: Facts and Figures 2019   URL: https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx [accessed 2019-10-11]
    37. Thomsen EK, Hemingway C, South A, Duda KA, Dormann C, Farmer R, et al. ResistanceSim: development and acceptability study of a serious game to improve understanding of insecticide resistance management in vector control programmes. Malar J 2018 Nov 13;17(1):422 [FREE Full text] [CrossRef] [Medline]
    38. Kharrazi H, Lasser EC, Yasnoff WA, Loonsk J, Advani A, Lehmann HP, et al. A proposed national research and development agenda for population health informatics: summary recommendations from a national expert workshop. J Am Med Inform Assoc 2017 Jan;24(1):2-12 [FREE Full text] [CrossRef] [Medline]
    39. Gamache R, Kharrazi H, Weiner JP. Public and population health informatics: The bridging of big data to benefit communities. Yearb Med Inform 2018 Aug;27(1):199-206 [FREE Full text] [CrossRef] [Medline]
    40. Kharrazi H, Weiner JP. IT-enabled community health interventions: challenges, opportunities, and future directions. EGEMS (Wash DC) 2014;2(3):1117 [FREE Full text] [CrossRef] [Medline]
    41. Phan MH, Keebler JR, Chaparro BS. The development and validation of the game user experience satisfaction scale (GUESS). Hum Factors 2016 Dec;58(8):1217-1247. [CrossRef] [Medline]
    42. Charlier N, Zupancic N, Fieuws S, Denhaerynck K, Zaman B, Moons P. Serious games for improving knowledge and self-management in young people with chronic conditions: a systematic review and meta-analysis. J Am Med Inform Assoc 2016 Jan;23(1):230-239. [CrossRef] [Medline]
    43. de Sá KC, Martins MG, da Costa CA, Barbosa JLV, Righi RR. A mapping study on mobile games for patients of chronic diseases. J Med Syst 2017 Sep;41(9):138. [CrossRef] [Medline]
    44. Holtz BE, Murray K, Park T. Serious games for children with chronic diseases: a systematic review. Games Health J 2018 Oct;7(5):291-301. [CrossRef] [Medline]
    45. Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs 2009 Feb;65(2):259-269. [CrossRef] [Medline]
    46. Drummond D, Monnier D, Tesnière A, Hadchouel A. A systematic review of serious games in asthma education. Pediatr Allergy Immunol 2017 May;28(3):257-265. [CrossRef] [Medline]
    47. Kharrazi H, Watters C, Oore S. Improving behavioral stages in children by adaptive applications. J Inf Technol Healthc 2008;6(1):83-95 [FREE Full text]
    48. Watters C, Oore S, Shepherd M, Abouzied A, Cox A, Kellar M, et al. Extending the Use of Games in Health Care. In: Proceedings of the 39th Annual Hawaii International Conference on System Sciences.: IEEE Computer Society; 2006 Presented at: HICSS'06; January 4-6, 2006; Kauia, HI. [CrossRef]
    49. Kharrazi H, Vincz L. Increasing physical activity by implementing a behavioral change intervention using pervasive personal health record system: an exploratory study. In: Proceedings of the International Conference on Universal Access in Human-Computer Interaction.: Springer; 2011 Presented at: UAHCI'11; July 9-14, 2011; Orlando, FL. [CrossRef]
    50. Kharrazi H, Vincz L. CiteSeerX. 2011. Using Health Information Technology and Behavior Change Model to Increase Exercise Adherence in Obesity   URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.457.7048&rep=rep1&type=pdf [accessed 2019-10-11]
    51. Mayorov A. The 71st Scientific Session of American Diabetes Association (ADA), June 24-28, 2011, San-Diego. In: Diabetes mellitus.: American Diabetes Association; 2011 Sep 15 Presented at: The 71st Scientific Session of American Diabetes Association (ADA); Jun 24, 2011; San-Diego, CA p. 120   URL: https:/​/professional.​diabetes.org/​abstract/​can-computer-game-improve-adherence-treatment-children-type-1-diabetes [CrossRef]
    52. Lu AS, Kharrazi H, Gharghabi F, Thompson D. A systematic review of health videogames on childhood obesity prevention and intervention. Games Health J 2013 Jun;2(3):131-141 [FREE Full text] [CrossRef] [Medline]
    53. Theng Y, Lee JW, Patinadan PV, Foo SS. The use of videogames, gamification, and virtual environments in the self-management of diabetes: a systematic review of evidence. Games Health J 2015 Oct;4(5):352-361. [CrossRef] [Medline]

    Edited by G Eysenbach; submitted 22.01.19; peer-reviewed by P Haubruck, B Ingadottir, R Alkoudmani, E Da Silva, MS Aslam, J Kofranek; comments to author 21.03.19; revised version received 07.09.19; accepted 12.12.19; published 05.03.20

    ©Nahid Sharifzadeh, Hadi Kharrazi, Elham Nazari, Hamed Tabesh, Maryam Edalati Khodabandeh, Somayeh Heidari, Mahmood Tara. Originally published in JMIR Serious Games (http://games.jmir.org), 05.03.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.