We conducted a scoping review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement as a more robust methodological approach [25]. The search was performed between July 2019 and August 2019.

Before starting the review, we manually checked for relevant articles in the authors’ languages (ie, French, Italian, and Spanish) on the first 20 pages of Google Scholar. Since we did not find any article on “gamification” AND “vaccine” in corresponding languages, we confirmed our initial choice to restrict our search to English-language papers. The following search terms and related variations were used: all (“vaccine*” OR “vaccination” OR “immunization*”) AND all (“serious game*” OR “video game*” OR “therapeutic game*” OR “online game*” OR “game*” OR “game app*” OR “mobile game*” OR “digital game*” OR “gamif*”).

Studies were selected from a search of the following major electronic databases: PubMed, Google Scholar, Journal of Medical Internet Research, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences Collection, and SocINDEX via EBSCOhost. A supplementary manual search was performed to identify additional relevant publications by reviewing the reference lists of the included articles and using ResearchGate.

Only peer-reviewed studies written in English in the decade 2009-2019 were included, regardless of the location of the study, type of vaccination under study, and study population. All types of study designs were included (eg, quantitative, qualitative, and mixed-methods studies; systematic reviews; meta-analyses; randomized controlled trials; and other experimental studies like pretests and posttests). Posters, preprints, and conference proceedings were excluded. Studies were included only if they used a gamification technique or tool to deliver informative or educative messages on vaccination. All types of digital games or gamified elements were included, from serious games (content gamification) to gamified Web-based quizzes (structural gamification) [26]. Articles not presenting the description and evaluation of a concrete gamified digital tool were excluded (eg, articles reporting the results of a survey on users’ needs and perspectives on games for vaccination).

Records identified in the literature search were evaluated in a 3-step approach. First, all identified titles and abstracts were screened for eligibility and coded by one researcher according to the selection criteria. Second, relevant articles were retrieved, and full-text articles were read independently by the researcher in charge of coding and extracting all data and by a second researcher. Third, a final list of publications for full-text review was established and validated by the 2 researchers. Any discrepancies were reviewed by a third researcher and finally resolved through consensus.

The 2 researchers conducted a quality assessment using the quality assessment method presented by Connolly et al [27]. This method assesses the overall weight of empirical evidence for the positive impact and outcomes of games. Each final paper included in the review was given a score of 1 (low), 2 (medium), or 3 (high) across the following 5 criteria: (1) appropriateness of the research design, with a score of 3 for randomized controlled trials, 2 for quasi-experimental controlled studies, and 1 for case studies, single subject-experimental, pretest/posttest, and other types of quantitative and qualitative studies; (2) appropriateness of methods and analysis; (3) representativeness and generalizability of the findings; (4) relevance of the focus of the study; and (5) relevance of the findings and their discussion. The total score for each paper was calculated by adding the scores of all 5 dimensions, resulting in a range from 5 to 15 points. Following the studies by Connolly et al [27] and Johnson et al [13] using the same quality assessment method, we categorized articles with a score of ≤8 points as weaker evidence, articles with a rating >8 to 12 points as moderate evidence, and articles with a rating >12 points as stronger evidence. We calculated the average score for each study and measured the weighted Cohen's kappa coefficient to test interrater reliability.

Data were sorted in categories, including author, year of publication, country, journal, research design, sample size and characteristics, type of vaccine, theory used, game content, game modality, gamification element(s), data analysis, type of outcomes, and mean quality score. For game modality and gamification elements, we based our coding on the work by Hamari et al [21].

To assess the effectiveness of gamified digital tools for vaccination, we took into account 3 types of outcomes: behavior (eg, real actions like receiving a vaccination or intent to get vaccinated), cognition (eg, increased knowledge of the topic or vaccine literacy [28]), and usability/acceptability (eg, appreciation of the intervention). Furthermore, detailed outcomes of the evaluation of each game were individually reported.

Each study presented 1 gamified digital tool, with the exception of 1 study that described and evaluated 2 tools: one on vaccination and one on antibiotics [31]. Only data concerning the vaccination game were taken into account for this scoping review.

The final 7 articles eligible for review were rated for the quality of the evidence: 2 articles were categorized as providing weaker evidence [24,32], 2 articles were categorized as providing moderate evidence [33,34], and 3 articles were categorized as providing stronger evidence [29,31,35]. The weighted Cohen's kappa coefficient was 0.71, suggesting good agreement between raters [36]. Concerning methodologies, 2 studies used a randomized controlled trial design [29,34], 1 study used a controlled experiment with posttest questionnaires [32], 1 study used only focus groups [33], and 1 study used only pretest and posttest questionnaires [24]. A mixed methods approach was used by 3 studies [30,31,35], balancing quantitative data from questionnaires with qualitative data from interviews or focus groups.

Game modalities and gamification elements were diverse (eg, avatars, challenges, informative feedback, points, levels, leaderboards, storytelling), but role play and reward systems were present in all studies. Of the 7 games, 4 were Web-based [24,31,32,34], whereas the other 3 were mobile apps [30,33,35]. One study [35] also used a social networking site. Of the 7 studies, 3 described tools that were cocreated by several stakeholders including health professionals, developers, and end users [24,33,35]. All gamified digital tools were promoted and funded through health authorities for public health purposes and universities for educational purposes. Of the 7 gamified digital tools, 5 were serious games [24,31,33-35], and the other 2 were implemented as a quiz [30] and website [32].

Infectious diseases were the specific focus of 6 studies: 3 studies addressed HPV [33-35], 2 studies addressed influenza [24,32], and 1 study addressed the combination of measles, mumps, and rubella [30]. The remaining study addressed vaccination in general [31].

Tools were mostly aimed at young players: 3 tools were developed for school-aged children [24,31,33], and 2 tools were developed for university students [32,34]. For 1 of the tools developed for children [33], evaluation data were collected also from parents. The remaining 2 tools were developed for the general population [30,35].

All studies but one [27] explicitly incorporated one or more behavioral theories: self-determination theory [33], health belief model [33,34], self-concept theory [34], theory of reasoned action and planned behavior [34], game theory [24,32], social value orientation [32], nudge theory [35], and empowerment model [30].

Positive effects of gamified interventions were reported in all 7 studies across at least one of the 3 outcomes we considered (ie, behavior, cognition, and usability/acceptability). For behavior, data were available for 5 studies [24,30-32,34]; for cognition, data were available for 3 studies [30,31,35]; and for usability/acceptability, data were available for 4 studies [30,31,33,35].

Table 1 outlines the characteristics of the 7 articles included in the review presented in chronological order. See Multimedia Appendix 1 for a description of the games and outcomes.

We identified 7 studies reporting data on the evaluation of the effectiveness of digital tools using gamification on the topic of vaccination. All 7 studies presented positive results in terms of pre-established outcomes (ie, behavior, cognition, and usability/acceptability) confirming that gamified digital tools can facilitate communication of vaccination-related messages and contribute to increased vaccination uptake. These results agree with those of previous studies demonstrating that gamification can contribute to changed behaviors and increased knowledge as well as be appreciated by users with regard to health-related topics [38]. When combining gamification and emerging technologies, results might be even more promising [39]. Digital tools have the advantage of being ubiquitous without time and space constraints [40], and the inclusion of gamified features might increase their appeal and acceptability, as reported in our study. Based on these assumptions, the design of future interventions should consider the use of both new technologies and gamification.

Previous research described existing serious games used for vaccination [23]. However, selected tools dated back to the year 2015 and were just listed and fully described without any appraisal of their effectiveness. Furthermore, only serious games were analyzed in this previous study [23], without considering other digital tools that were not full-blown games but used gamified features [20]. Through our review, we provide an update, through the year 2019, of existing digital games for vaccination and an evaluation of these games. Given the speed at which technology changes and improves, monitoring new digital tools is essential. Most importantly, to our knowledge, our study is the first scoping review synthesizing data on the evaluation of the effectiveness of different types of digital tools using gamification, not exclusively serious games.

We classified games per type of outcome. The 4 studies [24,30,32,34] reporting data on behavior showed a remarkable increase in the intent to get vaccinated and a positive attitude towards vaccination. Similar results on behavior change have been found in serious games for oral hygiene [41], asthma [42], and fruit and vegetable consumption [43]. Effectiveness in terms of behavior change might be explained by the fact that gamification tends to improve the involvement and motivation of users who feel more convinced of their decision after playing [44,45]. Games are thought to provide a good medium for increasing self-efficacy and changing behavior as they offer the opportunity for a new experience in a safe environment, without real-life consequences to making wrong decisions [46].

Positive findings on cognition were reported by 3 studies [30,31,35], corresponding to increased knowledge and literacy about vaccination. This agrees with a meta-analysis of serious digital games for healthy lifestyle promotion [46]. According to this previous scientific work, health knowledge is easier to influence than other outcomes, but the impact of a change in knowledge is not as strong as influencing a person’s intent to change behavior. These positive results might be explained by the learning-by-doing approach used in gamified digital tools where players learn through exploration and experimentation.

In terms of usability/acceptability, results on this outcome were reported by 3 studies [30,31,35]. Effectiveness in terms of usability is potentially justified by the co-construction process. Including different stakeholders like health professionals and game developers is time-consuming but is useful for developing games that will appeal to the target audience [47]. One game promoting healthy eating in children [48] and one game to help young people quit smoking [49] underlined the advantages of using a co-construction approach to increase commitment to and acceptability of the games.

While all 7 gamified digital tools were effective across the 3 types of outcomes, they presented different characteristics (ie, game features, targeted audience, and targeted disease for vaccination). Among successful game features, role play or characterization, earning and losing tokens, and advancing levels were the 3 modalities that were the most used and accepted by users across all studies. As for role play or characterization, FightHPV [35] and Stop the Spread [31] were appreciated by users because they included a set of characters that were considered really amusing. In particular, the avatars in Vax! [34] allowed users to better appraise risk perception for infectious diseases as well as to increase one’s intent to receive vaccination. Presenting a story with a character with which users could identify might be preferred because the character refers to one’s doubts and knowledge and helps virtually measure the impact of one’s decision. Character identification can improve risk perception and encourage vaccination uptake [50], since, within the game, the harms of nonvaccination can be virtually self-experimented. Furthermore, following a herd immunity approach [51], some of the games under study made users interact and confront with other imaginary characters to explain the collective dimension and community-level impact of vaccination. Immersive story telling could also have enhanced engagement and subsequent retention of key messages. As for earning and losing tokens, like in I-Vax [32], Land of Secret Gardens [33], and Morbiquiz [30], the presence of tokens in a game might encourage the players and maintain their motivation. The explanation can be found in behavioral theories: reinforcement and punishment contingencies are equally effective as long as they challenge the user [52]. Participants feel their role in the game is active, which makes them more engaged with the game. As for advancing levels, we know from the Stop the Spread game [31] that players enjoy the steady increase in difficulty as the game progresses. However, if games are too difficult, like the first version of FightHPV [35], users stop playing and cannot learn if they do not advance. Thus, it is important to design the game so that players feel challenged but not frustrated. Like earning and losing tokens, advancing levels maintains the interest and motivation of the player. Finally, results showed that using a serious game format, like in most of the studies [24,31,33-35], was not more effective than the other two formats (ie, quiz [30] and website [32]), thus confirming that using gamifying features might be as effective as a full-blown game. Future research including more studies is needed to validate this hypothesis.

As for the targeted audience, apart from 1 study involving parents who were a little skeptical about the trustworthiness of a gamified approach [33], all other studies targeted young people (children, students, young parents) who were attracted by digital gamified tools and represented the most captive audience. About 90% of teenagers play video games [53], and millennials are simultaneously technologically adept with and shaped by technology [54]. As future parents and adults making decisions about their health and the health of their relatives, young people represent a core target group for interventions addressing vaccination coverage.

As for the type of targeted vaccine, all games except Stop the Spread [31] targeted a specific vaccine. This might have been a good strategy to clearly explain one vaccine at a time without confusing the player. Targeting other vaccines like hepatitis, for instance, with new games might be interesting to have a complete spectrum of vaccines explained through gamified digital tools. On the other hand, the design of games seems to be independent from the type of vaccine. This means that the same game might be transferable to all types of vaccine. This could be the case of games like Morbiquiz [30], VAX! [34], and FightHPV [35], whose designs can be easily adapted to other vaccines or vaccine-related diseases.

Finally, the main findings of this review include the relevance of incorporating behavioral theories within the game conception. Almost all gamified digital tools under study were based on solid theories and proved to be effective in facilitating understanding and appraisal of information about vaccines and behavior change endpoints. As suggested by previous literature, theory-driven interventions are more efficacious than those that are not [55], which might further explain the positive outcomes of synthesized tools supported by the gamification approach.

Limitations of this review were the comparative paucity of included studies and the marked heterogeneity of their study design and contents. Publication bias should also be considered, as small studies with negative findings would likely not be published. Furthermore, the scores of some articles were low [33,34] for various reasons including a limited study focus, a research design that was not completely appropriate, nongeneralizable findings, or small sample sizes. Only 3 articles [30,31,34] presented a solid evaluation methodology relying on quantitative and qualitative pretest and posttest data. We also relied on the work of a single coder, which might have introduced systematic bias. However, by omitting a second coder, we wished to ensure consistency in the study selection. Finally, all 7 studies presented positive short-term outcomes, while no long-term impact assessment of the games was conducted.

Based on our study, we can suggest how to improve the design of future interventions. Especially during this period of disinformation about vaccination circulating on the internet, gamified digital tools can help provide more accurate information, while being fun and engaging. These recommendations might ensure that produced games are attractive, validated, and effective, especially if they are produced with researchers and professionals in the vaccine domain.

First, the educational perspective is fundamental for all games, but it must be implicit. Interventions should be focused on increasing knowledge and influencing behavior about vaccination without clearly presenting their final aim. Users might unconsciously learn by playing without having the feeling that they are following a training class or didactic presentation. It is important to use a narrative approach with an appropriate story line to engage, motivate, and empower users throughout the learning process. However, the information provided must be factual and trustworthy.

Second, emotional engagement between the player and the environment should be created. Fun contributes to such engagement. Using characters and avatars is a good strategy to capture and maintain the attention of the players. Avatars can self-represent the player in a simulation or role-playing game so that players are immersed in the game; if users are active, their chances to learn increase. Emotional engagement is also reinforced with levels and tokens. Their use is highly recommended for future gamified digital tools. Provide a leaderboard for competition not only with other players but also with oneself as a personal challenge. However, game functions like recovery aids and the pace of the games should be adapted to the public, to avoid frustration with levels that are too difficult.

Third, a balance should be found between simplicity and attractiveness. Future games should pay attention to the design and aesthetics of the game. Using concise text might help with retention of information and knowledge about vaccination, while not stopping the flow of the game. Amusing characters and animations are necessary to make the game appealing.

Fourth, games should take into consideration their target audience. The best solution to achieve this is to co-construct the games with all concerned stakeholders, including end users, through a design-thinking approach [56]. In general, games addressing children and young adults (especially boys) have the advantage of targeting a population who is familiar with video games and computers in general. This might not be the case for older adults.

Finally, before designing a gamified digital intervention, it is important to consider its costs, which are usually very high (at least US $10,000).

Our review suggests that there is a need to continue developing gamified digital tools but also to evaluate the impact of existing and future tools. Great enthusiasm for these games has led to many being produced with insufficient validation of their effectiveness. To better evaluate gamified digital tools in the short term, medium term, and long term, preferred study designs are randomized controlled trials or any other experimental design with a control group, combined with longitudinal data collection. Using a mixed methods design would also be beneficial to comprehensively capture users’ opinions and satisfaction with the game. Like in our scoping review, the outcomes to measure might be behavioral indicators like individual vaccination records or intent to get vaccinated, cognitive indicators like an increase in knowledge or vaccine literacy, and indicators of usability/acceptability of the intervention. If possible, all 3 types of outcomes should be measured to better assess the overall qualities of the games. Instruments like the Mobile App Rating Scale [37] could be used for this purpose. These evaluation studies will eventually help us understand how gamified digital tools can change vaccination uptake and coverage.

Gamification is an innovative and promising option to consider when designing vaccination-related interventions addressed to the general public and young people in particular, especially for those who are hesitant about vaccination. Based on the findings of this review, health professionals, health promotion and prevention specialists, and developers are encouraged to use game-based features in interventions aimed to endorse vaccination uptake in order to increase their acceptability and consequent effectiveness. Theory-driven gamified digital tools are preferred.