This paper is in the following e-collection/theme issue:

Reviews (41) Vaccination and Immunization in the Digital Age (347) COVID-19 Vaccination (128) Vaccination Sentiment and Anti-Vaccination Infodemiology (50) Behavior Change (655) Gamification (117) Games and Gamification for Health (610) Digital Health Reviews (1191) Serious Games for Health and Medicine (647)

Published on in Vol 12 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47257, first published .
Digital Gamification Tools to Enhance Vaccine Uptake: Scoping Review

Digital Gamification Tools to Enhance Vaccine Uptake: Scoping Review

Digital Gamification Tools to Enhance Vaccine Uptake: Scoping Review

Authors of this article:

Hina Hakim1 Author Orcid Image ;   S Michelle Driedger2 Author Orcid Image ;   Dominique Gagnon3 Author Orcid Image ;   Julien Chevrier4 Author Orcid Image ;   Geneviève Roch5, 6, 7 Author Orcid Image ;   Eve Dubé3, 8 Author Orcid Image ;   Holly O Witteman1, 7, 9 Author Orcid Image
Article Authors Cited by Tweetations (2) Metrics

    Review

    1Department of Family and Emergency Medicine, Université Laval, Québec City, QC, Canada

    2Department of Community Health Sciences, University of Manitoba, Winnipeg, MB, Canada

    3Direction des risques biologiques, Institut national de santé publique du Québec, Quebec City, QC, Canada

    4Bibliothèque Louise-Lalonde-Lamarre, Polytechnique Montréal, Montréal, QC, Canada

    5Faculty of Nursing, Université Laval, Quebec City, QC, Canada

    6Centre hospitalier universitaire (CHU) de Québec-Université Laval, Université Laval, Quebec City, QC, Canada

    7VITAM Research Centre for Sustainable Health, Université Laval, Quebec City, QC, Canada

    8Département d’anthropologie, Université Laval, Quebec City, QC, Canada

    Corresponding Author:

    Hina Hakim, PhD

    Department of Family and Emergency Medicine

    Université Laval

    1050 avenue de la Médecine, Pavillon Ferdinand-Vandry

    Québec City, QC, G1V 0A6

    Canada

    Phone: 1 418 656 2131

    Email: hina.hakim.1@ulaval.ca


    Background: Gamification has been used successfully to promote various desired health behaviors. Previous studies have used gamification to achieve desired health behaviors or facilitate their learning about health.

    Objective: In this scoping review, we aimed to describe digital gamified tools that have been implemented or evaluated across various populations to encourage vaccination, as well as any reported effects of identified tools.

    Methods: We searched Medline, Embase, CINAHL, the Web of Science Core Collection, the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, Academic Search Premier, PsycInfo, Global Health, and ERIC for peer-reviewed papers describing digital gamified tools with or without evaluations. We also conducted web searches with Google to identify digital gamified tools lacking associated publications. We consulted 12 experts in the field of gamification and health behavior to identify any papers or tools we might have missed. We extracted data about the target population of the tools, the interventions themselves (eg, type of digital gamified tool platform, type of disease/vaccine, type and design of study), and any effects of evaluated tools, and we synthesized data narratively.

    Results: Of 1402 records, we included 28 (2%) peer-reviewed papers and 10 digital gamified tools lacking associated publications. The experts added 1 digital gamified tool that met the inclusion criteria. Our final data set therefore included 28 peer-reviewed papers and 11 digital gamified tools. Of the 28 peer-reviewed papers, 7 (25%) explained the development of the tool, 16 (57%) described evaluation, and 2 (7%) reported both development and evaluation of the tool. The 28 peer-reviewed papers reported on 25 different tools. Of these 25 digital gamified tools, 11 (44%) were web-based tools, 8 (32%) mobile (native mobile or mobile-enabled web) apps, and 6 (24%) virtual reality tools. Overall, tools that were evaluated showed increases in knowledge and intentions to receive vaccines, mixed effects on attitudes, and positive effects on beliefs. We did not observe discernible advantages of one type of digital gamified tool (web based, mobile, virtual reality) over the others. However, a few studies were randomized controlled trials, and publication bias may have led to such positive effects having a higher likelihood of appearing in the peer-reviewed literature.

    Conclusions: Digital gamified tools appear to have potential for improving vaccine uptake by fostering positive beliefs and increasing vaccine-related knowledge and intentions. Encouraging comparative studies of different features or different types of digital gamified tools could advance the field by identifying features or types of tools that yield more positive effects across populations and contexts. Further work in this area should seek to inform the implementation of gamification for vaccine acceptance and promote effective health communication, thus yielding meaningful health and social impacts.

    JMIR Serious Games 2024;12:e47257

    doi:10.2196/47257

    Keywords

    digital gamified tools; digital game; vaccination; gamification; vaccine uptake; scoping review; review method; vaccine; gamified; COVID-19; COVID; SARS-CoV-2; health behaviour; health behavior; health promotion; behavior change; behaviour change 


    Vaccination is one of the most cost-effective methods of preventing the spread of vaccine-preventable diseases. If vaccination coverage falls below the thresholds that are safe for the prevention of epidemic transmission, the incidence of vaccine-preventable diseases increases [1,2]. For example, measles returned over the past 2 decades, and the incidence of measles in the European Union increased in 2017-2018 [3].

    In 2019, prior to the COVID-19 pandemic, the World Health Organization identified vaccine hesitancy (ie, the reluctance or refusal to be vaccinated despite the availability of vaccination services) as 1 of the top 10 threats to worldwide health [4]. Vaccine hesitancy is one of the several reasons some people are un- or undervaccinated [5-9]. Interventions addressing vaccine hesitancy are therefore necessary to promote vaccine acceptance and uptake. As the contributors of vaccine acceptance are diverse, no single intervention will solve this issue [10]. Multicomponent interventions tailored to local barriers to vaccine acceptance and uptake are known to be the most effective [11,12]. Misinformation and conspiracy theories spread online, where extensive antivaccine content is shared [13-15], potentially negatively influencing views about vaccines [16,17]. Efforts have been made to counter vaccine misinformation and mistrust by targeting various groups, such as parents, non–health care workers [18,19], and adolescents [20], and delivering information about the risks and benefits of different types of vaccines, for instance, human papillomavirus (HPV) vaccination [21] and measles, mumps, and rubella vaccines [22,23]. Along with traditional communication strategies, the use of other strategies to inform and educate about immunizations, for example, with digital gamified tools, may help encourage vaccine uptake.

    Gamification is defined as the use of game design elements in nongame contexts [24]. It includes several aspects and features, such as fun interfaces, immediate success or feedback, reward systems (levels, point scores, badges), challenges and competitions, team playing, avatars, and quizzes. Previous studies have used gamification to achieve desired health behaviors [25-27] or facilitate their learning about health [28]. Gamification draws on elements from serious games, meaning fully developed digital games used to train and educate players [29,30]. For example, a serious game “Land of Secret Gardens” facilitates conversations about HPV with preteens. In the game, preteens need to protect their bodies with a “potion,” which offers a metaphor for the HPV vaccine [31]. However, serious games and digital gamified tools are distinct but related concepts. Serious games use gaming as a central and primary medium [32]. In contrast, digital gamified tools (eg, apps) or gamified interventions are not complete game experiences but have gaming features, such as rewards systems, scoring of points, or engaging users in different challenges [33]. In this study, we defined digital gamified tools as digital apps with the aforementioned gaming features. Our definition includes serious games that meet the criteria, that is, they must include such gaming features. This scoping review provides insight into the reported effects of digital gamified tools to increase vaccine uptake. Our review built upon existing reviews in the field by including a comprehensive search of both published literature and online tools, as well as an examination of both the characteristics and the reported effects of these digital tools. This review was distinct in that it focused specifically on digital gamified tools and their effects, rather than simply the effectiveness of gamification in general. In doing so, this review aimed to fill a gap in the literature by providing evidence-based answers to the question of whether gamification “works” to increase vaccine uptake. Therefore, the objectives of this scoping review [34] were, first, to review digital gamified tools that have been implemented or evaluated across various populations to encourage vaccine uptake and, second, to describe any reported effects of the identified tools in terms of influence on users’ knowledge or behavior toward vaccination. Our research questions can therefore be summarized as follows:

    • What digital gamified tools intended to encourage vaccination exist and have been described in the literature?
    • Do these tools demonstrate any effects on knowledge, attitudes, beliefs, and behaviors about vaccination?

    Search Strategy

    For peer-reviewed papers, we searched Medline (Ovid), Embase (Ovid), CINAHL (EBSCO), the Web of Science Core Collection, the Cochrane Database of Systematic Reviews (Ovid), the Cochrane Central Register of Controlled Trials (Ovid), Academic Search Premier (EBSCO), PsycInfo (Ovid), Global Health (Ovid), and ERIC (Ovid) with no language or date restrictions. The proposed search terms were, for example, “vaccine,” “vaccination,” “immunization,” “video games,” “gamification,” “application,” and “virtual reality” (see Multimedia Appendix 1 for the full search strategy). The search was conducted on January 26 and 27, 2022.

    We also conducted an online Google search on May 5, 2022, for any digital tools with gamified features that deliver informative or educative messages on vaccination. The search terms we used were “vaccination,” “immunization,” “electronic game,” “computer game,” “mobile game,” “interactive game,” and “digital game” (see Multimedia Appendix 1 for the full search strategy). We reviewed the first 30 results for each search, as it is rare for users to click past the third page of 10 search results per page, and therefore, researchers analyzing medical content available on the web often use 30 as a threshold [35-37]. On May 6, 2022, we conducted the same searches in private browsing mode to ascertain whether our results had been affected by a “filter bubble” [38], that is, the way Google search results are adapted to one’s previous browsing activity. Our search strategy was constructed and reviewed by 2 librarians. Following the librarians’ advice, we expanded our search strategy to include ERIC and Global Health databases.

    Study Selection and Screening Process

    We used PICO (Population, Intervention, Comparison, and Outcome) to structure study inclusion and exclusion criteria (Table 1). Our population of interest was the general public or any subgroup, including health care professionals and students. We sought studies describing tools with gamification techniques or gamified elements, including gamified web-based quizzes to deliver informative or educative messages on vaccination. Posters, preprints, editorials, conference proceedings, news bulletins, and paper-based or board games were excluded. Our comparator was any control, including offering no education on vaccination or comparing participants before and after an intervention. Our outcomes of interest included common outcomes associated with vaccine uptake, namely knowledge (comprehension, understanding), attitudes (for or against vaccination), beliefs (perceived benefits, perceived risks), and behaviors toward vaccines (vaccination intention [ie, intention to get vaccinated or not get vaccinated] and vaccine uptake [ie, receiving or not receiving a vaccine]). We excluded papers that did not present the description or evaluation of a concrete digital gamified tool.

    Table 1. Inclusion and exclusion criteria.
    ComponentInclusion criteriaExclusion criteriaQuestion related to the criteria
    Type of report
    • Original paper
    • Evaluated intervention or digital gamified tool
    • Posters, preprints, and conference proceedings
    • Modeling or simulation study
    • Brochures
    • Editorials
    • Bulletins
    		Has the study or research described the development of the tool and evaluated it?
    Population
    • General public (any subgroup)
    • Professionals
    • Students
    		N/AaWho is the audience for whom the key message was intended?
    Intervention
    • Tools with gamification technique or gamified elements, including gamified web-based quizzes to deliver informative or educative messages on vaccination
    • Any study or gamification tools not intended for vaccination/vaccine uptake
    • Studies or apps to reduce vaccine pain and fears and to report immunization status or record keeping, surveillance or vaccine coverage apps, contact-tracing or early detection apps
    • Paper games, board games (not digital)
    • Videos with no gamified element included
    		Does the study or tool aim to deliver an informative or educative message on vaccination?
    Comparator
    • Any control, including offering no education or no digital gamified tool
    		N/AN/A
    Outcome
    • Common outcomes that encourage vaccine uptake: knowledge (comprehension, understanding), attitudes (for or against vaccination, beliefs (risk perception, etc), behaviors toward vaccines (vaccination intention [ie, intention to get vaccinated or not get vaccinated] and vaccine uptake [ie, receiving or not receiving a vaccine])
    • Outcomes not related to the encouragement of vaccine uptake
    		Has the study or tool been evaluated for the outcomes that encourage vaccine uptake?

    aN/A: not applicable.

    For Google-searched digital gamified tools, our inclusion and exclusion criteria used the same specifications regarding population and intervention. We did not apply comparison and outcome criteria to web-based tools because we did not expect these to report evaluation studies.

    We reported this review according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (see the PRISMA checklist in Multimedia Appendix 2) [39]. We registered our protocol on the Open Science Framework [40].

    Expert Consultations

    After extracting information from peer-reviewed papers and tools identified via a Google search, we contacted experts in the field of digital gamified tools (eg, developers and researchers working on the topic in Canada and worldwide who were already known to the research team) to complement our online searches and ensure completeness. Specifically, we sent emails to 12 experts about the results of our searches and asked them to alert us to any games or papers we might have missed.

    Data Charting

    We developed a form in Microsoft Excel to guide the charting of data. We pretested and reviewed the form with team members to ensure we were accurately and adequately capturing relevant data. Data charting occurred independently with verification. Specifically, a reviewer (author HH) identified and screened all studies and digital gamified tools for their eligibility. Screening results were verified by a second reviewer (author DG). The data charting was then performed by a reviewer (HH) and again verified by a second reviewer (DG). Any conflicts throughout screening or data charting were resolved by a third reviewer (author ED). From the included papers, we charted data about (1) the type and design of study (developmental or evaluation study, user testing, randomized controlled trial, etc), (2) the vaccine(s) addressed (COVID-19, HPV, etc), (3) the purpose of the study or intervention, (4) the digital gamified tool platform (web based, native mobile app, mobile-enabled web app, virtual reality), and (5) the characteristics of study participants. For the evaluated interventions, we charted data about preselected outcomes that are widely used to predict health-related behaviors and to assess outcomes in studies of interventions about vaccination and immunization [11-14]. Specifically, we extracted data about the tools’ effects on knowledge, attitudes, beliefs (perceived benefits, perceived risks), and behavioral intentions. Emotional, cultural, and social factors can also influence a decision about vaccination [29,30]. Therefore, we also extracted data about other outcomes that the studies may have evaluated. Because we sought to understand all possible effects, we did not prespecify any of these as a primary outcome. We organized the extracted data in tables and synthesized them descriptively.

    Quality Assessment

    To assess the quality of the studies that evaluated their interventions, we used the Mixed Methods Appraisal Tool (MMAT) developed by Pluye et al [41]. Two reviewers independently conducted the quality assessment, resolving disagreements through discussion until reaching a consensus. A third and a fourth reviewer (authors HH and HW) intervened to settle any remaining conflicts.

    Data Synthesis

    We summarized data using a narrative approach involving framework and content analysis. We classified each digital gamified tool platform using the 4 types of digital gamified tools: web-based tool, native mobile app, mobile-enabled web app, virtual reality tool. For the type of digital gamified tool, we classified web-based tools that explicitly noted their suitability for mobile use (eg, by smartphone or tablet) as mobile-enabled apps. We classified web-based tools without such an explicit statement as web based only, even though they may be functional on mobile devices. For the type and design of study, we grouped randomized designs together, including traditional randomized controlled trials with only 2 study arms and factorial designs with more than 2 study arms. Although these methods are not exactly the same, they all use randomization to minimize potential biases and are therefore functionally equivalent for our purposes of understanding what kinds of evaluations have been undertaken [42]. We summarized the main characteristics of tools, including PICO elements, in a tabular display. We used the PRISMA 2020 flowchart to describe the process of study selection [43].


    Papers Identified and Scope of Literature

    We identified a total of 2082 records through database searches. After removing duplicates, we screened 1402 (67.3%) database records. Through Google searches, we identified 10 digital gamified tools and 2 papers. In a private browsing mode search, there was no change in search results. Of the 12 experts contacted, 2 (17%) responded and suggested 2 papers and 2 links, of which 1 (50%) digital gamified tool met the inclusion criteria and was included in our review. Through these methods, our final data set included 28 (2%) peer-reviewed papers and 11 digital gamified tools. Figure 1 shows our PRISMA diagram.

    ‎
    Figure 1. PRISMA flow diagram. PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

    Of the 28 peer-reviewed papers, 7 (25%) explained the development of the tool, 16 (57%) described evaluation, and 2 (7%) reported both development and evaluation of the tool (Table 2). To report our results, we grouped studies together that reported the same tool, meaning 28 peer-reviewed papers reporting on 25 different tools. Of these 25 digital gamified tools, 11 (44%) were web-based tools, 7 (28%) mobile (native mobile or mobile-enabled web) apps, 6 (24%) virtual reality tools, and 1 (4%) offered in both mobile and web-based versions (for details, see Table 2). The most common single vaccines addressed in the tools were influenza (n=6, 24%, tools) and HPV (n=6, 24%, tools). Other tools addressed COVID-19 (n=2, 8%); measles, mumps, influenza, and smallpox (n=2, 8%); a hypothetical disease (n=2, 8%); other vaccine-preventable diseases (n=6, 24%); and the role vaccines play in preventing the spread of disease with no particular vaccine specified (n=1, 4%). Of the 10 digital gamified tools identified via a Google search and 1 suggested by the expert (a total of 11 digital gamified tools; see Table 3), the largest group (n=5, 45%) addressed COVID-19, and the rest were about other vaccine-preventable diseases. The 11 gamified elements identified in the Google search and expert feedback identified 6 types of gamified elements: reward points, serious games, physical trading cards, certificates, role-playing, and quizzes (see Table 3). The most common type was reward points, which appeared in 5 (45%) cases. Two cases used serious games, one case used physical trading cards and reward points, one case used certificates, one case used role-playing, and one case used quizzes. Additional characteristics of the studies included (eg, country of origin, sample size, participant characteristics) are detailed in Multimedia Appendix 3 [31,44-70]. The expanded versions of Table 2 [31,44-70] and Table 3 [71-81] are provided in Multimedia Appendix 4.

    Table 2. General information about the studies.
    Type of study and author(s)Type of digital gamified tool platformType of disease/vaccineType and design of study (development or evaluation, iterative design, randomized controlled trial, etc)
    Evaluation studies

    		Betsch and Böhm [44]Web-based toolHypotheticalEvaluation: online experiment

    		Carolan et al [45]Web-based toolMeasles, mumps, influenza, and smallpoxEvaluation: pre-post study

    		Cates et al [31]Web-based toolHPVaEvaluation: pilot randomized controlled trial

    		Dale et al [46]Native mobile appInfluenzaEvaluation: nonrandomized trial

    		Darville et al [47]Web-based toolHPVEvaluation: randomized controlled trial

    		Eley et al [48], McNulty et al [49]Web-based toolBacteria, vaccine-preventable diseaseEvaluation: quantitative followed by qualitative research design

    		Fadda et al [50], Fadda et al [51]Native mobile appMMR vaccinesEvaluation: mixed methods research design

    		Ibuka et al [52]Web-based toolHypothetical diseaseEvaluation: experimental design

    		Kaufman and Flanagan [53]Web-based toolNot reportedEvaluation: experimental design

    		Lee et al [54]Native mobile appInfluenzaEvaluation: randomized controlled trial

    		Mitchell et al [55], Laplana [56]Web-based toolInfluenzaEvaluation: pre-post study

    		Mottelson et al [57]Virtual reality toolCOVID-19Evaluation: randomized controlled trial (2×2 factorial design)

    		Nowak et al [58]Virtual reality toolInfluenzaEvaluation: one-way between-subjects design with random assignment

    		Real et al [59]Virtual reality toolInfluenzaEvaluation: quasi- randomized controlled trialb

    		Woodall et al [60]Mobile-enabled web appHPVEvaluation: clinic-cluster randomized trial

    		Vandeweerdt et al [61]Virtual reality toolCOVID-19Evaluation: randomized controlled trial
    Development studies

    		Amresh et al [62]Web-based toolHPVDevelopment: iterative design

    		Bertozzi et al [63] (data extracted for the game related to vaccines)Web-based toolInfluenzaDevelopment: iterative design

    		Carolan et al [64]Web-based toolMeasles, mumps, influenza, and smallpoxDevelopment: iterative design

    		de Araujo Lima et al [66]Native mobile appVaccine-preventable diseasesDevelopment: heuristic evaluation by users, content evaluation by experts

    		Kafai et al [65]Virtual realityDragon swooping cough virus to reflect real-life features of infectious viruses, such as Ebola.Development: user feedback via surveys (asking users questions) and log files (observing user behaviors)

    		Real et al [67]Native mobile appHPVDevelopment: usability testing

    		Streuli et al [68]Virtual realityPediatric vaccinesDevelopment: Community-based participatory research and co-design
    Development and evaluation studies

    		Davies et al [69]Mobile or web app (multiple formats available)Hepatitis BDevelopment and evaluation: Participatory Action Research

    		Ruiz-López et al [70]Native mobile appHPVDevelopment and evaluation: Iterative design and evaluation via questionnaire

    aHPV: human papillomavirus.

    bAllocation to a study arm was performed according to work schedules, which are often arbitrary. We therefore considered this quasi-randomization.

    Table 3. Tools from Google search and expert suggestions.
    Digital gamified toolType of disease/vaccineType of digital gamified tool platformGamification elements (eg, rewards, role-playing, leaderboard, serious game)
    Antidote COVID-19 [71]COVID-19Native mobile appReward points
    The Vaccination Game [72]H11N7 and influenzaWeb-based toolSerious game
    Help take down COVID-zilla! [73]COVID-19Web-based toolRole-playing
    Just the Vax! [74]Vaccine-preventable diseaseWeb-based toolReward points
    COVID Invaders [75]COVID-19Web-based toolReward points
    Vax Pack Hero [76]Vaccine-preventable diseaseWeb-based toolReward points and physical trading cards
    Flu\'s Clues [77]InfluenzaWeb-based toolCertificate of completion for solving the influenza mystery
    Virus Fighter [78]COVID-19, influenza, Ebola, measlesWeb-based toolSerious game
    Immunization411: for preteens and teens’ online training [79]Tdap meningococcal vaccine, varicella, HPVa, influenzaWeb-based toolReward points
    COVID Chronicles [80]COVID-19Web-based toolReward points
    I Boostb [81]Vaccine-preventable diseaseWeb-based toolQuiz

    aHPV: human papillomavirus.

    bSuggested by an expert.

    The studies were conducted in 26 different countries, with the majority of studies coming from the United States (n=13, 46%, studies) and the United Kingdom (n=5, 18%, studies). Study populations included students at various levels (elementary school to college, specialty programs, eg, nursing and pediatric residency), parents of vaccine-eligible children, adults from the general population, members of particular sociocultural communities (eg, immigrants, Indigenous peoples), and convenience samples, such as players of a game, attendees of a conference, and employees of an organization. Sample sizes ranged from 8 to 50,286. Whenever papers reported study participant characteristics such as age, sex, gender, ethnocultural identity, or socioeconomic levels, we extracted summary data, as shown in Multimedia Appendix 3.

    Reported Effects of Evaluated Interventions

    In total, 18 (64%) of 28 studies evaluated at least 1 of our outcomes of interest, while 11 (39%) studies reported the effects of the evaluated interventions on more than 1 outcome of interest. Summarized outcomes and their MMAT quality assessments are shown in Table 4. Multimedia Appendix 5 provides full details.

    Table 4. Outcomes of evaluation studies included.
    Type of digital gamified tool platform and studyKnowledge (comprehension/understanding, etc)Attitudes (for/against vaccination, etc)Beliefs (risk perceptions, etc)Behavioral intentions (getting vaccinated or not, etc)Others (eg, emotions)MMATa quality score
    Web-based tool

    		Betsch and Böhm [44]—bNegative vaccine attitudes with compulsory vaccination—Decreased vaccine uptake with compulsory vaccinationIncreased level of anger with compulsory vaccination60% quality criteria met

    		Carolan et al [45]—No significant effect on attitudes towards vaccination——Increased confidence in information needs80% quality criteria met

    		Cates et al [31]Increase in knowledge about immunization——Positive increase in intentions to vaccinateIncrease in vaccination self-efficacy, decisional balance towards vaccination100% quality criteria met

    		Darville et al [47]——Positive effects on beliefs towards vaccinationIncrease in intentions to vaccinate—60% quality criteria met

    		Eley et al [48], McNulty et al [49]Improvements in knowledge about immunization————100% quality criteria met

    		Ibuka et al [52]————Free riding in vaccination decisions decreases vaccine acceptance80% quality criteria met

    		Kaufman and Flanagan [53]The digital version of the game was less effective at facilitating learningThe digital version of the game was less effective at attitude change——The digital version of the game was perceived to be complicated to use20% quality criteria met

    		Mitchell et al [55], Laplana [56]Increase in knowledgePositive increase in attitudes for vaccination—Increase in vaccine uptake after accessing the game—80% quality criteria met (Mitchell et al [55])
    Mobile app

    		Dale et al [46]———Positive increase in intentions to vaccinate—80% quality criteria met

    		Fadda et al [50], Fadda et al [51]Improvements in knowledge about immunization——Increase in intentions to vaccinateIncrease in psychological empowerment and confidence in the decision80% quality criteria met (Fadda et al [50], Fadda et al [51])

    		Lee et al [54]———Increase in intentions to vaccinate—80% quality criteria met

    		Woodall et al [60]——Increase in beliefs towards vaccinationIncrease in intentions to vaccinateIncrease in vaccine confidence40% quality criteria met

    		Ruiz-López et al [70]Increase in knowledge after playing the game————100% quality criteria met
    Virtual reality tool

    		Mottelson et al [57]———Increase in vaccination intention when both the personal and collective benefit of COVID-19 vaccination was communicatedIncrease in COVID-19 empathy, vaccination recommendation, and vaccination readiness80% quality criteria met

    		Nowak et al [58]——Positive effects on beliefs towards vaccinationIncrease in intentions to vaccinate—100% quality criteria met

    		Real et al [59]—Increase in attitudes in favour of vaccination———60% quality criteria met

    		Vandeweerdt et al [61]———Increase in intentions to vaccinateVirtual reality intervention increases a sense of collective responsibility100% quality criteria met
    Mobile or web app (multiple formats available)

    		Davies et al [69]Increase in knowledge about immunization————80% quality criteria met

    aMMAT: Mixed Methods Appraisal Tool.

    bNot reported.

    Effects on Knowledge (Includes Comprehension/Understanding, etc)

    Overall, the 28 included studies suggested that digital gamified tools may positively influence knowledge. Of 7 (25%) studies that assessed knowledge, 6 (86%) showed an increase in knowledge about immunization in general [31,48,51,55,69,70]. All these 6 (86%) studies were of high quality (≥80%). One study of low quality (≤25%) reported that a digital game is less effective at increasing knowledge compared to its original board game format [53]. When considering only the high-quality (≥80%) studies, we observed that digital gamified tools are associated with increased knowledge.

    Effects on Attitudes (for or Against Vaccination)

    Overall, digital gamified tools appeared to have mixed effects on attitudes toward vaccination. Of 5 (18%) of 28 studies that assessed attitudes, 2 (40%), one of high quality (≥80%) and the other of medium quality (60%), showed an increase in positive attitudes toward vaccination [55,59]. In addition, 2 (40%) studies, one of high quality (≥80%) and the other of low quality (20%), reported no or less effect on attitudes toward vaccination [45,53], and 1 (20%) study comparing voluntary and compulsory vaccines in a game context showed negative attitudes regarding compulsory vaccination [44]. When considering only the high-quality (≥80%) studies, we observed inconsistent effects on attitudes.

    Effects on Beliefs (Perceived Benefits, Perceived Risks)

    Overall, digital gamified tools demonstrated positive effects on beliefs toward vaccination. In total, 3 (11%) of 28 studies, 1 (33%) of high quality (100%) and 2 (67%) of medium quality (60% and 40%), evaluated the effects of digital gamified tools on beliefs toward vaccination. All 3 (100%) studies showed positive effects on beliefs toward vaccination [47,58,60]. When considering only the high-quality (≥80%) studies, we observed that digital gamified tools are associated with positive beliefs about vaccines.

    Effects on Behavioral Intentions

    Overall, the 28 included studies suggested that digital gamified tools may positively influence intentions to receive vaccines. In total, 11 (39%) studies evaluated the effects of digital gamified tools on behavioral intentions with regard to vaccines. Of these 11 studies, 1 (9%) of medium quality (60%) showed a decrease in vaccination intention when compulsory vaccination was introduced within a game context [44], whereas 10 (91%) studies, 3 (30%) of medium quality (60% and 40%) and 7 (70%) of high quality (≥75%), showed increased intentions to vaccinate [31,46,47,51,54,55,57,58,60,61]. When considering only the high-quality (≥80%) studies, digital gamified tools appeared to be consistently associated with increased vaccination intention.

    Other Outcomes

    In total, 9 (32%) of 28 studies have also evaluated the effects of digital gamified tools on other outcomes. Of these, 4 (44%) studies reported an increase in confidence in vaccines (medium quality=40%) [60], confidence in information needs (high quality=80%) [45], decisional balance in support of vaccination (high quality=100%) [31], and confidence in vaccine decisions (high quality=80%) [50]. In addition, 1 (11%) study of high quality (80%) reported an increase in empathy toward those vulnerable to COVID-19 and vaccination recommendations [57], and 2 (22%) studies of high quality (100% and 80%) reported an increase in vaccination self-efficacy and readiness [31,57]. An increase in psychological empowerment (high quality=80%) [51] and in emotions such as anger toward compulsory vaccination (medium quality=60%) [44] was also reported by 2 (22%) studies. One study of high quality (80%) reported that the concept of free riding decreases vaccine acceptance [52], whereas another study of high quality (100%) reported that virtual reality intervention increases collective responsibility [61]. When considering only the high-quality (≥80%) studies, we observed a variety of positive effects associated with digital gamified tools, including confidence in vaccines, confidence in decisions about vaccines, empathy toward vulnerable people, collective responsibility, psychological empowerment, and vaccination self-efficacy and readiness.

    Effects of the Platform (Web Based, Mobile, Virtual Reality)

    The study designs of the 28 included papers did not permit us to formally compare the effects of different platforms in a robust way. Upon inspection, there did not appear to be a strong effect of the platform. In other words, we did not observe evidence in favor of web-based, mobile, or virtual reality apps over the other 2 types of apps.


    Principal Findings

    The broad objective of this scoping review was to map the state of the science regarding digital gamified tools and their effects. In other words, we wished to answer a common question at the intersection of public health and digital health: does gamification encourage vaccination and influence knowledge, attitudes, beliefs, and behaviors related to vaccination? By mapping both published literature and tools currently available online, we observed 2 principal findings.

    First, our results suggest that gamification can increase predictors of vaccine uptake, such as knowledge, attitudes, beliefs, behaviors, and vaccination intention. This finding is similar to the findings of a previous review by Montagni et al [82] suggesting that gamification can contribute to changed behaviors and improved knowledge of vaccination. Similarly, other reviews have suggested the potential benefits of gamification for non–vaccination-related behavior change, such as a systematic review suggesting that gamification interventions could be a feasible way to improve health-related outcomes among cancer survivors [83] and another review suggesting their effectiveness in improving physical activity [84]. Such previous work became even more relevant during the COVID-19 pandemic, as many jurisdictions sought to optimize vaccine uptake in the context of an “infodemic” (ie, overabundance of information, true, false, and misleading, about the pandemic and recommended preventive behaviors) [85]. Half of the digital gamified tools identified in our web search addressed COVID-19, suggesting an active interest in using a gamified approach in the pandemic context. Recent research by Plechatá et al [86] published after our data extraction steps were complete suggested that explaining the concept of herd immunity with gamification has a positive impact on the COVID-19 vaccination intention.

    Second, our review suggests that although gamification has the potential to enhance the impact of education strategies, gamified tools alone may not wholly address gaps in vaccine acceptance and uptake. Although some of the identified tools did increase vaccination, the increases did not fully close gaps between previous and desired vaccine uptake. This finding aligns with those of Tozzi et al [87], which suggested that promising results could be achieved by combining gamification with educative and informative tools to improve immunization programs. This finding also aligns with previous reviews suggesting the use of digital gamified interventions as a public health tool of interest in enhancing vaccine uptake [82,88]. Further research published by Real et al [89] after our systematic search similarly observed that integrating gamification, such as virtual reality, in training modules enhances uptake of the HPV vaccine. Integrating gamified features may work because they make digital tools acceptable and more fun to use and may reduce the chances of people feeling pushed toward vaccination. In parallel, gamification may be a promising strategy for increasing knowledge, skills, and confidence among health professionals engaging in discussions about vaccines with their patients [90,91].

    In addition to these findings drawn directly from our review of the included tools, we offer a broader observation based on the contents of this scoping review, along with the larger landscape of vaccine acceptance research: context is key. Although an engaging approach may work for some groups or in some situations, it may be less well accepted among other groups and in other situations. For instance, a casual and approachable style of communication will work for the younger audience to convey vaccine information but might be deemed insufficient to health care professionals in a more formal setting, such as hospitals. A good understanding of the factors associated with low vaccine acceptance at the local level is needed prior to developing gamified tools [92]. Future research in this area should consider possible contextual factors, such as local culture, social and demographic characteristics of users, and different influences on vaccine hesitancy and acceptance in different regions. To help better match games to the context(s) in which they will be played, when developing games, developers and researchers may wish to consider involving potential players from different contexts early and often. This aligns with previous work [93,94] suggesting that involving users earlier in developing tools may help in designing interventions suitable for a targeted context. One of the examples in our review was an intervention by Cates et al [31] designed to explain HPV vaccines to teenagers using a “secret garden” theme. Involving potential game players early in the development of the game may have contributed toward its positive effects on vaccination intention.

    The implications of this research extend beyond the immediate reported effects of gamified tools and delve into the strategic dimensions of public health policy and communication efforts. Considering the insights gleaned from the findings, this study supports a comprehensive and well-informed approach to integrating gamification into strategies for promoting vaccination. As gamification continues to demonstrate its potential in enhancing vaccine uptake, it is crucial to navigate this terrain thoughtfully, considering the various factors that influence its impact. This includes not only the technological and behavioral aspects but also the larger sociocultural context in which vaccination decisions are made. Therefore, our study emphasizes the importance of a comprehensive approach that fosters a mutually beneficial relationship between technological innovation, evidence-based strategies, and an intricate understanding of local contexts. This approach has the potential to make gamification a sustainable and adaptable tool in the arsenal of public health interventions, rather than just a passing trend.

    The review does not find a clear advantage for any platform in terms of reported effects. It was challenging to measure the impact of the platforms on behavioral outcomes and calls for more focused research to better understand the specific elements within each platform that drive behavior change. In essence, our study suggests that the reported effects of an app may not be solely determined by its platform but rather by the strategic incorporation of mechanics and elements that facilitate the desired behavior change.

    Gamification can influence knowledge, attitudes, and beliefs about vaccines, which can affect vaccine uptake. This is consistent with theories of change proposing that cognitive changes can lead to behavioral outcomes. Although our study mainly examines the immediate effects of gamification on these cognitive aspects, it also offers some implications for using gamification as a potentially viable strategy to improve vaccine acceptance.

    Strengths and Limitations

    Our study has 5 main limitations. First, because we aimed to capture all relevant evidence and examples, as is typical in a scoping review, we included a broad range of study designs and did not draw conclusions about the relative advantages or disadvantages of different game platforms and features. Given the rapid growth within this field of research, it would be difficult to truly prioritize evidence according to quality criteria at this point. In the future, it may be possible to conduct a systematic review and meta-analysis, restricting included studies to randomized experiments or randomized controlled trials. Such future work may include approaches such as a network meta-analysis to allow for comparison of the effects of different game types or game features. Based on the existing literature, it is difficult to conclude whether certain games are more or less likely to achieve their aims. Second, our results may be influenced by publication bias. It is possible that groups that have developed digital gamified tools that showed disappointing results simply did not publish their studies. This bias could lead to an overestimation of the reported effects of these tools. This highlights the importance of further research to fully understand the real impact of these tools and thus accurately inform policy decisions about the development and use of these tools. Third, and related to the previous 2 points, the rapid growth in this area may mean that we missed more recent evidence in literature published after January 2022 and web searches after May 2022. Fourth, the majority of digital gamified tools on vaccination represented in publications and online were developed in high-income countries. This finding aligns with the findings of previous work by Ohannessian et al [88], who also reported a predominance of high-income countries. This may reflect more widespread internet access and resources for developing digital gamified tools in high-income countries. It may also reflect publication bias in the scientific literature (ie, there may be fewer papers written about digital gamified tools in lower-income countries) and online (ie, tools developed and published in lower-income countries may not be ranked highly by search engines and therefore may not have appeared in our web searches). Tools developed in lower-income countries may also take different forms; for example, they may be text message–based interventions (with or without gamification) rather than web-based tools and therefore would be less likely to be identified in web searches. Analog games from high-income countries were similarly excluded from the scope of our study [95]. Nondigital games, such as board and card games, have demonstrated positive impacts on educational knowledge, cognitive function, and social interactions [96,97]. Such games can support diverse learning across subjects and settings, fostering interactions that develop skills, such as computational thinking and teamwork, and have positive impacts on academic achievement and vocabulary acquisition compared to digital games [97-99]. We restricted our scoping review to digital gamified tools because the review was intended to provide an evidence base for digital game development. Although nondigital games are also potentially useful interventions, the implementation and distribution of such interventions is more challenging, especially in a geographically dispersed country, such as Canada. Fifth, and finally, as we used Google and private browsing in Google, there may be a possibility that different search engines would provide different results.

    This study also has 2 main strengths. First, by systematically examining the current literature and currently available tools online, we were able to offer an updated overview of the potential effects of including gamification in digital tools about vaccination. Second, by conducting a scoping review to broadly map the literature, future work can more easily identify and select key outcomes for systematic reviews and meta-analyses in this domain.

    Conclusion

    Digital gamified tools have the potential to improve vaccine uptake by increasing knowledge and promoting positive attitudes, beliefs, behaviors, and vaccination intention. Further evaluations of these innovative digital tools, including head-to-head comparisons of different features and different platforms, will add more knowledge about what works and what does not in order to achieve public health goals more efficiently. In the wider context of health policy, digital gamified tools may be useful components of multifaceted strategies to improve vaccination rates throughout society.

    Acknowledgments

    This work was supported by a research grant from the Canadian Institutes of Health Research (GA3177725). The authors gratefully acknowledge the assistance of Frédéric Bergeron (librarian) for reviewing the search strategy. We appreciate the assistance of Charles Racine and Créscence Joëlle Mefou Tasong in conducting the quality appraisal of the studies that evaluated their interventions.

    Authors' Contributions

    All authors provided substantial contributions to this paper’s conception and edits and approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Search strategy.

    DOCX File , 20 KB
    Multimedia Appendix 2

    PRISMA-ScR checklist.

    PDF File (Adobe PDF File), 498 KB
    Multimedia Appendix 3

    Characteristics of the studies included in the review.

    DOCX File , 25 KB
    Multimedia Appendix 4

    Expanded version of Table 2 (general information about the studies) and Table 3 (tools from Google search and expert suggestions).

    DOCX File , 28 KB
    Multimedia Appendix 5

    MMAT quality assessment. MMAT: Mixed Methods Appraisal Tool.

    XLSX File (Microsoft Excel File), 92 KB
    1. Omer SB, Salmon DA, Orenstein WA, deHart MP, Halsey N. Vaccine refusal, mandatory immunization, and the risks of vaccine-preventable diseases. N Engl J Med. May 07, 2009;360(19):1981-1988. [CrossRef]
    2. Phadke VK, Bednarczyk RA, Salmon DA, Omer SB. Association between vaccine refusal and vaccine-preventable diseases in the United States: a review of measles and pertussis. JAMA. Mar 15, 2016;315(11):1149-1158. [FREE Full text] [CrossRef] [Medline]
    3. Plans-Rubió P. Low percentages of measles vaccination coverage with two doses of vaccine and low herd immunity levels explain measles incidence and persistence of measles in the European Union in 2017-2018. Eur J Clin Microbiol Infect Dis. Sep 10, 2019;38(9):1719-1729. [CrossRef] [Medline]
    4. Ten health issues WHO will tackle this year. World Health Organization. URL: http://tinyurl.com/39cd7tuf [accessed 2021-10-04]
    5. Attwell K, Betsch C, Dubé E, Sivelä J, Gagneur A, Suggs LS, et al. Increasing vaccine acceptance using evidence-based approaches and policies: insights from research on behavioural and social determinants presented at the 7th Annual Vaccine Acceptance Meeting. Int J Infect Dis. Apr 2021;105:188-193. [FREE Full text] [CrossRef] [Medline]
    6. MacDonald NE, Dube E, Comeau JL. Have vaccine hesitancy models oversimplified a complex problem to our detriment? The Adapted Royal Society of Canada vaccine uptake framework. Vaccine. Jun 23, 2022;40(29):3927-3930. [FREE Full text] [CrossRef] [Medline]
    7. Dubé E, Gagnon D, MacDonald N, Bocquier A, Peretti-Watel P, Verger P. Underlying factors impacting vaccine hesitancy in high income countries: a review of qualitative studies. Expert Rev Vaccines. Nov 2018;17(11):989-1004. [CrossRef] [Medline]
    8. MacDonald NE, Comeau J, Dubé; Graham J, Greenwood M, Harmon S, et al. Royal society of Canada COVID-19 report: enhancing COVID-19 vaccine acceptance in Canada. FACETS. Jan 01, 2021;6(1):1184-1246. [CrossRef]
    9. Dolu İ, Turhan Z, Yalnız Dilcen H. COVID-19 vaccine acceptance is associated with vaccine hesitancy, perceived risk and previous vaccination experiences. Disaster Med Public Health Prep. Dec 23, 2021;17:e97. [FREE Full text] [CrossRef] [Medline]
    10. Behavioural and social drivers of vaccination: tools and practical guidance for achieving high uptake. World Health Organization. URL: http://tinyurl.com/3uteanyc [accessed 2023-01-10]
    11. Dubé E, Gagnon D, MacDonald NE, SAGE Working Group on Vaccine Hesitancy. Strategies intended to address vaccine hesitancy: review of published reviews. Vaccine. Aug 14, 2015;33(34):4191-4203. [FREE Full text] [CrossRef] [Medline]
    12. Omer SB, Benjamin RM, Brewer NT, Buttenheim AM, Callaghan T, Caplan A, et al. Promoting COVID-19 vaccine acceptance: recommendations from the Lancet Commission on Vaccine Refusal, Acceptance, and Demand in the USA. Lancet. Dec 11, 2021;398(10317):2186-2192. [FREE Full text] [CrossRef] [Medline]
    13. Mills M, Rahal C, Brazel D, Yan J, Gieysztor S. COVID-19 vaccine deployment: behaviour, ethics, misinformation and policy strategies. The Royal Society; the British Academy. URL: http://tinyurl.com/2bc87r5f [accessed 2024-02-10]
    14. Oehler RL. On measles, vaccination, social media activism and how to win back our role as our patients’ best advocates. Clin Infect Dis. Jan 02, 2020;70(2):338-340. [CrossRef] [Medline]
    15. Tasnim S, Hossain M, Mazumder H. Impact of rumors or misinformation on coronavirus disease (COVID-19) in social media. SocArXiv preprint posted online 2020. [FREE Full text] [CrossRef]
    16. Witteman HO, Zikmund-Fisher BJ. The defining characteristics of Web 2.0 and their potential influence in the online vaccination debate. Vaccine. May 28, 2012;30(25):3734-3740. [CrossRef] [Medline]
    17. Haase N, Schmid P, Betsch C. Impact of disease risk on the narrative bias in vaccination risk perceptions. Psychol Health. Mar 2020;35(3):346-365. [CrossRef] [Medline]
    18. Kaufman J, Ryan R, Walsh L, Horey D, Leask J, Robinson P, et al. Face-to-face interventions for informing or educating parents about early childhood vaccination. Cochrane Database Syst Rev. May 08, 2018;5(5):CD010038. [FREE Full text] [CrossRef] [Medline]
    19. Lendacki FR, Forst LS, Mehta SD, Kerins JL. COVID-19 vaccination requirements, encouragement and hesitancy among non-health care, non-congregate workers in Chicago: results from the WEVax survey. BMC Public Health. May 25, 2023;23(1):951. [FREE Full text] [CrossRef] [Medline]
    20. Golden SD, Moracco KE, Feld AL, Turner KL, DeFrank JT, Brewer NT. Process evaluation of an intervention to increase provision of adolescent vaccines at school health centers. Health Educ Behav. Dec 2014;41(6):625-632. [CrossRef] [Medline]
    21. Kharbanda EO, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to promote human papillomavirus vaccination. Vaccine. Mar 21, 2011;29(14):2537-2541. [CrossRef] [Medline]
    22. Shourie S, Jackson C, Cheater F, Bekker H, Edlin R, Tubeuf S, et al. A cluster randomised controlled trial of a web based decision aid to support parents' decisions about their child's measles mumps and rubella (MMR) vaccination. Vaccine. Dec 05, 2013;31(50):6003-6010. [FREE Full text] [CrossRef] [Medline]
    23. Andriani Y, Rusmil K, Akbar I. Measles-rubella immunization health education using animated videos and text messages via WhatsApp. Midwifery. 2020. URL: https://www.academia.edu/download/84881544/52501-208272-1-PB.pdf [accessed 2024-02-10]
    24. Groh F. Gamificationtate of the art definition and utilization. 2012 Presented at: RTMI’12: 4th Seminar on Research Trends in Media Informatics; February 14, 2012, 2012;39-46; Ulm, Baden-Württemberg.
    25. Lister C, West JH, Cannon B, Sax T, Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games. Aug 04, 2014;2(2):e9. [FREE Full text] [CrossRef] [Medline]
    26. Johnson D, Deterding S, Kuhn K, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv. Nov 2016;6:89-106. [FREE Full text] [CrossRef] [Medline]
    27. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform. Jul 2017;71:31-48. [FREE Full text] [CrossRef] [Medline]
    28. Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games. Dec 12, 2013;1(1):e3. [FREE Full text] [CrossRef] [Medline]
    29. Sawyer B, Rejeski D. Serious Games: Improving Public Policy through Game-Based Learning and Simulation. Washington, DC. Woodrow Wilson International Center for Scholars; 2002.
    30. Alvarez J, Djaouti D. An introduction to serious game - definitions and concepts. In: Fauquet-Alekhine P, Soler L, editors. Serious Games & Simulation for Risks Management. Paris. LARSEN Science; 2011;11-15.
    31. Cates JR, Fuemmeler BF, Stockton LL, Diehl SJ, Crandell JL, Coyne-Beasley T. Evaluation of a serious video game to facilitate conversations about human papillomavirus vaccination for preteens: pilot randomized controlled trial. JMIR Serious Games. Dec 03, 2020;8(4):e16883. [FREE Full text] [CrossRef] [Medline]
    32. de Carvalho CV, Coelho A. Game-based learning, gamification in education and serious games. Computers. Mar 04, 2022;11(3):36. [CrossRef]
    33. Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry. 2016;7:215. [FREE Full text] [CrossRef] [Medline]
    34. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. Oct 2020;18(10):2119-2126. [CrossRef] [Medline]
    35. Hargrave DR, Hargrave UA, Bouffet E. Quality of health information on the internet in pediatric neuro-oncology. Neuro Oncol. Apr 2006;8(2):175-182. [FREE Full text] [CrossRef] [Medline]
    36. Petrescu P. Google organic click-through rates in 2014. MOZ Blog. 2014. URL: https://moz.com/blog/google-organic-click-through-rates-in-2014 [accessed 2024-02-10]
    37. van der Marel S, Duijvestein M, Hardwick JC, van den Brink GR, Veenendaal R, Hommes DW, et al. Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis. Dec 2009;15(12):1891-1896. [CrossRef] [Medline]
    38. Resnick P, Garrett R, Kriplean T, Munson S, Stroud N. Bursting your (filter) bubble: strategies for promoting diverse exposure. 2013. Presented at: CSCW '13: Computer Supported Cooperative Work; February 23-27, 2013, 2013; San Antonio, TX. [CrossRef]
    39. PRISMA for scoping reviews. PRISMA. URL: http://tinyurl.com/23j8b9aw [accessed 2024-02-10]
    40. Gamification tool- enviromental scan. OSF Registries. URL: https://osf.io/nv8af [accessed 2024-02-09]
    41. Pluye P, Gagnon M, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. Apr 2009;46(4):529-546. [CrossRef] [Medline]
    42. Montgomery AA, Peters TJ, Little P. Design, analysis and presentation of factorial randomised controlled trials. BMC Med Res Methodol. Nov 24, 2003;3:26. [FREE Full text] [CrossRef] [Medline]
    43. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
    44. Betsch C, Böhm R. Detrimental effects of introducing partial compulsory vaccination: experimental evidence. Eur J Public Health. Jun 2016;26(3):378-381. [CrossRef] [Medline]
    45. Carolan K, Verran J, Crossley M, Redfern J, Whitton N, Amos M. Impact of educational interventions on adolescent attitudes and knowledge regarding vaccination: a pilot study. PLoS One. 2018;13(1):e0190984. [FREE Full text] [CrossRef] [Medline]
    46. Dale LP, White L, Mitchell M, Faulkner G. Smartphone app uses loyalty point incentives and push notifications to encourage influenza vaccine uptake. Vaccine. Jul 26, 2019;37(32):4594-4600. [CrossRef] [Medline]
    47. Darville G, Anderson–Lewis C, Stellefson M, Lee Y, MacInnes J, Pigg R, et al. Customization of avatars in a HPV digital gaming intervention for college-age males: an experimental study. Simul Gaming. Oct 17, 2018;49(5):515-537. [CrossRef]
    48. Eley CV, Young VL, Hayes CV, Verlander NQ, McNulty CAM. Young people’s knowledge of antibiotics and vaccinations and increasing this knowledge through gaming: mixed-methods study using e-Bug. JMIR Serious Games. Feb 01, 2019;7(1):e10915. [FREE Full text] [CrossRef] [Medline]
    49. McNulty CAM, Lecky DM, Farrell D, Kostkova P, Adriaenssens N, Koprivová Herotová T, et al. e-Bug Working Group. Overview of e-Bug: an antibiotic and hygiene educational resource for schools. J Antimicrob Chemother. Jun 2011;66(Suppl 5):v3-12. [FREE Full text] [CrossRef] [Medline]
    50. Fadda M, Galimberti E, Fiordelli M, Romanò L, Zanetti A, Schulz PJ. Effectiveness of a smartphone app to increase parents' knowledge and empowerment in the MMR vaccination decision: a randomized controlled trial. Hum Vaccin Immunother. Nov 02, 2017;13(11):2512-2521. [FREE Full text] [CrossRef] [Medline]
    51. Fadda M, Galimberti E, Fiordelli M, Schulz PJ. Evaluation of a mobile phone–based intervention to increase parents’ knowledge about the measles-mumps-rubella vaccination and their psychological empowerment: mixed-method approach. JMIR Mhealth Uhealth. Mar 07, 2018;6(3):e59. [FREE Full text] [CrossRef] [Medline]
    52. Ibuka Y, Li M, Vietri J, Chapman GB, Galvani AP. Free-riding behavior in vaccination decisions: an experimental study. PLoS One. 2014;9(1):e87164. [FREE Full text] [CrossRef] [Medline]
    53. Kaufman; Flanagan. Lost in translation: comparing the impact of an analog and digital version of a public health game on players? perceptions, attitudes, and cognitions. Int J Gaming Comput Mediat Simul. Jul 2013;5(3):1-9. [FREE Full text] [CrossRef]
    54. Lee W, Stück D, Konty K, Rivers C, Brown CR, Zbikowski SM, et al. Large-scale influenza vaccination promotion on a mobile app platform: a randomized controlled trial. Vaccine. Apr 16, 2020;38(18):3508-3514. [CrossRef] [Medline]
    55. Mitchell G, Leonard L, Carter G, Santin O, Brown Wilson C. Evaluation of a 'serious game' on nursing student knowledge and uptake of influenza vaccination. PLoS One. Jan 14, 2021;16(1):e0245389. [FREE Full text] [CrossRef] [Medline]
    56. Laplana JP. BJN Awards 2019: innovation runner up: creating a staff flu vaccination game. Br J Nurs. Aug 08, 2019;28(15):1032-1034. [CrossRef] [Medline]
    57. Mottelson A, Vandeweerdt C, Atchapero M, Luong T, Holz C, Böhm R, et al. A self-administered virtual reality intervention increases COVID-19 vaccination intention. Vaccine. Nov 05, 2021;39(46):6746-6753. [FREE Full text] [CrossRef] [Medline]
    58. Nowak GJ, Evans NJ, Wojdynski BW, Ahn SJG, Len-Rios ME, Carera K, et al. Using immersive virtual reality to improve the beliefs and intentions of influenza vaccine avoidant 18-to-49-year-olds: considerations, effects, and lessons learned. Vaccine. Jan 29, 2020;38(5):1225-1233. [CrossRef] [Medline]
    59. Real FJ, DeBlasio D, Beck AF, Ollberding NJ, Davis D, Cruse B, et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal. Acad Pediatr. 2017;17(4):431-435. [CrossRef] [Medline]
    60. Woodall WG, Zimet G, Kong A, Buller D, Reither J, Chilton L, et al. Vacteens.org: a mobile web app to improve HPV vaccine uptake. Front Digit Health. 2021;3:693688. [FREE Full text] [CrossRef] [Medline]
    61. Vandeweerdt C, Luong T, Atchapero M, Mottelson A, Holz C, Makransky G, et al. Virtual reality reduces COVID-19 vaccine hesitancy in the wild: a randomized trial. Sci Rep. Mar 17, 2022;12(1):4593. [FREE Full text] [CrossRef] [Medline]
    62. Amresh A, Chia-Chen A, Baron C. A game based intervention to promote HPV vaccination among adolescents. 2019. Presented at: SeGAH 2019: IEEE 7th International Conference on Serious Games and Applications for Health; August 5-7, 2019, 2019; Kyoto, Japan. [CrossRef]
    63. Bertozzi E, Krilov L, Walker D. Successful game development partnerships between academics and physicians: two case studies. IJGCMS. Jul 2013;5(3):97. [CrossRef]
    64. Carolan K, Verran J, Amos M, Crossley M, Redfern J, Whitton N, et al. SimFection: a digital resource for vaccination education. J Biol Educ. May 24, 2018;53(2):225-234. [CrossRef]
    65. Kafai Y, Fields D, Giang M, Fefferman N, Sun J, Kunka D, et al. Designing for massive engagement in a tween community: participation, prevention, and philanthropy in a virtual epidemic. 2017. Presented at: IDC '17: 2017 Conference on Interaction Design and Children; June 27-30, 2017, 2017; Stanford, CA. [CrossRef]
    66. de Araujo Lima ID, de Leon CGRMP, Ribeiro LM, da Silva ICR, Vilela DM, Fonseca LMM, et al. A serious game (Immunitates) about immunization: development and validation study. JMIR Serious Games. Feb 18, 2022;10(1):e30738. [FREE Full text] [CrossRef] [Medline]
    67. Real FJ, Rosen BL, Bishop JM, McDonald S, DeBlasio D, Kreps GL, et al. Usability evaluation of the novel smartphone application, HPV Vaccine: Same Way, Same Day, among pediatric residents. Acad Pediatr. 2021;21(4):742-749. [CrossRef] [Medline]
    68. Streuli S, Ibrahim N, Mohamed A, Sharma M, Esmailian M, Sezan I, et al. Development of a culturally and linguistically sensitive virtual reality educational platform to improve vaccine acceptance within a refugee population: the SHIFA community engagement-public health innovation programme. BMJ Open. Sep 14, 2021;11(9):e051184. [FREE Full text] [CrossRef] [Medline]
    69. Davies J, Bukulatjpi S, Sharma S, Caldwell L, Johnston V, Davis JS. Development of a culturally appropriate bilingual electronic app about hepatitis B for indigenous Australians: towards shared understandings. JMIR Res Protoc. Jun 10, 2015;4(2):e70. [FREE Full text] [CrossRef] [Medline]
    70. Ruiz-López T, Sen S, Jakobsen E, Tropé A, Castle PE, Hansen BT, et al. FightHPV: design and evaluation of a mobile game to raise awareness about human papillomavirus and nudge people to take action against cervical cancer. JMIR Serious Games. Apr 08, 2019;7(2):e8540. [FREE Full text] [CrossRef] [Medline]
    71. New mobile game in collaboration with WHO raises COVID-19 vaccine awareness. Eurogamer. URL: http://tinyurl.com/4p5snk46 [accessed 2024-02-09]
    72. The vaccination game. mrcwimm. URL: https://mrcwimm.itch.io/the-vaccination-game [accessed 2024-02-09]
    73. Help take down COVID-zilla!: protect your friends and family. Alberta Health Services. URL: http://tinyurl.com/yp82u75j [accessed 2024-02-09]
    74. Just the vax!. Vaccine Education Center. URL: https://media.chop.edu/data/files/vaccine-trivia-game/index.html [accessed 2024-02-09]
    75. COVID invaders. Get One Desk. URL: https://www.getonedesk.com/covid-invaders [accessed 2024-02-09]
    76. Vax pack hero. Vax Pack Hero. URL: https://vaxpackhero.com/ [accessed 2024-02-09]
    77. Flu's clues: virtual flu game. Families Fighting Flu. URL: https://www.familiesfightingflu.org/virtual-flu-game/ [accessed 2024-02-09]
    78. Virus fighter. Virus Fighter. URL: https://www.virusfighter.org/ [accessed 2024-02-09]
    79. Immunization411: for preteens and teens’ online training. Missouri Department of Health and Senior Services. URL: http://tinyurl.com/553ztd28 [accessed 2024-02-09]
    80. COVID chronicles. COVID Chronicles. URL: http://tinyurl.com/yc73frrh [accessed 2024-02-09]
    81. I boost. Public Health Association of BC. URL: https://iboostimmunity.com/ [accessed 2024-02-09]
    82. Montagni I, Mabchour I, Tzourio C. Digital gamification to enhance vaccine knowledge and uptake: scoping review. JMIR Serious Games. May 18, 2020;8(2):e16983. [FREE Full text] [CrossRef] [Medline]
    83. Ning Y, Jia Z, Zhu R, Ding Y, Wang Q, Han S. Effect and feasibility of gamification interventions for improving physical activity and health-related outcomes in cancer survivors: an early systematic review and meta-analysis. Support Care Cancer. Dec 30, 2022;31(1):92. [CrossRef] [Medline]
    84. Mazeas A, Duclos M, Pereira B, Chalabaev A. Evaluating the effectiveness of gamification on physical activity: systematic review and meta-analysis of randomized controlled trials. J Med Internet Res. Jan 04, 2022;24(1):e26779. [FREE Full text] [CrossRef] [Medline]
    85. Infodemic. World Health Organization. URL: https://www.who.int/health-topics/infodemic [accessed 2023-01-24]
    86. Plechatá A, Vandeweerdt C, Atchapero M, Luong T, Holz C, Betsch C, et al. Experiencing herd immunity in virtual reality increases COVID-19 vaccination intention: evidence from a large-scale field intervention study. Comput Hum Behav. Feb 2023;139:107533. [FREE Full text] [CrossRef] [Medline]
    87. Tozzi AE, Gesualdo F, D'Ambrosio A, Pandolfi E, Agricola E, Lopalco P. Can digital tools be used for improving immunization programs? Front Public Health. 2016;4:36. [FREE Full text] [CrossRef] [Medline]
    88. Ohannessian R, Yaghobian S, Verger P, Vanhems P. A systematic review of serious video games used for vaccination. Vaccine. Aug 31, 2016;34(38):4478-4483. [CrossRef] [Medline]
    89. Real FJ, Ollberding NJ, Meisman AR, DeBlasio DJ, Pero MB, Davis D, et al. Impact of a virtual reality curriculum on human papillomavirus vaccination: a pilot trial. Am J Prev Med. Nov 2022;63(5):865-873. [FREE Full text] [CrossRef] [Medline]
    90. Richman AR, Torres E, Wu Q, Eldridge D, Lawson L. The evaluation of a digital health intervention to improve human papillomavirus vaccine recommendation practices of medical students. J Cancer Educ. Aug 17, 2023;38(4):1208-1214. [CrossRef] [Medline]
    91. Raikhel AV, Blau K, Alberty K, Cornia P, Rodriguez RA, Steinberg KP, et al. Vax the Max, a gamification intervention for COVID-19 vaccination task engagement in the inpatient setting. Am J Med Qual. 2023;38(1):47-56. [FREE Full text] [CrossRef] [Medline]
    92. Brewer NT, Chapman GB, Rothman AJ, Leask J, Kempe A. Increasing vaccination: putting psychological science into action. Psychol Sci Public Interest. Dec 2017;18(3):149-207. [CrossRef] [Medline]
    93. Vaisson G, Provencher T, Dugas M, Trottier M, Chipenda Dansokho S, Colquhoun H, et al. User involvement in the design and development of patient decision aids and other personal health tools: a systematic review. Med Decis Making. Apr 2021;41(3):261-274. [CrossRef] [Medline]
    94. Hakim H, Bettinger JA, Chambers CT, Driedger SM, Dubé E, Gavaruzzi T, et al. A web application about herd immunity using personalized avatars: development study. J Med Internet Res. Oct 30, 2020;22(10):e20113. [FREE Full text] [CrossRef] [Medline]
    95. Epstein D. Game design and behaviour change using Vaxcards: a collectable card game to increase vaccine confidence. Monash University. 2022. URL: http://tinyurl.com/bdhxbbfj [accessed 2024-02-10]
    96. Noda S, Shirotsuki K, Nakao M. The effectiveness of intervention with board games: a systematic review. Biopsychosoc Med. 2019;13:22. [FREE Full text] [CrossRef] [Medline]
    97. Bayeck R. Examining board gameplay and learning: a multidisciplinary review of recent research. Simul Gaming. Apr 16, 2020;51(4):411-431. [CrossRef]
    98. Talan T, Doğan Y, Batdı V. Efficiency of digital and non-digital educational games: a comparative meta-analysis and a meta-thematic analysis. J Res Technol Educ. Apr 20, 2020;52(4):474-514. [CrossRef]
    99. Naderi S, Moafian F. The victory of a non-digital game over a digital one in vocabulary learning. Comput Educ Open. Dec 2023;4:100135. [CrossRef]

    ‎
    HPV: human papillomavirus
    MMAT: Mixed Methods Appraisal Tool
    PICO: Population, Intervention, Comparison, and Outcome
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

    Edited by G Eysenbach, T Leung; submitted 14.03.23; peer-reviewed by R Starling, M Coccia, D Epstein, R Delaney; comments to author 23.07.23; revised version received 22.09.23; accepted 02.01.24; published 29.02.24.

    Copyright

    ©Hina Hakim, S Michelle Driedger, Dominique Gagnon, Julien Chevrier, Geneviève Roch, Eve Dubé, Holly O Witteman. Originally published in JMIR Serious Games (https://games.jmir.org), 29.02.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.