Original Paper
- Lauren Bloomfield1*, BHlthSc, MPH, DrPH ;
- Julie Boston2*, BSc, Grad Dip, PhD ;
- Martin Masek2*, BEng, PhD ;
- Lesley Andrew2*, BSc, MSc, PhD ;
- Donna Barwood2, BEd, MEd, PhD ;
- Amanda Devine2, BAppSc, BEd, PhD
1School of Medicine, The University of Notre Dame Australia, Fremantle, Australia
2Edith Cowan University, Joondalup, Australia
*these authors contributed equally
Corresponding Author:
Julie Boston, BSc, Grad Dip, PhD
Edith Cowan University
270 Joondalup Drive
Joondalup, 6027
Australia
Phone: 61 0863045702
Email: julie.boston@ecu.edu.au
Abstract
Background: Invasive meningococcal disease (IMD) is a serious, vaccine-preventable infectious disease that can be life-threatening. Teaching adolescents about the early detection and prevention of IMD can be challenging in a school environment, with educators reporting they lack confidence or expertise to cover this in the classroom environment. Professional guest educators are an alternative to cover specialist topics such as IMD; however, time and resourcing constraints can mean that these educators are not always available. Serious games may be an alternative to face-to-face education, where complex health information may be delivered via self-directed gameplay.
Objective: This study aims to develop a serious game that can replace a face-to-face educator in a classroom setting to educate adolescents aged 12 years to 15 years. This study evaluates the efficacy of the Meningococcal Immunisation Awareness, Prevention and Protection app (MIApp), a serious game designed to replicate the information provided in a 30-minute face-to-face presentation provided by a trained educator.
Methods: This clustered, randomized controlled equivalence trial involved students (Years 7-10) from 6 secondary schools across metropolitan Western Australia who completed pre- and postintervention questionnaires with a follow-up at 3 months postintervention to measure the primary outcome of IMD knowledge acquisition following this self-guided intervention. The findings were compared with changes in an active control (comparison) group who received an in-class educational presentation about IMD transmission and protection. A questionnaire was developed to assess 9 key areas of knowledge. Median scores for knowledge pre- and postintervention were collected from a self-administered assessment of this questionnaire and, at 3 months postintervention, were compared between groups. A knowledge score of +/–2/16 was determined a priori to meet the criteria for equivalence. Participants who used MIApp were also asked a series of questions to assess the enjoyment of and engagement with the game.
Results: Of the 788 participating students, the median postintervention correct score in both the MIApp and control cohorts was 14/16 (87.5% correct responses), compared with the median pre-intervention correct score of 6/16 (37.5% correct responses), representing a significant (P<.001) increase in IMD knowledge in both groups. Improvements were retained in both groups 3 months after the initial intervention (median correct score: 11/16 in the intervention group; 12/16 in the control group; P=.86), demonstrating the efficacy of MIApp to deliver health education about IMD transmission and protection, although response rates in the follow-up cohort were low (255/788, 32.4%).
Conclusions: MIApp met the predetermined threshold for equivalence, demonstrating similar improvements in knowledge posttrial and at the 3-month follow-up. Participating adolescents considered the MIApp game more enjoyable than a presentation, with equivalent improvements in knowledge. Serious games could represent a constructive tool to help teachers impart specialized health education.
doi:10.2196/60755
Keywords
Introduction
An estimated 94% of Australian teenagers are smartphone users, with research showing 82% of teenagers frequently use these devices for digital gaming [9 in 10 Aussie teens now have a mobile (and most are already on to their second or subsequent handset). Roy Morgan. Aug 22, 2016. URL: http://www.roymorgan.com/findings/6929-australian-teenagers-and-their-mobile-phones-june-2016-201608220922 [accessed 2024-12-07] 1]. The emergence of digital technologies has led to a generation with new ways of thinking, learning, and interacting with each other [Emmerich K, Masuch M, Schmidt R. Digital Entertaining Health Applications: Researching the Fundamentals of a Cause-Effect Relationship. 2014. Presented at: Workshops on Advances in Computer Entertainment Conference; November 11-14, 2014; Funchal, Portugal. [CrossRef]2]. Game-based learning occurs through “an environment where game content and gameplay enhance knowledge and skill acquisition, and where game activities involve problem solving spaces and challenges that provide players/learners with a sense of achievement” [Qian M, Clark K. Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior. Oct 2016;63:50-58. [CrossRef]3]. In the school environment, digital games are increasingly being considered a constructive learning tool to captivate and intrinsically motivate adolescents’ engagement [Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]4-Wouters P, van Nimwegen C, van Oostendorp H, van der Spek E. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology. May 2013;105(2):249-265. [CrossRef]6]. The concept of “serious games” generally refers to those digital games in which the intention is to provide meaningful learning while simulating real-world scenarios that are typically aligned with educational standards [Ronimus M, Eklund K, Pesu L, Lyytinen H. Supporting struggling readers with digital game-based learning. Education Tech Research Dev. Feb 20, 2019;67(3):639-663. [CrossRef]7-Zyda M. From visual simulation to virtual reality to games. Computer. Sep 2005;38(9):25-32. [CrossRef]9].
Research suggests serious games can promote active learning, build knowledge, engage and motivate students, support higher-level thinking, and increase learner independence [Newhouse C. STEM the boredom: engage students in the Australian curriculum using ICT with problem-based learning and assessment. J Sci Educ Technol. Sep 14, 2016;26(1):44-57. [CrossRef]10]. Studies have also demonstrated the attractiveness of digital games as a learning environment to support social, emotional, and cognitive outcomes in young people [Annetta LA, Minogue J, Holmes SY, Cheng M. Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education. Aug 2009;53(1):74-85. [CrossRef]11-Young MF, Slota S, Cutter AB, Jalette G, Mullin G, Lai B, et al. Our princess is in another castle. Review of Educational Research. Mar 01, 2012;82(1):61-89. [CrossRef]13], with evidence illustrating their influence on the long-term retention of knowledge [Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]4,Annetta LA, Minogue J, Holmes SY, Cheng M. Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education. Aug 2009;53(1):74-85. [CrossRef]11]. Serious games appear to be well suited to specific mandated curriculum learning areas like science, technology, engineering, and mathematics [Barwood D, Smith S, Miller M, Boston J, Masek M, Devine A. Transformational game trial in nutrition education. Australian Journal of Teacher Education. Apr 2020;45(4):18-29. [CrossRef]14-Masek M, Boston J, Lam C, Corcoran S. Improving mastery of fractions by blending video games into the Math classroom. Computer Assisted Learning. Apr 27, 2017;33(5):486-499. [CrossRef]16] and offer additional learning opportunities to engage adolescents in new emerging fields of study such as health promotion [Papastergiou M. Exploring the potential of computer and video games for health and physical education: A literature review. Computers & Education. Nov 2009;53(3):603-622. [CrossRef]17,Andrew L, Barwood D, Boston J, Masek M, Bloomfield L, Devine A. Serious games for health promotion in adolescents - a systematic scoping review. Educ Inf Technol (Dordr). 2023;28(5):5519-5550. [FREE Full text] [CrossRef] [Medline]18].
Given their growing appeal, serious games are increasingly being explored as a novel alternative to the traditional classroom delivery of health promotion messages to adolescents [Emmerich K, Masuch M, Schmidt R. Digital Entertaining Health Applications: Researching the Fundamentals of a Cause-Effect Relationship. 2014. Presented at: Workshops on Advances in Computer Entertainment Conference; November 11-14, 2014; Funchal, Portugal. [CrossRef]2,DeSmet A, Van Ryckeghem D, Compernolle S, Baranowski T, Thompson D, Crombez G, et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev Med. Dec 2014;69:95-107. [FREE Full text] [CrossRef] [Medline]5,Barber NC, Stark LA. Online resources for understanding outbreaks and infectious diseases. CBE Life Sci Educ. Mar 02, 2015;14(1):fe1. [FREE Full text] [CrossRef] [Medline]19,Fontes P, Boaventura P, Azevedo J. Mobile games to improve healthier lifestyle amongst youth: "are we there yet" or should we move forward? 2014. Presented at: Workshops on Advances in Computer Entertainment Conference; November 11-14, 2014; Funchal, Portugal. [CrossRef]20]. Serious games for health promotion can stimulate real-life situations, providing adolescents with explorative learning environments [Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertainment Computing. Dec 2013;4(4):231-247. [CrossRef]21], and can be adapted to the developmental stage, educational level, and personal interest, allowing the player to self-manage their educational goals [Espinosa-Curiel IE, Pozas-Bogarin EE, Hernández-Arvizu M, Navarro-Jiménez ME, Delgado-Pérez EE, Martínez-Miranda J, et al. HelperFriend, a serious game for promoting healthy lifestyle behaviors in children: design and pilot study. JMIR Serious Games. May 06, 2022;10(2):e33412. [FREE Full text] [CrossRef] [Medline]22].
Moreover, serious games can offer support for the delivery of health education around issues that may not be easy for teachers to address due to time, stigma, and their own lack of knowledge [Espinosa-Curiel IE, Pozas-Bogarin EE, Hernández-Arvizu M, Navarro-Jiménez ME, Delgado-Pérez EE, Martínez-Miranda J, et al. HelperFriend, a serious game for promoting healthy lifestyle behaviors in children: design and pilot study. JMIR Serious Games. May 06, 2022;10(2):e33412. [FREE Full text] [CrossRef] [Medline]22,Ricciardi F, De Paolis LT. A comprehensive review of serious games in health professions. International Journal of Computer Games Technology. 2014;2014:1-11. [CrossRef]23]. In addition, a lack of resources for expert presenters to deliver face-to-face content across large geographical areas means that regional and remote areas are less likely to receive such interventions, furthering the inequities between metropolitan and rural populations. A large body of research exists to illustrate how serious games can be designed specifically for topics such as nutrition [Barwood D, Smith S, Miller M, Boston J, Masek M, Devine A. Transformational game trial in nutrition education. Australian Journal of Teacher Education. Apr 2020;45(4):18-29. [CrossRef]14,Baranowski T, Ryan C, Hoyos-Cespedes A, Lu AS. Nutrition education and dietary behavior change games: a scoping review. Games Health J. Jun 2019;8(3):153-176. [FREE Full text] [CrossRef] [Medline]24,Holzmann SL, Schäfer H, Groh G, Plecher DA, Klinker G, Schauberger G, et al. Short-term effects of the serious game "Fit, Food, Fun" on nutritional knowledge: a pilot study among children and adolescents. Nutrients. Aug 30, 2019;11(9):A. [CrossRef] [Medline]25], sexual health [Chu SKW, Kwan ACM, Reynolds R, Mellecker RR, Tam F, Lee G, et al. Promoting sex education among teenagers through an interactive game: reasons for success and implications. Games Health J. Jun 2015;4(3):168-174. [CrossRef] [Medline]26,DeSmet A, Shegog R, Van Ryckeghem D, Crombez G, De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J. Apr 2015;4(2):78-90. [CrossRef] [Medline]27], substance use [Rodriguez DM, Teesson M, Newton NC. A systematic review of computerised serious educational games about alcohol and other drugs for adolescents. Drug Alcohol Rev. Mar 2014;33(2):129-135. [CrossRef] [Medline]28], and mental health [David OA, Costescu C, Cardos R, Mogoaşe C. How effective are serious games for promoting mental health and health behavioral change in children and adolescents? A systematic review and meta-analysis. Child Youth Care Forum. Jul 31, 2020;49(6):817-838. [CrossRef]29], among others. This ranges from understanding strategies that can be used to explain complex concepts to pedagogical approaches that capture the social and cultural nuances of communities or age-related groups. However, despite 2 meta-analyses suggesting serious games represent promising learning tools [DeSmet A, Van Ryckeghem D, Compernolle S, Baranowski T, Thompson D, Crombez G, et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev Med. Dec 2014;69:95-107. [FREE Full text] [CrossRef] [Medline]5,Wouters P, van Nimwegen C, van Oostendorp H, van der Spek E. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology. May 2013;105(2):249-265. [CrossRef]6], limited and inconsistent evidence of serious game efficacy in delivering health education content remains an issue [DeSmet A, Shegog R, Van Ryckeghem D, Crombez G, De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J. Apr 2015;4(2):78-90. [CrossRef] [Medline]27,Kato PM. Evaluating efficacy and validating games for health. Games Health J. Feb 2012;1(1):74-76. [CrossRef] [Medline]30].
For digital games to become a key tool for the support of traditional modalities of teaching and learning, there is a need to evaluate the efficacy of these games. The challenge for teachers is to ensure the serious game is curriculum-aligned, fit-for-purpose, and effective as a resource to meet the needs of a range of student ages and settings and that it can extend or complement learning. Developing serious games for health education, however, is time-consuming and challenging. It requires multidisciplinary team members to address stated objectives, encompassing elements of design, content, format, and pedagogy to the psychology of behavioral change [Sharifzadeh N, Kharrazi H, Nazari E, Tabesh H, Edalati Khodabandeh M, Heidari S, et al. Health education serious games targeting health care providers, patients, and public health users: scoping review. JMIR Serious Games. Mar 05, 2020;8(1):e13459. [FREE Full text] [CrossRef] [Medline]31]. Adapting complex health information for suitability with younger audiences within an educational environment can pose several further challenges for game developers. These challenges include maintaining player engagement and ensuring the game effectively reflects national curriculum standards and learning goals and outcomes.
The Meningococcal Immunisation Awareness, Prevention and Protection app (MIApp) is a serious game developed in Western Australia (WA) to deliver education via a digital platform about the transmission of meningococci and protection targeted at adolescents aged 12 years to 15 years. Invasive meningococcal disease (IMD), caused by Neisseria meningitidis bacteria, is a serious, vaccine-preventable infectious disease that is rare but life-threatening [Meningococcal Infection Notifications 2018. 32 WA Department of Health. 2018. URL: http://ww2.health.wa.gov.au/Articles/N_R/Notifiable-infectious-disease [accessed 2020-09-01] 32]. IMD can result in ongoing disability ranging from major neurological complications, including hearing loss, cognitive deficit, and seizure, to severe non-neurological effects, such as the amputation of a limb and death [Sadarangani M, Scheifele DW, Halperin SA, Vaudry W, Le Saux N, Tsang R, et al. investigators of the Canadian Immunization Monitoring Program‚ ACTive (IMPACT). Outcomes of invasive meningococcal disease in adults and children in Canada between 2002 and 2011: a prospective cohort study. Clin Infect Dis. Apr 15, 2015;60(8):e27-e35. [CrossRef] [Medline]33]. Adolescents are at an increased risk of meningococcal carriage, a precursor for the development of invasive disease [Burman C, Serra L, Nuttens C, Presa J, Balmer P, York L. Meningococcal disease in adolescents and young adults: a review of the rationale for prevention through vaccination. Hum Vaccin Immunother. 2019;15(2):459-469. [FREE Full text] [CrossRef] [Medline]34]. Multilevel public health measures to reduce the risk of IMD include vaccination; minimization of behaviors known to increase transmission, such as intimate kissing and openly sneezing and coughing; and education for the early identification and antibiotic administration for suspected cases [Invasive Meningococcal Disease: CDNA National Guidelines for Public Health Units. Communicable Diseases Network Australia. Jul 01, 2024. URL: https://www.health.gov.au/sites/default/files/2024-07/invasive-meningococcal-disease-cdna-national-guidelines-for-public-health-units.pdf [accessed 2024-12-07] 35]. Anecdotal experience of educators suggests teachers in WA are hesitant to introduce the subject of IMD to students without adequate support from a reliable information source. Current educational strategies for IMD are limited by the availability of (often volunteer) staff to provide face-to-face presentations to school-aged children and by the coordination of organizing these, especially in rural and remote areas.
The primary aim of the study was to evaluate the efficacy of MIApp, a serious game that delivers health education for adolescents about IMD, compared with that of a face-to-face presentation.
The associated objectives were as follows: (1) evaluate the efficacy of the serious game MIApp on knowledge about meningococcal carriage, disease signs and symptoms, and vaccination among adolescents; (2) assess the acceptability of, usability of, and overall student experience with MIApp. The overall purpose of the investigation was to determine whether the delivery via an iPad of a fun, engaging, age-appropriate, self-navigating MIApp could be used as a valuable pedagogical tool to help educators better engage with young people around health issues they may find challenging to address. The secondary aim was to assess the retention of knowledge 3 months postintervention.
Methods
Design
The development of the educational MIApp was part of a broader initiative funded by Lotterywest, the WA Department of Health, the Amanda Young Foundation (AYF), and Edith Cowan University (ECU) between 2019 and 2022. The project aimed to raise awareness and promote IMD prevention among adolescents, a high-risk group. MIApp, a serious game, integrates thoughtfully chosen gamification mechanics and educational components. The app was developed using an iterative co-design approach, with collaboration among the funding partners and a steering committee of experts in IMD, public health, serious game development, and learning design, as well as input from high school students and end users. The app was developed with the education team of the AYF and was designed to reflect the information in the current face-to-face presentation. Health Promotion Officers developed these resources and designed them to deliver key health messages at an age-appropriate level.
The resulting narrative emerged as an overworld that brings to life the reality of meningococcal infection through interactive gameplay where the players are tasked with solving the source of IMD infection. Through MIApp, the students are introduced to the content including communicable disease, IMD, the concept of “healthy carriers,” signs and symptoms of infection, and preventive health care strategies (“ways” to be safe and remain safe). An extended MIApp user guide outlining all the game's features is available from the AYF website [MIApp Years 7 to 10. The Amanda Young Foundation. URL: https://amandayoungfoundation.org.au/miapp-years-7-to-10/ [accessed 2024-12-07]
36] [ MIApp Extended User Guide.Multimedia Appendix 6
Following the launch of MIApp in mid-2021, formal evaluation began in 2021 and was completed in 2022. We compared 2 education delivery models in the study: the intervention group, who used MIApp [Multimedia Appendices 2-Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]4], and the active control (comparison) group, who received an in-class educational presentation [ Amanda Young Foundation classroom presentation.Multimedia Appendix 5
Both models of delivery took approximately 50 minutes. Pre- and postintervention questionnaires were conducted to examine student responses to 9 key knowledge areas of IMD (see Textbox 1). The MIApp research design was a cluster randomized controlled trial, following data analysis from the pilot. Students were recruited across 4 year groups from a range of secondary schools. Recruitment and initial pre-post testing occurred between April 2021 and March 2021, with follow-up surveys sent 3 months after the school visit. The number of participating schools recruited was adjusted depending on the number of enrolled students per school and, consequently, the consent rate at each site, with recruitment ceasing when the prespecified sample size was reached.
Key knowledge areas to be assessed
- Meningococcal infection is life-threatening but rare
- Early signs and symptoms, progression of illness
- Common symptoms of invasive meningococcal disease to look out for
- What to do if you are experiencing symptoms
- The importance of vaccination as a prevention tool
- Which strains are covered by vaccination
- The significance of “healthy carriers” in disease transmission
- How the bacteria spread and behaviors that can result in transmission
- The importance of avoiding behaviors that could spread the bacteria
Participants
Adolescents, being at increased risk for meningococcal carriage, were the target demographic for this study. Participants were high school students in year levels 7, 8, 9, and 10; aged 12 years to 15 years; and in the metropolitan area of Perth, WA. Schools were purposely selected using an online directory of the School Index of Community Socio-Educational Advantage (ICSEA) [Find a School. Australian Curriculum, Assessment and Reporting Authority. URL: https://www.myschool.edu.au/ [accessed 2024-12-07] 37]. The ICSEA is a scale of socioeducational advantage computed for each school, allowing comparisons between schools based on the level of educational advantage or disadvantage that students bring to their academic studies. School facilities, resources, and staffing do not influence ICSEA. Sampling and recruitment aimed to acquire an equal proportion of male and female students between years 7 and 10 and across the ICSEA school values in the intervention and control groups.
The sample size calculation was based on an expected cluster size of 100 participants, with an intracluster correlation coefficient of 0.02 and an anticipated effect size of 0.4 (Cohen d). The design effect was calculated as 1 + (expected average cluster size – 1) × intracluster correlation coefficient. This resulted in a design effect of 2.98. After adjusting for clustering, an adequate sample size of 201 participants was required. To account for an expected 20% attrition, we aimed for an initial total sample size of 750 participants, leading to an expected final sample size of 600.
The effect size of 0.4 was selected as a moderate expected difference between the intervention and control arms, given the nature of the intervention and previous literature on similar studies. The design effect calculation and the adjustment for clustering follow standard methods for cluster randomized trials as described by Donner and Klar [Klar N, Donner A. Design and Analysis of Cluster Randomization Trials in Health Research. London, England. Wiley; 2000. 38]. Cohen d was used during the design phase to guide the sample size estimation, though nonparametric methods were used in the analysis phase due to the skewness of the data.
A total of 6 secondary schools across metropolitan Perth, WA, gave consent and participated in the research study. Parent and student information sheets informed participants that the trial would compare MIApp with a face-to-face presentation. The consent process was undertaken by the schools, with the research team only engaging with students who had returned consent. All students who returned consent were eligible to participate in the study. Those who did not return consent participated in a different teacher-led activity chosen at the school’s discretion. It was assumed that all students had a base level of computer literacy suitable for participation in the trial.
Due to mandatory lockdowns and COVID-19 rules, the study team could not travel to regional areas within the time limit of the study, and consequently, only metropolitan schools were included. Recruited schools were asked to provide an aggregate number of classes per year group (years 7, 8, 9, and 10) and students per class who were eligible to participate. Despite differences in student ICSEA scores between the MIApp and control groups, with the MIApp group having more significant variation in socioeducational backgrounds, the median was the same for each group.
Randomization Procedure
This was a clustered randomized controlled equivalence trial. Randomization was performed at a class level within each school. Following school enrollment, each site provided a class list to the research team indicating the number of classes available for participation at each level. Paper consent was collected by the school, who determined the day on which students were eligible to participate based on the provision of consent. Classes within each school were given a unique identifier and stratified by year level. These were then stratified by class level and randomly allocated to receive the intervention or control in sequential order by research team members with no affiliation to the school or knowledge of the class composition. Due to the nature of the intervention, blinding or allocation concealment was not applied. The arm of participation was recorded in the data as a numerical code (1, 2) that was not revealed to the analyst during the statistical analysis before preparing the data for reporting.
Instrumentation
All participating students were required to complete pre- and postintervention questionnaires designed to measure the primary outcome: change in knowledge of IMD. To assess the 9 key knowledge areas, 16 questions were developed ( MiApp Post Study Questionnaire.Multimedia Appendix 1
Textbox 1). Participants developed a unique identifier based on elements of their name and date of birth. The participants in the intervention group were also asked to rank on a 5-point scale a series of questions about the study’s secondary outcomes: the participants’ perception of usability and engagement with the game and, more generally, about the delivery of educational messages using serious games as a platform [Brooke J. SUS - A quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability evaluation in industry (Vol. 189). New York, NY. Routledge; 1996:4-7.39]. Data on knowledge acquisition, timing, and behavior were also obtained to gain further insight into the overall usability, learnability, and experience of the MIApp game.
The validity and reliability of the research instrument were established in several ways during the project. First, the instrument was closely examined by the research team and the steering committee to ensure alignment with formative research, curriculum text, clarity, and ease of language for the age group and the functioning and facilities within the instrument, such as response mechanisms. The instrument was then pilot tested for the construct validity of the topics included. Minor revisions were made to discordant wording. Test-retest reliability was not performed for the questionnaire items.
Procedures
Following agreement to participate in the MIApp study and written informed consent provided by the school principal, schools were asked to provide an aggregate number of classes per year group (years 7, 8, 9, and 10) and students per class eligible to participate. A parent or carer and student (participant) information sheet and consent form were disseminated by the schools before the provision of consent and content delivery. This information sheet outlined the rationale for the study, the risks and benefits to participants, and the proposed activity undertaken as part of the research.
The educational content for the intervention and control groups was based on the AYF guidelines to support safer, healthier, and more physically active living by young Australians by creating meaningful, valuable, and strengths-based learning focused on IMD awareness and prevention. They reflect WA's Healthy and Physical Education (HPE) learning outcomes via the WA P-10 Syllabus for HPE and the Australian Curriculum for HPE [Health and Physical Education. School Curriculum and Standards Authority. URL: https://k10outline.scsa.wa.edu.au/home/teaching/curriculum-browser/health-and-physical-education [accessed 2024-12-07]
40]. The intervention, MIApp, is a new software application developed by ECU. MIApp is a self-directed, serious game designed to be completed in 30 minutes. It targets students in years 7 through 10 to raise awareness and bring to life the realities of meningococcal infection. At the Detect and Protect Agency, players investigate a case to solve the source of a mystery infection. Players engage with the learning content and key messages delivered in the AYF presentation through a combination of games, quizzes, simulations, and interactive activities. The control group received a face-to-face educational presentation delivered by a presenter from the AYF. This is the primary mode of education developed by the AYF and includes a secondary school PowerPoint presentation adapted to fit the allocated research time of 30 minutes. The presentation delivers 9 key messages about IMD including prevention and identification of symptoms and preventative health care strategies (see Textbox 1). Embedded in the presentation is a short interactive quiz and a video highlighting real-life stories of young people who have caught and survived the disease.
Classes within each participating school were randomized to the intervention or control arm stratified by year group, with one-half of the classes at each school receiving either the app or the face-to-face presentation. At the discretion of individual schools, classrooms or locations were set up for the research trials. A requirement was that rooms be separate, so no interference was experienced with either mode of delivery. All questionnaires were completed by the participants using the Qualtrics survey platform [Qualtrics. URL: https://www.qualtrics.com/ [accessed 2024-12-07] 41] on an iPad during the initial onsite visit and via an emailed link for the 3-month follow-up.
Tablets were distributed to all the participants, followed by a briefing on the scope of the research and instructions for the session. Each session (control and intervention) took 50 minutes comprised of 5 minutes to 10 minutes for the pre-intervention questionnaire, 30 minutes for either the intervention (MIApp) or control (presentation), and a further 5 minutes to 10 minutes for the postintervention questionnaire. There was limited interaction between the researchers and participants in order to mirror the use in a classroom environment. Participants were first asked to create a unique identifier that could be used to link the questionnaire data collected before and after their session and again at the 3-month follow-up. The identifier used the first 2 letters of their first name, the day of their date of birth, and the last 2 letters of their last name. MIApp cohort participants also used this unique identifier to link survey responses to in-app data gathered by MIApp. Participants were not required to remember the username they created for the follow-up survey, and no identifying information about the student was collected by the app.
Data Analysis
Statistical analysis was conducted using R version 4.0.3 [R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria.; 2021. URL: https://www.R-project.org/ [accessed 2024-12-07]
42]. “Differences in correct” scores of the 16 pre and postintervention questions were used to compare the MIApp and control cohorts ( MiApp Post Study Questionnaire.Multimedia Appendix 1
Ethics Approval
Ethics approval for the study was obtained from the Human Research Ethics Committee at ECU (reference: 2019-00736-BLOOMFIELD). The parents of all participants were provided with participant information and a consent form outlining the rationale for the trial and the activities of the intervention and control groups.
Trial Registration
As this was not clinical research, it was not registered on the Australian New Zealand Clinical Trials Registry. A protocol was not published prior to commencement.
Results
Approximately 1500 students were invited to participate in the trial by the 6 schools invited to participate. A total of 788 participants from the 6 trial schools consented and completed the pre- and postintervention questionnaires and were eligible for inclusion in the primary analysis: 442 in the intervention arm and 346 in the control arm (Figure 1). Although different demographic patterns were observed across the 6 participating schools, approximately one-half of the students were in year 7 (397/788, 50.4%), were female (397/788, 50.4%), and self-reported not knowing anything about meningococcal disease at the start of the trial (395/788, 50.1%) (
Table 1). The demographic and self-reported pre-knowledge characteristics of the study population by group are shown in
Table 2. There were no statistically significant differences between the groups, which suggests a good balance of these properties following group randomization.

Demographics | School 1 (n=173), n (%) | School 2 (n=56), n (%) | School 3 (n=138), n (%) | School 4 (n=198), n (%) | School 5 (n=214), n (%) | School 6 (n=9), n (%) | Total sample (N=788), n (%) | |
ICSEAa | 1100 | 1013 | 1032 | 1110 | 1120 | 890 | —b | |
Year | ||||||||
7 | 111 (64.2) | 26 (46.4) | 0 (0) | 46 (23.2) | 214 (100) | 0 (0) | 397 (50.4) | |
8 | 62 (35.8) | 5 (8.9) | 0 (0) | 31 (15.7) | 0 (0) | 6 (66.7) | 104 (13.2) | |
9 | 0 (0) | 25 (44.6) | 138 (100) | 24 (12.1) | 0 (0) | 3 (33.3) | 190 (24.1) | |
10 | 0 (0) | 0 (0) | 0 (0) | 97 (49) | 0 (0) | 0 (0) | 97 (12.3) | |
Gender | ||||||||
Male | 67 (38.7) | 31 (55.4) | 76 (55.1) | 93 (47) | 91 (42.5) | 3 (33.3) | 361 (45.8) | |
Female | 102 (59) | 21 (37.5) | 56 (40.6) | 97 (49) | 116 (54.2) | 5 (55.6) | 397 (50.4) | |
Other | 4 (2.3) | 4 (7.1) | 6 (4.3) | 8 (4) | 7 (3.3) | 1 (11.1) | 30 (3.8) |
aICSEA: Index of Community Socio-Educational Advantage.
bNot applicable.
Demographic characteristics and preknowledge | MIApp (n=442) | Control (n=346) | Total sample (n=788) | P valuea | ||||||
Year, n (%) | .19 | |||||||||
7 | 215 (48.6) | 182 (52.6) | 397 (50.4) | |||||||
8 | 60 (13.6) | 44 (12.7) | 104 (13.2) | |||||||
9 | 118 (26.7) | 72 (20.8) | 190 (24.1) | |||||||
10 | 49 (11.1) | 48 (13.9) | 97 (12.3) | |||||||
Gender, n (%) | .94 | |||||||||
Male | 200 (45.2) | 161 (46.5) | 361 (45.8) | |||||||
Female | 225 (50.9) | 172 (49.7) | 397 (50.4) | |||||||
Other | 17 (3.8) | 13 (3.8) | 30 (3.8) | |||||||
Baseline knowledgeb, n (%) | .62 | |||||||||
Nothing | 218 (49.3) | 177 (51.2) | 395 (50.1) | |||||||
Some | 217 (49.1) | 166 (48) | 383 (48.6) | |||||||
A lot | 7 (1.6) | 3 (0.9) | 10 (1.3) | |||||||
ICSEAc, median (IQR) | 1100 (1032-1120) | 1100 (1100-1120) | —d | .02 |
aPearson chi-square test for categorical variables, and the Kruskal-Wallis test for continuous variables.
bStudents were asked to nominate which of these 3 statements was the most accurate: I don\'t know anything about meningococcal disease, I know a little about meningococcal disease, and I know a lot about meningococcal disease.
cICSEA: Index of Community Socio-Educational Advantage.
dNot applicable.
Because they answered “I’m not sure/I don’t know” to all the questions on the survey, 33 participants in the control group (33/346, 9.5%) and 18 participants in the MIApp group (18/442, 4.1%) had a score of 0 for the pre-intervention survey. The median pre-intervention correct score in both the MIApp and control cohorts was 6/16 (MIApp: IQR 4-8; control: IQR 3-9), and the median postintervention correct score in both the MIApp and control cohorts was 14/16 (MIApp: IQR 12-15; control: IQR 13-15). The median improvement in score for the total cohort was 7/16 (IQR 5-10). The median improvements in score were 7/16 (IQR 5-9) for the MIApp arm and 8/16 (IQR 5-11) for the control arm (P<.001). Statistical equivalence in mean improvement between arms was demonstrated, with the confidence intervals for the mean difference falling within 2 points (MIApp vs control: mean difference=–1.036, lower limit=–1.569, upper limit=–0.503). Score improvement differed by year group, with students in years 9 and 10 having slightly lower median improvements in score (7 and 6, respectively) than those in years 7 and 8, who both had median improvements of 8 (P<.001). No statistically significant differences in median score improvement were noted by gender (P=.20; Table 3).
Characteristic | MIAppa (n=442) | F2Fb (n=346) | Total sample (N=788) | P valuec | ||||||||||
Pre-intervention, median (IQR) | Postintervention, median (IQR) | Change in the score, median (IQR) | Pre-intervention, median (IQR) | Postintervention, median (IQR) | Change in the score, median (IQR) | Pre-intervention, median (IQR) | Postintervention, median (IQR) | Change in the score, median (IQR) | ||||||
Year | <.001 | |||||||||||||
7 | 6 (3.5-8) | 14 (12-15) | 7 (5-9.5) | 6 (3-8) | 14 (13-15) | 8 (6-11) | 6 (3-8) | 14 (13-15) | 8 (5-10) | |||||
8 | 6 (3.75-8) | 14 (13-15) | 7 (4.75-10) | 6 (2-8) | 15 (14-15) | 9 (6-12) | 6 (3-8) | 14 (13-15) | 8 (5-10.25) | |||||
9 | 7 (4-8) | 14 (13-15) | 7 (5-10) | 6 (4-9) | 14 (13-15) | 8 (5-11) | 6 (4-8) | 14 (13-15) | 7 (5-10) | |||||
10 | 7 (6-9) | 14 (12-15) | 5 (4-8) | 7 (5.75-9.25) | 15 (14-15) | 7 (4-9) | 7 (6 9) | 14 (13-15) | 6 (4-8) | |||||
Gender | .20 | |||||||||||||
Male | 7 (4-9) | 14 (12-15) | 7 (4-9) | 6 (3-8) | 14 (13-15) | 8 (5-11) | 6 (3-8) | 14 (13-15) | 7 (5-10) | |||||
Female | 4 (4-8) | 14 (13-15) | 7 (5-10) | 6 (3-9) | 14 (13-15) | 8 (5-11) | 6 (4-9) | 14 (13-15) | 8 (5-10) | |||||
Other | 6 (5-8) | 13 (8-14) | 5 (2-8) | 6 (2-7) | 14 (13-15) | 8 (6-11) | 6 (4-7.75) | 13 (11-14) | 6.5 (2.5-9) |
aMIApp: Meningococcal Immunisation Awareness, Prevention and Protection app.
bF2F: face to face.
cKruskal-Wallis test.
Table 4 shows the percentage of MIApp participants responding to questions about attitudes toward and engagement with the game; because these questions were not compulsory, not all students responded. Most participants responded positively: “I thought MIApp was easy to use” (348/426, 81.7%); “It was clear when I was playing what I was supposed to do next” (342/427, 80.1%); “MIApp was more enjoyable than a presentation” (326/426, 76.5%); “I felt very confident using MIApp” (315/423, 74.5%); and “I would like to use educational games like this more often” (313/424, 73.8%). Few found it difficult to use or needed help: “I needed to learn a lot more before I could understand MIApp” (58/426, 13.6%); “Finding things in MIApp was too hard” (45/428, 10.5%); “I found using MIApp a bit overwhelming” (37/424, 8.7%); and “I needed help from my teacher to be able to use MIApp” (27/427, 6.3%).
Statement | Agreement with the statement, n (%) | ||||
Year 7 (n=215) | Year 8 (n=60) | Year 9 (n=118) | Year 10 (n=49) | Total sample (n=442) | |
I would like to use educational games like this more often | 162 (79.4)a | 45 (76.3)b | 73 (64)c | 33 (70.2)d | 313 (73.8)e |
Finding things in MIApp was too hard | 23 (11.1)f | 3 (5.1)b | 10 (8.8)c | 9 (19.1)d | 45 (10.5)g |
It was clear when I was playing what I was supposed to do next | 166 (80.2)h | 48 (81.4)b | 91 (79.8)c | 37 (78.7)d | 342 (80.1)i |
I needed help from my teacher to be able to use MIApp | 15 (7.2)h | 2 (3.4)b | 5 (4.4)c | 5 (10.6)d | 27 (6.3)i |
MIApp was more enjoyable than a presentation | 159 (76.8)h | 46 (78)b | 86 (75.4)c | 35 (76.1)j | 326 (76.5)k |
I thought the features of on the screen on MIApp were confusing | 27 (13.1)l | 3 (5.1)b | 7 (6.2)m | 12 (25.5)d | 49 (11.5)n |
I thought MIApp was easy to use | 168 (81.6)l | 48 (81.4)b | 92 (80.7)c | 40 (85.1)d | 348 (81.7)k |
I found using MIApp a bit overwhelming | 17 (8.3)o | 3 (5.1)b | 12 (10.6)m | 5 (10.6)d | 37 (8.7)e |
I felt very confident using MIApp | 151 (73.7)o | 44 (74.6)b | 85 (75.9)p | 35 (74.5)d | 315 (74.5)q |
I needed to learn a lot more before I could understand MIApp | 29 (14.1)l | 9 (15.3)b | 14 (12.3)c | 6 (12.8)d | 58 (13.6)k |
an=204.
bn=59.
cn=114.
dn=47.
en=424.
fn=208.
gn=428.
hn=207.
in=427.
jn=46.
kn=426.
ln=206.
mn=113.
nn=425.
on=205.
pn=212.
qn=423.
A total of 255 participants from both groups completed nonduplicated (by unique identifier) responses for the 3-month follow-up survey, with a lower response rate for the MIApp arm (MIApp: 97/442, 22%; control: 158/346, 45.7%; P<.001). Of these, only 89 responses could be linked to the primary questionnaire results using the unique study identifier (MIApp: 50/442, 11.3%; control: 39/346, 11.3%; P=.99). The inability to match responses to the original survey for the majority of respondents precludes an analysis at follow-up by demographic factors.
The 3-month follow-up survey asked participants to answer the same set of questions related to the 9 key knowledge areas and indicate if they participated in the MIApp or control arm on the day of the trial. The median total correct scores at the 3-month follow-up were 11/16 (IQR 9-13) and 12/16 (IQR 9-13) for the MIApp and control arms, respectively (P=.86). A sensitivity analysis only including the 89 participants who could be matched to the trial surveys produced similar results (correct scores of 12/16 and 13/16 for the intervention and control arms, respectively).
Although there was a slight decline in knowledge at the 3-month mark, this again demonstrates that there was similar knowledge retention in both groups 3 months after the initial lesson. Overall, these results suggest very similar improvements in the pre-post intervention scores by intervention group, year, and gender. This suggests that students either receiving an educational session via an AYF presenter or playing the MIApp can expect to improve their knowledge of IMD in the 9 key areas outlined in the previous sections.
Discussion
Principal Findings
This study demonstrated the efficacy of a serious game, MIApp, in improving knowledge about IMD in high school–aged children at equivalent levels to that of a face-to-face presentation delivered by a content expert. Serious games are attracting significant attention for promoting health education among children and adolescents [Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]4-Wouters P, van Nimwegen C, van Oostendorp H, van der Spek E. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology. May 2013;105(2):249-265. [CrossRef]6,Charlier N, Zupancic N, Fieuws S, Denhaerynck K, Zaman B, Moons P. Serious games for improving knowledge and self-management in young people with chronic conditions: a systematic review and meta-analysis. J Am Med Inform Assoc. Jan 2016;23(1):230-239. [FREE Full text] [CrossRef] [Medline]45], yet there is still a need to understand their effectiveness [Espinosa-Curiel IE, Pozas-Bogarin EE, Hernández-Arvizu M, Navarro-Jiménez ME, Delgado-Pérez EE, Martínez-Miranda J, et al. HelperFriend, a serious game for promoting healthy lifestyle behaviors in children: design and pilot study. JMIR Serious Games. May 06, 2022;10(2):e33412. [FREE Full text] [CrossRef] [Medline]22,DeSmet A, Shegog R, Van Ryckeghem D, Crombez G, De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J. Apr 2015;4(2):78-90. [CrossRef] [Medline]27,Kato PM. Evaluating efficacy and validating games for health. Games Health J. Feb 2012;1(1):74-76. [CrossRef] [Medline]30]. Our literature review suggested that, although serious games may offer a viable alternative to traditional pedagogical methods in terms of knowledge improvement, behavioral and attitudinal change, and engagement, the varied nature of serious games used in health promotion and the lack of standardized tools to measure outcomes mean that the results are heterogeneous, often with small sample sizes and high attrition [Andrew L, Barwood D, Boston J, Masek M, Bloomfield L, Devine A. Serious games for health promotion in adolescents - a systematic scoping review. Educ Inf Technol (Dordr). 2023;28(5):5519-5550. [FREE Full text] [CrossRef] [Medline]18].
This study compared the efficacy of the serious game MIApp with a traditional classroom health education model on knowledge about IMD in a sample of secondary school students. The results demonstrated that this serious game (1) significantly improved knowledge about the disease among adolescents, (2) had an outcome statistically equivalent to that of the current face-to-face educational approach, and (3) was easy to use and more enjoyable. Importantly, the study showed that gains in knowledge from playing the serious game MIApp were sustained 3 months later at equivalent levels to those in the control arm.
In line with recent research, this study supports the notion that age-appropriate, self-navigating serious games may be considered a positive learning tool for delivering school-based health education [Holzmann SL, Schäfer H, Groh G, Plecher DA, Klinker G, Schauberger G, et al. Short-term effects of the serious game "Fit, Food, Fun" on nutritional knowledge: a pilot study among children and adolescents. Nutrients. Aug 30, 2019;11(9):A. [CrossRef] [Medline]25,Chu SKW, Kwan ACM, Reynolds R, Mellecker RR, Tam F, Lee G, et al. Promoting sex education among teenagers through an interactive game: reasons for success and implications. Games Health J. Jun 2015;4(3):168-174. [CrossRef] [Medline]26,Froome HM, Townson C, Rhodes S, Franco-Arellano B, LeSage A, Savaglio R, et al. The effectiveness of the Foodbot Factory mobile serious game on increasing nutrition knowledge in children. Nutrients. Nov 06, 2020;12(11):1. [CrossRef] [Medline]46,Haruna H, Hu X, Chu SKW, Mellecker RR. Initial validation of the MAKE framework: a comprehensive instrument for evaluating the efficacy of game-based learning and gamification in adolescent sexual health literacy. Ann Glob Health. Feb 28, 2019;85(1):A. [FREE Full text] [CrossRef] [Medline]47]. One fundamental feature of serious games is that they are “fun” and pedagogically sound [Ronimus M, Eklund K, Pesu L, Lyytinen H. Supporting struggling readers with digital game-based learning. Education Tech Research Dev. Feb 20, 2019;67(3):639-663. [CrossRef]7-Zyda M. From visual simulation to virtual reality to games. Computer. Sep 2005;38(9):25-32. [CrossRef]9]. As illustrated by the MIApp study, adolescents find digital game–based learning more enjoyable than a traditional classroom presentation, characteristics that are acknowledged as essential for effective engagement [Starks K. Cognitive behavioral game design: a unified model for designing serious games. Front Psychol. 2014;5:28. [FREE Full text] [CrossRef] [Medline]48]. By providing a captivating learning environment, serious games can intrinsically and extrinsically motivate adolescents to promote active learning to understand health issues better [Papastergiou M. Exploring the potential of computer and video games for health and physical education: A literature review. Computers & Education. Nov 2009;53(3):603-622. [CrossRef]17,Kato PM. Evaluating efficacy and validating games for health. Games Health J. Feb 2012;1(1):74-76. [CrossRef] [Medline]30,Sailer M, Hense JU, Mayr S, Mandl H. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior. Apr 2017;69:371-380. [CrossRef]49]. Research also suggests that, through the provision of interactive engagement and immediate feedback, serious games can maintain students’ involvement for a longer period [Gee JP. Pleasure, learning, video games, and life: the projective stance. E-Learning and Digital Media. Sep 01, 2005;2(3):211-223. [CrossRef]50-Shute VJ, Ventura M, Bauer M, Zapata-Rivera D. Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In: Ritterfeld U, Cody M, Vorderer P, editors. Serious games: Mechanisms and effects. New York, NY. Routledge; 2009:317-343.52], supporting higher-level thinking and increasing learner independence [Newhouse C. STEM the boredom: engage students in the Australian curriculum using ICT with problem-based learning and assessment. J Sci Educ Technol. Sep 14, 2016;26(1):44-57. [CrossRef]10].
With increasing numbers of children and adolescents in Australia using digital games [9 in 10 Aussie teens now have a mobile (and most are already on to their second or subsequent handset). Roy Morgan. Aug 22, 2016. URL: http://www.roymorgan.com/findings/6929-australian-teenagers-and-their-mobile-phones-june-2016-201608220922 [accessed 2024-12-07] 1], together with the immediate transition to digital learning during the COVID-19 pandemic [Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, et al. Digital technologies in the public-health response to COVID-19. Nat Med. Aug 2020;26(8):1183-1192. [CrossRef] [Medline]53], serious games offer a unique educational platform to improve the reach of health education [DeSmet A, Van Ryckeghem D, Compernolle S, Baranowski T, Thompson D, Crombez G, et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev Med. Dec 2014;69:95-107. [FREE Full text] [CrossRef] [Medline]5,Rodriguez DM, Teesson M, Newton NC. A systematic review of computerised serious educational games about alcohol and other drugs for adolescents. Drug Alcohol Rev. Mar 2014;33(2):129-135. [CrossRef] [Medline]28,Ravyse WS, Seugnet Blignaut A, Leendertz V, Woolner A. Success factors for serious games to enhance learning: a systematic review. Virtual Reality. Sep 20, 2016;21(1):31-58. [CrossRef]54,Chow CY, Riantiningtyas RR, Kanstrup MB, Papavasileiou M, Liem GD, Olsen A. Can games change children’s eating behaviour? A review of gamification and serious games. Food Quality and Preference. Mar 2020;80:103823. [CrossRef]55]. As a self-guided resource not limited to staff facilitating an education session, serious games can support teachers to provide health education around topics they have limited knowledge about or may find difficult to discuss [Ronimus M, Eklund K, Pesu L, Lyytinen H. Supporting struggling readers with digital game-based learning. Education Tech Research Dev. Feb 20, 2019;67(3):639-663. [CrossRef]7]. Moreover, this study revealed that students using MIApp also retained their IMD knowledge over 3 months, as did those who attended a face-to-face educational presentation. Although research has suggested serious games might be more beneficial for long-term retention of health information due to their enjoyable format [Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]4], to date, most studies have focused primarily on short-term pre- and posttests [Holzmann SL, Schäfer H, Groh G, Plecher DA, Klinker G, Schauberger G, et al. Short-term effects of the serious game "Fit, Food, Fun" on nutritional knowledge: a pilot study among children and adolescents. Nutrients. Aug 30, 2019;11(9):A. [CrossRef] [Medline]25].
Although this study was conducted with schools in primarily medium-to-high ICSEA areas of Perth, WA, as shown in Table 2, its relevance needs to be assessed in rural settings where cases of IMD are higher; the preliminary findings of this study suggest MIApp could represent an emerging resource in the space of meningococcal education for adolescents and young adults in Australia. Key features underlying MIApp are known to promote motivational learning, improve knowledge, and promote engagement, which can increase educational achievement [Arnab S, Lim T, Carvalho M, Bellotti F, de Freitas S, Louchart S, et al. Mapping learning and game mechanics for serious games analysis. Brit J Educational Tech. Jan 05, 2014;46(2):391-411. [CrossRef]56,Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games. Dec 12, 2013;1(1):e3. [FREE Full text] [CrossRef] [Medline]57], and are likely responsible for the positive findings observed in our study [Froome HM, Townson C, Rhodes S, Franco-Arellano B, LeSage A, Savaglio R, et al. The effectiveness of the Foodbot Factory mobile serious game on increasing nutrition knowledge in children. Nutrients. Nov 06, 2020;12(11):1. [CrossRef] [Medline]46,Shute VJ, Ventura M, Bauer M, Zapata-Rivera D. Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In: Ritterfeld U, Cody M, Vorderer P, editors. Serious games: Mechanisms and effects. New York, NY. Routledge; 2009:317-343.52,DeSmet A, Thompson D, Baranowski T, Palmeira A, Verloigne M, De Bourdeaudhuij I. Is Participatory Design Associated with the Effectiveness of Serious Digital Games for Healthy Lifestyle Promotion? A Meta-Analysis. J Med Internet Res. Apr 29, 2016;18(4):e94. [FREE Full text] [CrossRef] [Medline]58]. MIApp is freely downloadable from the App store [MIApp. App Store. URL: https://apps.apple.com/us/app/miapp/id1623282445 [accessed 2024-12-07]
59] on personal devices or is available on the AYF website [The Amanda Young Foundation. URL: https://www.amandayoungfoundation.org.au/ [accessed 2024-12-07]
60] as a desktop version, accessible to all, including those regional and rural communities that previously would not have had the resources or access to receive the face-to-face guest presentations. Findings of this study can help teachers to feel more assured that students are gaining the same learning opportunity through an interactive and engaging app as they would receive from a qualified presenter.
As shown in previous research [Chu SKW, Kwan ACM, Reynolds R, Mellecker RR, Tam F, Lee G, et al. Promoting sex education among teenagers through an interactive game: reasons for success and implications. Games Health J. Jun 2015;4(3):168-174. [CrossRef] [Medline]26,Sharifzadeh N, Kharrazi H, Nazari E, Tabesh H, Edalati Khodabandeh M, Heidari S, et al. Health education serious games targeting health care providers, patients, and public health users: scoping review. JMIR Serious Games. Mar 05, 2020;8(1):e13459. [FREE Full text] [CrossRef] [Medline]31,Haruna H, Hu X, Chu SKW, Mellecker RR. Initial validation of the MAKE framework: a comprehensive instrument for evaluating the efficacy of game-based learning and gamification in adolescent sexual health literacy. Ann Glob Health. Feb 28, 2019;85(1):A. [FREE Full text] [CrossRef] [Medline]47,Arnab S, Lim T, Carvalho M, Bellotti F, de Freitas S, Louchart S, et al. Mapping learning and game mechanics for serious games analysis. Brit J Educational Tech. Jan 05, 2014;46(2):391-411. [CrossRef]56], engaging key stakeholders in the co-design, development, testing, and refining of the serious game MIApp was essential; adopting a co-design approach with all stakeholders is useful in promoting health communication interventions and enhancing the acceptability, usability, and utility of the serious game [Chu SKW, Kwan ACM, Reynolds R, Mellecker RR, Tam F, Lee G, et al. Promoting sex education among teenagers through an interactive game: reasons for success and implications. Games Health J. Jun 2015;4(3):168-174. [CrossRef] [Medline]26]. For MIApp, collaborators included an interdisciplinary team of researchers, experts, and community partners with varied expertise in education, health and computer science, and game development. This collaboration facilitated important development and design improvements of the game interface, mechanism, health content, and pedagogy for students and ensured that the social and cultural interests of the targeted adolescent group were included.
For greater influence, serious games for health education and promotion should be designed in a way that enables them to be embedded in national curriculum, as this gives teachers the opportunity to combine gameplay with the mandated curriculum to address learning outcomes [Li M, Tsai C. Game-based learning in science education: a review of relevant research. J Sci Educ Technol. Feb 6, 2013;22(6):877-898. [CrossRef]15,Masek M, Boston J, Lam C, Corcoran S. Improving mastery of fractions by blending video games into the Math classroom. Computer Assisted Learning. Apr 27, 2017;33(5):486-499. [CrossRef]16,Andrew L, Barwood D, Boston J, Masek M, Bloomfield L, Devine A. Serious games for health promotion in adolescents - a systematic scoping review. Educ Inf Technol (Dordr). 2023;28(5):5519-5550. [FREE Full text] [CrossRef] [Medline]18]. MIApp was aligned specifically with the teaching and learning HPE curriculum as outlined by the Australian curriculum and, more specifically, the lower secondary year-level syllabi for the HPE Learning Area in WA [Health and Physical Education. School Curriculum and Standards Authority. URL: https://k10outline.scsa.wa.edu.au/home/teaching/curriculum-browser/health-and-physical-education [accessed 2024-12-07] 40]. Additionally, a free downloadable teaching and learning curriculum companion of activities was developed for teachers to assist them to cognitively build and support student understanding of concepts and health-enhancing skills and dispositions related to meningococcal awareness and prevention and is available from the AYF website. These learning activities have also been aligned to meet the teaching and learning requirements of the HPE curriculum.
The overall educational package draws on 5 interrelated ideas underpinning the pedagogy in the delivery of HPE in Australia. The broader dissemination of this tool by teachers as a public health intervention could support the digital implementation of national efforts to reduce IMD among adolescents. The demonstrated efficacy of MIApp means that educators, particularly in rural and remote areas without previous access to specialist education sessions about IMD, can deploy this game in the classroom to provide their students with self-guided education supported by teaching resources.
Limitations
There were several limitations to this research. First, a major limitation to this study was the low response rate and match rate for the 3-month follow-up survey. There was a relatively low rate of complete responses (255/788, 32.4%) for the 3-month follow-up; the response rate was significantly lower for the MIApp group (97/442, 21.9%) than the control group (158/346, 45.7%). Attrition for this follow-up survey was expected to be low, and our literature review highlighted significant issues with attrition experienced by researchers running similar interventions [Andrew L, Barwood D, Boston J, Masek M, Bloomfield L, Devine A. Serious games for health promotion in adolescents - a systematic scoping review. Educ Inf Technol (Dordr). 2023;28(5):5519-5550. [FREE Full text] [CrossRef] [Medline]18]. Although attrition was built into the sample size calculation, the 3-month follow-up response rate was still much lower than anticipated. Future projects may consider methods to improve response rates, including incentives for completing follow-up surveys.
Further, only 35% of completed surveys could be linked to the identifiers for the pre-postintervention arm, due to students entering an identifier that did not match the one created on the day of trial. The method for creating a unique identifier involved using parts of key demographic information (name and date of birth) that were considered robust and unlikely to change over the study period. Despite this assumption, 65% of identifiers and school codes did not match those created on the day of the trial. A more robust identifier may be considered for future trials where follow-up data need to be linked to data collected on the day of the trial.
Finally, a significantly lower proportion of responses in the 3-month follow-up was received from those in the intervention arm than those in the control arm. The implications of this difference are that, potentially, respondents from the intervention arm were systematically different from those who did not respond. If respondents reflect those who were more likely to retain information, this may have biased the results. The overall low response rate in both groups may suggest this bias would be present in both; however, residual bias from the difference in response rates between groups may still be present. A sensitivity analysis using only surveys that were able to be linked to the primary trial data, which also allows independent confirmation that the correct trial arm was selected on the follow-up survey, yielded a response rate that was not significantly different between groups. In addition, the data from these groups confirmed the 1-point difference in correct responses between the intervention and control arms reported in the larger follow-up cohort, which was within the predetermined equivalence threshold. This increases our confidence that the results reported in the larger follow-up sample were reflective of the larger cohort.
Despite these limitations, the completed surveys suggested that many key messages delivered via either MIApp or in a face-to-face presentation are retained 3 months after a single 30-minute session. Future studies assessing the longer-term efficacy of serious games for health promotion may consider outcome measures of behavior change that do not rely on participant response rate (eg, vaccination and infection rates).
The overall findings of this study suggest that serious games, such as MIApp, represent an effective strategic avenue for the use of digital technology to engage adolescents in learning about key health issues. The research found both delivery methods to be similar in improving knowledge and attitudes toward meningococcal carriage and disease signs, symptoms, and prevention. Coupled with greater reported enjoyment, if well-designed, serious games could be used in national health education. The strengths of this study included the engagement of a range of key stakeholders in its planning and design to improve outcomes and ensure specificity of knowledge relating to the curriculum and possibilities for content inclusion in the teaching and learning of HPE in Australia. Although further investigations are needed to explore any minor changes required for the delivery in the diverse socioeconomic, cultural, and geographic settings of Australia, this study has illustrated the efficacy of the MIApp game. Moreover, data generated in this study can support evidence for the development of future serious games for adolescent health education worldwide. Today, in our digital landscape, greater understanding of the game learning interests of adolescent students is paramount to actively promote their active engagement in health education.
Acknowledgments
Funding for this study was provided by a Lotterywest Grant (2019–00736-BLOOMFIELD), with additional funding from the Western Australia Department of Health, the Amanda Young Foundation (AYF), and Edith Cowan University (ECU).
Data Availability
The data sets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Authors' Contributions
LB, AD, LA, DB, and MM conceptualized the study. LB and AD curated the data, acquired funding, and designed the methodology. LB and JB performed the formal analysis. JB, AD, and DB performed the investigation. AD, JB, and DB provided project administration. AD provided the resources. MM provided the software. AD, JB, and MM supervised the study. LB created the visualizations. LB, JB, and DB wrote the original draft of the manuscript. LB, AD, LA, JB, DB, and MM reviewed and edited the manuscript.
Conflicts of Interest
None declared.
Multimedia Appendix 5
Amanda Young Foundation classroom presentation.
PDF File (Adobe PDF File), 843 KBMultimedia Appendix 7
CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) form for the Meningococcal Immunisation Awareness, Prevention and Protection app (MIApp) study.
PDF File (Adobe PDF File), 894 KBReferences
- 9 in 10 Aussie teens now have a mobile (and most are already on to their second or subsequent handset). Roy Morgan. Aug 22, 2016. URL: http://www.roymorgan.com/findings/6929-australian-teenagers-and-their-mobile-phones-june-2016-201608220922 [accessed 2024-12-07]
- Emmerich K, Masuch M, Schmidt R. Digital Entertaining Health Applications: Researching the Fundamentals of a Cause-Effect Relationship. 2014. Presented at: Workshops on Advances in Computer Entertainment Conference; November 11-14, 2014; Funchal, Portugal. [CrossRef]
- Qian M, Clark K. Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior. Oct 2016;63:50-58. [CrossRef]
- Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs. Feb 2009;65(2):259-269. [CrossRef] [Medline]
- DeSmet A, Van Ryckeghem D, Compernolle S, Baranowski T, Thompson D, Crombez G, et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev Med. Dec 2014;69:95-107. [FREE Full text] [CrossRef] [Medline]
- Wouters P, van Nimwegen C, van Oostendorp H, van der Spek E. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology. May 2013;105(2):249-265. [CrossRef]
- Ronimus M, Eklund K, Pesu L, Lyytinen H. Supporting struggling readers with digital game-based learning. Education Tech Research Dev. Feb 20, 2019;67(3):639-663. [CrossRef]
- Zainuddin Z. Students' learning performance and perceived motivation in gamified flipped-class instruction. Computers & Education. Nov 2018;126:75-88. [CrossRef]
- Zyda M. From visual simulation to virtual reality to games. Computer. Sep 2005;38(9):25-32. [CrossRef]
- Newhouse C. STEM the boredom: engage students in the Australian curriculum using ICT with problem-based learning and assessment. J Sci Educ Technol. Sep 14, 2016;26(1):44-57. [CrossRef]
- Annetta LA, Minogue J, Holmes SY, Cheng M. Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education. Aug 2009;53(1):74-85. [CrossRef]
- Plass JL, Homer BD, Kinzer CK. Foundations of game-based learning. Educational Psychologist. Feb 06, 2016;50(4):258-283. [CrossRef]
- Young MF, Slota S, Cutter AB, Jalette G, Mullin G, Lai B, et al. Our princess is in another castle. Review of Educational Research. Mar 01, 2012;82(1):61-89. [CrossRef]
- Barwood D, Smith S, Miller M, Boston J, Masek M, Devine A. Transformational game trial in nutrition education. Australian Journal of Teacher Education. Apr 2020;45(4):18-29. [CrossRef]
- Li M, Tsai C. Game-based learning in science education: a review of relevant research. J Sci Educ Technol. Feb 6, 2013;22(6):877-898. [CrossRef]
- Masek M, Boston J, Lam C, Corcoran S. Improving mastery of fractions by blending video games into the Math classroom. Computer Assisted Learning. Apr 27, 2017;33(5):486-499. [CrossRef]
- Papastergiou M. Exploring the potential of computer and video games for health and physical education: A literature review. Computers & Education. Nov 2009;53(3):603-622. [CrossRef]
- Andrew L, Barwood D, Boston J, Masek M, Bloomfield L, Devine A. Serious games for health promotion in adolescents - a systematic scoping review. Educ Inf Technol (Dordr). 2023;28(5):5519-5550. [FREE Full text] [CrossRef] [Medline]
- Barber NC, Stark LA. Online resources for understanding outbreaks and infectious diseases. CBE Life Sci Educ. Mar 02, 2015;14(1):fe1. [FREE Full text] [CrossRef] [Medline]
- Fontes P, Boaventura P, Azevedo J. Mobile games to improve healthier lifestyle amongst youth: "are we there yet" or should we move forward? 2014. Presented at: Workshops on Advances in Computer Entertainment Conference; November 11-14, 2014; Funchal, Portugal. [CrossRef]
- Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertainment Computing. Dec 2013;4(4):231-247. [CrossRef]
- Espinosa-Curiel IE, Pozas-Bogarin EE, Hernández-Arvizu M, Navarro-Jiménez ME, Delgado-Pérez EE, Martínez-Miranda J, et al. HelperFriend, a serious game for promoting healthy lifestyle behaviors in children: design and pilot study. JMIR Serious Games. May 06, 2022;10(2):e33412. [FREE Full text] [CrossRef] [Medline]
- Ricciardi F, De Paolis LT. A comprehensive review of serious games in health professions. International Journal of Computer Games Technology. 2014;2014:1-11. [CrossRef]
- Baranowski T, Ryan C, Hoyos-Cespedes A, Lu AS. Nutrition education and dietary behavior change games: a scoping review. Games Health J. Jun 2019;8(3):153-176. [FREE Full text] [CrossRef] [Medline]
- Holzmann SL, Schäfer H, Groh G, Plecher DA, Klinker G, Schauberger G, et al. Short-term effects of the serious game "Fit, Food, Fun" on nutritional knowledge: a pilot study among children and adolescents. Nutrients. Aug 30, 2019;11(9):A. [CrossRef] [Medline]
- Chu SKW, Kwan ACM, Reynolds R, Mellecker RR, Tam F, Lee G, et al. Promoting sex education among teenagers through an interactive game: reasons for success and implications. Games Health J. Jun 2015;4(3):168-174. [CrossRef] [Medline]
- DeSmet A, Shegog R, Van Ryckeghem D, Crombez G, De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J. Apr 2015;4(2):78-90. [CrossRef] [Medline]
- Rodriguez DM, Teesson M, Newton NC. A systematic review of computerised serious educational games about alcohol and other drugs for adolescents. Drug Alcohol Rev. Mar 2014;33(2):129-135. [CrossRef] [Medline]
- David OA, Costescu C, Cardos R, Mogoaşe C. How effective are serious games for promoting mental health and health behavioral change in children and adolescents? A systematic review and meta-analysis. Child Youth Care Forum. Jul 31, 2020;49(6):817-838. [CrossRef]
- Kato PM. Evaluating efficacy and validating games for health. Games Health J. Feb 2012;1(1):74-76. [CrossRef] [Medline]
- Sharifzadeh N, Kharrazi H, Nazari E, Tabesh H, Edalati Khodabandeh M, Heidari S, et al. Health education serious games targeting health care providers, patients, and public health users: scoping review. JMIR Serious Games. Mar 05, 2020;8(1):e13459. [FREE Full text] [CrossRef] [Medline]
- Meningococcal Infection Notifications 2018. 32 WA Department of Health. 2018. URL: http://ww2.health.wa.gov.au/Articles/N_R/Notifiable-infectious-disease [accessed 2020-09-01]
- Sadarangani M, Scheifele DW, Halperin SA, Vaudry W, Le Saux N, Tsang R, et al. investigators of the Canadian Immunization Monitoring Program‚ ACTive (IMPACT). Outcomes of invasive meningococcal disease in adults and children in Canada between 2002 and 2011: a prospective cohort study. Clin Infect Dis. Apr 15, 2015;60(8):e27-e35. [CrossRef] [Medline]
- Burman C, Serra L, Nuttens C, Presa J, Balmer P, York L. Meningococcal disease in adolescents and young adults: a review of the rationale for prevention through vaccination. Hum Vaccin Immunother. 2019;15(2):459-469. [FREE Full text] [CrossRef] [Medline]
- Invasive Meningococcal Disease: CDNA National Guidelines for Public Health Units. Communicable Diseases Network Australia. Jul 01, 2024. URL: https://www.health.gov.au/sites/default/files/2024-07/invasive-meningococcal-disease-cdna-national-guidelines-for-public-health-units.pdf [accessed 2024-12-07]
- MIApp Years 7 to 10. The Amanda Young Foundation. URL: https://amandayoungfoundation.org.au/miapp-years-7-to-10/ [accessed 2024-12-07]
- Find a School. Australian Curriculum, Assessment and Reporting Authority. URL: https://www.myschool.edu.au/ [accessed 2024-12-07]
- Klar N, Donner A. Design and Analysis of Cluster Randomization Trials in Health Research. London, England. Wiley; 2000.
- Brooke J. SUS - A quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability evaluation in industry (Vol. 189). New York, NY. Routledge; 1996:4-7.
- Health and Physical Education. School Curriculum and Standards Authority. URL: https://k10outline.scsa.wa.edu.au/home/teaching/curriculum-browser/health-and-physical-education [accessed 2024-12-07]
- Qualtrics. URL: https://www.qualtrics.com/ [accessed 2024-12-07]
- R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria.; 2021. URL: https://www.R-project.org/ [accessed 2024-12-07]
- Kruskal WH, Wallis WA. Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association. Dec 1952;47(260):583-621. [CrossRef]
- Rusticus SA, Lovato CY. Applying tests of equivalence for multiple group comparisons: demonstration of the confidence interval approach. Practical Assessment, Research, and Evaluation. 2011;16(1):7. [CrossRef]
- Charlier N, Zupancic N, Fieuws S, Denhaerynck K, Zaman B, Moons P. Serious games for improving knowledge and self-management in young people with chronic conditions: a systematic review and meta-analysis. J Am Med Inform Assoc. Jan 2016;23(1):230-239. [FREE Full text] [CrossRef] [Medline]
- Froome HM, Townson C, Rhodes S, Franco-Arellano B, LeSage A, Savaglio R, et al. The effectiveness of the Foodbot Factory mobile serious game on increasing nutrition knowledge in children. Nutrients. Nov 06, 2020;12(11):1. [CrossRef] [Medline]
- Haruna H, Hu X, Chu SKW, Mellecker RR. Initial validation of the MAKE framework: a comprehensive instrument for evaluating the efficacy of game-based learning and gamification in adolescent sexual health literacy. Ann Glob Health. Feb 28, 2019;85(1):A. [FREE Full text] [CrossRef] [Medline]
- Starks K. Cognitive behavioral game design: a unified model for designing serious games. Front Psychol. 2014;5:28. [FREE Full text] [CrossRef] [Medline]
- Sailer M, Hense JU, Mayr S, Mandl H. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior. Apr 2017;69:371-380. [CrossRef]
- Gee JP. Pleasure, learning, video games, and life: the projective stance. E-Learning and Digital Media. Sep 01, 2005;2(3):211-223. [CrossRef]
- Prensky M. Digital game-based learning. Comput. Entertain. Oct 2003;1(1):21-21. [CrossRef]
- Shute VJ, Ventura M, Bauer M, Zapata-Rivera D. Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In: Ritterfeld U, Cody M, Vorderer P, editors. Serious games: Mechanisms and effects. New York, NY. Routledge; 2009:317-343.
- Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, et al. Digital technologies in the public-health response to COVID-19. Nat Med. Aug 2020;26(8):1183-1192. [CrossRef] [Medline]
- Ravyse WS, Seugnet Blignaut A, Leendertz V, Woolner A. Success factors for serious games to enhance learning: a systematic review. Virtual Reality. Sep 20, 2016;21(1):31-58. [CrossRef]
- Chow CY, Riantiningtyas RR, Kanstrup MB, Papavasileiou M, Liem GD, Olsen A. Can games change children’s eating behaviour? A review of gamification and serious games. Food Quality and Preference. Mar 2020;80:103823. [CrossRef]
- Arnab S, Lim T, Carvalho M, Bellotti F, de Freitas S, Louchart S, et al. Mapping learning and game mechanics for serious games analysis. Brit J Educational Tech. Jan 05, 2014;46(2):391-411. [CrossRef]
- Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games. Dec 12, 2013;1(1):e3. [FREE Full text] [CrossRef] [Medline]
- DeSmet A, Thompson D, Baranowski T, Palmeira A, Verloigne M, De Bourdeaudhuij I. Is Participatory Design Associated with the Effectiveness of Serious Digital Games for Healthy Lifestyle Promotion? A Meta-Analysis. J Med Internet Res. Apr 29, 2016;18(4):e94. [FREE Full text] [CrossRef] [Medline]
- MIApp. App Store. URL: https://apps.apple.com/us/app/miapp/id1623282445 [accessed 2024-12-07]
- The Amanda Young Foundation. URL: https://www.amandayoungfoundation.org.au/ [accessed 2024-12-07]
Abbreviations
AYF: Amanda Young Foundation |
ECU: Edith Cowan University |
HPE: Healthy and Physical Education |
ICSEA: Index of Community Socio-Educational Advantage |
IMD: invasive meningococcal disease |
MIApp: Meningococcal Immunisation Awareness, Prevention and Protection app |
WA: Western Australia |
Edited by A Coristine; submitted 20.05.24; peer-reviewed by K Erenli, M McMillan; comments to author 29.07.24; revised version received 20.09.24; accepted 19.11.24; published 11.02.25.
Copyright©Lauren Bloomfield, Julie Boston, Martin Masek, Lesley Andrew, Donna Barwood, Amanda Devine. Originally published in JMIR Serious Games (https://games.jmir.org), 11.02.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.