This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.
Approximately 10%-12% of New Zealand children and young people have long-term physical conditions (chronic illnesses) and are more likely to develop psychological problems, particularly anxiety and depression. Delayed treatment leads to worse health care and poorer long-term outcomes. Recently, eHealth interventions, especially those based on principles of cognitive behavioral therapy and biofeedback, have been shown to be moderately effective in reducing anxiety. However, these modalities have rarely been combined. Young people have expressed a preference for well-designed and technology-based support to deal with psychological issues.
This study aims to co-design and evaluate the acceptability and usability of a cognitive behavioral therapy and biofeedback-based, 5-module eHealth game called
Users gave
This study provides preliminary evidence for
Australian New Zealand Clinical Trials Network Registry (ANZCTR) ACTRN12616001253493; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=371443
Long-term physical conditions (also known as chronic illnesses), defined as those lasting more than 3 months and impairing functioning, are common, affecting 10%-12% of children globally [
Psychological problems are more likely in children and young people with long-term physical conditions [
Access to and effectiveness of treatments for psychological problems in children and young people with long-term physical conditions are currently limited. Although they are traditionally addressed using generic psychotherapies, such as cognitive behavioral therapy (CBT) and pharmacotherapies such as anxiolytic or antidepressant medication, there is limited evidence that such therapies are effective for this population [
Over the past few decades, the increasing popularity of smart technology, release of gamified and app-based interventions, and calls from international organizations, such as The Lancet Global Mental Health Group [
Traditional psychological therapies often include a component of psychologically or chemically induced relaxation. There is increasing evidence that newer, more technology-based forms of therapy, such as biofeedback, may achieve similar results, either alone or in combination with traditional therapies [
In a recent study, New Zealand young people with long-term physical conditions confirmed that anxiety is the most significant psychological issue that they face [
The open trial, conceptualized by 3 authors (HT, KS, and SM), used a mixed-methods design, including quantitative analysis of anxiety symptoms and quality of life outcomes, intervention use, and qualitative analysis of participant feedback.
A total of 24 young people aged between 10 and 17 years were recruited from a tertiary children’s hospital in Auckland, New Zealand, between October 2018 and May 2020. Eligible participants had any type of long-term physical condition lasting for longer than 3 months (eg, asthma, diabetes, cancer, and cystic fibrosis) and measurable levels of anxiety (eg, specific phobia, generalized anxiety, and nonspecific anxiety) and may or may not have had comorbid mental health conditions. Eligible participants were of any ethnicity, intellectually and physically able to use the intervention, and understood enough English to play the game and provide informed consent or assent with paired parental consent if they were aged <16 years. Participants who did not meet all these criteria and those who had recently undertaken or were undertaking CBT or other forms of psychotherapy, biofeedback therapy, or pharmacotherapy with anxiolytic medication within the past 6 months were excluded because the effect of those therapies could confound the impact of the intervention.
Illustrative images from the game—clockwise from top left: bridge of starship; learning about the origins of anxiety; exploring the anxiety monster; planet of the mind.
The primary outcomes of the open trial were evaluated as follows: (1) acceptability of the prototype intervention (ie, is the content and format acceptable to users?) was quantitatively assessed via user ratings of overall acceptability and helpfulness on scales from 0 to 10 and qualitatively assessed via feedback during semistructured interviews following completion of the game; (2) usability of the intervention (ie, is it usable?) was quantitatively assessed using the System Usability Scale (SUS) [
Schedule of outcome measurement.
Outcome | Start of game | Start of each module | Completion of game | 3 months following completion |
Acceptability | N/Aa | N/A | User feedback via questionnaires and semistructured interviews | N/A |
Usability | N/A | N/A | System Usability Scale, user feedback via semistructured interviews | N/A |
Effectiveness | GAD-7b, SCASc, Likert VASd, PedsQLe | Likert VAS | GAD-7, SCAS, Likert VAS, Peds QL | GAD-7, SCAS, Peds QL |
aN/A: not applicable.
bGAD-7: Generalized Anxiety Disorder-7 item.
cSCAS: Spence Children’s Anxiety Scale.
dVAS: visual analog scale.
ePedsQL: Pediatric Quality of Life Inventory.
Quantitative data were analyzed by our biostatistician (CF) using Excel (version 16, Microsoft Inc) and SPSS (version 25, IBM Corp). Analyses included basic descriptive statistics (eg, number of sessions completed, number of times device accessed, duration of use, changes in anxiety score, and demographic characteristics of the sample). McNemar chi-square tests and one-tailed
This study received ethics approval from the New Zealand Health and Disability Ethics Committee (HDEC, 16/CEN/136) on September 30, 2016. The lower age limit for participation was initially set at 12 years but later extended down to 10 years following a period of slow recruitment. Invitations to participate in the study were forwarded to potential participants through their own clinicians to minimize coercion using a direct approach. Verbal and written consent was obtained directly for those aged >16 years and via their parents with paired participant assent for those aged <16 years. Participants were free to discontinue engagement at any stage without consequence, and this was made clear to them. Although plans were made for any unanticipated distress occurring during participation to be managed by immediate referral to the hospital-based pediatric consultation-liaison mental health team, of which the lead author (HT) is a team member, this never occurred. Data were securely stored on a department server and kept securely for 10 years (or 10 years following younger participants’ 16th birthday) according to the ethics committee requirements.
A total of 15 participants aged between 8 and 16 years, of mixed gender (10 males and 5 females) and with different long-term physical conditions (cancer, asthma, bronchiectasis, cystic fibrosis, Alport syndrome, and others) provided feedback, 2 of which on multiple occasions. User feedback was incorporated to address technical issues, make instructions clearer, and develop the game’s look and feel. Examples of user feedback during the first sprint and the use of this feedback are provided in
Generally positive feedback regarding look or feel, for example, “It’s fun,” “I liked how some monsters chase you, and others need to be found.”
None
Technical issue identified: “Only one little bug, getting stuck in the block.”
To be fixed by game developer
Unsure whether different colored crystals are the same
Clarification to be added to introduction to module 3
Hard to recall positive and negative feelings when asked
Summary list to be added to the end of module 1
Re. ideal audience for the game: “I think younger kids, probably 8-15 years, any (boys and girls)”
None, current game probably appropriate for target age range
A total of 32 participants (different from those who participated in the co-design process) were referred by their clinicians to participate in the open trial of the
Participant characteristics (n=24).
Characteristics | Values | ||
Age (years), mean (range) | 14 (10-17) | ||
|
|||
|
Male | 9 (38) | |
|
Female | 15 (63) | |
|
|||
|
Cancer | 4 (17) | |
|
Transplant (heart, liver, and kidney) | 4 (17) | |
|
Epilepsy | 2 (8) | |
|
Juvenile idiopathic arthritis | 2 (8) | |
|
Nut allergy | 2 (8) | |
|
Other | 10 (42) |
Participants gave
Qualitative feedback regarding acceptability.
Theme and subtheme | Supporting examples (participant number) | |
Helpfulness for managing anxiety |
“I enjoyed the games and thought the game gave quite good techniques.” [P13] “During the games where you had to keep your heart rate down, and breathing exercises, I did find ways to slow down my breathing, and calm my heart rate, which was good.” [P8] “It taught me a lot of breathing skills.” [P14] “The game points out very helpful things that you don’t really think about.” [P15] |
|
|
||
|
|
|
|
|
“The game was fairly easy to control and fairly smooth running.” [P5] “It was informative and the animations were fun.” [P4] “The heart rate monitor was fun - to see where my heart was at.” [P1] “It was really fun and I would do it again.” [P18] |
|
|
|
|
|
“The game was too difficult in module 3.” [P4] “Bit too much talking and felt like module 2 was the same as module 1.” [P20] “I don’t feel like the game was for my age (15 years) and not enough shooting.” [P5] |
Participants had mixed views on the usability of
System Usability Scale subscales.
System Usability Scale itema | Values, mean (SD; range) |
I thought the game was easy to use (+) | 4.18 (0.92; 1-5) |
I found the various functions in this game were well-integrated (+) | 3.77 (1.05; 2-5) |
I would imagine that most people would learn to use this game very quickly (+) | 3.91 (0.87; 1-5) |
I felt very confident using the game (+) | 3.84 (1.20; 2-5) |
I think that I would like to use this game frequently (+) | 2.22 (1.02; 1-4) |
I found the game unnecessarily complex (−) | 1.91 (1.11; 1-5) |
I found the game very cumbersome to use (−) | 2.95 (0.67; 1-5) |
I think that I would need support of a technical person to be able to use this game (−) | 1.45 (0.81; 1-3) |
I thought there was too much inconsistency in this game (−) | 1.66 (1.20; 1-4) |
I needed to learn a lot of things before I could get going with this game (−) | 1.57 (0.90; 1-4) |
a(+) higher scores indicate greater usability; (−) lower scores indicate decreased usability.
Time taken to complete each module and the whole game.
|
Completion (n=24), n (%) | Values, mean (SD; range) |
Module 1 | 23 (96) | 12.4 days (41.1; 11 minutes-142.9 days) |
Module 2 | 23 (96) | 5.6 days (7.2; 14 minutes-19.7 days) |
Module 3 | 19 (79) | 25.3 days (38.6; 26 minutes-128.3 days) |
Module 4 | 17 (71) | 13.6 days (18.7; 19 minutes-51.1 days) |
Module 5 | 16 (67) | 3.8 minutes (0.0006; 3.0 minutes-5.0 minutes) |
Total | N/Aa | 79.4 days (9.52; 12.0 days-243.9 days)b |
aN/A: not applicable.
bOn the basis of participants with completed data for all five modules.
“Some controls were a bit touchy and pressing the back button on the tablet would reset the progress on that module.” [P1]
“Module three was difficult to pass.” [P11]
“The games sometimes took a while to get the hang of.” [P3]
“Just at home in my room.” [P19]
“At home.” [P15]
“Sometimes, if I didn’t know what to do...I asked my parents, or my bigger brother.” [P15]
“[My mum] was actually quite involved; she just asked questions about it.” [P19]
“Make cut scenes skippable and add sections/chapters to each module.” [P8]
“Add a pause button that automatically pauses the game if you leave, so you don’t lose progress.” [P15]
“Disable the back button or use a different tablet.” [P1]
“Have less backstory about the Starship and a more detailed description on how to play the mini-games.” [P13]
“Add a double jump bar.” [for module 3; P17]
Participants reported concordant changes in anxiety using three separate scales: GAD-7 [
Change in anxiety on General Anxiety Disorder-7 item, Spence Children’s Anxiety Scale, and Likert visual analog scales and quality of life on the Pediatric Quality of Life Inventory scale.
|
Generalized Anxiety Disorder, 7-item scale | Spence Children’s Anxiety Scale | Likert visual analog scale | Pediatric Quality of Life Inventory scale | ||||||||
|
Pre | Post | 3 months | Pre | Post | 3 months | Pre | Post | Pre | Post | ||
Participant, n | 24 | 21 | 22 | 23 | 21 | 22 | 18 | 16 | 24 | 21 | ||
Value, mean (SD; range) | 9.9 (5.4; 0-21) | 5.3 (3.2; 1-12) | 6.2 (4.7; 0-21) | 35.7 (16.0; 13-69) | 26.1 (13.9; 12-62) | 26.0 (16.0; 8-79) | 6.2 (1.5; 3.5-9.0) | 3.8 (1.9; 0-7.5) | 63.7 (15.9; 28.33-93.33) | 68.0 (15.5; 40-88.33) | ||
N/Aa | <.001 | .001 | N/A | <.001 | .005 | N/A | .001 | N/A | .04 |
aN/A: not applicable.
Our findings provide preliminary evidence that
We believe that positive design features of
Despite being co-designed with its target audience, further minor modifications are warranted before the RCT to address some of the intervention’s technical aspects and improve its usability and acceptability. The fact that the mean length of time taken to complete the intervention exceeded expectations may be related to a number of issues. Extensive completion times for module 1 are likely to be because of participants being assisted by us to access module 1 during the onboarding process but not actually commencing or completing it until a later date. The delay in completing module 3 is likely to be related to participants getting stuck while playing the embedded platform game. Finally, although the requirement for real-world mastery of a chosen source of anxiety during module 4 may have proved challenging for users, this remains a vital means of generalizing therapeutic knowledge into practice [
The strengths of this study are the co-design of
Description of content of the game and purpose of each module.
Change in symptom severity on the Generalized Anxiety Disorder Scale.
Body Signs, Relaxation, Active Helpful Thoughts, Victory Over Your Fears, Enjoy
cognitive behavioral therapy
ecological momentary assessment
Generalized Anxiety Disorder, 7-item
Heart Rate Variability
Pediatric Quality of Life Inventory
randomized controlled trial
Spence Children’s Anxiety Scale
System Usability Scale
This research was conducted as part of the lead author’s PhD at the University of Auckland, New Zealand. This study’s material costs and participant salaries were funded by the Department of Psychological Medicine and the lead author’s PhD student research fund.
HT developed Starship Rescue as part of his PhD studies at the University of Auckland. As such, he is the owner of the intellectual property for this work and stands to gain financially from successful commercialization.