This paper is in the following e-collection/theme issue:

Formative Evaluation and Development of Games (76) Mental Health Games (83) Serious Games for Health and Medicine (692) Games and Gamification for Health (657) Mental Health Issues in Adolescence (457) Gamification (129)

Published on in Vol 12 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47513, first published .
Effects of a Serious Game for Adolescent Mental Health on Cognitive Vulnerability: Pilot Usability Study

Effects of a Serious Game for Adolescent Mental Health on Cognitive Vulnerability: Pilot Usability Study

Effects of a Serious Game for Adolescent Mental Health on Cognitive Vulnerability: Pilot Usability Study

Authors of this article:

Eva De Jaegere1 Author Orcid Image ;   Kees van Heeringen1 Author Orcid Image ;   Peter Emmery2 Author Orcid Image ;   Gijs Mommerency3 Author Orcid Image ;   Gwendolyn Portzky1 Author Orcid Image
Article Authors Cited by Tweetations (2) Metrics

    1Department of Head and Skin, Faculty of Medicine and Health Sciences, Ghent University, , Ghent, , Belgium

    2University Psychiatric Centre KU Leuven, KU Leuven, , Leuven, , Belgium

    3Department of Child and Adolescent Psychiatry, Ghent University Hospital, , Ghent, , Belgium

    Corresponding Author:

    Eva De Jaegere, PhD


    Background: Adolescent mental health is of utmost importance. E-mental health interventions, and serious games in particular, are appealing to adolescents and can have beneficial effects on their mental health. A serious game aimed at improving cognitive vulnerability (ie, beliefs or attitudes), which can predispose an individual to mental health problems, can contribute to the prevention of these problems in adolescents.

    Objective: This study aimed to assess the feasibility of the prototype of a serious game called “Silver.”

    Methods: The prototype of the serious game was developed using a user-centered participatory design. The prototype of Silver focused on 1 aspect of a serious game for improving cognitive vulnerability in adolescents, that is, the recognition and identification of cognitive distortions. Through the game, players were required to identify and classify the character’s thoughts as helpful or unhelpful. Upon successful advancement to the next level, the task becomes more challenging, as players must also identify specific types of cognitive distortions. A pre- and posttest uncontrolled design was used to evaluate the game, with a 1-week intervention phase in which participants were asked to play the game. Participants aged 12-16 years were recruited in schools. The outcomes of interest were the recognition of cognitive distortions and presence of participants’ cognitive distortions. The game was also evaluated on its effects, content, and usefulness.

    Results: A total of 630 adolescents played Silver and completed the assessments. Adolescents were significantly better at recognizing cognitive distortions at the pretest (mean 13.09, SD 4.08) compared to the posttest (mean 13.82, SD 5.09; t629=−4.00, P<.001). Furthermore, their cognitive distortions decreased significantly at the posttest (mean 38.73, SD 12.79) compared to the pretest (mean 41.43, SD 10.90; t629=7.98, P<.001). Participants also indicated that the game helped them recognize cognitive distortions. Many participants considered the game appealing (294/610, 48.2%) but boring (317/610, 52%) and preferred a more comprehensive game (299/610, 49%).

    Conclusions: Findings from this study suggest that a serious game may be an effective tool for improving cognitive vulnerability in adolescents. The development of such a serious game, based on the prototype, is recommended. It may be an important and innovative tool for the universal prevention of mental health problems in adolescents. Future research on the effects of the game is warranted.

    JMIR Serious Games 2024;12:e47513

    doi:10.2196/47513

    Keywords

    e-health; cognitive psychology; cognitive distortion; cognitive vulnerability; digital health; serious games; adolescent mental health; prototype; adolescent; prevention; eHealth 


    Mental illness is one of the biggest health burdens worldwide, and adolescent mental health is a particular global concern [1]. According to the United Nations’ recently published report on children’s mental health, approximately 1 in 7 adolescents aged between 10 and 19 years experience a diagnosed mental disorder globally [2]. Approximately 40% of these disorders are attributable to anxiety and depression [1]. The psychological distress and economic costs due to adolescent mental health problems are enormous. In 2021, the invisible economic cost due to mental health problems in adolescents was estimated at US $387 billion per year globally [2]. Moreover, these mental health problems are important risk factors for self-harm and suicide among adolescents [3,4].

    Early access to treatment can improve outcomes, but limited treatment resources result in long waiting lists and undertreatment [5]. Society, and health systems in particular, need to cope with the increasingly high demands to reduce costs and prevent self-harm and suicide among adolescents. E-mental health interventions are already being used across health care and may be more accessible, engaging, and acceptable options [6-8]. These can take various forms, such as text-based programs, multimedia and interactive programs, virtual reality–based programs, and serious games [9-12]. Evidence of their beneficial effects is increasingly provided [6,13-15]. Regarding serious games, a recent review concluded that the limited evidence indicates a beneficial effect on reducing mental health problems [11,16-19]. Thus, there is a massive potential for serious games to be a new, emerging treatment modality that is more acceptable and engaging, as it uses game mechanics, especially for adolescents who are major users of computerized games in the present-day digital world. However, more research is needed [11,18,20].

    This study therefore aimed at providing further evidence of the beneficial effects of serious games by piloting a prototype (“Silver”) developed to counter cognitive characteristics, which are known to increase the vulnerability to common mental health problems such as depression and anxiety [21]. More specifically, the prototype targets adolescents (aged 12-16 years), aiming to gain insight into cognitive distortions (ie, negative, biased thoughts that influence people’s interpretation of themselves or the world [22]) and their effect on feelings and behaviors. Cognitive distortions, including all-or-nothing thinking, overgeneralization, or mind reading, can contribute to the development of mental health issues such as depression or anxiety during adolescence [23-25]. Identifying and modifying these distortions and replacing them with more balanced, helpful thoughts—that is, cognitive restructuring—is a common technique used in cognitive behavioral therapy (CBT) [22], which has been demonstrated to prevent mental health issues during this vulnerable development phase [26,27] and improve mental well-being [28]. Given the relationship between cognitive distortions and adolescent mental health, as well as the potential of CBT-based techniques to address these distortions, this study aimed to investigate the specific impact of the prototype on adolescents’ cognitive processing. Therefore, the primary hypothesis was that adolescents would improve at recognizing and categorizing helpful and unhelpful thoughts (ie, cognitive distortions) after playing Silver. Furthermore, playing the game was expected to lead to a decline in cognitive distortions. Finally, the usability of Silver was assessed, focusing on the appeal of the game.


    Participants and Recruitment

    Participants were aged between 12 and 16 years and had a smartphone or tablet. Participants were excluded if they were not proficient in Dutch. Recruitment took place from August 2017 to October 2017 via schools.

    Ethical Considerations

    School directors of 8 secondary schools with different curricula across Flanders (ie, the Dutch-speaking region in Belgium) consented to participate in the study. Parents or guardians were informed about the study and given the opportunity to decline their child’s participation (opt out). Web-based assent was obtained from the adolescents before the start of the study. Participants did not receive any form of compensation for their involvement in this study. The data were deidentified prior to analysis to safeguard participants’ privacy. The study was approved by the Commission for Medical Ethics of the University Hospital Ghent (Belgian registration B670201731975).

    Design and Procedure

    The prototype was evaluated using a pre- and posttest uncontrolled design with a 1-week intervention phase. Before participants received access to the prototype, they were asked to fill in a web-based questionnaire (pretest). Immediately after completing the questionnaire, they received access to Silver, which they were asked to play daily for 1 week. After 1 week, they were asked to fill in the second web-based questionnaire (posttest).

    Intervention

    The prototype of the serious game Silver aims to reduce cognitive distortions in adolescents. The prototype is based on a cognitive behavioral framework and focuses on 1 element of mental health improvement, that is, gaining insight into cognitive distortions and their effects on feelings and behaviors. The prototype was designed and developed in a cocreative manner, in which the target users themselves (ie, adolescents aged 12-16 years) were involved, as well as a clinical child psychologist, a child and adolescent psychiatrist, and professional game designers. The design, therefore, was user centered and participatory. The cocreation process was managed by the company that developed the game’s prototype.

    The game is set up in 3 different worlds inhabited by anthropomorphic animals. Each world has different chapters that can be played. The more a player progresses in the game, the more difficult it becomes. The game always starts with an animal that is stuck in his or her mind and therefore is in a “glitch.” The incident preceding the “glitch” is explained through a flashback where the player is shown how this came about (see Figure 1). The events represent difficult situations that are very relatable for adolescents (eg, not getting likes on a social media post). Afterward, the player is shown the character’s thoughts. Each time after reading a thought, the player must decide whether it is a helpful or unhelpful thought (ie, cognitive distortion). If the thoughts are correctly recognized, the unhelpful thoughts are fired upon by little robots and the helpful thoughts return to the character’s head. When enough helpful thoughts have been collected, the character is released from his or her “glitch” and the chapter is completed. At higher levels, the player is also asked to indicate which type of unhelpful thought it is. At first, 2 types of cognitive distortions are introduced, that is, future thinking and all-or-nothing thinking. Afterward, 1 more type of cognitive distortion is added, that is, mind reading (see Figure 2). Thus, the further you progress through the game, the more difficult it becomes as the thoughts can be categorized into more types of cognitive distortions. In this way, the player learns to gain insight into the different types of cognitive distortions.

    ‎
    Figure 1. Screenshot of Silver, showing a flashback to the incident before the glitch. “Maar vanochtend stak zijn moeder nog een kakelvers woestijnvosje in zijn lunchbox.” means “But this morning, his mother stuck another brand-new desert fox in his lunch box.”
    ‎
    Figure 2. Screenshot of Silver, showing the process of identifying types of cognitive distortions. “Aan tafel zijn ze al zeker met mij aan het lachen.” means “At the table, they are already definitely laughing at me.“; “Helpend” means “Helpful”; “Toekomst” means ”Future”; “Gedachten lezen” means “Read minds”; and “Zwart-wit” means “Black and white.”

    Measures

    All outcome measures were collected via self-report questionnaires, which were administered on the web.

    Demographics

    Sociodemographic information (ie, gender, age, and education) was assessed at the pretest. Data on sex (male, female, or other), age (in years), and education (first or second grade and type of curriculum, ie, general secondary education, technical secondary education, secondary education in the arts, and vocational education) were obtained. In addition, participants were asked if they had ever been in therapy for psychological problems.

    Media and Game Use

    At baseline, data were collected regarding participants’ use of media by asking participants what type of media they used and how often they used it on a 5-point scale (1=“never”; 5=“daily”). Items regarding game use assessed whether they ever played computer or video games and whether they still played them. Furthermore, participants were asked on which devices they played the games and how often they did this on a 5-point scale (1=“never”; 5=“daily”). They also gave an estimate about their knowledge about games, ranging from 1= “no knowledge” to 4=“expert.”

    Primary Outcome Measure: Recognition of Negative and Positive Automatic Thoughts

    The primary outcome measure focused on recognizing helpful and unhelpful thoughts. A questionnaire was developed, in which 20 items of the Dysfunctional Attitude Scale, Form A, Dutch translation [29,30] were used. Participants were asked to classify each item as helpful, unhelpful, or “I do not know.” Items were scored as true or false. Scores ranged from 0 to 20, with higher scores indicating a better identification of negative and positive automatic thoughts. The internal consistency of the scale in this study was α=.82.

    Secondary Outcome Measure: Presence of Cognitive Distortions

    The Children’s Negative Cognitive Error Questionnaire–Revised [31] is a 16-item self-report questionnaire that assesses cognitive distortions in those aged 9-17 years. The questionnaire consists of 5 subscales that measure 5 categories of cognitive distortions: “underestimation of the ability to cope,” “personalizing without mind reading,” “mind reading,” “selective abstraction,” and “overgeneralizing.” In each item, a situation is described, followed by a possible thought about the situation. Participants are asked to rate on a 5-point Likert scale how much the thought corresponds to what they would think in that situation, ranging from “almost exactly like I would think” (5 points) to “not at all like I would think” (1 point). Total scores range from 16 to 80, with a higher score reflecting more distorted cognitive errors. The total scale has a good level of internal consistency and good test-retest reliability [31,32].

    Game Evaluation

    At the posttest, participants were asked to rate various statements regarding the effects, content, and usefulness of the game. Items were rated on a 5-point Likert scale (1=“completely disagree”; 5=“completely agree”). An example of an item is “By playing the game I will be able to recognize my own cognitive distortions.” Participants were also asked how they would rate the game overall on a scale from 0 to 10.

    Statistical Analysis

    Power and sample size could not be based on previous studies due to a lack of comparable studies. An effect size of 0.3 was assumed. To detect such an effect size with α=.05 and β=.80, a total sample of 500 participants was calculated. However, since a possible high dropout of 70% to 75% was expected [33], the total required sample size was estimated at 1753.

    Differences between the participants of the study and those who dropped out during the study, as well as differences between those who played the game (ie, gamers) and those who did not play it (ie, nongamers), were examined with χ2 tests (for categorical variables) and 2-tailed independent-sample t tests (for continuous variables). Mean changes between the pre- and posttests were carried out using 2-tailed paired-samples t tests. The corresponding effect sizes were assessed using Cohen d. A significance level of .05 was used for all outcome analyses. All data were analyzed using SPSS software (version 27; IBM Corp).


    Sociodemographic Characteristics and Baseline Outcome Measure

    A total of 1654 adolescents signed up to take part in the study. Among these, 1140 took part in the pre- and posttests. Table 1 presents the differences between the participants who completed both the pre- and posttests and those who only completed the pretest.

    Table 1. Sociodemographic characteristics and baseline outcome measures of participants who completed both the pre- and posttests and participants who only completed the pretest.
    CharacteristicsPre- and posttests (n=1140)Pretest only (n=514)P value
    Sex, n (%)
    Female700 (61.4)303 (58.9).34
    Male429 (37.6)206 (40.1).34
    Other11 (1)5 (1).59
    Age (y), mean (SD)13.40 (1.32)13.86 (1.36).001
    Education curriculum, n (%)
    First-grade GSEa306 (26.8)84 (16.4)<.001
    First-grade VEb55 (4.8)79 (15.4)<.001
    Second-grade GSE513 (45)158 (30.8)<.001
    Second-grade TSEc173 (15.2)56 (10.9).02
    Second-grade SEAd20 (1.8)51 (9.9)<.001
    Second-grade VE73 (6.4)85 (16.6)<.001
    Treatment for psychological problems, n (%)
    Never been to therapy984 (86.3)419 (81.7).02
    More than a year ago68 (6)32 (6.2).83
    Less than a year ago50 (4.4)31 (6).15
    In therapy38 (3.3)31 (6).01
    Baseline outcome measures, mean (SD)
    Recognizing automatic thoughts12.64 (4.37)11.51 (4.82)<.001
    CNCEQ-Re40.81 (10.97)40.66 (12.13).82

    aGSE: general secondary education.

    bVE: vocational education.

    cTSE: technical secondary education.

    dSEA: secondary education in the arts.

    eCNCEQ-R: Children’s Negative Cognitive Error Questionnaire–Revised.

    Of the 1140 adolescents who completed the pre- and posttests, 510 (44.7%) reported that they did not engage with the game (hereafter referred to as nongamers). The primary reason for nonengagement was technical problems (233/510, 45.7%) such as inability to download the game. Other reasons included a lack of time or forgetfulness (198/510, 38.8%), a disinterest in the game (51/510, 10%), and other unspecified reasons (59/510, 5.5%). In contrast, 630 (55.3%) of the 1140 adolescents indicated that they played the game (hereafter referred to as gamers). There were no significant differences in all baseline sociodemographic characteristics between gamers and nongamers except for type of education curriculum (P=.02; see Table 2). Additionally, adolescents who played the game scored significantly higher on recognizing automatic thoughts (mean 13.09, SD 4.08 vs mean 12.08, SD 4.65; t1020=–3.86, P<.001) and significantly higher on the Children’s Negative Cognitive Error Questionnaire–Revised (mean 41.43, SD 10.90 vs mean 40.05, SD 11.01; t1138=–2.10, P=.04) at baseline (see Table 2).

    Table 2. Sociodemographic characteristics and baseline outcome measures of gamers and nongamers in schools.
    CharacteristicsGamers (n=630)Nongamers (n=510)P value
    Sex, n (%)
    Female371 (58.9)329 (64.5).053
    Male251 (39.8)178 (34.9).09
    Other8 (1.3)3 (0.6).24
    Age (y), mean (SD)13.43 (1.30)13.36 (1.33).40
    Education curriculum, n (%)
    First-grade GSEa186 (29.5)120 (23.5).02
    First-grade VEb25 (4)30 (5.9).13
    Second-grade GSE264 (41.9)249 (48.8).02
    Second-grade TSEc99 (15.7)74 (14.5).57
    Second-grade SEAd14 (2.2)6 (1.2).18
    Second-grade VE42 (6.7)31 (6.1).69
    Treatment for psychological problems, n (%)
    Never been to therapy542 (86)442 (86.7).76
    More than a year ago39 (6.2)29 (5.7).72
    Less than a year ago31 (4.9)19 (3.7).33
    In therapy18 (2.9)20 (3.9).32
    Baseline outcome measures, mean (SD)
    Recognizing automatic thoughts13.09 (4.08)12.08 (4.65)<.001
    CNCEQ-Re41.43 (10.90)40.05 (11.01).04

    aGSE: general secondary education.

    bVE: vocational education.

    cTSE: technical secondary education.

    dSEA: secondary education in the arts.

    eCNCEQ-R: Children’s Negative Cognitive Error Questionnaire–Revised.

    Media and Game Use

    Table 3 shows media and game use of the gamers versus the nongamers. The nongamers were significantly less likely to play games currently (270/510, 52.9% vs 405/630, 64.3%; χ21=15.02, P<.001) and were significantly more likely to have “no knowledge” about games than the gamers (69/510, 13.5% vs 54/630, 8.6%; χ21=7.20, P=.007).

    Table 3. Media and game use of gamers and nongamers in schools.
    Media and game useGamers (n=630)Nongamers (n=510)P value
    Media use, n (%)
    Smartphone use615 (97.6)496 (97.3).70
    Tablet use466 (74)360 (70.6).20
    Desktop use226 (35.9)179 (35.1).79
    Laptop use501 (79.5)409 (80.2).78
    Game console use367 (58.3)279 (54.7).23
    Game playing, n (%)
    Ever601 (95.4)476 (93.3).13
    Currently405 (64.3)270 (52.9)<.001
    Game knowledge, n (%)
    No knowledge54 (8.6)69 (13.5).007
    Beginner260 (41.3)211 (41.4).97
    Advanced255 (40.5)198 (38.8).57
    Expert61 (9.7)32 (6.3).04

    Outcome Measures

    Table 4 shows the mean changes in the outcome measures from pre- to posttest and its effect sizes for the group that played the game. The gamers significantly improved in recognizing automatic thoughts (P<.001) and had significantly fewer distorted cognitive errors (P<.001). On both measures, the difference represented a small effect size (0.16 and 0.23, respectively).

    Table 4. Pre- and posttest scores on outcome measures and 2-tailed paired t test results.
    Outcome measuresPretest score, mean (SD)Posttest score, mean (SD)t test (df)d (95% CI)P value
    Recognizing automatic thoughts13.09 (4.08)13.82 (5.09)−4.00 (629)0.16 (−1.09 to −0.37)<.001
    CNCEQ-Ra41.43 (10.90)38.73 (12.79)7.98 (629)0.23 (3.36 to 7.98)<.001

    aCNCEQ-R: Children’s Negative Cognitive Error Questionnaire–Revised.

    Game Evaluation

    A total of 610 gamers gave a score on the various evaluation items (see Table 5). Besides the high number of neutral responses, they generally moderately or highly agreed with the items. The median overall satisfaction rating of 599 gamers, which was scored on a scale of 1 to 10, was 6 and the mean was 5.51 (SD 2.30).

    Table 5. Game evaluation ratings (n=610).
    StatementsDisagree, n (%)Neutral, n (%)Agree, n (%)Score, mean (SD)
    By playing the game I learned about different ways of thinking.134 (22)278 (45.6)198 (32.5)2.10 (0.73)
    By playing the game I will recognize my unhelpful thoughts.123 (20.2)244 (40)243 (39.8)2.20 (0.75)
    The game helps me to reflect more upon my thoughts.164 (26.9)272 (44.6)174 (28.5)2.02 (0.75)
    I can empathize with the stories in the game.202 (33.1)204 (33.4)204 (33.4)2.00 (0.82)
    I think the game is beautifully made.96 (15.7)220 (36.1)294 (48.2)2.32 (0.73)
    I find the game difficult.283 (46.4)198 (32.5)129 (21.1)1.75 (0.78)
    I find the game boring.122 (20)171 (28)317 (52)2.32 (0.79)
    I would like it if the game was not only about thoughts but also about feelings and relaxation.102 (16.7)209 (34.3)299 (49)2.32 (0.74)

    Principal Findings

    The aim of this pilot study was to test the efficacy and usability of the prototype of Silver, a serious game aimed at reducing cognitive distortions in adolescents and thus decreasing cognitive vulnerability to mental health problems. The trial supports our hypothesis that automatic thoughts are better recognized after playing Silver. This enhancement in cognitive awareness was also reflected in the evaluation of the game, wherein gamers indicate that playing the game helps in recognizing unhelpful thoughts. Furthermore, after having played the game, adolescents showed fewer cognitive distortions than before playing. These findings are in line with prior studies, underscoring the positive impact of serious games on cognitive beliefs and modification [19,34-37]. This study contributes to the growing evidence on digital interventions that incorporate core components of CBT, such as cognitive restructuring, and their beneficial effects on well-being and mental health issues [38]. Silver’s emphasis on identifying and mitigating cognitive distortions aligns closely with the principles of rational emotive behavior therapy (REBT), a type of CBT [39]. This therapeutic approach focuses on the identification, challenge, and substitution of irrational beliefs with rational counterparts, alongside learning to manage emotions and behavior in a more helpful way. Literature suggests that serious games focusing on REBT techniques seem to have a strong positive effect in mitigating symptoms of depression and anxiety [19]. Considering the significant outcomes associated with the current version of Silver, which primarily targets cognitive distortions, an expanded version of the game that includes elements intended to address emotional and behavioral aspects could potentially have a greater positive impact on mental health. The inclusion of these additional components could further enhance the game’s therapeutic effectiveness, aligning with REBT’s approach to mental health improvement [19].

    Regarding the evaluation of Silver, the majority (294/610, 48.2%) of gamers indicated that Silver is appealing. Research has shown that nonappealing interfaces may be off-putting and may cause adolescents to disengage from the game. The cocreation of Silver with the target audience probably has ensured that it has attractive aesthetics [40]. In contrast, the majority (317/610, 52%) also perceived Silver as boring. Adolescents may experience serious games in such a manner since they often have very didactic content that does not match commercial, off-the-shelf games, and as a result, the adolescents may cease playing these games [40]. This dichotomy can be attributed to the prototype’s focus on a single aspect of cognitive vulnerability (ie, cognitive distortions), which, although important, may lack the variety necessary to sustain players’ interest over time. Therefore, this may be a critical area for further development in diversifying the game’s content and mechanics. The game could be broadened to include a range of elements. This could mitigate the issue of monotony, thereby improving overall engagement and effectiveness [19]. Furthermore, the gamers indicated that they would like a more comprehensive game that also deals with feelings and relaxation. Such a game may be more eventful, more fascinating, and less boring.

    Preceding a discussion of potential implications of the study findings for the development of serious games, methodological issues need to be addressed. First, as this was a pre- and posttest study, a control group was lacking. Therefore, we were unable to compare the effects on the gamers with those in a random control group that did not play the game. A convenience sample was used, which can lead to a selection bias and consequently underrepresent or overrepresent particular groups. Efforts were made to counter this as much as possible by recruiting a large number of adolescents from 8 different schools with various curricula. However, it is also unclear why some adolescents agreed to take part in the study but others did not. As the sample was not chosen at random, the inherent bias in convenience sampling means that any generalizations of findings must be made with caution. Second, a large group did not adhere to the study protocol. The main reason for dropout were technical difficulties. These were largely due to the prototypic nature of our app; as it was not readily available on app stores, it required adolescents to undertake multiple steps before receiving access to the game. Moreover, participants’ feedback showed time constraints and forgetfulness as additional factors for not engaging with the game. To mitigate these issues in future studies, it is imperative to streamline the app’s accessibility, potentially by securing its availability on common digital distribution platforms. Furthermore, incorporating human support may serve to enhance participant engagement and possibly the overall effectiveness of the intervention [41]. Addressing these aspects is critical for improving study adherence and ensuring the robust evaluation of the app’s therapeutic potential. Furthermore, participants who dropped out were older and had a lower education level. Additionally, they may have experienced more mental health problems, as baseline measurements showed that they were less skilled at recognizing automatic thoughts and were currently more likely to be in treatment for psychological problems. The nongamer group encompassed more adolescents with little interest in games, as they currently played no games or had less knowledge about them. In addition, they were less skilled at recognizing automatic thoughts but had fewer cognitive distortions themselves. However, high attrition rates are not uncommon when studying e-mental health interventions. A systematic review and meta-analysis of computer-based psychological treatments showed an overall dropout rate of 57%, which further increased to 74% in unsupported digital programs [6,42]. In this study, the adolescents were also not offered any support, and this may have had a major effect on adherence. Adding human support could decrease the attrition rate and may even increase the effectiveness of the serious game [43,44]. Moreover, the effectiveness of serious games is often assessed in pragmatic study trials. The real-life settings in which these studies are carried out can also have an impact on the attrition rate. Nevertheless, this type of study can improve the generalizability of the results, as the environments in which they will be implemented are similar to the ones in the trials [45]. Third, no standardized questionnaire for the recognition of automatic thoughts was used. The questionnaire was based on an existing standardized questionnaire [29,30] but was adapted for this study. Fourth, participants often responded with “neutral” in the game evaluation. Although they may have used the neutral midpoint response because they did not comprehend the items or were undecided, offering these neutral responses may decrease the quality and reliability of a questionnaire, particularly in adolescents, since they are more sensitive to pleasing by selecting a neutral answer. Future studies should consider omitting the neutral midpoint [46,47]. Fifth, the study’s emphasis on quantitative measures may have introduced acquiescence bias. Adding qualitative methodologies may provide a more nuanced perspective of the participants’ experiences with the prototype. Future studies should consider using a mixed methods approach to enhance understanding of the intervention’s impact [48].

    Lastly, the eligibility criterion requiring participants to possess a smartphone or tablet may have introduced a selection bias, potentially excluding adolescents without access to such technology or those reluctant to use it. This could inadvertently reinforce the digital divide, that is, inequalities in accessing and using information and communication technologies [49], and limit the generalizability of the findings. Future research should address this limitation by using more inclusive recruitment strategies to minimize technological barriers and ensure broader participation.

    It is difficult to assess the effect of these methodological issues on the validity of the current findings, more so as the current results are difficult to compare to those from similar previous studies. The few available studies of serious games aimed at cognitive training had targeted adults or children with particular mental health problems such as anxiety, alcohol use disorder, or attention-deficit/hyperactivity disorder [11,20]. To the best of the authors’ knowledge, only 2 serious games were studied regarding effects on emotional resilience or mental health promotion in a community sample of adolescents. The results of these studies are comparable to our findings [20,35,36].

    Conclusions

    In conclusion, and keeping the limitations mentioned above in mind, this pilot study demonstrates the promising effects of the Silver prototype. Notably, participants exhibited not only an enhanced ability to recognize cognitive distortions but also a significant decrease in their own experiences of such distortions after engaging with the prototype. This observation suggests a potential positive influence on cognitive characteristics, which are commonly associated with mental health issues. It is therefore recommended that a serious game aimed at decreasing cognitive vulnerability and therefore improving mental health in a general population of adolescents should be developed further and that its efficacy should be studied in future research. This study provides a few cues for further research. The dropout of adolescents who may have the greatest need for cognitive restructuring is a matter of concern, and reasons and remedies for this worrisome issue should be targeted in future research. Randomized controlled trials should be used to further explore the effects of serious games on adolescents in the general population, preferably using an active control group that engages with a different type of digital intervention [50]. Follow-up periods should be sufficiently long to study potential preventative effects among adolescents (Reynard et al [20]).

    Acknowledgments

    Funding for this study was provided by the Flemish government. We are very grateful to the participants, teachers, school directors, and game company whose time and efforts made this work possible.

    Conflicts of Interest

    None declared.

    1. Kuehn BM. Lack of adolescents’ mental health care is a global challenge. JAMA. Nov 16, 2021;326(19):1898. [CrossRef] [Medline]
    2. United Nations Children’s Fund. The state of the world’s children 2021: on my mind – promoting, protecting and caring for children’s mental health. UNICEF; Oct 2021. URL: https://www.unicef.org/media/114636/file/SOWC-2021-full-report-English.pdf [Accessed 2024-04-26]
    3. Bridge JA, Goldstein TR, Brent DA. Adolescent suicide and suicidal behavior. J Child Psychol Psychiatry. 2006;47(3-4):372-394. [CrossRef] [Medline]
    4. Hawton K, Saunders KEA, O’Connor RC. Self-harm and suicide in adolescents. Lancet. Jun 23, 2012;379(9834):2373-2382. [CrossRef] [Medline]
    5. NICEimpact mental health. National Institute for Health and Care Excellence (NICE); 2019. URL: https://www.issup.net/files/2019-05/NICEimpact-mental-health.pdf [Accessed 2024-04-26]
    6. Hollis C, Falconer CJ, Martin JL, et al. Annual research review: digital health interventions for children and young people with mental health problems – a systematic and meta-review. Child Psychology Psychiatry. Apr 2017;58(4):474-503. [CrossRef] [Medline]
    7. Perry Y, Werner-Seidler A, Calear AL, Christensen H. Web-based and mobile suicide prevention interventions for young people: a systematic review. J Can Acad Child Adolesc Psychiatry. 2016;25(2):73-79. [Medline]
    8. Torous J, Bucci S, Bell IH, et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry. Oct 2021;20(3):318-335. [CrossRef] [Medline]
    9. Bell IH, Nicholas J, Alvarez-Jimenez M, Thompson A, Valmaggia L. Virtual reality as a clinical tool in mental health research and practice. Dialogues Clin Neurosci. Jun 2020;22(2):169-177. [CrossRef] [Medline]
    10. Dellazizzo L, Potvin S, Luigi M, Dumais A. Evidence on virtual reality–based therapies for psychiatric disorders: meta-review of meta-analyses. J Med Internet Res. Aug 19, 2020;22(8):e20889. [CrossRef] [Medline]
    11. Dewhirst A, Laugharne R, Shankar R. Therapeutic use of serious games in mental health: Scoping review. BJPsych open. Feb 2, 2022;8(2):e37. [CrossRef] [Medline]
    12. Freeman D, Reeve S, Robinson A, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med. Oct 2017;47(14):2393-2400. [CrossRef] [Medline]
    13. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. Jan 2018;47(1):1-18. [CrossRef] [Medline]
    14. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. Apr 29, 2021;8(4):e25847. [CrossRef] [Medline]
    15. Radovic A, Vona PL, Santostefano AM, Ciaravino S, Miller E, Stein BD. Smartphone applications for mental health. Cyberpsychol Behav Soc Netw. Jul 2016;19(7):465-470. [CrossRef] [Medline]
    16. Abd-Alrazaq A, Al-Jafar E, Alajlani M, et al. The effectiveness of serious games for alleviating depression: systematic review and meta-analysis. JMIR Serious Games. Jan 14, 2022;10(1):e32331. [CrossRef] [Medline]
    17. Abd-Alrazaq A, Alajlani M, Alhuwail D, et al. The effectiveness of serious games in alleviating anxiety: systematic review and meta-analysis. JMIR Serious Games. Feb 14, 2022;10(1):e29137. [CrossRef] [Medline]
    18. David OA, Costescu C, Cardos R, Mogoaşe C. How effective are serious games for promoting mental health and health behavioral change in children and adolescents? A systematic review and meta-analysis. Child Youth Care Forum. Dec 2020;49(6):817-838. [CrossRef]
    19. Eve Z, Turner M, Di Basilio D, et al. Therapeutic games to reduce anxiety and depression in young people: a systematic review and exploratory meta-analysis of their use and effectiveness. Clin Psychology Psychoth. 2024;31(1):e2938. [CrossRef]
    20. Reynard S, Dias J, Mitic M, Schrank B, Woodcock KA. Digital interventions for emotion regulation in children and early adolescents: systematic review and meta-analysis. JMIR Serious Games. Aug 19, 2022;10(3):e31456. [CrossRef] [Medline]
    21. Hjemdal O, Friborg O, Stiles TC, Martinussen M, Rosenvinge JH. A new scale for adolescent resilience: grasping the central protective resources behind healthy development. Meas Eval Couns Dev. 2006;39(2):84-96. [CrossRef]
    22. Beck JS. Cognitive Behavior Therapy: Basics and Beyond. 2nd ed. Guilford Press; 2011:xix-391. ISBN: 978-1-60918-504-6
    23. Marton P, Churchard M, Kutcher S. Cognitive distortion in depressed adolescents. J Psychiatry Neurosci. May 1993;18(3):103-107. [Medline]
    24. Muris P, Field AP. Distorted cognition and pathological anxiety in children and adolescents. Cogn Emot. 2008;22(3):395-421. [CrossRef]
    25. Smith EM, Reynolds S, Orchard F, Whalley HC, Chan SW. Cognitive biases predict symptoms of depression, anxiety and wellbeing above and beyond neuroticism in adolescence. J Affect Disord. Dec 1, 2018;241:446-453. [CrossRef] [Medline]
    26. James AC, Reardon T, Soler A, James G, Creswell C. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database Syst Rev. Nov 16, 2020;11(11):CD013162. [CrossRef] [Medline]
    27. Weisz JR, McCarty CA, Valeri SM. Effects of psychotherapy for depression in children and adolescents: a meta-analysis. Psychol Bull. Jan 2006;132(1):132-149. [CrossRef] [Medline]
    28. Powell J, Hamborg T, Stallard N, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res. Dec 31, 2012;15(1):e2. [CrossRef] [Medline]
    29. Raes F, Hermans D, van den Broeck K, Eelen P. Kort Instrumenteel: de Nederlandstalige versie van de Dysfunctional Attitude Scale – vorm A (DAS-A-NL). Gedragstherapie. 2005;38:285-294. URL: https://psycnet.apa.org/record/2006-01720-004 [Accessed 2024-04-26]
    30. Weissman AN, Beck AT. Development and validation of the Dysfunctional Attitude Scale: a preliminary investigation. Presented at: Annual Meeting of the American Educational Research Association; Mar 27 to 31, 1978; Toronto, ON. URL: https://eric.ed.gov/?id=ED167619 [Accessed 2024-04-26]
    31. Maric M, Heyne DA, van Widenfelt BM, Westenberg PM. Distorted cognitive processing in youth: the structure of negative cognitive errors and their associations with anxiety. Cognit Ther Res. Feb 2011;35(1):11-20. [CrossRef] [Medline]
    32. Schwartz JS, Maric M. Negative cognitive errors in youth: specificity to anxious and depressive symptoms and age differences. Behav Cogn Psychother. Sep 2015;43(5):526-537. [CrossRef] [Medline]
    33. De Jaegere E, van Landschoot R, van Heeringen K, et al. The online treatment of suicidal ideation: a randomised controlled trial of an unguided web-based intervention. Behav Res Ther. Aug 2019;119:103406. [CrossRef] [Medline]
    34. David OA, Cardoș RAI, Matu S. Changes in irrational beliefs are responsible for the efficacy of the REThink therapeutic game in preventing emotional disorders in children and adolescents: mechanisms of change analysis of a randomized clinical trial. Eur Child Adolesc Psychiatry. Mar 2019;28(3):307-318. [CrossRef] [Medline]
    35. David OA, Predatu R, Cardoș RAI. Effectiveness of the REThink therapeutic online video game in promoting mental health in children and adolescents. Internet Interv. Apr 20, 2021;25:100391. [CrossRef] [Medline]
    36. Huen JM, Lai ES, Shum AK, et al. Evaluation of a digital game-based learning program for enhancing youth mental health: a structural equation modeling of the program effectiveness. JMIR Ment Health. Oct 7, 2016;3(4):e46. [CrossRef] [Medline]
    37. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MFG. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. BMJ. Apr 18, 2012;344:e2598. [CrossRef] [Medline]
    38. Babbage CM, Jackson GM, Davies EB, Nixon E. Self-help digital interventions targeted at improving psychological well-being in young people with perceived or clinically diagnosed reduced well-being: systematic review. JMIR Ment Health. Aug 26, 2022;9(8):e25716. [CrossRef] [Medline]
    39. Ellis A, Dryden W. The Practice of Rational Emotive Behavior Therapy. 2nd ed. Springer; 2007. [CrossRef]
    40. Garrido S, Millington C, Cheers D, et al. What works and what doesn’t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. Nov 13, 2019;10:759. [CrossRef] [Medline]
    41. Tremain H, McEnery C, Fletcher K, Murray G. The therapeutic alliance in digital mental health interventions for serious mental illnesses: narrative review. JMIR Ment Health. Aug 7, 2020;7(8):e17204. [CrossRef] [Medline]
    42. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. Jun 2012;32(4):329-342. [CrossRef] [Medline]
    43. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions - a systematic review. Internet Interv. Oct 2014;1(4):205-215. [CrossRef]
    44. Shum AK, Lai ES, Leung WG, et al. A digital game and school-based intervention for students in Hong Kong: quasi-experimental design. J Med Internet Res. Apr 5, 2019;21(4):e12003. [CrossRef] [Medline]
    45. Baumel A, Edan S, Kane JM. Is there a trial bias impacting user engagement with unguided e-mental health interventions? A systematic comparison of published reports and real-world usage of the same programs. Transl Behav Med. Nov 25, 2019;9(6):1020-1033. [CrossRef] [Medline]
    46. Borgers N, Sikkel D, Hox J. Response effects in surveys on children and adolescents: the effect of number of response options, negative wording, and neutral mid-point. Qual Quant. Feb 2004;38:17-33. [CrossRef]
    47. Thornton L, Gardner LA, Osman B, et al. A multiple health behavior change, self-monitoring mobile app for adolescents: development and usability study of the Health4Life app. JMIR Form Res. Apr 12, 2021;5(4):e25513. [CrossRef] [Medline]
    48. Bryman A. Integrating quantitative and qualitative research: how is it done? Qual Res J. Feb 2006;6(1):97-113. [CrossRef]
    49. Scheerder A, van Deursen A, van Dijk J. Determinants of internet skills, uses and outcomes. A systematic review of the second- and third-level digital divide. Telemat Inform. Dec 2017;34(8):1607-1624. [CrossRef]
    50. Zayeni D, Raynaud JP, Revet A. Therapeutic and preventive use of video games in child and adolescent psychiatry: a systematic review. Front Psychiatry. Feb 6, 2020;11:36. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    REBT: rational emotive behavior therapy

    Edited by Gunther Eysenbach; submitted 22.03.23; peer-reviewed by Joanna Chu, Paul Matthews; final revised version received 08.03.24; accepted 24.03.24; published 09.05.24.

    Copyright

    © Eva De Jaegere, Kees van Heeringen, Peter Emmery, Gijs Mommerency, Gwendolyn Portzky. Originally published in JMIR Serious Games (https://games.jmir.org), 9.5.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.