This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.
Working memory capacity has been found to be impaired in adolescents with various psychological problems, such as addictive behaviors. Training of working memory capacity can lead to significant behavioral improvements, but it is usually long and tedious, taxing participants’ motivation to train.
This study aimed to evaluate whether adding game elements to the training could help improve adolescents’ motivation to train while improving cognition.
A total of 84 high school students were allocated to a working memory capacity training, a gamified working memory capacity training, or a placebo condition. Working memory capacity, motivation to train, and drinking habits were assessed before and after training.
Self-reported evaluations did not show a self-reported preference for the game, but participants in the gamified working memory capacity training condition did train significantly longer. The game successfully increased motivation to train, but this effect faded over time. Working memory capacity increased equally in all conditions but did not lead to significantly lower drinking, which may be due to low drinking levels at baseline.
We recommend that future studies attempt to prolong this motivational effect, as it appeared to fade over time.
Psychological problems that occur during adolescence are often associated with deficiencies in self-regulation [
Heavy drinking in youth has previously been associated with suboptimal cognitive control functions (eg, [
Despite its efficacy in specific adolescent groups, motivation is an important moderator of cognitive training efficacy [
There have been several attempts to gamify cognitive training paradigms. For example, Prins et al [
The City Builder game was designed as a so-called game-shell [
This pilot study describes the results of 10 sessions of alcohol-related WMC training using the City Builder game. We compared 3 conditions (all including the alcohol-related context): the gamified WMC training using the City Builder game (henceforth referred to as the
Finally, a practical problem that can occur in an experimental comparison of a training task with and without game elements is that although all participants complete the same assessments after the training, only those in the gamified condition have been rewarded during training. As these participants may have been getting used to being rewarded for their effort, the lack of rewards in the posttraining assessment could negatively affect their motivation, and in effect their performance, potentially distorting the assessment of the training effect in an unwanted way [
Participants were 84 adolescents from a high school in the Netherlands aged between 13 and 16 years (mean age 13.7 [SD 0.7] years; 40% [34/84] boys). Participants trained during normal school hours in 14 groups of 6 students. They were randomly assigned to 1 of the 3 training conditions stratified for age, gender, and school class. Participants in each group were allocated to the same condition (as a form of clustered randomization) to prevent them from comparing the gamified and nongamified versions among each other. There were 24 students (4 groups) in the placebo condition, 30 students (5 groups) in the standard WMC training condition, and another 30 students (5 groups) in the gamified WMC training condition. The training took place in 2 cohorts: 7 groups (2 placeboes, 3 standard WMC training, and 2 gamified WMC training) finished training before Christmas break; the other 7 groups started after Christmas. The second cohort filled in an additional questionnaire assessing motivation to train after each session. Due to personal reasons, 3 students (2 from the placebo and 1 from the standard WMC training condition) dropped out during the study. The study’s target sample size was between 25 and 30 participants per condition, which was based on similar studies [
Before the study, parental consent was obtained for each adolescent, and at baseline, adolescents were informed about the training procedure and the reward for participation, which was a maximum of 15 euros, consisting of 5 euros for doing the baseline and posttraining assessments and an additional 1 euro for each completed training session. The training itself was not presented to the participants as an alcohol intervention; rather, it was presented as a new “computer training” that was to be tested, which could help them to gain more “mental control” over their behavior, such as (excessive) alcohol use. To keep the students motivated to continue training in all conditions, it was announced that the training money was only awarded when a minimum of 8 training sessions were completed. The training was done on university laptops in groups of 6 adolescents, whereas the assessment sessions, which were the same in all 3 conditions, were done in groups of 12 students on school personal computers. After the baseline assessment, participants performed 10 daily training sessions on school days during the next 2 weeks. When a training session was missed because of an important school activity, an extra training session was planned for a total of 10 training opportunities per participant. Finally, there was a posttraining assessment session.
This training was based on the Chessboard task by Dovis et al [
This version was exactly the same as the standard WMC training, except that the sequence length was always kept at 3 to prevent a training effect while presenting a visually similar experience (cf [
This version was also similar to the standard WMC training but was embedded within a game context, the City Builder game ([
As shown in
The City Builder game. Left pane: the game screen; Right pane: the working memory capacity (WMC) training task is presented overlaying the game screen. During instructions, the game is shown in the background (as pictured); when the trials start, the background blacks out entirely.
Procedure during training sessions.
Version of working memory capacity training | Standard | Placebo | Gamified |
Training block 1 (9 min) | 20 trials | 25 trials | 20 trials |
Break 1 (3 min) | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, enjoy break in silence |
Training block 2 (9 min) | 20 trials | 25 trials | 20 trials |
Break 2 (3 min) | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, enjoy break in silence |
Optional extra training block (5 min)b | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, or enjoy break in silence | Continue training, read magazine, or enjoy break in silence |
aDuring the first session, participants in the gamified working memory capacity training condition always started the first break with a 1-min introduction to the game.
bDuring the last session, the second break lasted for 8 min, and the extra training block was omitted, as there was no next session to spend the bonus points in.
WMC was assessed using the Self-Ordered Pointing Task (SOPT; [
Besides the number of bonus trials done per session (ie, during both breaks as well as in the final, optional training block) as a behavioral measure of motivation, 2 self-report questions were also added in the second cohort: “How much were you looking forward to this task?” and “How much did you like this task?,” both scored on a 10-point scale ranging from 1 (not at all) to 10 (very much). After the training, participants were asked about their previous game experience, as well as how much fun they thought the training had been, on a 5-point Likert scale from 1 (a lot of fun) to 5 (very boring); how difficult they thought the training had been, on a 5-point Likert scale from 1 (very difficult) to 5 (very easy); and how often they would continue doing the training if it would be made available at home, on a 5-point Likert scale from 1 (never) to 5 (very often).
As heavy drinking does occur at this age in the Netherlands [
To check for baseline differences in intelligence quotient (IQ), a subselection of 30 items from Raven Standard Progressive Matrices (RPM [
Before running the analyses, all dependent variables were screened for univariate outliers (scores removed more than 3 SDs from the group mean), which resulted in the exclusion of 2 outliers on the AUDIT, 1 on the Five-Shots Questionnaire, 4 on the TLFB, 1 on the SOPT sum score, 2 on the RAPI18, 1 on the SOPT, 1 on the BAS Fun seeking, and 2 on the BAS Reward responsiveness subscales. Due to technical problems, the data of 4 participants at baseline, TLFB data for 3 participants, and RPM data for 1 participant were lost. All analyses were thus performed on the remaining number of participants.
The hypothesized effects of training condition over time were ascertained through the use of factorial repeated measures analyses of variance, using condition as a between-subjects factor (with 3 levels: standard, placebo, and gamified), and time as a within-subjects factor (with 2 levels: before and after the training). Motivation was compared on several measures using regular analyses of variance (or nonparametric variants thereof, in those cases where one or more statistical testing assumptions were violated), as well as a growth model analysis on the number of bonus trials done during each session. Finally, an exploratory analysis of variance was performed using the percentages on specific squares following the alcohol picture.
Due to various reasons (eg, illness), some participants missed one or more sessions but were allowed to continue training. Five participants, however, did not complete the full assessments and were therefore excluded from the relevant prepost analyses. In total, 29 participants completed the full training in the gamified WMC training condition; 27 in the standard WMC training condition and 20 in the placebo condition. Levels of drinking were very low at baseline. The average sum score on the AUDIT was 1.2 (SD 2.3), with 52 participants having a sum score of 0, and 0.4 (SD 1.1) on the RAPI18. Therefore, it was decided to include these 2 long-term measures again after training to make sure this finding was stable. This was the case. There were no baseline differences in age, gender, IQ, impulsivity, or WMC between conditions (all
There was a main effect of time on WMC as measured with the SOPT sum score,
As another measure of motivation to train, we looked at the total number of bonus trials done during each session (ie, during both breaks as well as in the final, optional training block), where we numbered the sessions per participant (see
Robust Maximum Likelihood was used as an estimator to account for the nonnormality. The first step taken was a confirmatory factor analysis (CFA) on the total number of bonus trials during each session (cf [
In the final step, we looked at change over time using a growth model of sessions 2 through 10, again with the bonus trials count variables as latent growth indicators. Several models were compared, first constraining groups to be equal or not (ie, assuming there were or there were no group differences), and subsequently constraining only the slopes to be equal or not (ie, assuming there were or there were no differences in the decrease of bonus trial counts), and the intercepts to be equal or not (ie, assuming there were or there were no baseline differences in bonus trial counts). The best model fit in terms of Akaike Information Criterion (AIC [
Training outcomes by group.
Measure | Standard | Placebo | Gamified | Total |
SOPTa sum score pretraining, mean (SD) | 55.4 (4.5) | 55.1 (4.8) | 56.2 (4.5) | 55.6 (4.6) |
SOPT sum score posttraining, mean (SD) | 57.4 (5.3) | 57.3 (4.3) | 55.9 (4.8) | 56.8 (4.9) |
TLFBb sum score pretraining, mean (SD) | 0.3 (0.6) | 0.2 (0.5) | 0.1 (0.2) | 0.2 (0.5) |
TLFB sum score posttraining, mean (SD) | 0.3 (0.7) | 0.1 (0.2) | 0.0 (0.2) | 0.1 (0.4) |
aSOPT: Self-Ordered Pointing Task.
bTLFB: Timeline Followback; shows the number of standardized drinks during the week before the assessment.
Motivations by group.
Measure | Standard | Placebo | Gamified | Total | ||
How much fun was the training? (mean [SD])a | 3.7 (0.7) | 3.2 (0.9) | 3.1 (0.7) | 3.3 (0.8) | .006b,c | |
Would you like to have the training at home? (yes; absolute [%]) | 2 (7) | 0 (0) | 5 (17) | 7 (9) | .10 | |
How often would you train at home? (mean [SD])d | 1.4 (0.7) | 1.3 (0.5) | 1.7 (0.8) | 1.5 (0.7) | .13b | |
How much were you looking forward to this task (the SOPT)? (mean [SD]e,f) | 1.3 (1.8) | 1.0 (0.8) | −0.6 (1.2) | 0.4 (1.6) | .003c | |
How much did you like this task (the SOPT)? (mean [SD]e,f) | 1.1 (1.8) | 0.5 (1.1) | −0.1 (1.8) | 0.4 (1.7) | .21 | |
Number of training sessions completed (mean [SD]) | 8.8 (1.1) | 8.4 (1.1) | 9.1 (0.8) | 8.8 (1.0) | .04g |
a5-point Likert scale from 1 (a lot of fun) to 5 (very boring).
bNonparametric Kruskal-Wallis test was applied due to violation of normality.
c
d5-point Likert scale from 1 (never) to 5 (very often).
eMean (SD) of change score. Change score is defined as the difference between the pre- and posttraining assessment scores.
f10-point grade from 1 (low) to 10 (high).
g
Average number of bonus trials per session. Error bars indicate 95% CI.
Average number of bonus trials per session.
Session | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Placebo, mean (SD) | 34.8 (19.2) | 14.7 (17.8) | 0.2 (0.4) | 1.0 (3.2) | 0.4 (1.1) | 0.5 (1.9) | 0.1 (0.2) | 0.1 (0.3) | 0.2 (0.8) | 0.0 (0.0) |
Standard, mean (SD) | 23.2 (11.8) | 10.7 (11.1) | 7.3 (8.4) | 2.9 (6.6) | 2.0 (7.0) | 0.8 (3.8) | 0.9 (3.9) | 0.4 (1.4) | 0.6 (2.5) | 1.3 (3.4) |
Gamified, mean (SD) | 16.5 (9.8) | 13.0 (10.0) | 9.2 (9.9) | 5.6 (8.5) | 4.9 (7.5) | 3.2 (6.5) | 2.0 (5.8) | 1.5 (5.7) | 1.8 (4.7) | 1.6 (4.7) |
Error percentages on specific squares.
Measure | Standard | Placebo | Gamified | Total | ||
Error percentage on squares directly following the alcohol picture | 24.2 (5.8) | 5.8 (3.8) | 24.7 (4.8) | 19.1 (9.8) | <.001a,b | |
Error percentage on squares not directly following the alcohol picture | 24.3 (5.9) | 6.8 (4.3) | 24.2 (5.2) | 19.3 (9.5) | <.001a,b | |
Ratio of errors directly following the alcohol picture over those that do not | 1.00 (0.08) | 0.85 (0.13) | 1.03 (0.09) | 0.97 (0.12) | <.001a,b | |
Error percentage on squares directly following the alcohol picture | 24.2 (5.8) | 24.7 (4.8) | 24.5 (5.2) | .33a | ||
Error percentage on squares not directly following the alcohol picture | 24.3 (5.9) | 24.2 (5.2) | 24.2 (5.5) | .66a | ||
Ratio of errors directly following the alcohol picture over those that do not | 100.3 (8.2) | 103.0 (8.7) | 101.6 (8.5) | .22 | ||
Average sequence lengthc | 5.5 (0.8) | 5.6 (0.7) | 5.5 (0.7) | .41 |
aNonparametric Kruskal-Wallis test, which was applied due to violation of normality.
b
cThe average number of squares shown per trial.
To determine the influence of the alcohol picture during the encoding phase of the training trials, we looked at the percentage of errors made specifically on squares that directly followed the alcohol picture versus the error percentage on squares that did not directly follow the alcohol picture. Overall, error percentages were different between the training conditions, but this was mainly because in the placebo condition, all sequences had exactly 3 squares, and thus fewer errors were made. When this condition was excluded, the standard and gamified WMC training conditions did not differ (see
In this pilot study, we investigated the beneficial effects of a serious game environment on adolescents’ motivation to do cognitive training. Although no relevant differences were found in the primary outcome measure (WMC), several interesting findings were obtained regarding motivation to train. First, the self-reported motivation questions posed after the training was completed showed mixed results, with participants only having a slight preference against the standard WMC training. This may indicate that participants did not like the game more than they liked the placebo WMC training, but it can also mean that they merely lost interest over time. Other than the nongamified training versions, the gamified WMC training, being presented as a game, likely increased participants’ expectations of its entertainment value. If the game then did not fully satisfy these expectations over the 10 sessions of training, this may have influenced the motivation assessment after the training. As such, it is advisable to assess motivation to train at multiple points in time to see if there might be an initial effect that fades over time. This can be achieved with a behavioral measure of motivation, such as the number of training trials done beyond the minimum amount required. This number was found to be higher in the gamified WMC training condition than in the nongamified conditions, but it also declined over time in all conditions.
Regarding the bonus trial analysis, the fact that the first session showed a much higher numbers of bonus trials in all conditions, compared with the following sessions, may actually make sense from a theoretical standpoint. Given that during the first session, all versions of the training were new to the participants, when the option to do extra trials was first presented, it may have been curiosity rather than motivation that drove participants to do some bonus trials. From the second session forward, though, this option was no longer novel, suggesting that actual motivation to train would have taken over.
It should be noted that the wish to spend the points collected through training by playing the game during the breaks may have limited the time available for doing bonus trials. This may have inadvertently led to an underestimation of the motivation to train. Relatedly, the number of bonus trials may have been skewed a little due to the fact that, on average, bonus trials in the placebo condition were shorter than those in the active training conditions. This might explain the initial peak in the placebo condition in session 1, while also underscoring the fact that the decline in sessions 2 and 3, which is attributed to motivation, is also most notable in this condition. As we unfortunately did not record the time spent on doing bonus trials or playing the game, the number of bonus trials was the only behavioral measure of motivation we were able to analyze. Future studies should therefore consider also looking at the time spent on bonus trials and playing the game as additional behavioral measures of motivation.
A theoretical explanation for the declining motivational effect found, in terms of fewer bonus trials done per session over time, could be that the points awarded during training may be acting primarily as extrinsic motivators [
In line with previous motivational results [
The second motivational finding concerns participants’ motivation to perform well on the study’s main cognitive outcome measure: the pre- and posttraining WMC assessments (SOPT). Although WMC was found to increase over time in all training conditions, which could indicate a practice effect, where participants’ performance increased due to having done the task before, motivation to complete the task had increased after the training in the nongamified conditions but had decreased in the gamified WMC training condition. This finding is in line with our hypothesis that the rewarding nature of the gamified WMC training condition may negatively affect motivation to complete assessment tasks afterward. Although it is unclear if, and to what degree, this motivational effect may have influenced the assessment of the actual training gain, it does have important implications for future research aiming to validate serious games, compared with their nonrewarding, original counterparts. Incorporating the assessment task in the game and having a mini-assessment at the start of every training session (cf [
The results presented in this paper do have to be interpreted with some caution because of several limitations. First, no training effects were found on drinking behavior; however, alcohol use was very low at baseline in this sample. As it obviously could not get much lower through training, no inferences on the effects of (gamified) cognitive training on drinking behavior should be made based on this study. It would be interesting for future intervention research to include adolescents with cognitive deficits and at risk for problematic alcohol use [
Despite these limitations, to the best of our knowledge, this study is the first to demonstrate that WMC training in adolescents can benefit from the use of game elements by increasing motivation to train. It follows that the challenge for future research will be in trying to prolong this effect, for example, by making bigger, more immersive games that last longer (although this is quite a challenge, even in commercial gaming). By closely monitoring the levels of motivation throughout the study, as well as by managing participants’ expectations about the entertainment value of the training, which may still be an important factor in determining the training outcome, more insight may be acquired into the specific effectiveness of the use of game elements in cognitive training. Finally, future research could also apply gamified WMC training in specific at-risk groups, such as adolescents who have specific difficulties with traditional training approaches due to attention- or motivation-related problems. Moderation analyses can then be used to reveal individual differences in the effectiveness of the gamified training, identifying those who could benefit the most from these motivational features.
attention-deficit/hyperactivity disorder
Akaike information criterion
Alcohol Use Disorder Identification Test
Bayesian information criterion
confirmatory factor analysis
intelligence quotient
Rutgers alcohol problem index
Raven standard progressive matrices
self-ordered pointing task
Alcohol Timeline Followback
working memory capacity
The authors wish to thank the students involved in the data collection for this study: Nathan Bleijenberg, Bouwien Westerhuis, Emmily Harwig, Marcia Dominicus, and Wendy Kuijn. WJB is supported by the National Initiative Brain and Cognition Grant 433-11-1510, and TEG and RWW by the VICI grant 453-08-001, both funded by the Dutch National Science Foundation (NWO). TEG is also supported by the ERAB grant EA 12 39.
None declared.