The training consisted of 4 sessions, at least 1 day apart, over the course of 2 weeks. The first and last training sessions were combined with the assessment tasks in our laboratory; the 2 remaining training sessions were done at home. During the first session, participants were informed about the study’s goal, gave digital informed consent, and were randomly assigned to the VPT-P (n=33), VPT-R (n=30), or VPT-G (n=33) condition. They continued with digital versions of the Alcohol Use Disorders Identification Test (AUDIT), a short Readiness to Change Questionnaire (RCQ), the Timeline Followback (TLFB) questionnaire, the Alcohol Use Questionnaire (AUQ), and a Motivation to Train Questionnaire (MTQ). After the questionnaires, they completed the VPT and VST baseline assessment and finished the session with the first VPT training. The following second and third sessions solely consisted of the VPT training task. The last session started with the fourth VPT training, after which they performed the VPT and VST posttraining assessments and the AUQ, TLFB, RCQ, and MTQ questionnaires, supplemented by a brief set of questions about the training itself (EVAL). To evaluate drinking behavior after the training, a follow-up TLFB was filled in via email 2 weeks after session 4.

A sample of undergraduate students (n=96, mean age 21.2 years, SD 1.8, range 18-28 years, 71%, 68/96 female) was recruited through the university laboratory’s website, based on their drinking behavior (≥5 standard glasses of alcohol on average per week for males; ≥4 for females). Participants received study credits or €30 for taking part in the experiment. The study was approved by the Ethics Committee of the University of Amsterdam (Protocol Number: 2015-DP-4215).

Alcohol use and problems were measured with 3 questionnaires: A TLFB [24,25] was used to measure alcohol consumption per day over the past week and also included a question about the number of binge drinking occasions during the past 30 days (>5 standard glasses of alcohol consumed during one occasion for male participants; >4 for females). An adapted version of the AUQ [26] was used to assess drink-specific alcohol consumption over the past 6 months. For analyses, Mehrabian and Russell’s [26] equation 2 was used to calculate the habitual alcohol consumption (HAC), including those items regarding consumption of beer, wine, and liquor, as well as our added items concerning alcohol pops. Alcohol-related problems were measured with the AUDIT [27], which includes 10 multiple-choice questions regarding alcohol consumption and alcohol-related problems. The overall AUDIT score ranges between 0 and 40, with ≥8 indicating an increased risk of alcohol-related problems in normal samples and ≥11 in student samples [28].

Motivation to train was assessed using a self-developed 4-item questionnaire, each item rated on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree.” Motivation to change was assessed using a shortened version of the RCQ [29], consisting of 3 multiple-choice items. The EVAL questions concerned how they rated the training overall. See Multimedia Appendix 1 for an overview of translated questions.

Attentional bias was measured using assessment versions of the VST and VPT paradigms. In the VST [23,30], participants were shown a grid of 4 by 4 pictures of beverages, where only 1 was of a different type: 1 alcoholic among 15 nonalcoholic beverages, or vice versa. The instruction was to find and click the deviant type of beverage as quickly and accurately as possible. To focus visual attention to the center of the grid, each trial started with a fixation cross in the center of the screen the participants had to mouse over in order to start the trial. When an incorrect response was given, feedback was given and the trial had to be redone. The task consisted of 6 blocks of 18 trials, using active (person drinking) or passive (bottle or glass only) pictures of alcoholic and nonalcoholic beverages or neutral pictures of 5- or 7-petaled flowers, following the schedule in Table 1. The order of the blocks containing beverages was counterbalanced over participants. A progress bar indicated the number of trials left in each block. The attentional bias scores for active and passive stimuli were computed by subtracting the respective average reaction times for selecting alcoholic beverages from the average reaction times for selecting nonalcoholic beverages. Given that faster reaction times on alcohol trials suggest an attentional bias toward alcohol, a positive bias score thus indicated a bias toward alcohol.

In the VPT, participants were shown pairs of alcohol and nonalcoholic beverages followed after 500 milliseconds by a small arrow probe in the location of one of the pictures, pointing upward or downward. The instruction was to press the keyboard’s arrow key corresponding to the arrow’s direction as quickly and accurately as possible. The task consisted of 168 critical trials with pairs of beverages and 32 filler trials with pairs of neutral objects (office supplies), presented in random order over 3 blocks: a starting block of 10 neutral practice trials, then 2 test blocks of 100 trials (84 critical trials). In one of the two test blocks, the pictures disappeared as the arrow became visible (“Go” block); in the other block the pictures remained visible as the arrow probe was superimposed (“Stay” block). The Stay trials were included as they might better detect difficult disengagement from alcohol cues [31,32], while the standard Go trials may be a better measure of rapid allocation and maintenance of attention on alcohol cues. The order of these blocks was counterbalanced over participants. Location of alcohol picture (left or right) and arrow (at alcohol-related or not-alcohol-related stimulus) was fully counterbalanced. The filler trials were included to slightly mask the contingency between arrow placement and the content of the pictures, as well as maintain participants’ attention on the task and avoid anticipatory responses. When an incorrect response was given, the trial had to be redone after feedback was given, and the arrow direction was reapplied randomly. A progress bar indicated the number of trials left in each block. All picture pairs were matched by size and colors (see Figure 1). All stimuli originated from the Amsterdam Beverage Picture Set (ABPS [33]). Attentional bias scores for Stay and Go trials were computed by respectively subtracting the average reaction time for correct trials with the arrow at the location of the alcoholic picture from the average reaction time on correct trials where the arrow was at the location of the nonalcoholic picture. Correction trials following incorrect responses were excluded, as reaction times on those trials were deemed unreliable owing to the stimuli being the same as in the previous trials; reaction times greater than 3000 milliseconds were coded as too late and were also excluded. Given that faster responses on alcohol trials suggest an attentional bias toward alcohol, a positive bias score thus indicated a bias toward alcohol.

The VPT training tasks were identical to the assessment version, except that after the practice block there was only 1 training block of 156 critical and 16 filler trials, where Stay and Go trials were presented randomly. Additionally, in the VPT-R and VPT-G the arrow always appeared at the location of the nonalcoholic beverage, thus training attention away from the alcoholic stimuli. In the VPT-G condition, participants trained using the Shots game (S van Schie, unpublished data, 2014). The Shots game was functionally identical to the VPT-R training, while looking like a slot machine game with 2 spinning wheels (see Figure 1; note that although the Shots game looks like a slot machine, it has no gambling elements to it. This was also explained to the participants). The game elements used here constitute an integrated gamification of the VPT paradigm as defined by the CBM gamification model by Boendermaker and colleagues [22]. It mainly uses a coin-based reward system (see Step 1 [22]) and nicer looking graphics, animations (eg, the spinning wheels with pictures of beverages), and sound effects (eg, when spinning the wheels or pressing a button; see Step 3 [22]). The participant is rewarded for correct and fast responses (using time bonuses and special bonus trials), requiring a coin in order to spin the wheels (ie, start a new trial) and eventually the possibility of reaching a new level (a new look for the machine). The game used picture stimuli similar to those in the ABPS but slightly edited to fit the graphical style of the game.

After the first training session, 2 participants in the VPT-R condition dropped out of the study and were excluded from the training effect analyses. An additional 6 participants failed to do the follow-up assessment and were excluded from the TLFB training effect analysis. Furthermore, baseline data from 2 participants on the RCQ question 3, and 1 participant on the VPT, were missing because of technical problems and therefore excluded from the relevant analyses.

At baseline, participants had consumed an average of 15.09 standard units of alcohol (SD 11.46) during the previous 7 days and binged on average on 6.65 occasions (SD 3.48) during the previous 30 days. The AUQ average HAC score was 230.91 (SD 17.17) and with 94% (90/96) of participants scoring ≥8 and 71% (68/96) scoring ≥11, indicating hazardous drinking in a large proportion of the sample [27,28]. See Table 2 for an overview of baseline characteristics. No significant group differences were detected at baseline.

All dependent variables were screened for univariate extreme outliers based on inspection of stem-and-leaf and box plots. When they were present, or one of the general linear model (GLM) assumptions were violated, a nonparametric method for factorial repeated-measures analysis of variance was used: the Aligned Rank Transform analysis of variance [34], in which data are aligned and then ranked as a preprocessing step, before applying GLM procedures (these results are marked with *).

This study aimed to decrease attentional bias toward alcohol and hazardous drinking behavior in young adults by using a CBM-A training with game elements. After training, there was an overall decline in attentional bias on the VPT task, but this effect was primarily driven by the regular VPT training condition, where a stronger bias was observed at baseline. The training effect did not generalize to the VST task, nor was there a decline in alcohol use after the training. Motivation to train decreased equally in all conditions, indicating that the training indeed became boring over time, but also that the motivational elements of the Shots game could not sufficiently counteract this effect. Interestingly, motivation to change, with respect to planning to drink less in the future, increased in the regular and placebo training but decreased in the game training condition. Moreover, participants in the game condition indicated a lower motivation to continue training compared with the other conditions. Participants also differed with regard to overall speed and accuracy of responses during the training, which may be due to the more complex nature of the gamified version of the training or the level of engagement in the training.

These findings regarding the motivational effects of the training may have important implications on the potential risks involved with using certain types of game elements. Although the gamified training arguably looked fancier than the regular training, the game elements in this study merely consisted of upgraded visuals and a coin-based reward system. There was no story line and only limited progression and personalization options were available in the game. As such, it is likely that these rather minimal game elements alone were insufficient to increase motivation to train in our student sample. Moreover, most participants will likely compare any gamified training with what they believe a game experience should look and feel like [22]. If the game training experience then disappoints, motivation could indeed take a dive, even going below the level observed in the regular training conditions, because expectations were higher to begin with. This was perhaps reflected in our finding that participants in the game condition, specifically, were less motivated to continue training, as well as start drinking less, after the training. Finally, these results may also be related to the visual and auditory game elements used, which might have distracted participants from the training elements, rendering it less effective. Indeed, the standard training condition with no game elements did show a small change in attentional bias. Moreover, motivation to change increased only in the nongame conditions. It could be that the exposure to alcohol cues gave participants a push toward a readiness to change but only when there were no distracting game elements surrounding those cues. Although this last point is speculative, it is clear that some game elements may be detrimental not only to motivation to train but also to the training mechanisms themselves. More research is necessary in order to determine which game elements are best suited for cognitive training, and CBM-A in particular. For example, more levels, a background story, or the introduction of character development throughout the multiple sessions of the game could benefit participants’ motivation. However, such additional elements would also have to be tested to see what their effect is on cognitive bias and related behavior.

Some limitations apply to this study. First, despite hazardous drinking in a substantial part of the sample, the modest training effects in this sample may be partially due to a relatively small alcohol attentional bias at baseline, which is a known moderator of training effects [13,35,36]. However, it should be noted that this notion does not necessarily make CBM-A inappropriate in such samples. For example, Schoenmakers and colleagues [21] detected no bias at baseline in their sample but still found a positive bias after training in their control group and a negative bias in their experimental group. Furthermore, this study included a total of 624 critical training trials divided over 4 sessions. Although this number is similar to that used in other research (eg, [19]), other attentional bias modification studies have used markedly larger numbers (eg, Schoenmakers et al [21], where participants completed 2640 training trials over 5 sessions). Given the very likely dose-response relationship between use and effectiveness of cognitive training paradigms, the amount of training practice may have prevented the training from efficiently changing attentional bias. Finally, a recent meta-analysis [37] concluded that Web-based CBM-A studies usually show smaller effect sizes than laboratory-based studies. Although the assessments took place in the laboratory, it is possible that the option to train at home had a negative effect on participants’ motivation, for example, by making participants take the training less seriously.

In sum, the novel game-like approach used in this study proved insufficient to motivate young adults to train, in comparison with a regular CBM-A training. In fact, some aspects of motivation appeared to deteriorate rather than improve, suggesting that gamification can have drawbacks if not done optimally. It could be concluded from this study that a point-based reward system in combination with fancy graphics does not satisfy participants’ expectations of what constitutes a game. Because one expects a game to be fun, this may have detrimental effects on motivation. Moreover, when those game elements distract participants from the training elements, they may actually impair performance. A second notion that can be taken from this study is that the observed attentional biases toward alcohol as measured with the VPT in this heavy drinking student sample were remarkably low. Whether this has implications for the presence of attentional bias in adolescent samples in general or merely pertains to the VPT paradigm as a valid assessment measure of attentional bias remains to be determined by future research. If nothing else, however, these results underscore the importance of careful scientific evaluation before serious games are used as interventions.