This paper is in the following e-collection/theme issue:

Games for Cognitive Assessment (61) Games and Gamification for Health (650) Cognitive and Neurorehabilitation (262) Cognitive Training for the Elderly (95)

Published on in Vol 8, No 2 (2020): Apr-Jun

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14030, first published .
Controlling for Placebo Effects in Computerized Cognitive Training Studies With Healthy Older Adults From 2016-2018: Systematic Review

Controlling for Placebo Effects in Computerized Cognitive Training Studies With Healthy Older Adults From 2016-2018: Systematic Review

Controlling for Placebo Effects in Computerized Cognitive Training Studies With Healthy Older Adults From 2016-2018: Systematic Review

Authors of this article:

Alexander Masurovsky1 Author Orcid Image
Article Authors Cited by (9) Tweetations (1) Metrics

    Review

    Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany

    Corresponding Author:

    Alexander Masurovsky, BA

    Berlin School of Mind and Brain

    Humboldt-Universität zu Berlin

    Unter den Linden 6

    Berlin, 10099

    Germany

    Phone: 49 30 2093 89768

    Email: masurova@student.hu-berlin.de


    Background: Computerized cognitive training has been proposed as a potential solution to age-related cognitive decline. However, published findings from evaluation studies of cognitive training games, including metastudies and systematic reviews, provide evidence both for and against transferability from trained tasks to untrained cognitive ability. There continues to be no consensus on this issue from the scientific community. Some researchers have proposed that the number of results supporting the efficacy of cognitive training may be inflated due to placebo effects. It has been suggested that placebo effects need to be better controlled by using an active control and measuring participant expectations for improvement in outcome measures.

    Objective: This review examined placebo control methodology for recent evaluation studies of computerized cognitive training programs with older adult subjects, specifically looking for the use of an active control and measurement of expectations.

    Methods: Data were extracted from PubMed. Evaluation studies of computerized cognitive training with older adult subjects (age ≥50 years) published between 2016 and 2018 were included. Methods sections of studies were searched for (1) control type (active or passive) and subtype (active: active-ingredient or similar-form; passive: no-contact or passive-task); (2) if expectations were measured, how were they measured, and whether they were used in analysis; and (3) whether researchers acknowledged a lack of active control and lack of expectation measurement as limitations (where appropriate).

    Results: Of the 19 eligible studies, 4 (21%) measured expectations, and 9 (47%) included an active control condition, all of which were of the similar-form type. The majority of the studies (10/19, 53%) used only a passive control. Of the 9 studies that found results supporting the efficacy of cognitive training, 5 were for far transfer effects. Regarding the limitations, due to practical considerations, the search was limited to one source (PubMed) and to search results only. The search terms may have been too restrictive. Recruitment methods were not analyzed, although this aspect of research may play a critical role in systematically forming groups with different expectations for improvement. The population was limited to healthy older adults, while evaluation studies include other populations and cognitive training types, which may exhibit better or worse placebo control than the studies examined in this review.

    Conclusions: Poor placebo control was present in 47% (9/19) of the reviewed studies; however, the studies still published results supporting the effectiveness of cognitive training programs. Of these positive results, 5 were for far transfer effects, which form the basis for broad claims by cognitive training game makers about the scientific validity of their product. For a minimum level of placebo control, future evaluation studies should use a similar-form active control and administer a questionnaire to participants at the end of the training period about their own perceptions of improvement. Researchers are encouraged to think of more methods for the valid measure of expectations at other time points in the training.

    JMIR Serious Games 2020;8(2):e14030

    doi:10.2196/14030

    Keywords

    computerized cognitive training; brain training; placebo; active control; elderly; older adults 


    Cognitive Training as a Solution for Age-Related Cognitive Decline

    As the many of the world’s nations face increasingly older populations, much attention has been given to how to limit the deleterious effects of aging on cognitive functioning, which are marked by a decrease in performance on a number of cognitive tests in the domains of memory, speed of processing, executive functioning, attention, and visual perception [1-4]. One proposed solution is computerized cognitive training, which has shown some promise in slowing age-related cognitive decline [5-7]. However, evaluation studies of computerized cognitive training have shown that it is still unclear whether success at training transfers to improved cognitive ability (see the following section for an explanation of transfer effects). Two opposing scientific consensus statements on this matter, signed by hundreds of scientists each, were published in 2014 [8,9]. Various studies have found significant effects of cognitive training on far transfer tests (indicating effects on broader categories of cognitive functioning) [5,6,10] or that cognitive training does little more than to improve abilities on near transfer tests (indicating effects on tasks similar to the training), with no indication of far transfer [11-13]. With such discrepancy in the field, some researchers have turned to analysis of the methodology used in cognitive training studies and identified, among other issues, a failure to properly control for placebo effects [12,14,15], which are systematic factors related to, but separate from, the training itself that may have been a causal component of observed effects [16] (see the “What Is the Placebo Effect” section for a more detailed explanation).

    Transfer Effects: Near and Far

    The term “transfer effect” indicates a significant, positive change on an outcome measure that is separate from the training itself and observed after completion of the training. The level of ecological validity for a transfer effect is denoted by the preceding word “near,” “far,” and sometimes “real-world” or “daily life.” This review is concerned with claims of near and far transfer. If an outcome measure is very similar to the tasks in the training, this is considered a measure of near transfer. For example, a training game might have a gamified version of an n-back task, with slight variations; this would make an n-back test one of near transfer for this game. Other tests of specific elements of working memory might also be considered tests of near transfer. Far transfer is assessed by tests of broad cognitive domains, such as memory, processing speed, or cognitive control. More comprehensive tests of working memory or a conglomeration of tests of different working memory elements would be considered tests of far transfer. I am not aware of any official methods to determine whether a particular outcome measure is one of near or far transfer. Rather, the aforementioned general guidelines are used in this review to categorize studies as reporting near or far transfer in the event that they do not label their own findings as such. Any result indicating either near or far transfer effects will be referred to in this review as having a positive result.

    What Is the Placebo Effect?

    The placebo effect has been studied for over 100 years [17] and is well-known in pharmaceutical research, especially research involving pain [18-20] and mood disorders [21-23]. In such studies, a sham treatment is administered in order to ensure that any positive response of the treatment is not due to the “symbols, rituals, and behaviors embedded in the clinical encounter” [24], such as the patient’s own expectations for improvement or social interactions with the clinician. Placebo effects do not necessarily replace an effect of the treatment; they can also occur alongside and interact with an effective treatment. For instance, a warm and caring demeanor on the part of the clinician administering medication may interact with the chemical components of the medication itself to produce an even stronger reduction in the patient’s subjective experience of pain [25]. The problem occurs when this effect systematically occurs differently in one experimental condition than in the other. If this phenomenon occurs without being detected by researchers, it could appear to be an effect caused by the experimentally manipulated variable (the medication). When properly controlled for, a control condition should elicit an equal placebo response to the experimental condition, strengthening evidence that any effect found was due to the treatment itself.

    Controlling for the Placebo Effect in Cognitive Training Studies: Active and Passive Controls

    Pharmaceutical research uses double-blinding, random assignment, and utilization of a placebo treatment to minimize differences in possible placebo effects between control and experimental conditions. Two proposed solutions in cognitive training are the use of a carefully designed active control condition and the measurement of participant expectations for improvement [14]. In the following two sections, I give definitions of active controls and passive controls as well as two subtypes for each type of control that describe common methods for their implementation.

    Active Controls: Active-Ingredient and Similar-Form Types

    An active control game is meant to generate the same expectations for improvement on a cognitive assessment as the experimental training game, without providing the essential elements of the training that drive the effect on cognition. An ideal active control game, then, might be identical to the experimental game in every respect except for one, the hypothesized “active ingredient” that drives the effect of the game on cognitive performance [14,26]. For the purposes of this review, this will be referred to as the active-ingredient type of active control. In one example, in a multitasking game created to improve interference resolution (believed to be a component of executive control), Mishra et al [26] created a nearly identical game to serve as the active control but focused the challenge on the attention element of the game, rather than the interference element. In the second example, in a cognitive training game targeting multitasking ability, Anguera et al [27] created a single-tasking version of the game for the active control condition.

    A similar-form type of active control mimics the training game in form but differs in details of the gameplay in more than one way. For instance, Cujek and Vranic [28] tested a computerized version of a challenging card game as cognitive training and used a computerized dice game as an active control. Between the conditions, the constants were the time spent playing a game at a computer and interaction with the experimenter. Ideally, participants would be unable to figure out from the gameplay if they were in the experimental or control condition, and expectations for improvement between conditions would remain constant. While perhaps less powerful than an active-ingredient type at generating similar expectations to the experimental training, the similar-form type is less demanding on study resources and is better suited to control for expectations than a passive control.

    Passive Controls: No-Contact and Passive-Task Types

    A common practice in cognitive training studies is to use a passive control game [12]. This review uses the term no-contact passive control to refer to studies where control subjects do not engage in any training and take only the pretest and posttest of cognition, often not speaking to researchers in between. This review uses the term passive-task passive control to refer to studies that have control subjects engage in an activity that shares little similarity with the experimental training program, such as watching educational DVDs or reading about cognitive training literature, and would be very unlikely to generate the same sorts of expectations (though expectation data could theoretically contradict this proposition).

    Evidence for Poor Placebo Control in Computerized Cognitive Training Studies

    In a paper published in 2013, Boot et al [14] suggested that differences in expectations for improvement between groups could be responsible for some or all of the published positive results in the cognitive training literature. In several studies of the effects of video games on cognition [29,30], fast-paced action video games were tested as potential cognitive enhancers, with slower-paced video games as the control condition. Boot and colleagues [14] surveyed a sample uninvolved in other video game studies for general expectations regarding the video games tested in those studies. The results of the survey predicted the observed results found by those studies of video game play on cognition, supporting the claim by Boot et al [14] that placebo effects could not be ruled out as a driver of some or all of the observed effects. In 2016, a large-scale systematic review of cognitive training literature by Simons et al [12] concluded that, among other methodological issues, many studies still lacked an active control group and were therefore, at best, weak evidence for the effectiveness of the cognitive training games tested.

    Study Groups May Systematically Form Expectations That Impact Results

    There is evidence that self-selection bias may be systematically contaminating results of cognitive training studies. Simply changing the method of recruitment created groups that would perform differently on outcome measures, despite being matched in nearly every other way. Overtly advertising the opportunity to participate in a cognitive training study (eg, “Brain Training & Cognitive Enhancement”) recruited a sample that achieved higher on tests of cognition after training, compared to a sample that was recruited via a neutral advertisement (eg, “Email Today & Participate in a Study”) [15]. Groups were matched for intelligence and motivation, and the groups performed equally well on the training tasks. The only measurable difference, other than performance on outcome measures, was that questionnaire data indicated the overtly recruited group had a stronger belief that intelligence is malleable rather than a fixed property of genetics. This study provides evidence that overt recruitment methods alone can create samples biased towards belief in the effectiveness of cognitive training, which can interact with the experimental condition to generate an improvement on outcome measures.

    There are signs that overt methods are a common method of recruitment. Foroughi et al [15] checked recruitment methods for studies in a meta-analysis by another author [31] and found that 17 of the 19 studies had used overt recruitment methods. In this review, recruitment methods were not assessed; however, it is notable that the majority of studies may be unintentionally and systematically recruiting subjects that hold optimistic beliefs about the power of cognitive training.

    Other Evidence of Optimism About the Effectiveness of Cognitive Training

    People generally believe that cognitive training is effective [32]. Furthermore, participants indicated higher or lower optimism for the effectiveness of cognitive training after the researchers showed them brief statements about the topic. The majority of elderly participants in one cognitive training study believed they had improved, even though no significant improvement had been found at the group level. Goghari and Lawlor-Savage [33] provided an analysis of expectation data; for actual cognitive training data, see the 2017 article by Goghari and Lawlor-Savage [34]. This indicates there may be a general optimism in elderly populations that cognitive training is effective.

    Placebo Effect Versus a Motivation Effect

    There is another way to consider expectation effects: For cognitive training to be effective, a person needs to believe in it in order to approach the training in a motivated way. When is it a placebo effect, and when is it an interaction between the training program and motivation? I will differ on this question here. The question underlying this review was not the mechanism of how expectations may influence observed effects or even if expectations influence observed effects, but rather, in light of evidence that expectations may impact performance on outcome measures in cognitive training studies, are recent studies measuring and properly controlling for expectations between groups?

    Why Is it Imperative to Control for a Placebo Effect in Cognitive Training Research?

    Unsupported claims about cognitive training carrying the blessing of scientific research can have direct consequences for society. Such claims may encourage people to spend time or money on cognitive training games, especially older adults that may be experiencing age-related cognitive decline and are therefore vulnerable to the marketing tactics of game makers. One example of such a claim about cognitive training studies is “What they show is that for the average person, our exercises truly speed up and sharpen the brain” [35]. Studies, such as that by Grönholm-Nyman et al [11], and meta-analyses, such as that by Sala and Gobet [36], have provided evidence against the existence of far transfer from cognitive training studies, but enough positive results exist [5,10] that the debate continues.

    Population: Healthy Older Adults

    Cognitive ability may begin to decline due to age starting as early as 25 years old [37], resulting in a decrease in functionality for functions such as memory, speed of processing, executive functioning, attention, and visual perception [1-4]. Rather than viewing the new arrival of computerized cognitive training with fear or distrust, there are signs that older adults are generally optimistic about the effectiveness of such training [32,33]. There is some evidence that cognitive training may result in improvement on cognitive tests and effectively slow age-related cognitive decline in older adults [5-7], especially for narrow transfer effects [12,38]. However, there is an opportunity cost: If cognitive training is not effective at improving cognition or slowing decline, older adults could be wasting valuable time that could be spent exercising, socializing, or engaging in other activities for which there is stronger evidence of a benefit to cognition [12,39].

    Objective and Scope of This Review

    In light of the diverging results in the cognitive training literature, evidence for potential widespread placebo effects in cognitive training studies should be taken seriously. If studies are not controlling for placebo effects, the possibility cannot be ruled out that this is a systematic methodological problem, falsely inflating the number of positive results. This review examined cognitive training literature from the previous 2 years (2016-2018) in studies of healthy older adults, specifically to determine the proportion of current studies focused on this population that properly controlled for potential placebo effects. Placebo control was assessed primarily by whether participant expectations for improvement were measured (in any form) and whether the study included an active control condition. Special attention was paid to studies that report far transfer effects. Studies with poor placebo control that find positive results for far transfer effects form the basis for the most extreme and potentially most egregious claims in the cognitive training game industry.


    Registration

    This review was not pre-registered. However, it was structured based on the PRISMA guidelines [40]. This review was conducted to fulfill a paper assignment for the Master’s Program at the Berlin School of Mind and Brain at Humboldt-Universität zu Berlin.

    Eligibility Criteria

    Inclusion criteria were as follows: randomized controlled trials (including exploratory studies); participants ≥50 years old; participants not diagnosed with any cognitive disorder, including mild cognitive impairment; administration of the experimental cognitive training in electronic form, such as on a computer or tablet; outcomes were assessments of cognitive ability; and studies published between 2016 and 2018.

    Exclusion criteria were as follows: training interventions of which the target was motor ability, such as gait; an experimental variable in a form other than that of computerized cognitive training, such as transcranial direct current stimulation (tDCS) dosage, exercise, or viewing instructional videos; reviews, metastudies, and other nonrandomized controlled trials; participants younger than the minimum age or with diagnosed cognitive impairment or neurodegenerative disease; studies published before 2016; and studies unavailable in an English translation.

    The criteria were established to be able to make a generalization about current placebo control practices in studies that fit the aforementioned criteria. Motor activity as a target of cognitive training involves different enough sorts of training and tests than ones used for measurement of other cognitive abilities that it would widen the scope of the review too much. The time range was limited to the past 2 years so that the review can serve as a measurement of current standards of placebo control in cognitive training research. A healthy population was chosen because methods for recruitment, training, and testing of impaired populations may vary widely from those of healthy populations. Methodological practices in healthy populations of this age group may also be more easily comparable to those of healthy populations in other age groups. The definition of an older adult varies between studies, but the age was set to ≥50 years to be more inclusive.

    Information Sources

    Studies were identified via a search of PubMed Online performed on December 1, 2018 by the study author. Each item that appeared in the search result was evaluated according to the inclusion and exclusion criteria.

    Search

    The formula of search terms entered into PubMed was: (“2016/01/01”[Date - Publication] : “3000”[Date - Publication]) AND (cognitive[Title/Abstract]) AND (training[Title/Abstract]) AND ((game[Title/Abstract]) OR (computer[Title/Abstract])) AND ((older adult*[Title/Abstract]) OR (elderly[Title/Abstract])) NOT ((physical[Title]) OR (motor[Title/Abstract]) OR (exercise[Title]) OR (movement[Title]) OR (walking[Title/Abstract]) OR (gait[Title/Abstract]) OR (MCI[Title/Abstract]) OR (mild cognitive impairment[Title/Abstract]) OR (Alzheimer*[Title/Abstract])).

    Study Selection

    Eligibility was assessed according to the previously described inclusion and exclusion criteria. Following the search, titles and abstracts of each returned result were reviewed for search criteria. If eligibility could not be determined in this way for a given paper, the methods section was inspected more closely. Eligibility was possible to determine for all studies using this method.

    Data Collection Process

    The aforementioned search query was constructed by trial and error until most of the undesired study types (eg, motor ability as an outcome measure) were not returned in search results and many results for the desired study types (cognitive training game evaluations) were. A printout of results for the official search was saved. Each study on the list was then systematically checked for inclusion and exclusion criteria, first by looking at the title and abstract, then in the Methods section if needed. Data items of interest (see next section) were recorded in an Excel document for each included study.

    Data Items

    The following data items were coded for each study. First, the measurement of expectations was coded as yes or no. If “yes” was determined for expectations measured, then how they were measured and if the expectation measurements were used in the analysis of the cognitive training results were recorded. Second, the control type (active or passive) was recorded. If the control type was active, the subtype (active-ingredient or similar-form) was recorded. If the control type was passive, the subtype (no-contact or passive-task) was recorded. Third, positive results were defined as a significantly higher score on the outcome measures for the experimental group. If positive results were found, the type (near transfer, far transfer, or both) was recorded. Fourth, if expectations were not measured, whether the authors mentioned this as a limitation in the discussion section (yes or no) was recorded. Fifth, if an active control was not used, whether the authors mentioned this as a limitation in the discussion section (yes or no) was recorded.

    The reason for coding for positive results was to calculate the proportion of studies that found positive results that were performed without good placebo control (as defined by this review) compared to the overall number of included studies. This number was used as a metric of the proportion of cognitive training studies with methodology that may be biasing results and therefore, consensus in favor of efficacy.

    Near and far transfer were determined based on whether the authors claimed their results reflected near or far transfer and on an investigation of the description of the outcome measure by the reviewer to see if these outcome measures were meant to reflect more general measures of cognition (far transfer) or specific tests of subcategories (near transfer).

    Effect sizes were not recorded as this is review was meant as an inquiry into methodology, not as a synthesis of results.

    Summary Measures and Synthesis of Results

    A summary of the data collected for the listed items is presented in the Results section. Conclusions were drawn about whether cognitive training studies are properly controlling for placebo, based on the proportion of studies measuring expectations and of those using an active control.


    Study Selection

    The initial search of PubMed returned 38 results. Of these, 19 met the eligibility criteria, and 19 were excluded. Studies were excluded if: they were preregistered trials that had not yet returned results (3 articles), they were not evaluation studies of cognitive training on cognitive performance (15 articles), or were focused on subjects’ perceptions of the benefits of playing these games (1 article). The 15 articles excluded for being nonevaluation studies were comprised of: 2 correlational studies, 1 usability study, 1 survey of preferences (not expectations), 1 study with “flow” as an outcome measure (a measure of player absorption into a game), 1 evaluation study of tDCS on cognitive ability, 1 that used electroencephalogram gamma activity as an outcome measure, 1 that included a cohort of younger adults, and 6 studies that fell into one of the following categories: metastudy, review, conference proceedings, or opinion essay. Another paper was excluded because, although it stemmed from a cognitive training evaluation study, its focus was on subjects’ perceptions of the benefits of playing these games [33].

    While it was not a study of a training game, 1 paper that was a study of general iPad training in improving cognition was included [41] because it was computerized training (via tablet) and followed the clinical evaluation format. Although it used several arcade-style games as the cognitive training and had other signs of methodological problems with reporting and possibly English translation issues (ie, used a dementia screening examination as a test of global cognition), 1 study [42] was included because it presented itself as an evaluation of “electronic cognitive training games” in its title; it was returned in the search results and had the potential to affect general perceptions of cognitive training game effectiveness.

    Study Characteristics

    Studies That Measured Expectations

    Of the 19 included studies, only 4 (21%) studies made any attempt to measure expectations to improve their experimental and control interventions (Table 1). Of those 4 studies, 3 used the expectation data in the interpretation of results. Of the 4 studies, 2 used an active control of the similar-form type, while 2 used a no-contact, passive control group (see next section for results on the use of active controls). Of the 4 studies that measured expectations, 2 found positive results for near transfer effects, and none found positive results for far transfer. Of the 15 studies that did not measure expectations, 5 mentioned this as a potential limitation of their study, while 10 did not, which suggests that the authors were either not aware of this as or did not believe this to be a potential confounding factor.

    Table 1. Studies that measured expectations.
    Search result #StudyControl typeControl subtypeHow were expectations measured?Used in analysis of results?Positive result?Transfer
    6Souders et al (2017) [38]ActiveSimilar-formExpectation for improvement in various cognitive modalities measured at end of interventionYesYesNear
    14Hynes (2016) [43]PassiveNo-contactOne question on questionnaire following training: “How much do you think you benefited from the training?” Rating: 2.62 of 4 (SD 0.67)YesNo—a
    21Guye and von Bastian (2017) [44]ActiveSimilar-formThree questions after the post-assessment: improvement in trained tasks, in untrained tasks, and in everyday life tasksYesNo—a
    30Yeo et al (2018) [45]PassiveNo-contactOne question after assessment: did your memory improve?NoYesNear

    aPositive results were not reported.

    Active Control

    Of the 19 included studies, 9 (47%) included an active control in the study design. None of these utilized an active-ingredient type game, such as that suggested by Mishra et al [46], where the control game mirrors the experimental in every way except for one factor hypothesized to drive the effect of training on cognitive ability. All 9 used a similar-form type of active control, where both groups play a game presented in a similar form for a similar amount of time, although details of the games themselves may vary (eg, the puzzle game suite, administered on a tablet, that served as the active control for the tablet-based training game suite used by Souders et al [38]). Of these studies, 3 also included a passive control group. Of the 9 studies using an active control, 7 found positive results: 6 for near transfer and 1 for far transfer.

    Passive Control

    Of the 19 included studies, 10 used a passive control and no active control group. Of these 10 studies, 5 used passive-task control groups, which completed simple tasks such as meeting the experimenter or reading information pamphlets about cognitive training. The other 5 studies used no-contact control groups that simply took the cognitive assessment tests before and after the experimental group had completed its training. Of the 10 studies that did not use an active control, 9 (90%) reported a significant positive effect of training, when compared with the control; 1 of these studies measured expectations but in a limited capacity and did not include this measurement in analysis of results. Of these 9 positive results, 5 were for near transfer effects, 2 were for far transfer effects, and 2 were for both near transfer and far transfer effects. Of the 10 studies that used only a passive control, only 4 mentioned this as a limitation to their study results, suggesting a potential lack of awareness on the part of the authors of the remaining 6 studies. Of these 6 studies that did not mention this limitation, 2 wrongly claimed to have included an “active placebo” condition (both studies had the same principle author; see the Discussion for more information).

    Far Transfer

    A significant, positive effect of training on tests of far transfer were reported by 6 studies. None of the studies measured expectations. Only 1 included an active control [28], 4 used passive-task controls, and 1 used a no-contact passive control.

    Results of Included Studies

    Table 2 presents the results of the included studies.

    Table 2. Results of included studies.
    Search result #StudyControl typeControl subtypeDescription of conditionsExpectations measured?Known limitation: expect-ations?Known limitation: active control?Positive result?TransferDescription of result
    4Cujzek & Vranic (2017) [28]ActiveSimilar-formExperimental: challenging computerized card game; active control: computerized dice-rolling gameNoNo—aYesFarReasoning test (D-48) was different enough from skills required for the card game for authors to conclude that this was evidence of far transfer. Effect was maintained at 4-month follow up.
    6Souders et al (2017) [38]ActiveSimilar-formExperimental: Mind Frontiers suite; active control: puzzle game suiteYes—b—aYesNearCorsi block tapping test (memory) was similar to game task. However, expectations for improvement may actually have MASKED observed improvement.
    7Kühn et al (2017) [47]BothActive: similar-form; passive: no-contactExperimental: Inhibition training game on a tablet; active control: general-purpose cognitive training platform; passive: pretests and posttests onlyNoNo—aYesNearExperimental group showed significant improvement on inhibition task while other groups did not.
    8Ballesteros et al (2017) [48]ActiveSimilar-formExperimental: Luminosity games; active control: The Sims or SimCity BuildNoNo—aNo—cNonsignificant trend on n-back task for training; significant effect for group on oddball task in favor of active control
    10Pereira-Morales et al (2018) [49]PassivePassive-task2 experimental groups: computerized training and computerized + pen and paper training; control: read brochure about cognitive trainingNoNoNoYesFarExperimental groups improved on cognitive tests more than passive control group. Experimental groups could not be considered active controls for each other because they were the same training, although one had additional training.
    12Lussier et al (2017) [50]PassivePassive-task2 experimental groups: VPTd vs FPTe; passive control: computer classesNoNoNoYesBothVPT had larger effect than FPT for near transfer and smaller effect for far transfer.
    13Toril et al (2016) [51]PassivePassive-taskExperimental: Luminosity, 15 1-hr training sessions; control: met with experimenter once a monthNoYesYesYesBothJigsaw puzzle task, digit forward (short-term memory), and Faces I and Faces II (episodic memory), Corsi Blocks; also maintained after 3 months (except for Corsi blocks)
    14Hynes (2016) [43]PassiveNo-contactExperimental: watched videos about cognitive training and played adaptive online training games; control: took pretests and posttestsYes—bNoNo—cNo difference found; therefore, no evidence of transfer effects (but: exploratory study)
    15Vaportzis et al (2017) [52]PassiveNo-contactExperimental: learned how to use regular apps on a tablet; passive control: only took pretests and posttestsNoNoYesYesNearImproved processing speed compared to control group
    17Lussier et al (2017) [53]PassivePassive-task2 experimental conditions: heterogeneous training context and homogeneous training context; control: computer lessonsNoNoNoYesNearTraining groups both had significantly better scores on near transfer tests than control. Heterogeneous training led to steeper improvement of dual-task coordination learning curve.
    18Grönholm-Nyman et al (2017) [11]ActiveSimilar-formExperimental: task-switching training games; control: fun games (Tetris, Angry Birds, and Bejeweled)NoYes—aYesNearVery limited: only one near-transfer effect found: overall accuracy on a rule-based part. Training group also improved on the training tasks (very near transfer).
    21Guye and von Bastian (2017) [44]ActiveSimilar-formExperimental: WMf training games; control: visual search training gamesYes—b—aNo—cBayesian analysis supported evidence for the null hypothesis. Expectation data went in opposite direction of observed result.
    22Chan et al (2016) [41]PassivePassive-taskExperimental: learned how to use a tablet; 2 control groups: games and radio programs at home, social groups met to discuss topics (to control for social interaction of training but limit new learning)NoNoNoYesFariPad training improved processing speed and episodic memory compared with controls.
    24Perrot et al (2018) [54]BothActive: similar-for; passive: no-contactExperimental: Kawashima Brain Training; active control: Super Mario Brothers; passive control: only pretests and posttestsNoYes—aYesNearExperimental training led to higher Stroop score than active control; both experimental training and active control had higher matrix reasoning scores than passive control; active control was significantly better at Corsi block test, spatial relations test, and number comparison test.
    25Nouchi et al (2016) [55]ActiveSimilar-formExperimental: processing-speed training game; active control: knowledge quiz training gameNoNo—aYesNearImprovements in processing speed, inhibition, and mood (depression scale) compared to active control
    27Belchior et al (2018) [56]BothActive control: similar-form; passive control: no-contactExperimental: PositScience Insight (visual attention and processing speed); active control: Crazy Taxi; passive control: took pre and post-testsNoYes—aYesNearCognitive training improved visual attention and processing speed.
    30Yeo et al (2018) [45]PassiveNo-contactExperimental: BrainFit software, which was controlled via BCI/dry EEGg headband; passive control: took pretests and posttestsYes—bYesYesNearMen in intervention group outperformed men in control group on RBANSh total score and subscore of Delayed Memory and Language.
    31Ordonez et al (2017) [42]PassiveNo-contactExperimental: Actively Station cognitive training game suite, a series of games, many involving physical movement; passive control: took pretests and posttestsNoNoYesYesFarTraining group improved on global cognition, verbal fluency, memory complaints, and mood compared with control. Reported a significant result for language, which is contradicted by the data table, unless what was truly meant was verbal fluency. Other methodological problem: ACE-Ri is a dementia screening test.
    34Sosa and Lagana (2018) [57]PassiveNo-contactExperimental: Brain Age; passive control: pretests and posttestsNoYesYesYesNearTraining group scored higher in brief syllable count and arithmetic assessments.

    aThe study included active control (not a limitation).

    bThe study controlled for expectations (not a limitation).

    cThe study reported no transfer effects.

    dVPT: variable priority training.

    eFPT: fixed priority training.

    fWM: working memory.

    gEEG: electroencephalogram.

    hRBANS: Repeatable Battery for the Assessment of Neuropsychological Status.

    iACE-R: Addenbrooke\'s Cognitive Examination Revised.

    Synthesis of Results

    The summary of the control for expectations is shown in Table 3. The first two columns report the proportion of included studies that took methodological steps identified in this review to control the influence of expectations: measuring expectations and including an active control condition. The third and fourth columns show the proportion of studies whose results may be misleading due to concerns raised in this review: those that did not include an active control, yet found a significant effect for their intervention, and those that additionally found a far transfer effect.

    Table 3. Summary of expectation control (n=19).
    StudiesMeasured expectations, n (%)Used active control (all similar-form), n (%)No active control + positive result, n (%)No active control + far transfer results, n (%)
    Studies that fit the criteria4 (21)9 (47)9 (47)5 (26)

    Summary of Evidence

    This review found that 4 of 19 studies (21%) measured expectations; therefore, the majority of studies (15/19, 79%) did not find it necessary to measure expectations for improvement. Of these 15 studies that did not measure expectations, 9 studies also did not mention this as a limitation of the study. A smaller majority of studies (10/19, 53%) did not include an active control condition, 6 of which did not mention this as a limitation. Perhaps most troubling is that 9 of 10 of the studies that did not include an active control also found a positive result; therefore, 9 of the 19 studies included here (47%) have published results reporting positive effects and did not properly control for placebo effects.

    Studies That Measured Expectations: Further Analysis

    Expectations were measured in some form in 4 studies. Of these, 2 found positive results, 1 found null results, and 1 found evidence supporting the null hypothesis using Bayesian analysis.

    Souders et al [38] provided the strongest evidence for a positive result that was not due to undetected placebo effects. They used an active control and administered a detailed expectation questionnaire. They found that the experimental group improved more than the active control group on the Corsi Block Tapping test, a test of memory similar to the training game task and therefore a near transfer result. The expectation data suggest that their subjects expected more improvement from the puzzle games (the active control) than from the intervention games, strengthening the claim that this was not a placebo effect. The authors suggest that expectation effects may even have gone in the opposite direction, potentially masking a bona fide effect from the training on cognition.

    Guye and von Bastian [44] provide strong, expectation-supported data against the effectiveness of working memory training. Using Bayesian analysis, they found evidence in favor of the null hypothesis, suggesting that the computerized working memory training regimen they used had no greater effect than the active control condition on cognitive ability. To gather data on expectations, they asked whether their subjects believed they had improved in three areas: on training tasks, on untrained tests of cognition, and on real-life tasks. The expectation data suggest that participants believed that they had improved as a result of the training. The results of this study support other research [13,58], suggesting that working memory training in general may not confer any near or far transfer effects on cognition in healthy subjects.

    In an exploratory study with a small sample, Hynes [43] found no effect for a computerized cognitive training program on a battery of cognitive tests, including a reasoning test correlated with fluid intelligence and a test of general intelligence. Responses to a question gauging participant expectations suggest that participants had moderate expectations for improvement, at least following the training period (mean 2.62/4, SD 0.67). The study used a no-contact passive control design, which provides weak control of potential placebo effects; however, it may strengthen a null result, under the assumption that expectations for improvement would be higher for the training group than the no-contact group. A potential rebuttal to this assumption is that the training program could discourage participants and lower expectations for improvement. In this case, however, participants indicated on a single expectation question, administered at the end of outcome testing, that they believed they had improved. More expectation questions could have potentially helped to better understand participant expectations.

    Yeo et al [45] found a sex-dependent, near transfer effect for cognitive training on scores of the Repeatable Battery for the Assessment of Neuropsychological Status, which tests a number of cognitive domains including language, memory, attention, and visuospatial construction. Expectation data were collected in the form of one question presented after the outcome measures, asking participants if they believed their memory had improved as a result of the training; the data suggest that they did believe that it had. While the authors interpreted this as a subjective marker of cognitive improvement, it also could mean that their positive results are weakened by the expectation data, as they cannot rule out the possibility that positive expectations were responsible for the sex-dependent increase in performance. Their design used a no-contact control; thus, the expectation question was only administered to the cognitive training group. It cannot therefore be known whether expectations for improvement differed between conditions.

    False Definition of Active Control

    Two studies, both with Lussier as the first author, claimed to include an “active placebo” condition [50,53]; however, an examination of the methods indicated that both studies used a passive control condition of the passive-task type, according to the definitions in this review. This distinction is important because a passive control provides weaker control for potential placebo effects than an active control. The training conditions in both studies were variations on group-administered, computer-based cognitive training; the control condition in both studies was computer classes, teaching how to use software such as Microsoft Excel. While the participants engaged in learning in both conditions, expectations for improvement on cognitive assessments could be very different for a training program that exists for the purpose of improving cognition, rather than to simply teach particular skills. Expectations were not measured, so there is no reason to believe that expectations for improvement were held constant across conditions. There is an argument to be made that the two experimental conditions in each study served as active control conditions for each other; however, the authors did not present nor analyze their data in this way [50,53].

    Recommendations for Researchers of Cognitive Training Games

    Measuring expectations can add to the practical workload of conducting a study. However, as demonstrated by some of the studies in this review, there are ways to measure expectations without significantly adding to the burden of data collection. All studies that collected expectation data that were reviewed here did so in the form of a questionnaire administered after the outcome measures. While potentially problematic because it cannot indicate what expectations were before or during training, it provides at least some insight into what expectations subjects held by the end of the training. This post hoc expectation data, while limited, can still be used in analysis of the observed results from the outcome tests. The study by Souders et al [38] serves as a model of expected data collection using a variety of expectation questions at the end of the training (note: Walter R. Boot, of Boot et al [14], was an author on this study and presented analysis of expectations towards the experimental and control games in a separate paper [59]). This study is also commendable for its use of an active control, of the similar-form type. Their experimental design strongly supports their claim that observed results were not likely to be placebo effects.

    Using a variety of expectation questions can help elucidate whether participants formed expectations for improvement in specific cognitive modalities. Two of the studies reviewed here [43,45] used only one question to assess expectations and accordingly lacked a detailed understanding of participant expectations. The study by Hynes [43] was exploratory, so a fully placebo-controlled design may not have been practically feasible; however, it serves as an example of how simply administering more questions could help to clarify participant expectations. The mean score for responses to this study’s expectation question implied participants thought they had improved due to the training; further questions could clarify whether they found the training discouraging, for instance, or if they believed they had improved on one specific modality, such as memory, but not another. The study by Yeo et al [45] could have also benefited from additional expectation questions, as their expectation data reveal very limited information about participant expectations.

    An active control provides all of the same benefits as a passive control, plus other benefits, such as expectation control. Beyond practical feasibility, there is no compelling methodological reason to forego use of an active control in favor of a passive control. No-contact passive controls offer some insight into the effects of repeated testing, but no control for any other placebo-related factors [16]. Passive-task control conditions can control for social interaction effects, but likely do not control for expectation effects, as participants can likely determine if they are in the control or training condition and form expectations for improvement accordingly.

    More work can be done to find ways to measure expectations, both for the conditions used in specific studies and for general expectations regarding various types of cognitive training. Boot et al [14] used a separate sample to measure expectations; Rabipour et al [32] surveyed people about expectations for cognitive training in general. Foroughi et al [15] examined the effects of recruitment methods on performance after training and measured belief in the ability to improve one’s own intelligence. There are, perhaps, creative new ways in which researchers could assess expectations before and during cognitive training or to gain insight into other placebo-related factors [16], such as social interaction, that may be impacting performance.

    Limitations

    This review was necessarily limited in its scope: Only one researcher was available, and it was conducted as one assignment to fill a paper requirement for a Master’s program. Only one search site was used (PubMed), the Works Cited sections of included papers were not screened for potential additions that matched the review criteria, and this review was not preregistered.

    The search terms may have been too restrictive by only returning results that contained the terms “game” or “computer” in the title or abstract. One study of cognitive training, by Goghari and Lawlor-Savage [34] was not found by this search query. This study measured expectations and reported a null result. It would have slightly changed the results of this review by increasing the number of studies that attempted to measure expectations from 4/19 to 5/20.

    Recruitment methods were not assessed in this review. Recruitment methods may play an important role in controlling for placebo effects [15] and would be a good consideration for future methodological reviews.

    Other types of studies that fell outside the scope of this review should be reviewed for placebo control methodology, as they are also potentially subject to placebo effects. Studies involving populations of unhealthy older adults, for instance, are numerous and should be considered; cognitive training may or may not be an effective way to mitigate degeneration and related symptoms due to neurodegenerative disorders such as Alzheimer’s disease or even for less severe conditions such as mild cognitive impairment. Combined studies of neurostimulation, such as tDCS, with cognitive training may yield effects not present with tDCS or cognitive training alone. Such studies form a recent area of cognitive training research and should be examined for placebo-control methodology as well.

    Conclusions

    In summary, this review found that the majority of computerized cognitive training studies with older adult samples from the past 2 years, which were included in this systematic review, have not measured expectations or properly controlled for potential placebo effects by including an active control. Methodologically speaking, only one study here [28] found placebo-controlled evidence for far transfer effects of cognitive training, although some that used an active control found evidence for near transfer effects. Studies that did measure expectations did so by administering an expectation questionnaire at the end of the training period. Cognitive training evaluation studies should, at a minimum, include a similar-form active control and measure expectations at the end of the training period. Ideally, future evaluation studies will include an active-ingredient type of active control and find creative new ways to measure expectations and control for other unintended types of placebo effects.

    If cognitive training can truly be effective, scientists have just as much of a duty to find evidence for this possibility as they do to find evidence against it. Well-controlled studies that rule out placebo as the reason for any observed effects strengthen the result, whether as evidence for or against the effectiveness of cognitive training.

    Acknowledgments

    I acknowledge support for the article processing fee from the German Research Foundation (DFG) and Open Access Publication Fund of Humboldt-Universität zu Berlin. Tuition for the Berlin School of Mind and Brain Master’s program is also made nearly free for all students. This project received no other funding. Thanks to Dr Michael Gaebler at the Max Planck Institute for Human Cognitive and Brain Sciences for providing mentorship in the writing of this paper and to Dr Dustin J Souders at Purdue University for providing a copy of one of his papers via ResearchGate.

    Conflicts of Interest

    The author does contract work for a company called neomento, which creates virtual reality exposure therapy solutions for therapists (eg, for anxiety disorders) but is currently not involved with cognitive training. There are no other potential conflicts of interest to declare.

    1. Deary IJ, Corley J, Gow AJ, Harris SE, Houlihan LM, Marioni RE, et al. Age-associated cognitive decline. Br Med Bull 2009 Sep;92:135-152. [CrossRef] [Medline]
    2. Yakhno NN, Zakharov VV, Lokshina AB. Impairment of memory and attention in the elderly. Neurosci Behav Physiol 2007 Mar;37(3):203-208. [CrossRef] [Medline]
    3. Salthouse TA. Memory aging from 18 to 80. Alzheimer Dis Assoc Disord 2003;17(3):162-167. [CrossRef] [Medline]
    4. Salthouse TA. The processing-speed theory of adult age differences in cognition. Psychological Review 1996;103(3):403-428. [CrossRef]
    5. Chiu H, Chu H, Tsai J, Liu D, Chen Y, Yang H, et al. The effect of cognitive-based training for the healthy older people: A meta-analysis of randomized controlled trials. PLoS One 2017 May;12(5):e0176742 [FREE Full text] [CrossRef] [Medline]
    6. Lampit A, Hallock H, Valenzuela M. Computerized cognitive training in cognitively healthy older adults: a systematic review and meta-analysis of effect modifiers. PLoS Med 2014 Nov;11(11):e1001756 [FREE Full text] [CrossRef] [Medline]
    7. Wolinsky FD, Vander Weg MW, Howren MB, Jones MP, Martin R, Luger TM, et al. Interim analyses from a randomised controlled trial to improve visual processing speed in older adults: the Iowa Healthy and Active Minds Study. BMJ Open 2011 Nov 21;1(2):e000225-e000225 [FREE Full text] [CrossRef] [Medline]
    8. A Consensus on the Brain Training Industry from the Scientific Community?. 2014. Max Planck Institute for Human Development and Stanford Center on Longevity   URL: http:/​/longevity3.​stanford.edu/​blog/​2014/​10/​15/​the-consensus-on-the-brain-training-industry-from-the-scientific-community/​ [accessed 2019-03-13] [WebCite Cache]
    9. Cognitive Training Data. Cognitive Training Data Response Letter   URL: https://www.cognitivetrainingdata.org/the-controversy-does-brain-training-work/response-letter/ [accessed 2019-03-13] [WebCite Cache]
    10. Mewborn CM, Lindbergh CA, Stephen Miller L. Cognitive Interventions for Cognitively Healthy, Mildly Impaired, and Mixed Samples of Older Adults: A Systematic Review and Meta-Analysis of Randomized-Controlled Trials. Neuropsychol Rev 2017 Dec;27(4):403-439. [CrossRef] [Medline]
    11. Grönholm-Nyman P, Soveri A, Rinne JO, Ek E, Nyholm A, Stigsdotter Neely A, et al. Limited Effects of Set Shifting Training in Healthy Older Adults. Front Aging Neurosci 2017 Mar;9:69 [FREE Full text] [CrossRef] [Medline]
    12. Simons DJ, Boot WR, Charness N, Gathercole SE, Chabris CF, Hambrick DZ, et al. Do "Brain-Training" Programs Work? Psychol Sci Public Interest 2016 Oct;17(3):103-186. [CrossRef] [Medline]
    13. Melby-Lervåg M, Redick TS, Hulme C. Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of "Far Transfer": Evidence From a Meta-Analytic Review. Perspect Psychol Sci 2016 Jul;11(4):512-534 [FREE Full text] [CrossRef] [Medline]
    14. Boot WR, Simons DJ, Stothart C, Stutts C. The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects. Perspect Psychol Sci 2013 Jul;8(4):445-454. [CrossRef] [Medline]
    15. Foroughi CK, Monfort SS, Paczynski M, McKnight PE, Greenwood PM. Placebo effects in cognitive training. Proc Natl Acad Sci U S A 2016 Jul 05;113(27):7470-7474 [FREE Full text] [CrossRef] [Medline]
    16. Fontaine KR, Williams MS, Hoenemeyer TW, Kaptchuk TJ, Dutton GR. Placebo effects in obesity research. Obesity (Silver Spring) 2016 Apr 30;24(4):769-771 [FREE Full text] [CrossRef] [Medline]
    17. Kaptchuk TJ. Intentional ignorance: a history of blind assessment and placebo controls in medicine. Bull Hist Med 1998;72(3):389-433. [CrossRef] [Medline]
    18. Atlas LY, Wager TD. A meta-analysis of brain mechanisms of placebo analgesia: consistent findings and unanswered questions. Handb Exp Pharmacol 2014;225:37-69. [CrossRef] [Medline]
    19. Colloca L, Benedetti F. Placebos and painkillers: is mind as real as matter? Nat Rev Neurosci 2005 Jul;6(7):545-552. [CrossRef] [Medline]
    20. Turner JA, Deyo RA, Loeser JD, Von Korff M, Fordyce WE. The importance of placebo effects in pain treatment and research. JAMA 1994 May 25;271(20):1609-1614. [Medline]
    21. Furmark T, Appel L, Henningsson S, Ahs F, Faria V, Linnman C, et al. A link between serotonin-related gene polymorphisms, amygdala activity, and placebo-induced relief from social anxiety. J Neurosci 2008 Dec 03;28(49):13066-13074 [FREE Full text] [CrossRef] [Medline]
    22. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 2008 Mar;5(2):e45 [FREE Full text] [CrossRef] [Medline]
    23. Shapiro AK. A contribution to a history of the placebo effect. Syst. Res 2007 Jan 17;5(2):109-135. [CrossRef]
    24. Hall KT, Loscalzo J, Kaptchuk TJ. Genetics and the placebo effect: the placebome. Trends Mol Med 2015 May;21(5):285-294 [FREE Full text] [CrossRef] [Medline]
    25. Enck P, Bingel U, Schedlowski M, Rief W. The placebo response in medicine: minimize, maximize or personalize? Nat Rev Drug Discov 2013 Mar;12(3):191-204. [CrossRef] [Medline]
    26. Mishra J, de Villers-Sidani E, Merzenich M, Gazzaley A. Adaptive training diminishes distractibility in aging across species. Neuron 2014 Dec 03;84(5):1091-1103 [FREE Full text] [CrossRef] [Medline]
    27. Anguera JA, Boccanfuso J, Rintoul JL, Al-Hashimi O, Faraji F, Janowich J, et al. Video game training enhances cognitive control in older adults. Nature 2013 Sep 05;501(7465):97-101 [FREE Full text] [CrossRef] [Medline]
    28. Cujzek M, Vranic A. Computerized tabletop games as a form of a video game training for old-old. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 2017 Nov;24(6):631-648. [CrossRef] [Medline]
    29. Green CS, Bavelier D. Action-video-game experience alters the spatial resolution of vision. Psychol Sci 2007 Jan;18(1):88-94 [FREE Full text] [CrossRef] [Medline]
    30. Green CS, Sugarman MA, Medford K, Klobusicky E, Daphne Bavelier. The effect of action video game experience on task-switching. Comput Human Behav 2012 May;28(3):984-994 [FREE Full text] [CrossRef] [Medline]
    31. Au J, Sheehan E, Tsai N, Duncan GJ, Buschkuehl M, Jaeggi SM. Improving fluid intelligence with training on working memory: a meta-analysis. Psychon Bull Rev 2015 Apr;22(2):366-377. [CrossRef] [Medline]
    32. Rabipour S, Davidson P. Do you believe in brain training? A questionnaire about expectations of computerised cognitive training. Behav Brain Res 2015 Dec 15;295:64-70 [FREE Full text] [CrossRef] [Medline]
    33. Goghari VM, Lawlor-Savage L. Self-Perceived Benefits of Cognitive Training in Healthy Older Adults. Front Aging Neurosci 2018 Apr;10:112 [FREE Full text] [CrossRef] [Medline]
    34. Goghari VM, Lawlor-Savage L. Comparison of Cognitive Change after Working Memory Training and Logic and Planning Training in Healthy Older Adults. Front Aging Neurosci 2017 Feb;9:39 [FREE Full text] [CrossRef] [Medline]
    35. Brain HQ. How we know it works   URL: https://www.brainhq.com/world-class-science/brainhq-effective/how-we-know-it-works [accessed 2019-03-13] [WebCite Cache]
    36. Sala G, Gobet F. Does Far Transfer Exist? Negative Evidence From Chess, Music, and Working Memory Training. Curr Dir Psychol Sci 2017 Dec;26(6):515-520 [FREE Full text] [CrossRef] [Medline]
    37. Salthouse TA. When does age-related cognitive decline begin? Neurobiol Aging 2009 Apr;30(4):507-514 [FREE Full text] [CrossRef] [Medline]
    38. Souders DJ, Boot WR, Blocker K, Vitale T, Roque NA, Charness N. Evidence for Narrow Transfer after Short-Term Cognitive Training in Older Adults. Front Aging Neurosci 2017 Feb;9:41 [FREE Full text] [CrossRef] [Medline]
    39. Dresler M, Sandberg A, Ohla K, Bublitz C, Trenado C, Mroczko-Wąsowicz A, et al. Non-pharmacological cognitive enhancement. Neuropharmacology 2013 Jan;64:529-543. [CrossRef] [Medline]
    40. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    41. Chan MY, Haber S, Drew LM, Park DC. Training Older Adults to Use Tablet Computers: Does It Enhance Cognitive Function? Gerontologist 2016 Jun;56(3):475-484 [FREE Full text] [CrossRef] [Medline]
    42. Ordonez TN, Borges F, Kanashiro CS, Santos CCDN, Hora SS, Lima-Silva TB. Actively station: Effects on global cognition of mature adults and healthy elderly program using eletronic games. Dement Neuropsychol 2017 Jun;11(2):186-197 [FREE Full text] [CrossRef] [Medline]
    43. Hynes SM. Internet, home-based cognitive and strategy training with older adults: a study to assess gains to daily life. Aging Clin Exp Res 2016 Oct;28(5):1003-1008. [CrossRef] [Medline]
    44. Guye S, von Bastian CC. Working memory training in older adults: Bayesian evidence supporting the absence of transfer. Psychol Aging 2017 Dec;32(8):732-746. [CrossRef] [Medline]
    45. Yeo SN, Lee TS, Sng WT, Heo MQ, Bautista D, Cheung YB, et al. Effectiveness of a Personalized Brain-Computer Interface System for Cognitive Training in Healthy Elderly: A Randomized Controlled Trial. J Alzheimers Dis 2018 Oct;66(1):127-138. [CrossRef] [Medline]
    46. Mishra J, Anguera JA, Gazzaley A. Video Games for Neuro-Cognitive Optimization. Neuron 2016 Apr 20;90(2):214-218 [FREE Full text] [CrossRef] [Medline]
    47. Kühn S, Lorenz RC, Weichenberger M, Becker M, Haesner M, O'Sullivan J, et al. Taking control! Structural and behavioural plasticity in response to game-based inhibition training in older adults. Neuroimage 2017 Aug 01;156:199-206. [CrossRef] [Medline]
    48. Ballesteros S, Mayas J, Prieto A, Ruiz-Marquez E, Toril P, Reales JM. Effects of Video Game Training on Measures of Selective Attention and Working Memory in Older Adults: Results from a Randomized Controlled Trial. Front Aging Neurosci 2017 Nov;9:354 [FREE Full text] [CrossRef] [Medline]
    49. Pereira-Morales AJ, Cruz-Salinas AF, Aponte J, Pereira-Manrique F. Efficacy of a computer-based cognitive training program in older people with subjective memory complaints: a randomized study. Int J Neurosci 2018 Jan;128(1):1-9. [CrossRef] [Medline]
    50. Lussier M, Bugaiska A, Bherer L. Specific transfer effects following variable priority dual-task training in older adults. Restor Neurol Neurosci 2017 Mar;35(2):237-250. [CrossRef] [Medline]
    51. Toril P, Reales JM, Mayas J, Ballesteros S. Video Game Training Enhances Visuospatial Working Memory and Episodic Memory in Older Adults. Front Hum Neurosci 2016 May;10:206 [FREE Full text] [CrossRef] [Medline]
    52. Vaportzis E, Martin M, Gow AJ. A Tablet for Healthy Ageing: The Effect of a Tablet Computer Training Intervention on Cognitive Abilities in Older Adults. Am J Geriatr Psychiatry 2017 Aug;25(8):841-851 [FREE Full text] [CrossRef] [Medline]
    53. Lussier M, Brouillard P, Bherer L. Limited Benefits of Heterogeneous Dual-Task Training on Transfer Effects in Older Adults. J Gerontol B Psychol Sci Soc Sci 2017 Sep 01;72(5):801-812. [CrossRef] [Medline]
    54. Perrot A, Maillot P, Hartley A. Cognitive Training Game Versus Action Videogame: Effects on Cognitive Functions in Older Adults. Games Health J 2019 Mar;8(1):35-40. [CrossRef] [Medline]
    55. Nouchi R, Saito T, Nouchi H, Kawashima R. Small Acute Benefits of 4 Weeks Processing Speed Training Games on Processing Speed and Inhibition Performance and Depressive Mood in the Healthy Elderly People: Evidence from a Randomized Control Trial. Front Aging Neurosci 2016;8:302 [FREE Full text] [CrossRef] [Medline]
    56. Belchior P, Yam A, Thomas KR, Bavelier D, Ball KK, Mann WC, et al. Computer and Videogame Interventions for Older Adults' Cognitive and Everyday Functioning. Games Health J 2019 Apr 29;8(2):129-143 [FREE Full text] [CrossRef] [Medline]
    57. Sosa GW, Lagana L. The effects of video game training on the cognitive functioning of older adults: A community-based randomized controlled trial. Arch Gerontol Geriatr 2019;80:20-30. [CrossRef] [Medline]
    58. Karbach J, Verhaeghen P. Making working memory work: a meta-analysis of executive-control and working memory training in older adults. Psychol Sci 2014 Nov;25(11):2027-2037 [FREE Full text] [CrossRef] [Medline]
    59. Boot W, Souders D, Charness N, Blocker K, Roque N, Vitale T. The Gamification of Cognitive Training: Older Adults’ Perceptions of and Attitudes Toward Digital Game-Based Interventions. In: Zhou J, Salvendy G, editors. Human Aspects of IT for the Aged Population. Design for Aging. ITAP 2016. Lecture Notes in Computer Science. Switzerland: Springer International Publishing; 2016:290-300.

    ‎
    ACE-R: Addenbrooke's Cognitive Examination Revised
    EEG: electroencephalogram
    FPT: fixed priority training
    RBANS: Repeatable Battery for the Assessment of Neuropsychological Status
    tDCS: transcranial direct current stimulation
    VPT: variable priority training
    WM: working memory

    Edited by G Eysenbach; submitted 15.03.19; peer-reviewed by N Roque, M Nahum, F Lanfranchi, S Ge, MS Aslam; comments to author 30.09.19; revised version received 28.02.20; accepted 29.03.20; published 26.06.20

    Copyright

    ©Alexander Masurovsky. Originally published in JMIR Serious Games (http://games.jmir.org), 26.06.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.