The age distribution of the world’s population is projected to dramatically shift over the next few decades as improved health care continues to extend life expectancy [1]. By 2050, more than one-fourth (27%) of the European population is expected to be aged >65 years. Although medicine can prolong relative physical health [2], offsetting age-related changes in cognitive health is growing in importance [3]. Although recent literature suggests that cognitive measures may inflate age-related decreases in performance [4-6] and disregard an increase in crystallized intelligence (eg, general knowledge and vocabulary [7]), a decrease in cognitive performance is one of the most feared aspects of aging [8]. Consequently, there is growing pressure to prolong active and healthy aging—successful aging [9]—by targeting age-related changes in cognitive abilities such as memory and executive functions (EFs; eg, inhibition and working memory [WM]) [10,11].

Numerous serious games for cognitive training in aging (SGCTAs) are being developed to mediate age-related cognitive changes. However, there is much debate in the literature regarding their efficacy. In fact, in 2014, a total of 2 teams of researchers published contradictory open letters. The first letter by a group of 70 scientists refuted the efficacy of such training [12]. The second letter by another group of 133 scientists claimed the opposite, supporting the benefit of cognitive training [13]. These letters were followed by an extensive review [14] suggesting that SGCTAs, in general, can improve performance on the trained game and associated activities—near transfer. However, the review cautions that there is insufficient evidence to suggest that these changes can generalize to activities that are not directly associated with the game—far transfer. As cognitive training games and interventions are a “means to enhance performance on other tasks” [14], it seems critical to measure their effects using far-transfer measures that gauge daily activities. Such far-transfer measures would gauge cognitive abilities through performance on a different task that is mediated by the trained cognitive functions.

Effective communication and speech perception play an extensive role in many daily activities and have evident effects on general health and well-being [15]. Difficulty in understanding speech in adverse conditions (eg, noisy background or while conducting another task) forms one of the most prevalent complaints among older adults [16]. These difficulties decrease the participation of older adults in social and professional interactions, thus limiting their independence and increasing feelings of loneliness. Growing evidence suggests that a decrease in speech processing, in turn, has a negative effect on mental health, general well-being, and even longevity [17-21]. The social restrictions imposed by the COVID-19 pandemic further limit interactions and other opportunities for cognitive exercise (eg, work and volunteering). Indeed, current restrictions have been found to increase loneliness and depression in older age [22], even after vaccinations were made available [23] and severe social restrictions were lifted [24]. These, together with limited access to health care services as a result of the pandemic [25,26], illustrate the necessity to create effective SGCTAs that can directly affect spoken communication in adverse conditions, even while social distancing. We suggest that testing speech processing as a far-transfer task could demonstrate the impact of training with SGCTAs, on the daily lives of older adults.

In this exploratory study, we used an eye-tracking paradigm to assess whether training EFs with the SGCTA Effectivate generalizes to improved speech processing in adverse listening conditions for older adults. This will serve to validate a real-time measure of speech perception—Eye tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL)—as a gauge of far-transfer efficacy of SGCTAs designed to train EFs and provide a case study for developers and academics on the use of far-transfer metrics.

Many SGCTA developers are targeting EFs because of their prominent role in healthy cognitive aging (refer to the seminal work by Salthouse [27,28]). EFs, which include WM and inhibition, enable active maintenance and manipulation of bottom-up information with top-down information in memory, especially during the performance of a concurrent task [29-31]. The literature suggests that individuals with better EFs are able to hold more incoming information and incorporate and manipulate it more easily, even under adverse conditions such as distractions (ie, external noise) and memory preload (ie, remembering the context of a conversation [32,33]). Therefore, it is not surprising that EFs play a significant role in speech processing [34].

Consider a scenario in which an older adult is driving his grandson in a car and radio music is playing. The grandson says, “grandpa, have you seen the DOLL?” The older listener must perform the following tasks:

As mentioned previously, EFs, especially WM, are essential to perform this complex task and have been shown to be affected by aging in the following ways: (1) stream segregation slows with aging, (2) decrease in the efficiency of inhibition impairs the ability to reject incorrect lexical candidates, (3) decrease in cognitive resources can impair speech perception, and (4) age-related hearing loss distorts the perception of bottom-up signals.

First, stream segregation slows with aging [35]. Reduced WM capacity has been linked to limitations in inhibition [36]. This affects the ability to separate relevant speech from irrelevant background noise. For example, in a study by Janse [37], when speech was presented in background noise, poor inhibitory abilities led to greater interference by the competing noise, which impaired speech perception in older adults.

Second, an age-related decrease in the efficiency of inhibition [15,38] impairs the ability to reject incorrect lexical candidates as the context unfolds in time [39,40].

Third, age-related decreases in cognitive resources, specifically in EFs and WM [38], can impair speech perception, as suggested by the Framework for Understanding Effortful Listening [41]. The Framework for Understanding Effortful Listening is an adaptation of the capacity model of attention by Nobel laureate Daniel Kahneman, which conceptualizes the relationship between mental resource capacity and cognitive demands. According to this model, mental resources have limited capacity. The presence of background noise or another resource-consuming task (eg, driving) can impede and slow down speech processing for people with lower resource capacities.

Finally, age-related hearing loss distorts the perception of bottom-up signals, providing impoverished input to the central nervous system. To mitigate these effects, older adults rely heavily on the linguistic context in word recognition, often to an even greater degree than younger adults [34,42,43]. Efficient context processing depends on WM capacity and information processing speed [44]. As mentioned previously, these capabilities decline with age [28,45] and can affect older adults’ ability to use context during word recognition. Depleted WM capacity can also affect the ability to temporarily remember words from a given linguistic context for later use [46].

In summary, cognitive performance is intertwined with speech perception, especially in older age. Age-related difficulties in speech perception are not only affected by reduced cognitive abilities but can also accelerate the rate of cognitive decline. A total of 2 Lancet reports [15,47] on dementia prevention highlighted improving auditory and speech accessibility as the number one modifiable risk factor in middle to late life. In fact, the relative weight of speech accessibility in preventing dementia is estimated to be higher than in tackling smoking, diabetes, hypertension, and obesity altogether. As Lin [48] suggests in his Aging and Cognitive Health Evaluation in Elders model [49], degraded speech processing affects cognitive resilience in aging by decreasing physical activities, social interactions, communication, and related brain functions. Hence, it is plausible to assume that training EFs should enable participants to juggle informational weight more gracefully and process speech faster, in turn, improving their quality of life and well-being.

To test the effect of cognitive training on speech processing, this study used eye tracking. We used a noninvasive infrared light source and high-precision camera that collects reflections from the eye and records the exact location of the eye gaze on the display at a rate of 500 samples per second. As the word unfolds in time, eye gaze data are time locked with what is being heard by the listener. By recording the participant’s eye movements in relation to the visual display and auditory stimuli, eye tracking provides a highly sensitive and continuous measure of spoken word processing. Unlike overt non–real-time responses (participant verbally or physically responding after the word has been heard), the covert rapidity of an eye movement allows one to determine the point in time at which the listener is able to isolate the target word from its competitors through the difference in fixations on the target and competitor over time. Although non–real-time responses, such as pointing at the screen, may be affected by age-related motor slowing, covert eye movement speed and accuracy are relatively unaffected [50].

To specifically gauge the cognitive mechanisms involved in speech processing under adverse conditions, our laboratory adapted the Visual World eye-tracking paradigm [51] to include a concurrent task (increasing memory load) and noise (increasing distractions), creating the E-WINDMIL [44]. In E-WINDMIL, listeners hear Hebrew sentences such as “point at the box” while viewing a visual display on a computer screen that contains 4 objects. In this example (Figure 1), the display shows a picture of the named object heard by the participant, box /ar.gaz/, along with three other objects: a phonological competitor (eg, an onset competitor that shares the first syllable with the target, rabbit /ar.nav/) and 2 additional objects that are neither semantically nor phonologically related to the heard target object or its name. Participants are asked to touch the picture of the object as quickly and accurately as possible while their eye gaze is recorded. Rather than analyzing the slower overt touch response, only the eye gaze is taken into account in later analysis.

As real-life speech processing is often accompanied by other tasks, before the onset of the spoken instructions, participants are also asked to retain in memory either 1 or 4 digits (low or high memory load, respectively) for later recall. A discrimination score, which is the difference between the proportion of eye gaze fixations to the target (image representing the heard word) and the phonological competitor, was used to assess the 2 groups. The higher the difference, the more efficient the listener is in discriminating the spoken target from its signal-sharing competitor. Using the same eye-tracking paradigm, Nitsan et al [52,53] showed that listeners with larger WM capacity were able to identify the target word (and reject the signal-sharing competitor) earlier than a matched group with lower capacity. These findings suggest that improving one’s cognitive capacity might improve speech processing in adverse conditions.

A total of 2 groups of older adults were tested twice on the E-WINDMIL speech processing task. One group received no cognitive training, whereas the other group followed the Effectivate SGCTA training protocol for 6 weeks. We aimed to test whether a short training period using Effectivate would engender a significant far-transfer change in speech processing ability. As the tested SGCTA does not involve any type of auditory training, improved performance on the E-WINDMIL speech processing task would provide strong support for the far-transfer effect and demonstrate the use of far-transfer measures in gauging training success for the validation of SGCTAs.

We hypothesized that if the tested SGCTA, Effectivate, improves generalized EFs, speech processing in adverse conditions, as measured by performance on the E-WINDMIL, would improve for the training group. Specifically, the difference in discrimination scores between the training group and control group would not be significant in the first test session, although an advantage for the training group would be found in the second test session (after training). This would suggest that training had a significant impact on real-time speech processing, above and beyond practice with the E-WINDMIL task.

A total of 54 older adults were recruited via phone calls from the Reichman University’s older adult research volunteer group and randomly assigned to either the cognitive training or control group. Although the groups cannot be said to reflect the diversity of the global older adult population, they are representative of the population residing in central Israel, where the study was conducted. Of these 54 individuals, 8 (15%) did not return for the second eye-tracking session, and 6 (11%) were excluded because of failure in eye movement recording or loss of eye-tracking signal. Recruitment was continuous for the duration of 6 months. Owing to the COVID-19 pandemic, participant recruitment and data collection were limited and terminated earlier than expected. The training group comprised 50% (20/40) of older adults (mean age 65.65, SD 4.8 years; 14/20, 70% were women). The control group comprised 50% (20/40) of older adults (mean age 69.05, SD 3.8 years; 13/20, 65% were women) from the study by Baharav et al [54]. All participants met the research inclusion criteria (refer to Textbox 1 for details). As shown in Table 1, the 2 groups had similar gender distribution (P=.74). Hearing acuity (pure tone average), years of education, and forward digit span scores did not differ significantly between the 2 groups (P=.51, P=.74, and P=.76, respectively). However, participants in the training group were slightly younger (t₃₈=2.48; P=.02). All the participants provided written informed consent.

Ethics approval for this study was obtained from the Reichman University (Interdisciplinary Center Herzliya) institutional review board (P_1920119). This study was conducted in line with the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) checklist (Multimedia Appendix 1).

Table 2 presents the accuracy percentage for each experimental condition—the percentage of trials in which participants both correctly selected the corresponding object on the visual display (indicating correct spoken word recognition) and correctly recalled the preload digits (indicating correct digit recall). A Mann-Whitney independent-sample nonparametric test confirmed that WM load, test session, and participant group did not have significant effects on accuracy, with P>.17 for all 4 tests.

We analyzed target discrimination scores (following the methodology of previous studies [60-63]) reflecting the listeners’ ability to discriminate the target word from its phonological competitor. The proportion of fixations on the competitor was subtracted from the proportion of fixations on the target within 250-millisecond time bins, starting from 250 milliseconds after the word onset to 1500 milliseconds. In this measure, the higher the value, the better listeners can discriminate the target from its competitor; values approaching 0 reflect an inability to discriminate between the target and competitor objects. Mixed-design repeated-measures ANOVAs were conducted for each 250-millisecond time bin, with three within-participants factors—WM load (high and low), test session (first and second), and condition (onset vs offset sound sharing)—and one between-participant factor—participant group (training and control). In each analysis, planned comparisons compared the effect of the participant group on discrimination scores in the first and second test sessions to verify whether differences between groups were related to the intervention (ie, significant effect only in the second session). Significant interactions of the test session with the participant group were noted in two of the five tested time bins: early processing 250 to 500 milliseconds and late processing 1250 to 1500 milliseconds, as discussed in the following section (Figures 3 and 4). The remaining three time bins (500-750 milliseconds, 750-1000 milliseconds, and 1000-1250 milliseconds) did not show any significant interaction; thus, they will not be discussed further.

The interaction between the test session and the participant group was found to be approaching significance (F_1,38=3.881; P=.06; η_p²=0.093). Planned comparison indicated no significant difference between the 2 groups in the first session (F_1,38=0.056; P=.81), whereas the second session produced higher discrimination scores in the training group (F_1,38=7.371; P=.01; η_p²=0.162). This suggests that improved performance can be related to the intervention itself. The effect of the participant group was marginally significant (F_1,38=3.048; P=.07; η_p²=0.085), with slightly higher discrimination scores in the training group (as in the previous analysis, this difference can be related to the second session) and no significant main effect for test session (F_1,38=0.224; P=.64). No significant triple interactions were found for the participant group, test session, or any other tested variables (WM load or condition).

A significant interaction between the test session and the participant group was found (F_1,38=4.220; P<.05; η_p²=0.100). Planned comparisons indicated that although the 2 groups did not significantly differ in the first session (F_1,38=1.689; P=.20), the second session showed higher discrimination scores in the training group than in the control group (F_1,38=9.003; P=.005; η_p²=0.192), suggesting that improved performance could be related to the intervention itself. A significant main effect of the participant group was noted (F_1,38=6.722; P=.01; η_p²=0.150), with higher discrimination scores in the training group (emanating from higher scores in the second session) and no significant main effect for the test session (F_1,38=0.108; P=.74). No significant triple interactions were found for the participant group, test session, or any other tested variables (WM load or condition).

In summary, in early and late processing (250-500 milliseconds and 1250-1500 milliseconds after word onset, respectively), performance did not differ between the 2 groups in the first test session. However, in the second test session after training with the Effectivate SGCTA, the training group surpassed the control group. These effects can be taken to suggest that the SGCTA training improved performance, over and above any effect of test-retest repetition.

In this exploratory study, we aimed to validate an eye-tracking paradigm, the E-WINDMIL, which tests real-time speech processing in adverse conditions as a gauge for the far-transfer efficacy of SGCTAs. Specifically, we tested whether training EFs in the visual modality with the SGCTA Effectivate generalizes to improved speech processing in adverse listening conditions (auditory modality) for older adults.

The training group, with 50% (20/40) of the older adults, was tested before and after 6 weeks of training on the SGCTA Effectivate. The control group, with another 50% (20/40) of the older adults, did not undergo any specific cognitive training. Before training, no significant differences in E-WINDMIL performance were noted between the control group and the training group. However, after training with Effectivate, the training group outperformed the control group in early word processing (indicated by eye movements, 250-500 milliseconds after word onset) and late word processing (1250-1500 milliseconds after word onset). The early processing advantage may suggest improved stream segregation between the spoken target word and noise [52,53], when WM was otherwise occupied. The late processing advantage alludes to improved decision-making processes (ie, using accumulated evidence) once the word had been completely heard [55]. Our results provide early support for the efficacy of the E-WINDMIL speech processing paradigm as a far-transfer measure of cognitive training with the SGCTA Effectivate. This is of special interest as the tested SGCTA did not train any auditory task or spoken language processing.

Challenges in determining the effectiveness of any cognitive intervention stem from the ongoing debate: Do we use near-transfer or far-transfer metrics? [14] In other words, is it sufficient to indicate improved performance on the trained task or should research indicate improved performance on a daily task, far from training, to suggest the generalizability of training? This exploratory study demonstrates the efficacy of using a far-transfer measure that involves speech processing in adverse conditions to discern the impact EF training has on daily life activities.

Speech processing in adverse conditions presents an excellent gauge of the generalizability of cognitive training. As speech processing is resource demanding, the fewer resources listeners have, the more they will be affected by adverse conditions such as background noise. Speech processing involves holding ongoing speech strings in memory and integrating words and phrases to create coherent meaning; thus, it is considered to be dependent on WM and other attentional resources [52,56]. The results of this study suggest the prowess of training to create a generalized cognitive effect, as a few weeks of training on the Effectivate SGCTA was sufficient to improve speech processing in adverse conditions (above and beyond test-retest learning effects).

This improvement can be interpreted in light of the crucial role of EFs, especially WM, in speech processing in adverse conditions. According to the Ease of Language Understanding model [44], explicit WM resources are drawn from a central pool to compensate for the loss of automatic matching between the input and lexical representations when the sound input is degraded by adverse listening conditions. Other studies have demonstrated a direct link between WM capacity and the ability to inhibit irrelevant information. This ability is necessary to separate the speech signal from background noise and reject competing words in the mental lexicon. Thus, our results suggest that training EFs using SGCTAs might have a generalized effect on real-life daily tasks. Returning to our example in the introduction, with improved WM capacity, the older adult will be better able to understand his grandson saying, “Grandpa, have you seen the DOLL?” rather than DOG (sound-sharing alternative) while driving a car (WM load) with the radio playing (adverse listening conditions) in the following ways:

Given the pivotal role of speech processing in successful aging [64], this change may have a lasting positive impact on the quality of life in older age.

The advantage of using the adapted visual world paradigm E-WINDMIL lies in its increased ecological validity—measuring a daily task (speech processing) that is important to the perseverance of well-being and performance in older age [17-19]. Eye tracking used by E-WINDMIL is better suited to test older adults’ speech processing than more traditional speech tests involving overt responses, such as verbal or keypress response. It is not influenced by an age-related slowing of motor speed, which often affects non–real-time speech tests [65]. Unlike many other speech processing tasks that assess the processing of a single word in ideal listening conditions, the E-WINDMIL asks listeners to retain digits for later recall (a task designed to weigh on WM resources) while presenting speech in noise. In this way, the E-WINDMIL paradigm acknowledges that speech in real-life scenarios is often experienced along with noise while the listener is engaged in other cognitively demanding tasks (eg, following the context of the sentence as it unfolds and driving). Moreover, eye tracking has been shown to be a sensitive measure for speech processing in various studies, suggesting that speech processing is costly in terms of WM processing and perhaps even mediated by it [29,34,56,66,67].

Previous adaptations of E-WINDMIL found eye tracking to be very sensitive to differences in cognitive reserve. Hadar et al [56] found that minimizing available cognitive resources can slow down processing in this task. Nitsan et al [52,53] found that individuals with higher cognitive reserve outperform individuals with lower reserve while using E-WINDMIL. This advantage, attributed by the authors to the use of cognitive resources for speech processing, was indicated in the later time bins—similar to the current findings. Other studies also found that differences in cognition were indicated in later word processing with older listeners in particular [61,68]. A recent study by Harel-Arbeli et al [46] attributed the advantages seen in later time bins to decision-making processes. In their study, using a similar eye-tracking paradigm, the spoken target word was preceded by a spoken predictive context presented in a quiet environment. An advantage of young adults over older adults, based on the age-related difference in cognitive resources, was present mainly in the late time bin when the full word had been spoken. Taken together, it appears that improved processing in the late time bin may reflect improved cognitive resources (eg, WM and inhibition).

The current data also indicated training-related advantages in processing during early time bins, when only the first phoneme of the word is being processed. This suggests that cognitive training improved target word stream formation and auditory stream segregation between the target word and noise [55,69]. Indeed, this early process of stream segregation has been linked not only to sensory processes but also to the deployment of cognitive resources. Cognition is necessary for the inhibition of the noise stream and selective focus on the target word stream, leading to stream segregation [70,71]. Stream segregation is essential for speech processing and represents one of the major hurdles for older adults in social interactions [72]. Indeed, age-related auditory sensory degradation can specifically impair processes related to stream segregation in aging [73]. This early time bin training advantage may also be related to the early benefits noted in the literature as a result of removing background noise [60] and increasing the lexical frequency of the spoken word [74] using similar eye-tracking paradigms. In summary, the performance advantage in the early time bin associated with SGCTA training may reflect an increase in cognitive reserve.

This study should be taken as a first step in supporting the effectiveness of the tested SGCTA, and it does not serve as a recommendation or suggestion to use SGCTAs in general or specifically the Effectivate SGCTA. This study was ongoing at the beginning of the COVID-19 pandemic and was halted because of national quarantine. Therefore, we were unable to amass a larger group of participants. Moreover, we were unable to recruit an active control group to undergo an alternative form of cognitive training. Future studies should attempt to replicate the results with an active control to ensure that the observed effects were not related to possible social desirability or lack of participant blinding but to the specific cognitive training, Effectivate. However, we note that the experimenters administering the study were blinded to the condition, and the experimental tool was relatively immune to social desirability. Such replications should also more carefully match participants across all groups. Indeed, on average, participants in the control group were older by a few years than those in the training group. We also note that participants in this study did not form a representative sample of the older adult population, specifically given the cognitive and linguistic inclusion criteria. Although future studies should aim to include more diverse samples, these criteria are common in research with this population [75-77].

We demonstrated that the Effectivate SGCTA is sufficiently powerful to induce changes, even in cognitively healthy older adults, and that the E-WINDMIL test is sufficiently sensitive to detect such changes. Our preliminary results are the first step, suggesting the ability of the SGCTA Effectivate to engender far cognitive transfer. Future studies should also try to relate our results to other more traditional cognitive measures and questionnaires tapping users’ subjective evaluation of their quality of life.

This exploratory study presents an early foray into the potential of speech processing in adverse conditions as a far-transfer gauge of SGCTAs. This is in line with previous studies that used gamification in cognitive decline research [78-81]. Results present a preliminary indicator of the SGCTA Effectivate’s potential to engender such far transfer from visual cognitive training to auditory speech processing after only a few weeks of training. Following training, older adults were better able to differentiate between the spoken target word and its sound-sharing competitor under adverse conditions (noise and digit memorization). We suggest that this change in performance represents a real-world improvement in a daily task that is directly related to successful aging. Thus, it shows the potential of the training to have a significant impact on the user’s daily life. We advocate that cognitive training should showcase evidence-based improvement in daily far-transfer tasks that can change the user’s quality of life, as opposed to merely showing changes in traditional pen-and-paper cognitive measures. As serious games are a means of improving performance in other tasks, games developed to the highest standards should seek out far-transfer validation methods. We hope that the increased demand for far-transfer metrics will bolster research efforts within the academic community to develop new far-transfer gauges of cognitive ability and call on serious game developers to adopt far-transfer metrics, such as E-WINDMIL, into their gauges for validity and success.

This study investigated aging through the lens of speech processing, a novel vantage point, which can illuminate interconnected attentional mechanisms known to be affected by aging. Most importantly, speech processing is an essential daily task performed across social interactions, leisure, and employment [72]. Impaired speech processing may have severe implications for older adults across all aspects of life. Therefore, we encourage adding tests of speech processing, especially in adverse conditions, to the arsenal of tools used to test the efficacy of EF training in aging. Furthermore, we suggest paying attention to speech processing in aging when considering accessibility and inclusion in serious game design.

In addition to being a novel and important test metric for aging, real-time speech processing metrics may also prove beneficial to testing other populations such as children with the neurodevelopmental disorder, attention-deficit/hyperactivity disorder (ADHD). As the most prevalent neurodevelopmental disorder in children, ADHD is associated with lifelong impairment, with symptoms reflecting a deficit in EFs such as inhibitory control, attentional regulation, and WM [82,83]. Given ADHD’s high prevalence and detrimental effect on the quality of life and well-being, many serious games are being developed to train EFs in ADHD. As is the case with SGCTAs, there is much debate in the literature regarding their efficacy [84]. Expanding on our findings, we suggest further exploration using E-WINDMIL to test the far-transfer efficacy of serious computerized games designed for children and adults with ADHD along with other promising populations that could benefit. We hope that the creation and use of universally accepted far-transfer metrics will determine gold standard serious games that will help us prolong cognitive functions and improve well-being with age and throughout life.