This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.
Mild cognitive impairment (MCI), the intermediate cognitive status between normal cognitive decline and pathological decline, is an important clinical construct for signaling possible prodromes of dementia. However, this condition is underdiagnosed. To assist monitoring and screening, digital biomarkers derived from commercial off-the-shelf video games may be of interest. These games maintain player engagement over a longer period of time and support longitudinal measurements of cognitive performance.
This paper aims to explore how the player actions of Klondike Solitaire relate to cognitive functions and to what extent the digital biomarkers derived from these player actions are indicative of MCI.
First, 11 experts in the domain of cognitive impairments were asked to correlate 21 player actions to 11 cognitive functions. Expert agreement was verified through intraclass correlation, based on a 2-way, fully crossed design with type consistency. On the basis of these player actions, 23 potential digital biomarkers of performance for Klondike Solitaire were defined. Next, 23 healthy participants and 23 participants living with MCI were asked to play 3 rounds of Klondike Solitaire, which took 17 minutes on average to complete. A generalized linear mixed model analysis was conducted to explore the differences in digital biomarkers between the healthy participants and those living with MCI, while controlling for age, tablet experience, and Klondike Solitaire experience.
All intraclass correlations for player actions and cognitive functions scored higher than 0.75, indicating good to excellent reliability. Furthermore, all player actions had, according to the experts, at least one cognitive function that was on average moderately to strongly correlated to a cognitive function. Of the 23 potential digital biomarkers, 12 (52%) were revealed by the generalized linear mixed model analysis to have sizeable effects and significance levels. The analysis indicates sensitivity of the derived digital biomarkers to MCI.
Commercial off-the-shelf games such as digital card games show potential as a complementary tool for screening and monitoring cognition.
ClinicalTrials.gov NCT02971124; https://clinicaltrials.gov/ct2/show/NCT02971124
Mild cognitive impairment (MCI) is a clinical entity defined as a transitional state between normal and pathological aging, where one or more cognitive domains are significantly impaired, yet activities of daily living are still preserved [
As such, digital games may help by providing
Today, the focal point of research assessing cognitive performance is
This study aims to explore the possibility of using COTS card games to screen cognition among patients living with MCI. This study involved 11 experts in the domain of MCI coming together to craft 23 candidate digital biomarkers from the digital card game Klondike Solitaire. Subsequently, a data acquisition campaign was set up involving 46 participants: 23 (50%) healthy older adults and 23 (50%) older adults with MCI. The participants were asked to play 3 games on a tablet. We examined the game data for differences at a group level for the candidate digital biomarkers using a generalized linear mixed model (GLMM) analysis. The results show that of the 23 candidate digital biomarkers, 12 (52%) differed significantly between both groups, taking age, tablet experience, and Klondike Solitaire experience into account. By providing a methodological approach and an exploratory study for crafting digital biomarkers, as well as articulating the rationale and the different steps taken, we hope to inform future researchers who aim to leverage the use of COTS video games to yield digital biomarkers.
Persons diagnosed with MCI show a deficit in cognition in at least one cognitive domain that cannot be attributed to age or any other disease; yet, they do not fulfill the diagnosis of dementia [
Typically, the process leading to a diagnosis of MCI is set into motion by a cognitive complaint from the older adult, relative, or (informal) caregiver, followed by a presumptive identification through a screening test. The cognitive screening tests used most often to detect MCI are the Montreal Cognitive Assessment (MoCA) [
Therefore, this presumptive identification is in turn followed by an elaborate neuropsychological assessment (ie, a cognitive test battery) and possibly a biomarker scan or a neuroimaging scan [
Serious (digital) games are “games that do not have entertainment, enjoyment, or fun as their primary purpose [
Serious games may provide certain advantages for the assessment of cognitive performance
Although serious games show promising results and have merit for both patient and physician, serious games are at risk of being dismissed as “chocolate-covered broccoli”: neuropsychological tests embellished with a thin layer of gameplay [
This lack of sustained engagement contrasts with surveys on gameplay among older adults. A large-scale (N=3737) survey of older adults’ attitude toward video games, conducted in 2019 by the American Association of Retired Persons [
COTS games may have the ability to retain players over a longer period and to support
Recent studies on using COTS games to measure cognitive impairment have generated promising results. The study by Jimison et al [
Across the aforementioned studies, different lines of reasoning have been given to justify the game of choice as suitable for neuropsychological evaluation. The study by Grabbe [
One of the most popular card games among older adults is Klondike Solitaire, also known as Patience, Fascination, or even just Solitaire [
This popular card game is played with a standard 52-card deck, with 28 (54%) cards dealt in 7 build stacks and the other 24 (46%) cards placed in a pile, as can be seen in
Klondike Solitaire. The seven build stacks can be seen at the bottom, the suit stacks are at the top left. The pile of undealt cards can be seen in the top right.
Given the popularity of Klondike Solitaire among the older population, and given the need for engaging, ecologically valid, scalable tools to assist in the screening and monitoring of MCI, this paper aims to investigate the feasibility of Klondike Solitaire game sessions to yield digital biomarkers of MCI. To this end, the study comprised the following investigations:
an exploration of the digital biomarkers of cognitive performance, based on the player actions (PAs) of Klondike Solitaire and an evaluation of the candidate digital biomarkers captured in Klondike Solitaire to measure the differences between healthy older adults and older adults living with MCI.
To explore the potential digital biomarkers of cognitive performance in Klondike Solitaire, we first conducted an expert consensus study, involving 11 experts. In this first part of the paper, we discuss the 3 steps taken to compile a final list of 23 candidate digital biomarkers.
To transform gameplay into PAs, a methodical approach was applied. In all, 4 researchers in the field of human-computer interaction carried out the following tasks. First, the literature on Klondike Solitaire was studied, ranging from scientific work [
These game events were then converted to PAs; they were described as an action of the player rather than as an event of the game. Next, all these PAs were transformed into their negative equivalents, for example,
Average of the experts’ ratings for each player action and cognitive function.
Player actions | Mental flexibility, mean (SD) | Inhibitory control, mean (SD) | Working memory, mean (SD) | Selective attention, mean (SD) | Visuospatial ability, mean (SD) | Object recognition, mean (SD) | Apraxia, mean (SD) | Cognitive planning, mean (SD) | Processing speed, mean (SD) |
PAa 1. Player takes a lot of time to think of a move. | 1.64 (1.12) | 0.73 (1.01) | 1.82 (0.4) | 1.55 (0.82) | 1.18 (0.98) | 1.27 (0.79) | 0.27 (0.47) | ||
PA 2. Player takes a lot of time to move the card. | 0.73 (1.01) | 0.73 (1.1) | 0.64 (0.5) | 0.64 (0.67) | 1.45 (1.04) | 0.64 (0.67) | 1.64 (0.92) | 0.91 (0.83) | |
PA 3. Player does not move a suitable card from the talon to the suit stack. | 0.73 (0.9) | 1.45 (0.93) | 1.64 (0.67) | 0.73 (1.01) | 0.91 (1.04) | ||||
PA 4. Player does not move a suitable card from the build stack to the suit stack. | 1.82 (0.98) | 0.91 (0.83) | 1.82 (0.75) | 1.36 (0.92) | 1.36 (0.92) | 0.27 (0.47) | 1.73 (0.79) | 1 (1) | |
PA 5. Player does not move a suitable card from the talon to the build stack. | 1.27 (0.9) | 1.91 (0.7) | 1.55 (1.04) | 1.73 (0.79) | 0.18 (0.4) | 0.82 (1.08) | |||
PA 6. Player does not move a suitable card from 1 build stack to another build stack. | 1.09 (0.94) | 1.45 (0.93) | 1.64 (0.92) | 0.18 (0.4) | 0.73 (1.01) | ||||
PA 7. Player does not place an ace immediately on an empty suit stack. | 1.27 (1.01) | 0.73 (1.01) | 1 (0.77) | 1.18 (1.17) | 0.45 (0.69) | 1.09 (1.04) | |||
PA 8. Player does not place a king on an empty build stack. | 1.45 (1.13) | 0.73 (0.9) | 1 (0.77) | 1.55 (1.13) | 0.36 (0.67) | 1 (0.89) | |||
PA 9. Player moves cards without benefit (eg, moving a jack from 1 queen to another). | 1.45 (1.04) | 1.82 (0.98) | 1.64 (1.03) | 0.82 (0.75) | 1.45 (1.21) | 0.18 (0.4) | 0.45 (0.52) | ||
PA 10. Player flips a lot through the pile. | 1.73 (1.01) | 1.82 (1.08) | 1 (0.89) | 1.45 (1.13) | 1 (1) | 0.91 (0.7) | |||
PA 11. Player moves a card onto a card with the same color. | 1.73 (1.1) | 1 (1) | 1.82 (1.17) | 0.27 (0.65) | 1.36 (0.81) | 0.45 (0.69) | |||
PA 12. Player moves a card to another card with the wrong number (eg, placing a 5 on a 10). | 1.18 (1.08) | 1.91 (0.94) | 1.09 (1.04) | 2.09 (0.94) | 0.45 (0.69) | 1.45 (0.93) | 0.36 (0.5) | ||
PA 13. Player selects the cards with a very bad precision (taps on edge of card or next to the card). | 0.45 (0.69) | 0.73 (0.79) | 0.27 (0.47) | 0.64 (0.81) | 0.82 (0.75) | 0.45 (0.82) | 0.45 (0.69) | ||
PA 14. Player starts tapping on the playfield with no apparent target (with short intervals, fidget tapping). | 0.73 (0.79) | 0.27 (0.47) | 0.82 (0.87) | 0.73 (0.9) | 0.45 (0.52) | 1.55 (1.29) | 0.91 (1.04) | 0.73 (0.9) | |
PA 15. Player presses the undo button a lot. | 1.82 (0.6) | 1.73 (1.1) | 1.36 (1.12) | 0.64 (0.67) | 0.64 (0.67) | 0.73 (1.01) | 1.27 (1.1) | ||
PA 16. Player requests a lot of hints. | 1.91 (1.04) | 1.73 (1.01) | 1.45 (0.93) | 0.64 (0.81) | 1 (0.77) | 0.45 (0.69) | 1.18 (0.75) | ||
PA 17. Player takes a very long time to finish games. | 1 (1.34) | 1.64 (1.21) | 1.09 (0.83) | 1.18 (0.98) | 0.91 (0.83) | ||||
PA 18. Player does not have a high score in the game. | 1.91 (1.04) | 1.45 (0.93) | 1.36 (0.92) | 0.91 (0.94) | 1.55 (1.04) | ||||
PA 19. Player does not win a lot of games (low win ratio). | 1.82 (1.08) | 2 (1) | 1.36 (0.92) | 1.18 (0.87) | 1 (0.89) | 1.64 (0.81) | |||
PA 20. Player’s scores in different games vary greatly. | 1.64 (1.12) | 0.73 (0.9) | 0.73 (0.9) | 0.64 (0.92) | 1.82 (1.08) | ||||
PA 21. Player’s win ratio decreases rapidly as the game’s level of difficulty increases. | 1.91 (0.94) | 1.18 (0.87) | 1.09 (0.94) | 0.82 (0.87) | 1.64 (1.03) |
aPA: player action.
bValues scoring moderately strong are in italics.
A set of cognitive functions was drafted in 5 phases (
The 5 phases through which the cognitive functions were defined. CDR: Clinical Dementia Rating, MoCA: Montreal Cognitive Assessment, MMSE: Mini-Mental State Examination.
In the next step, the experts were asked to rate the extent to which each PA was related to a specific cognitive function.
These experts were recruited from 2 leading memory clinics in Belgium using a snowball sampling method. Of the 11 experts, 3 (27%) held Doctor of Medicine degrees and were experienced in treating cognitive decline, whereas the other 8 (73%) were neuropsychologists; 7 (64%) were women; and 4 (36%) were men. The average age of the experts was 45 (SD 13.3) years, and their average working experience was 20 (SD 14) years. In all, 3 coauthors of this paper (LVA, PD, and FFB) also participated as experts. None of the experts were compensated for participating in the study.
Before they began the rating process, the experts received a standardized introduction comprising a video that explained all concepts of the game [
Next, each expert received a coding sheet in which they could map the 21 PAs to the 9 cognitive functions. Each cell had to be filled in according to the following 4-point scale:
0: This cognitive function has no significant correlation to the PA. 1: This cognitive function correlates weakly to the PA. 2: This cognitive function correlates moderately to the PA. 3: This cognitive function correlates strongly to the PA.
Finally, they were also given the choice to explain their train of thought in an optional clarification column.
The intraclass correlation (ICC) for each PA as variables of interest with cognitive functions was computed. In addition, we computed ICCs for each of the cognitive functions as variables of interest, with all PAs considered as observations [
We found that the ICCs for all PAs scored higher than 0.75, suggesting good to excellent reliability according to the study by Koo et al [
An overview of the associations between individual PAs and cognitive functions, according to the expert mapping, is presented in
These PAs were captured through the game as potential digital biomarkers, that is, measurable factors of the game, such as score, duration of the game, and detailed moves. These candidate digital biomarkers were enriched with additional information about the game. This contextualization is important to ensure an unambiguous interpretation of the cognitive information derived from the gameplay. For example, whereas a game played with many moves on the pile may indicate that a player progressed in the game, it may indicate equally that the player did not realize that they were stuck. By calculating the percentage of pile moves by dividing the number of pile moves by the total number of moves, a more informative metric can be obtained. In this manner, 23 potential digital biomarkers were defined that we further classified into 1 of 6 categories: time-based, performance-based, error-based, execution-based, auxiliary-based, and result-based. Time-based digital biomarkers are biomarkers related to the speed of PAs. Performance-based digital biomarkers are biomarkers related to optimal gameplay (ie, if the game was played according to strategies that ensure optimal performance). Error-based digital biomarkers relate to making incorrect moves according to the rules of Klondike Solitaire. Auxiliary-based digital biomarkers are interactions that are not part of the core gameplay (ie, requesting undo moves and hints). Execution-based digital biomarkers relate to the accuracy in moving cards and the presence of accidental taps. Finally, result-based digital biomarkers are biomarkers that evaluate the final outcome of the game (eg, how far did the participant get in the game). An overview of all digital biomarkers and their contextualizations is presented in
Digital biomarkers related to the player actions in Klondike Solitairea.
Related PAb | Digital biomarker | Description | Contextualization | Value | |||||
|
|||||||||
|
PA 1 | Think time | Time spent thinking of a move. Defined as the time necessary to find and touch a suitable card | Average (SD) | Number (ms) | ||||
|
PA 2 | Move time | Time spent moving cards. Defined as the time necessary to move a suitable card to the destination | Average (SD) | Number (ms) | ||||
|
PA 1, PA 2 | Total time | Total time to make a move. Defined as the combination of think time and move time | Average (SD) | Number (ms) | ||||
|
|||||||||
|
PA 3 | Final β error | Whether there were still moves possible when quitting a game | None | Boolean | ||||
|
PA 3, PA 4, PA 5, PA 6 | β error | Number of pile moves made with moves remaining on the board divided by the total number of pile moves | Percentage | 0%-100% | ||||
|
PA 7 | Ace β error | Number of missed opportunities to place an ace on the suit stacks divided by the total number of game moves | Percentage | 0%-100% | ||||
|
PA 8 | King β error | Number of missed opportunities to place a king on an empty spot divided by the total number of game moves | Percentage | 0%-100% | ||||
|
PA 10 | Pile move | Number of pile moves divided by the total number of board moves | Percentage | 0%-100% | ||||
|
|||||||||
|
PA 11, PA 12 | Successful move | Number of successful moves divided by the total number of board moves | Percentage | 0%-100% | ||||
|
PA 11, PA 12 | Erroneous move | Number of erroneous moves divided by the total number of board moves | Percentage | 0%-100% | ||||
|
|||||||||
|
PA 13 | Accuracy | Accurateness of selecting a card, defined by how close to the center a card was touched | Average (SD) | 0%-100% | ||||
|
PA 14 | Taps | Number of actuations on nonuser interface elements | None | Number | ||||
|
|||||||||
|
PA 15 | Undo move | Number of undo moves requested. | Percentage | 0%-100% | ||||
|
PA 16 | Hint move | Number of hints requested | Percentage | 0%-100% | ||||
|
|||||||||
|
PA 17 | Game time | Total time spent playing a game | None | Number (ms) | ||||
|
PA 18 | Score | Final score of a game | None | Number | ||||
|
PA 19 | Solved | Whether the game was completed. Indicator of how successfully the game was played | None | Boolean | ||||
|
PA 19 | Cards moved | Number of cards selected for each move. An additional indicator of how successfully the game was played; as games of Klondike Solitaire progress, longer stacks of cards are moved | Average (SD) | Number |
aPlayer actions (PAs) 20 and 21 were not captured because the single-point-in-time setup would not allow a comparison of scores and win ratios with ranging difficulty over time. In addition, PA 9 was not tested because the current software would not allow for detection of these moves.
bPA: player action.
The aim of this second study is to explore the potential of these candidate digital biomarkers of cognitive performance. Relying on 46 participants, we captured data and performed a GLMM analysis to examine the differences between healthy participants and those diagnosed with MCI.
In total, 46 participants—23 (50%) healthy participants and 23 (50%) with MCI—were enrolled. The older adults with MCI were recruited by 2 leading memory clinics in Belgium. Healthy participants were recruited from multiple senior citizen organizations, using a snowball sampling method. All healthy participants were aged 65 years or older, were fluent in written and verbal Dutch, had 20/20 (corrected) vision, no motor impairments, and lived independently or semi-independently at home, in a service flat, or at a care home. The exclusion criteria for healthy participants were subjective-memory concerns expressed by the participant, caretaker, or clinician. In addition, they were screened using the MMSE, MoCA, and CDR Scale. To minimize the risk of including potential individuals with MCI among the healthy participants, cutoff scores of 27 on the MMSE, 26 on the MoCA, and 0 on the CDR Scale were enforced. The participants living with MCI had been formally diagnosed with multiple-domain aMCI by 1 of the 2 collaborating memory clinics, based on the diagnostic criteria described in the study by Petersen [
Demographic and neuropsychological data (N=46).
|
Healthy participants (n=23) | Participants diagnosed with MCIa (n=23) | |||
Age (years), mean (SD) | 70 (5.4) | 80 (5.2) | |||
|
|||||
|
Levels 1-2 | 5 (22) | 4 (17) | ||
|
Levels 3-4 | 7 (30) | 13 (57) | ||
|
Levels 5-6 | 11 (48) | 6 (26) | ||
|
|||||
|
Female | 11 (47) | 13 (57) | ||
|
Male | 12 (53) | 10 (43) | ||
|
|||||
|
Daily | 12 (52) | 3 (13) | ||
|
Weekly | 2 (9) | 2 (9) | ||
|
Monthly | 0 (0) | 2 (9) | ||
|
Yearly or less | 2 (9) | 1 (4) | ||
|
Never | 7 (30) | 15 (65) | ||
|
|||||
|
Daily | 3 (13) | 7 (30) | ||
|
Weekly | 6 (26) | 8 (35) | ||
|
Monthly | 3 (13) | 2 (9) | ||
|
Yearly or less | 11 (47) | 6 (26) | ||
|
Never | 0 (0) | 0 (0) | ||
MMSEc score, mean (SD) | 29.61 (0.65) | 26.17 (1.75) | |||
MoCAd score, mean (SD) | 28.09 (1.28) | N/Ae | |||
CDRf Scale score, mean (SD) | 0 (0) | N/A |
aMCI: mild cognitive impairment.
bISCED: International Standard Classification of Education.
cMMSE: Mini-Mental State Examination.
dMoCA: Montreal Cognitive Assessment.
eN/A: not applicable.
fCDR: Clinical Dementia Rating.
All game sessions were completed on a Lenovo Tab3 10 Business tablet (Lenovo Group Ltd) running Android 6.0 (Google LLC). A Solitaire app created by Bielefeld [
Each observation was made between 9 AM and 5 PM in the home environment of the participant to ensure a familiar and comfortable environment. Each observation took between 2 and 3 hours and consisted of 2 main parts:
a game session where game-based digital biomarkers of Klondike Solitaire were collected on a tablet and a neuropsychological examination where cognitive information was collected.
Each game session started with a standardized 5-minute introduction during which the tablet, the game mechanics, and possible touch interactions were explained. This was followed by a practice game, identical for all participants, where questions to the researcher were allowed and the participant could get used to the touch controls. Data from this practice game were not used for analysis. After this practice game, each participant played 3 games in succession. The order and games were identical across all participants. All games were purposefully chosen through prior playtesting, in that they were solvable and varied in difficulty level. During these 3 games, no questions were allowed, and gameplay continued until the participants either finished the game or indicated that they deemed further progress impossible. All game sessions and cognitive evaluations were conducted by the same researcher to avoid differences arising from researcher bias.
This study is in accordance with the declaration of Helsinki and General Data Protection Regulation compliant. Ethical approval was provided by the ethics committee of UZ/KU Leuven, Belgium (CTC S59650). Because of the fragile nature of our participants’ health, utmost care was taken when providing information to them about the game sessions. The tests were conducted only after we received written informed consent.
To assess the difference between the healthy participants and those diagnosed with MCI, a GLMM analysis was performed using R software (The R Foundation for Statistical Computing) [
Continuous digital biomarkers (eg, think time average) were modeled using a GLMM with the identity link function. Binary outcomes (eg, solved or not solved) were modeled using a GLMM with the logit link function. The significance of the effect of MCI was determined using the likelihood ratio test, which compares the model with a model without the effect of MCI, both estimated without restricted maximum likelihood [
The results of the GLMM analysis on the effect of MCI are presented below. A visualization of digital biomarker performance for all groups across all games is presented in
Performance on time-based digital biomarkers for both groups. MCI: mild cognitive impairment.
Generalized linear mixed model analysis results for each digital biomarker.
Digital biomarker | Value, constant (SD) | Value, β (SD) | Value, |
||||||
|
|||||||||
|
Think time average | –1371.778 (1415.444) | 1119.947 (405.815) | .006 | 0.416 (0.904) | ||||
|
Think time SD | –814.527 (1720.073) | 1112.533 (490.53) | .02 | 0.211 (0.655) | ||||
|
Move time average | –508.575 (373.89) | 156 (95.547) | .1 | 0.257 (0.579) | ||||
|
Move time SD | –856.605 (847.852) | 323.599 (202.032) | .1 | 0.137 (0.419) | ||||
|
Total time average | –912.419 (2149.177) | 1278.263 (573.839) | .02 | 0.318 (0.870) | ||||
|
Total time SD | 206.569 (2676.062) | 1315.598 (673.665) | .04 | 0.176 (0.715) | ||||
|
|||||||||
|
Final β error | –7.233 (4.131) | 0.435 (0.922) | .65 | 0.096 (0.068) | ||||
|
β error percentage | –7.203 (33.849) | 6.108 (6.879) | .36 | 0.089 (0.371) | ||||
|
Ace β error percentage | –0.132 (0.629) | 0.051 (0.137) | .73 | 0.023 (0.209) | ||||
|
King β error percentage | –3.682 (5.918) | 0.907 (1.323) | .48 | 0.028 (0.230) | ||||
|
Pile move percentage | 71.759 (24.052) | 13.333 (4.88) | .006 | 0.097 (0.513) | ||||
|
|||||||||
|
Successful move percentage | 87.486 (9.443) | –8.913 (3.595) | .02 | 0.104 (0.795) | ||||
|
Erroneous move percentage | 9.486 (6.529) | 3.624 (1.651) | .03 | 0.081 (0.466) | ||||
|
|||||||||
|
Accuracy average | 92.134 (9.167) | –3.817 (1.903) | .04 | 0.246 (0.805) | ||||
|
Accuracy SD | 4.519 (3.746) | 0.137 (0.772) | .85 | 0.056 (0.196) | ||||
|
Taps | –5.113 (10.704) | 5.334 (2.762) | .05 | 0.098 (0.500) | ||||
|
|||||||||
|
Undo move percentage | 0.228 (0.955) | 0.135 (0.205) | .49 | 0.008 (0.151) | ||||
|
Hint move percentage | –0.58 (0.87) | –0.311 (0.204) | .12 | 0.046 (0.491) | ||||
|
|||||||||
|
Gametime | –167427.7 (187325.5) | 93211.27 (53699.25) | .08 | 0.198 (0.690) | ||||
|
Score | 29.03 (1389.752) | –744.433 (286.576) | .009 | 0.105 (0.612) | ||||
|
Solved | –2.954 (4.578) | –2.63 (1.007) | .008 | 0.186 (0.152) | ||||
|
Cards moved average | 1.111 (0.262) | –0.119 (0.054) | .03 | 0.061 (0.093) | ||||
|
Cards moved SD | 0.135 (0.705) | –0.38 (0.147) | .009 | 0.072 (0.152) |
For time-based digital biomarkers (
For performance-based digital biomarkers (
Performance on performance-based digital biomarkers for both groups. MCI: mild cognitive impairment.
For error-based digital biomarkers (
Performance on error-based digital biomarkers for both groups. MCI: mild cognitive impairment.
For execution-based digital biomarkers (
Performance on execution-based digital biomarkers for both groups. MCI: mild cognitive impairment.
For result-based digital biomarkers (
Performance on result-based digital biomarkers for both groups. MCI: mild cognitive impairment.
With regard to auxiliary-based digital biomarkers (
Performance on auxiliary-based digital biomarkers for both groups. MCI: mild cognitive impairment.
MCI is a neurological disorder that is linked to an increased risk of developing dementia. As such, early detection of cognitive deterioration is essential for timely diagnosis and for allowing tailored care and treatment. Collecting digital biomarkers through COTS games may help by providing cognitive information through behavior traces of activities already integrated into the daily life of older adults. In this study, we investigated in particular whether Klondike Solitaire game sessions could yield digital biomarkers. In the paragraphs below, we discuss our findings and reflect on the different potential digital biomarkers, their relationship with cognitive functions, and the ethical implications of their use for cognitive assessment purposes.
Of the 23 candidate digital biomarkers, 12 (52%) differed significantly between older adults with MCI and a healthy control group. This supports the use of digital card games for monitoring cognitive performance and possibly detecting differences in cognitive performance caused by MCI.
Although the overall findings are promising, not all candidate biomarkers performed equally. In the case of time-based digital biomarkers, the biomarkers related to coming up with a move—think time average and think time SD—were significantly affected by MCI. In contrast, the biomarkers related to the actual physical movement of cards—move time average and move time SD—were not significantly affected. Total time average (
In the case of performance-based digital biomarkers, in contrast with expectations, none of the biomarkers related to β errors were proven to differ significantly. Upon rewatching gameplay, it became clear that there were two different types of β errors: strategic and unintentional. However, because of the current configuration of the app, it was impossible to discriminate between the two types. This is discussed further in the
Equally, the results indicated that participants with MCI made more mistakes because both error-based digital biomarkers (ie, successful move percentage and erroneous move percentage) were significant. In contrast, none of the auxiliary-based digital biomarkers differed significantly. Upon inspecting the data, it was noted that neither group consistently used these functionalities, which may have contributed to the lack of significance.
Finally, of the 5 digital biomarkers in the result-based category, 4 (80%) were significant, 3 (60%) with
In sum, our findings are in accordance with those of the study by Jimison et al [
In this study, the participants with MCI were diagnosed with multiple-domain aMCI based on the diagnostic criteria described in the study by Petersen [
COTS games also have their limitations. First, neuropsychological assessments are typically designed to assess a broad yet targeted spectrum of cognitive functions. Moreover, different tests are devised to measure 1 cognitive function in particular. COTS games, and more particularly digital card games, were found to be more limited in terms of the cognitive functions that they can specifically assess. When using COTS games, it may be hard to separate the evaluations of specific cognitive functions. In this study, experts judged every single PA to be moderately to strongly related to at least one cognitive function.
Second, using COTS games as an instrument to measure cognitive performance and possibly flag MCI necessitates ethical reflection. We envisioned that COTS games would be used only in accordance with the informed consent of the patient, with the positive aspiration that this could aid in the longitudinal monitoring of cognitive deterioration, more accurately measuring cognitive performance and variance. This project grew out of an ambition to escape the limitations of serious games and provide meaningful play to older adults. Nonetheless, we have to acknowledge that we may have transformed an activity previously considered enjoyable and innocent into an instrumental activity that may even trigger a sense of being under health surveillance [
Third, it has to be noted that deriving digital biomarkers from digital games may not be relevant for all older adults. Not everyone is an avid gamer, and even avid gamers may have preferences for different game genres. In addition, these preferences might change over time [
Finally, the interaction between health care professional and patient, often stimulating and motivating in and of itself, is crucial for full assessment. Hence, we argue that COTS games for screening and monitoring of MCI should not be used as a replacement for current neuropsychological examination but rather as a source of additional information.
In contrast with expectations, β error–related digital biomarkers proved to be insignificant. Upon inspecting the games of both groups, it became clear that there are two types of β errors: build stack β errors and suit stack β errors. The former represents missed moves among the build stacks. These errors were rarely made on purpose and occurred fewer times in the healthy participants’ group, based on observation. In contrast, the latter represent missed moves between build stacks and suit stacks. We observed that this latter category was used strategically to prevent the inability to place future cards. Our observations suggest that these occurred more often in the healthy participants’ group. However, because of the current configuration of the app, it was impossible to discriminate between these two types of β errors. Hence, this points to the importance of further contextualization and refinement of the measurement of β errors, and biomarkers in general, which should be addressed in future work.
An a priori power analysis [
In addition, because of the average age difference between the 2 groups, we chose a GLMM for our statistical analysis because it can factor in confounding effects. A side exploration included using trained machine learning models on the same data set to predict age instead of MCI. These models were found to be less performant than the ones modeling MCI, underscoring that the effect of MCI was greater than the effect of age in our data set. Nevertheless, it is a limitation that we have to acknowledge and take into account while interpreting the results.
This study provides insight into the cognitive functions addressed while playing digital card games and assesses the potential of digital card game sessions for screening for MCI. To this end, 11 experts in neuropsychology or geriatrics mapped the associations of PAs in Klondike Solitaire with cognitive functions. On the basis of this exercise, which showed that the experts agreed that PAs were related to cognitive functions, 23 potential digital biomarkers of cognitive performance were crafted. A GLMM analysis, taking the effects of age, tablet experience, and Klondike Solitaire experience into account, compared digital biomarker performance between a group consisting of people living with MCI and a healthy control group. We found that of the 23 digital biomarkers, 12 (52%) had a significant and sizeable effect, despite the strict inclusion criteria and natural variations in human cognition. These exploratory results support the notion of detecting MCI through Klondike Solitaire game sessions.
Intraclass correlations of the cognitive functions and player actions with 95% CIs.
amnestic mild cognitive impairment
Clinical Dementia Rating
commercial off-the-shelf
generalized linear mixed model
intraclass correlation
mild cognitive impairment
Mini-Mental State Examination
Montreal Cognitive Assessment
nonamnestic mild cognitive impairment
player action
None declared.