This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.
Background
Cognitive impulsivity is a multifaceted construct associated with symptom severity, functional impairment, and poor quality of life in eating disorders (EDs) and mental ill-health. However, objective assessment of cognitive impulsivity is piecemeal and complex, with many assessment tools lacking psychometric evaluation and ecological validity. Furthermore, validated assessment tools are rarely perceived to be engaging or meaningful by individuals who complete them, limiting their utility in research and forming a barrier to clinical translation. Thus, although cognitive impulsivity predicts treatment engagement and outcomes, it is rarely assessed or addressed in a clinical context.
Objective
We aimed to redesign and evaluate the Cognitive Impulsivity Suite (CIS), a validated gamified assessment battery of cognitive impulsivity, through user-centered co-design, agile game development, and user-centered evaluation. This collaborative study partnered researchers with individuals with lived experiences (LEs) of EDs and commonly co-occurring mental ill-health, and game development experts.
Methods
In a sequential mixed methods design, we first defined user requirements through inductive thematic analysis of data from 2 focus groups incorporating 7 individuals with LE of EDs and commonly co-occurring mental health conditions (mean age 27.6, SD 7.03 y; 4 women, 2 men, 1 nonbinary), researchers, clinicians, and technology professionals. Agile game development was achieved through 6-week-long sprints, involving game developers and a play-testing team of researchers. During LE evaluation, we collected and analyzed data from an expanded sample (n=18; mean age 30.5, SD 6.56 y; 12 women, 3 men, 2 nonbinary), using a pragmatic blending of qualitative and quantitative research methods. This included inductive and deductive thematic analysis of “thinking aloud” data, descriptive statistics, and ANOVA tests of the Game User Experience Satisfaction Scale, short version (GUESS-18) surveys.
Results
We co-designed guiding principles and ideas for aesthetics, story concepts, and gameplay features, which closely aligned with leading theories of psychological well-being, clinical evidence concerning ED recovery, and gamification frameworks. Qualitative evaluations of the new tool, CIS Papillon Park, showed user engagement and motivation were driven by opportunities for autonomy, personal accomplishment, and simulated interpersonal relationships, aligning with approaches to gamification based on self-determination theory. In quantitative evaluation, a mean GUESS-18 composite score of 45.9 (SD 9.85) showed CIS Papillon Park achieved sound overall user satisfaction, with subscale scores revealing strengths in usability, narrative, visual and audio aesthetics, and personal gratification.
Conclusions
The contributions of this study are grounded in its integration of user-centered co-design and evaluation, agile game development, and theory-driven approaches to cognitive assessment and gamification, to redesign and evaluate a validated online task battery. The resulting CIS Papillon Park combines appealing aesthetics, gamification elements that address cognitive, emotional, and social needs, and accessible playing experiences, which maximized user satisfaction and engagement while prioritizing psychological safety. Next steps involve psychometric evaluation and dissemination.
Cognitive impulsivity is a multifaceted construct that describes rapid, unplanned reactions without sufficient regard for their potential consequences, which arise from alterations in multiple processes, including attentional control, reflection, and feedback monitoring or shifting [1-3]. Distinctive patterns of cognitive impulsivity are well-documented characteristics of eating disorders (EDs) [4-8]. Furthermore, similar patterns are observed in conditions that commonly co-occur with EDs, such as mood and anxiety disorders, obsessive-compulsive disorder, substance and behavioral addictions, and attention-deficit/hyperactivity disorder [9-14]. Although causal pathways remain uncertain, elevated cognitive impulsivity has been shown to predict future onset of these disorders [15], to become more pronounced with illness duration [16], and to predict treatment outcomes [17]. Furthermore, research indicates greater severity of cognitive dysfunction in EDs is associated with symptom severity, functional impairment, and poorer quality of life [18,19]. Importantly, objective assessment of cognitive impulsivity has improved our understanding of shared and unique risk, maintenance, and treatment response mechanisms across EDs and commonly co-occurring conditions [20-24]. Nomothetic assessment of cognitive impulsivity has also facilitated the identification of novel treatment targets and neurocognitive interventions, while idiographic assessment may help identify which individuals will most benefit from those treatments [25-30]. As such, incorporating objective assessment of cognitive functioning in clinical assessment, diagnosis, and treatment planning may offer an opportunity to bridge critical treatment gaps, and help us to better understand and respond to complex clinical phenomena, such as diagnostic crossover, heterogeneity, and comorbidity [31-33]. However, despite persistent calls for this translation, objective assessment of cognitive impulsivity remains absent from routine clinical practice [34,35].
Specific barriers have been identified that limit the research impact and translation of cognitive impulsivity assessments. The first concern is the quality and quantity of existing tools, whereby many measure discrete facets of cognitive impulsivity and fail to predict real-world impulsive behaviors [36], and lack robust psychometric or normative evaluation [24,37-41]. Furthermore, the sheer array of tools, paradigms, and analysis methods creates onerous administration for clinicians and researchers, and limits replicability, comprehensiveness, and comparison [42,43]. Finally, a few of the evidence-based tools are well-accepted or engaging to users, leading to performance invalidity [44-46], limiting research participation and uptake in clinical or self-assessment contexts [47-53]. One approach to address these barriers is gamification—the process of creating opportunities for subjective value [54] through the introduction of game-like elements and mechanics [55]. Central to this approach is that gamification activates intrinsic motivational processes, thus drawing on self-determination theory, which posits human behavior is motivated by needs for competence, autonomy, and relatedness [56]. Gamification also reflects principles from other influential theories and frameworks, including social learning theory, social cognitive theory, behaviorism, and flow [57]. Gamification elements may therefore target a range of mechanisms to strengthen intrinsic motivation—cognitive (eg, achievement, progression, and mastery), emotional (eg, enjoyment, satisfaction, and immersion), or social (eg, collaboration and teamwork) [58,59]. Evidence suggests these can be achieved through various “gameful affordances” [54], for example, decomposing demanding activities into clear, achievable subtasks; creating meaning through narrative, characters, and storytelling; providing feedback, rewards, and visible indicators of progress; enabling exploration and customization; including teammates and social supports; and calibrating difficulty to optimal levels for each player [58-61]. Together with game design elements, such as aesthetic appeal, these in turn foster the desired player behavior or outcome, such as long-term user engagement, satisfaction, and consistent effort [57,62-66]. The Cognitive Impulsivity Suite (CIS) is an online task battery that was designed using these principles [3]. The CIS comprehensively assesses key cognitive processes associated with impulsive behaviors in an integrated environment, featuring a cohesive “Wild West” story and visual design. Multiple mechanisms are assessed in a single session, allowing outcomes to be analyzed and interpreted in a holistic manner, while using gamification to maximize consumer engagement [67-69]. Research has demonstrated this integrated task structure to be valid and reliable in assessing cognitive impulsivity in clinical and community samples, whether administered online or in-person [3,70,71]. However, it has also highlighted opportunities to broaden the reach and enhance the impact of the tool, particularly by advancing its clinical translation, and to improve user satisfaction, including for those with lived experience (LE) of EDs and commonly co-occurring conditions [72,73].
Lack of input from users is a second well-documented barrier limiting the research impact and clinical translation of cognitive assessments and similar technology-enabled tools [74-78]. In response, there is growing acceptance of the need to meaningfully engage users throughout the design, implementation, and evaluation of mental health interventions [79,80], service improvements [81,82], and priorities and processes for research [83-86]. Human- and user-centered design approaches to these health innovations, which prioritize user needs and iterative development processes, have been shown to maximize their usability and satisfaction, and support the translation of evidence-based innovations from research to practice [87-89]. Similarly, co-design approaches, which incorporate multiple stakeholders, including users with LE, can help to maximize the relevance and benefits of health innovations for those users [90,91], promote behavior change [92], support empowerment, uptake, and engagement [93,94], and reduce waste [95,96]. However, although emerging research suggests user-centered and lived-experience informed co-design is well suited to developing and improving cognitive assessment tools, this remains rare [78,97-99]. Additionally, though usability and user satisfaction are recognized as critical to long-term uptake and translation of health innovations, few gamified assessments or interventions are evaluated on this basis [42,88,100].
Our aims for this study were to redesign the CIS and to evaluate user satisfaction, using a collaborative sequential mixed methods co-design framework, informed by both evidence and experience [89,101]. Specifically, our researchers synthesized research evidence and stakeholder feedback to identify the need to reimagine the CIS and to ensure it retained its validated scientific structure (ie, the same cognitive constructs would be assessed in the same manner). Individuals with LEs would co-design guiding principles and ideas for game concepts and features through focus groups, alongside technology experts and researchers (refer to Ideation and Design Phase section), and would leverage their LEs to test and evaluate the new product (refer to Evaluation Phase section). Clinicians would cofacilitate focus groups and provide guidance to researchers to support the psychological well-being of participants. Technology experts would produce the new games through an iterative agile development process in close collaboration with researchers (refer to Production Phase section).
MethodsOverview
We used a sequential mixed methods co-design approach to reimagine the CIS assessment tool [102,103]. We adopted a pragmatic paradigm to achieve our objectives [104], centering users with LE as key informants at critical stages of the design process to maximize usability and satisfaction with the assessment tool [87], integrating researcher perspectives to retain the mechanistic structure of the original CIS games [101], and acknowledging the resource-limited context [89,92]. As such, our methodology adopts a framework that combines user-centered approaches to specifying requirements and evaluating the design, with rapid, low-cost production [105-107]. Here, we define co-design as a participatory process in which stakeholders with various forms of relevant expertise collaborate to address a problem concerning a system or tool [108-110]. This methodology was driven by an ethical commitment to elevate the voices of individuals with LE who will be most impacted by our research [111,112]. Furthermore, the research team values the deep experiential insights and “practical knowing” that can be accessed through participatory inquiry and researcher reflexivity, while also endorsing a critical realist perspective that values quantitative measurement of psychological phenomena [113]. Given this standpoint and our research goals, we adopted a thematic analysis (TA) approach to qualitative data analysis, a family of analytical methods that centers participant perspectives, and that acknowledges the active role of the researcher [114-116].
Details of research designs, participants, data collection, and data analysis for each phase (ideation and design, production, and evaluation) are reported below according to American Psychological Association mixed methods journal article reporting standards [117].
Ideation and Design PhaseResearch Design Overview
The qualitative ideation and design phase aimed to specify user requirements and produce a detailed design brief to guide the reproduction of the CIS. This was conducted through 2 focus group co-design sessions, which incorporated individuals with LEs of EDs and general mental health concerns, researchers, clinicians, and technology professionals. Our LE co-design approach in this phase was informed by the Australian Eating Disorder Research and Translation Center, which funded the production and evaluation phases of this project [86,118], and by user-centered design principles for eHealth and serious games [87,107,119].
Ideation and Design Study ParticipantsResearcher Description
The research team includes a PhD candidate with experience in qualitative and quantitative research, psychological assessment, and software development life-cycle (EC), a clinician-researcher with experience in EDs, addiction, and trauma (AA), a psychologist (LH), and a researcher with experience in qualitative and quantitative research and measurement development (CM). Noting researcher reflexivity and self-reflection as a critical aspect of reflexive TA [120-122], the research team reflected on their positionality and perspectives relevant to the research objectives and aimed to faithfully represent the user requirements and design preferences identified from this phase. Researcher identity and positionality statements are provided in Table S2 in Multimedia Appendix 1.
Participants
A total of 7 individuals participated in the focus groups. Participant characteristics are reported in Table 1. While there is no universally agreed-upon sample size for TA research [123,124], this sample size was deemed practical to manage [125] and sufficient to achieve the research goals based on previous research [78]. Importantly, we aimed to produce one acceptable solution to our identified research problem, rejecting the concepts of a single correct solution or “data saturation,” which do not form part of TA approaches to qualitative inquiry [121,126].
aTwo design-phase participants chose not to report their weight.
bARFID: avoidant or restrictive food intake disorder.
cEDNOS: eating disorder not otherwise specified.
dOSFED: other specified feeding and eating disorder (including atypical anorexia, and BED/BN of limited frequency or duration).
eADHD: attention-deficit/hyperactivity disorder.
fAnxiety disorders (including but not limited to generalized anxiety disorder, panic disorder).
gBPD: borderline personality disorder.
hOCD: obsessive-compulsive disorder.
Researcher-Participant Relationship
There was an established relationship between the research team attending the focus groups (AA, EC, and LH) who had collaborated on the development of the original CIS [3]. There were no preexisting relationships between the research team and any of the participants. Ethical concerns relevant to this research context included the unequal power relationship between researchers and participants. In addition to informed consent and ethical review procedures, psychologist, LH, attended the focus groups with the expressed role of supporting participant well-being.
Participant Recruitment and Selection
We recruited participants aged 18‐40 years who had current or past diagnoses of any ED. For safety and compliance, exclusion criteria were currently experiencing psychotic symptoms, bipolar disorder, and suicidality. Additionally, we excluded individuals who had previously completed the CIS. Participants were recruited using Meta advertisements, existing participant databases, and through local and national ED organizations (eg, InsideOut Institute, Australia & New Zealand Academy of Eating Disorders, and Scientific Work in Anorexia Nervosa & Other Eating Disorders).
Of the 29 who expressed interest in the study, 22 met eligibility criteria. A total of 7 participants were selected and attended the first focus group, with 6 participants returning for the second focus group (Figure 1). Focus groups were held in person at Monash University between June and July 2023. Participant characteristics are reported in Table 1.
Design-phase participant flowchart.
The figure depicts a flowchart of participants who were enrolled, screened, allocated, and completed the ideation and design phase of the CIS redevelopment project from April to July 2023.
Ideation and Design Data Collection
Participants provided informed consent and completed screening and enrollment questionnaires using REDCap (Research Electronic Data Capture) [127,128]. Eligible individuals were contacted to confirm eligibility and be briefed on focus group procedures. Participants attended 2 focus group sessions, scheduled 1 month apart, which were cofacilitated by EC and AA, and attended by a game development professional and a psychologist (LH). Each focus group lasted 3 hours.
Participants were asked open questions to generate ideas for a gamified cognitive assessment tool that should appeal to diverse audiences with EDs and commonly co-occurring conditions (refer to section S4 in Multimedia Appendix 1 for focus group guides). To avoid influencing ideas, participants were not shown visuals depicting the original game. Instead, brief details of the CIS’s general purpose, structure, and basic schematics were shared. A brief feedback survey collected their group and research experiences, such that improvements could be made for the second session (section S4.2 in Multimedia Appendix 1) [129-131]. In the second focus group, we honed design concepts and features and clarified key topics of conversation from the first session. Following the second session, participants were again invited to complete a brief survey to evaluate their experiences. Throughout these sessions, the facilitators, psychologist, and technology professional engaged in active discussions and whiteboarding with participants, prompting iterative review, expansion, and refinement of ideas. Sessions were audio recorded using Echo360 or Zoom software, and transcripts were redacted to protect anonymity.
Ideation and Design Data Analysis
Data were analyzed according to the 6 stages of reflexive TA by Braun and Clarke [114]. An inductive approach was used to explore participants’ ideas and perspectives, while acknowledging the positivist, cognitive-behavioral perspective inherent in psychological assessments and the constraints imposed by the existing validated task structure. Reflexive TA further acknowledges that themes are actively generated in a subjective process, whereby the researcher interprets meaning from participant statements [126,132]. Final transcripts were coded by EC using NVivo 14 (Lumivero) [133].
In the first stage of reflexive TA, deep familiarization, transcripts and recordings were reviewed. The second stage involved initial coding, whereby an open coding approach to label segments of text within the data was used. In stage 3, generating themes, codes were grouped into categories to generate the main themes. Stage 4 involved reviewing themes to ensure a rich understanding of the data, whereby AA, LH, and CM were consulted to provide feedback on the drafted themes and possible interpretations. Themes were further refined in stage 5, theme finalization. Finally, stage 6, interpreting and reporting, involved the research teams’ continued revision of the codes and themes during the write-up phase.
Production PhaseResearch Design Overview
The production phase involved implementing the design ideas and guiding principles defined in the Ideation and Design Phase section. Here, we engaged Noble Steed Games (NSG) as an expert partner to design and produce the revised CIS. The new CIS was produced in an agile development process, involving 6 week-long iterative and collaborative sprints between NSG and the research team between March and April 2025. This approach was informed by guidelines and case studies concerning the pragmatic integration of user-centered design and LE co-design principles with the challenges of producing highly technical solutions that require expert knowledge, within a context of limited time and resources [88,92,105,106].
Production Participants
The production team involved 4 members of the NSG team, alongside the first author (EC). A play testing team was formed to assess usability and the extent to which the new CIS achieved user requirements and research objectives [119]. This team involved EC, CM, 3 student researchers, and 4 members of the AVG research group. All play testing members were familiar with LE co-design guiding principles and design ideas, and with the original CIS.
Production Procedures
NSG used CIS Choices in the Wild West, co-designed guiding principles, and cogenerated design ideas to inform their production of the new version of the CIS, referred to as Papillon Park (section S4, Multimedia Appendix 1). For the Reflection mini-game, NSG sourced copyright-free images from Unsplashed and Pexels, which were processed by EC to standardize size, color, contrast, and luminance properties using the magick package in R (R Core Team) [134]. During each sprint, NSG produced a specific deliverable (eg, wireframes for each mini-game; scene, stimulus, and character art; and scripts and instruction text), which were reviewed and refined in consultation with EC. Once available, playable prototypes were deployed to secure Monash servers, such that game functionality and data capture could be tested. These prototypes were further refined in response to play testing feedback, for example, by modifying stimulus timing and on-screen positioning to ensure target constructs were being effectively assessed. Following completion of the agile production period, ad hoc updates and refinements were defined by EC and implemented by NSG in response to LE play testing and evaluations. No research data were collected or analyzed during this phase.
Evaluation PhaseResearch Design Overview
The evaluation phase aimed to explore user experience satisfaction and the extent to which the design of CIS Papillon Park met user requirements [87,119]. To achieve these goals, we adopted a convergent mixed methods design. The GUESS-18 questionnaire was selected as an effective tool for quantitative data collection [135,136] and analyzed using descriptive and inferential statistics [116,137,138]. “Thinking aloud” was selected as an appropriate method for qualitative data collection, whereby participants provide a verbal commentary while playing the games [139,140], which was analyzed using deductive and inductive cycles of TA. While this method departs from the “Big Q” paradigm described by Braun and Clarke [115], this approach allowed us to answer our research questions by structuring our analysis according to the user requirements defined during the ideation and design phase, and is well-suited to convergent mixed methods approaches [116].
Evaluation Study ParticipantsParticipants
We sought to retain participants from the ideation and design phase while recruiting additional participants to expand our sample size and composition. We planned to divide this sample into 3 approximately equal groups to examine whether participation in the ideation and design phase, or the process of “thinking aloud” while playing, influenced subsequent quantitative evaluations of user experiences and satisfaction.
Participant Recruitment and Selection
Participants who contributed to the ideation and design phase were invited to the present phase via email. Additional participants were recruited through Meta advertisements and existing participant databases. Identical inclusion and exclusion criteria used in the ideation and design phase were used. Of the total, 5 of the ideation and design phase participants returned for the evaluation phase. Of the 43 additional recruited participants, 27 met eligibility criteria, and 13 completed the study. All 5 participants from the ideation and design phase were invited to attend 1-hour Zoom sessions to play CIS Papillon Park while providing “thinking aloud” feedback. Newly recruited participants were split into 2 groups, such that 6 also completed Zoom “thinking aloud” sessions, and 7 played the game in their own time, according to their availability and preferences (Figure 2). Sessions took place in April 2025.
Evaluation-phase participant flowchart.
The figure depicts a flowchart of participants who were enrolled, screened, allocated, and completed the evaluation phase of the CIS redevelopment project from March to May 2025.
Characteristics of the 18 participants who completed the evaluation phase are summarized in Table 2.
Lived experience evaluation phase participant characteristics (n=18; counts may sum to >18 due to cases with multiple diagnoses).
Demographic characteristic
Value
Age (y), mean (SD; range)
30.46 (6.56; 19‐42)
BMI (kg/m2), mean (SD; range)
23.78 (6.94; 15.6-42.7)a
Race or ethnicity, n
Caucasian (White Australian, European, or North American)
11
Asian
7
Gender identity (self-identified), n
Cisgender woman
12
Cisgender man
3
Transgender woman
0
Transgender man
1
Nonbinary or gender diverse
2
Current eating disorder diagnoses, n
ARFIDb
2
Anorexia nervosa
3
Binge eating disorder
0
Bulimia nervosa
1
EDNOSc or OSFEDd
2
None
10
Historic eating disorder diagnoses, n
ARFID
1
Anorexia nervosa
1
Binge eating disorder
5
Bulimia nervosa
6
EDNOS or OSFED
4
Other lifetime psychiatric diagnoses, n
ADHDe
6
Anxietyf
11
Autism
3
BPDg
1
Depression
8
OCDh
2
Other
2
aFour evaluation-phase participants chose not to report their weight.
bARFID: avoidant or restrictive food intake disorder.
cEDNOS: eating disorder not otherwise specified.
dOSFED: other specified feeding and eating disorder (including atypical anorexia, and BED/BN of limited frequency or duration).
eADHD: attention-deficit/hyperactivity disorder.
fAnxiety disorders (including but not limited to generalized anxiety disorder, panic disorder).
gBPD: borderline personality disorder.
hOCD: obsessive-compulsive disorder.
Sample Size, Power, and Precision
We did not perform an a priori power analysis. The target sample size of 21 participants was instead determined by the depth-oriented approach of the qualitative component of the evaluation phase, in accordance with established guidelines for this methodology and previous research [78,123].
Setting
Participants were provided with a link to access the CIS Papillon Park online and were instructed to use a computer with a keyboard (ie, not a phone or tablet). Participants were informed that the CIS would take approximately 40 minutes to complete and to complete the games in a single sitting. Participants completing “thinking aloud” sessions were provided with the link during their Zoom session, while those playing in their own time received it via email through REDCap. The 3 mini-games were presented in a randomized order, and server logs were used to confirm completion.
Quantitative Measures
The Game User Experience Satisfaction Scale, short version (GUESS-18) was used to gather quantitative feedback [135,136]. GUESS-18 is a psychometrically validated tool with 9 subscales that assess how various game attributes contribute to user satisfaction (Table S3.1 in Multimedia Appendix 1). Participants rated 18 items (eg, “I think the game is visually appealing”) on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree) with a neutral midpoint of 4 (neither agree nor disagree). One item (“I feel bored while playing the game”) is reverse-coded. Subscale scores (1-7) are calculated by averaging numeric ratings for component questions, and a composite score (9-63) is obtained by summing subscale scores, with higher scores indicating greater user satisfaction. In this study, internal reliability for the composite scale was excellent (Cronbach α=0.93, Zumbo ordinal α=0.95). For subscales, refer to Table S3.2 in Multimedia Appendix 1.
After critically analyzing the alignment of the GUESS-18 subscale constructs to the co-designed guiding principles in the ideation and design phase (Table S3.3 in Multimedia Appendix 1), 2 items were developed to explore theme, No Trigger Warnings Needed (“I feel the games are appropriate for someone with lived experiences like mine,” “I feel the games are appropriate for someone with lived experiences like mine,” “I feel the games are triggering for someone with lived experiences like mine” [reverse coded]). Space for additional open-text feedback was provided at the end of the instrument.
Evaluation Data Collection
Participants provided informed consent and completed screening and enrollment questionnaires online via REDCap [127,128].
Qualitative Data Collection
Of the 18 participants, 11 confirmed their availability for “thinking aloud” sessions to take place via Zoom [139]. During the session, participants were asked to access CIS Papillon Park online and provide a running commentary of their thought process as they proceeded through the game. Participants were advised that the facilitator (EC) would simply observe unless they became quiet, in which case a prompt would be provided. “Thinking aloud” sessions were audio- and video-recorded and transcribed using Zoom.
The remaining 7 participants preferred to complete CIS Papillon Park in their own time and were invited to provide qualitative feedback using a free text box in the online feedback questionnaire.
Quantitative Data Collection
After completing the CIS Papillon Park, participants were directed to complete the GUESS-18 questionnaire and the additional 2 items concerning the appropriateness of the tool for individuals with lived and living experiences of EDs and general mental health concerns (section S3.1 in Multimedia Appendix 1).
Evaluation Data AnalysisQualitative Data Analysis
Transcripts were reviewed for accuracy and uploaded to NVivo 15 for analysis [141]. First author, EC, undertook deep familiarization with the data. Data were coded to first identify technical or usability issues that could be rapidly resolved by NSG (eg, statements concerning misunderstandings of game instructions). EC then conducted deductive and inductive cycles of TA [116]. The co-designed guiding principles were used as a coding framework for deductive TA, with statements coded for insights into the extent to which the new tool met these user requirements. Data were then revisited with an inductive lens, seeking further insights into user experiences that had not been captured during deductive analysis. Finally, EC and CM reviewed codes, initial themes, and interpretations for clarity and narrative consistency. Themes continued to be refined during the write-up and reporting phase.
Quantitative Data Analysis
GUESS-18 responses were scored using Microsoft Excel and analyzed and visualized in R Studio [142]. There were no missing data. Means and SDs were calculated for total user satisfaction and for each of the 9 subscales according to published guidelines [135,136]. Exploratory omnibus ANOVA tests were used to examine group differences in GUESS-18 overall and subscale satisfaction between the 3 groups of participants (ie, designers, newly recruited “thinking aloud” participants, and those who completed the games in their own time).
Data Integration
The integration of qualitative and quantitative feedback data was intended to provide greater value than each alone [143]. Qualitative evaluation using a psychometrically sound tool facilitates comparison with other serious games and gamified psychological assessments, which is challenging when using only qualitative data, but lacks depth, and does not distinguish between the 3 component games of the overall tool. Conversely, qualitative data allowed us to understand why users felt satisfied or dissatisfied with particular elements of their experiences, and the variability in their experiences between the 3 mini-games. To achieve these goals, we aimed to synthesize and interpret qualitative and quantitative data at individual, group, and sample levels, and present an integrated narrative of our findings in the results section [137].
Ethical Considerations
Ethical approval for all research procedures was provided by Monash University Human Research Ethics Committee (reference 29393), and all research participants provided informed written consent for the data collection and analysis described herein. Data were deidentified to protect participant privacy and confidentiality. No participants can be identified by the information included within the main paper or supplementary file, where participants are referred to individually (eg, in quotations), pseudonyms have been used. Participants with LE were compensated Aus $60 (approximately US $44) for each focus group (or Aus $20 [approximately US $15] for Zoom make-up sessions) and Aus $40 (approximately US $29) for the evaluation phase.
ResultsIdeation and Design Phase ResultsOverview
In total, three overarching themes were defined with a total of 9 subthemes that describe guiding principles for redeveloping the CIS: (1) aesthetic appeal and a progressive story create an immersive user experience; (2) achieving individual and collaborative goals fosters player motivation and purpose; and (3) relatedness, variety, and autonomy: fulfilling key psychological needs enhances user engagement. The meanings of these themes and their subthemes are explained below, with illustrative quotes provided in Table 3.
Summary of co-designed guiding principles and illustrative focus group quotes.
Theme and subtheme
Illustrative quotes
Aesthetic appeal and a progressive story create an immersive user experience
1.1 An interactive journey
“So, there’s a narrative, overarching kind of structure that you have to follow, and the entire story is just the mission.” [Ang]
“I like collecting things along the way … or like, just like interacting with objects in a way that doesn’t necessarily add anything to your character or development, or anything, but just little interactions that are there.” [Erin]
1.2 Whimsical worlds have broad appeal
“There are a lot of like popular games that take a lot of elements of the real world, but then they adapt it into like their own art style, I guess, like aesthetic. It’s just, very NICE to look at. And yeah, I guess the animations and the storyline, it’s just very relaxing.” [Georgie]
“I guess, I just find them very happy and appealing to look at … whimsical, is kind of the word that comes to mind.…. I guess it’s a bit childish, but it’s like happy childhood. It’s sort of like happy childhood memories. Yeah, it’s got that whimsical nature to it, but it still blends in a bit of reality.” [Fiona]
“I think when I’m playing games, quite often, I would be doing it to kind of get out of the real world. I think a lot of people who just use games as like a form of escapism.” [Chloe]
Achieving individual and collaborative goals fosters player motivation and purpose
2.1 Growing and “leveling up”
“I really like being able to level up and acquire certain skills.” [Erin]
“With the experience, then you sort of level-up… you can purchase items that help you … or certain sorts of clothing, hairstyle, something like that.” [Ang]
2.2 A prosocial purpose
“For me, that’s a quest … you find the person or the clues to save humanity.” [Bethany]
“Oh yeah! Rescuing the towns people! That would be so cute!” [Fiona]
“I was thinking of growing something, potentially.” [Dylan]
“Yeah, whether it’s a garden or a forest. Just … more plant life, the better you do.” [Dylan]
“I feel like it’s all in a game. You know that this is virtual. It’s not real to me. I think others probably feel differently though.” [Ang]
2.3 Positive insight
“I was thinking perhaps keeping it more towards, say, benefit versus less benefit as opposed to danger versus no danger.” [Dylan]
“I like that it’s not catastrophic, you know? You don’t lose entirely.” [Ang]
“Interpretation is more helpful. A leader board is a terrible idea.” [Bethany]
“Because it just has that social comparison aspect.” [Erin]
“Yeah, even though I’m trying, everyone else is doing so much better than I am.” [Bethany]
Relatedness, variety, and autonomy: fulfilling key psychological needs enhances user engagement
3.1 Batman needs his Alfred
“You’re Batman, and you need your Alfred.” [Bethany]
“I think belonging to a group does enhance a game, whether it’s a community or a gang or whatever. It creates more responsibility. And I really like to be able to get to know the people in your community or team, or whatever, and them all having their own little personalities and stories.” [Erin]
“A diverse set of characters is more engaging, and each of the interactions adding to the storyline, adding details to this world. I feel that would make it more engaging.” [Georgie]
3.2 Variety is the spice of game-life
“I think, depending on how often in the game this is going to occur, [it’s important] for it to not be the same every time …. I think I’d get a bit annoyed.” [Erin]
“I like games with time limits …” [Chloe]
“Whoa, so stressful!” [Ang]
“Yes but keeps it really interesting.” [Chloe]
“Where it’s not too difficult where it’s frustrating, but it’s not too straightforward.” [Georgie]
3.3 Choose your own adventure
“I was thinking of a Harry Potter game I used to play when I was a kid, and you could, choose different wands. And that would give you different abilities.” [Chloe]
“Yeah, Like different characters, different paths? Yeah. It’s like the first game that I ever played was Pokémon. You choose the different sort of Pokémon, right? And then based on the different type, you have different abilities and so on.” [Ang]
“I like changing the environment … in terms of customisation.” [Bethany]
3.4 No trigger warnings needed
“The main thing that I was thinking about was in some games you just get a little bit too much freedom with customizing your character. Most of the time people just end up creating, I guess what they wish they looked like. But then it just gets a bit triggering or problematic once you get to adjust heights or body shape and size and stuff. So, yeah, I feel too much customization of characters in games would be triggering.” [Georgie]
“I think diversity in size, shape, all that - that’s a big one. Size and shape in video games, there’s not much diversity. So, I think that’s really important.” [Erin]
“I think there’s going to be a lot less issues around inclusivity and stuff if you’re just playing from behind the camera.” [Erin]
“I think, if you’re the one looking in, you’ve got less issues, with things like gender representation. If you’re representing other people or characters, they need to be diverse.” [Bethany]
Theme 1: Aesthetic Appeal and a Progressive Story Create an Immersive User ExperienceSubtheme 1.1: An Interactive Journey
Participants described a preference for first-person game-play and a sense of advancement and forward momentum within a cohesive narrative structure. For example, the playing character would proceed through a journey or quest across a series of tasks, completing these as missions toward their overall goal. Participants also agreed that the playing experience would be improved by exploring varied environments as they progressed through the story.
Subtheme 1.2: Whimsical Worlds Have Broad Appeal
Participants described wanting the games to have an appealing art style, with characters and environments that are cartoonish, simple, and colorful, and a playing experience that is gentle and accessible. They consistently drew on examples such as Nintendo’s Super Mario and Animal Crossing, and there was consensus that these features appeal to “gamers” and “nongamers” alike. Participants described a sense of escapism in these whimsical worlds as a key attraction to gaming. They also described the enjoyment of completing relatively simple or familiar tasks that have been rendered funny, quirky, surprising, or novel by the whimsical and cartoonish environment, or by the abilities (or lack thereof) of their playing character.
Theme 2: Achieving Individual and Collaborative Goals Fosters Player Motivation and PurposeSubtheme 2.1: Growing and Leveling Up
Participants discussed the most meaningful goals and rewards to be gaining knowledge, skills, and expertise. These gains might contribute directly to the overall game goal, achieve personal growth, or be used to augment the character or their environment.
Subtheme 2.2: A Prosocial Purpose
Participants resonated with having an overall objective that helped others, although one participant felt this was less important. Participants agreed, however, that rewards should be nonmonetary, and that in-game “currency” should be instrumental to achieving their prosocial goal, rather than “capitalistic.” Examples included rescuing townspeople, growing plant life, or taking care of realistic or mythical animals.
Subtheme 2.3: Positive Insight
Participants discussed that leaderboards and comparisons to other players’ performance could be motivating; however, consensus was reached that feedback should avoid social comparison. The participants agreed that poor performance should be “safe,” such that no in-game characters would be harmed or imperiled if the player gained relatively little in-game currency or points. As much as possible, outcomes should be neutral or positive and avoid damage or significant losses, and feedback should be normalizing and supportive.
Theme 3: Relatedness, Variety, and Autonomy: Fulfilling Key Psychological Needs Enhances User EngagementSubtheme 3.1: Batman Needs His Alfred
Participants felt games played within a mental health context should incorporate positive in-game relationships. One example was through a constant supporting presence of a sidekick, or as one participant stated, “You’re Batman, and you need your Alfred.” In addition, participants felt the playing character could be part of a virtual team or be depicted engaging positively with nonplayable characters (NPCs) from the broader community. This subtheme, therefore, is closely related to pursuing a prosocial purpose.
Subtheme 3.2: Variety Is the Spice of Game Life
It was emphasized that incorporating as much variety as possible within the constraints of the task structure would be key to maintaining player motivation and would prevent the player experience from becoming monotonous. Participants also felt an appropriate level of difficulty and challenge to be important; while too little can be boring, too much becomes stressful and a barrier to engagement.
Subtheme 3.3: Choose Your Own Adventure
Participants requested the ability to make customizations and to interact with or construct the environment around them. These experiences were felt to create richness and immersion in games, as well as providing a sense of autonomy that fosters active engagement.
Subtheme 3.4: No Trigger-Warnings Needed
To maintain psychological safety, the participants agreed on the importance of avoiding the ability to customize or alter human forms and of diversity in the depiction of human and human-like characters. Avoiding stimuli or gameplay relating to food and eating, body image, alcohol, drugs or gambling, violence or crime, and gendered, racial, or cultural stereotypes was also discussed as critical for individuals who share their lived and living experiences.
In addition to the themes described above, the focus groups generated many specific ideas for each of the component games, which were summarized and communicated to the game developers (section S4 in Multimedia Appendix 1). Importantly, the focus group participants also evaluated their ideation and design phase experience positively (section S4.2 in Multimedia Appendix 1).
Production Phase Results
To achieve the co-designed themes while maintaining the validated structure of the original CIS, our game development partner produced the CIS Papillon Park with 3 component mini-games featuring a cohesive community garden theme. Cognitive Impulsivity is assessed as users proceed through Horticulture Hero (attentional control), Fauna Finder (reflection), and Verdant Venture (feedback monitoring or shifting), which can be presented in set or randomized order. Each game occurs within a unique garden environment and begins with a brief practice mode before the scored blocks (Figure 3).
Introductory screens for CIS Papillon Park component mini-games: Horticulture Hero (attentional control), Fauna Finder (reflection), and Verdant Venture (feedback monitoring/shifting).
The figure depicts 3 colorful cartoon images of a greenhouse workbench, each overlaid with 3 images from the mini-games, explaining their goals and response keys.
Users are led through the games by 2 virtual helpers—“Grandma Pixie” and “Winston” (Figure 4), and interact with diverse NPCs who reinforce their progress and the positive impact it has on the community. At the conclusion of each mini-game, users receive their total points and are offered a choice between 2 rewards to decorate the community garden. Their progress is visually depicted with their chosen reward and new growth.
The figure depicts an Australian ringtail possum wearing glasses and a yellow bow tie seated on a stool, alongside an older adult woman dressed in pink and leaning on a wooden walking stick. These virtual helpers provide instruction, performance feedback, and companionship to the user throughout CIS Papillon Park.
Winston and Grandma Pixie.
Evaluation Phase ResultsQualitative Outcomes
Qualitative evaluations were consistent with the 3 superordinate themes and subthemes that participants described during phase 1. The meanings of participant evaluations are described below, with illustrative quotations provided in Table 4.
Summary of thematic evaluations and illustrative quotes.
Theme and subtheme
Illustrative quotes
Aesthetic appeal and a progressive story create an immersive user experience
1.1 An interactive journey
“Overall, I quite liked the flow of the storyline and the characters that I interacted with…. It’s a very well-developed experience.” [Liza]
“I like that it’s got a little ongoing story to kind of …. I feel like it like with that, and mixing things up, it kind of adds to that gamified element by like giving you something to work towards.” [Jay]
1.2 Whimsical worlds have broad appeal
“I found the graphics and background music/side effects of the game to be very calming and immersive.” [Lily]
“I like the way that this is designed. I’ve done similar things to this before, where, um, I guess they’re assessing a similar sort of thing where it’s hit the space bar when a particular number comes up on screen. But I think the way that this is … visually it’s a lot more ... interesting, which is nice. It’s got nice bright colors, which makes it a little bit more fun to look at, versus your kind of black and white numbers on a screen.” [Jay]
Achieving individual and collaborative goals fosters player motivation and purpose
2.1 Growing and “leveling up”
“You probably get a little better at this game as it goes on … as you get more familiar with it.” [Tom]
“The ‘practice’ rounds of each game helped allay any anxiety I had about learning how to play the game and I found them all quite easy to pick up quickly.” [Olivia]
“I would say that overall, it’s an interestingly positive experience, because, firstly, it’s not a steep learning curve. Not at all. I like and how simple it is, and how easy it is to pick up.” [Ang]
“It did make me disheartened to not be able to figure out the pattern for this game and made me question whether I wasn’t smart enough.” [Millie]
2.2 A prosocial purpose
“[The NPC’s] were great. I think they were all very likable.” [Jess]
“Maybe more people will come and visit and bring me [Winston] presents … I want that!” [Grace]
“I’m wondering if these random visitors are gonna come back into the game, - if they like if they play a role, or if they are just … there.” [Erin]
“Grandmother’s a bit patronizing, isn’t she?” [Bethany]
2.3 Positive insight
“The sound effects when I got answers right/wrong were very cute … and I felt a tinge of sadness when getting an answer wrong, but I didn’t inherently feel upset for the rest of the game.” [Liza]
“I wouldn’t say it’s easy not to beat yourself up about it …. but I like how low stakes this is.” [Ang]
“I was left wondering how I had performed versus other people playing the game in this study.” [Liza]
Relatedness, variety, and autonomy: fulfilling key psychological needs enhances user engagement
3.1 Batman needs his Alfred
“I think the art is a really good motivator because you get to see little Winston and Grandma Pixie … Who doesn’t like a happy Winston?” [Alex]
“I like his happy sound … if you’re happy, I’m happy.” [Grace]
“That’s three 200’s in a row, Winston. I hope you’re happy, little buddy.” [Ang]
“I’m gonna end up in the negatives. Sorry, Winston.” [Lily]
3.2 Variety is the spice of game-life
“The [horticulture hero] game was interesting and I liked that the plants changed each round.” [Liza]
“It’s nice to see the variety of points change this time” [Alex]
“I enjoyed the blurry animal guess game most - it wasn’t the same choices repeated and it got you to engage cognitively with time urgency.” [Millie]
“I feel like the whole design of this game is just to test my patience!” [Jess]
“[There were] too many rounds to make it to the next stage. It became quite tedious, boring and irritating.” [Jo]
“If I was playing this game under normal circumstances, like at home, I might just turn it off, because it’s really long.” [Chloe]
3.3 Choose your own adventure
“Oooh, there’s a bird or a mushroom … I want the bird. (Gasps). It’s a bird fountain! Oh my gosh that’s so cool. OK, yes, I agree, wonderful choice, I know!” [Grace]
“That’s cute … I like how you can customize it.” [Erin]
“In the blurry game [Fauna Finder] … I felt I could actually engage and make a difference rather than something allocated/randomized.” [Lexi]
“The first game was confusing as it was not clear what made each plant a successful choice [and why there wasn’t 100 points, but only − 100, 50 and 200] and made me think the success was randomized like a gambling game rather than anything indicative of your own performance and choices.” [Alex]
3.4 No trigger warnings needed
“I really like the characters around here, because I can see that a lot of thought’s gone into their design.... Very interesting gender-neutral choice.” [Ang]
“It was nice that there was like a diverse range of different people that came in.” [Fiona]
“The choice of animals - since you’ve got such realistic pictures - gorgeous for animal lovers, and they’re not generally one’s people get too icked out about - possibly the mice, but, like, if you were showing pictures of snakes, I wouldn’t be able to play.” [Bethany]
“I wonder what would happen to somebody who had a phobia of frogs.” [Ang]
Theme 1: Aesthetic Appeal and a Progressive Story Create an Immersive User ExperienceSubtheme 1.1: Interactive Journey
Perspectives concerning the game story, context, and environment were consistently positive. Participants enjoyed the overall story and community garden theme, and that each game took place within a different environment to provide variety and achieve the desired sense of progression.
Subtheme 1.2: Whimsical Worlds
All participants praised the art style and felt the visual and sound designs enhanced their playing experience. Participants described the designs as appealing, calm, and soothing, and as effectively “drawing players in.” They felt these designs were key to maintaining their motivation and drew positive comparisons to standard cognitive assessments they had experienced.
Theme 2: Achieving Individual and Collaborative Goals Fosters Player Motivation and PurposeSubtheme 2.1: Growing and “Leveling Up”
Participants described feeling that they were able to learn and improve their skills in positive terms. However, the degree to which participants endorsed this feeling varied across the 3 mini-games; the most positive statements related to Fauna Finder, which participants felt was more skill-based, and the more negative statements related to Verdant Venture, which participants tended to find frustrating.
Subtheme 2.2: A Prosocial Purpose
The community NPCs were found to be likable and appealing, and participants enjoyed seeing their achievements benefit Winston, their virtual companion. However, while some participants enjoyed interacting with the NPCs and learning that their performance was benefiting the community, others did not connect with this intended feature or found the NPC interactions to be protracted. Grandma Pixie’s communication style also inspired some more polarized responses.
Subtheme 2.3: Positive Insight
Participants felt the game created a gentle, low-stakes environment that still engaged them emotionally and cognitively—enough to care about their performance, but not enough to feel stressed—and felt their interactions with Winston effectively supported this. Participants provided more mixed feedback relating to the lack of context for point totals; some participants wanted to understand their performance in comparison to other players or the maximum available, while others appreciated that this was not a feature.
Theme 3: Relatedness, Variety, and Autonomy: Fulfilling Key Psychological Needs Enhances User EngagementSubtheme 3.1: Batman Needs His Alfred
All participants responded positively to Winston’s companionship. Participants commented that they were motivated to perform on behalf of Winston, engaging emotionally with his audio and visual performance feedback.
Subtheme 3.2: Variety Is the Spice of Game-Life
Participants reported high enjoyment, engagement, and motivation when stimuli, responses, and rewards were varied, and when they felt their behavior was consequential. As with Growing and leveling-up above, these experiences varied across the mini-games, with participants typically finding Horticulture Hero to be easy but boring, Fauna Finder to be the most purposeful and engaging, and Verdant Venture to be the most difficult, random, and confusing. Moreover, 2 participants felt the game was overall too long.
Subtheme 3.3: Choose Your Own Adventure
Although opportunities for customization were limited, the ability to select from available rewards and to see those rewards depicted as garden decorations was appreciated and enhanced player engagement and fulfillment.
Subtheme 3.4: No Trigger Warnings Needed
Finally, participants praised the diverse character depictions and “wholesome” gameplay features. They felt the production phase had been attentive to inclusivity and emotional safety, while also highlighting the need to consider individual sensitivities to the animals depicted in Fauna Finder.
Quantitative Outcomes
Evaluations of the games were overall positive, with a mean GUESS-18 composite score of 45.9 (SD=9.85; 72.9%, 57.3‐88.5), and subscale scores ranging from 4.1 (neutral) for “Creative Freedom” to 6.4 (agree-to-strongly agree) for “Usability or Playability” (Table 5; Figure 5). There were no significant differences in mean composite or subscale scores between the groups of participants who were involved in the ideation and design phase, those who completed the “thinking aloud” evaluation over Zoom, and those who evaluated the games independently (Table 5).
Game User Experience Satisfaction Scale, short version user satisfaction survey results.
Measure
All (n=18), mean (SD)
Designers (n=5), mean (SD)
Zoom (n=6), mean (SD)
Own time (n=7), mean (SD)
Group differences (ANOVA)a
F test (df)
P value
Composite scoreb
45.9 (9.85)
47.4 (4.98)
42.4 (10.38)
47.9 (12.25)
0.55 (2, 15)
.59
Subscale scoresc
Usability or playability
6.4 (0.73)
6.7 (0.27)
6.3 (0.76)
6.4 (0.94)
0.52 (2, 15)
.60
Personal gratification
5.8 (0.75)
5.8 (0.76)
5.7 (0.88)
5.9 (0.75)
0.10 (2, 15)
.91
Visual aesthetics
5.6 (1.33)
5.8 (0.57)
5.3 (1.40)
5.8 (1.73)
0.22 (2, 15)
.81
Narratives
5.4 (1.63)
6.1 (0.82)
5.1 (1.77)
5.3 (1.98)
0.56 (2, 15)
.58
Audio aesthetics
5.1 (1.79)
5.5 (1.70)
4.9 (1.69)
4.9 (2.14)
0.19 (2, 15)
.82
Play engrossment
4.8 (1.06)
5.0 (0.35)
4.3 (1.21)
5.1 (1.21)
1.14 (2, 15)
.35
Social connectivity
4.5 (1.25)
4.6 (0.96)
3.9 (1.46)
4.9 (1.25)
0.94 (2, 15)
.41
Enjoyment
4.3 (1.75)
4.2 (0.76)
3.3 (1.72)
5.3 (1.89)
2.62 (2, 15)
.11
Creative freedom
4.1 (1.63)
3.7 (1.44)
3.8 (1.29)
4.6 (2.05)
0.55 (2, 15)
.59
aAchieved power for 1-way omnibus ANOVA tests was calculated as 0.74 using G*Power 3.1 (α=.05, sample n=18, 3 groups, and average effect size of F=0.76).
bComposite scores range 9‐63 with neutral midpoint 27.
cSubscale scores range 1‐7 with neutral midpoint 4; higher scores indicate greater user satisfaction.
Game User Experience Satisfaction Scale, short version subscale ratings.
The figure presents a bar graph depicting GUESS-18 subscale ratings for the full evaluation sample (n=18). Bars indicate mean ratings for each subscale, and error bars indicate the SD, arranged from the subscale with the highest user satisfaction (usability or playability) to the lowest (creative freedom).
Ratings for the additional questionnaire items concerning No Trigger Warnings Needed produced an overall mean score of 5.72 (SD 1.22). Ratings differed significantly by group (F2,15=7.64; P<.01) with significantly higher ratings in the 2 groups providing “thinking aloud” feedback than in the group playing in their own time (scoring and ANOVA outcomes for these items are provided in Table S3.4 in Multimedia Appendix 1).
Integration
Qualitative and quantitative evidence converged to allow us to understand which design features and gamification elements were most effective in facilitating long-term user motivation and engagement. Specifically, both qualitative and quantitative data demonstrated that the “wholesome” environment, cohesive and progressive story, attractive visuals, and appealing audio created a positive emotional user experience, which in turn facilitated immersion and enjoyment.
Qualitative evaluations also complemented quantitative data by allowing users to explain why particular subscales were rated higher or lower. For example, the inclusion of virtual helpers, practice rounds, retries, a progress bar, a timer or speed element, and visual depiction of achievements were key to user motivation and underpinned the high levels of usability or playability and personal gratification. Similarly, qualitative feedback explained that perceiving the themes defined during co-design concerning desires for autonomy (choose your own adventure) and variety (variety is the spice of game life) to be lacking resulted in the lowest GUESS ratings for the “enjoyment” and “creative freedom” subscales. Importantly, qualitative feedback showed users perceived the 3 mini-games that form the CIS Papillon Park to vary according to these characteristics, as well as their perceived level of difficulty, such that user motivation and engagement varied significantly between each mini-game. This variability had not been captured in the quantitative user satisfaction ratings.
Qualitative and quantitative evaluations diverged in relation to the GUESS-18 ratings for social connectivity and the overwhelmingly positive appraisals of Winston. Further analysis indicated that lower ratings for social connectivity were associated with qualitative feedback that NPC interactions, and the game overall, were too long.
Discussion
We aimed to redesign and evaluate the CIS, a validated gamified assessment battery of cognitive impulsivity, through a sequential mixed methods approach to user-centered co-design, agile game development, and user-centered evaluation [3,89,106]. We first defined LE user requirements, generating themes and subthemes that described overall guiding principles, and producing specific design ideas. The agile game development process resulted in a functioning suite of gamified assessments, the CIS Papillon Park, being deployed to secure Monash University servers. During LE evaluation, our qualitative analysis confirmed that the new tool largely met the co-designed user requirements. Quantitative ratings aligned closely, showing the CIS Papillon Park achieved strong overall user satisfaction according to the GUESS-18 composite scale, with strengths in usability, narrative, visual and audio aesthetics, and personal gratification.
Escapism into attractive virtual worlds featuring cohesive narrative storytelling and accessible gameplay, and a sense of personal growth and achievement, were identified as critical themes and key strengths of the gamified assessment tool produced in this study. Aesthetic and narrative characteristics are well-known to increase immersion and pleasure in gaming [144,145]. Furthermore, the gamification mechanisms we incorporated to foster personal accomplishment, including tasks with clear goals and rounds, points and performance feedback, in-game rewards, and visible depiction of achievements, are some of the most commonly used [59,65]. These characteristics were the most strongly endorsed in the qualitative and quantitative evaluations, confirming the new games to be broadly appealing and fulfilling, and reinforcing previous research demonstrating the value of aesthetics and gamification to user engagement and satisfaction with cognitive assessments [63,64,146]. These principles also align closely with the theoretical grounding of gamification [56,57,61], tapping into cognitive and emotional mechanisms to generate intrinsic motivation to engage with CIS Papillon Park [58,59]. In the context of cognitive assessments, engagement and intrinsic motivation serve critical instrumental purposes, encouraging uptake and continued, authentic effort, and thus valid assessment outcomes [45,46,62,66].
We also incorporated gamification elements that expand on those most used in mental health interventions, for example, the presence of virtual helpers [65]. Winston, Grandma Pixie, and the community NPCs reflected key ways in which our participants characterized mental health and positive functioning, including affirming interpersonal relationships and the pursuit of prosocial goals. Difficulties in these areas are well-documented features of EDs [147-152]. For example, socializing problems, social isolation, and loneliness have been shown to have a detrimental, bidirectional relationship with ED symptoms [152,153], while social connectedness may protect against ED onset and facilitate recovery [154-156]. As such, re-engaging with interpersonal relationships and meaningful life pursuits form common targets of harm reduction models of care and psychosocial recovery support for individuals living with EDs [157,158]. Within a gaming context, social interaction and collaboration toward mutual, prosocial goals are recognized as key not only to player engagement but also to realizing psychological benefits from gaming [58,59,66,159,160]. While real-time, multiplayer cooperation cannot readily be recreated within the context of cognitive assessment, evaluations suggested we had been successful in modeling meaningful collaboration and prosociality through reinforcing interactions with an endearing “side-kick,” although somewhat less successful in achieving this goal through interactions with members of a broader virtual community [161,162].
A key insight that emerged from the integration of qualitative and quantitative evaluation was the variability in user satisfaction between the 3 mini-games that form the CIS Papillon Park [143]. In total, 3 key elements distinguished user satisfaction—perceived autonomy and self-determination, variety, and level of difficulty. Our synthesized evaluation showed that gamification elements, including practice rounds, retries, a progress bar, and timer or speed, were key to achieving a level of difficulty that maximized user satisfaction [59,73]. Perceived lack of variety, autonomy, and creativity received lower quantitative ratings and inconsistent qualitative evaluations. These findings aligned closely with a previous study, where enjoyment helped people keep playing and maintained performance in easy games, while motivation facilitated performance and enjoyment in difficult games [163]. These findings reflect the inherent conflict between the repetition and standardization needed to achieve valid and reliable data in cognitive assessments, and user goals such as novelty and self-directed exploration that make games fun and engrossing, highlighting the difficulty of balancing the constraints of structured assessments with engaging game elements [62,63]. They also indicate that combining multiple gamification strategies, whereby elements enhance or compensate for one another, may be important in determining long-term user engagement [52,164].
User-centered co-design approaches are now commonly used to design mental health services and interventions leveraging web, mobile, and artificial intelligence technologies [48,74,78,99,100,165]. Less commonly, these approaches can be used to design and evaluate valid and reliable psychological assessment measures [42,88]. In developing the CIS Papillon Park, we took on board growing impetus to meaningfully incorporate LE perspectives throughout the research process [95,129]. Respectfully collaborating with individuals with lived and living experience in both design and evaluation is therefore a key strength of this study, underpinning its novel contribution to the field. Through evaluation, we were able to confirm user design priorities had been faithfully represented at all stages of the research process and were present in the final product [111,166]. Importantly, in addition to defining what to include in the new product, the co-designed guiding principles defined what not to include. It is important to note that some gamification elements might be harmful to certain populations, as exemplified by discussions concerning a leaderboard in this study [59]. Gamification is inherently experiential and subjective [54]; therefore, gamification and user-centered design work most effectively together, such that users define which gamification elements will be most safe and meaningful for them [167]. By following up on user-centered co-design with LE evaluations, we confirmed CIS Papillon Park maintained psychological safety for individuals with LEs of EDs and mental ill-health [168].
In terms of limitations, our co-design framework and qualitative methodologies meant we collaborated with relatively small groups of participants, and our quantitative evaluation analysis was somewhat underpowered. However, we nonetheless aimed that they would be representative of the target populations for the resulting assessment tool. We were therefore pleased to engage with groups that comprised varied current and historical experiences with disordered eating, that represented a broad range of experiences of commonly co-occurring mental ill-health, and that encompassed diverse identities and personal characteristics. These sentiments were shared by our participants, who endorsed the co-design groups and research processes as positive experiences, emphasizing their inclusivity [129,130]. This project was undertaken in Australia with participants of White European, White North American, and Asian ancestry; thus, its findings may be less relevant in other cultures. Our methodology incorporated various forms of expertise to achieve a practical goal of reimagining an existing cognitive assessment tool [101,104,116,169]. As such, we incorporate key principles of qualitative research but depart from certain canonical aspects of “Big Q” approaches [115,122]. Further methodological limitations include the challenges of integrating user-centered co-design and agile development in a resource-limited context [89,92,105,106]. In terms of the quantitative results, our ANOVA analyses were slightly underpowered, such that there is an elevated risk that we failed to detect differences in user satisfaction between groups.
Ultimately, this project intends to enhance the impact and reach of the CIS by facilitating its potential for clinical translation and adoption by end users [170]. Validated tools with streamlined administration and analysis pipelines that are well-accepted by users are urgently needed to support current moves toward transdiagnostic, dimensional, and individualized care, and address the significant treatment gap for individuals living with EDs and mental ill-health [171-174]. Importantly, such individuals who are experiencing cognitive challenges as part of their symptom profile are eager for these issues to be addressed in treatment, yet feel these needs are rarely met [35,175,176], experiences that were highlighted in our focus group discussions. Including objective assessment of cognitive impulsivity in a clinical context may thus allow practitioners to identify individuals who would most benefit from targeted cognitive interventions or augmented treatments, and to achieve data-driven personalization of treatment approaches [177-179]. Next steps will therefore involve a detailed psychometric evaluation of the CIS “Papillon Park,” confirming it maintains the properties of the CIS “Choices in the Wild West,” before dissemination. Additional qualitative research is also planned, engaging clinicians and LE participants as key stakeholders to co-design feedback reports and implementation strategies.
The unique contributions of this study include effectively blending user-centered co-design, agile game development, and theory-based cognitive assessment and gamification to reimagine a validated assessment tool. By following up on LE co-design with qualitative and quantitative evaluations, we identified a combination of aesthetics, gamification elements, and accessible playing experiences that maximized user satisfaction, engagement, and motivation while prioritizing psychological safety. This underscores the importance of gamified assessments in the mental health field providing opportunities for users to experience satisfaction of psychological needs, including autonomy, personal achievement, and social support. By effectively engaging users, CIS Papillon Park therefore holds promise to facilitate valid and reliable assessment of cognitive impulsivity in individuals experiencing EDs and common co-occurring mental health symptoms. Incorporating well-accepted objective cognitive assessment tools in routine clinical care is needed to support the individualization and prioritization of interventions, and thus support treatment engagement and recovery for these underserved conditions.
The authors declare that no generative artificial intelligence tools were used during the conduct of this research, nor in the preparation of this manuscript.
The authors wish to thank Noble Steed Games for their expertise and creativity in producing Cognitive Impulsivity Suite "Papillon Park" and acknowledge Torus Games for the original Cognitive Impulsivity Suite "Choices in the Wild West," and for their contributions to the focus group co-design process. EC thanks Assistant Prof Trevor Chong and Dr Justin Mahlberg for their guidance and support, and Kayla Flannagan for her assistance with participant recruitment. Finally, we extend our gratitude to the participants of this study for generously sharing their time and perspectives.
Funding
EC is supported by an Australian Research Training Program scholarship and by a TranslatED scholarship from the Australia Eating Disorders Research and Translation Centre funded by the Commonwealth Department of Health under the National Leadership in Mental Health Program. AVG is funded by an Australian National Health and Medical Research Council Investigators (grant 2009464), and the Australia Eating Disorders Research and Translation Centre funded by the Commonwealth Department of Health.
Data Availability
The deidentified datasets generated and analyzed during this study are available from the corresponding author on reasonable request.
EC: Conceptualization, methodology, participant recruitment and screening, focus group facilitation, data analysis, writing – original draft, writing – review and editing
CM: Data analysis, writing – review and editing
AA: Conceptualization, methodology, participant recruitment and screening, focus group facilitation, writing – review and editing
LH: focus group facilitation, participant support, writing – review and editing
AVG: Conceptualization, supervision, writing – review and editing
None declared.
AbbreviationsCIS
Cognitive Impulsivity Suite
ED
eating disorder
GUESS-18
Game User Experience Satisfaction Scale, short version
LE
lived experience
NPC
nonplayable character
NSG
Noble Steed Games
REDCap
Research Electronic Data Capture
TA
thematic analysis
ReferencesMacKillopJWeaferJC GrayJOshriAPalmerAde WitHThe latent structure of impulsivity: impulsive choice, impulsive action, and impulsive personality traitsPsychopharmacology (Berl)201609233183361337010.1007/s00213-016-4372-027449350NiggJTAnnual Research Review: on the relations among self-regulation, self-control, executive functioning, effortful control, cognitive control, impulsivity, risk-taking, and inhibition for developmental psychopathologyJ Child Psychol Psychiatry20170458436138310.1111/jcpp.1267528035675Verdejo-GarciaATiegoJKakoschkeNA unified online test battery for cognitive impulsivity reveals relationships with real-world impulsive behavioursNat Hum Behav2021115111562157710.1038/s41562-021-01127-334045720HirstRBBeardCLColbyKAQuittnerZMillsBMLavenderJMAnorexia nervosa and bulimia nervosa: a meta-analysis of executive functioningNeurosci Biobehav Rev2017128367869010.1016/j.neubiorev.2017.08.01128851577SmithKEMasonTBJohnsonJSLavenderJMWonderlichSAA systematic review of reviews of neurocognitive functioning in eating disorders: the state-of-the-literature and future directionsInt J Eat Disord20180851879882110.1002/eat.2292930102781HowardMGregertsenECHindochaCSerpellLImpulsivity and compulsivity in anorexia and bulimia nervosa: a systematic reviewPsychiatry Res20201129311335410.1016/j.psychres.2020.11335432781364StewardTMestre-BachGVintró-AlcarazCDelay discounting of reward and impulsivity in eating disorders: from anorexia nervosa to binge eating disorderEur Eat Disord Rev20171125660160610.1002/erv.254329057603KrugIDangABLuEOoiWLPortingaleJMilesSA narrative review on the neurocognitive profiles in eating disorders and higher weight individuals: insights for targeted interventionsNutrients202412231624441810.3390/nu1624441839771039SchwarzKMoessnangCSchweigerJITransdiagnostic prediction of affective, cognitive, and social function through brain reward anticipation in schizophrenia, bipolar disorder, major depression, and autism spectrum diagnosesSchizophr Bull2020041046359260210.1093/schbul/sbz07531586408Chavez-BaldiniUNiemanDHKeestraAThe relationship between cognitive functioning and psychopathology in patients with psychiatric disorders: a transdiagnostic network analysisPsychol Med20230153247648510.1017/S003329172100178134165065AbramovitchAShortTSchweigerAThe C factor: cognitive dysfunction as a transdiagnostic dimension in psychopathologyClin Psychol Rev2021068610200710.1016/j.cpr.2021.10200733864968SchretlenDJFinleyJCADel BeneVAVarvarisMThe ubiquity of cognitive impairment in human illness: a systematic review of meta-analysesArch Clin Neuropsychol2025052140486387710.1093/arclin/acae11339667720HiltonRATozziLNesamoneySTransdiagnostic neurocognitive dysfunction in children and adolescents with mental illnessNat Mental Health20242329930910.1038/s44220-023-00199-6East-RichardCR. -MercierANadeauDCellardCTransdiagnostic neurocognitive deficits in psychiatry: a review of meta-analysesCanadian Psychology / Psychologie canadienne202061319021410.1037/cap0000196FrancesconiMFlouriEHarrisonAChange in decision-making skills and risk for eating disorders in adolescence: a population-based studyEur Psychiatry20201013631e9310.1192/j.eurpsy.2020.9233046157Miranda-OlivosRTestaGLucasIClinical factors predicting impaired executive functions in eating disorders: the role of illness durationJ Psychiatr Res202112144879510.1016/j.jpsychires.2021.09.04234601380TestaGGraneroRMisiolekAImpact of impulsivity and therapy response in eating disorders from a neurophysiological, personality and cognitive perspectiveNutrients202211251423501110.3390/nu1423501136501041ÖzcanÖÖÇevreliBA comparative neuropsychological assessment of anorexia nervosa, bulimia nervosa, binge eating disorder and healthy control: a cross-sectional study in female university studentsBMC Psychol2025113131120910.1186/s40359-025-03515-041185048Hamady-SaadHEngel-YegerBA systematic review on executive functions in female adolescents with eating disorders: implications on daily life and relevance to rehabilitationJ Child Adolesc Ment Health20251136223726910.2989/17280583.2024.243836739840915BickelWKAthamnehLNBassoJCExcessive discounting of delayed reinforcers as a trans-disease process: update on the state of the scienceCurr Opin Psychol20191230596410.1016/j.copsyc.2019.01.00530852411FinebergNAChamberlainSRGoudriaanAENew developments in human neurocognition: clinical, genetic, and brain imaging correlates of impulsivity and compulsivityCNS Spectr201402191698910.1017/S109285291300080124512640BrooksSSchiothHHimmerichHJáuregui-LoberaIImpulsivity and compulsivity in anorexia nervosa: cognitive systems underlying variation in appetite restraint from an RDoC perspectiveAnorexia and Bulimia Nervosa2019IntechOpen978-1-78923-981-2ThomasKSBirchREJonesCRGVanderwertRENeural correlates of executive functioning in anorexia nervosa and obsessive-compulsive disorderFront Hum Neurosci20221684163310.3389/fnhum.2022.84163335693540GoschkeTDysfunctions of decision-making and cognitive control as transdiagnostic mechanisms of mental disorders: advances, gaps, and needs in current researchInt J Methods Psychiatr Res20140123 Suppl 1Suppl 1415710.1002/mpr.141024375535TchanturiaKLounesNHolttumSCognitive remediation in anorexia nervosa and related conditions: a systematic reviewEur Eat Disord Rev20141122645446210.1002/erv.232625277720DunlopKAWoodsideBDownarJTargeting neural endophenotypes of eating disorders with non-invasive brain stimulationFront Neurosci201610303010.3389/fnins.2016.0003026909013StamenovaVLevineBEffectiveness of goal management training® in improving executive functions: a meta-analysisNeuropsychol Rehabil20191229101569159910.1080/09602011.2018.143829429540124EichenDMMathesonBEAppleton-KnappSLBoutelleKNNeurocognitive treatments for eating disorders and obesityCurr Psychiatry Rep2017091996210.1007/s11920-017-0813-728744627HaynosAFAndersonLMAskewAJCraskeMGPetersonCBAdapting a neuroscience-informed intervention to alter reward mechanisms of anorexia nervosa: a novel direction for future researchJ Eat Disord20210526916310.1186/s40337-021-00417-534039415LevinsonCAChristianCBeckerCBHow idiographic methodologies can move the clinical-science field forward to integrate personalized treatment into everyday clinical care and improve treatment outcomesClin Psychol Sci202501131698210.1177/21677026231217316DalgleishTBlackMJohnstonDBevanATransdiagnostic approaches to mental health problems: current status and future directionsJ Consult Clin Psychol20200388317919510.1037/ccp000048232068421Sauer-ZavalaSGutnerCAFarchioneTJBoettcherHTBullisJRBarlowDHCurrent definitions of “transdiagnostic” in treatment development: a search for consensusBehav Ther20170148112813810.1016/j.beth.2016.09.00428077216HambletonAPepinGLeAPsychiatric and medical comorbidities of eating disorders: findings from a rapid review of the literatureJ Eat Disord202209510113210.1186/s40337-022-00654-236064606KavanaughBCCancilliereMKSpiritoANeurocognitive heterogeneity across the spectrum of psychopathology: need for improved approaches to deficit detection and interventionCNS Spectr20200625343644410.1017/S109285291900108131131779AllottKBryceSDouglasKStaintonAWoodSJBowieCRFormulating cognitive functioning to guide personalised treatment for people diagnosed with mental disordersActa Psychiatr Scand202605153557358710.1111/acps.1381139722475Martínez-PerníaDOlavarríaLFernández-ManjónBThe limitations and challenges in the assessment of executive dysfunction associated with real-world functioning: the opportunity of serious gamesAppl Neuropsychol Adult202532255757310.1080/23279095.2023.217443836827177MonticoneMGaleotoGBerardiATofaniMGaleotoGBerardiATofaniMAuxiliadora MarquezMPsychometric properties of assessment toolsMeasuring Spinal Cord Injury: A Practical Guide of Outcome Measures2021Springer International Publishing71510.1007/978-3-030-68382-5_2978-3030683825EnkaviAZEisenbergIWBissettPGLarge-scale analysis of test-retest reliabilities of self-regulation measuresProc Natl Acad Sci U S A20190319116125472547710.1073/pnas.181843011630842284HedgeCPowellGSumnerPThe reliability paradox: why robust cognitive tasks do not produce reliable individual differencesBehav Res Methods2018065031166118610.3758/s13428-017-0935-128726177HowiesonDCurrent limitations of neuropsychological tests and assessment proceduresClin Neuropsychol20190233220020810.1080/13854046.2018.155276230608020SalemATrierweilerCSde LinsEComputerized instruments for assessment of executive function: a scoping review with emphasis on game-based assessmentAppl Neuropsychol Adult2026012011610.1080/23279095.2026.261374041557642WileyKRobinsonRMandrykRLThe making and evaluation of digital games used for the assessment of attention: systematic reviewJMIR Serious Games202108993e2644910.2196/2644934383674KesselsRPCImproving precision in neuropsychological assessment: bridging the gap between classic paper-and-pencil tests and paradigms from cognitive neuroscienceClin Neuropsychol20190233235736810.1080/13854046.2018.151848930394172DeRightJJorgensenRSI just want my research credit: frequency of suboptimal effort in a non-clinical healthy undergraduate sampleClin Neuropsychol201529110111710.1080/13854046.2014.98926725494327SweetJJHeilbronnerRLMorganJEAmerican Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingeringClin Neuropsychol2021083561053110610.1080/13854046.2021.189603633823750Van DykeSAMillisSRAxelrodBNHanksRAAssessing effort: differentiating performance and symptom validityClin Neuropsychol20132781234124610.1080/13854046.2013.83544724028487BorghoutsJEikeyEMarkGBarriers to and facilitators of user engagement with digital mental health interventions: systematic reviewJ Med Internet Res20210324233e2438710.2196/2438733759801HickieIBDavenportTABurnsJMProject Synergy: co-designing technology-enabled solutions for Australian mental health services reformMed J Aust201910211 Suppl 7S7S3S3910.5694/mja2.5034931587276LyonARBrunsEJUser-centered redesign of evidence-based psychosocial interventions to enhance implementation-hospitable soil or better seeds?JAMA Psychiatry20190117613410.1001/jamapsychiatry.2018.306030427985SchuellerSMNearyMO’LoughlinKAdkinsECDiscovery of and interest in health apps among those with mental health needs: survey and focus group studyJ Med Internet Res20180611206e1014110.2196/1014129891468TorousJNicholasJLarsenMEFirthJChristensenHClinical review of user engagement with mental health smartphone apps: evidence, theory and improvementsEvid Based Ment Health20180821311611910.1136/eb-2018-10289129871870LumsdenJSkinnerACoyleDLawrenceNMunafoMAttrition from web-based cognitive testing: a repeated measures comparison of gamification techniquesJ Med Internet Res201711221911e39510.2196/jmir.847329167090PedersenMKDíazCMCWangQJMeasuring cognitive abilities in the wild: validating a population-scale game-based cognitive assessmentCogn Sci202306476e1330810.1111/cogs.1330837354036HuotariKHamariJA definition for gamification: anchoring gamification in the service marketing literatureElectron Markets201702271213110.1007/s12525-015-0212-zWarsinskySSchmidt-KraepelinMRankSThiebesSSunyaevAConceptual ambiguity surrounding gamification and serious games in health care: literature review and development of Game-Based Intervention Reporting Guidelines (GAMING)J Med Internet Res20210910239e3039010.2196/3039034505840RyanRMDeciELSelf-determination theory and the facilitation of intrinsic motivation, social development, and well-beingAm Psychol200001551687810.1037//0003-066x.55.1.6811392867KrathJSchürmannLvon KorfleschHFORevealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learningComput Human Behav20211212510696310.1016/j.chb.2021.106963XiNHamariJDoes gamification satisfy needs? A study on the relationship between gamification features and intrinsic need satisfactionInt J Inf Manage2019064621022110.1016/j.ijinfomgt.2018.12.002KoivistoJHamariJThe rise of motivational information systems: a review of gamification researchInt J Inf Manage2019044519121010.1016/j.ijinfomgt.2018.10.013RyanRMRigbyCSPrzybylskiAThe motivational pull of video games: a self-determination theory approachMotiv Emot2006121230434436010.1007/s11031-006-9051-8CsikszentmihalyiMFlow: The Psychology of Optimal Experience1990Harper & Row978-0061339202KhaleghiAAghaeiZMahdaviMAA gamification framework for cognitive assessment and cognitive training: qualitative studyJMIR Serious Games2021051892e2190010.2196/2190033819164VermeirJFWhiteMJJohnsonDCrombezGVan RyckeghemDMLThe effects of gamification on computerized cognitive training: systematic review and meta-analysisJMIR Serious Games2020081083e1864410.2196/1864432773374ChetitahMMüllerJDesernoLWaltmannMvon MammenSGamification framework for reinforcement learning-based neuropsychology experiments20230412FDG 2023Apr 12-14, 202310.1145/3582437.3587190ChengVWSDavenportTJohnsonDVellaKHickieIBGamification in apps and technologies for improving mental health and well-being: systematic reviewJMIR Ment Health2019062666e1371710.2196/1371731244479HassanLDiasAHamariJHow motivational feedback increases user’s benefits and continued use: a study on gamification, quantified-self and social networkingInt J Inf Manage2019064615116210.1016/j.ijinfomgt.2018.12.004LumsdenJEdwardsEALawrenceNSCoyleDMunafòMRGamification of cognitive assessment and cognitive training: a systematic review of applications and efficacyJMIR Serious Games2016071542e1110.2196/games.588827421244LongBSimsonJBuxó-LugoAWatsonDGMehrSAHow games can make behavioural science betterNature New Biol202301613794443343610.1038/d41586-023-00065-636650244MirandaATPalmerEMIntrinsic motivation and attentional capture from gamelike features in a visual search taskBehav Res Methods20140346115917210.3758/s13428-013-0357-723835649TiegoJVerdejo-GarciaAAndersonAKoutoulogenisJBellgroveMAMechanisms of cognitive disinhibition explain individual differences in adult attention deficit hyperactivity disorder traitsCortex20231016717819610.1016/j.cortex.2023.06.01337567053RajKSegraveRVerdéjo-GarciaAYücelMCognitive inflexibility and repetitive habitual actions are associated with problematic use of the internetAddict Behav20230413910760010.1016/j.addbeh.2022.10760036608592MandrykRLBirkMVToward game-based digital mental health interventions: player habits and preferencesJ Med Internet Res20170420194e12810.2196/jmir.690628428161MantellRHwangYIJDarkMEvaluating the user experience and usability of game-based cognitive assessments for older people: systematic reviewJMIR Aging202506118e6525210.2196/6525240499156GrahamAKWildesJEReddyMMunsonSABarr TaylorCMohrDCUser-centered design for technology-enabled services for eating disordersInt J Eat Disord20191052101095110710.1002/eat.2313031313370MasonJPownallMPalmerAAzevedoFInvestigating lay perceptions of psychological measures: a registered reportSoc Psychol Bull2023111813210.32872/spb.9383LinardonJShatteAMesserMFirthJFuller-TyszkiewiczME-mental health interventions for the treatment and prevention of eating disorders: an updated systematic review and meta-analysisJ Consult Clin Psychol2020118811994100710.1037/ccp000057532852971Abu-OdahHSaidNBNairSCIdentifying barriers and facilitators of translating research evidence into clinical practice: a systematic review of reviewsHealth Soc Care Community202211306e3265e327610.1111/hsc.1389835775332JarmanHKMcLeanSARodgersRInforming mHealth and web-based eating disorder interventions: combining lived experience perspectives with design thinking approachesJMIR Form Res20221031610e3838710.2196/3838736315225SchleiderJLThe fundamental need for lived experience perspectives in developing and evaluating psychotherapiesJ Consult Clin Psychol20230391311912110.1037/ccp000079836795434AxelssonMSchønningVBocktingCLived experiences: a focus group pilot study within the MentALLY project of mental healthcare among European usersBMC Health Serv Res202007120160510.1186/s12913-020-05454-532611345PalmerVJWeavellWCallanderRThe participatory zeitgeist: an explanatory theoretical model of change in an era of coproduction and codesign in healthcare improvementMed Humanit20190945324725710.1136/medhum-2017-01139829954854HawkeLDSheikhanNYBastidas-BilbaoHRodakTExperience-based co-design of mental health services and interventions: a scoping reviewSSM - Mental Health202406510030910.1016/j.ssmmh.2024.100309ObeidNMcVeyGSealeEPreskowWNorrisMLCocreating research priorities for anorexia nervosa: the Canadian eating disorder priority setting partnershipInt J Eat Disord20200553539240210.1002/eat.2323432011022AouadPHambletonAMarksPSetting the top 10 eating disorder research and translation priorities for AustraliaAust N Z J Psychiatry2023095791281129110.1177/0004867422112875436196038Papastavrou BrooksCKafleEButtNCo-producing principles to guide health research: an illustrative case study from an eating disorder research clinicRes Involv Engagem20230920918410.1186/s40900-023-00460-337730642MaguireSPepinGCalvertSLet’s Co! The how-to of working within a co-production framework to devise, design and execute collaborative multisite clinical trialsBr J Clin Psychol2026012210.1111/bjc.7001741567105GöttgensIOertelt-PrigioneSThe application of human-centered design approaches in health research and innovation: a narrative review of current practicesJMIR Mhealth Uhealth2021126912e2810210.2196/2810234874893TzimourtaKDHuman-centered design and development in digital health: approaches, challenges, and emerging trendsCureus202506176e8589710.7759/cureus.8589740548156MohammedAHAbdul WahabNIbrahimNEnhancing mHealth applications through user-centred design: a conceptual framework for improved healthcare accessibility and user experienceJCRINN2025101152410.24191/jcrinn.v10i1.491BerryNMachinMAinsworthJDeveloping a theory-informed smartphone app for early psychosis: learning points from a multidisciplinary collaborationFront Psychiatry20201160286110.3389/fpsyt.2020.60286133362612BlandfordAGibbsJNewhouseNPerskiOSinghAMurrayESeven lessons for interdisciplinary research on interactive digital health interventionsDigit Health20184205520761877032510.1177/205520761877032529942629DuffyABoroumandzadNShermanALChristieGRiadiIMorenoSExamining challenges to co-design digital health interventions with end users: systematic reviewJ Med Internet Res2025031427e5017810.2196/5017840085834NealeJBowenAMLessons for uptake and engagement of a smartphone app (SURE Recovery) for people in recovery from alcohol and other drug problems: interview study of app usersJMIR Hum Factors202203891e3303810.2196/3303835258474VoorheisPPetchJPhamQKuluskiKMaximizing the value of patient and public involvement in the digital health co-design process: a qualitative descriptive study with design leaders and patient-public partnersPLOS Digit Health202310210e000021310.1371/journal.pdig.000021337878566SlatteryPSaeriAKBraggePResearch co-design in health: a rapid overview of reviewsHealth Res Policy Syst202002111811710.1186/s12961-020-0528-932046728KauppinenKKeikhosrokianiPKhanSHuman-centered design and benefit realization management in digital health care solution development: protocol for a systematic reviewJMIR Res Protoc2024052113e5612510.2196/5612538772023SuomiAFreemanBBanfieldMFramework for the Engagement of People with a Lived Experience in Program Implementation and Research2017Black Dog InstituteBonessCLHebdenHLardierDTEngaging people with lived experience of psychological disorders: current research and future directions for community-engaged measure development in psychological scienceClin Psychol Sci20250713472073910.1177/2167702624130433940746401MoserAKorstjensISeries: practical guidance to qualitative research. Part 5: co-creative qualitative approaches for emerging themes in primary care research: experience-based co-design, user-centred design and community-based participatory researchEur J Gen Pract20221228111210.1080/13814788.2021.201070035037811AnQKelleyMMHannersAYenPYSustainable development for mobile health apps using the human-centered design processJMIR Form Res202308257e4569410.2196/4569437624639MorleyCJoseKHallSEEvidence-informed, experience-based co-design: a novel framework integrating research evidence and lived experience in priority-setting and co-design of health servicesBMJ Open2024088148e08462010.1136/bmjopen-2024-08462039122385CreswellJWA Concise Introduction to Mixed Methods Research2021SAGE publications1544355777KelleUCombining qualitative and quantitative methods in research practice: purposes and advantagesQual Res Psychol20063429331110.1177/1478088706070839Shannon-BakerPMaking paradigms meaningful in mixed methods researchJ Mix Methods Res20161010431933410.1177/1558689815575861FaraoJMalilaBConradNMutsvangwaTRangakaMXDouglasTSA user-centred design framework for mHealthPLOS ONE2020158e023791010.1371/journal.pone.023791032813711CocktonGLárusdóttirMGregoryPCajanderÅCocktonGLárusdóttirMGregoryPCajanderÅIntegrating user-centred design in agile developmentIntegrating User-Centred Design in Agile Development2016Springer International Publishing146Ergonomics of Human-System Interaction Part 210: Human-Centred Design for Interactive Systems2019ISOShenNCo-approaches in mental health need conceptual clarityNat Rev Psychol2026511210.1038/s44159-025-00518-xVargasCWhelanJBrimblecombeJAllenderSCo-creation, co-design, co-production for public health - a perspective on definition and distinctionsPublic Health Res Pract20220615322322221110.17061/phrp322221135702744GrahamAKMunsonSAReddyMIntegrating user-centered design and behavioral science to design a mobile intervention for obesity and binge eating: mixed methods analysisJMIR Form Res2021051055e2380910.2196/2380933970114MusićSElwynRFountasGValuing the voice of lived experience of eating disorders in the research process: benefits and considerationsAust N Z J Psychiatry20220356321621810.1177/000486742199879433715452LignouSCapitaoLHamer-HuntJMSinghICo-production: an ethical model for mental health research?Am J Bioeth201908198495110.1080/15265161.2019.161987731544640LincolnYSGubaEGDenzinNKLincolnYSParadigmatic controversies, contradictions, and emerging confluences, revisitedThe Sage Handbook of Qualitative Research20114Sage Publications97128978-1412974172BraunVClarkeVUsing thematic analysis in psychologyQual Res Psychol200601327710110.1191/1478088706qp063oaBraunVClarkeVToward good practice in thematic analysis: Avoiding common problems and be(com)ing a knowing researcherInt J Transgend Health20232411610.1080/26895269.2022.212959736713144ProudfootKInductive/deductive hybrid thematic analysis in mixed methods researchJ Mix Methods Res20230717330832610.1177/15586898221126816LevittHMBambergMCreswellJWFrostDMJosselsonRSuárez-OrozcoCJournal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force reportAm Psychol201801731264610.1037/amp000015129345485PepinGBarrandSCalvertSQuick start guide for co-production in eating disorders researchA national project funded by the Australian Government, Department of Health and Aged Care20252026-05-27Australian Eating Disorder Research & Translation Centrehttps://insideoutresourcelibrary.s3.ap-southeast-2.amazonaws.com/Quick+Start+Guide+for+Co+Production.pdfMaximRIArnedo-MorenoJIdentifying key principles and commonalities in digital serious game design frameworks: scoping reviewJMIR Serious Games202503513e5407510.2196/5407540053743Olmos-VegaFMStalmeijerREVarpioLKahlkeRA practical guide to reflexivity in qualitative research: AMEE Guide No. 149Med Teach202204745311110.1080/0142159X.2022.205728735389310BraunVClarkeVCan I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern‐based qualitative analytic approachesCouns and Psychother Res202103211374710.1002/capr.12360BraunVClarkeVOne size fits all? What counts as quality practice in (reflexive) thematic analysisQual Res Psychol202107318332835210.1080/14780887.2020.1769238WutichABeresfordMBernardHRSample sizes for 10 types of qualitative data analysis: an integrative review, empirical guidance, and next stepsInt J Qual Methods202401231609406924129620610.1177/16094069241296206BraunVClarkeVConceptual and design thinking for thematic analysisQualitative Psychology20229132610.1037/qup0000196MoserAKorstjensISeries: practical guidance to qualitative research. Part 3: sampling, data collection and analysisEur J Gen Pract20181224191810.1080/13814788.2017.137509129199486BraunVClarkeVIs thematic analysis used well in health psychology? A critical review of published research, with recommendations for quality practice and reportingHealth Psychol Rev20231217469571810.1080/17437199.2022.216159436656762HarrisPATaylorRMinorBLThe REDCap consortium: building an international community of software platform partnersJ Biomed Inform2019079510320810.1016/j.jbi.2019.10320831078660HarrisPATaylorRThielkeRPayneJGonzalezNCondeJGResearch Electronic Data Capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics supportJ Biomed Inform20090442237738110.1016/j.jbi.2008.08.01018929686BurnsCKilkennyMFPurvisTProcesses underpinning successful co-design: lessons from a digital health projectHealth Expect202506283e7027210.1111/hex.7027240405447PetersSGuccioneLFrancisJEvaluation of research co-design in health: a systematic overview of reviews and development of a frameworkImplement Sci202409111916310.1186/s13012-024-01394-439261956TremblayMHamelCViau-GuayAGirouxDUser experience of the co-design research approach in eHealth: activity analysis with the course-of-action frameworkJMIR Hum Factors202208993e3557710.2196/3557735943783BraunVClarkeVDenzinNKThematic Analysis20236Sage Publications385401NVivo 14 editionLumivero20232026-05-27https://lumivero.com/resources/free-trial/nvivo/Advanced graphics and image-processing in RCRAN20252026-05-27https://cran.r-project.org/web/packages/magick/index.htmlKeeblerJRShelstadWJSmithDCChaparroBSPhanMHValidation of the GUESS-18: a short version of the Game User Experience Satisfaction Scale (GUESS)J Usability Stud2020161496210.5555/3532747.3532752PhanMHKeeblerJRChaparroBSThe development and validation of the Game User Experience Satisfaction Scale (GUESS)Hum Factors2016125881217124710.1177/001872081666964627647156FettersMDCurryLACreswellJWAchieving integration in mixed methods designs-principles and practicesHealth Serv Res201312486 Pt 22134215610.1111/1475-6773.1211724279835FettersMDFreshwaterDPublishing a methodological mixed methods research articleJ Mix Methods Res2015079320321310.1177/1558689815594687JørgensenAHThinking-aloud in user interface design: a method promoting cognitive ergonomicsErgonomics19900433450150710.1080/00140139008927157ChartersEThe use of think-aloud methods in qualitative research an introduction to think-aloud methodsBrock Education Journal200312210.26522/brocked.v12i2.38NVivo 15 editionLumivero20252026-05-27https://tinyurl.com/bdcvdmybAllaireJJRStudio: integrated development environment for R20252026-05-27Posit Softwarehttps://www.r-project.org/conferences/useR-2011/abstracts/180111-allairejj.pdfFettersMDFreshwaterDThe 1 + 1 = 3 integration challengeJ Mix Methods Res2015049211511710.1177/1558689815581222MillsKSpeaking of psychology: how video games can help kids learn and grow, with Susan [Podcast]20242026-05-27American Psychological Associationhttps://www.apa.org/news/podcasts/speaking-of-psychology/video-games-kids-learnLazzaroNWhy we play games: the four keys to player experienceXEODesign Inc20042026-05-27https://media.gdcvault.com/gdc04/slides/why_we_play_games.pdfKimHKAttraction and achievement as 2 attributes of gamification in healthcare: an evolutionary concept analysisJ Educ Eval Health Prof2024211010.3352/jeehp.2024.21.1038600768HayPAouadPLeAEpidemiology of eating disorders: population, prevalence, disease burden and quality of life informing public policy in Australia-a rapid reviewJ Eat Disord202302151112310.1186/s40337-023-00738-736793104HayPMitchisonDColladoAELGonzález-ChicaDAStocksNTouyzSBurden and health-related quality of life of eating disorders, including avoidant/restrictive food intake disorder (ARFID), in the Australian populationJ Eat Disord2017512110.1186/s40337-017-0149-z28680630WinklerLADChristiansenELichtensteinMBHansenNBBilenbergNStøvingRKQuality of life in eating disorders: a meta-analysisPsychiatry Res2014093021911910.1016/j.psychres.2014.05.00224857566JenkinsPEHosteRRMeyerCBlissettJMEating disorders and quality of life: a review of the literatureClin Psychol Rev20110231111312110.1016/j.cpr.2010.08.00320817335RiegerEVan BurenDJBishopMTanofsky-KraffMWelchRWilfleyDEAn eating disorder-specific model of interpersonal psychotherapy (IPT-ED): causal pathways and treatment implicationsClin Psychol Rev20100630440041010.1016/j.cpr.2010.02.00120227151RaykosBCMcEvoyPMFurslandASocializing problems and low self-esteem enhance interpersonal models of eating disorders: evidence from a clinical sampleInt J Eat Disord2017095091075108310.1002/eat.2274028649729LevineMPLoneliness and eating disordersJ Psychol20121461-224325710.1080/00223980.2011.60643522303623LinvilleDBrownTSturmKMcDougalTEating disorders and social support: perspectives of recovered individualsEat Disord201220321623110.1080/10640266.2012.66848022519898MakriEMichopoulosIGonidakisFInvestigation of loneliness and social support in patients with eating disorders: a case-control studyPsychiatry International20223214215710.3390/psychiatryint3020012UzunianLGVitalleM de SSocial skills: a factor of protection against eating disorders in adolescentesCien Saude Colet20151120113495350810.1590/1413-812320152011.1836201426602727WilliamsKDDobneyTGellerJSetting the eating disorder aside: an alternative model of careEur Eat Disord Rev201003182909610.1002/erv.98920099264HayPChinnDForbesDRoyal Australian and New Zealand College of psychiatrists clinical practice guidelines for the treatment of eating disordersAust N Z J Psychiatry2014114811977100810.1177/000486741455581425351912GranicILobelAEngelsRThe benefits of playing video gamesAm Psychol201401691667810.1037/a003485724295515BassanelliSVastaNBucchiaroneAMarconiAGamification for behavior change: a scientometric reviewActa Psychol (Amst)20220822810365710.1016/j.actpsy.2022.10365735767927BombariDSchmid MastMCanadasEBachmannMStudying social interactions through immersive virtual environment technology: virtues, pitfalls, and future challengesFront Psychol2015686910.3389/fpsyg.2015.0086926157414ChenJYCFragomeniGA mixed reality based social interactions testbed: a game theory approach20182026-05-2710th International Conference, VAMR 2018, Held as Part of HCI International 2018Jul 15-20, 2018Springer International Publishing AG4056https://link.springer.com/book/10.1007/978-3-319-91584-5HuYZUrakamiJWeiHTVombergLHChignellMLongitudinal analysis of sustained performance on gamified cognitive assessment tasksAppl Neuropsychol Adult202431450252610.1080/23279095.2022.203993135196467FriehsMADechantMVedressSFringsCMandrykRLEffective gamification of the stop-signal task: two controlled laboratory experimentsJMIR Serious Games202009883e1781010.2196/1781032897233SharpGDwyerBXieJCo-design of a single session intervention chatbot for people on waitlists for eating disorder treatment: a qualitative interview and workshop studyJ Eat Disord202503111314610.1186/s40337-025-01225-x40069853PhillipouACalvertSde BoerKLived experience-informed eating disorders research: an illustrative exampleEat Disord202634328129210.1080/10640266.2025.247122040014421FitzgeraldMRatcliffeGSerious games, gamification, and serious mental illness: a scoping reviewPsychiatr Serv202002171217018310.1176/appi.ps.20180056731640521TaherRBhanushaliPAllanSBridging the gap from medical to psychological safety assessment: consensus study in a digital mental health contextBJPsych Open2024063104e12610.1192/bjo.2024.71338828683DownsJBeyond the methodological binary: coproduction as the third pillar of mental health scienceBMJ Ment Health2025099281e30180710.1136/bmjment-2025-30180740930753van Dijk-de VriesAStevensAvan der WeijdenTBeurskensAHow to support a co-creative research approach in order to foster impact. The development of a co-creation impact compass for healthcare researchersPLoS ONE20201510e024054310.1371/journal.pone.024054333045033LevinsonCACusackCHuntRAFitterman-HarrisHFRalph-NearmanCHooperSThe future of the eating disorder field: inclusive, aware of systems, and personalizedBehav Res Ther20241218310464810.1016/j.brat.2024.10464839486192LevinsonCAKapadiaASandoval-AraujoLEVanzhulaIAMachryKMovement toward dimensional symptom models of comorbid obsessive-compulsive disorder and eating disordersAnnu Rev Clin Psychol20250521140743810.1146/annurev-clinpsy-081423-02083139929546BryantEMarksPGriffithsKTreating the individual: moving towards personalised eating disorder careJ Eat Disord20250491316310.1186/s40337-025-01246-640205596RobbinsTWGillanCMSmithDGde WitSErscheKDNeurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatryTrends Cogn Sci201201161819110.1016/j.tics.2011.11.00922155014BryceSChengNDaltonACognitive health treatment priorities and preferences among young people with mental illness: the your mind, your choice surveyEarly Interv Psychiatry2024021829410110.1111/eip.1343637198726MoritzSBernaFJaegerSWestermannSNagelMThe customer is always right? Subjective target symptoms and treatment preferences in patients with psychosisEur Arch Psychiatry Clin Neurosci201706267433533910.1007/s00406-016-0694-527194554LevinsonCAHuntRAKeshishianACUsing individual networks to identify treatment targets for eating disorder treatment: a proof-of-concept study and initial dataJ Eat Disord20211149114710.1186/s40337-021-00504-734736538Segura-SerraltaMCiscarSBlascoLContribution of executive functions to eating behaviours in obesity and eating disordersBehav Cogn Psychother20201148672573310.1017/S135246582000026032329428SchmidtUCampbellICTreatment of eating disorders can not remain “brainless”: the case for brain-directed treatmentsEur Eat Disord Rev20131121642542710.1002/erv.225724123463
Supplementary materials containing images of the original CIS games, researcher positionality statements, the Game User Experience Satisfaction Scale, short version survey, focus group question guides, and ideas generated during the ideation and design phase.