This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.
Cognitive behavioral therapy is an efficacious treatment for child anxiety disorders. Although efficacious, many children (40%-50%) do not show a significant reduction in symptoms or full recovery from primary anxiety diagnoses. One possibility is that they are unwilling to learn and practice cognitive behavioral therapy skills beyond therapy sessions. This can occur for a variety of reasons, including a lack of motivation, forgetfulness, and a lack of cognitive behavioral therapy skills understanding. Mobile health (mHealth) gamification provides a potential solution to improve cognitive behavioral therapy efficacy by delivering more engaging and interactive strategies to facilitate cognitive behavioral therapy skills practice in everyday lives (in vivo).
The goal of this project was to redesign an existing mHealth system called SmartCAT (Smartphone-enhanced Child Anxiety Treatment) so as to increase user engagement, retention, and learning facilitation by integrating gamification techniques and interactive features. Furthermore, this project assessed the effectiveness of gamification in improving user engagement and retention throughout posttreatment.
We redesigned and implemented the SmartCAT system consisting of a smartphone app for children and an integrated clinician portal. The gamified app contains (1) a series of interactive games and activities to reinforce skill understanding, (2) an in vivo skills coach that cues the participant to use cognitive behavioral therapy skills during real-world emotional experiences, (3) a home challenge module to encourage home-based exposure tasks, (4) a digital reward system that contains digital points and trophies, and (5) a therapist-patient messaging interface. Therapists used a secure Web-based portal connected to the app to set up required activities for each session, receive or send messages, manage participant rewards and challenges, and view data and figures summarizing the app usage. The system was implemented as an adjunctive component to brief cognitive behavioral therapy in an open clinical trial. To evaluate the effectiveness of gamification, we compared the app usage data at posttreatment with the earlier version of SmartCAT without gamification.
Gamified SmartCAT was used frequently throughout treatment. On average, patients spent 35.59 min on the app (SD 64.18) completing 13.00 activities between each therapy session (SD 12.61). At the 0.10 significance level, the app usage of the gamified system (median 68.00) was higher than that of the earlier, nongamified SmartCAT version (median 37.00,
The gamified system showed good acceptability, usefulness, and engagement among anxious children receiving brief cognitive behavioral therapy treatment. Integrating an mHealth gamification platform within treatment for anxious children seems to increase involvement in shorter treatment. Further study is needed to evaluate increase in involvement in full-length treatment.
The results of multiple independent randomized clinical trials provide evidence that cognitive behavioral therapy (CBT) is an efficacious treatment for childhood anxiety [
Although homework is routinely assigned, many anxious children struggle with homework completion [
Mobile health (mHealth) technologies present potential solutions to overcoming barriers to fostering home-based skills practice for children. First, the “always-carried” and “always-on” nature of smartphones creates an opportunity to deliver CBT interventions to children in natural settings during their everyday lives, an approach referred to as “ecological momentary intervention” (EMI) [
Despite this potential, the repetitive tasks (eg, self-monitoring and self-management) that characterize most mHealth apps can be exhausting and may lack intrinsic rewards [
The purpose of this project was multifaceted. First, we redesigned our existing mHealth system, titled “Smartphone-enhanced Child Anxiety Treatment” (SmartCAT), consisting of a smartphone app (SmartCAT app), a therapist portal (SmartCAT portal), and a two-way communication connecting them [
A user-centered design (UCD) approach was used to gather requirements and iteratively design the system, leveraging the SmartCAT 1.0 system that had been previously pilot-tested [
Although the skills coach in SmartCAT 1.0 was actively used, averaging 5.36 times per session, children and therapists suggested several potential improvements including developing more interactive and fun ways for the children to learn and practice CBT skills in daily life and also improving rewards to increase the rates of CBT skills practice. On the basis of this feedback, we redesigned and developed a new system using the iterative, user-centered approach described below.
The SmartCAT addresses barriers to home-based skills practice for children by (1) Providing automatic cues to children to practice skills at prescribed times and places, even when they forget to initiate skills practice on their own; (2) Motivating children to practice skills; and (3) Providing interactive ways to learn the skills and offering
The UCD process was conducted in three steps. The initial step of this UCD process was the development of design principles based on user information captured and interpreted by therapists that deliver CBT to anxious children. The therapists served as the interface between the users (ie, anxious children) and the designer or software developer. Meetings with therapists were conducted to brainstorm and identify design ideas and criteria. Such ideas as including interactive features, treatment engagement and adherence, and educational content were addressed. These design ideas and criteria were then translated into design principles, which were in turn used to evaluate the system. The results from the design principles development step provided general guidelines for implementation by software developers for the iterative system development step. Continuous input or feedback was provided by two therapists during the system development process. A formative usability study involving the children in the study was conducted following the system development process to collect feedback and discover usability problems. Revisions were made before the system was implemented as part of the clinical trial.
The initial UCD process revealed a conceptual model for the system that includes seven key components. By implementing these components in the system, CBT treatment outcomes can hopefully be improved. The individual and specific components of the system are outlined as follows:
The CBT components of the model were translated into several skill-builder modules that include an in vivo skills coach, a series of interactive games and activities to reinforce skill understanding, and a home challenge module to encourage home-based exposure (
Gamification aims to increase people’s engagements in real life activities and encourage specific human behaviors. To some extent, the concept is already being used in manual-based CBT treatment such as the Coping Cat program [
Recent advances in interactive mHealth technologies allow gamification concepts to be layered on top of activities provided by mobile apps. “Swarm” app, eg, rewards its users for checking into a new place by giving digital coins, badges, stickers, and statuses. These game mechanics serve dual functions—helping users learn to use the app and making a real-world experience more engaging. Digital coins and badges give the users a sense of accomplishment, whereas status changes such as “mayorships” allow users to compete with their friends.
In this project, the system was gamified so as to drive children’s engagement in completing their weekly skill-builder modules via an iterative process consisting of four steps. These steps are as follows:
Identify the end goals
Determine interesting activities to move patients toward the end goals: identify activities that are aligned with the goals. The activities should also capture the interest of the person. From a self-determination theory perspective, interest can be defined as an affect that occurs in the interaction between a person and an activity [
Competence: the need of people to gain mastery of tasks and learn different skills. When people feel that they have skill or expertise at doing something, they will be more likely to continue doing it. Opportunities to learn different skills, or be optimally challenged, can also improve a person’s level of competency [
Autonomy: refers to the need to feel in control when performing activities or tasks. The core concept of autonomy is freedom. Allowing individuals freedom in choosing has been shown to improve autonomy and, consequently, their intrinsic motivation [
Relatedness or connection: refers to the need to feel connected to others. People tend to internalize and accept values and practices from those to whom they feel connected and from contexts in which they experience a sense of belonging. Providing a possibility of social connectedness that conveys security can strengthen intrinsic motivation [
Although interest plays a central role in intrinsic motivation, it is not central to all motivated behavior. People often engage in instrumental activities for some desired outcome not related to the activity itself (being extrinsically motivated). External rewards such as points, money, gift cards, toys, or something tangible can motivate people to complete tasks. For gamification to truly motivate people, it has to target correct and intrinsically motivated activities, as well as provide external rewards for completing the activities [
Apply game design elements to improve user experience: key elements of gamification are applied to make activities feel more “playful.”
Evaluate effectiveness: depending on the goals defined in the initial step, gathering quantitative or qualitative data can assess the effectiveness of gamification. Quantitative data that includes engagement (time spent using the app, the number of digital points collected) and retention (the number of features completed between sessions) can be used to infer user behavior directly. Qualitative data such as user feedback, comments, concerns, frustrations, and suggestions can capture perceptions and attitudes toward gamified apps.
A total of 35 participants (aged 9-14 years; mean=11.19) met the criteria for the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-V) diagnosis of generalized anxiety disorder, social anxiety disorder, and/or separation anxiety disorder. These diagnoses are common in children, frequently cooccur, have a similar presentation, and respond to the same treatment approaches [
The participants included 5 participants enrolled in a beta testing phase and 30 participants enrolled in an open trial phase who completed treatment. The participants who enrolled in the beta testing phase received similar treatment to those who enrolled in the open trial phase.
Skill-builder modules
Module | Session | Description |
Skills coach | 1, 2, 3, 4, 5, 6, 7 | Guide the participant through developing a FEAR plan for a current or recent in vivo anxious experience. |
What’s the feeling? (game) | 1, 2, 3, 4a, 5a, 6a, 7a | Ask the participant to identify emotional and somatic symptoms from various scenarios (including anxiety, physical pain, and hunger). |
Chillax | 1, 2a, 3a, 4a, 5a, 6a, 7a | View or practice with a video demonstrating deep breathing techniques. |
Listen or practice with an mp3 audio file for progressive muscle relaxation. | ||
Thought-buster (game) | 2, 3a, 4, 5, 6, 7 | Ask the participant to identify anxious vs nonanxious self-talk or coping vs noncoping self-talk. |
Thought-swapper (game) | 3, 4, 5, 6, 7 | Ask the participant to identify coping self-talk that works best in a given situation. |
Problem-solver (game) | 3, 4, 5, 6, 7 | Generate and evaluate potential solutions to hypothetical problems. |
Challenger | 4, 5, 6, 7 | Therapist selects personally relevant home challenges from a menu on the portal; patient is prompted to develop a FEAR plan and complete these challenges via app. |
Show that I can | 1, 2, 3, 4, 5, 6, 7 | Therapist selects weekly task (adapted from the Coping Cat workbook) from a menu on the portal; patient is prompted to complete the task via app. |
aOptional.
Intrinsic and extrinsic motivators in target activities.
Activities | Intrinsic motivators | Extrinsic motivators |
Completing interactive skill-building modules (“What’s the feeling?”, Thought-buster, Thought-swapper, Problem-solver) | Specific modules are assigned for a particular session. As the session progresses, different modules with different challenges will be assigned (competence) | Tangible prizes (ie, accessories and makeup, small toys and games, and gift cards for older teens) |
Each module can be initiated independently (autonomy) | ||
Completing skills coach | As the session progresses, children are asked to come up with their own coping strategies instead of choosing from a provided checklist (competence and autonomy) | |
Completing at-home challenges (Challenger), Chillax, and Show that I can task | At-home challenges are discussed with the therapist in face-to-face sessions. Children can choose which challenges they want to complete (competence and autonomy). | |
Sending or replying to messages | Children can send messages to their therapist to ask therapeutic questions (relatedness or connection) | Attention, praise |
Actions, components, and mechanics
Actions | Components | Mechanics |
Initiate and complete skill-builder modules when requested to do so by app alarm | One point toward the target number of points (cumulative) | Collect a certain number of points. Therapists will assign the target points needed to redeem a selected prize. A collection of stars and a trophy will be displayed on the home screen. A progress bar and badges are displayed after the completion of actions. |
Initiate and complete skill-builder modules from within the app (on one’s own initiative). | Two points toward the target number of points (cumulative) | |
Complete all required modules for a particular session. | One star | Collect one star for each session. |
Complete all required modules for sessions 1, 2, and 3. | Silver trophy | Collect a silver trophy. |
Complete all required modules for sessions 4, 5, 6, and 7. | Gold trophy | Collect a gold trophy. |
After completing a phone screen, potential participants completed a clinical intake interview. To establish anxiety and exclusionary diagnoses, the Kiddie-Schedule for Affective Disorders and Schizophrenia for School-Aged Children-Present and Lifetime version [
The children were treated using the brief Coping Cat manual and workbook [
As part of the treatment, the therapist was required to complete several tasks via the clinician portal, which is accessible from a computer or a tablet (see
User engagement was defined as an indicator of the extent to which children interact with the app. User engagement data was reported using indications such as how much time the children spent on the app and the total number of app use during treatment.
App retention was defined as the extent to which children retain their willingness in completing skill-builder modules between sessions. Retention data was reported using the app use between sessions.
A Mann-Whitney
Portal tasks a therapist was required to complete.
Tasks | Start of session | End of session | Between sessions |
Enter custom locations and times | ✓ | ||
Select modules for upcoming week | ✓ | ||
Review skills coach or other module data from the week with child | ✓ | ||
Set target points for the following weeks | ✓ | ||
Send or receive messages | ✓ |
The app was developed using an Android software development kit (SDK). To accommodate new features (ie, low-power location monitoring and improved user interface), Android SDK version 4.2 or above was used. The minigames were developed using Unity, a cross-platform game engine developed by Unity Technologies. Unity allows the games to be run on top of Android or iPhone operating system (iOS, Apple Inc) devices.
The following key components of the system were implemented during the iterative system development process.
The reminder (
To complement time-based reminders, we also provide location-aware reminders using geofencing. Geofencing enables automatic detection of mobile objects as they enter or exit a geofence, which is a virtual boundary for a real-world area [
The reminders are integrated into a weekly plan for each child that is pushed to the child’s app. As shown in
Notes: an instructional message that will be shown on the message part of the app’s notification dialog
Time: the length of the event and the 2-hour window (ie, 4-6 PM, 5-7 PM, 6-8 PM, and 7-9 PM) of the day that a notification should pop up,
Session: each session is associated with a different set of skill-builder modules
Optional module: an indicator to include additional skill-builder modules.
The following minigames (see
Some anxious children are insufficiently skilled in recognizing somatic cues associated with different feelings (eg, anxiety, anger, boredom, and sadness) [
Clinical levels of anxiety can come from irrational or maladaptive thoughts, beliefs, or self-talk. In therapy sessions, the therapist teaches anxious children cognitive reframing techniques to modify the maladaptive nature of their self-talk. This requires the children to first recognize their self-talk. The “Thought-buster” module helps the child in classifying self-talk as either anxious or nonanxious. Self-talk in the app is presented as balloons that can be popped by tapping the screen and are randomized between screens.
Rational analysis of thoughts followed by a generation of coping thoughts marks another important task in cognitive restructuring processes. The “Thought-swapper” module guides the child in conducting rational analysis of a thought based on a hypothetical situation. For each hypothetical situation, an anxious thought presents in a thought bubble on top of the character. For each situation, the child needs to either counter the initial thought or intensify it. This way, the child can experiment and learn what coping thoughts will work best in a given situation and foster an understanding that thoughts can influence emotions.
Anxious children often present with problems they wish to resolve. The strategies (eg, avoidance and escape) these children have used to resolve problems in the past is often not an effective strategy for future difficulties. For example, anxious children might not leave their home to avoid panicky feelings. Although avoidance might be effective in reducing anxious distress in the short term, it is an ineffective strategy for dealing with future uncomfortable thoughts and feelings. During a face-to-face session, a CBT therapist leads the child through the steps in the problem-solving process.
The “Problem-solver” module provides an interactive way for the child to practice the four steps of problem solving: define the problem, come up with as many solutions as you can think of, evaluate all of the options, and pick one or two best solutions. To familiarize the child with these four steps, the module imitates an SMS text message (short message service, SMS) conversation between the child and his or her virtual friend who is experiencing a hypothetical problem from his or her hypothetical life (eg, performing at the talent show after school or going to a friend’s sleepover). Here the child must help his or her virtual friend solve the problem randomly generated each time the module is initiated.
To complement the games, we have included the “Chillax” module (see
SmartCAT reminders, weekly plan, minigames, and Chillax module screen.
The “skills coach” module (
As illustrated in
The therapist activates the “Challenger” module (see
To support therapist-patient interaction, we developed a secure messaging interface (
Skill-builder modules can be activated during instances of acute anxiety by launching the app. From the app’s home screen (
Skills Coach and Challenger module screen.
SmartCAT secure messaging system, app and portal home screen, and Show that I can module screen.
The portal allows therapists to monitor patients’ progress, as well as access their skills coach, Show that I can, and Challenger entries. The home screen of the portal can be seen illustrated in
We have also included the Show that I can module (
The child usage data revealed that the app was used frequently during treatment. On average, each child spent 35.59 min on the app (SD 64.18) completing 13.00 skill-builder modules per session (SD 12.61), suggesting high motivation during treatment.
The children were using different sets of skill-builder modules between sessions, suggesting their willingness to learn a varying set of skills. As illustrated in
SmartCAT 1.0 vs. SmartCAT 2.0 usage frequency. Usage data were collected after Session 1 and calculated at the end of Session 8.
User engagement and app retention by system.
System | Number of participants | Engagement (across duration of treatment) | App retention | |
Time spent in minutes (SD) | App use (SD) | App use per session (SD) | ||
SmartCAT 1.0 | 15 | 135.08 (56.48) | 36.13 (13.54) | 5.16 (3.03) |
SmartCAT 2.0 | 35 | 248.02 (327.41) | 90.40 (69.33) | 13.00 (12.61) |
SmartCAT 2.0 module usage between sessions.
Participants were satisfied with the visual appearance of the app, comfortable using the app, and making the app part of their daily routine. They stated that the app was easy to use and found it helpful when they were experiencing anxiety, as illustrated in the following quotes:
It is amazing, it can really help you.
I thought the app helped me out a lot. It was like therapist on a phone.
Was very easy to use and learn. Keep up the good work!
The app was very easy to use and wasn’t confusing at all.
On average, the app was used twice a day. The therapists could track participants’ weekly progress and could provide written reinforcements when required using the portal. The result of the implementation indicates that the gamified SmartCAT system has been used as expected and suggests that the inclusion of gamification can effectively increase user engagement and retention.
Although effective, the effects of gamification were not uniformly experienced by all participants. During the clinical trial, one participant did not use the app often, completing only 12 skill-builder modules throughout treatment. This participant was not motivated to use the app and was diagnosed and referred for depression treatment at posttreatment. This suggests that symptoms of depression may interfere with engagement. Six patients used the app more often—but less than an average of seven times—between sessions (<49 times across duration of treatment). This suggests that the implementation of gamification does not always lead to significant increases in user engagement and app retention. As previous studies on player motivation suggest, intrinsic and extrinsic motivators can differently influence the way people interact with game-like systems [
The project was implemented in an uncontrolled clinical trial involving a small number of patients, which must be taken into account in interpreting the results. The usage patterns observed at posttreatment may not reflect realistic usage patterns, as the patients who already have iPhones were not able to use the system on their own smartphones.
brief cognitive behavioral therapy
cognitive behavioral therapy
ecological momentary intervention
mobile health
software development kit
user-centered design
This project was funded in part by the National Institute of Mental Health (NIMH) grant #R34MH102666 (PI JS). GP and BP are also funded through the RERC on ICT “From Cloud to Smartphone—Accessible and Empowering ICT,” grants #90RE5018 and #90DP0064 from the National Institute for Disability, Independent Living, and Rehabilitation Research (NIDILRR). The authors would like to thank Marcie L Walker and Han-Tsung (Marcus) Min from the Department of Psychology, University of Pittsburgh, for their help with project and data management.
JS, BP, GP, and OL are the inventors of the SmartCAT mHealth system. JL is also the Chief Executive Officer (CEO) of Playpower Labs, Inc, which provided design and development services for SmartCAT games. He has no further ownership of the games or financial interest in the outcomes. PCK receives royalties from the sales of materials related to the treatment of anxiety in youth.