Published on in Vol 6, No 1 (2018): Jan-Mar

User-Centered Design of Learn to Quit, a Smoking Cessation Smartphone App for People With Serious Mental Illness

User-Centered Design of Learn to Quit, a Smoking Cessation Smartphone App for People With Serious Mental Illness

User-Centered Design of Learn to Quit, a Smoking Cessation Smartphone App for People With Serious Mental Illness

Original Paper

1Center for Addiction Science and Technology, Psychiatry and Behavioral Sciences, Duke University, Durham, NC, United States

2Design Use Build, Department of Human Centered Design and Engineering, University of Washington, Seattle, WA, United States

3Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States

4Chad Otis Illustration & Design, Seattle, WA, United States

Corresponding Author:

Roger Vilardaga, MA, PhD

Center for Addiction Science and Technology

Psychiatry and Behavioral Sciences

Duke University

2812 Erwin Road, Suite 403 Box 13

Durham, NC, 27705

United States

Phone: 1 9196813441


Background: Smoking rates in the United States have been reduced in the past decades to 15% of the general population. However, up to 88% of people with psychiatric symptoms still smoke, leading to high rates of disease and mortality. Therefore, there is a great need to develop smoking cessation interventions that have adequate levels of usability and can reach this population.

Objective: The objective of this study was to report the rationale, ideation, design, user research, and final specifications of a novel smoking cessation app for people with serious mental illness (SMI) that will be tested in a feasibility trial.

Methods: We used a variety of user-centered design methods and materials to develop the tailored smoking cessation app. This included expert panel guidance, a set of design principles and theory-based smoking cessation content, development of personas and paper prototyping, usability testing of the app prototype, establishment of app’s core vision and design specification, and collaboration with a software development company.

Results: We developed Learn to Quit, a smoking cessation app designed and tailored to individuals with SMI that incorporates the following: (1) evidence-based smoking cessation content from Acceptance and Commitment Therapy and US Clinical Practice Guidelines for smoking cessation aimed at providing skills for quitting while addressing mental health symptoms, (2) a set of behavioral principles to increase retention and comprehension of smoking cessation content, (3) a gamification component to encourage and sustain app engagement during a 14-day period, (4) an app structure and layout designed to minimize usability errors in people with SMI, and (5) a set of stories and visuals that communicate smoking cessation concepts and skills in simple terms.

Conclusions: Despite its increasing importance, the design and development of mHealth technology is typically underreported, hampering scientific innovation. This report describes the systematic development of the first smoking cessation app tailored to people with SMI, a population with very high rates of nicotine addiction, and offers new design strategies to engage this population. mHealth developers in smoking cessation and related fields could benefit from a design strategy that capitalizes on the role visual engagement, storytelling, and the systematic application of behavior analytic principles to deliver evidence-based content.

JMIR Serious Games 2018;6(1):e2




Smoking rates in the United States have been reduced to 15% in the past decades [1]. However, this downward trend is not present in people with serious mental illness (SMI) [2]. This population, which is characterized by people with chronic mental health symptoms and functional impairments that interfere with major life activities, typically encompasses individuals with a diagnosis of schizophrenia, schizoaffective, bipolar, and recurrent major depression [3]. People with SMI have smoking rates of up to 88% [4-6], have high levels of disease [7], and lose 25 years of life expectancy [8].

Survey research indicates that this population has rates of adoption of mobile technology that range between 72% and 81% [9,10], closing the gap with the 95% adoption of the general population [11], and thus presenting an opportunity to develop mobile smoking cessation apps that address the treatment needs of this population.

Despite the increasing number of digital interventions developed for people with SMI [12-17], no mobile intervention for smoking cessation has been described in the scientific literature for this vulnerable patient population. This shortage of smoking cessation mHealth research for SMI is not surprising when considering the larger context of smoking cessation mHealth. There are over 540 smoking cessation apps in the market [18], but only 2 have been tested in randomized controlled trials [19,20], and to our knowledge, there are no reports of their user-centered design research.

This lack of reports on the user-centered design process of mHealth interventions for smoking cessation is problematic, because (1) the determination of the active therapeutic ingredients delivered by an app should be the result of a careful design process, (2) poorly designed software systems have an impact on their ultimate efficacy and when not usable can be a waste of resources [21,22], and (3) unreported design research undermines design reproducibility and our body of knowledge. Thus, user-centered design research of mHealth interventions is not only an important step to ensure their efficacy and usability, but also an important way to advance our scientific knowledge.

People with SMI can have very low levels of adherence to digital interventions [12,23], which calls for an user-centered design process that addresses a series of known usability barriers in this population, such as persistent and moderate-to-severe mental health symptoms [24], low levels of educational attainment [25], cognitive deficits [26,27], and poor fine motor skills [28].

Prior Work

In previous research, we found a direct link between key demographic factors and engagement with SmartQuit, a smoking cessation app designed for the general population. Specifically, we demonstrated that experiencing mental health symptoms, being female, and having low levels of educational attainment predicted low levels of engagement with the app at a 3-month follow-up [29]. In subsequent research, we further identified specific usability barriers encountered by people with SMI when using NCI QuitPal, a smoking cessation app developed by the National Cancer Institute (NCI) [30]. Despite adherence to US Clinical Practice Guidelines (USCPG) for smoking cessation, our study directly showed that NCI QuitPal led to critical performance errors and low user experience among smokers with SMI, suggesting the need for new design approaches for this population.

Goal of This Study

From these initial studies, we planned to develop Learn to Quit, a smoking cessation app tailored to this often neglected and vulnerable population. Consistent with the need to report the user-centered design process of mHealth interventions, the aim of this paper was to describe the rationale, ideation, prototyping, design, user research, and final feature set of Learn to Quit, a smoking cessation app tailored to individuals with SMI that will be subsequently tested in a randomized controlled feasibility trial ( NCT03069482).

Our user-centered design methods and materials are summarized in Figure 1. This formative study was organized in 7 phases consistent with the user-centered design framework [31,32], which includes the following key activities: (1) understanding and specifying the context of use (phase I); (2) specifying the user and organizational requirements, such as the active ingredients of the behavior change intervention (phases II-III); (3) produce design solutions (phase IV); and (4) evaluate the design (phase V). Our design process also involved users throughout the design and development process, addressed the whole user experience (both usability and user experience), and incorporated multidisciplinary perspectives, which are key principles in user-centered design [31].

Phases VI and VII are not part of the user-centered design process per se, but are important steps in design implementation, which are also documented in the user-centered design literature [33]. Substantial progress in each phase was necessary to initiate meaningful progress in the following phase. However, at times, these phases overlapped with each other. For example, phase IV overlapped with phase V, because feedback from usability testing was used to modify or edit the original sketches and paper prototype. Conversely, completion of phase VI was a required step to initiate phases VI and VII. All study procedures were approved by the Institutional Review Board of the University of Washington.

Figure 1. Methods and materials.
View this figure

Phase I: Expert Panel

To better understand the needs of our target population, we sought to get the perspective of patients from this population and their providers. Expert guidance has been used in prior work to inform app development efforts [34]. We formed an expert panel composed of 2 social workers, 1 psychiatric case manager, 2 psychiatrists, and 2 smokers with SMI. After a project introduction, we addressed the panel with 2 research questions: (1) What are the biggest challenges for people with SMI to quit smoking? and (2) How could we design a highly engaging app for smokers with SMI? The first and second authors took notes from comments and observations, discussed them to identify agreements and focal points, and organized them for qualitative review. No formal thematic analysis of these interviews was conducted.

Phase II: Selection of Design Principles

Before designing the app, we prespecified 2 sets of design principles: (1) general learning principles based on applied behavior analysis and (2) design principles specific to individuals with SMI. Applied behavior analysis is a scientific discipline focused on developing strategies and behavior modification techniques in areas of social relevance based on principles of learning. Use of these principles has shown promise for the treatment of addiction in people with SMI [35,36], and these principles have been used in the design of games and other health apps [37]. Design principles specific to individuals with SMI have typically addressed the cognitive deficits and mental health symptoms encountered by this population. This approach has been used in previous work designing technologies for people with SMI [38], and it is supported by the general literature with regard to the importance of adjusting designs systems to meet the cognitive model of the user [39-41].

Phase III: Incorporating Evidence-Based Smoking Cessation Content

Smoking cessation app content was selected from behavior change interventions supported by the empirical literature (eg, clinical trials) and from process research suggesting a theoretical link between intervention components and the symptoms typically experienced by our target population (see below “Evidence-Based Smoking Cessation Content” subheading in the Results section). This process ensured the theoretical grounding of the app and its evidence-based foundation.

Phase IV: Ideation, Sketching, and Paper Prototyping

We used several user-centered design tools to ideate a prototype of the app. This included (1) the creation of personas, a technique aimed at increasing the designer’s emotional understanding of the end user by creating a short narrative of their motivations, context, and personal characteristics [42-44], and (2) sketching and paper prototyping, an important component of the design process consisting the use of paper drawings to quickly iterate on variations of app structure, app interactions, and the layout of content [42,45].

Phase V: Usability Testing of Paper Prototype


Sketches and images developed during the ideation and paper prototyping phase provided the basis for usability testing. To simulate the app experience, we used app prototyping software (POP, Marvel, London, UK). The software was installed on iPod Touch devices, and the prototype was presented to smokers with SMI during a single 45 min session. Our key inclusion criteria were as follows: (1) being an adult who smokes at least 5 cigarettes per day, (2) receiving outpatient mental health treatment and medication by a psychiatric provider, and (3) being fluent in

Usability testing procedures included (1) completing a series of tasks with the simulated app, (2) evaluation of user experience with semistructured interviews, and (3) rating the prototype using the system usability scale (SUS) [46]. Preceding our user interviews with a series of hands-on tasks provided the user with a more in-depth experience with the app prototype, and therefore allowed us to gather more meaningful and concrete feedback from users. Finally, our global assessment of usability, the SUS, was conducted at the very end to give users an opportunity to summarize their feedback. Given the iterative nature of this testing phase, we modified app design features after each participant and provided the new version to the following participant. This iterative process is standard in formative evaluations [47].

Our usability testing tasks evaluated the following elements of the app prototype: (1) an introductory tutorial, (2) overall Home Screen navigation (Figure 2, panel a), (3) overall Play Screen navigation (Figure 3, panel a), (4) access to technical coach feature (Figure 2, panel a), and (5) access to learning score and practice scores (Figure 2, panel a). Consistent with usability testing guidelines [47], our procedures reminded the interviewer to keep neutrality in response to user comments and behavior, read the script verbatim to each user, and observe and keep track of user comments and behavior. Finally, for the purposes of this study, we did not conduct observational coding of users’ behavior during task completion. Instead, we relied on the qualitative analysis of user feedback and the SUS.

Measures and Analysis

The SUS is a valid and reliable 10-item 5-point Likert scale with scores that range from 0 to 100. Higher scores indicate higher levels of usability, with scores above 68 indicating above-average usability [46]. The SUS can be further analyzed based on 2 subscales that measure usability and learnability of the software system. Learnability refers to the user’s level of ease in gaining proficiency with a software system. These subscales are interpreted following the same range and direction of the overall scale [48]. Semistructured interviews were recorded, transcribed, and analyzed using thematic analysis, an inductive qualitative method that organizes verbal content based on similarity, dependence, and proximity to identify key themes and opportunities for innovation [49-51].

Phase VI: Defining of the App’s Core Vision and Design Specifications

We created a design specifications document that laid out the app’s overall vision, look and feel of the interface, and its basic components and features [33]. This document was used to facilitate communication with a software development company.

Phase VII: Work With a Software Development Vendor

In this final phase, we worked with a software vendor to materialize the vision and design specifications that resulted from our formative study.

Figure 2. These wireframes represent how our app’s initial Home Screen evolved throughout our design process. From left to right: (a) Home Screen sketch, (b) pre-14 Home Screen, and (c) post-14 Home Screen. Wireframes (b) and (c) are examples of the 2 types of Home Screen status: dark green, indicating that the user is still completing the 14 modules of Learn to Quit, and light blue, indicating that the user already completed the Learn to Quit lessons and is ready for his first quit attempt.
View this figure
Figure 3. These wireframes represent how our app’s initial Play Screen evolved throughout our design process. From left to right: (a) Play Screen sketch, (b) pre-14 journey map, and (c) post-14 journey map. Wireframe (a) presents a character that needs to “jump” from stone to stone to “pick up” skills for quitting while navigating through a “swamp of urges.” In wireframe (b), the user has completed the “Finding Your North Star” lesson and practiced the “Your North Star for Quitting” skills module. Wireframe (c) presents a user who has completed all levels of the Learn to Quit journey and motivated by his values for quitting has metaphorically reached “Learn to Quit Land”.
View this figure

In this section, we describe the results of our formative study by focusing on a summary of our expert panel feedback, describing an initial paper prototype of the app and the results of its usability testing. This formative study resulted in a clearer definition of the app’s core vision and helped define the scope of work to be conducted by a software development company. Results from the remaining user-centered design processes and materials (ie, design principles, theory-based content, paper prototyping) are presented throughout a final section that lays out the app’s final characteristics and features and how they were informed by those processes and materials.

Results From Our Formative Study

Expert Panel Results

The panel emphasized the specific challenges faced by this population when trying to quit (Table 1). This included the panel members’ view that, as opposed to the general population, patients with SMI are more concerned about ongoing medical and mental health challenges (eg, metabolic syndrome, suicidality) rather than dying from cancer. Although not consistent with the behavioral economics literature [52], in the panel members’ experience as providers, motivations to quit in people with SMI are either different or less pronounced than in the general population. To add to this challenge, providers indicated that both withdrawal symptoms and ongoing mental health symptoms were a common concern in their patients when discussing the possibility of quitting smoking (eg, triggering a psychotic event). Needing preparation to quit was also emphasized, arguing that patients would prefer a system that teaches them a set of skills for quitting, rather than a system that emphasizes setting up a quit date and monitoring maintenance.

To solve this lack of motivation and concerns about the process of quitting, the panel outlined a series of strategies that could engage the users with this app. One of them was the use of meaningful images and storytelling. Some level of gamification was viewed as important to engage users with SMI. In addition, the panel argued that as a way to compensate for the overwhelming task of quitting smoking, the system should include progressive disclosures, make sure it rewards small victories, and add external motivation (eg, money saved). Social networking and monitoring of medication aids were also mentioned. Finally, a few other themes emerged during our discussion, including challenges related to the use of technology in this population, the social context of these patients (eg, risk of having their smartphone being stolen), and the benefits of personalization and provider support during the initial stages of nicotine withdrawal and beyond.

Initial Results From Ideation, Sketching, and App Prototyping

On the basis of input from our expert panel and the authors’ experience as clinical providers, we created 3 personas: (1) Esteban, a 55-year-old male with schizophrenia, 35 years of smoking history, psychiatrically stable, and with a range of medical complications; (2) Martin, a 31-year-old male with a diagnosis of bipolar disorder, who is a light smoker, and holds a stable job; and (3) Julia, a 43-year-old female with recurrent major depression and a history of drug use, who is taking care of a young daughter. These personas guided the design process and helped ensure that the initial prototype was responsive to the physical and emotional contexts of a broad spectrum of people with SMI, from psychotic to chronic affective disorders (see Multimedia Appendix 1).

These personas were used as inspiration to sketch our first wireframes (ie, sets of images displaying the functional elements of a website or app) and the overall app prototype, which was limited to a few basic components: A Home Screen (see Figure 2, panel a); a Play Screen (see Figure 3, panel a); a Tracking Screen (see Figure 4, panel a); a Quiz Screen (see Figure 5, panel a); a proof-of-concept smoking cessation module (see Figure 6, panel a); and an onboarding tutorial. To minimize cognitive demand in people with SMI, the app had only a few buttons (eg, “back,” “next,” and “Home”). These buttons were large, an important feature given the fine motor deficits observed in people with SMI [28] and findings from previous research in this population [30,38].

Table 1. Expert panel themes. Each of the insights of our expert panel are organized by a question and accompanied by a short description.
Questions and themesDescription
Question 1: What are the biggest challenges for people with serious mental illness to quit smoking?

Motivation to quitLife expectancy is not generally a motivation to quit in this population

Quitting without preparationEarly attempts to quit without enough preparation, and/or lacking a step-down quitting process

Withdrawal symptomsFear of experiencing withdrawal symptoms days after quitting

Mental health symptomsOngoing anxiety, depression, stress, and psychotic symptoms during the quitting process
Question 2: How could we design a highly engaging app for smokers with serious mental illness?

Meaningful visualsThe ability to display pictures of inspiring objects, people, sites, or pets

Having a “video game” feelThe appeal of video games or “game like” features (eg, “bingo”)

Social networkingThe possibility of sharing with peers

StorytellingThe use of interactive characters (eg, dog) for storytelling

Encouraging activationThe importance of increasing activation (eg, exercise) to facilitate quitting and cope with withdrawal

Progressive disclosureUnlocking bits and pieces of the app, such as a picture or a message, as the user makes progress

Rewarding small victoriesReinforcing small victories toward quitting (eg, one day without smoking)

Money savings motivationSmall money savings (eg, a few extra dollars a week)

Medication aidsMedication education and a system to help patients adhere to medication intake
Miscellaneous themes

Technological literacyLack of smartphone knowledge was viewed as a potential barrier to app use

Predominant use of AndroidThe need to build an app in Android was viewed as important to secure access in this population

Provider check-insFlexible check-ins (eg, text messages) were emphasized to enhance engagement with the app

Need for personalizationAn app that was customizable to each patient was deemed as important

StealingConcern that some patients with serious mental illness might have their devices stolen
Figure 4. These wireframes represent how our initial tracking feature evolved. From left to right: (a) tracking feature sketch, (b) cigarette tracking, and (c) personalized cigarette use feedback. As opposed to wireframe (a), in which we planned to use a single wireframe to collect all desirable tracking dimensions, in the final app, we used separate wireframes for each dimension (eg, smoking, mood). Note in (b) that users could report smoking half cigarette. Wireframe (c) is an example of personalized feedback following a user who reported smoking between 5 and 10 cigarettes.
View this figure
Figure 5. Wireframe examples of an initial sketch of a “Review Quiz” and a final Learning Module Quiz. Quizzes were presented at the end of the learning modules, and contained 3 questions each. From left to right: (a) Review Quiz sketch, (b) example of question for the “Key to Quitting” module, (c) feedback to correct answer that is followed by game reward sound, and (d) summary of quiz results, which indicates number of correct answers, best answer of all times, and number of practice stars gained (1 for each practice with a total of 3 per module).
View this figure
Figure 6. Selection of wireframes of a smoking cessation skill (ie, Use Your Five Senses) designed to encourage self-awareness of our 5 senses. From top to bottom: (a) sketch of Use Your Five Senses skill and (b) final Use Your Five Senses skill module. Wireframes in panel (b) include five 10-second timers to assist the user focus their attention. They provide visual and tactile cues to mark the end of each practice of focused attention.
View this figure
Table 2. Key baseline features of usability testing subjects and corresponding system usability scale (SUS) scores. Scores above the usability standard cut-off (>68) are indicated in italics.
Participant numberMental health treatmentYears in mental health (mean=25)Years smoking (mean=20)Cigarettes per day (mean=11)SUS usability (mean= 80)SUS learnability (mean=60)SUS total (mean= 74)
P1Case manager, psychiatric nurse259137210078
P2Case manager, psychiatric nurse29107887585
P3Case manager, psychiatrist203210442540
P4Case manager, psychiatrist393515815075
P5Case manager, psychiatrist121410948893
Usability Testing Results

A total of 5 daily smokers recruited from an outpatient mental health clinic participated in usability testing of the app prototype. They averaged 44 years of age (standard deviation [SD] 7.5), and the majority were female (4/5) and had less than a college education (4/5). Of the 5 participants, 1 was multiracial, 1 African American, and the rest were white. Our sample group smoked an average of 11 cigarettes per day (SD 3), and most had an extended smoking history (mean 20 years, SD 13). Although we did not conduct diagnostic interviews, all participants were patients from a community mental health clinic, had an assigned psychiatric case manager and a psychiatric provider, were currently taking psychiatric medication, and on average had received mental health treatment for 25 years (SD 10). See Table 2 for a breakdown of individual baseline characteristics.

System Usability Scale

Overall, participants’ levels of usability with the prototype were above the standard cutoff, suggesting that the initial prototype had promise. This was reflected in both the overall scale and the usability subscale. Conversely, the learnability subscale of the SUS did not reach the standard cutoff, although it reached high levels for 3 out of 5 individuals (see Table 2).

Thematic Analysis

Table 2 summarizes the results of the thematic analysis. To facilitate comparison with user testing of a nontailored app for smoking cessation in the same population (NCI QuitPal), we matched some of the themes resulting from our analysis with a previous usability testing study we conducted in smokers with SMI [30].

Key Learnings From Usability Testing of the Learn to Quit Prototype

Key learnings from usability testing can be organized in 3 areas: support for app design features, critical usability errors, and minor usability errors. First, we found that the following app design features improved the apps usability: (1) the prototype’s reduced number of app layers, (2) removing the need to use a keypad to enter and save information in the app, and (3) overall prototype’s simplicity. Although Learn to Quit’s simple app structure and navigation features were the result of a previous user-centered design study we conducted in the same population [30], this study allowed us to test a specific and concrete solution to those previously identified usability issues. User experience more directly confirmed that participants appreciated the simplicity of the design and the use of cartoons and gamification, and had a positive response to our paper prototype of a smoking cessation skill (see Tables 3 and 4).

Second, usability testing identified a critical usability error with our paper prototype. Specifically, it revealed that our original Home Screen was confusing to most users (see Figure 2, panel a). Users had difficulty in understanding the Home Screen structure and the purpose of each Home Screen subpanel. This is reflected in the prototype’s suboptimal SUS learnability score.

Finally, we identified minor usability problems with the prototype, including small font in the subpanels and confusion about specific language. These usability errors led to a final Home Screen that had a simpler layout and removed most of its original displays and content (see Figure 2, panels b and c).

Table 3. Usability testing results for Learn to Quit prototype (n=5) matched with comparable usability testing results from a previous user-centered design study (n=5) we conducted in a smoking cessation app designed for the general population (QuitPal) [30].

Smoking cessation app (QuitPal); (SUSa=65.5)
Learn to Quit paper prototype; (SUS= 74)
Difficulty entering information in the appUnable to “pull up the keypad”The need to use the keypad was removed from the prototype
Difficulty saving informationFailure to identify and press “save” button at the top of screenThe need to use a “save” button was removed from the prototype
Getting lost in app layers“It took me a long time to get back to that menu frame”No observed confusion about how to return to the Home Screen
Tremor and fine motor skills“[buttons were] too close together”P5b: “I like how the letters are big”

aSUS: System Usability Scale; scores above the usability standard cut-off (>68) are indicated in italics.

bP: Participant.

Table 4. Themes identified during usability testing of the Learn to Quit app prototype (n=5).

Representative quotea
Interested in gamification of smoking cessation skillsP1: “That would be so cool! A point every day”
P2: “[the most exciting] the points”
P4: “this would be kind of fun”; “So this is like a little game and people often play a lot of games”
P5: “That’s a good thing […] that builds confidence”
Drawn by cartoons and storytellingP3: “The cartoons, the whole thing. It’s got great spirit”
P4: “that’s very cute!”; “I like it, it’s cartoony-like”; “it is very eye-catching”
P5: “So you got a cartoon character! That’s what I was thinking. It was right on. That works for me”; “Mm, cartoon characters, yes!”
Appreciating simplicityP2: “It was simple, informative, easy to use”
P4: “[I like it]...when you’re a kid and you’re learning something new, it’s basic and it’s not all overstimulated, to put it that way…”
P5: “It just goes right in your mind”; “this looks really simple too and it looks good”
Proof of concept: Acceptance and Commitment Therapy module showed promiseP1: “I wish you guys could send it to me so that I could practice it and learn it”
P4: ”it’s like this too shall pass […], makes sense.”; “Mind and action skills. That’s pretty good. Because it’s like when you go to an AA”
P5: “It relaxed me”
Home Screen confusionP1: “It’s very small and I can’t see what it is.”
P2: “I don’t know if that’s an “I” or not.”
P3: “The first part of the app is so busy…”

aP: Participant.

Design Specifications and the App’s Core Vision

On the basis of this formative study, we created a technical document laying out the app’s structure and its core screens and features. We named the app Learn to Quit and synthesized the app’s core vision with the following: learn, practice, and play. Learning referred to the process of being exposed to daily modules that explain different smoking cessation concepts. Brief quizzes would help the user retain and learn those materials. Practice referred to the actual practice of smoking cessation skills in the form of brief daily exercises. Practice should lead to “mastery” of the learned materials. Play referred to the user’s opportunity to participate in a game comprising completing learning and practice modules and earning rewards along the way. The role of play was to promote higher levels of app engagement and commitment to learn.

Software Development Timeline

In May 2015, we filed a Report of Innovation at the University of Washington (ROI#47274) with the design specifications document of the app and proceeded to approach a company to develop a software-coded version of the app. Learn to Quit was built between August 2015 and October 2015. Smashing Ideas Inc. [53], a design and development agency, contributed to the refining and enhancement of the app prototype.

Learn to Quit’s Core Features

Evidence-Based Smoking Cessation Content

Learn to Quit’s main active ingredient is Acceptance and Commitment Therapy (ACT) [54]. We chose this behavior change approach because (1) it has promising results across multiple smoking cessation clinical trials [19,55-58]; (2) its components have shown to predict smoking cessation outcomes [59-61]; (3) it is an intervention originally developed to provides skills to cope with mental health symptoms [54]; and (4) it has been successfully adapted to people with SMI [62-65]. These 4 characteristics made ACT a suitable evidence-based approach for smoking cessation in this specific population.

ACT has 3 components relevant to smoking cessation: awareness, or the smoker’s ability to recognize smoking triggers and urges; openness, or the smoker’s willingness to experience smoking urges and triggers; and values activation, or the smoker’s active engagement in values-based activities related to health. The app content was therefore designed to deliver techniques related to each of those components, which predicted smoking cessation outcomes in previous research [59,60]. A secondary active ingredient of our intervention was adherence to key elements of USCPG for smoking cessation [66]. These guidelines include setting up a quit date, proper use of medication aids (eg, nicotine patch), and preparing for relapse. These 2 active ingredients (ACT and USCPG) were integrated to ensure an evidence-based design approach to app development.

We adapted ACT to a mobile format by creating 14 modules of ACT+USCPG content and 14 modules of exercises to practice smoking cessation skills (see Multimedia Appendix 2). We chose a 14-day program that would be consistent with USCPG, which recommends 2 weeks of preparation for quitting, and would balance the need for gradual exposure to smoking cessation content while providing a concrete timeline. Each module presented a short narrative that exemplified ACT+USCPG content and explained it from the perspective of someone with nicotine addiction and SMI. Each concept was presented in a variety of narratives and exercises to increase learning generalizability (see principle of multiple exemplar training in Multimedia Appendix 3). In addition, each lesson ended with a quiz designed to further enhance comprehension and retention (see Figure 5, panels b and c). We arranged these modules so that completing a lesson per day was necessary to unlock new lessons the next day (see principle of differential reinforcement of successive approximations in Multimedia Appendix 3).

Completion of skills modules was optional to increase the user’s perceived behavioral control [67]. ACT+USCPG content was developed and organized so that it would gradually increase in complexity as the user advances through each of the 3 levels (see section below on app gamification).

Simple Screens, Large Buttons, and a Predictable App Structure

To address the cognitive deficits observed in this population, we incorporated feedback from our formative study and followed recommendations from previous literature [30,38-41,68]. See Multimedia Appendix 3 for a list of key design principles we used to address SMI in our app. The result was an app with simple screens, large buttons, and a simple structure. The Home Screen (Figure 2, panels b and c) only included 2 buttons, 1 to access the Play Screen, which unlocked the module content described above, and 1 button to access the app settings. Usability testing suggested that the Home Screen should incorporate only a few elements. Thus, the Home Screen had as its focus a character surrounded by a circle of checkmarks, indicating overall progress in modules’ completion.

The Play Screen (Figure 3, panels b and c) represents a game in which each day the user takes a step forward toward completing a smoking cessation module. This screen had a linear structure, with older modules at the bottom and newer modules toward the top. Each module ranged between 6 and 24 screens, all of which had a reduced amount of semantic information, a simple color palette, consistent font size and type, and a predictable structure. All remaining wireframes in the app were presented in the form of cartoon scripts divided in 2 panels: the upper panel included the text and the lower panel a complementary image. We made each panel identical in size to maximize consistency and minimize cognitive load. This format was consistent with our panel’s emphasis on the use of storytelling and interactive characters to engage patients with SMI. We also avoided the use of videos and audio. We hypothesized that the use of sliding cartoons and vignettes could maximize retention and comprehension of smoking cessation skills, as this format provides the user with more control over the speed of presentation of smoking cessation content and the ability to stop or review a single vignette for as long as needed.

Gamification of Smoking Cessation Content

Consistent with our expert panel’s feedback, usability testing, and the empirical literature on the use of games not for entertainment, such as health or education (ie, serious games) [67,69], we designed an app that gamifies smoking cessation content. The main concept was designing a game in which the smoker metaphorically overcomes a swamp of urges and learns quitting skills along the way (see Figure 3). Gamification included interactive quizzes that provide immediate feedback about quiz results (see Figure 6, panel d), a component that has been correlated with long-term improvements in eHealth interventions [70]. Feedback included auditory cues for right or wrong answers and a reward system consisting of checkmarks, badges, cups, and crowns (see Applied Behavior Analysis section below). Repeated practice was encouraged with a reward system that provided stars every time a module was completed. This star system was consistent with the narrative of the app game (ie, module “Finding Your North Star”; see Multimedia Appendix 2). Sliding through module screens was also followed by video game sounds and feedback. We also added interactive elements to our tracking feature. Specifically, a 3-tiered system of personalized feedback was incorporated to diminish assessment burden. That is, we categorized all possible answers to a specific question (“How many cigarettes you smoked today?”) into 3 different levels and we created a custom message for each, acknowledging the users’ answer and encouragement to move forward in their journey (eg, smoking less than 5 cigarettes; see Figure 4, panels b and c).

Application of Principles of Applied Behavior Analysis

Behavioral principles (see Multimedia Appendix 3) informed the organization and delivery of smoking cessation content. These principles had the goals of (1) increasing retention, comprehension, and mastery of app content and (2) minimizing the impact of the cognitive deficits and low educational attainment of our target population [26,27,71]. For example, instead of providing the user with all smoking cessation content at once, we used the principle of successive approximations [72] to lay out increasingly complex ACT content throughout a 14-day period (see Multimedia Appendix 2 for a list of all the lessons and skills). Then, we applied the principle of multiple exemplar training [73]. Consistent with this principle, we presented multiple examples of a given concept or skill throughout multiple modules as a means to foster skills generalization to real-world settings.

In addition, the app used a combination of antecedent and consequential control strategies [72,74]. On the one hand, antecedent control was used in the form of notifications and Home Screen messages to prompt the individual to complete certain modules or tasks (Figure 2, panel b: “New Lesson unlocked! Tap button to Start”). On the other hand, consequential control was used by applying positive reinforcement arranged on a fixed ratio schedule [72]. More specifically, obtaining a new star was made contingent upon completing a module. This reinforcement ratio was used to encourage users to complete those modules at least 3 times (ie, a maximum of 3 stars could be obtained per module; see Figure 3, panels b and c), which could lead to a total of 84 stars (see Figure 2, panels b and c). Additional modules could be completed at any time but it was not incentivized with more stars.

A rewards scheme was also implemented to increase the reinforcing effect of responding correctly to lesson quizzes (see Figure 5, panel d). For example, an individual who gave 3 correct answers to 3 questions would receive a “crown.” Badges and cups were provided to users who responded 2 and 1 correct answers, respectively. Checkmarks were offered to users who completed the quiz but did not have any correct answer. Finally, we used negative reinforcement to promote daily completion of app modules. Specifically, the number of days in a row in which the user completed a module was indicated with a big bold number at the bottom of the Home Screen (see Figure 2, panel b). Not completing a new module one day was penalized with starting the count again from zero.

Emphasis in Visual Engagement and Storytelling

Because our expert panel and usability testing strongly supported the use of simple cartoons and visual storytelling, we created a gender-neutral character that rotated across the different stories and metaphors presented in each module (see Figures 2,3, and 6). The character enacts a variety of scenarios with the aim of exemplifying the experience of nicotine addiction and the smoking cessation skills offered in the app. Combination of imagery and text has been recommended in previous literature on mHealth in SMI populations [14]. The stories were told in very short sentences designed to avoid cognitive overload and maximize module completion. The purpose of these images and stories is to evoke an emotional connection and increase retention and comprehension of app content. For example, the Home Screen had 2 types of background: (1) dark green (Figure 2, panel b), to represent that the user is still completing the Learn to Quit journey and thus navigating through a “swamp of urges,” and (2) light blue (Figure 2, panel c), to indicate that the user has reached “Learn to Quit land” and thus is ready to quit. Similarly, the background of each module used a consistent and muted color scheme that maps into the 3 dimensions of the app’s theory-based content: (1) gray for Learning Modules, (2) blue for Mind Skills, and (3) red for Action Skills. The function of this color scheme was to offer a simple and predictable layout that would be more likely to fit the cognitive model of the user [40,41]. Previous research in people with SMI also indicated that bright colors can be overwhelming in this population, thus supporting our use of muted colors and a simplified color scheme [38].

Access to Technical Coaching

Our expert panel indicated that “technological illiteracy” was common in patients with SMI (Table 1). In the panel members’ opinion, many of their patients have never used a smartphone device, and therefore using successfully an app for smoking cessation could be a challenge. Furthermore, they commented on the complex social context of these patients and the potential need for more intense personal support. In our previous study testing a smoking cessation app in a small sample of this population [30], we observed some of these challenges, but we also noted that there was a wide range of technological literacy in this population, with some patients owning their own device and comfortably using smartphone apps. Therefore, we incorporated a simple technical coaching feature in our design that would serve as an aid to those that needed guidance, whereas at the same time maintaining the core vision of the app as a stand-alone intervention. This technical coaching feature consisted of a button in the settings section that allowed the user to call a preassigned technical coach. Eventually, this technical coaching role could be performed by addiction counselors or case managers and therefore serve to integrate the app within ongoing health care at a community mental health clinic or larger health care organization.

Principal Findings

This paper reports the rationale, ideation, design, user research, and final features of a novel smoking cessation app developed for people with SMI, a population in great need of novel smoking cessation treatment. Building this app involved a user-centered design process that carefully considered a series of design principles to maximize comprehension and retention of smoking cessation concepts, minimize the impact of known challenges in people with SMI, and ensure the effective delivery of evidence-based smoking cessation content.

Results from our user-centered design process informed the features included in the final app. First, informed by our formative study and as suggested by the literature [14], we gradually delivered evidence-based smoking cessation content using imagery and simple semantic content. The primary active ingredient of this evidence-based content was ACT, which is an intervention that has empirical support as a smoking cessation intervention [55-58] and as an intervention to treat individuals with SMI [62-65]. This intervention was further integrated with smoking cessation recommendations from USCPG [66] to ensure alignment with best clinical practices.

Second, as suggested by the literature [30,38-41,68] and our user testing of a paper prototype, we created an app with very simple screens, buttons, and a predictable app structure that led to promising usability and user experience results. The paper prototype we created used a minimal set of app buttons and a very simple app structure, which probably contributed to scores above the usability standard cut-off. Our thematic analysis of user experience interviews was in line with these usability scores and further reinforced the inclusion of our final set of app’s core features.

Third, input from a panel of experts in SMI led to the idea of incorporating app gamification, visual engagement, and storytelling [67,69,75]. Use of these elements is consistent with the serious games literature for smoking cessation published in this journal [67]. More specifically, the app offered a module to help users identify their core values for quitting and used these values as the game’s objective (ie, module “Your North Star for Quitting”), which increases the user’s perceived behavioral control and intrinsic motivation [67]. In addition, it offered a clearly structured game that had functional utility [67], that is, successfully completing the game involved being exposed to a series of content known to help people quit smoking.

Fourth, we implemented a number of applied behavior analysis principles to maximize retention and comprehension of app content [37,76]. This included the use of progressive disclosures of smoking cessation content, variation and repetition of that content, a reward system, and modules that gradually portrayed complex smoking cessation concepts (eg, psychological awareness). Application of these principles provided guidance to adjust our design to the needs of our population and provided a level of conceptual clarity that linked our work with the behavior change literature at large. For example, although the term “notification” is common in mHealth, this feature is essentially an antecedent control strategy, which has been extensively used in the behavior change literature in areas such as autism [77-79], individuals with dementia and cognitive impairments [80-82], and in cases of traumatic brain injury and poor executive functioning [83,84].

Finally, our development effort took into account implementation considerations: (1) it used an Android operating system, which according to our panel was a common platform among people with SMI and tends to dominate the market among people with lower socioeconomic status (eg, individuals with disabilities) [85,86], and (2) it addressed the potential need for more intense personal support by including a technical coach feature within the app. In the future, this coaching role could be performed by addiction counselors or case managers in community mental health clinics and address not just technical issues but also support to use of ACT skills to deal with smoking cravings or adherence to USCPG.

Comparison With Prior Work

The study reported in this paper is consistent with user-centered design research of mobile apps for depression [87], smoking cessation [30], and work with diverse populations [88]. Furthermore, Learn to Quit’s emphasis in gamification is consistent with an app developed for depression, SuperBetter [89].

To date, many apps have focused on the use of sensors and algorithms to track user context and provide personalized feedback [90-94]. Learn to Quit differs from this approach in the sense that it is based on a more traditional form of engagement, visual storytelling. Visual storytelling is a core form of engagement common across human cultures [95]. Storytelling and visual engagement are implicated in emotional arousal [96] and in attaching value and significance to sensory descriptions [97,98], thus contributing to learning and memory (eg, comprehension and retention of new content) [98,99]. Given the fact that motivational challenges are common among people with SMI (eg, negative symptoms of schizophrenia), we built an app that had visual storytelling at its core. Our main goal was to create an app with visuals and stories that were as engaging as possible, in combination with gamification, well-established behavior analytic principles, and the use of evidence-based smoking cessation content.

As stated in the introduction, we believe that the determination of the active therapeutic ingredients delivered by an app should be the result of a careful design process. However, user-centered design research could lead to stakeholder recommendations that are not consistent with evidence-based practices or theory-based principles of change. Adherence to evidence-based practices or theory-based principles of change might not always be emphasized in user-centered design research, yet it is a key activity of the design process inherent in the original user-centered design guidelines [31,32]. As shown in our Methods section, we emphasized this key activity by incorporating phases II and III and by making an effort to ensure that our final design integrated concerns brought up by each of these phases.

Finally, a relevant aspect of this app is that it could be a good example of the concept of universal design [100], for which systems tailored to specific groups of individuals with certain disabilities or challenges become inherently usable to larger groups of the population without those challenges. For example, even though the app addresses core usability barriers experienced in people with SMI, in doing so, it explains in simple terms very complex psychological concepts which might engage smokers of young age, low literacy groups, and the general public. Furthermore, the app’s systematic use of behavior analytic principles to increase comprehension and retention of ACT and USCPG is broadly applicable. Therefore, Learn to Quit’s design approach could be generalizable to related mental health conditions, health behaviors, or the general public.


This study had several limitations. First, the number of patients with SMI in the expert panel group (n=2) and the usability testing study (n=5) could have been small, leaving to question whether a larger sample of people with SMI could have led to more feedback and opportunities for innovation. There is debate among user-centered design researchers about the most cost-efficient number of subjects to identify usability errors [101,102], with the most traditional approach suggesting a sample size of 5 [101]. Recruiting individuals with SMI is particularly challenging; therefore, we believe our sample size was justified.

Second, our methods could have been more rigorous in several aspects. Specifically, results from our expert panel were not transcribed and analyzed using a complete set of qualitative methods, which could have led to the identification of additional themes. However, rather than a thorough and comprehensive analysis of provider input, the goal of this expert panel was to quickly gather initial insights and impressions that would orient our imminent design process. Additionally, usability testing did not include observational coding of user behavior. As reported in similar studies [30,87,88], this could have provided more concrete usability feedback and complement the results of the SUS and the thematic analysis of our transcripts. A third methodological limitation is that we did not conduct diagnostic interviews of our subjects. Despite this, all subjects received psychiatric treatment and were recruited from a community mental health clinic, which, according to regulations by the US Department of Health and Human Services [3], requires serving individuals with SMI status.

Third, the app’s tracking feature provided personalized feedback based on participants’ responses to self-reported ratings of mood and smoking behavior. However, this level of personalization did not take into consideration each individual’s baseline (eg, certain smoking reductions could be large or small depending on the individual’s baseline), limiting its impact for personalization. Likewise, the current app system does not take into account a variety of quitting scenarios (eg, individuals who quit before the end of the program) and how these scenarios interact with app content. Future versions could take into account these personal scenarios to strengthen Learn to Quit’s usefulness and level of personalization.

Finally, this paper focuses on the user-centered design research of a paper prototype leading to the development of the Learn to Quit app. Although this report does not provide data about the usability and user experience of the final Learn to Quit app, it allowed us to transparently report in more detail its user-centered design process. A separate report of Learn to Quit’s usability and user experience in its target population is under review elsewhere.


This is the first paper to systematically describe the rationale, ideation, design, and user research of a smoking cessation app specifically designed for people with SMI, a population with alarmingly high rates of nicotine addiction and in high need of novel smoking cessation treatments. The feasibility and acceptability of this app will be subsequently tested in a randomized controlled feasibility trial ( NCT03069482).

User-centered design is a critical process for the development of mobile interventions for individuals with SMI. However, because emotional and cognitive challenges are present in less severe forms of mental illness or can be present in other health conditions (eg, cancer patients), the results of this study might be generalizable to other areas of mHealth research. Therefore, while ideating and designing digital interventions, mHealth developers might consider capitalizing on the role of visual engagement, storytelling, and the systematic application of behavior analytic principles to deliver evidence-based content.


The authors thank Francis J McClernon, PhD, for comments and feedback that greatly improved the final manuscript. Roger Vilardaga received funding support from the National Institute of Drug Abuse (K99DA037276 and R00DA037276). Learn to Quit is the intellectual property of the University of Washington (© 2015-2016 University of Washington). Julie A. Kientz’s spouse is the cofounder of Senosis Health, a start-up company in the area of health technologies for diagnosis, monitoring, and treatment, which was recently acquired by Google.

Authors' Contributions

RV envisioned the app rationale and intervention, ideated app design and features, sketched imagery, conducted research activities, interpreted results, and wrote this manuscript. JR coordinated research activities, sketched imagery to be included in each module, and contributed to the overall conduct of design and research activities. EZ transcribed the recordings of the usability testing procedure, conducted thematic analysis of the interviews, discussed emerging themes with first author, and sketched imagery to be included in each module. JK provided user-centered design expertise in the development of the software app for behavior change and smoking cessation and contributed to early drafts of this manuscript. RR contributed to form the expert panel, provided serious mental illness expertise and feedback about the overall app design, and contributed to late drafts of this manuscript. CO created high-fidelity designs and illustrations and contributed to improvements in interaction design and the final user experience of the app. KH contributed to late drafts of this manuscript and to the layout of the manuscript structure and sections.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Primary and secondary personas that guided Phase IV of the user-centered design process.

PNG File, 197KB

Multimedia Appendix 2

Evidence-based smoking cessation content of Learn to Quit app.

PDF File (Adobe PDF File), 195KB

Multimedia Appendix 3

Design principles.

PDF File (Adobe PDF File), 242KB

  1. Jamal A, King BA, Neff LJ, Whitmill J, Babb SD, Graffunder CM. Current Cigarette Smoking Among Adults -- United States, 2005-2015. MMWR Morb Mortal Wkly Rep 2016;65(44):1205-1211. [CrossRef]
  2. Cook BL, Wayne GF, Kafali EN, Liu Z, Shu C, Flores M. Trends in smoking among adults with mental illness and association between mental health treatment and smoking cessation. J Am Med Assoc 2014 Jan 08;311(2):172-182 [FREE Full text] [CrossRef] [Medline]
  3. Substance Abuse and Mental Health Services Administration. 2016. Behind the Term: Serious Mental Illness   URL: [WebCite Cache]
  4. Cook BL, Wayne GF, Kafali EN, Liu Z, Shu C, Flores M. Trends in smoking among adults with mental illness and association between mental health treatment and smoking cessation. J Am Med Assoc 2014 Jan 08;311(2):172-182 [FREE Full text] [CrossRef] [Medline]
  5. Dickerson F, Stallings CR, Origoni AE, Vaughan C, Khushalani S, Schroeder J, et al. Cigarette smoking among persons with schizophrenia or bipolar disorder in routine clinical settings, 1999-2011. Psychiatr Serv 2013 Jan;64(1):44-50. [CrossRef] [Medline]
  6. Lasser K, Boyd JW, Woolhandler S, Himmelstein DU, McCormick D, Bor DH. Smoking and mental illness: a population-based prevalence study. J Am Med Assoc 2000;284(20):2606-2610. [Medline]
  7. Scott D, Happell B. The high prevalence of poor physical health and unhealthy lifestyle behaviours in individuals with severe mental illness. Issues Ment Health Nurs 2011;32(9):589-597. [CrossRef] [Medline]
  8. Colton CW, Manderscheid RW. Congruencies in increased mortality rates, years of potential life lost, and causes of death among public mental health clients in eight states. Prev Chronic Dis 2006 Apr;3(2):A42 [FREE Full text] [Medline]
  9. Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE. Mobile technologies among people with serious mental illness: opportunities for future services. Adm Policy Ment Health 2013 Jul;40(4):340-343 [FREE Full text] [CrossRef] [Medline]
  10. Firth J, Cotter J, Torous J, Bucci S, Firth JA, Yung AR. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: a meta-analysis of cross-sectional studies. Schizophr Bull 2015 Sep 22;42(2):448-455. [CrossRef] [Medline]
  11. Pew Research Center. Pewinternet. Mobile Fact Sheet Internet   URL: [accessed 2017-09-01] [WebCite Cache]
  12. Killikelly C, He Z, Reeder C, Wykes T. Improving adherence to web-based and mobile technologies for people with psychosis: systematic review of new potential predictors of adherence. JMIR Mhealth Uhealth 2017 Jul 20;5(7):e94 [FREE Full text] [CrossRef] [Medline]
  13. Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, et al. Designing websites for persons with cognitive deficits: design and usability of a psychoeducational intervention for persons with severe mental illness. Psychol Serv 2007 Aug;4(3):202-224 [FREE Full text] [CrossRef] [Medline]
  14. Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013 Dec;36(4):289-296 [FREE Full text] [CrossRef] [Medline]
  15. Palmier-Claus J, Rogers A, Ainsworth J, Machin M, Barrowclough C, Laverty L, et al. Integrating mobile-phone based assessment for psychosis into people?s everyday lives and clinical care: a qualitative study. BMC Psychiatry 2013 Mar 2013 Jan 23;13(1):12. [CrossRef]
  16. Ferron JC, Brunette MF, McHugo GJ, Devitt TS, Martin WM, Drake RE. Developing a quit smoking website that is usable by people with severe mental illnesses. Psychiatr Rehabil J 2011;35(2):111-116. [CrossRef] [Medline]
  17. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull 2012 May;38(3):414-425 [FREE Full text] [CrossRef] [Medline]
  18. Bricker JB, Copeland W, Mull KE, Zeng EY, Watson NL, Akioka KJ, et al. Single-arm trial of the second version of an acceptance & commitment therapy smartphone application for smoking cessation. Drug Alcohol Depend 2017 Jan 01;170:37-42. [CrossRef] [Medline]
  19. Bricker JB, Mull KE, Kientz JA, Vilardaga R, Mercer LD, Akioka KJ, et al. Randomized, controlled pilot trial of a smartphone app for smoking cessation using acceptance and commitment therapy. Drug Alcohol Depend 2014 Oct 1;143:87-94. [CrossRef] [Medline]
  20. Buller DB, Borland R, Bettinghaus EP, Shane JH, Zimmerman DE. Randomized trial of a smartphone mobile application compared to text messaging to support smoking cessation. Telemed J E Health 2014 Mar;20(3):206-214 [FREE Full text] [CrossRef] [Medline]
  21. Hafner K. NYtimes. Wanted by the Police: A Good Interface Internet   URL: [WebCite Cache]
  22. Rutirasiri C. Entrepreneur. Making the Business Case for Human-Centered Design Internet   URL: [accessed 2017-09-01] [WebCite Cache]
  23. Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [CrossRef] [Medline]
  24. Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res 2001 Dec;36(6 Pt 1):987-1007 [FREE Full text] [Medline]
  25. Kessler R, Chiu W, Demler O, Walters E. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun 01;62(6):617. [CrossRef] [Medline]
  26. Green MF, Kern RS, Heaton RK. Longitudinal studies of cognition and functional outcome in schizophrenia: implications for MATRICS. Schizophr Res 2004 Dec 15;72(1):41-51. [CrossRef] [Medline]
  27. Austin MP, Mitchell P, Goodwin GM. Cognitive deficits in depression: possible implications for functional neuropathology. Br J Psychiatry 2001 Mar;178:200-206 [FREE Full text] [Medline]
  28. Schwartz BL, Rosse RB, Veazey C, Deutsch SI. Impaired motor skill learning in schizophrenia: implications for corticostriatal dysfunction. Biol Psychiatry 1996 Feb;39(4):241-248. [CrossRef]
  29. Zeng EY, Vilardaga R, Heffner JL, Mull KE, Bricker JB. Predictors of utilization of a novel smoking cessation smartphone app. Telemed J E Health 2015 Jul 14;21(12):998-1004. [CrossRef] [Medline]
  30. Vilardaga R, Rizo J, Kientz JA, McDonell MG, Ries RK, Sobel K. User experience evaluation of a smoking cessation app in people with serious mental illness. Nicotine Tob Res 2016 May;18(5):1032-1038. [CrossRef] [Medline]
  31. ISO. Ergonomics of human-system interaction -- Part 210: Human-centred design for interactive systems   URL: [accessed 2017-11-08] [WebCite Cache]
  32. International Organization for Standardization. ISO 13407Human-centred design processes for interactive systems Internet   URL: [accessed 2017-11-14] [WebCite Cache]
  33. Hartson R, Pyla P. The UX Book: Process and Guidelines for Ensuring a Quality User Experience. 1 edition. Amsterdam, Boston: Morgan Kaufmann; 2012.
  34. Bhattacharya A, Vilardaga R, Kientz JA, Munson SA. Lessons from Practice: Designing Tools to Facilitate Individualized Support for Quitting Smoking. USA: ACM; 2017 Presented at: ACM Trans Comput Hum Interact; 5/6/2017; Denver p. 3057-3070   URL: [CrossRef]
  35. McDonell M, Howell D, McPherson S, Cameron J, Srebnik D, Roll J, et al. Voucher-based reinforcement for alcohol abstinence using the ethyl-glucuronide alcohol biomarker. J Appl Behav Anal 2012;45(1):161-165 [FREE Full text] [CrossRef] [Medline]
  36. McDonell MG, Leickly E, McPherson S, Skalisky J, Srebnik D, Angelo F, et al. A randomized controlled trial of ethyl glucuronide-based contingency management for outpatients with co-occurring alcohol use disorders and serious mental illness. Am J Psychiatry 2017 Apr 01;174(4):370-377. [CrossRef] [Medline]
  37. Dallery J, Kurti A, Erb P. A new frontier: integrating behavioral and digital technology to promote health behavior. Behav Anal 2015 May;38(1):19-49 [FREE Full text] [CrossRef] [Medline]
  38. Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, et al. Designing websites for persons with cognitive deficits: design and usability of a psychoeducational intervention for persons with severe mental illness. Psychol Serv 2007 Aug;4(3):202-224 [FREE Full text] [CrossRef] [Medline]
  39. Stephanidis C. User Interfaces for All: Concepts, Methods,Tools. Hillsdale, NJ, USA: CRC Press; 2000.
  40. Norman D, Berkrot P. The Design of Everyday Things. Old Saybrook: Tantor Audio; 2011.
  41. Dalal NP, Quible Z, Wyatt K. Cognitive design of home pages: an experimental study of comprehension on the World Wide Web. Inf Process Manag 2000 Jul;36(4):607-621. [CrossRef]
  42. Cooper A, Reimann R, Cronin D. About Face 3: The Essentials of Interaction Design. 3rd edition. Indianapolis, IN: Wiley; 2007.
  43. LeRouge C, Ma J, Sneha S, Tolle K. User profiles and personas in the design and development of consumer health technologies. Int J Med Inform 2013 Nov;82(11):e251-e268. [CrossRef] [Medline]
  44. Miaskiewicz T, Kozar KA. Personas and user-centered design: how can personas benefit product design processes? Design Studies 2011 Sep;32(5):417-430. [CrossRef]
  45. Buxton B. Sketching User Experiences: Getting the Design Right and the Right Design. 1 edition. San Francisco, Calif: Morgan Kaufmann; 2007.
  46. Sauro J. A practical guide to the system usability scale: Background, benchmarks & best practices. Denver: Measuring Usability LLC; 2011.
  47. Rubin J, Chisnell D, Spool J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. 2 edition. Indianapolis, IN: Wiley; 2008.
  48. Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. In: Human Centered Design. Berlin, Heidelberg: Springer; 2009.
  49. Rogers Y, Sharp H, Preece J. Interaction Design: Beyond Human - Computer Interaction. West Sussex: John Wiley & Sons; 2011.
  50. IDEO. Human-Centered Design Toolkit: An Open-Source Toolkit To Inspire New Solutions in the Developing World. 2 edition. S. l. Canada: IDEO; 2011.
  51. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
  52. Khwaja A, Silverman D, Sloan F. 2006. Time Preference, Time Discounting, and Smoking Decisions Internet   URL:
  53. Smashing Ideas.   URL: [accessed 2017-09-01] [WebCite Cache]
  54. Hayes S, Strosahl K, Wilson K. Acceptance and Commitment Therapy: The Process and Practice of Mindful Change. Second Edition: Guilford Press; 2011.
  55. Bricker JB, Mann SL, Marek PM, Liu J, Peterson AV. Telephone-delivered acceptance and commitment therapy for adult smoking cessation: a feasibility study. Nicotine Tob Res 2010 Apr;12(4):454-458. [CrossRef] [Medline]
  56. Gifford EV, Kohlenberg BS, Hayes SC, Pierson HM, Piasecki MP, Antonuccio DO, et al. Does acceptance and relationship focused behavior therapy contribute to bupropion outcomes? A randomized controlled trial of functional analytic psychotherapy and acceptance and commitment therapy for smoking cessation. Behav Ther 2011 Dec;42(4):700-715. [CrossRef] [Medline]
  57. Hernández-López M, Luciano MC, Bricker JB, Roales-Nieto JG, Montesinos F. Acceptance and commitment therapy for smoking cessation: a preliminary study of its effectiveness in comparison with cognitive behavioral therapy. Psychol Addict Behav 2009 Dec;23(4):723-730. [CrossRef] [Medline]
  58. Bricker J, Wyszynski C, Comstock B, Heffner JL. Pilot randomized controlled trial of web-based acceptance and commitment therapy for smoking cessation. Nicotine Tob Res 2013 Oct;15(10):1756-1764 [FREE Full text] [CrossRef] [Medline]
  59. Vilardaga R, Heffner JL, Mercer LD, Bricker JB. Do counselor techniques predict quitting during smoking cessation treatment? A component analysis of telephone-delivered Acceptance and Commitment Therapy. Behav Res Ther 2014 Oct;61:89-95 [FREE Full text] [CrossRef] [Medline]
  60. Heffner JL, Vilardaga R, Mercer LD, Kientz JA, Bricker JB. Feature-level analysis of a novel smartphone application for smoking cessation. Am J Drug Alcohol Abuse 2015 Jan;41(1):68-73. [CrossRef] [Medline]
  61. Bricker JB, Schiff L, Comstock BA. Does avoidant coping influence young adults' smoking?: a ten-year longitudinal study. Nicotine Tob Res 2011 Oct;13(10):998-1002 [FREE Full text] [CrossRef] [Medline]
  62. Gaudiano BA, Herbert JD. Acute treatment of inpatients with psychotic symptoms using Acceptance and Commitment Therapy: pilot results. Behav Res Ther 2006 Mar;44(3):415-437. [CrossRef] [Medline]
  63. Gaudiano BA, Nowlan K, Brown LA, Epstein-Lubow G, Miller IW. An open trial of a new acceptance-based behavioral treatment for major depression with psychotic features. Behav Modif 2013 May;37(3):324-355 [FREE Full text] [CrossRef] [Medline]
  64. White R, Gumley A, McTaggart J, Rattrie L, McConville D, Cleare S, et al. A feasibility study of Acceptance and Commitment Therapy for emotional dysfunction following psychosis. Behav Res Ther 2011 Dec;49(12):901-907. [CrossRef] [Medline]
  65. Bach P, Hayes SC. The use of acceptance and commitment therapy to prevent the rehospitalization of psychotic patients: A randomized controlled trial. J Consult Clin Psychol 2002;70(5):1129-1139. [CrossRef]
  66. 2008 PHS Guideline Update Panel, Liaisons, and Staff. Treating tobacco use and dependence: 2008 update U.S. Public Health Service Clinical Practice Guideline executive summary. Respir Care 2008 Sep;53(9):1217-1222 [FREE Full text] [Medline]
  67. El-Hilly AA, Iqbal SS, Ahmed M, Sherwani Y, Muntasir M, Siddiqui S, et al. Game on? Smoking cessation through the gamification of mHealth: a longitudinal qualitative study. JMIR Serious Games 2016 Oct 24;4(2):e18 [FREE Full text] [CrossRef] [Medline]
  68. Choi J, Bakken S. Web-based education for low-literate parents in Neonatal Intensive Care Unit: Development of a website and heuristic evaluation and usability testing. Int J Med Inform 2010 May 9;79(8):565-575. [CrossRef]
  69. Miller AS, Cafazzo JA, Seto E. A game plan: Gamification design principles in mHealth applications for chronic disease management. Health Informatics J 2014 Jul 1;22(2):184-193. [CrossRef] [Medline]
  70. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug;13(3):e52 [FREE Full text] [CrossRef] [Medline]
  71. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):593-602. [CrossRef] [Medline]
  72. Catania A, Brigham T. Handbook of Applied Behavior Analysis: social and Instructional Processes. Michigan: Irvington Publishers; 1978.
  73. Cooper J, Heron T, Heward W. Applied Behavior Analysis. 2 edition. Upper Saddle River, NJ: Pearson; 2007:978.
  74. Glynn T. Antecedent control of behaviour in educational contexts. Educ Psychol (Lond) 2010 Nov 25;2(3-4):215-229. [CrossRef]
  75. Blohm I, Leimeister JM. Gamification: Design of IT-Based Enhancing Services for Motivational Support and Behavioral Change. Bus Inf Syst Eng 2013 Jun 14;5(4):275-278. [CrossRef]
  76. Morford ZH, Witts BN, Killingsworth KJ, Alavosius MP. Gamification: the intersection between behavior analysis and game design technologies. Behav Anal 2014 May;37(1):25-40 [FREE Full text] [CrossRef] [Medline]
  77. Koegel RL, Shirotova L, Koegel LK. Antecedent stimulus control: using orienting cues to facilitate first-word acquisition for nonresponders with autism. Behav Anal 2009;32(2):281-284 [FREE Full text] [Medline]
  78. Ross DE, Greer RD. Generalized imitation and the mand: inducing first instances of speech in young children with autism. Res Dev Disabil 2003;24(1):58-74. [Medline]
  79. Tsiouri I, Schoen SE, Paul R. Enhancing the application and evaluation of a discrete trial intervention package for eliciting first words in preverbal preschoolers with ASD. J Autism Dev Disord 2012 Jul;42(7):1281-1293. [CrossRef] [Medline]
  80. Engberg S, Sereika SM, McDowell BJ, Weber E, Brodak I. Effectiveness of prompted voiding in treating urinary incontinence in cognitively impaired homebound older adults. J Wound Ostomy Continence Nurs 2002 Sep;29(5):252-265. [Medline]
  81. Adkins VK, Mathews RM. Prompted voiding to reduce incontinence in community-dwelling older adults. J Appl Behav Anal 1997;30(1):153-156 [FREE Full text] [CrossRef] [Medline]
  82. Feliciano L, Vore J, LeBlanc LA, Baker JC. Decreasing entry into a restricted area using a visual barrier. J Appl Behav Anal 2004;37(1):107-110 [FREE Full text] [CrossRef] [Medline]
  83. Zencius AH, Wesolowski MD, Burke WH, McQuade P. Antecedent control in the treatment of brain-injured clients. Brain Injury 2009 Jul 03;3(2):199-205. [CrossRef]
  84. Ylvisaker M, Turkstra LS, Coelho C. Behavioral and social interventions for individuals with traumatic brain injury: a summary of the research with clinical implications. Semin Speech Lang 2005 Nov;26(4):256-267. [CrossRef] [Medline]
  85. Edwards J. Business Insider. These Maps Show That Android Is For Poor People Internet   URL: [accessed 2017-09-01] [WebCite Cache]
  86. Shaffer D. WebpageFX. iPhones Dominate Smartphone Market Share for Internet Usage Internet   URL: [accessed 2017-09-01] [WebCite Cache]
  87. Stiles-Shields C, Montague E, Lattie EG, Schueller SM, Kwasny MJ, Mohr DC. Exploring user learnability and learning performance in an app for depression: usability study. JMIR Hum Factors 2017 Aug 11;4(3):e18 [FREE Full text] [CrossRef] [Medline]
  88. Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016 Dec;31(12):1417-1426. [CrossRef] [Medline]
  89. Roepke AM, Jaffee SR, Riffle OM, McGonigal J, Broome R, Maxwell B. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J 2015 Jun;4(3):235-246. [CrossRef] [Medline]
  90. Rabbi M, Ali S, Choudhury T, Berke E. Passive and in-situ assessment of mental and physical well-being using mobile sensors. Proc ACM Int Conf Ubiquitous Comput 2011;2011:385-394 [FREE Full text] [CrossRef] [Medline]
  91. Berke EM, Choudhury T, Ali S, Rabbi M. Objective measurement of sociability and activity: mobile sensing in the community. Ann Fam Med 2011;9(4):344-350 [FREE Full text] [CrossRef] [Medline]
  92. Wang R, Aung M, Abdullah S, Brian R, Campbell A, Choudhury T, et al. CrossCheck: Toward Passive Sensing Detection of Mental Health Changes in People with Schizophrenia. USA: ACM; 2016 Presented at: Proceedings of the 2016 ACM International Joint Conference on Pervasive Ubiquitous Computing; 9/12/2016; Heidelberg.
  93. Rahman T, Adams A, Ravichandran R, Zhang M, Patel S, Kientz J, et al. DoppleSleep: A Contactless Unobtrusive Sleep Sensing System Using Short-range Doppler Radar. USA: ACM; 2015 Presented at: Proceedings of the 2015 ACM International Joint Conference on Pervasive Ubiquitous Computing Internet; 7/11/2015; Osaka.
  94. Consolvo S, McDonald D, Toscos T, Chen M, Froehlich J, Harrison B, et al. Activity Sensing in the Wild: A Field Trial of Ubifit Garden. USA: ACM; 2008 Presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Internet; 4/5/2008; Florence, Italy.
  95. Bruner J. Making Stories: Law, Literature, Life. Cambridge: Harvard University Press; 2003.
  96. Barraza JA, Zak PJ. Empathy toward strangers triggers oxytocin release and subsequent generosity. Ann N Y Acad Sci 2009 Jun;1167:182-189. [CrossRef] [Medline]
  97. Grabenhorst F, Rolls ET. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci 2011 Feb;15(2):56-67. [CrossRef] [Medline]
  98. Conway BR, Rehding A. Neuroaesthetics and the trouble with beauty. PLoS Biol 2013 Mar;11(3):e1001504 [FREE Full text] [CrossRef] [Medline]
  99. Ishai A, Fairhall SL, Pepperell R. Perception, memory and aesthetics of indeterminate art. Brain Res Bull 2007 Jul 12;73(4-6):319-324. [CrossRef] [Medline]
  100. The Center for Universal Design.   URL: [WebCite Cache]
  101. Nielsen J, Landauer T. A Mathematical Model of the Finding of Usability Problems. USA: ACM; 1993 Presented at: Proceedings of the INTERACTION and CHI Conference on Human Factors in Computing Systems Internet; 1993; New York, NY.
  102. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003 Aug;35(3):379-383. [CrossRef]

ACT: Acceptance and Commitment Therapy
NCI: National Cancer Institute
SMI: serious mental illness
SUS: system usability scale
USCPG: US Clinical Practice Guidelines

Edited by A Powell; submitted 03.09.17; peer-reviewed by M Levin, S Schueller, K Bold, C Bullen; comments to author 06.10.17; revised version received 10.11.17; accepted 25.11.17; published 16.01.18


©Roger Vilardaga, Javier Rizo, Emily Zeng, Julie A Kientz, Richard Ries, Chad Otis, Kayla Hernandez. Originally published in JMIR Serious Games (, 16.01.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.