Sketches and images developed during the ideation and paper prototyping phase provided the basis for usability testing. To simulate the app experience, we used app prototyping software (POP, Marvel, London, UK). The software was installed on iPod Touch devices, and the prototype was presented to smokers with SMI during a single 45 min session. Our key inclusion criteria were as follows: (1) being an adult who smokes at least 5 cigarettes per day, (2) receiving outpatient mental health treatment and medication by a psychiatric provider, and (3) being fluent in

Usability testing procedures included (1) completing a series of tasks with the simulated app, (2) evaluation of user experience with semistructured interviews, and (3) rating the prototype using the system usability scale (SUS) [46]. Preceding our user interviews with a series of hands-on tasks provided the user with a more in-depth experience with the app prototype, and therefore allowed us to gather more meaningful and concrete feedback from users. Finally, our global assessment of usability, the SUS, was conducted at the very end to give users an opportunity to summarize their feedback. Given the iterative nature of this testing phase, we modified app design features after each participant and provided the new version to the following participant. This iterative process is standard in formative evaluations [47].

Our usability testing tasks evaluated the following elements of the app prototype: (1) an introductory tutorial, (2) overall Home Screen navigation (Figure 2, panel a), (3) overall Play Screen navigation (Figure 3, panel a), (4) access to technical coach feature (Figure 2, panel a), and (5) access to learning score and practice scores (Figure 2, panel a). Consistent with usability testing guidelines [47], our procedures reminded the interviewer to keep neutrality in response to user comments and behavior, read the script verbatim to each user, and observe and keep track of user comments and behavior. Finally, for the purposes of this study, we did not conduct observational coding of users’ behavior during task completion. Instead, we relied on the qualitative analysis of user feedback and the SUS.

The SUS is a valid and reliable 10-item 5-point Likert scale with scores that range from 0 to 100. Higher scores indicate higher levels of usability, with scores above 68 indicating above-average usability [46]. The SUS can be further analyzed based on 2 subscales that measure usability and learnability of the software system. Learnability refers to the user’s level of ease in gaining proficiency with a software system. These subscales are interpreted following the same range and direction of the overall scale [48]. Semistructured interviews were recorded, transcribed, and analyzed using thematic analysis, an inductive qualitative method that organizes verbal content based on similarity, dependence, and proximity to identify key themes and opportunities for innovation [49-51].

The panel emphasized the specific challenges faced by this population when trying to quit (Table 1). This included the panel members’ view that, as opposed to the general population, patients with SMI are more concerned about ongoing medical and mental health challenges (eg, metabolic syndrome, suicidality) rather than dying from cancer. Although not consistent with the behavioral economics literature [52], in the panel members’ experience as providers, motivations to quit in people with SMI are either different or less pronounced than in the general population. To add to this challenge, providers indicated that both withdrawal symptoms and ongoing mental health symptoms were a common concern in their patients when discussing the possibility of quitting smoking (eg, triggering a psychotic event). Needing preparation to quit was also emphasized, arguing that patients would prefer a system that teaches them a set of skills for quitting, rather than a system that emphasizes setting up a quit date and monitoring maintenance.

To solve this lack of motivation and concerns about the process of quitting, the panel outlined a series of strategies that could engage the users with this app. One of them was the use of meaningful images and storytelling. Some level of gamification was viewed as important to engage users with SMI. In addition, the panel argued that as a way to compensate for the overwhelming task of quitting smoking, the system should include progressive disclosures, make sure it rewards small victories, and add external motivation (eg, money saved). Social networking and monitoring of medication aids were also mentioned. Finally, a few other themes emerged during our discussion, including challenges related to the use of technology in this population, the social context of these patients (eg, risk of having their smartphone being stolen), and the benefits of personalization and provider support during the initial stages of nicotine withdrawal and beyond.

On the basis of input from our expert panel and the authors’ experience as clinical providers, we created 3 personas: (1) Esteban, a 55-year-old male with schizophrenia, 35 years of smoking history, psychiatrically stable, and with a range of medical complications; (2) Martin, a 31-year-old male with a diagnosis of bipolar disorder, who is a light smoker, and holds a stable job; and (3) Julia, a 43-year-old female with recurrent major depression and a history of drug use, who is taking care of a young daughter. These personas guided the design process and helped ensure that the initial prototype was responsive to the physical and emotional contexts of a broad spectrum of people with SMI, from psychotic to chronic affective disorders (see Multimedia Appendix 1).

These personas were used as inspiration to sketch our first wireframes (ie, sets of images displaying the functional elements of a website or app) and the overall app prototype, which was limited to a few basic components: A Home Screen (see Figure 2, panel a); a Play Screen (see Figure 3, panel a); a Tracking Screen (see Figure 4, panel a); a Quiz Screen (see Figure 5, panel a); a proof-of-concept smoking cessation module (see Figure 6, panel a); and an onboarding tutorial. To minimize cognitive demand in people with SMI, the app had only a few buttons (eg, “back,” “next,” and “Home”). These buttons were large, an important feature given the fine motor deficits observed in people with SMI [28] and findings from previous research in this population [30,38].

A total of 5 daily smokers recruited from an outpatient mental health clinic participated in usability testing of the app prototype. They averaged 44 years of age (standard deviation [SD] 7.5), and the majority were female (4/5) and had less than a college education (4/5). Of the 5 participants, 1 was multiracial, 1 African American, and the rest were white. Our sample group smoked an average of 11 cigarettes per day (SD 3), and most had an extended smoking history (mean 20 years, SD 13). Although we did not conduct diagnostic interviews, all participants were patients from a community mental health clinic, had an assigned psychiatric case manager and a psychiatric provider, were currently taking psychiatric medication, and on average had received mental health treatment for 25 years (SD 10). See Table 2 for a breakdown of individual baseline characteristics.

On the basis of this formative study, we created a technical document laying out the app’s structure and its core screens and features. We named the app Learn to Quit and synthesized the app’s core vision with the following: learn, practice, and play. Learning referred to the process of being exposed to daily modules that explain different smoking cessation concepts. Brief quizzes would help the user retain and learn those materials. Practice referred to the actual practice of smoking cessation skills in the form of brief daily exercises. Practice should lead to “mastery” of the learned materials. Play referred to the user’s opportunity to participate in a game comprising completing learning and practice modules and earning rewards along the way. The role of play was to promote higher levels of app engagement and commitment to learn.

In May 2015, we filed a Report of Innovation at the University of Washington (ROI#47274) with the design specifications document of the app and proceeded to approach a company to develop a software-coded version of the app. Learn to Quit was built between August 2015 and October 2015. Smashing Ideas Inc. [53], a design and development agency, contributed to the refining and enhancement of the app prototype.

Learn to Quit’s main active ingredient is Acceptance and Commitment Therapy (ACT) [54]. We chose this behavior change approach because (1) it has promising results across multiple smoking cessation clinical trials [19,55-58]; (2) its components have shown to predict smoking cessation outcomes [59-61]; (3) it is an intervention originally developed to provides skills to cope with mental health symptoms [54]; and (4) it has been successfully adapted to people with SMI [62-65]. These 4 characteristics made ACT a suitable evidence-based approach for smoking cessation in this specific population.

ACT has 3 components relevant to smoking cessation: awareness, or the smoker’s ability to recognize smoking triggers and urges; openness, or the smoker’s willingness to experience smoking urges and triggers; and values activation, or the smoker’s active engagement in values-based activities related to health. The app content was therefore designed to deliver techniques related to each of those components, which predicted smoking cessation outcomes in previous research [59,60]. A secondary active ingredient of our intervention was adherence to key elements of USCPG for smoking cessation [66]. These guidelines include setting up a quit date, proper use of medication aids (eg, nicotine patch), and preparing for relapse. These 2 active ingredients (ACT and USCPG) were integrated to ensure an evidence-based design approach to app development.

We adapted ACT to a mobile format by creating 14 modules of ACT+USCPG content and 14 modules of exercises to practice smoking cessation skills (see Multimedia Appendix 2). We chose a 14-day program that would be consistent with USCPG, which recommends 2 weeks of preparation for quitting, and would balance the need for gradual exposure to smoking cessation content while providing a concrete timeline. Each module presented a short narrative that exemplified ACT+USCPG content and explained it from the perspective of someone with nicotine addiction and SMI. Each concept was presented in a variety of narratives and exercises to increase learning generalizability (see principle of multiple exemplar training in Multimedia Appendix 3). In addition, each lesson ended with a quiz designed to further enhance comprehension and retention (see Figure 5, panels b and c). We arranged these modules so that completing a lesson per day was necessary to unlock new lessons the next day (see principle of differential reinforcement of successive approximations in Multimedia Appendix 3).

Completion of skills modules was optional to increase the user’s perceived behavioral control [67]. ACT+USCPG content was developed and organized so that it would gradually increase in complexity as the user advances through each of the 3 levels (see section below on app gamification).

To address the cognitive deficits observed in this population, we incorporated feedback from our formative study and followed recommendations from previous literature [30,38-41,68]. See Multimedia Appendix 3 for a list of key design principles we used to address SMI in our app. The result was an app with simple screens, large buttons, and a simple structure. The Home Screen (Figure 2, panels b and c) only included 2 buttons, 1 to access the Play Screen, which unlocked the module content described above, and 1 button to access the app settings. Usability testing suggested that the Home Screen should incorporate only a few elements. Thus, the Home Screen had as its focus a character surrounded by a circle of checkmarks, indicating overall progress in modules’ completion.

The Play Screen (Figure 3, panels b and c) represents a game in which each day the user takes a step forward toward completing a smoking cessation module. This screen had a linear structure, with older modules at the bottom and newer modules toward the top. Each module ranged between 6 and 24 screens, all of which had a reduced amount of semantic information, a simple color palette, consistent font size and type, and a predictable structure. All remaining wireframes in the app were presented in the form of cartoon scripts divided in 2 panels: the upper panel included the text and the lower panel a complementary image. We made each panel identical in size to maximize consistency and minimize cognitive load. This format was consistent with our panel’s emphasis on the use of storytelling and interactive characters to engage patients with SMI. We also avoided the use of videos and audio. We hypothesized that the use of sliding cartoons and vignettes could maximize retention and comprehension of smoking cessation skills, as this format provides the user with more control over the speed of presentation of smoking cessation content and the ability to stop or review a single vignette for as long as needed.

Consistent with our expert panel’s feedback, usability testing, and the empirical literature on the use of games not for entertainment, such as health or education (ie, serious games) [67,69], we designed an app that gamifies smoking cessation content. The main concept was designing a game in which the smoker metaphorically overcomes a swamp of urges and learns quitting skills along the way (see Figure 3). Gamification included interactive quizzes that provide immediate feedback about quiz results (see Figure 6, panel d), a component that has been correlated with long-term improvements in eHealth interventions [70]. Feedback included auditory cues for right or wrong answers and a reward system consisting of checkmarks, badges, cups, and crowns (see Applied Behavior Analysis section below). Repeated practice was encouraged with a reward system that provided stars every time a module was completed. This star system was consistent with the narrative of the app game (ie, module “Finding Your North Star”; see Multimedia Appendix 2). Sliding through module screens was also followed by video game sounds and feedback. We also added interactive elements to our tracking feature. Specifically, a 3-tiered system of personalized feedback was incorporated to diminish assessment burden. That is, we categorized all possible answers to a specific question (“How many cigarettes you smoked today?”) into 3 different levels and we created a custom message for each, acknowledging the users’ answer and encouragement to move forward in their journey (eg, smoking less than 5 cigarettes; see Figure 4, panels b and c).

Behavioral principles (see Multimedia Appendix 3) informed the organization and delivery of smoking cessation content. These principles had the goals of (1) increasing retention, comprehension, and mastery of app content and (2) minimizing the impact of the cognitive deficits and low educational attainment of our target population [26,27,71]. For example, instead of providing the user with all smoking cessation content at once, we used the principle of successive approximations [72] to lay out increasingly complex ACT content throughout a 14-day period (see Multimedia Appendix 2 for a list of all the lessons and skills). Then, we applied the principle of multiple exemplar training [73]. Consistent with this principle, we presented multiple examples of a given concept or skill throughout multiple modules as a means to foster skills generalization to real-world settings.

In addition, the app used a combination of antecedent and consequential control strategies [72,74]. On the one hand, antecedent control was used in the form of notifications and Home Screen messages to prompt the individual to complete certain modules or tasks (Figure 2, panel b: “New Lesson unlocked! Tap button to Start”). On the other hand, consequential control was used by applying positive reinforcement arranged on a fixed ratio schedule [72]. More specifically, obtaining a new star was made contingent upon completing a module. This reinforcement ratio was used to encourage users to complete those modules at least 3 times (ie, a maximum of 3 stars could be obtained per module; see Figure 3, panels b and c), which could lead to a total of 84 stars (see Figure 2, panels b and c). Additional modules could be completed at any time but it was not incentivized with more stars.

A rewards scheme was also implemented to increase the reinforcing effect of responding correctly to lesson quizzes (see Figure 5, panel d). For example, an individual who gave 3 correct answers to 3 questions would receive a “crown.” Badges and cups were provided to users who responded 2 and 1 correct answers, respectively. Checkmarks were offered to users who completed the quiz but did not have any correct answer. Finally, we used negative reinforcement to promote daily completion of app modules. Specifically, the number of days in a row in which the user completed a module was indicated with a big bold number at the bottom of the Home Screen (see Figure 2, panel b). Not completing a new module one day was penalized with starting the count again from zero.

Because our expert panel and usability testing strongly supported the use of simple cartoons and visual storytelling, we created a gender-neutral character that rotated across the different stories and metaphors presented in each module (see Figures 2,3, and 6). The character enacts a variety of scenarios with the aim of exemplifying the experience of nicotine addiction and the smoking cessation skills offered in the app. Combination of imagery and text has been recommended in previous literature on mHealth in SMI populations [14]. The stories were told in very short sentences designed to avoid cognitive overload and maximize module completion. The purpose of these images and stories is to evoke an emotional connection and increase retention and comprehension of app content. For example, the Home Screen had 2 types of background: (1) dark green (Figure 2, panel b), to represent that the user is still completing the Learn to Quit journey and thus navigating through a “swamp of urges,” and (2) light blue (Figure 2, panel c), to indicate that the user has reached “Learn to Quit land” and thus is ready to quit. Similarly, the background of each module used a consistent and muted color scheme that maps into the 3 dimensions of the app’s theory-based content: (1) gray for Learning Modules, (2) blue for Mind Skills, and (3) red for Action Skills. The function of this color scheme was to offer a simple and predictable layout that would be more likely to fit the cognitive model of the user [40,41]. Previous research in people with SMI also indicated that bright colors can be overwhelming in this population, thus supporting our use of muted colors and a simplified color scheme [38].

Our expert panel indicated that “technological illiteracy” was common in patients with SMI (Table 1). In the panel members’ opinion, many of their patients have never used a smartphone device, and therefore using successfully an app for smoking cessation could be a challenge. Furthermore, they commented on the complex social context of these patients and the potential need for more intense personal support. In our previous study testing a smoking cessation app in a small sample of this population [30], we observed some of these challenges, but we also noted that there was a wide range of technological literacy in this population, with some patients owning their own device and comfortably using smartphone apps. Therefore, we incorporated a simple technical coaching feature in our design that would serve as an aid to those that needed guidance, whereas at the same time maintaining the core vision of the app as a stand-alone intervention. This technical coaching feature consisted of a button in the settings section that allowed the user to call a preassigned technical coach. Eventually, this technical coaching role could be performed by addiction counselors or case managers and therefore serve to integrate the app within ongoing health care at a community mental health clinic or larger health care organization.