SpaPlay is a digital social online game developed to motivate women to make healthy eating choices and to exercise. It was developed through a close partnership between academia and industry, spanning more than 5 years, and involving all stages of game design and development. The current version of SpaPlay is a browser-based video game accessible to players via personal computers and laptops. It requires an Internet connection and a user-generated username and password. The content of the game centers on a virtual spa resort that needs to be developed and maintained, similar to other popular games like Farmville or We Rule. Figure 1 presents several screen shots of game elements. SpaPlay bridges real and virtual worlds, and game progress is contingent on activities completed in real life around physical activity and healthy eating. For example, eating a salad or taking a 10-minute walk would earn players points towards developing the spa (eg, building facilities, accumulating ratings, playing mini games). Gamification and principles of the Self-Determination theory are used to sustain players’ interest and engagement. Further, since weight loss takes time to self-observe, more immediate rewards in the game (eg, power-ups and customization options for the user’s avatar) or real life (eg, coupons from associated vendors) are used to keep players engaged.

Figure 2 explicitly demonstrates the relationship between game play elements, gamification principles and the self-determination theory. The two core game mechanics through which experience points are earned are Quests and Sparks. Quests are sets of physical activities or dietary tasks that the players complete within the span of a week and target longer-term commitment to health behaviors. An example includes taking 1-2 flights of stairs twice within a week; substituting a sugary beverage for water 5 times a week; or eating a fruit instead of a snack twice in a week. Sparks are short, single-time tasks that can include stretching for 5 minutes or adding spinach to a sandwich. Social Sparks are dietary or exercise activities in which the user engages with others in the real world. Both Sparks and Quests allow users to set their own short and longer-term goals and chose from a variety of activities that the game offers. They promote autonomy in the way that the players design and enact their own program. They also build social connectivity through in-game and real-life group-based activities (eg, taking a walk with a friend), promoting relatedness.

It is important to note a few additional game features. For instance, healthy behaviors are introduced through the game interface, which both teaches exercise methods, such as yoga through the yoga mini game, and proper diet through recommendations and diet mini games, such as the chef game, which challenges players to create healthy dishes with healthy recipes. Self-tracking of physical activity is enabled through a connected Fitbit sensor, allowing users to log and measure external activities. Finally, the game incorporates social features that allow users to interact with friends and display measures of their progress.

To evaluate the game, we recruited women from an urban New England area. Participants self-selected for the study in response to online solicitations sent through listservs and handouts and flyers posted at several public health and non-for-profit organization sites. Inclusion criteria were being ≥ 21 years old, speaking English, and reporting being at least at the contemplation stage of change for engaging in exercise and nutrition on the transtheoretical model of change [34], as described below. Of the 60 women initially recruited, 13 (21.7%) dropped out within the first week of the study. Reasons cited included lack of gaming experience and inability to provide adequate time commitment to game play. The final sample was comprised of 47 women who partook in the 90-day longitudinal study, with global assessments taken at baseline, 1-, 2-, and 3- month follow-ups. The study was approved by the Institutional Review Board at Northeastern University.

After the online consent, participants completed a baseline survey and were given instructions with game access. Following the baseline, participants were contacted via emails at months 1, 2, and 3 with a request to fill out follow-up surveys assessing main study outcomes (BMI and nutrition knowledge). A research assistant emailed participants once a month to check on progress and address any questions or concerns. All participants were awarded a $20 gift certificate at study completion.

Telemetry game play data were automatically recorded and time-stamped throughout the game play period. Data consisted of time-stamped actions, such as logins, type of Quests or Sparks selected, completion of Sparks or Quests, game activities such as picking up trash, score changes, and social interactions.

Two major study outcomes were BMI and nutritional knowledge. Participant self-reported weight and height was collected at baseline, 1-, 2-, and 3- months. BMI was calculated using the formula: weight (kg)/height (m)² [35].

Nutritional knowledge was assessed at 1, 2, and 3 months with the General Nutrition Knowledge questionnaire for adults. The questionnaire is comprised of 53 items evaluating an individual’s knowledge of nutritional and dietary needs. It has high test-retest reliability and construct validity established through expert review [36]. To reduce burden, due to the length of the questionnaire, we purposefully omitted assessing participants on their nutrition knowledge at baseline.

Readiness to change behaviors in domains of exercise, nutrition and consumption of sweetened beverages was assessed at baseline by the Readiness to Change Questionnaire, a 16-item instrument with high levels of test-retest reliability and predictive validity for behavior change [37]. For this study, we omitted information on the beverage-related items, and readiness to change on exercise and nutrition domains were used as covariates.

First, we examined play patterns based on objective telemetry data. Frequency of play was determined from daily game logins, which were tracked from the automatically recorded telemetry data. Daily play was noted as present (1) or absent (0). The total number of play days was computed for each person. In addition, the play intensity index was computed as the length of time between logins. With daily play, an individual mean would be expected to equal 0. Less frequent play would naturally translate into a higher mean value. Play regularity index was computed as the standard deviation between logins. For a regular play pattern (eg, every 2 days), it would be expected to be 0, and increase with irregular play. Figure 3 presents an example of login data from one study participant with corresponding play intensity and play regularity indices. Further, to capture the nature of game activities for each individual, we computed the total number of food- and exercise- related Sparks and Quests, and the number of game and social activities. Descriptive statistics were computed for all 9 indicators of game play for the entire sample.

Second, we examined whether background baseline variables differentiated between play patterns. To account for data non-normality, log transformations were applied to total days of play and the total number of food and exercise Quests and Sparks. General linear models were run with 7 predictors including education, ethnicity, age, history of weight loss program participation, baseline BMI, history of gaming 1-2 hours per session, and baseline stage of change on exercise and healthy food intake questionnaires. While repeated testing jeopardizes the overall type I error rate, given the preliminary nature of the study, we did not adjust the p-value and kept it at the .05 level for each model.

Third, to examine changes in BMI and nutritional knowledge over the course of gameplay, multilevel modeling (MLM) [38,39] was used to test for effects of time and game adherence on the study outcomes while controlling for major demographics (ie, age, ethnicity, and educational level), history of game play, and baseline readiness to change stage. MLM is designed to account for nested data structure with observations clustered within participants, for missing data with some participants omitting certain observations, and for inter-individual differences in trajectories with possible variability in intercept and slope values. The following model, as specified in Figure 4 was fit to the data, where the residuals are normally distributed, ε_it∼N(0,R_i), with variance R_i following the compound symmetry structure, and u_0i and u_1i are random intercept and slope parameters with corresponding variances, τ₀₀ and τ₁₀. In this equation, BMI for a given person i measured at time t is modeled as a function of, where time is centered at the 3^rdmonth follow-up (with values of -1, -2, and -3 corresponding to observations at 60 days, 30 days, and baseline, respectively). γ₀₀ is the BMI value at 3 months, controlling for study covariates Cov_i, γ₁₀ is the effect of game play on BMI, and is a linear rate of change in BMI in increments of 30 days (1 month). A similar model was fitted to Nutrition Knowledge as the outcome. Both were fitted in the nonlinear mixed effects (nlme) package in R, which is free, open-source statistical software [40]. Final models included predictors significant at the .05 alpha level.

The sample was comprised of women with an average age of 30 years (SD 7.3), 81% (38/47) reporting a Bachelors or Masters degree, 61% (30/47) being White, 20% (10/47) Asian, 8% (4/47) Black, and 2% (1/47) Latina. Of these participants, 6% (3/47) were concurrently enrolled in a weight loss program, while 32.7% (16/47) had previously completed weight loss programs. Average BMI at the beginning of the study was 26.98 (SD 5.6), which is considered “overweight” by the CDC’s criteria [35]. A number of participants (8/47, 17%) reported daily video game usage, 33% (26/47) reported less than daily but more than weekly usage, 25% (12/47) reported playing weekly or less frequently, and 6% (3/47) did not provide information about their video game behavior. Of those who did report playing video games, 72% (34/47) reported playing 1-2 hours per gaming session.

For behavior change stages on nutrition, 60% (28/47) of participants were in the contemplation stage, 34% (16/47) were in the preparation stage and 4% (247) were in the action stage. For physical activity, 55% (26/47) were in the contemplation stage, 38% (18/47) were in the preparation stage and 4% (2/47) were in the action stage.

Table 1 summarizes descriptive statistics for game play indicators. On average, participants played about 7 days (SD 12.5), 25% of participants did not utilize the game at all between the beginning and the end of the study, and 75% logged in fewer than 7 times over the course of the study. Food and exercise Sparks were the most popular activities. Distributions of Sparks were very skewed, with several players engaging in several hundreds of Sparks over the course of 90 days, but the majority engaging in a few (median of 8 for food and 4 for exercise Sparks). The average number of days between logins was 24.65 (SD 14.23), and the play regularity index averaged at 16.48 (SD 7.66).

Analyses of game play data revealed several predictors that differentiated between different levels of engagement with the game (Table 2). Two major predictors emerged. First, those who reported being in the preparation or action stages of eating healthy at baseline had a higher total number of logins, had shorted gaps between logins, played the game more consistently, and completed more exercise Sparks (all P values < .05). Second, those who reported at baseline spending 1-2 hours playing video games per session had fewer total logins and completed fewer exercise and food-related Sparks and Quests (all P values < .05).

Figure 5 presents observed BMI trajectories, with the large line summarizing the sample average, and thinner lines summarizing data from 10 randomly selected participants. Overall, the trajectory decreased over the course of three months. Average BMI at the beginning of the study was 26.98 (SD 5.6) and 26.09 (SD 5.27) at the end of the study, corresponding to Cohen’s d effect size of 1.28 (large) based on a paired-sample t test.

Table 3 presents MLM results for BMI data. As expected from the graphical summary, the effect of time was significant (P<.001), with the average estimated rate of change of .27 per month on the BMI scale. Controlling for age, individuals who completed more exercise Quests tended to have higher final BMI (P=.043); and individuals who played with less regularity also tended to have higher final BMI (P=.079).

For nutrition knowledge, the average value at 30 days was 73.30 (± 13.59) compared to the mean at 90 days of 78.68 (± 12.66). This increase corresponds to Cohen’s d effect size of |.856| (large) based on the paired t-test. Results of the MLM are summarized in Table 4. Overall, knowledge increased with time (P<.001), with the estimated increase of 2.003 units per month in the study. Individuals who completed more exercise Quests had a marginally higher final level of nutrition knowledge (P=.089).

A very promising result of the study is that the participants lost weight over the course of the three months. A significant and large change of almost one point on the BMI scale was observed, with the sample mean of 27 (SD 5.6) at baseline and 26.1 (SD 5.26) at the 3-month follow-up. The only other study that reported on BMI changes in the context of a game for diabetes management in adults [11] also found a drop in BMI, although of a smaller magnitude, .7 on the BMI scale over the course of 6 months. Sample demographics vary greatly between these 2 studies. At the same time, it is very encouraging to observe such a large effect. Further, individuals who completed more game activities (ie, cleaning the island, playing mini games), reported being in contemplation stage vs. preparation or action, were older, and who played 1-2 hours of video games per session tended to have higher final BMI scores.

According to the Transtheoretical Model [41-42], matching the intervention to cognitions and behaviors of individuals is of utmost important for ensuring success. Individualization of interventions for health behaviors, including diet and exercise, based on the individual’s readiness to change helps to move him or her along the behavior change trajectory. This theory found previous empirical support in traditional forms of interventions for dietary behaviors [43-45]. In the context of games, Lin et al. [14] carried out a qualitative study of participants engaged in step count linked to a virtual fish avatar. They found that those in the pre-contemplation or maintenance stages (both ends of the scale) were the least likely to change their daily steps. Findings from our study further support the relationship between behaviors and cognitions assessed through individuals’ readiness to change and their BMI. This has implications not only on how individuals are recruited but also clarifies the target audience and raises possibilities for individualizing game content explicitly depending on individual characteristics.

For nutrition knowledge, we also observed a large increase from the first month of assessment to follow up. While nutritional knowledge was intentionally not assessed at baseline, it is encouraging to see that the increase continued even after the first month of game play. Thus, it is likely that current findings are an underestimate of the game effect on nutritional knowledge. Similar findings of a large increase in knowledge of a food pyramid have been previously observed with a computer game to promote healthy diet in young adults (average age = 20 years) after 1 month of game play [10]. In our study, individuals who completed more food Quests and, interestingly, fewer exercise Sparks demonstrated higher levels of knowledge at post-test. In addition, women further along on the readiness to change their physical activity scale had significantly higher post-scores. These results support the fact that nutritional knowledge can be changed in a course of three months, even sporadic, game play intervention.

The overall low utilization of the SpaPlay program by the participants was a disappointing finding. On average, participants played for a week out of the 3 months, with only 50% playing more than 2 days. Gaps between logins were long, averaging 25 days (SD 14.23), and players largely exhibited an irregular play pattern as indicated by sporadic logins, estimated as an average standard deviation between logins of 16.48 (SD 7.66). Our previous usability studies [27,32] provided evidence for more frequent and consistent game play and generally good acceptance. Lower summaries from this study may be an artifact of design differences. The length of the qualitative study was 1 month, with interviews conducted regularly at one-week intervals, and interview questions informed by the play data [33]. This setup likely enforced more play than would occur naturally, without external reinforcement. These results may be a function of the instructions given to participants at baseline, where women were not explicitly told how often they should play the game. Rather, participants were expected to utilize the game as they would in real-life settings.

Further, we have examined game elements that were most frequently used. The participants favored briefer “Spark” activities over longer “Quests” that required a week-long commitment. It appears that short game features with immediate reinforcement were more appealing. A similar trend was found in our qualitative study and was largely expected.

We also found that some players engaged with the game more extensively than others, indicating a greater suitability of the game for some participants. This was supported by high variability scores in almost all game play elements, except for game and social activities. Women who, at baseline, reported being in the preparation and action stages of change on their diet played SpaPlay more frequently and consistently than those in the contemplation stage; they also completed more exercise Sparks. A theoretical application of the thranstheoretical model to game play adherence in the context of narrative-based games for health was proposed by Yin, Bickmore, and Montfort [46]; however, no empirical data were provided. Our findings present evidence that adherence to game play does depend on one’s cognitions and behaviors (as in preparation and action stages of change) and should be considered when designing and potentially individualizing games. Further research is needed to understand the construct of stages of changes more intimately in the context of games for health.

Another finding is that women who report playing video games 1-2 hours at a time had lower play frequency and consistency in the current game and completed fewer Sparks and Quests. Thus, a preference for spending a lot of time playing video games does not naturally translate into playing a health-related game. Taylor [47] discussed motivational factors for women who participate in multiplayer online gaming environments, which include social interaction, mastery, status, team participation, and exploration. The different style and context of SpaPlay seems to not appeal to this subgroup of women. This finding is important in understanding who might adopt or disregard this type of intervention. Interviewing women who engage in heavy bursts of game play could have shed more light into the nuances of the mismatch between their preferences and the current game, and could be a topic of a future investigation.

Results of the current study should be interpreted with caution in light of certain limitations. First, this study included a convenience sample of participants interested in playing computer games and who were at least contemplating losing weight. While this is a limitation, the sample is reflective of the population for whom we considered the game to be the most suitable and thus might resemble individuals who would seek a computer game to change their diet and exercise habits.

Second, the lack of a control group precludes complete certainty around the cause of weight loss and increase in nutritional knowledge. Given the pilot nature of the study, participants served as their own controls, which is strong but not absolute evidence of the game effectiveness. This could be an avenue of future research.

Third, more study follow-up data would have been beneficial for learning detailed lessons about players’ experiences. While our qualitative data from the previous study provides insight about satisfaction and acceptability [27,32], more data could provide additional insights into what works and what does not work when playing the game in more naturalistic settings. This should be considered in future studies, while considering participants’ fatigue from questionnaire responses.

Finally, all study measures were based on self-report. Since the current study is a part of an ongoing process of SpaPlay development, a new iteration of the game integrates objective measures of physical activity via Fitbit that is directly linked to game environment. Further investigations will make use of these objective data.

As a concept, “games for health” is relatively new, with very few studies systematically and comprehensively evaluating validity and effectiveness. The current study incrementally contributes to the field, and highlights the complexity of several issues related to adherence, deployment in real life, and individualization. However, it is encouraging to see the ability of the game to change BMI and nutritional knowledge, both important targets of many health interventions.