This study used a 2-study quantitative design in which the 2 studies addressed complementary but distinct research questions. In Study 1, the theory of planned behavior (TPB) [27] and SDT [21] informed the multicomponent intervention. In educational and behavioral research contexts, baseline pretesting may influence participants’ subsequent attitudes or behaviors through familiarity, learning, or recall, thereby threatening internal validity [28]. This phenomenon is commonly referred to as the practice effect, or pretest sensitization. A Solomon 4-group randomized design was used to evaluate the effect of a 12-week multicomponent app-supported physical activity intervention on university students’ core literacy in physical education and health, while accounting for potential pretest sensitization. Because the intervention incorporated structured physical activity, app-based monitoring and feedback, health education, gamified activities, peer interaction, goal setting, reminders, and researcher support, the experimental contrast estimated the effect of the intervention package rather than the isolated effect of the sports app.

In Study 2, we used the technology acceptance model [15] and SDT [21], an independent cross-sectional survey sample, and structural equation modeling (SEM) to examine the direct and indirect associations among sports app involvement, digitalization, gamification, and core literacy in physical education and health. In the Study 2 model, digitalization and gamification were specified as functional and experiential app-related dimensions, respectively.

The questionnaire drew on published instruments and relevant literature and was contextualized for sports app users and university students. Wording and contextual relevance were reviewed by 5 physical education experts, 3 doctoral students, and 8 master’s students, followed by student tryouts and refinements. Items were rated on a 5-point Likert-type scale, with higher scores indicating higher levels of the corresponding construct.

Before the formal Study 2 survey, the 4 instruments were evaluated using a pilot sample of 192 university students. Item screening was performed using the full pilot sample, after which the responses were divided into 2 subsamples comprising 96 university students each. One subsample was used for exploratory factor analysis of the app-related scales, and the other subsample was used for confirmatory factor analysis (CFA) of the retained measurement structures. The prespecified 3-domain core literacy structure was evaluated directly by CFA. Structural hypothesis testing used the subsequent independent 790-participant formal Study 2 sample. The detailed factor loadings, composite reliability (CR), and average variance extracted (AVE) are reported in Multimedia Appendix 1 (Table S2).

The Sports App Involvement Scale was adapted from the leisure involvement framework of McIntyre and Pigram [30] and contextualized for sports app use. The 10-item instrument retained 3 dimensions: attractiveness (4 items), self-expression (3 items), and life centrality (3 items). In the pilot validation, the 3-factor measurement model showed acceptable fit (χ²/df=1.479, standardized root mean square residual [SRMR]=0.038, RMSEA=0.071, and comparative fit index [CFI]=0.968).

Digitalization was assessed using an 8-item scale adapted from the monitoring and feedback measures reported by Nelson et al [18], with item wording contextualized from activity trackers to sports apps. The monitoring dimension comprises 4 items addressing the amount, presentation, constructiveness, and frequency of information, whereas the feedback dimension comprises 4 items addressing perceived accuracy, usefulness, effectiveness, and reliability. In the pilot validation, the 2-factor measurement model showed acceptable fit (χ²/df=1.365, SRMR=0.038, RMSEA=0.062, and CFI=0.977).

The Gamification Scale was developed for the sports app context, rather than being directly adopted from a single existing instrument. The dimensional structure of this scale was informed by the SDT and prior research on motivational and game-related experiences [21,22]. Following literature review and expert refinement, gamification was operationalized using 9 items across 3 dimensions: sense of mastery, interaction fluency, and intrinsic motivation. In the pilot validation, the 3-factor measurement model showed acceptable fit (χ²/df=1.228, SRMR=0.042, RMSEA=0.049, and CFI=0.979).

The Physical Education and Health Core Literacy Scale was developed for this study based on the 3-domain framework specified in the Chinese Physical Education and Health Curriculum Standards and informed by earlier Chinese instruments assessing core literacy among university students. The initial scale included 30 items covering motor competence, health behaviors, and sportsmanship. However, items D9, D14, D29, and D30 were removed during the pilot item analysis. Items D9, D14, and D29 did not meet the prespecified extreme-group discrimination criterion (t≥3.00), whereas item D30 had a corrected item-total correlation of 0.374, which was below the 0.400 criterion. The resulting 26-item scale comprised 8 motor competence items, 11 health behavior items, and 7 sportsmanship items and was used in the formal survey. Item removal was guided by prespecified psychometric criteria while retaining the representation of all 3 theoretically defined domains, thereby preserving the intended 3-domain structure of the scale. The retained 3-factor measurement model showed acceptable fit in the pilot CFA (χ²/df=1.267, SRMR=0.045, RMSEA=0.053, and CFI=0.948).

In both studies, core literacy in physical education and health and its 3 dimensions were assessed using participant-reported questionnaires. The sources, adaptation procedures, and final structures of the 4 Study 2 instruments are summarized in Multimedia Appendix 1 (Table S3).

Study 1 analyses were conducted using SPSS (version 27.0). The primary analysis was a 2×2 factorial ANOVA of postintervention scores, with pretest exposure and intervention condition as between-participant factors. As a supporting analysis, postintervention scores in the pretested experimental and control groups were compared using analysis of covariance (ANCOVA), adjusting for the corresponding baseline scores. Group-by-baseline interaction terms were examined to assess the homogeneity of regression slopes. These analyses included participants with available postintervention outcomes. Baseline characteristics of completers and noncompleters in the pretested groups were compared using Welch independent-samples t tests and Fisher exact tests, as appropriate.

Sensitivity to the 13 missing postintervention assessments was examined using reference-based resampling with 10,000 replications. Within each pretest stratum, missing outcomes in the intervention groups were sampled with replacement from the observed outcome vectors in the corresponding control group, assuming no intervention benefit among intervention-group noncompleters. Missing control-group outcomes were sampled from the corresponding observed control-group outcomes. The 3 outcomes were sampled jointly to preserve their within-participant correlation structure. The structural absence of pretest measurements in the unpretested groups was not treated as missing data. This sensitivity analysis supplemented the complete-case analyses; it did not constitute a conventional intention-to-treat analysis.

Study 2 analyses used SPSS (version 27.0) and Amos (version 29.0). Internal consistency was assessed using Cronbach α and discriminant validity using the heterotrait-monotrait ratio. Common method effects were examined using Harman’s single-factor test and a CFA comparison of the hypothesized 4-factor and 1-factor models. Model fit was evaluated using χ²/df, CFI, Tucker-Lewis index (TLI), RMSEA, and SRMR. Descriptive statistics and Pearson correlations were also calculated.

The primary age- and sex-adjusted latent-variable SEM estimated direct and indirect associations, with bias-corrected (BC) CIs obtained from 5000 bootstrap resamples. The latent-variable specification was retained to account for measurement error in the construct indicators. The inclusion of only 8 universities provided limited university-level information for a full multilevel latent model. The primary SEM assumed independence between participants, and its SEs and bootstrap CIs were not adjusted for university-level clustering.

University-level clustering was described using intercept-only random-effects models, intraclass correlation coefficients, and design effects. Two secondary analyses used standardized composite scores. One added BMI and 7 university indicator variables to the age- and sex-adjusted models. The other applied university-clustered robust standard errors to the age- and sex-adjusted composite-score models, with a finite-sample correction and t-based inference using 7 df, implemented in Python (version 3.13.5) with statsmodels (version 0.14.6). These analyses assessed the consistency of the 6 composite-score paths; cluster-adjusted CIs for the primary latent-model indirect associations were not estimated.

Of the 120 randomized participants, 107 (89.2%) completed the postintervention assessment: 93% (28/30) in the PE group, 83% (25/30) in the PC group, 83% (25/30) in the UE group, and 97% (29/30) in the UC group. In total, 45.8% (49/107) of the participants who completed the postintervention assessment were men; the complete-case sample had a mean age of 20.56 (SD 1.59) years. Detailed participant characteristics are provided in Multimedia Appendix 1, Table S4, and the reasons for 10.8% (13/120) noncompletions are shown in Figure 3.

Among the 60 pretested participants, 53 completed the postintervention assessment and 7 did not. Completers and noncompleters did not differ significantly at baseline in sex, age, anthropometric characteristics, or core literacy measures (Multimedia Appendix 1, Table S5). No exercise-related adverse events were reported.

The primary analysis in Study 1 was a 2×2 factorial ANOVA of posttest scores, with pretest exposure and intervention conditions as the between-group factors. Significant intervention main effects were observed for self-reported motor competence (F_1,103=33.869, P<.001, ηp²=0.247), health behaviors (F_1,103=21.548, P<.001, ηp²=0.173), and sportsmanship (F_1,103=30.261, P<.001, ηp²=0.227), with higher scores in the intervention condition (Table 2).

Pretesting showed significant main effects on motor competence (F_1,103=22.290, P<.001, ηp²=0.178) and health behaviors (F_1,103=19.489, P<.001, ηp²=0.159), but not on sportsmanship (F_1,103=3.478, P=.07, ηp²=0.033). The Pretest×Intervention interaction was not statistically significant for motor competence (F_1,103=0.011, P=.92, ηp²=0.000), health behaviors (F_1,103=0.008, P=.93, ηp²=0.000), or sportsmanship (F_1,103=0.072, P=.79, ηp²=0.001), providing no evidence that the intervention effect differed according to pretest exposure.

As a supporting analysis among the pretest groups, posttest outcomes were compared between the PE and PC groups using an ANCOVA while controlling for baseline scores. The group-by-baseline interaction terms were not statistically significant across the three outcomes, supporting the assumption of homogeneity in the regression slopes.

After adjustment for baseline scores, the intervention group had higher self-reported posttest scores than the control group for motor competence (F_1,50=12.703, P<.001, ηp²=0.203), health behaviors (F_1,50=9.481, P=.003, ηp²=0.159), and sportsmanship (F_1,50=14.713, P<.001, ηp²=0.227). These results were consistent with those of the primary factorial analysis (Table 3).

Under the specified control-reference assumptions, the reference-based resampling sensitivity analysis yielded results consistent with those of the complete-case analysis. Across all 10,000 replicates, the main effect of the intervention remained statistically significant for motor competence, health behaviors, and sportsmanship. The median intervention-effect estimates for motor competence, health behaviors, and sportsmanship were F_1,116=27.60 (ηp²=0.192), F_1,116=17.94 (ηp²=0.134), and F_1,116=24.52 (ηp²=0.175), respectively. The detailed simulation summaries are provided in Multimedia Appendix 1 (Table S6).

Detailed exploratory Tukey honestly significant difference comparisons are provided in Multimedia Appendix 1 (Table S7). In brief, the corresponding intervention-control contrasts were statistically significant for all 3 outcomes under the pretested and unpretested conditions; exact P values are reported in Table S7 in Multimedia Appendix 1. However, as the pretest × intervention interactions were not statistically significant, these pairwise contrasts were interpreted as descriptive rather than primary evidence.

The final Study 2 sample comprised 790 university students, of whom 409 (51.8%) were men. The mean age of this sample was 20.33 (SD 1.78) years, with an age range of 18 to 25 years. Detailed demographic characteristics and university-specific sample sizes ranging from 61 to 117 per university are presented in Multimedia Appendix 1 (Table S8). The intraclass correlation coefficients for sports app involvement and digitalization were 0.010 and 0.002, respectively, whereas those for gamification and core literacy in physical education and health were approximately 0; the corresponding design effects were 1.99, 1.18, 1.00, and 1.00, respectively (Multimedia Appendix 1, Table S9).

In the pilot CFA subsample (n=96), the standardized factor loadings for the retained items ranged from 0.604 to 0.834. AVE values across the 11 first-order dimensions ranged from 0.508 to 0.617, and CR values ranged from 0.757 to 0.934, supporting convergent validity (Multimedia Appendix 1, Table S2).

The final Study 2 sample had good internal consistency, with Cronbach α values of 0.890 for sports app involvement, 0.848 for digitalization, 0.814 for gamification, and 0.924 for core literacy in physical education and health. In the formal 4-factor measurement model, standardized dimension-level factor loadings ranged from 0.703 to 0.906, CR values ranged from 0.770 to 0.865, and AVE values ranged from 0.527 to 0.683. The heterotrait-monotrait ratios among the four focal constructs ranged from 0.426 to 0.638, with all values below the conservative 0.85 criterion, supporting the empirical distinction among the four constructs (Multimedia Appendix 1, Tables S10 and S11).

Harman single-factor test identified nine factors with eigenvalues >1, with the first factor accounting for 25.1% of the total variance. The hypothesized 4-factor model showed an acceptable fit (χ²₃₈=129.39, χ²/df=3.405, CFI=0.978, TLI=0.968, RMSEA=0.055, SRMR=0.030), whereas the 1-factor model showed a poor fit (χ²₄₄=1401.80, χ²/df=31.859, CFI=0.670, TLI=0.587, RMSEA=0.198, SRMR=0.119). These findings argue against a single dominant common factor; however, common method effects cannot be ruled out because the focal measures were self-reported and collected simultaneously.

Descriptive statistics and correlations are presented in Table 4. Correlations among the 4 focal constructs were positive, ranging from r=0.386 to r=0.538 (all P<.001).

The primary single-level latent-variable model showed an acceptable fit for the cross-sectional data (χ²₅₄=185.892, χ²/df=3.442, CFI=0.968, TLI=0.953, RMSEA=0.056, 90% CI 0.047‐0.064). Sports app involvement was associated with core literacy in physical education and health (β=.135, P=.01), digitalization (β=.653, P<.001), and gamification (β=.279, P<.001). Digitalization was associated with core literacy (β=.374, P<.001) and gamification (β=.486, P<.001), and gamification was associated with core literacy (β=.135, P=.02; Table 5 and Figure 4).

Inference from this primary model was based on the assumption of independence between participants.

The primary latent model was adjusted for age and sex and assumed independence between participants; its SEs were not adjusted for university-level clustering.

Bootstrap analysis showed a model-estimated total association of 0.385 (BC 95% CI 0.296‐0.478). The direct association was 0.113 (BC 95% CI 0.016‐0.210), accounting for 29.5% of the model-estimated total association, and the total indirect association was 0.271 (BC 95% CI 0.194‐0.361), accounting for 70.5%. The specific indirect associations through digitalization (0.204, BC 95% CI 0.126‐0.290), gamification (0.031, BC 95% CI 0.004‐0.077), and digitalization followed by gamification (0.036, BC 95% CI 0.005‐0.081) accounted for 53.1%, 8.2%, and 9.3% of the model-estimated total association, respectively (Table 6).

In the secondary sensitivity path analysis based on standardized composite scores, adding BMI and 7 university indicator variables to the age- and sex-adjusted models resulted in only small changes in the 6 focal coefficients (absolute changes ≤0.010), and all 6 focal paths remained statistically significant (all P<.001; Multimedia Appendix 1, Table S9, panel B). In the university-clustered analysis of the age- and sex-adjusted composite-score models, all 6 focal paths remained positive and statistically significant (all P<.05; Multimedia Appendix 1, Table S9, panel C). These secondary results supported the consistency of the 6 composite-score paths.