Participants were recruited in France between November and December 2024 through a study advertisement posted on LinkedIn. Interested individuals contacted the research team and were scheduled for an in-person laboratory session. Data collection took place in controlled laboratory settings at the Centre for Research in Psychology: Cognition, Psychism, and Organizations (CRP-CPO, University of Picardie Jules Verne) and the Lorraine Laboratory of Psychology and Neuroscience of Behavioral Dynamics (2LPN, University of Lorraine). All sessions were conducted in quiet rooms without distractions to ensure standardized testing conditions.

The study used convenience sampling with self-selection. A public recruitment post was published on LinkedIn, inviting adults to participate in a laboratory-based study. Interested individuals self-referred by contacting the research team and were screened for eligibility based on the inclusion and exclusion criteria. Eligible individuals were then scheduled for an in-person laboratory session.

The intended sample size was approximately 40 participants, primarily determined by feasibility constraints for in-laboratory testing and by the objective of detecting at least moderate convergent associations in an early-stage validation study. The achieved sample size was 39 participants for the full session. For the teamwork module, the analytic sample comprised 35 participants because game data for that module were not available for 4 participants.

With a sample size of 39 participants, the study has approximately 80% power (2-sided α=.05) to detect correlations of r or ρ=0.44 or greater. For the teamwork module (n=35), the corresponding threshold was approximately 0.46. Therefore, the study was not designed to reliably detect small associations. No interim analyses or stopping rules were prespecified.

This convergent validity study compared 3 assessment modalities embedded in the serious game Yuzu with corresponding reference instruments administered outside the game. Active-empathic listening was assessed in Yuzu using a questionnaire module based on the AELS [38,39]. The reference measure was the original AELS questionnaire [38,39]. Decision-making under uncertainty was assessed in Yuzu using an adaptation of the IGT [40]. The reference measure was the original IGT administered online [43]. Teamwork-related interpersonal behavior was assessed in Yuzu through structured dialogue choices, operationalized using the SYMLOG framework [41]. The reference measure was the simplified SYMLOG questionnaire by Blumberg [42].

To evaluate whether performance could be biased by participants’ relationships with digital technology, measures of computer literacy and technophilia were collected. In addition, participants completed a game experience questionnaire based on the playability concept [44] and a demographic questionnaire during the debriefing phase. No other covariates were included in the primary inferential analyses.

Yuzu is a single-player serious game for soft skills assessment that integrates multiple modalities within a single narrative 3D environment. Players navigate a space station, interact with nonplayable characters, complete tasks, and make decisions under constraints (Figure 2).

All actions are recorded through embedded trackers, producing behavioral indicators such as response patterns and choice sequences. Yuzu was designed by a multidisciplinary team (work psychology and ergonomics, engineering, and user interface design) in collaboration with the 2LPN research laboratory (University of Lorraine, France).

Active listening is commonly assessed using self-report scales because it reflects cognitive and emotional processes that can be difficult to capture through performance tasks. The AELS is a widely used and validated tool for measuring listening quality in interpersonal communication [38,39]. It distinguishes 3 components: sensing, processing, and responding. In Yuzu, these components served as the conceptual framework for an original in-game questionnaire module. Items were specifically formulated for the serious game, and responses were collected using a 5-point Likert-type scale. The module yields an overall active listening score and 3 subscores corresponding to sensing, processing, and responding. These were compared with the reference AELS questionnaire [38]. The interface of the active listening module is shown in Figure 3.

Within the narrative, this module appears after the player awakens from a prolonged sleep: the station’s physician administers a short questionnaire to verify the participant’s condition and adaptation to the situation.

We selected the IGT [40] as the reference paradigm for decision-making under uncertainty. In the classical IGT, participants complete 100 trials and choose among 4 decks (A, B, C, and D). Decks A and B are disadvantageous over the long term (high immediate gains but long-term losses), whereas decks C and D are advantageous (lower immediate gains but long-term benefits). Expected learning is reflected in an increasing preference for advantageous decks over the course of the task.

In Yuzu, the core principles of the IGT were retained and transposed into a serious game format in which participants collected meteorite samples using drones across 4 excavation sites. The 100 trials were divided into 10 blocks of 10 choices each. Consistent with prior work [45], the first 40 trials (blocks 1-4) were treated as an exploration phase and the remaining 60 trials (blocks 5-10) as an exploitation phase. The interface of the game-based IGT adaptation is shown in Figure 4.

Interpersonal behavior in team settings was operationalized using the SYMLOG framework [41], which positions actions along core social dimensions such as dominance, affiliation, and task orientation. The Yuzu teamwork scenario places the player in an emergency situation in which a spacecraft approaches the space station dangerously and rapid decisions must be made. Participants completed 4 sets of dialogue choices during interactions with nonplayer characters. Response options varied in both content and interpersonal tone (eg, more or less dominant or friendly), and some proposals originated from Earth as an external authority, allowing players to either follow or reject directives.

Each response option was precategorized along 3 axes aligned with SYMLOG: dominance vs submission, friendliness vs hostility, and acceptance vs rejection of norms and tasks. For each participant, mean scores were computed for these dimensions to derive an in-game interpersonal profile, which was compared with the simplified SYMLOG questionnaire [42] administered outside the game. The dialogue-choice interface used in the teamwork module is shown in Figure 5.

All statistical tests used a 2-sided α level of .05. For equivalence testing, the TOST procedure was conducted at α=.05; therefore, 90% CIs are reported for equivalence tests, whereas 95% CIs are reported elsewhere. Missing data were limited to the teamwork module: 4 of 39 participants (10.3%) had missing in-game teamwork data because the module data were not saved. These cases were excluded from teamwork analyses only, resulting in an analytic sample size of 35 participants. No imputation was performed. Because the missingness was caused by a technical data-saving issue, it was assumed to be unrelated to the underlying teamwork construct; however, this assumption cannot be formally verified.

Results are reported in the order of the hypotheses stated in the Introduction section. Listening and teamwork hypotheses were tested using Spearman rank correlations with bootstrap CIs. Decision-making equivalence was tested using TOST and complemented by a paired 2-tailed t test and an effect size estimate.

Convergent validity was supported for active listening. Yuzu’s active listening score showed a strong association with the reference AELS total score (Spearman ρ=0.890, 95% CI 0.804-0.948; P<.001; 95% CIs estimated using bootstrap resampling [BCa, 5000 resamples]).

Agreement between the 2 versions was evaluated using Bland-Altman analysis (Table 2; Figure 7). The mean bias was close to 0, and the limits of agreement indicated no large systematic disagreement between the formats.

The Lin concordance correlation coefficient (CCC) indicated good-to-excellent agreement between the 2 versions (CCC=0.899, 95% CI 0.817-0.946). Overall, these results support convergent validity and close agreement between Yuzu’s listening module and the reference AELS.

The analyses supported the equivalence between the Yuzu adaptation and the reference IGT during the exploitation phase. The TOST 90% CI for the mean difference fell entirely within the prespecified equivalence interval of −5 to +5, and both one-sided tests were significant (Table 3; Figure 8). The paired 2-tailed t test was not statistically significant, and the standardized effect size was small (Table 3).

To describe learning dynamics, Figure 9 shows the evolution of the mean net score across blocks of 10 trials for both modalities. In both conditions, performance increased across blocks, with a similar shift from exploration to exploitation beginning at block 5.

Convergent validity was not supported for the teamwork module. Yuzu-derived teamwork profiles were clustered toward a highly cooperative style, consistent with a ceiling effect. Spearman rank correlations between Yuzu’s 3 dimensions and the corresponding questionnaire dimensions were not statistically significant (Table 4).

Descriptive distributions suggested that the questionnaire differentiated participants across the 3 axes, whereas the Yuzu-derived scores were compressed and shifted in a way consistent with limited variance (Table 5).

Taken together, these results suggest that serious games can support convergent validity for some soft-skill constructs and modalities, but that validity evidence is highly dependent on how the construct is operationalized in gameplay and on the extent to which the measurement modality introduces method-related variance [50,51]. In practice, modules that closely preserve the structure of a reference instrument (eg, questionnaire-based formats) or a well-defined task paradigm (eg, feedback-based decision tasks) may translate more readily into game-based formats, whereas dialogue-choice approaches to interpersonal profiling may require stronger differentiation in scenario design and response options to generate sufficient variance for construct validation.

The very high correlation and strong agreement between Yuzu and the AELS indicate that the in-game questionnaire module produced scores that were closely aligned with the reference instrument. One interpretation is that the narrative embedding of the questionnaire did not distort the construct being measured while preserving the conceptual structure of the original scale [38,39]. However, because both measures rely on self-reported judgments, shared method variance is a plausible contributor to the observed strength of the association [51]. In addition, response biases associated with self-report, including impression management and socially desirable responding, may remain present even when the questionnaire is integrated into a game narrative [24]. Therefore, the present findings support convergent validity with an established self-report instrument, but they do not yet demonstrate that Yuzu captures listening behavior as it unfolds in real interactions.

Equivalence results indicate that the Yuzu adaptation preserved the decision-making tendencies captured by the reference IGT during the exploitation phase. This finding suggests that the gamified interface and narrative context did not substantially alter the balance between immediate rewards and long-term outcomes that the task is intended to probe [40]. The parallel block-wise trajectories are consistent with the interpretation that participants learned from feedback in a comparable manner across both modalities, in line with previous work [45].

This result is noteworthy because it suggests that the structural properties of a well-defined cognitive paradigm can be preserved when transposed into a game narrative, provided that the core contingency structure, here the asymmetric reward schedule across decks, remains intact. From a measurement standpoint, task-based modules of this kind may be more robust to gamification than questionnaire or dialogue-based formats, because scoring relies on behavioral sequences rather than self-report, thereby reducing susceptibility to socially desirable responding [24]. Future work should examine whether this equivalence holds across different gamified implementations and population subgroups.

The absence of convergent validity for the teamwork module can plausibly be explained by restricted variance and a ceiling effect in the in-game indicators. If most participants select options that are clearly cooperative and norm-conforming, the resulting scores cannot differentiate interpersonal styles and will correlate poorly with questionnaire measures that capture stable individual differences. This aligns with the general expectation that restriction of range attenuates correlations [52]. This pattern may also reflect the scenario’s incentive structure. Cooperation might have been framed as the most sensible option for narrative success, making alternative responses appear irrational or socially inappropriate. Such tendencies are compatible with the broader literature on socially desirable responding, in which respondents may favor options that maintain a positive self-image or align with perceived expectations [24].

Another possibility is a construct mismatch. The simplified SYMLOG questionnaire assesses broad interpersonal tendencies, whereas the Yuzu dialogue choices may have captured context-specific decision strategies under an emergency narrative. Like situational judgment tests, the dialogue-choice module may primarily capture respondents’ implicit trait policies or knowledge about which interpersonal behaviors are effective in a given situation, whereas the SYMLOG questionnaire targets broader dispositional teamwork tendencies [53]. In that case, the lack of convergence could indicate that the game module measured situational behavior shaped by the scenario rather than the trait-like interpersonal profile assessed by the questionnaire.

From a measurement perspective, the current dialogue-choice format may also suffer from limited behavioral sampling. With only a small number of choice points, measurement precision is limited, and individual scores become sensitive to idiosyncratic responses to specific prompts. This issue is consistent with discussions of scenario-based assessment formats, where item sampling and option design strongly influence reliability and construct representation [4,54]. Increasing the number of observations, diversifying situations, and designing options that are less transparently “good” or “bad” could improve score reliability and increase variance.