The Cantonal Ethics Committee of Zurich, Switzerland, reviewed the study protocol and issued a declaration of no objection (Business Management System for Ethics Committees Number Req-2020-00059). We collected all data under written informed consent by all participants.

This investigator-initiated, prospective, randomized, eye-tracking study investigated anesthesia providers’ visual attention on the patient monitor during simulated critical anesthesia events. We used three different patient monitor screen modalities (Figure 1). We collected all eye-tracking data during a simulation study conducted in May 2020 at the University Hospital Zurich in Switzerland [20].

We included 52 anesthesia teams consisting of a nurse anesthetist and an anesthesiologist. In randomized order, each team simulated the scenario of severe bronchospasm, myocardial infarction, and malignant hyperthermia once. Each simulated scenario lasted 10 minutes. For each scenario, the teams used a different screen modality: only conventional patient monitoring (ie, number- and waveform-based monitoring), only the avatar-based patient monitoring (visual-patient-avatar), or split-screen monitoring consisting of both modalities shown side-by-side simultaneously (Figure 1). In a randomized order, we chose one of the team members to be the team leader. During the scenarios, mentioned leader (ie, either the nurse anesthetist or the physician) wore a mobile eye-tracking device (Pupil Invisible; Pupil Labs, GmbH) while the team managed the critical incident together. We used Research Randomizer Version 4.0 [28] to randomize the order of the scenarios, respective team leaders, and screen modality.

We conducted the study in a backup operating room with an analogous setup as the study center’s active operating rooms. To enhance the simulation fidelity, we used real medications and airway management tools in addition to a state-of-the-art, full human patient simulator (HAL S301; Gaumard Scientific Company, Inc). We used a Philips InteliVue MX500 (Koninklijke Philips NV) patient monitor to closely resemble the study center’s MX550 and MX800 monitors. In the simulation environment, we tagged the patient monitor as our area of interest using pre-generated QR codes. These QR codes enable the Pupil Player software to automatically detect mentioned areas of interest, a prerequisite for automated processing. In Multimedia Appendix 1, we present a video sequence showing an anesthesia team managing a critical anesthesia event, providing a good overview of the simulation environment, the patient monitor used, and the eye-tracking footage.

Before starting each scenario, we calibrated the eye-tracking device to the participant. We recorded the subject's field of view as a video feed with Pupil Invisible, a mobile eye-tracking device similar in shape and size to regular glasses. The device was connected to a mobile phone that participants carried in their pockets, which served as a power source and storage unit. After each recording, we uploaded the eye-tracking data to a research server and made backup copies on physical hard drives.

For the analysis, we first manually checked the eye-tracking data's quality. Then, for the post hoc semi-automated video analysis, we used Pupil Labs proprietary software Pupil Player on an Acer Aspire V15 Nitro laptop (Acer Inc). Within Pupil Player, we delimited the patient monitor as our area of interest, using the surface tracker plugin. Using the fixations detector plugin and cropping the videos to the start and end of the 10-minute simulation (designated by a bell signal), we were then able to automatically export all fixations and their durations as Microsoft Excel spreadsheets (version 16.58; Microsoft Corporation). During post hoc editing, all recordings were manually stopped five times per recording to ensure the accuracy of the boundaries of the areas of interest. Multimedia Appendix 2 shows an example sequence of the analyzed eye-tracking data in Pupil Player.

As our primary outcome, we defined visual attention as fixation count and dwell time. As a fixation, we counted every instance where the subject’s gaze rested in one single location within the area of interest for more than 100ms. The dwell-time corresponds to the cumulative time in seconds that the participant’s gaze was focused on the predefined area of interest. In addition to visual attention, we collected potentially influencing variables such as screen type (only conventional, only avatar or split-screen monitoring), scenario (bronchospasm, myocardial infarction, or malignant hyperthermia), sequence of the scenarios, profession (nurse anesthetist or anesthesiologist), and work experience (in years). Furthermore, we looked at the relation of our primary outcome and the participant’s task performance. The task performance was based on the time required for participants to perform critical diagnostic and therapeutic tasks during the scenarios [20]. An example of such a therapeutic task in the malignant hyperthermia scenario is stopping the trigger or administrating dantrolene.

In this exploratory study design, a power calculation was not performed. For descriptive statistics, we report means with standard deviation and medians with IQRs for continuous data and numbers and percentages for categorical data. We used mixed linear models to analyze fixations count and dwell time. In the models, we included the potentially influential variables task performance, screen type, scenario, sequence of the scenarios, profession, and work experience as covariates. We used R version 4.0.5 (R Foundation for Statistical Computing,) to analyze all data and used Prism 9 (GraphPad Software Inc) to create all figures. We considered a P-value of less than .05 to determine statistical significance.

Overall, participants had a median (IQR) of 198 (93-318) fixations on the monitor with a median (IQR) dwell time of 30.2 (14.9-51.3) seconds. This means that participants spent around 5% of their visual attention on the patient monitor screen during the 10-minute simulation.

Comparing the fixations on the two avatar-based screen modalities with the fixations to conventional monitoring, the mixed linear model yielded a coefficient of 4.89 fixations (95% CI –59.57 to 69.35; P=.89) for only avatar-based monitoring and a coefficient of -4.33 fixations (95%CI: –74.55 to 65.90; P=.91) for the split-screen modality. This means that participants using only avatar-based monitoring in the simulated scenarios had about five fixations more, and participants using the split-screen modality had about four fixations less than when they used conventional monitoring. However, those results were not statistically significant.

Comparing the dwell time on the avatar-based screen modalities with the dwell time on the conventional monitor using an adjusted, mixed linear model, we found a coefficient of 1.56 seconds (95% CI –8.98 to 12.09; P=.78) for only avatar-based monitoring and a coefficient of –0.02 seconds (95% CI –11.50 to 11.46; P=1.00) for the split-screen modality. This means that participants using only avatar-based monitoring in the simulated scenarios looked around 1.6 seconds longer on the monitor screen. Participants using the split-screen spent as much time on the patient monitor as those using conventional monitoring. However, those results were also not statistically significant.

Regarding task performance, the mixed linear models showed that the fixation count and the dwell time decreased with better task performance regardless of the screen modality used (fixation: coefficient –1.39; 95% CI –2.44 to –0.34; P=.02 and dwell time: coefficient –0.23; 95%CI –0.4 to –0.06; P=.01). Thus, with each percentage point of better task performance, the number of fixations decreased by about 1.39, and the dwell time diminished by 0.23 seconds.

Regarding the potentially influential variables, profession, work experience, and sequence of the scenarios, the mixed linear models yielded no evidence for differences in fixations and dwell time (Figure 3). However, the scenarios themselves differed with respect to fixation and dwell time (Figure 3). Participants had significantly more fixations (coefficient 141.97; 95% CI 75.62 to 208.32; P<.001) and a higher dwell time (coefficient 23.23; 95%CI 12.48 to 34.08; P<.001) on the patient monitor in the myocardial infarction scenario.

Further, we found that for the conventional half of the split-screen modality, participants had 158 (IQR 63-226) fixations and a dwell time of 24.3 (IQR 10.0-36.8) seconds. For the avatar-based half of the split-screen modality, participants had 44 (IQR 28-84) fixations and a dwell time of 6.8 (IQR 4.3-13.3) seconds. Using a Mann-Whitney test to compare both halves of the split-screen, we found that subjects had significantly fewer fixations (P<.001) and significantly less dwell time (P=.001) on the avatar part of the patient monitor.

This study investigated whether avatar-based patient monitoring influences the visual attention of anesthesia providers on the patient monitor. We assessed 99 eye-tracking videos of anesthesia personnel managing simulated critical anesthesia events. We found no significant difference in visual attention when anesthesia providers used avatar-based or conventional patient monitoring in simulated critical anesthesia events using adjusted, mixed linear models.

Anesthesia personnel devoted about 5% of their time to the patient monitor. These results are consistent with findings under real-life conditions [29]. However, other simulation studies reported higher percentages of dwell time on the patient monitor [29,30]. The high fidelity of our simulations may explain these differences. We used an in-situ simulation design and enhanced the simulation’s fidelity further by using real medications and airway management tools in addition to a state-of-the-art, full human patient simulator. Thus, it is conceivable that a very realistic simulation is more likely to align participant behavior with outcomes under real-life conditions than a simulation with lower fidelity.

We hypothesized that the anesthesia provider's visual attention on the patient monitor would decrease with avatar-based patient monitoring due to accelerated and simplified information transfer found in the previous studies [17-19,26,27]. The basic idea behind this hypothesis is that the qualitative visualization of the patient avatar may lead to a quicker overview of the patient's situation [27,31]. In addition, avatar-based patient monitoring highlights pathophysiological changes, eliminating the time-consuming task of creating a mental model from the various numerical values of the conventional patient monitor [5,6,32]. This may speed up the perception of the situation, lead to fewer fixations and less dwell time on irrelevant vital signs and consequently lead to less visual attention on the patient monitor. In other words, the anesthesia provider knows where to look and can therefore perceive the necessary information with less visual attention. However, this eye-tracking study did not confirm our hypothesis. We found no significant difference in participants' visual attention when using avatar-based monitoring compared to conventional patient monitoring. Unfamiliarity with the novel, avatar-based technology may have masked its potential benefits and may serve as a possible explanation for the finding as all participants used the visual-patient-avatar for the first time. Benefits and acceptance of a new technology depend heavily on users' exposure [33].

However, we found significant differences between the two halves of the split-screen modality. Participants had significantly more visual attention on the conventional part than on the avatar part of the split-screen monitor. This may indicate an interaction effect after all. Perhaps the avatar drew their attention to something (eg, a vital sign outside the norm) that they checked on the conventional screen. Because the qualitative visualization provided by the avatar is intuitive and quickly understood [27,31], participants paid relatively little visual attention to the avatar. To verify the qualitative input, participants had to extract information from the various numbers and waveforms on the conventional screen, a time-consuming task [5,6,32].

Qualitative data collection on how participants used the patient monitors might have helped clarify the result concerning the split-screen modality. Mixed methods are often more powerful than purely quantitative data analysis for such complex human factors work [34].

We found that an increase in the anesthesia team’s task performance was associated with decreased visual attention. The correlation between the two parameters supports the notion that visual attention and task performance act as indirect indicators of situation awareness [4,30,35-38]. The distribution of visual attention determines what is in the perceptual field and, therefore, contributes to the sensory input, an essential aspect of perception (ie, situation awareness level I) [32]. Good clinical performance in managing the simulated critical anesthesia events comes at the end of good decision-making [38,39], which requires a sufficient understanding (ie, situation awareness level II) and a projection of the situation's near future (ie, situation awareness level III) [32]. A combination of eye tracking and performance measures simultaneously determined what information had been seen and to what degree this information had been perceived and comprehended by the anesthesia provider, giving us a good idea about all three levels of situation awareness achieved (Figure 4).

This study may contribute exciting aspects to the current debate on how to best measure situation awareness [4,30,40-42]. Questionnaires to be used during simulations were proposed and validated as direct measurement tools for situation awareness [4,40]. These direct measurement methods require pausing the simulation to answer the questionnaire before resuming the task again [40]. Evidently, this instrument for assessing situation awareness has limited application in clinical reality, as there is no time to stop the treatment of a critical patient to interview the treating physician. For this reason, we propose the combination of the indirect measurement parameters visual attention and task performance as a surrogate when measurements of situation awareness in clinical praxis are wanted.

The three simulated emergencies received different amounts of visual attention. In the myocardial infarction scenario, participants looked at the patient monitor more frequently and had a higher dwell time (Figure 3). In this simulation, there was the additional option of displaying a 12-lead electrocardiogram on the patient monitor, necessary to diagnose myocardial infarction. Although we manually cropped the 12-lead electrocardiogram sequence because we did not analyze it in the context of conventional patient monitoring or avatar-based patient monitoring, participants may have searched for a relatively long time on the monitor to activate the 12-lead electrocardiogram function. This circumstance may explain the significantly increased visual attention on the patient monitor in this scenario.

Furthermore, we found a tendency (not significant) that anesthesia nurses had more fixations and a higher dwell time on the patient monitor (Figure 3). Anesthesia nurses perform important preparatory tasks but generally spend less time with direct anesthesia management. This circumstance may result in anesthesia nurses paying slightly more visual attention to the patient monitor to gain the same level of situational awareness as an anesthesiologist whose main task is anesthesia management.

Our study had several limitations. First, we had to exclude more than one-third of all simulations from our analysis due to technical issues or poor data quality of the eye-tracking footage. Although we used one of the latest mobile eye-tracking devices on the market, we faced several challenges while recording the data: calibrating the glasses for ocular pathologies (eg, alternating squinting or wearing prescription glasses), battery-life issues, or the device slipping of the participants face during physical tasks (eg, manual resuscitation). This shows that despite the massive development of eye-tracking hardware and software in recent years, the technology is still error-prone. Second, all study participants were unfamiliar with avatar-based patient monitoring. Therefore, the results may vary as anesthesia personnel becomes accustomed to this new technology. Third, the median work experience of our participants is relatively low at four years. Finally, we conducted this study in a university hospital in Europe. Therefore, results may differ under other conditions and in other parts of the world.

The study had several strengths. First, we reduced selection bias by balanced participant selection and consistent randomization of the scenario sequence, team leader, and screen-modality. Second, a rigorous manual quality check of the eye-tracking recordings ensured excellent data quality, allowing us to replace error-prone manual eye-tracking analysis with automated analysis. Third, we attempted to circumvent the bias of authenticity inherent in all simulation-based studies [43] through our in-situ study design and our efforts to represent clinical reality as accurately as possible in this high-fidelity simulation study. Finally, we used eye-tracking hardware that was no more distracting than wearing regular glasses to produce high-quality video footage.

We found no significant difference in visual attention when anesthesiologists used the novel avatar-based or the conventional patient monitoring in simulated critical anesthesia events. However, when using the split-screen displaying the conventional monitoring alongside the avatar-based monitoring, significantly less visual attention was paid to the avatar side of the screen. This may indicate an interaction effect. Perhaps the avatar drew the participant's attention to a vital sign outside the norm that they checked on the conventional screen. Because the qualitative visualization provided by the avatar is intuitive and quickly understood, participants paid relatively little visual attention to the avatar. To verify the qualitative input on the conventional monitor screen seems to have taken more time. In addition, we identified visual attention in conjunction with task performance as a valuable surrogate for situational awareness as it covers all three levels of situational awareness.