Demographic information was analyzed with summary statistics. Multiple choice responses to statements made by VANESSA were collected and analyzed by subgroups including gender, clinical experience, job title, counseling resources used, previous perinatal counseling training, and ethics research focus. Statistical analyses included Kruskal-Wallis and Mann-Whitney tests to evaluate various study subgroups, as the data did not follow a normal distribution. A P value <.05 was considered significant. Responses to the open-ended questions were analyzed using a grounded theory analysis approach. Two study team members (PM and RU) evaluated the open-ended responses. An initial list of codes was identified by analyzing the data. Individual codes were discussed further and collapsed into major themes. The final themes were reached after thorough discussion from the two readers. A third study team member (MG) was consulted for discrepancies.

A total of 24 neonatal providers participated in the pilot study (Table 2). The group was evenly divided between those with less than 10 years of neonatal clinical experience and those with more than 10 years of experience. Most participants reported using medical research and mortality calculators as resources for their perinatal counseling (Table 2). Only the group of Neonatologists who attended a biweekly neonatal ethics interest group listed counseling literature as a resource. Didactic lectures on perinatal counseling and perinatal counseling simulation use were infrequently utilized as resources by participants in this study. Most participants had been previously trained via clinical observation. No participants felt that VANESSA was unnecessarily complex and 96% (22/23) felt they could use VANESSA without the support of a technical person. Few respondents (3/23, 13%) disagreed with the statement that VANESSA was realistic and only one participant felt that she did not respond as other patients would.

Of the 672 emotions presented, participants accurately identified VANESSA’s emotion 78.9% (530/672) of the time. As expected, giving participants context through text and audio dialogue did improve their accuracy of emotional identification (Figure 2). When given no contextual dialogue participants were fairly accurate at 74.4% (192/258). By adding text dialogue, respondents improved to 81.7% (291/356) when the context was given. Participants’ confidence in how accurate they were at identifying emotions lagged slightly behind their actual accuracy (Figure 3). When analyzed by each emotion, we found that participants were easily able to identify happy (89.8%, 219/244), afraid (78.0%, 192/246), and angry (80.5%, 161/200) emotions but were less accurate at identifying the sad (63.8%, 134/210) emotion.

Participants chose empathic responses to VANESSA 75.0% (81/108) of the time. The response chosen most often was, “I can see this is upsetting.” The nonempathic response most often chosen was, “I have more information to share with you, may I go on?” This response accounted for 81% (22/27) of all the nonempathic choices.

The group of Neonatologists who attended a biweekly neonatal ethics interest group were more likely to choose empathic responses (P=.01) but were not more likely than the other groups to correctly recognize VANESSA’s emotions. We also assessed differences based on gender, clinical experience, job title, counseling resources used, and counseling training. There were no statistically significant differences between these groups.

The qualitative analysis of the two open-ended questions of, “What would you do?” and, “What are my options?” posed by VANESSA yielded over 50 codes. The codes were distilled into four themes: eliciting the mother’s values, sharing the counselor’s values, this is a difficult choice, and the desire to give more information to aid the decision.

Participants focused on eliciting the mother's values:

Sharing the counselor's values was demonstrated through statements such as:

Participants also acknowledged the difficult nature of the decision and recognized that there were different approaches:

Few participants gave the definite answer not to resuscitate the soon-to-be-born baby when asked, “What would you do?” Some participants expressed that they didn’t know what they would do if they were in that situation:

Many participants offered more information:

Others reflected the question back to VANESSA in an attempt to elicit her goals and values:

Our study findings demonstrated the feasibility and potential utility of an emotionally expressive virtual perinatal counseling simulator. The “happy,” “afraid,” and “angry” emotions were identified with an accuracy of 80% to 90% (192/246 to 219/244). A high level of emotion recognition by participants interacting with a female virtual SP in a simulated prenatal counseling session is encouraging and is consistent with previous studies that show that recognition of emotion is most accurate on female faces [7,12]. The “sad” emotion was identified accurately only 63.8% (134/210) of the time. This discrepancy persisted despite providing additional text or audio context through scripted conversations. It is possible that the intensity of the “sad” emotion animation was not adequate or that it may have been perceived by participants as a blend of emotions when only one forced choice response was available [13]. Previous studies noted similar accuracy levels for happiness, afraid, sadness, and angry using photographic images of facial expressions of emotion [14].

It is important for health care providers to accurately perceive emotions to provide the appropriate support and empathy for patients who are struggling with a diagnosis or those who are coping with a loss. The responses of a subgroup of participants who attended a biweekly neonatal ethics interest group were significantly more empathic toward VANESSA. This finding is consistent with counseling literature that shows that health care providers with interest in ethics had more empathy toward their patients and demonstrates that counseling approaches employed with virtual SPs may parallel those of actual encounters [4].

Virtual prenatal counseling training may be valuable to medical and advanced practice provider training programs. A survey of neonatology program directors revealed an interest in standardizing prenatal counseling training [6]. A prenatal counseling simulator could be used for just-in-time training for residents and fellows and could serve as a way for experienced health care providers to get feedback on their prenatal counseling.

Our qualitative analysis of participant responses to VANESSA's open-ended questions yielded several themes related to the health care providers’ approaches when faced with a difficult question. Most of our participants did attempt to elicit the mother's values rather than presenting their own, but fewer acknowledged that this was a difficult decision with uncertainty in the outcome. Review of the literature notes that families find these two themes to be very important in their counseling [15]. We think this issue underlines the need for further improvement in how we communicate with our patients and underscores the value of virtual SP simulators in research on prenatal counseling, which will be a focus in the next phase of VANESSA's development.

The limitations of our study were that it was conducted at a single academic center and it had a small sample size. Some strengths of our study are that our participants were representative of a large academic neonatology practice, our participants were evenly split between highly experienced providers and moderately experienced providers, and we had participation from both NNPs and physicians.

In conclusion, this pilot study shows that a perinatal counseling simulation is feasible and can yield useful data on perinatal counseling communication. Our participants showed a high rate of emotion recognition and empathy in their responses. Further work needs to be done to develop our prototype further but demonstrating the recognition of VANESSA’s emotions has laid a solid foundation for additional research to validate this approach.