This paper is in the following e-collection/theme issue:

Serious Games for Health and Medicine (683) Graduate and Postgraduate Education for Health Professionals (97) Simulation (102)

Published on in Vol 6, No 2 (2018): April-June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9611, first published .
Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study

Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study

Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study

Authors of this article:

Patrick Motz1 Author Orcid Image ;   Megan Gray1 Author Orcid Image ;   Taylor Sawyer1 Author Orcid Image ;   Jennifer Kett2 Author Orcid Image ;   Douglas Danforth3 Author Orcid Image ;   Kellen Maicher3 Author Orcid Image ;   Rachel Umoren1 Author Orcid Image
Article Authors Cited by (10) Tweetations (1) Metrics

    Original Paper

    1Division of Neonatology, Department of Pediatrics, University of Washington, Seattle, WA, United States

    2Division of Neonatology, Department of Pediatrics, Mary Bridge Children's Hospital, Tacoma, WA, United States

    3Medical Simulation, Obstetrics and Gynocology, Ohio State University, Columbus, OH, United States

    Corresponding Author:

    Rachel Umoren, MD

    Division of Neonatology

    Department of Pediatrics

    University of Washington

    1959 NE Pacific St

    Seattle, WA,

    United States

    Phone: 1 206 543 3675

    Fax:1 206 543 8926

    Email: rumoren@uw.edu


    Background: Prenatal counseling at the limits of newborn viability involves sensitive interactions between neonatal providers and families. Empathetic discussions are currently learned through practice in times of high stress. Decision aids may help improve provider communication but have not been universally adopted. Virtual standardized patients are increasingly recognized as a modality for education, but prenatal counseling simulations have not been described. To be valuable as a tool, a virtual patient would need to accurately portray emotions and elicit a realistic response from the provider.

    Objective: To determine if neonatal providers can accurately identify a standardized virtual prenatal patient’s emotional states and examine the frequency of empathic responses to statements made by the patient.

    Methods: A panel of Neonatologists, Simulation Specialists, and Ethicists developed a dialogue and identified empathic responses. Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA), a screen-based simulation of a woman at 23 weeks gestation, was capable of displaying anger, fear, sadness, and happiness through animations. Twenty-four neonatal providers, including a subgroup with an ethics interest, were asked to identify VANESSA’s emotions 28 times, respond to statements, and answer open-ended questions. The emotions were displayed in different formats: without dialogue, with text dialogue, and with audio dialogue. Participants completed a post-encounter survey describing demographics and experience. Data were reported using descriptive statistics. Qualitative data from open ended questions (eg, “What would you do?”) were examined using thematic analysis.

    Results: Half of our participants had over 10 years of clinical experience. Most participants reported using medical research (18/23, 78%) and mortality calculators (17/23, 74%). Only the ethics-interested subgroup (10/23, 43%) listed counseling literature (7/10, 70%). Of 672 attempts, participants accurately identified VANESSA’s emotions 77.8% (523/672) of the time, and most (14/23, 61%) reported that they were confident in identifying these emotions. The ethics interest group was more likely to choose empathic responses (P=.002). Participants rated VANESSA as easy to use (22/23, 96%) and reported that she had realistic dialogue (15/23, 65%).

    Conclusions: This pilot study shows that a prenatal counseling simulation is feasible and can yield useful data on prenatal counseling communication. Our participants showed a high rate of emotion recognition and empathy in their responses.

    JMIR Serious Games 2018;6(2):e8

    doi:10.2196/games.9611

    Keywords

    prenatal counseling; simulation; ethics 


    One out of every ten babies is born prematurely [1]. It has become standard practice for health care providers to offer expectant mothers with premature labor a prenatal consultation. This prenatal consultation addresses the complications of premature birth and gives parents a chance to engage in a dialogue about what to expect for their baby. The prenatal consultation becomes even more critical when babies are very premature and may be born at the limits of medical capacity to successfully provide life-sustaining care, otherwise known as the limits of viability [2]. Families may make life or death decisions based on the information given to them by their health care provider. Prior studies show that most parents wish to participate in decision-making in these kinds of situations [3]. However, parents often cannot recall the therapeutic options that are discussed or find that their decision-making is not impacted by physicians’ predictions of survival and outcomes [3]. Rather, psychosocial influences such as religion, spirituality, and hopefulness more readily guided their decisions [2]. In some cases, up to a quarter of parents prefer to relinquish decision making autonomy, either to physicians or by leaving the situation in “God's hands” [2,4].

    Despite the gravity of these conversations, there is evidence that communication during prenatal consultations could be significantly improved, and there have also been calls for a more standardized approach to perinatal counseling [4]. For example, some researchers have proposed a framework with visual aids to help parents better understand the outcomes of babies born at the limit of viability [5]. However, this approach of providing more standardized information does not always meet the needs of parents. Parents need to feel understood and supported as they advocate for their baby in a collaborative and compassionate environment [3].

    Over the last two decades, the use of standardized patients (SPs) for health provider communication training has increased [4]. However, their use in prenatal counseling is limited and there is evidence that even with training, SP encounters are prone to recall bias that may lead to inconsistent feedback [6]. Virtual SPs may be a more accessible and cost-effective approach. However, in these emotionally charged conversations, the ability of the virtual patient to project a recognizable emotion is a key element to creating a valid user experience [7]. We hypothesize that a standardized virtual patient simulator called Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA), with the capacity to express emotions, will be a feasible approach to developing health care provider communication skills in prenatal counseling.

    We set out to achieve the following objectives:

    1. Enhance a virtual SP simulator with animations reflecting four primary emotional states.
    2. Evaluate the degree to which practicing Neonatologists and neonatal nurse practitioners (NNPs) can correctly identify the virtual SP’s emotional states and the frequency of empathic responses to statements made by the patient.
    3. Examine differences in participants’ responses to questions posed by the virtual patient.

    This observational study was approved by the Seattle Children’s Hospital Institutional Review Board. All attending Neonatologists, Neonatal-Perinatal medicine fellows, and NNPs from the University of Washington and Seattle Children's Hospital were eligible to participate, and all Neonatologists who attended a biweekly neonatal ethics interest group participated. Providers who did not routinely provide perinatal counseling were excluded.

    VANESSA, a prototype virtual standardized perinatal patient of a woman pregnant at 23 weeks gestation, was adapted from a medical history-taking virtual patient simulator [8] with animated emotional responses on the Unity 3D platform [9]. The VANESSA simulator was programmed to display the emotions of anger, fear, happiness, and sadness through animations of facial expressions and body language. An SP case and potential responses were developed with input from attending Neonatologists who provided extensive perinatal counseling services (Table 1). Using the VANESSA interface with the potential patient responses programmed, the scenario was deployed to participants [10]. A structured dialogue of the scenario was programmed to include the full potential range of emotions. Potential responses to VANESSA’s dialogue were developed with each set containing both empathic and nonempathic options. Demographic data and feedback on the product were collected using an online survey [11].

    Participants were emailed a link to the online module and associated survey. First, participants were shown video clips of VANESSA displaying emotions with no dialogue (out of context) and asked to identify the emotion she was expressing. Participants were then taken through the prenatal counseling scenario with text displays of both patient and provider dialogue (in context with text) and asked to identify the emotions (Figure 1).

    The online module ended with participants participating in the counseling with opportunities to choose responses to the patient’s statements. The interactive counseling section provided audio from the SP and text-based responses to move the scenario forward. Participants were again asked to identify VANESSA’s emotions during the case (in context with audio). The concordance between the displayed emotion and participant responses was determined for each context. At the end of the encounter, participants were asked two open-ended questions by the simulator: (1) “What would you do?” and (2) “What are my options?” Participants responded by typing into a text box. There was no limit to the length of the participant responses.

    Once the scenario was finished, the participants filled out a post-encounter survey that included demographic information, years of experience, and formal training in counseling or perinatal counseling. The survey elicited participants’ impressions of the usability of the simulator using a 5-point Likert scale (from 1=strongly agree to 5=strongly disagree).

    Table 1. Excerpt of VANESSA dialogue.
    TopicSample DialogueAnimation
    IntroductionDr. X: Hi, I’m the neonatal provider on-call. Your obstetrician asked me to meet with you to discuss your baby with you.
    VANESSA: Hi, thank you for coming.    		Happy
    Assessing patient’s comfort/ interest in talkingDr. X: Is this an okay time to talk?
    VANESSA: My husband is gone for the day, but I can talk now. (Patient animation—Sad)    		Sad
    Assess current understandingDr. X: What is your understanding of what might happen in the next few days?
    VANESSA: It sounds like the baby is coming. I’m only six months along. (Patient animation—Afraid)    		Afraid
    Relationship buildingDr. X: Have you picked a name for the baby?
    VANESSA: I’m going to call him Robert, after my dad.
    Dr. X: That’s a great name. (Patient animation—Happy)    		Happy
    ‎
    Figure 1. VANESSA prototype interface.
    View this figure

    Statistical Analyses

    Demographic information was analyzed with summary statistics. Multiple choice responses to statements made by VANESSA were collected and analyzed by subgroups including gender, clinical experience, job title, counseling resources used, previous perinatal counseling training, and ethics research focus. Statistical analyses included Kruskal-Wallis and Mann-Whitney tests to evaluate various study subgroups, as the data did not follow a normal distribution. A P value <.05 was considered significant. Responses to the open-ended questions were analyzed using a grounded theory analysis approach. Two study team members (PM and RU) evaluated the open-ended responses. An initial list of codes was identified by analyzing the data. Individual codes were discussed further and collapsed into major themes. The final themes were reached after thorough discussion from the two readers. A third study team member (MG) was consulted for discrepancies.


    Participant Demographics

    A total of 24 neonatal providers participated in the pilot study (Table 2). The group was evenly divided between those with less than 10 years of neonatal clinical experience and those with more than 10 years of experience. Most participants reported using medical research and mortality calculators as resources for their perinatal counseling (Table 2). Only the group of Neonatologists who attended a biweekly neonatal ethics interest group listed counseling literature as a resource. Didactic lectures on perinatal counseling and perinatal counseling simulation use were infrequently utilized as resources by participants in this study. Most participants had been previously trained via clinical observation. No participants felt that VANESSA was unnecessarily complex and 96% (22/23) felt they could use VANESSA without the support of a technical person. Few respondents (3/23, 13%) disagreed with the statement that VANESSA was realistic and only one participant felt that she did not respond as other patients would.

    Emotional Identification

    Of the 672 emotions presented, participants accurately identified VANESSA’s emotion 78.9% (530/672) of the time. As expected, giving participants context through text and audio dialogue did improve their accuracy of emotional identification (Figure 2). When given no contextual dialogue participants were fairly accurate at 74.4% (192/258). By adding text dialogue, respondents improved to 81.7% (291/356) when the context was given. Participants’ confidence in how accurate they were at identifying emotions lagged slightly behind their actual accuracy (Figure 3). When analyzed by each emotion, we found that participants were easily able to identify happy (89.8%, 219/244), afraid (78.0%, 192/246), and angry (80.5%, 161/200) emotions but were less accurate at identifying the sad (63.8%, 134/210) emotion.

    Empathic Response to VANESSA

    Participants chose empathic responses to VANESSA 75.0% (81/108) of the time. The response chosen most often was, “I can see this is upsetting.” The nonempathic response most often chosen was, “I have more information to share with you, may I go on?” This response accounted for 81% (22/27) of all the nonempathic choices.

    The group of Neonatologists who attended a biweekly neonatal ethics interest group were more likely to choose empathic responses (P=.01) but were not more likely than the other groups to correctly recognize VANESSA’s emotions. We also assessed differences based on gender, clinical experience, job title, counseling resources used, and counseling training. There were no statistically significant differences between these groups.

    Qualitative Analysis of Participant Responses

    The qualitative analysis of the two open-ended questions of, “What would you do?” and, “What are my options?” posed by VANESSA yielded over 50 codes. The codes were distilled into four themes: eliciting the mother’s values, sharing the counselor’s values, this is a difficult choice, and the desire to give more information to aid the decision.

    Table 2. Demographics of study participants.
    Demographicsn (%)
    Gender
    Male8 (35)
    Female16 (65)
    Profession
    Neonatal nurse practitioner7 (29)
    Physician17 (71)
    Clinical experience
    >10 years12 (50)
    <10 years12 (50)
    Previous counseling training
    Clinical observation23 (96)
    Workshop4 (17)
    Simulation9 (38)
    Communication workshop6 (25)
    Resources used for perinatal counseling
    Medical research18 (78)
    Counseling literature7 (30)
    Lectures5 (22)
    Simulation3 (13)
    Mortality calculator17 (74)
    ‎
    Figure 2. Accuracy of emotion recognition versus participants’ impression of accurate emotional recognition.
    View this figure
    ‎
    Figure 3. Participants’ empathic versus nonempathic responses.
    View this figure

    Participants focused on eliciting the mother's values:

    Let's talk more about how you're feeling so I can help you to [the] best answer for your family.
    There is no right or “expert” answer, but I am here to help you consider what is best for your baby given your values and unique situation.

    Sharing the counselor's values was demonstrated through statements such as:

    Like you, I would want the best for my baby; whatever that might be.
    I would want to make a decision together with my partner. Is there any way for your husband to come be with you?

    Participants also acknowledged the difficult nature of the decision and recognized that there were different approaches:

    This decision is difficult and overwhelming. Everyone approaches these life challenges differently with different priorities.

    Few participants gave the definite answer not to resuscitate the soon-to-be-born baby when asked, “What would you do?” Some participants expressed that they didn’t know what they would do if they were in that situation:

    I don't know. I've seen very loving families do different things.

    Many participants offered more information:

    I want to give you as much information as I can, so you and your family can make the decision that is the best for you.

    Others reflected the question back to VANESSA in an attempt to elicit her goals and values:

    Tell me what is important for your baby and your family.

    Principal Findings

    Our study findings demonstrated the feasibility and potential utility of an emotionally expressive virtual perinatal counseling simulator. The “happy,” “afraid,” and “angry” emotions were identified with an accuracy of 80% to 90% (192/246 to 219/244). A high level of emotion recognition by participants interacting with a female virtual SP in a simulated prenatal counseling session is encouraging and is consistent with previous studies that show that recognition of emotion is most accurate on female faces [7,12]. The “sad” emotion was identified accurately only 63.8% (134/210) of the time. This discrepancy persisted despite providing additional text or audio context through scripted conversations. It is possible that the intensity of the “sad” emotion animation was not adequate or that it may have been perceived by participants as a blend of emotions when only one forced choice response was available [13]. Previous studies noted similar accuracy levels for happiness, afraid, sadness, and angry using photographic images of facial expressions of emotion [14].

    It is important for health care providers to accurately perceive emotions to provide the appropriate support and empathy for patients who are struggling with a diagnosis or those who are coping with a loss. The responses of a subgroup of participants who attended a biweekly neonatal ethics interest group were significantly more empathic toward VANESSA. This finding is consistent with counseling literature that shows that health care providers with interest in ethics had more empathy toward their patients and demonstrates that counseling approaches employed with virtual SPs may parallel those of actual encounters [4].

    Virtual prenatal counseling training may be valuable to medical and advanced practice provider training programs. A survey of neonatology program directors revealed an interest in standardizing prenatal counseling training [6]. A prenatal counseling simulator could be used for just-in-time training for residents and fellows and could serve as a way for experienced health care providers to get feedback on their prenatal counseling.

    Our qualitative analysis of participant responses to VANESSA's open-ended questions yielded several themes related to the health care providers’ approaches when faced with a difficult question. Most of our participants did attempt to elicit the mother's values rather than presenting their own, but fewer acknowledged that this was a difficult decision with uncertainty in the outcome. Review of the literature notes that families find these two themes to be very important in their counseling [15]. We think this issue underlines the need for further improvement in how we communicate with our patients and underscores the value of virtual SP simulators in research on prenatal counseling, which will be a focus in the next phase of VANESSA's development.

    Limitations

    The limitations of our study were that it was conducted at a single academic center and it had a small sample size. Some strengths of our study are that our participants were representative of a large academic neonatology practice, our participants were evenly split between highly experienced providers and moderately experienced providers, and we had participation from both NNPs and physicians.

    Conclusions

    In conclusion, this pilot study shows that a perinatal counseling simulation is feasible and can yield useful data on perinatal counseling communication. Our participants showed a high rate of emotion recognition and empathy in their responses. Further work needs to be done to develop our prototype further but demonstrating the recognition of VANESSA’s emotions has laid a solid foundation for additional research to validate this approach.

    Conflicts of Interest

    None declared.

    1. Frey H, Klebanoff N. The epidemiology, etiology, and costs of preterm birth. Semin Fetal Neonatal Med 2016 Apr;21(2):68-73. [CrossRef] [Medline]
    2. Payot A, Gendron S, Lefebvre F, Doucet H. Deciding to resuscitate extremely premature babies: how do parents and neonatologists engage in the decision? Soc Sci Med 2007 Apr;64(7):1487-1500. [CrossRef] [Medline]
    3. Boss RD, Hutton N, Sulpar LJ, West AM, Donohue PK. Values parents apply to decision-making regarding delivery room resuscitation for high-risk newborns. Pediatrics 2008 Sep;122(3):583-589. [CrossRef] [Medline]
    4. Stokes TA, Watson KL, Boss RD. Teaching antenatal counseling skills to neonatal providers. Semin Perinatol 2014 Feb;38(1):47-51. [CrossRef] [Medline]
    5. Sobczak-Hoeft SM, Finer N. Standardizing perinatal consults to improve trainee physician competency and decrease maternal anxiety. J Neonatal Perinatal Med 2008 Jul 23;1(3):181-187.
    6. Turner JL, Dankoski ME. Objective structured clinical exams: a critical review. Fam Med 2008 Sep;40(8):574-578. [Medline]
    7. de Borst AW, de Gelder B. Is it the real deal? Perception of virtual characters versus humans: an affective cognitive neuroscience perspective. Front Psychol 2015;6:576 [FREE Full text] [CrossRef] [Medline]
    8. Maicher K, Danforth D, Price A, Zimmerman L, Wilcox B, Liston B, et al. Developing a conversational virtual standardized patient to enable students to practice history-taking skills. Simul Healthc 2017 Apr;12(2):124-131. [CrossRef] [Medline]
    9. Canonical Ltd. London, UK; 2015 Jan 01. Unity   URL: https://unity3d.com/ [accessed 2018-04-25] [WebCite Cache]
    10. Articulate. New York; 2016 Jan 01. Storyline Version 3   URL: https://articulate.com/p/storyline-3 [accessed 2018-04-25] [WebCite Cache]
    11. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
    12. Wells LJ, Gillespie SM, Rotshtein P. Identification of Emotional Facial Expressions: Effects of Expression, Intensity, and Sex on Eye Gaze. PLoS One 2016;11(12):e0168307 [FREE Full text] [CrossRef] [Medline]
    13. Wu Y, Babu SV, Armstrong R, Bertrand JW, Luo J, Roy T, et al. Effects of virtual human animation on emotion contagion in simulated inter-personal experiences. IEEE Trans Vis Comput Graph 2014 Apr;20(4):626-635. [CrossRef] [Medline]
    14. Ekman P, Friesen W, Ellsworth P. Emotion in the human face: guidelines for research and an integration of findings. Oxford: Pergamon Press; Oct 2013:22.
    15. Janvier A, Lorenz J, Lantos J. Antenatal counselling for parents facing an extremely preterm birth: limitations of the medical evidence. Acta Paediatr 2012 Aug;101(8):800-804. [CrossRef] [Medline]
    16. Tucker EB, McKenzie F, Fadel WF, Matthias MS, Salyers MP, Barnato AE, et al. Using simulation to assess the influence of race and insurer on shared decision making in periviable counseling. Simul Healthc 2014 Dec;9(6):353-359 [FREE Full text] [CrossRef] [Medline]
    17. Kett JC. Prenatal Consultation for Extremely Preterm Neonates: Ethical Pitfalls and Proposed Solutions. J Clin Ethics 2015;26(3):241-249. [Medline]
    18. Bastek TK, Richardson DK, Zupancic JAF, Burns JP. Prenatal consultation practices at the border of viability: a regional survey. Pediatrics 2005 Aug;116(2):407-413. [CrossRef] [Medline]
    19. Griswold KJ, Fanaroff JM. An evidence-based overview of prenatal consultation with a focus on infants born at the limits of viability. Pediatrics 2010 Apr;125(4):e931-e937. [CrossRef] [Medline]
    20. Paul DA, Epps S, Leef KH, Stefano JL. Prenatal consultation with a neonatologist prior to preterm delivery. J Perinatol 2001;21(7):431-437. [Medline]
    21. Lyles AA, Amresh A, Huberty J, Todd M, Lee RE. A Mobile, Avatar-Based App for Improving Body Perceptions Among Adolescents: A Pilot Test. JMIR Serious Games 2017 Mar 02;5(1):e4 [FREE Full text] [CrossRef] [Medline]
    22. Yee W, Sauve R. What information do parents want from the antenatal consultation? Paediatr Child Health 2007 Mar;12(3):191-196 [FREE Full text] [Medline]
    23. Zupancic J, Kirpalani H, Barrett J, Stewart S, Gafni A, Streiner D, et al. Characterising doctor-parent communication in counselling for impending preterm delivery. Arch Dis Child Fetal Neonatal Ed 2002 Sep;87(2):F113-F117 [FREE Full text] [Medline]
    24. Grobman WA, Kavanaugh K, Moro T, DeRegnier R, Savage T. Providing advice to parents for women at acutely high risk of periviable delivery. Obstet Gynecol 2010 May;115(5):904-909 [FREE Full text] [CrossRef] [Medline]
    25. Keenan HT, Doron MW, Seyda BA. Comparison of mothers' and counselors' perceptions of predelivery counseling for extremely premature infants. Pediatrics 2005 Jul;116(1):104-111. [CrossRef] [Medline]
    26. Partridge JC, Martinez AM, Nishida H, Boo N, Tan KW, Yeung C, et al. International comparison of care for very low birth weight infants: parents' perceptions of counseling and decision-making. Pediatrics 2005 Aug;116(2):e263-e271. [CrossRef] [Medline]
    27. Boss RD, Donohue PK, Roter DL, Larson SM, Arnold RM. “This is a decision you have to make”: using simulation to study prenatal counseling. Simul Healthc 2012 Aug;7(4):207-212. [CrossRef] [Medline]
    28. Kaempf JW, Tomlinson M, Arduza C, Anderson S, Campbell B, Ferguson LA, et al. Medical staff guidelines for periviability pregnancy counseling and medical treatment of extremely premature infants. Pediatrics 2006 Jan;117(1):22-29. [CrossRef] [Medline]
    29. Swinton C, Lantos J. Current empirical research in neonatal bioethics. Acta Paediatr 2010 Dec;99(12):1773-1781. [CrossRef] [Medline]
    30. Vollmer DD, Fair K, Hong YA, Beaudoin CE, Pulczinski J, Ory MG. Apps seeking theories: results of a study on the use of health behavior change theories in cancer survivorship mobile apps. JMIR Mhealth Uhealth 2015 Mar 27;3(1):e31 [FREE Full text] [CrossRef] [Medline]
    31. Bekele E, Bian D, Peterman J, Park S, Sarkar N. Design of a Virtual Reality System for Affect Analysis in Facial Expressions (VR-SAAFE); Application to Schizophrenia. IEEE Trans Neural Syst Rehabil Eng 2017 Dec;25(6):739-749. [CrossRef] [Medline]
    32. Rizzo AA, Neumann U, Enciso R, Fidaleo D, Noh JY. Performance-driven facial animation: basic research on human judgments of emotional state in facial avatars. Cyberpsychol Behav 2001 Aug;4(4):471-487. [CrossRef] [Medline]
    33. American College of ObstetriciansGynecologists, Society for Maternal-Fetal Medicine. ACOG Obstetric Care Consensus No. 3: Periviable Birth. Obstet Gynecol 2015 Nov;126(5):e82-e94. [CrossRef] [Medline]
    34. Halpern J. Empathy and patient-physician conflicts. J Gen Intern Med 2007 May;22(5):696-700 [FREE Full text] [CrossRef] [Medline]
    35. Price D, International Guidelines 2000 Conference on Cardiopulmonary ResuscitationEmergency Cardiac Care. New neonatal resuscitation program guidelines. Can Fam Physician 2001 Jun;47:1263-1264 [FREE Full text] [Medline]
    36. Graafland M, Dankbaar M, Mert A, Lagro J, De WL, Schuit S, et al. How to systematically assess serious games applied to health care. JMIR Serious Games 2014 Nov 11;2(2):e11 [FREE Full text] [CrossRef] [Medline]

    ‎
    NNP: neonatal nurse practitioner
    SP: standardized patient
    VANESSA: Virtual Antenatal Encounter and Standardized Simulation Assessment

    Edited by G Eysenbach; submitted 08.12.17; peer-reviewed by S Perry, T Antunes; comments to author 20.01.18; revised version received 05.03.18; accepted 07.03.18; published 11.05.18

    Copyright

    ©Patrick Motz, Megan Gray, Taylor Sawyer, Jennifer Kett, Douglas Danforth, Kellen Maicher, Rachel Umoren. Originally published in JMIR Serious Games (http://games.jmir.org), 11.05.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.