This paper is in the following e-collection/theme issue:

Serious Games for Health and Medicine (641) Extended Reality, Virtual Reality and Virtual Worlds (509)

Published on in Vol 6, No 4 (2018): Oct-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10839, first published .
Virtual Reality Clinical Research: Promises and Challenges

Virtual Reality Clinical Research: Promises and Challenges

Virtual Reality Clinical Research: Promises and Challenges

Authors of this article:

Bernie Garrett1 Author Orcid Image ;   Tarnia Taverner1 Author Orcid Image ;   Diane Gromala2 Author Orcid Image ;   Gordon Tao3 Author Orcid Image ;   Elliott Cordingley4 Author Orcid Image ;   Crystal Sun1 Author Orcid Image
Article Authors Cited by (118) Tweetations (24) Metrics

    Viewpoint

    1School of Nursing, University of British Columbia, Vancouver, BC, Canada

    2School of Interactive Arts and Technology, Simon Fraser University, Surrey, BC, Canada

    3Rehabilitation Sciences, University of British Columbia, Vancouver, BC, Canada

    4Faculty of Science, University of British Columbia, Vancouver, BC, Canada

    *all authors contributed equally

    Corresponding Author:

    Bernie Garrett, RN, PhD

    School of Nursing

    University of British Columbia

    T201-2211 Wesbrook Mall

    Vancouver, BC, V6T 2B5

    Canada

    Phone: 1 16047538942

    Email: bernie.garrett@ubc.ca


    Background: Virtual reality (VR) therapy has been explored as a novel therapeutic approach for numerous health applications, in which three-dimensional virtual environments can be explored in real time. Studies have found positive outcomes for patients using VR for clinical conditions such as anxiety disorders, addictions, phobias, posttraumatic stress disorder, eating disorders, stroke rehabilitation, and for pain management.

    Objective: This work aims to highlight key issues in the implementation of clinical research for VR technologies.

    Methods: A discussion paper was developed from a narrative review of recent clinical research in the field, and the researchers’ own experiences in conducting VR clinical research with chronic pain patients.

    Results: Some of the key issues in implementing clinical VR research include theoretical immaturity, a lack of technical standards, the problems of separating effects of media versus medium, practical in vivo issues, and costs.

    Conclusions: Over the last decade, some significant successes have been claimed for the use of VR. Nevertheless, the implementation of clinical VR research outside of the laboratory presents substantial clinical challenges. It is argued that careful attention to addressing these issues in research design and pilot studies are needed in order to make clinical VR research more rigorous and improve the clinical significance of findings.

    JMIR Serious Games 2018;6(4):e10839

    doi:10.2196/10839

    Keywords

    virtual reality; clinical research; VR standards; VR theory; VR immersion; VR presence 


    Contemporary research on computer-based virtual reality (VR) dates back to the early 1980s, although devices for presenting stereoscopic imagery (ie, using a slightly different image for each eye) such as the stereoscope started in the 1830s [1]. The exploration of VR use in clinical applications is accelerating rapidly with the advent of more powerful computer and graphics processors capable of rendering real-time three-dimensional (3D) imagery, and the availability of relatively low-cost VR headsets such as the Oculus Rift or HTC Vive (see Figure 1).

    As researchers with significant experience in researching VR for clinical applications, we have identified some major issues in the development of clinical VR research. Substantial challenges remain with theoretical ambiguity and immaturity, a lack of technical standards, problems of media versus medium, practical in vivo issues, and economic feasibility.

    Background

    There has been rapid growth in the reported use of VR in the treatment of a variety of clinical conditions, such as acute and chronic pain management [2-9], anxiety disorders [10-12], phobias [13-15], posttraumatic stress disorder (PTSD) [16-18], eating disorders [19], autism [20], and rehabilitation [21-26]. Additionally, its use in professional health care education has also been expanding rapidly [22,27-32].

    One early clinical application of VR was for the treatment of acrophobia [33]. Graduated exposure to virtual environments with foot bridges, balconies, and a glass elevator were used with a railing placed around the user in the real world for them to hold on to. The intervention was reported as effective. Over the last 20 years, VR clinical applications have expanded to address other phobias and anxiety disorders. The most common approaches in this field have been to model virtual environments after existing exposure therapies using graduated exposure to a VR version of the object or situation that causes distress and use of VR cognitive behavioral therapies [34-37]. For PTSD, virtual environments have been used to simulate complex traumatic scenarios under control to treat war survivors [18,38]. Similarly, VR has been used in the treatment of body image and eating disorders [39-41]. These approaches leverage education, visual feedback, and simulations of critical situations to improve body self-perception.

    These studies largely focus on health outcomes to determine the efficacy of VR treatments. While they reported positive clinical effects over a variety of VR experiences, they often pay limited attention to the nature of the hardware and software used. Furthermore, the VR therapies usually relied on custom virtual environments. However, the literature often lacks commentary on the design and development of them. Despite these limitations, VR-based treatments for treating fear-related and anxiety disorders appear to be the most established clinical applications of VR.

    Another key area has been in the use of VR for pain management. The mechanism of VR pain control is primarily thought to be distractive, although the precise mode of action remains uncertain [6,42,43]. For example, VR has been used to manage acute pain during in-hospital treatments for burn patients [5,44-47]. Using VR for needle-stick pain has also been researched [6,48]. In the field of chronic pain, VR has also been applied [8,49-51]. Several researchers have explored VR use to treat phantom limb pain [52-55]. VR allows clinicians to present patients with a virtual representation of their missing limb. Through perception and motor training, patients experienced relief from phantom limb pain by seeing their virtual limb move in accordance to their voluntary motor signals.

    ‎
    Figure 1. VR clinical application papers published by year (PubMed).
    View this figure

    In the rehabilitation field, there has also been great interest in pairing assistive technologies (eg, robotics, treadmills, wheelchair on rollers, wearable sensors) and VR. The primary goals have been the development of tools that support patient motivation to engage with rehabilitation and to leverage the logistical advantages of digital technology, namely performance monitoring, telehealth, and patient self-management. Additionally, translating physiotherapy exercises and activity training into the VR space allows for much greater control over and variety of scenarios.

    For example, robot-assisted upper limb therapy paired with VR visual feedback allows for graded exercises contextualized in a videogame environment [56-58]. In wheelchair simulators, VR enables users to practice wheelchair navigation skills in more dangerous situations such as traffic crossings and crowds without risk [59,60].

    Clinical VR research to date has generally been positive, but overall research in this field is in the early stages and faces technical and theoretical hurdles. Most studies have used non-standardized techniques and tools in small-scale pilot studies. Over the last 4 years, the authors have conducted several clinical VR research projects [6,8,49] and found a number of challenges in the field that may limit the validity and generalizability of the work.


    Theoretical Ambiguity and Immaturity

    As with the development of any new discipline, establishing a sound theoretical basis and standards is key to the growth of the field. However, there exists some theoretical ambiguity in the field due in part to its immaturity. Overall, VR may be considered as a growing field, defined by both its technology and its effects. The desired effect is to create an immersive experience, whereby the user is placed in a simulated environment that looks and feels as engaging as the real world. The person in this synthetic environment has a specific sense of self-location within it, can move to explore it, feels that the space surrounds them, and can interact with the objects within it. Overall, they feel a sense of presence in this environment, and their actions partially determine what happens within it [61,62].

    Technically, the sense of immersion in a VR environment is largely achieved through visual and auditory stimuli that simulate 3D visual and auditory cues available in the real world. Haptic feedback can also contribute to this immersion. Visually, this is delivered to the user via a head-mounted display, which presents the computer-generated imagery (CGI) of the VR scene from the perspective of each of the user’s eyes. The literature suggests that immersion is largely influenced by both visual and audio qualities, although a universally accepted definition is yet to emerge [63-67]. Immersion has been defined as the extent to which a user feels present in the CGI environment, rather than in their actual physical environment [68,69]. In computer science, immersion has more often been defined in terms of the technology and by the extent to which the computer is able to deliver an inclusive, extensive, surrounding, and vivid illusion of reality to the senses of the participant [67]. Therefore, immersion is often referenced by technical considerations, such as field of view and positioning of the virtual body in the CGI. The inclusion of stereoptic imagery is widely thought to be the dominant factor that enhances the immersive experience. Other technical factors, such as greater display resolution or increased field of view are also significant [52,70].

    Presence, on the other hand, refers to the sense of being within an environment that is generated through technological means [68,71]. It is viewed as the sense of actually being in a constructed world [68,69,71,72]. Two experiential and technology-dependent dimensions are considered to contribute to a sense of presence. The first dimension is vividness, or the production of a sensory-rich, mediated environment. The second is interactivity, defined as a user’s ability to engage with the environment and modify its form or alter events through interaction with it. An analogy would be that you can become immersed through the text in a book, but feel a sense of presence in the story only when you feel you are actually there experiencing the events.

    This differentiation of immersion from presence (which is seen as more of a subjective element) is fairly well established in computer science, but less so in clinical VR research, where the terms are often used interchangeably. For clinical use, a technical definition of immersion is limited, as it ignores the participant as a co-constructor of the experience. Therefore, concepts of presence and telepresence [68] are likely more useful to clinical applications. An immersive virtual environment can be considered to be a computer-generated environment that elicits the user’s sense of presence or “being there.” It can be seen as an environment that produces an esthetic perception connected to the ideal of total immersion in virtual space involving the willing suspension of disbelief [69,71,73]. In clinical contexts, this sense of presence is likely the key element of interest that differentiates the impact of VR from other distractive and cognitive approaches. Assessment tools that separate these aspects, such as the igroup Presence Questionnaire have been developed [74]. However, clinical VR literature rarely discusses these theoretical aspects nor provides robust theoretical explanations of how VR theory applies to the specific problem under investigation. As VR is essentially a technology-mediated phenomenon, this lack of theoretical distinction, between what actually constitutes a VR experience, at the least, makes meaningful comparisons between clinical studies complex.

    Adding more complexity is the issue that the actual nature of the effect of VR on the clinical problem of interest is also often unknown. For example, VR environments are hypothesized to reduce pain by mediating cognitive attentional and distractive mechanisms. The use of VR might act directly and indirectly on pain perception in different ways by altering neurological signaling pathways involving attention, emotion, concentration, memory, touch, and the auditory and visual senses. However, there are competing theoretical explanations of pain and the exact mechanisms of how VR may attenuate it remain unclear [48,75-82]. It has been theorized that VR analgesia stems from the neurobiological interactions of areas of the brain that produce analgesic effect by regulating visual, auditory, and touch sensory experiences [80]. Hoffman et al state that VR works predominantly via distraction. Pain requires attentiveness, and humans have been found to have limited controlled attentional resources [83]. The level and impact of the distraction can depend on the level of the immersion—the more immersive the VR, the more effective in reducing pain [84]. Furthermore, using functional magnetic resonance imaging (fMRI) brain scanning, correlation in pain-related brain activity and subjective pain report was reported, thus, demonstrating the impact of VR on pain-related brain activity in all five regions of the brain [83].

    VR has been shown to alter the sense of an individual’s presence to that of being in a virtual world, therefore changing features of the individuals experience associated with sensory affective and cognitive processes.

    The validity of clinical VR research also needs to be considered in the context of the theory development process. Overall, there are five major processes that occur in the development and establishment of a theory: (1) creating conceptual meaning, (2) structuring and generalizing the theory, (3) generating the theoretical relationships, (4) applying the theory, and then (5) theory validation by testing in different real-world applications [85]. At this stage of VR development for clinical use, the underpinning theory has yet to reach the higher levels of established validity.

    Standardized Implementation

    The type of VR technology implemented varies greatly between clinical studies. It is arguable that the current state of the art is very much technologically led rather than theoretically led, with each new iteration of clinical research using the latest VR applications and hardware with disparate approaches for a variety of clinical conditions. As the hardware and software continue to advance rapidly, studies even a year apart may be using completely different hardware or software and, in many cases, the technology is only vaguely defined [6,12,42].

    Three-Dimensional Versus Two-Dimensional

    Many clinical studies have used the term VR to describe significantly different multimedia technologies, including two-dimensional (2D) video screen presentations, 2D-rendered images presented on screens [86] and head-mounted displays [9,46,87], 360-degree 2D presentations on head-mounted displays [88], or computer-assisted virtual environment (CAVE) room-scale projection systems [89,90]. Others used 3D-rendered VR in motion-tracked stereoscopic head-mounted displays, with a wide field of view [8,9,91-93]. There are similar differences in audio use in these studies, with some using positional stereo sound (ie, location-specific sound that moves as the user moves their head) and others using non-spatial audio. Although health outcomes may be comparable, the nature and value of 3D versus 2D applications have not been widely explored in clinical applications.

    Study Design

    In addition to the theoretical issues, the nature of VR study design itself represents another significant hurdle. Systematic reviews/meta-analyses illustrate that many of these studies are statistically underpowered, although positive statistical results are frequently claimed [6,12,18,94-96]. To establish clinical efficacy of a therapy, large-scale quality randomized controlled trials are required. Comparative clinical studies also require a suitable control environment to contrast with the VR experience. Few studies make an adequate attempt to address this and frequently neglect to differentiate the effects of the media from the medium itself (both theoretically and in practice). For example, the medium of VR could be the use of VR technology and a head-mounted display to render a 360-degree stereoscopic and stereo audio environment with which a person can interact. The media may be a puzzle-solving interactive VR computer game, a VR rollercoaster ride, or a 3D-rendered high-definition video experience of a beach environment. Failing to explore if it is the VR experience itself or the medium used that is eliciting an effect is problematic. A good design will contrast a VR experience with a non-VR equivalent of the same experience, controlling for the effects of the medium compared to the media. These issues likely reflect some degree of confirmation bias among researchers, but this illustrates the need to implement larger-scale high-quality clinical VR studies.

    Usability and Technical Proficiency

    Another more practical challenge faced by clinical researchers is the usability of VR systems and the level of technical proficiency required to run them. Although current VR iterations are designed to be more user friendly, significant technical limitations remain. The use of head-mounted displays is problematic for some patients. They are cumbersome, particularly for patients with head or neck injuries, or for those who are particularly susceptible to eye strain. Additionally, VR applications are generally not usable by people with cognitive or significant visual deficiencies, as they are unable to access existing VR interfaces. Also, prolonged exposure to a screen a few centimeters from the eyes often leads to eye strain or headaches and represents an ongoing issue with VR systems [97,98]. Users with limited head or neck mobility often reported the systems were uncomfortable to use [8]. Furthermore, most advanced head-mounted displays have a cable tether that can be a distraction from the experience or a tripping hazard for older patients.

    Cybersickness, as a side effect of VR, is also well documented and limits use by many patients, particularly those taking medications that can cause nausea [8,99-103]. Newer systems that operate at room-scale (ie, where the user can walk around in a pre-determined area) have addressed this to some extent, but many patients also have limited mobility and must use the system in a seated position. This gives rise to another problem: most VR applications are currently designed to be used as either room-scale or seated, with few working well in both configurations. The issue derives from the fact that room-scale VR navigation affords the user much greater range of motion to physically approach virtual items, while the seated position requires a set visual height, longer reaching movements, and controller-based navigation of the environment. The environment design and implementation requirements generally do not transfer well from seated to room-scale and vice versa. Many VR systems have implemented teleportation navigation systems to support moving through larger distances to overcome this issue, but again those designed for room scale use do not adjust well to use from a seated position.

    The design and game paradigm of many VR experiences itself can also prove challenging for older patients. For users who have grown up with computer games, the nature of VR experiences is more readily understandable: traversing 3D-rendered worlds, using menus, navigating levels, storing and retrieving items, saving progress, solving game puzzles, and relating button-presses to abstract actions are all mechanics learned through experience. This alignment of VR with recreational gaming is exemplified by the marketing and delivery of HTC Vive and Oculus Rift VR applications through the Steam online gaming platform. Most clinical users are likely to be older adults, who have no such videogame literacy and often find learning these elements frustrating and distracting to their VR experience. Little work exists exploring the VR preferences of these users and the VR market is firmly dominated by the younger consumer.

    Lab Versus In Vivo Practical Issues

    Much of the existing VR research has taken place in lab or clinic settings. These environments can be optimized for VR systems. However, much remains to be known about the effects of regular and prolonged VR use for real-world and home applications, where they will be used for many chronic conditions. Certainly, there are common challenges for research requiring any kind of at-home technology implementation such as logistics, remote technical support, learning curve, and compliance. However, there are a few unique challenges to consider in the implementation of VR systems outside the lab.

    Current VR systems require dedicated space and are susceptible to interference. Room-scale systems require a 5 ft2 space, which may be intrusive to a patient’s living space. Cables may pose tripping hazards. Infrared sensors, such as those used by systems such as the HTC Vive, may be interfered with by devices such as TV remotes, resulting in display cutting out, choppy visuals, and loss of tracking, thus disrupting the user’s experience. Other environmental factors that disrupt infrared tracking, such as climate and reflection of light off windows or mirrors, can be easily mitigated in a lab setting but can be more difficult to cope with in a home. Furthermore, calibration for motion tracking of VR equipment is sensitive and thus movement of equipment must be minimized. Effective installation of VR equipment while still maintaining the usability of the home space is challenging and may be further complicated if there are pets or children in the home.

    For clinical research, where a study may take weeks or months, these technological burdens are important to negotiate with participants in advance. Despite these challenges, our experience has shown that research participants are often enthusiastic and willing to accommodate the various needs of the equipment and research study. However, these attitudes may not necessarily carry over to commercial or non-research contexts.

    Costs

    Finally, the cost of VR still presents a challenge to implementing large-scale trials [11]. Although costs of head-mounted displays are dropping, quality VR environments still require high-end computer systems with advanced graphics processing to run them. VR applications are also expensive to develop. The current cost of a full system to run a quality VR clinical experience is around US $2,500 per unit plus maintenance costs, making clinical research with multiple users costly. As with any information technology, attrition of value is also rapid; newer technologies rapidly make older systems obsolete. A practical assumption of minimal resale value of a VR system after 3 years is not unreasonable.


    Although clinical VR research looks promising, significant theoretical and practical challenges remain, such as theoretical ambiguity and immaturity, lack of technical standards, differentiating effects of media versus medium, value of 2D versus 3D applications, study design, usability, conducting in vivo research, and economic feasibility. Defining the impact of presence in clinical VR studies and differentiating the concept of presence from immersion (as they are often used synonymously) is a problem, and current research designs are often ill-equipped to differentiate the role of VR from confounding factors. More robust study designs contrasting VR experience with an equivalent non-VR control are required.

    Practical challenges also remain, as existing high-end VR systems remain cumbersome and require technical proficiency to use. VR systems are not always user-friendly for patients. Moreover, issues of eye and neck strain and cybersickness remain as practical barriers to wider use. For those undertaking clinical VR research, it is important to keep these issues in mind during efforts to improve the evidence base for these technologies as health interventions.

    Conflicts of Interest

    None declared.

    1. Rheingold H. Virtual reality. London: Simon & Schuster; 1991.
    2. Wright JL, Hoffman HG, Sweet RM. Virtual reality as an adjunctive pain control during transurethral microwave thermotherapy. Urology 2005 Dec;66(6):1320. [CrossRef] [Medline]
    3. Hoffman HG, Patterson DR, Seibel E, Soltani M, Jewett-Leahy L, Sharar SR. Virtual reality pain control during burn wound debridement in the hydrotank. Clin J Pain 2008 May;24(4):299-304. [CrossRef] [Medline]
    4. Thomas DA. Virtual reality research continues to progress at the National Institutes of Health. Cyberpsychol Behav Soc Netw 2014 Jun;17(6):334 [FREE Full text] [CrossRef] [Medline]
    5. Maani CV, Hoffman HG, Morrow M, Maiers A, Gaylord K, McGhee LL, et al. Virtual reality pain control during burn wound debridement of combat-related burn injuries using robot-like arm mounted VR goggles. J Trauma 2011 Jul;71(1 Suppl):S125-S130 [FREE Full text] [CrossRef] [Medline]
    6. Garrett B, Taverner T, Masinde W, Gromala D, Shaw C, Negraeff M. A rapid evidence assessment of immersive virtual reality as an adjunct therapy in acute pain management in clinical practice. Clin J Pain 2014 Dec;30(12):1089-1098. [CrossRef] [Medline]
    7. Wolitzky K, Fivush R, Zimand E, Hodges L, Rothbaum B. Psychol Health. 2005. Effectiveness of virtual reality distraction during a painful medical procedure in pediatric oncology patients   URL: https://www.tandfonline.com/doi/abs/10.1080/14768320500143339 [WebCite Cache]
    8. Garrett B, Taverner T, McDade P. Virtual Reality as an Adjunct Home Therapy in Chronic Pain Management: An Exploratory Study. JMIR Med Inform 2017 May 11;5(2):e11 [FREE Full text] [CrossRef] [Medline]
    9. Tashjian V, Mosadeghi S, Howard A, Lopez M, Dupuy T, Reid M, et al. Virtual Reality for Management of Pain in Hospitalized Patients: Results of a Controlled Trial. JMIR Ment Health 2017 Mar 29;4(1):e9 [FREE Full text] [CrossRef] [Medline]
    10. Diemer J, Mühlberger A, Pauli P, Zwanzger P. Virtual reality exposure in anxiety disorders: impact on psychophysiological reactivity. World J Biol Psychiatry 2014 Aug;15(6):427-442. [CrossRef] [Medline]
    11. Lindner P, Miloff A, Hamilton W, Reuterskiöld L, Andersson G, Powers MB, et al. Creating state of the art, next-generation Virtual Reality exposure therapies for anxiety disorders using consumer hardware platforms: design considerations and future directions. Cogn Behav Ther 2017 Sep;46(5):404-420. [CrossRef] [Medline]
    12. Parsons TD, Rizzo AA. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Ther Exp Psychiatry 2008 Sep;39(3):250-261. [CrossRef] [Medline]
    13. Hirsch JA. Virtual reality exposure therapy and hypnosis for flying phobia in a treatment-resistant patient: a case report. Am J Clin Hypn 2012 Oct;55(2):168-173. [CrossRef] [Medline]
    14. Garcia-Palacios A, Hoffman H, Carlin A, Furness TA, Botella C. Virtual reality in the treatment of spider phobia: a controlled study. Behav Res Ther 2002 Sep;40(9):983-993. [Medline]
    15. Herrero R, García-Palacios A, Castilla D, Molinari G, Botella C. Virtual reality for the induction of positive emotions in the treatment of fibromyalgia: a pilot study over acceptability, satisfaction, and the effect of virtual reality on mood. Cyberpsychol Behav Soc Netw 2014 Jun;17(6):379-384. [CrossRef] [Medline]
    16. Maples-Keller JL, Yasinski C, Manjin N, Rothbaum BO. Virtual Reality-Enhanced Extinction of Phobias and Post-Traumatic Stress. Neurotherapeutics 2017 Jul;14(3):554-563 [FREE Full text] [CrossRef] [Medline]
    17. Josman N, Reisberg A, Weiss PL, Garcia-Palacios A, Hoffman HG. BusWorld: an analog pilot test of a virtual environment designed to treat posttraumatic stress disorder originating from a terrorist suicide bomb attack. Cyberpsychol Behav 2008 Dec;11(6):775-777. [CrossRef] [Medline]
    18. Gonçalves R, Pedrozo AL, Coutinho ESF, Figueira I, Ventura P. Efficacy of virtual reality exposure therapy in the treatment of PTSD: a systematic review. PLoS One 2012;7(12):e48469 [FREE Full text] [CrossRef] [Medline]
    19. Ferrer-Garcia M, Gutiérrez-Maldonado J, Riva G. Virtual Reality Based Treatments in Eating Disorders and Obesity: A Review. J Contemp Psychother 2013 Jun 25;43(4):207-221. [CrossRef]
    20. Didehbani N, Allen T, Kandalaft M, Krawczyk D, Chapman S. Virtual Reality Social Cognition Training for children with high functioning autism. Computers in Human Behavior 2016 Sep;62:703-711. [CrossRef]
    21. Merians A, Jack D, Boian R, Tremaine M, Burdea GC, Adamovich SV, et al. Virtual reality-augmented rehabilitation for patients following stroke. Phys Ther 2002 Sep;82(9):898-915. [Medline]
    22. Chirico A, Lucidi F, De LM, Milanese C, Napoli A, Giordano A. Virtual Reality in Health System: Beyond Entertainment. A Mini-Review on the Efficacy of VR During Cancer Treatment. J Cell Physiol 2016 Feb;231(2):275-287. [CrossRef] [Medline]
    23. Broeren J. Stud Health Technol Inform.: Göteborg University. Sahlgrenska Academy; 2007. Virtual Rehabilitation-Implications for Persons with Stroke   URL: https://gupea.ub.gu.se/handle/2077/7329 [accessed 2018-09-20] [WebCite Cache]
    24. Glegg SMN, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors influencing therapists' adoption of virtual reality for brain injury rehabilitation. Cyberpsychol Behav Soc Netw 2013 May;16(5):385-401. [CrossRef] [Medline]
    25. Casale R. Virtual reality in the rehabilitation of phantom limb pain: What are we doing and how do we measure it in research and in daily practice? 2013 Presented at: 12th Congress of European Forum for Research in Rehabilitation; 2013; Montescano, Italy   URL: http://www.mfprm.net/files/the-12th-mfprm-congress-abstract-book.pdf
    26. Levin MF, Weiss PL, Keshner EA. Emergence of virtual reality as a tool for upper limb rehabilitation: incorporation of motor control and motor learning principles. Phys Ther 2015 Mar;95(3):415-425 [FREE Full text] [CrossRef] [Medline]
    27. Merians AS, Poizner H, Boian R, Burdea G, Adamovich S. Sensorimotor training in a virtual reality environment: does it improve functional recovery poststroke? Neurorehabil Neural Repair 2006 Jun;20(2):252-267. [CrossRef] [Medline]
    28. Mikropoulos TA, Natsis A. Educational virtual environments: A ten-year review of empirical research (1999–2009). Computers & Education 2011 Apr;56(3):769-780. [CrossRef]
    29. Cecil J, Gupta A, Pirela-Cruz M. An advanced simulator for orthopedic surgical training. Int J Comput Assist Radiol Surg 2018 Feb;13(2):305-319. [CrossRef] [Medline]
    30. Perry S, Bridges SM, Zhu F, Leung WK, Burrow MF, Poolton J, et al. Getting to the Root of Fine Motor Skill Performance in Dentistry: Brain Activity During Dental Tasks in a Virtual Reality Haptic Simulation. J Med Internet Res 2017 Dec 12;19(12):e371 [FREE Full text] [CrossRef] [Medline]
    31. Cheng Y, Chiang H, Ye J, Cheng L. Enhancing empathy instruction using a collaborative virtual learning environment for children with autistic spectrum conditions. Computers & Education 2010 Dec;55(4):1449-1458. [CrossRef]
    32. Li L, Yu F, Shi D, Shi J, Tian Z, Yang J, et al. Application of virtual reality technology in clinical medicine. Am J Transl Res 2017;9(9):3867-3880 [FREE Full text] [Medline]
    33. Rothbaum BO, Hodges LF, Kooper R, Opdyke D, Williford JS, North M. Effectiveness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia. Am J Psychiatry 1995 Apr;152(4):626-628. [CrossRef] [Medline]
    34. Bouchard S, Côté S, St-Jacques J, Robillard GE, Renaud P. Effectiveness of virtual reality exposure in the treatment of arachnophobia using 3D games. Technol Health Care 2006;14(1):19-27. [Medline]
    35. Igna R, Stefan S, Onac I, Onac I, Ungur R, Szentagotai TA. Mindfulness-based cognitive-behavior therapy (MCBT versus virtual reality (VR) enhanced CBT, versus treatment as usual for chronic back pain. A clinical trial. J Evidence-Based Psychother 2014;14(2):229-247.
    36. Wiederhold B, Jang D, Gevirtz R, Kim S, Kim I, Wiederhold M. The treatment of fear of flying: a controlled study of imaginal and virtual reality graded exposure therapy. IEEE Trans Inform Technol Biomed 2002 Sep;6(3):218-223. [CrossRef]
    37. Wiederhold B, Riva G. Annual Review of Cybertherapy and Telemedicine 2013. In: Positive Technology and Health Engagement for Healthy Living and Active Ageing. Amsterdam, Netherlands: IOS Press; 2013.
    38. Maples-Keller JL, Price M, Rauch S, Gerardi M, Rothbaum BO. Investigating Relationships Between PTSD Symptom Clusters Within Virtual Reality Exposure Therapy for OEF/OIF Veterans. Behav Ther 2017 Dec;48(2):147-155. [CrossRef] [Medline]
    39. Serino S, Pedroli E, Keizer A, Triberti S, Dakanalis A, Pallavicini F, et al. Virtual Reality Body Swapping: A Tool for Modifying the Allocentric Memory of the Body. Cyberpsychol Behav Soc Netw 2016 Feb;19(2):127-133. [CrossRef] [Medline]
    40. Gutiérrez-Maldonado J, Wiederhold BK, Riva G. Future Directions: How Virtual Reality Can Further Improve the Assessment and Treatment of Eating Disorders and Obesity. Cyberpsychol Behav Soc Netw 2016 Feb;19(2):148-153. [CrossRef] [Medline]
    41. Wiederhold BK, Riva G, Gutiérrez-Maldonado J. Virtual Reality in the Assessment and Treatment of Weight-Related Disorders. Cyberpsychol Behav Soc Netw 2016 Feb;19(2):67-73. [CrossRef] [Medline]
    42. Triberti S, Repetto C, Riva G. Psychological factors influencing the effectiveness of virtual reality-based analgesia: a systematic review. Cyberpsychol Behav Soc Netw 2014 Jun;17(6):335-345. [CrossRef] [Medline]
    43. Dascal J, Reid M, Ishak WW, Spiegel B, Recacho J, Rosen B, et al. Virtual Reality and Medical Inpatients: A Systematic Review of Randomized, Controlled Trials. Innov Clin Neurosci 2017;14(1-2):14-21 [FREE Full text] [Medline]
    44. Maani CV, Hoffman H, Fowler M, Maiers A, Gaylord K, Desocio P. Combining ketamine and virtual reality pain control during severe burn wound care: one military and one civilian patient. Pain Med 2011 Apr;12(4):673-678 [FREE Full text] [CrossRef] [Medline]
    45. Hoffman HG, Meyer WJ, Ramirez M, Roberts L, Seibel EJ, Atzori B, et al. Feasibility of articulated arm mounted Oculus Rift Virtual Reality goggles for adjunctive pain control during occupational therapy in pediatric burn patients. Cyberpsychol Behav Soc Netw 2014 Jun;17(6):397-401 [FREE Full text] [CrossRef] [Medline]
    46. Kipping B, Rodger S, Miller K, Kimble R. Virtual reality for acute pain reduction in adolescents undergoing burn wound care: a prospective randomized controlled trial. Burns 2012 Aug;38(5):650-657. [CrossRef] [Medline]
    47. Jeffs D, Dorman D, Brown S, Files A, Graves T, Kirk E, et al. Effect of Virtual Reality on Adolescent Pain During Burn Wound Care. Journal of Burn Care & Research 2014;35(5):395-408. [CrossRef]
    48. Gold JI, Kim SH, Kant AJ, Joseph MH, Rizzo AS. Effectiveness of virtual reality for pediatric pain distraction during i.v. placement. Cyberpsychol Behav 2006 Apr;9(2):207-212. [CrossRef] [Medline]
    49. Gromala D, Song M, Yim J, Fox T, Barnes S, Nazemi M. Immersive VR: A Non-pharmacological Analgesic for Chronic Pain? 2011 Presented at: International Conference on Human Factors in Computing Systems; May 2011; Chicago, USA. [CrossRef]
    50. Oneal BJ, Patterson DR, Soltani M, Teeley A, Jensen MP. Virtual reality hypnosis in the treatment of chronic neuropathic pain: a case report. Int J Clin Exp Hypn 2008 Oct;56(4):451-462 [FREE Full text] [CrossRef] [Medline]
    51. Jones T, Moore T, Rose H, Choo J. The impact of virtual reality on chronic pain. The Journal of Pain 2016 Apr;17(4):S102-S103. [CrossRef]
    52. Cole J, Crowle S, Austwick G, Slater DH. Exploratory findings with virtual reality for phantom limb pain; from stump motion to agency and analgesia. Disabil Rehabil 2009;31(10):846-854. [CrossRef] [Medline]
    53. Bach F, Schmitz B, Maaß H, Cakmak H, Diers M, Bodman RB, et al. Using interactive immersive VR/AR for the therapy of phantom limb pain. 2010 Presented at: 13th International Conference on Humans and Computers Pages; December 8-10, 2010; Japan p. 183-187   URL: http://dl.acm.org/citation.cfm?id=1994529
    54. Gaggioli A, Amoresano A, Gruppioni E, Verni G, Riva G. A myoelectric-controlled virtual hand for the assessment and treatment of phantom limb pain in trans-radial upper extremity amputees: a research protocol. Stud Health Technol Inform 2010;154:220-222. [Medline]
    55. Murray CD, Pettifer S, Howard T, Patchick EL, Caillette F, Kulkarni J, et al. The treatment of phantom limb pain using immersive virtual reality: three case studies. Disabil Rehabil 2007 Sep 30;29(18):1465-1469. [CrossRef] [Medline]
    56. Glegg SMN, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors influencing therapists' adoption of virtual reality for brain injury rehabilitation. Cyberpsychol Behav Soc Netw 2013 May;16(5):385-401. [CrossRef] [Medline]
    57. Turolla A, Dam M, Ventura L, Tonin P, Agostini M, Zucconi C, et al. Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. J Neuroeng Rehabil 2013 Aug 01;10:85 [FREE Full text] [CrossRef] [Medline]
    58. Adamovich SV, Fluet GG, Tunik E, Merians AS. Sensorimotor training in virtual reality: a review. NeuroRehabilitation 2009;25(1):29-44 [FREE Full text] [CrossRef] [Medline]
    59. Mahajan HP, Dicianno BE, Cooper RA, Ding D. Assessment of wheelchair driving performance in a virtual reality-based simulator. J Spinal Cord Med 2013 Jul;36(4):322-332 [FREE Full text] [CrossRef] [Medline]
    60. Harrison A, Derwent G, Enticknap A, Rose FD, Attree EA. The role of virtual reality technology in the assessment and training of inexperienced powered wheelchair users. Disabil Rehabil 2002;24(11-12):599-606. [CrossRef] [Medline]
    61. Psotka J. Immersive training systems: Virtual reality and education and training. Instr Sci 1995 Nov;23(5-6):405-431. [CrossRef]
    62. Barfield W, Weghorst S. The Sense of Presence within Virtual Environments: A Conceptual Framework. In: Salvendy G, Smith M, editors. Human-Computer Interaction: Software and Hardware Interfaces. London: Elsevier; 1993:699-704.
    63. Bowman D, McMahan RP. Virtual Reality: How Much Immersion Is Enough? Computer 2007 Jul;40(7):36-43. [CrossRef]
    64. Rash CE. A 25-year retrospective review of visual complaints and illusions associated with a monocular helmet-mounted display. Displays 2008 Mar;29(2):70-80. [CrossRef]
    65. Hoffman H, Seibel E, Richards T, Furness T, Patterson D, Sharar S. Virtual reality helmet display quality influences the magnitude of virtual reality analgesia. J Pain 2006 Nov;7(11):843-850. [CrossRef] [Medline]
    66. Duh H, Lin J, Kenyon RV, Parker DE, Furness TA. Effects of Field of View on Balance in an Immersive Environment. Proceedings IEEE Virtual Reality 2001 2001. [CrossRef]
    67. Slater M, Wilbur S. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 1997 Dec;6(6):603-616 [FREE Full text] [CrossRef]
    68. Steuer J. Defining Virtual Reality: Dimensions Determining Telepresence. J Commun 1992;42(4):73-93. [CrossRef]
    69. Nechvatal J. Immersive Ideals/Critical Distances. Saarbrücken: Lambert Academic Publishing; 2009.
    70. Lin J, Duh H, Parker D, Abi-Rached H, Furness TA. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. Proceedings IEEE Virtual Reality 2002 2002:164. [CrossRef]
    71. Slater M, Lotto B, Arnold M, Sanchez-Vives MV. How we experience immersive virtual environments?: the concept of presence and its measurement. UB Journal of Psychology 2009;40(2):210.
    72. Osimo SA, Pizarro R, Spanlang B, Slater M. Conversations between self and self as Sigmund Freud--A virtual body ownership paradigm for self counselling. Sci Rep 2015 Sep 10;5:13899 [FREE Full text] [CrossRef] [Medline]
    73. Hoffman HG, Sharar SR, Coda B, Everett JJ, Ciol M, Richards T, et al. Manipulating presence influences the magnitude of virtual reality analgesia. Pain 2004 Sep;111(1-2):162-168. [CrossRef] [Medline]
    74. Schbert T, Regernbrecht H, Friedmann F. iGroup Presence Questionnaire (IPQ) Overview. 2016.   URL: http://www.igroup.org/pq/ipq/index.php [accessed 2018-07-31] [WebCite Cache]
    75. Schneider SM, Prince-Paul M, Allen MJ, Silverman P, Talaba D. Virtual reality as a distraction intervention for women receiving chemotherapy. Oncol Nurs Forum 2004;31(1):81-88. [CrossRef] [Medline]
    76. Melzack R, Wall PD. Pain mechanisms: a new theory. Science 1965 Nov 19;150(3699):971-979. [Medline]
    77. McCaul KD, Malott JM. Distraction and coping with pain. Psychol Bull 1984 May;95(3):516-533. [Medline]
    78. Melzack R. Evolution of the neuromatrix theory of pain. 2005 Jun Presented at: World Congress of World Institute of Pain; 2005; Barcelona, Spain p. 85-94. [CrossRef]
    79. Bantick SJ, Wise RG, Ploghaus A, Clare S, Smith SM, Tracey I. Imaging how attention modulates pain in humans using functional MRI. Brain 2002 Feb;125(Pt 2):310-319. [Medline]
    80. Gold J, Belmont KA, Thomas D. The neurobiology of virtual reality pain attenuation. Cyberpsychol Behav 2007 Aug;10(4):536-544. [CrossRef] [Medline]
    81. Sander WS, Eshelman D, Steele J, Guzzetta CE. Effects of distraction using virtual reality glasses during lumbar punctures in adolescents with cancer. Oncol Nurs Forum 2002;29(1):E8-E15. [CrossRef] [Medline]
    82. Mahrer NE, Gold JI. The use of virtual reality for pain control: a review. Curr Pain Headache Rep 2009 Apr;13(2):100-109. [Medline]
    83. Hoffman HG, Chambers GT, Meyer WJ, Arceneaux LL, Russell WJ, Seibel EJ, et al. Virtual reality as an adjunctive non-pharmacologic analgesic for acute burn pain during medical procedures. Ann Behav Med 2011 Apr;41(2):183-191 [FREE Full text] [CrossRef] [Medline]
    84. Wender R, Hoffman HG, Hunner HH, Seibel EJ, Patterson DR, Sharar SR. Interactivity Influences the Magnitude of Virtual Reality Analgesia. J Cyber Ther Rehabil 2009;2(1):27-33 [FREE Full text] [Medline]
    85. Chinn P, Kramer M. Integrated theory and knowledge development in nursing. St. Louis, MO: Mosby/Elsevier; 2013.
    86. Morris LD, Louw QA, Crous LC. Feasibility and potential effect of a low-cost virtual reality system on reducing pain and anxiety in adult burn injury patients during physiotherapy in a developing country. Burns 2010 Aug;36(5):659-664. [CrossRef] [Medline]
    87. Furman E, Jasinevicius T, Bissada NF, Victoroff K, Skillicorn R, Buchner M. Virtual reality distraction for pain control during periodontal scaling and root planing procedures. J Am Dent Assoc 2009 Dec;140(12):1508-1516. [Medline]
    88. Fassbender E, Heiden W. Atmosphaeres - 360° Video Environments for Stress and Pain Management. In: Lecture Notes in Computer Science. Heidelberg: Springer Link; 2014:48-58.
    89. Havig P, McIntire J, Geiselman E. Virtual reality in a cave: Limitations and the need for HMDs? Proc. SPIE 8041 2011. [CrossRef]
    90. Costa J, Robb J, Nacke L. Physiological acrophobia evaluation through in vivo exposure in a VR CAVE. 2011 Presented at: IEEE Games Media Entertainment; 2014; Toronto, Canada. [CrossRef]
    91. Markus LA, Willems KE, Maruna CC, Schmitz CL, Pellino TA, Wish JR, et al. Virtual reality: feasibility of implementation in a regional burn center. Burns 2009 Nov;35(7):967-969. [CrossRef] [Medline]
    92. Gold JI, Kant AJ, Kim SH, Rizzo A. Virtual anesthesia: The use of virtual reality for pain distraction during acute medical interventions. Seminars in Anesthesia, Perioperative Medicine and Pain 2005 Dec;24(4):203-210. [CrossRef]
    93. Maani C, Hoffman H, DeSocio P, Morrow M, Galin C, Magula J, et al. Pain Control During Wound Care For Combat-Related Burn Injuries Using Custom Articulated Arm Mounted Virtual Reality Goggles. J CyberTherapy Rehabil 2008;1(2):193 [FREE Full text]
    94. Lohse KR, Hilderman CGE, Cheung KL, Tatla S, Van der Loos HFM. Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PLoS One 2014;9(3):e93318 [FREE Full text] [CrossRef] [Medline]
    95. Powers MB, Emmelkamp PM. Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J Anxiety Disord 2008;22(3):561-569. [CrossRef] [Medline]
    96. Turner WA, Casey LM. Outcomes associated with virtual reality in psychological interventions: where are we now? Clin Psychol Rev 2014 Dec;34(8):634-644. [CrossRef] [Medline]
    97. Rosenfield M. Computer vision syndrome (a.k.a. digital eye strain). Optometry in Practice 2016;17(1):1-10.
    98. Leroy L, Fuchs P, Moreau G. Real-time adaptive blur for reducing eye strain in stereoscopic displays. ACM Trans Appl Percept 2012 Jun 01;9(2):1-18. [CrossRef]
    99. LaViola JJ. A discussion of cybersickness in virtual environments. SIGCHI Bull 2000 Jan 01;32(1):47-56. [CrossRef]
    100. Bonato F, Bubka A, Palmisano S. Combined pitch and roll and cybersickness in a virtual environment. Aviat Space Environ Med 2009 Nov;80(11):941-945. [Medline]
    101. Ling Y, Brinkman WP, Nefs HT, Qu C, Heynderickx IE. Cybersickness and anxiety in virtual environments. 2011 Presented at: Joint Virtual Reality Conference; 2011; Nottingham, UK p. 115.
    102. McCauley ME, Sharkey TJ. Cybersickness: Perception of Self-Motion in Virtual Environments. Presence: Teleoperators and Virtual Environments 1992 Jan;1(3):311-318 [FREE Full text] [CrossRef]
    103. Barrett J. Side effects of virtual environments: A review of the literature.: Defence Science and Technology Organisation Canberra (Australia); 2004.   URL: https://pdfs.semanticscholar.org/ab1b/4153e44abb4c1a1fcac5f2aaee847d30ecf3.pdf [accessed 2018-07-31] [WebCite Cache]

    ‎
    2D: two-dimensional
    3D: three-dimensional
    CGI: computer-generated imagery
    PTSD: posttraumatic stress disorder
    VR: virtual reality

    Edited by G Eysenbach; submitted 21.04.18; peer-reviewed by B Birckhead, B Atzori, S Thurnheer; comments to author 12.07.18; revised version received 31.07.18; accepted 07.08.18; published 17.10.18

    Copyright

    ©Bernie Garrett, Tarnia Taverner, Diane Gromala, Gordon Tao, Elliott Cordingley, Crystal Sun. Originally published in JMIR Serious Games (http://games.jmir.org), 17.10.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.