Malignant or benign tumors, advanced osteomyelitis, osteoradionecrosis, and complex fractures can lead to extensive bone and soft tissue defects with the need for reconstruction. Therefore, soft and hard tissue reconstruction is an important and commonly used procedure in oral and maxillofacial surgery [1]. Microvascular reconstruction with fibula or iliac crest bone transplants is one of the best options for mandible bone reconstruction [2,3], and it has the highest success rate and delivers the best functional and esthetic results [4,5]. Microvascular reconstruction has been performed using a conventional technique, specifically manual free transplant raising, adjustment, and insertion [6,7].

Given its numerous clinical advantages, a virtual surgical planning (VSP) approach prior to microvascular reconstruction of the jaw has seen increased popularity. This approach involves using preoperative medical imaging within computer-assisted surgery (CAS) applications [8,9]. The advantages of VSP include reduced ischemia, a shorter defect reconstruction time, a shorter surgical procedure [10], a shorter length of hospital stay [10], a lower number of necessary osteotomy revisions, an overall lower volume of bone removed, a lower rate of osseous injury [11], and a better match of removed bone volume to defect volume [12].

However, VSP is associated with a higher preoperative workload [8] and is therefore often delegated to younger fellows or technical staff [13,14]. Despite this, VSP still remains an integral part of the surgical process and should be done or supervised by the performing surgeon [15]. Furthermore, learning VSP is time-consuming and requires an appropriate amount of learning time, but these investments are required to ultimately achieve good clinical outcomes [15-18].

One of the main bottlenecks is the preparation of 3D models by segmentation [19]. This process is still performed on a 2D computer desktop screen (DS) with 2D controls, such as a mouse and keyboard [20], which however seems unsuitable per se for such a task [21]. Furthermore, working in a DS environment differs from working in a surgical site since the DS environment lacks stereoscopic vision, and the use of a mouse and keyboard does not resemble working with surgical instruments at all. This leads to discrepancies between the VSP performed in a DS environment and the surgery performed in an operating theater.

A potential way to overcome the disadvantages of VSP performed in a DS environment could be the use of virtual reality (VR) [22] since it offers stereoscopic vision, allows the user to work manually, and allows more focused work due to immersion [21,23]. VR environments better resemble realistic work routines in the operating theater, as users can rotate and flip anatomical structures and observe different intraoperative viewing angles [24], thus enabling an enhanced understanding of the anatomy [25]. Although VR more broadly has been applied to surgery, it has often been basic or partially immersive VR. This must be distinguished from enhanced, fully immersive, binocular head-based VR [26]. With the so-called second wave that began in 2012, this technology has significantly spread in the consumer market and must be distinguished from augmented reality–based systems, which have become widespread under the terms mixed or extended reality [27]. Areas of application for these new VR head-mounted displays (HMDs) are surgical education, surgical training, and surgical planning [28]. VR simulations have been shown to lead to an improvement in surgical skills among subjects [29]. VR was also used for the visualization of medical images in radiology [22]. Recently, VR has been used for spatial understanding [30] or in the course of multiuser conferences during surgical planning [31]. However, to our knowledge, the potential for performing all steps or substeps of VSP using VR HMDs has not yet been explored.

This study aimed to demonstrate the possible advantages of a VR environment through a substep of VSP, specifically the segmentation of the fibula (calf bone) and os coxae (hip bone), which is typically applied in oral and maxillofacial surgery, by conducting a training course in both DS and VR environments and comparing the results.

VSP in a traditional DS environment is still considered the gold standard, despite its disadvantages, such as being tedious and time consuming [43]. This negates the benefit of time saved in the operating theater due to additional time spent on VSP [13,14]. In this context, VR is a promising approach that might attenuate the mentioned disadvantages of VSP performed in a DS environment.

In this study, the processes of learning VSP, using segmentation as a surrogate parameter, between traditional DS and fully immersive VR environments were compared, as segmentation is a common substep in VSP, a bottleneck in terms of time [19,44], and a crucial factor that determines the accuracy of the final computer-aided design (CAD) result [45]. In general, segmentation is important for not only oral and maxillofacial surgery but also many other medical specialties that require 3D models created by segmentation for the subsequent steps in CAD [45]. Therefore, segmentation was used as a surrogate parameter to measure performance in VR compared to performance in a traditional DS environment during a structured training course. For a representative number of cases (n=156), all participants had to create models of the fibula and os coxae, which are commonly used for microvascular reconstruction of the mandible [2,3], in both environments.

Many studies have shown that VSP is beneficial to patients [14,46]. However, VSP is only feasible if the surgeon is proficient in VSP software. For this reason, the process of learning how to operate such CAS-related software has already been the subject of research in the literature. For example, learning curves upon repetition have been found for VSP for the treatment of orbital fractures or in orthognathic surgery [15,47]. In addition, differences in learning curves when it comes to CAD programs used in dentistry have been found between different software and professional groups (students, technicians, and dentists) [18,33]. Furthermore, the learning curve has been shown to be a function of the complexity of the task for model creation using CAD [48]. However, all these studies used applications in a 2D DS environment, where a computer mouse and keyboard were utilized. This type of work environment is counterintuitive to surgeons who are accustomed to ambidextrous, manual, and visuospatial activity while being very focused. In contrast, VR is more similar to surgical work and therefore offers a promising approach.

The results of our study showed that working in a VR environment is significantly more efficient than working in a DS environment. At the same time, VR has a much faster learning curve (Figure 7). To have a comparable measure between models with different volumes, we used segmentation volume per minute (cm³/min), assuming that it significantly reflects the number of attempts to solve the task during the acquisition of cognitive and motor skills [48]. At the end of the training program, this was almost twice as high in the VR environment compared to the DS environment. To account for a transfer effect from one method to the other due to the crossover study design, we split the groups in correspondence to similar studies [18,33] and adjusted our model for training method order. To exclude bias from the multiple steps of the VSP as described in other studies that have evaluated the learning process for an entire VSP/CAS based on the final surgical outcome [15,49], we focused on the process performance itself. The measurable stronger learning effect in the VR environment was also perceived by the participants (Table 2). This is important, as the faster learning of CAD software, such as that for VSP, is also accompanied by increased planning and surgical accuracy, and improved anatomical understanding of the target structures [15,47]. Independent of segmentation performance, it is assumed that for different DS software, with correspondingly longer training periods, the differences will eventually cease to exist [33]. However, this does not explain the large differences observed between the DS and VR environments.

These differences can be explained by the principally different natures of the DS and VR environments. Subjects in our study reported a high degree of anatomical orientation in the VR environment compared to the DS environment (P<.001). One reason for the superior anatomical orientation might be the possibility of obtaining different viewing angles in the VR environment. Specifically, Cha et al demonstrated this advantage [24]. Furthermore, in our study, subjects preferred to work with both hands using 2 VR controllers, appreciated the advantages of stereoscopic vision, and preferred the intuitive perception provided by the VR environment, which might explain their preference for the VR environment over the DS environment overall (Table 3). In summary, this study highlighted the benefits of spatial perception and working in 3D in a VR environment with good orientation in terms of the anatomy and strong similarity to surgical work. Similar results have been found in the use of nonfully immersive VR for CAS of the liver [21]. In the literature, various approaches to using at least nonfully immersive VR in the medical context can be found. For example, stereoscopic monitors have been used in oral and maxillofacial surgery to determine facial soft tissue or orbital volumes [50,51]. Furthermore, they have been used in neurosurgery to train surgeons to resect intracerebral tumors [52] and assess surgical risk as a rehearsal before surgeries [25,53].

Yet, all of the aforementioned approaches lacked the full immersion that comes with using VR HMDs and controllers in both hands, as in our study. In our setting, users held control elements in both hands and worked with them within arm’s reach (Figures 2 and 3). This could be another reason for the rapid learning and efficiency of working in VR, as proprioception can then be used. Therefore, subjects felt particularly comfortable, and the highest precision could be achieved [54]. In contrast, working with a computer mouse and keyboard in the DS environment does not allow this. For complex models (os coxae) in comparison to simpler models (fibula), this advantage was even more pronounced in VR than in DS. In addition to the advantages of the stereoscopic visualization of complex models in VR [55], the reasons for this are that complex models require more commands for their implementation [48] and, at the same time, bimanual work in VR is particularly faster [56].

Regarding the segmentation quality of the os coxae model, a certain discrepancy between the radiologists’ evaluations and that of the subjects appeared to be a shortcoming, as the subjects tended to overestimate themselves. However, this had no impact on clinical practice, since with a given clinically relevant score of ≤2 (good or excellent), we found no significant difference between the 2 environments. A possible reason for this overestimation could be that in the VR environment by default, usually only 1 slice (axial, coronal, or sagittal) was displayed. In contrast, all 3 slices were displayed in parallel in the DS application. One solution would be to display all 3 slices by default in VR. In addition, a different perception of the 3D models (stereoscopic during segmentation by participants vs monoscopic during evaluation by radiologists) could have influenced this. Another reason could be the resolution capacity of the used VR HMD, causing individual voxels to be blurred. Improved hardware or better delineation of individual voxels in the single-layer view could probably solve this.

Overall, our results suggest that the intrinsic feedback learning effect of VSP, which a surgeon achieves by multiple repetitions of VSP and consecutive surgeries [15], could potentially be at least partially replaced by a training program in this manner for novices in VSP. Therefore, the results of our study suggest that VR should be the environment of choice, given its similarity to actual surgical activity. Furthermore, subjects consistently reported less fatigue and more enjoyment while working in the VR environment (Tables 2 and 3). This was also in line with the SUS results, which clearly rated VR better than DS. The reduced exhaustion can be explained by the shorter time needed to complete the task and possibly by the manual 3D interactions in VR, resembling the surgical working environment more realistically (Table 2). Participants perceived the VR software to be superior to the DS software according to the UEQ results for multiple qualities of experience (Figure 8) and preferred the VR environment over the DS environment (Table 3). This was in line with the results of a study on VR user interfaces for medical marking on 3D models [57]. Yet, these differences are surprising, since both applications basically fulfill the same task.

Experiencing the burden of today’s required levels of documentation, surgeons have long been demanding to spend less time at the computer [58]. This could probably also explain the poor rating for the DS environment, which is similar to the typical unpopular work on a DS computer. However, VR has a more playful character, which might explain the higher ratings for the VR environment. However, this clear difference in rating also shows that VSP applications should be adapted to the needs of surgeons. VR seems to meet these requirements, at least in the case of our test persons. In this context, the following things should be considered. Using a VR environment can lead to VR sickness. However, this was not reported by any of the participants, probably because they were sitting and no dynamic scenes were used [59]. In addition, it should be considered that older adults take longer to adapt when learning new technologies [60]. In this regard, studies have shown that older adults also adapt well to VR [61]. Unlike DS with the use of a keyboard and mouse, in the performance of VSP by older adults, VR can be advantageous, as VR is closer to actual surgery due to the immersive environment, stereoscopic vision, and bimanual working. Another point that needs to be considered is cost. Such a system would have to be acquired first. On the other hand, the use of VR, as shown in our study, is time-saving and could thereby reduce the overall cost of VSP. However, an investigation of cost was outside the scope of our study.

Trials featuring the completion of VSP in a nonfully immersive manner have been described to be very promising in the field of liver surgery [21]. However, to our knowledge, no studies have yet assessed VSP performed completely in fully immersive VR. Future studies should address, aside from segmentation, the other steps of full VSP, with the goal to perform VSP completely in VR.

Apart from the approach shown in our study, semiautomated or fully-automated segmentation algorithms are increasingly being used to accelerate or completely take over segmentation [20]. Yet, our study is concerned about which is the better working environment to perform any kind of VSP. This is even more important since the most common segmentation algorithms require a high amount of time for manual postprocessing [45], and the sole use of fully automated algorithms has still not gained general acceptance in clinical practice due to their high sensitivity to image-related artifacts, a resulting inadequate structure recognition, and a low functional stability [20]. In the future, however, artificial intelligence and VR could be combined, especially to further accelerate tasks in VR.

We not only demonstrated the feasibility of segmentation in VR but also highlighted the advantages of this. Possible explanations for these findings are the more intuitive work interface provided by the VR environment, the better spatial orientation, and the stereoscopic vision [21], which were perceived as great advantages for the purposes of VSP.

Through the crossover design, the subjects represented their own control group, and with a total of 312 experiments, the study achieved good internal validity. To improve the generalization of the results, that is, the external validity, we performed the study with subjects having different surgical experience. However, a greater diversity of subjects could lead to different results. The external validity should be evaluated in a further study, and a multicenter study is preferred to include subjects with experience from other clinics. However, this does not diminish the significance of this study. As with the various phases of drug studies, it is worthwhile to first examine the internal validity. Based on the findings of this study, a subsequent study for good external validity can be planned with more subjects but fewer examinations per subject. Furthermore, it should be considered that 2 different software programs (3D Slicer and Elucis) were compared with each other. Although both programs have a comparable command set, a slight impact cannot be excluded. However, this would not explain the large differences observed.

In contrast to the DS environment, the VR environment offers a more rapid learning curve, provides the ability to work faster, results in increased time saved, and is less exhausting when engaging in segmentation as an important substep of VSP. The VR environment was rated by participants to be easier to learn and to be their environment of choice, possibly due to its similarity to actual surgical activities. Therefore, VR-based VSP could be better integrated into clinical practice. This study can serve as a basis for investigating other common substeps of VSP.