This study is a prospective, quantitative, and nonrandomized observational study examining the effects of 2 VR input systems (wFC and woFC) on brain activation in both younger adults and older adults. The research was conducted at the Motion Analysis Laboratory and the Medical Device Innovation Center of National Cheng Kung University (NCKU) in Tainan, Taiwan, from July 10, 2023, to August 11, 2023. The reporting adheres to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines for observational research [19].

Participants were recruited through posters displayed around the NCKU campuses and local Line groups. The eligibility criteria included being at least 21 years of age, not having conditions that would prevent normal use of the VR system (such as finger or hand amputations), and not experiencing motion sickness when using the VR system.

The study was approved by the Human Research Ethics Review Committee of NCKU (approval NCKU HREC-E-112-181-2) and was also registered in ClinicalTrials.gov (NCT06412887). Informed consent was obtained from all participants before data collection began, including consent for the primary or secondary analysis and publication of research data. All data were kept anonymous. Participants received a compensation of US $18 upon completing the experiment.

The hardware part of the VR Force Control Training System consists of 4 main components. A laptop computer running Unity (Unity Technologies) is labeled A in Figure 1. A head-mounted display, specifically the Meta Quest 2 (Meta Platforms Inc), is labeled B in Figure 1 and is used to display the game. A force-sensing system, built around a Flexiforce A301 force-sensitive resistor (FSR) from Tekscan, Inc, is labeled D in Figure 1. This FSR was calibrated using test weights ranging from 300 g to 7 kg with 5 trials per weight, each lasting 10 seconds. An Arduino Uno, labeled C in Figure 1, converts and sends the FSR readings to the laptop computer running Unity. Figure 1 illustrates the VR force control training system used in this study. The system’s stability was previously evaluated by measuring 3 different test weights over 3 days at 3-hour time intervals. The resulting intraclass correlation coefficient over 3 days was 0.75.

As mentioned in the previous sections, the game consists of 2 tasks that can be played using 2 input systems, both based on the hand-tracking features provided by the Oculus Interaction SDK. The first task requires users to memorize a 5-element list containing the order of the food to be fed to the baby. The second task involves picking up an interactive fork, selecting the appropriate food, and feeding it to the baby. In the woFC system, the user simply performs a thumb-index finger opposition to pick up the fork. In contrast, the wFC system is designed with the same thumb-index finger opposition in woFC system but with an additional isometric pinch contraction at 70 (SD 10%) maximum voluntary contraction (MVC), a value derived from a prior pilot study.

Once the fork is picked up in the wFC system, the user’s pinch force is continuously monitored and displayed on a force bar and provides real-time feedback as depicted in Figure 2. Exceeding or falling below the allowable pinch force range causes the fork to reset, requiring the users to initiate the grasp again. These modifications in the wFC system introduce an added layer of difficulty immersion compared with the woFC system. To achieve this functionality, custom code was written to modify the hand-tracking features of the Oculus Interaction SDK. Differences between wFC and woFC systems are summarized by Figure 3.

Brain activity was measured using NIRScout system (NIRx Medical Technologies). The setup consisted of 8 sources and 16 detector optodes, spaced 3 cm apart, forming 30 channels. These were arranged according to the international 10‐20 system Figure 4 to cover the regions of interest (ROIs) involved with hand movements, specifically the prefrontal cortex (PFC), premotor cortex (PMC), supplementary motor area (SMA), and primary motor cortex (M1). Data were collected over 2 wavelengths (760 nm and 850 nm) with a sampling rate of 7.81 Hz. The experiment followed a block-related design as illustrated in Figure 5. A trial in this study consisted of a single gameplay loop, followed by a rest event, each lasting 30 seconds. Measurements were taken during 5 consecutive trials using the gameplay loop wFC, followed by a rest event and then 5 consecutive trials using the gameplay loop woFC. During each gameplay loop the participants played the game using the assigned input system. In contrast, during the rest events, they were asked to close their eyes and enter a resting state.

The experiment procedures were as follows: First, participants entered an isolated room where they were briefed about the experiment and provided informed consent. Second, participants familiarized themselves with both input systems of the game, and the in-game maximum pinch voluntary contraction was recorded to calculate each user’s force threshold. Third, the fNIRS system was calibrated using the NIRStar software (NIRx Medical Technologies) to ensure acceptable signal quality across all channels. Finally, data collection proceeded according to the block-related design outlined in Figure 5, while Figure 6 shows a participant performing the experiment.

Game performance was quantified using 3 parameters: the amount of grasp initiated, pinch force coefficient of variation (COV), and memory task accuracy. As the amount of grasp initiated, a unitless metric was derived by counting the number of times the participants needed to pick up the fork in each gameplay loop, reflecting the number of times the participants’ pinch force exceeded or fell below the threshold. The pinch force COV was calculated by dividing the SD of the pinch force by the average pinch force during each gameplay loop, resulting in a unit of Newton/Newton. Memory task accuracy was calculated by dividing the number of correct foods fed by the total number of foods fed in each gameplay loop.

Raw fNIRS signals were processed using MATLAB R2023b (Mathworks) running the open-source app Homer3 (version 1.80.2) [20]. The signals were preprocessed manually and by using functions inside Homer3. These include visually inspecting the raw signals for noise and discontinuities, channel pruning, conversion from intensity to optical density, motion corrections, bandpass filtering, and conversion from optical density to concentration. Channel pruning was performed to eliminate signals that were too weak, too strong, or had a high SD. Motion correction was done using the Wavelet-based algorithms [21]. Bandpass filter was set at 0.01 Hz and 0.1 Hz was applied to eliminate the effects of heartbeat, breathing, and other Mayer waves [22]. The optical densities were then converted into concentration using the modified Beer-Lambert Law with the default partial path length factor of 1.

The hemodynamic response function (HRF) of HbO and HbR was estimated with general linear modeling using the ordinary least squares method [23]. The HRF was modeled by a consecutive sequence of Gaussian functions, with SD of 1 and a temporal spacing of 1 second between the mean over a period of −5 to 35 seconds [24]. The baseline period was set to 5 seconds before stimulus onset, and its mean was subtracted from the whole HRF (−5 seconds to 35 seconds) to perform baseline normalization. The mean HbO and HbR concentration for each ROI was calculated by averaging across the channels in each ROI from the period of 10 seconds to 35 seconds owing to the delay in movement-induced hemodynamic response [22]. The mean HbO and HbR concentration was then projected to the cortical surface of the Colin Atlas by registering the optode and channel locations in AtlasViewer (version 2.44) [25], a MATLAB toolbox for the anatomical interpretation of fNIRS data processed in Homer3.

The Pearson correlation between each ROI’s processed HbO and HbR data was calculated for each group, resulting in four 8×8 matrices representing 4 full network connections. For a more comprehensive evaluation of the network, 2 features were extracted for statistical analysis: connection ratio (CR) and connection strength (CS) [26]. To obtain CR and CS, the Pearson correlation coefficient of each matrix was first converted to normal distribution through the Fisher z transformation [27]. CR was defined as the ratio between the significant connections and the total number of connections in each network, while CS was defined as the average of absolute z values of a network. A connection was considered significant when its absolute z value was greater than 0.7.

The Mann-Whitney U test was used for comparing the game performance between younger adult and older adult populations. The Wilcoxon signed rank test was used to assess the statistical significance of differences in HbO and HbR responses, CS, and CR between the 2 input systems. The Spearman correlation was used to identify correlations between game scores and HbO and HbR responses. The level of significance was set at .05. For data involving multiple comparisons, the Benjamini-Hochberg procedure [28] was used to control the false discovery rate with a Benjamini-Hochberg critical value of 0.05.

The study included 12 eligible younger adults (mean age 25.00, SD 4.5 years) and 12 eligible older adults (mean age 73.00, SD 3.6 years). Of the participants, 23 were right-handed; the dominant hand of 1 participant was not specified.

Table 1 shows the average game scores per run and its comparison between both groups. Only the pinch force COV showed a statistically significant difference.

The mean HbO concentration for the younger adults and the older adults during gameplay is shown in Tables 2 and 3, respectively. Some ROIs show significantly higher HbO concentration when playing the game using the wFC, namely, the right premotor cortex (rPMC); right primary motor cortex (rM1) for the younger adults; and left primary motor cortex (lM1), right prefrontal cortex (rPFC), and rM1 for the older adults.

The mean HbR concentration for the younger adults and the older adults during gameplay is shown in Tables 4 and 5, respectively. Some ROIs show significantly lower HbR concentration when playing the game using the wFC, namely, the rPMC; rM1 for the younger adults; and lM1, rPFC, right supplementary motor area (rSMA), and rM1 for the older adults.

The correlation matrices between the HbO and HbR data of each ROI for both groups, while playing the game with and without force control (wFC and woFC), are presented in Figures 7 and 8, respectively. ROIs with higher correlations appear in reddish colors, while ROIs with less correlations appear in bluish colors.

The CS and CR for the younger adults and the older adults are shown in Table 6. The older adult group exhibited higher CS and CR when playing the game using wFC as compared with woFC.

Figures 9 and 10 present the temporal characteristics of the HRF (HbO and HbR) of both groups during gameplay. In Figure 9, the HbO temporal characteristics of the PFC, PMC, and M1 during wFC follow a rise after task onset that slowly decreases as the task progresses. The HbR follows the opposite pattern. As for the SMA, the left side, regardless of input system, shows a spike in the HbO and the HbR, followed by a constantly elevated response, while the HbO and the HbR of the right side stay at around baseline levels. Regarding the M1 during woFC, the HbO and the HbR on both the left and right sides show little to no response. For the older adult group, Figure 10 shows that all ROIs follow an increase in HbO after task onset, which slowly decreases as the task progresses, while the HbR follows an opposite pattern.

Figures 11 and 12 show the cortical activation maps during gameplay under both input systems for the younger adult and older adult groups, respectively. The red color signifies increases in HbO while blue signifies decreases in HbO. Based on both figures, the wFC elicits a larger area of activation than the woFC, irrespective of age groups. Age-related differences occur in the laterality of activation, with the older adults experiencing a more bilateral activation across 2 input systems.

This study investigated the effects of integrating force control into a VR hand-tracking system on brain activity in the younger adults and the older adults. Participants played the same game using 2 different control systems (wFC and woFC). Despite studies showing that there is a clear difference in game performance between younger adults and older adults, whether it be commercial games [29], exergames [30,31], cognitive games [32], and so forth, the results of this study did not (amount of grasp initiated and memory task accuracy). In the aforementioned studies, aging-related changes and the level of difficulty were identified as the main factors for the differences in game performance. As for this study, the difficulty influenced by the force threshold and number of elements in the food list could not have been challenging enough, which resulted in similar performance across age groups.

The findings of this study align with previous fNIRS research [12,33] on finger opposition movement as a primary input and the effects of different interaction modes in both immersive and nonimmersive VR games. These studies suggest that changes in interaction mode or input system can influence movement patterns, posture, perceived difficulty, and enjoyment, all of which may contribute to variations in brain activity intensity and the areas of activated brain regions. Looking further into how the 2 input systems (wFC and woFC) affected each ROIs reveals the following:

First, the addition of force control can introduce elements that can alter the movement strategies of the participants as reflected by the higher right PMC activity in the younger adults and higher rSMA activity in the older adults during wFC gameplay. Aside from motor planning, the PMC is also responsible for action observation and transforming visual information into motor commands. This includes the anticipation of the force output required to lift an object based on visual cues [34]. Furthermore, the precision of one’s pinch force control can also greatly affect the PMC activation, with tasks requiring a higher precision of pinch force control, such as holding a small object with a sustained force level of just above the point before the object slips, producing a much higher right PMC activity as compared with holding the small object with one’s normal and automatically scaled pinch force [35]. The right PMC activity of the younger adult group in this study also followed the behavior found in the previous studies. The lack of difference in the right PMC activation of the older adults when playing the game in both control systems can be due to their right PMC already being recruited to play the game using the woFC as a compensatory mechanism by the aging brain to recruit additional cortical areas when performing simple motor tasks [36,37]. Furthermore, this mechanism, along with the wFC’s requirement to follow a precise movement sequence (thumb and index finger opposition followed by sustained isometric contraction at a strict time window ~0.5 second), likely contributed to the higher rSMA activity exhibited by the older adults, as well as the weak to moderate correlation between the pinch force COV-HbO (r=0.393) and pinch force COV-HbR (r=−0.494). The SMA is directly involved in planning complex movements that require temporal accuracies, as revealed by studies requiring users to press a button 2.5 seconds after a stimulus [38-40] and in studies disrupting the function of the SMA via repetitive transcranial magnetic stimulation, revealing worse performance in a sequential pressing task.

Second gameplay using the wFC induces greater ipsilateral M1 activity in both groups than gameplay using the woFC. In the young and healthy population, simple movements involving the hands and fingers exclusively activate the contralateral M1, while older adults feature more bilateral activity in the motor regions in response to motor tasks [36,41-43]. Aside from being age dependent, the ipsilateral activity of the M1 could also be influenced by task requirements and complexity [44-47]. Tasks that require sustained isometric contraction can also induce bilateral activity in the M1, detectable by both fNIRS and functional magnetic resonance imaging (fMRI) [48-50]. The magnitude of the force during these tasks ranges from 20% to 80% MVC and lasts from 10 up to 180 seconds per trial. Results from these studies point out that the bilateral activity of the M1 can be achieved by finger tasks accompanied by 20%, 40%, and 60% MVC. The contralateral activity of the M1 does not scale directly with force output, and the ipsilateral M1 is directly affected by the magnitude of the force, as well as the duration of the muscle contraction. This study also obtained consistent results with the previously mentioned studies. The results of this study aligning with previous findings show that wFC gameplay induces higher rM1 activity than woFC gameplay in both groups. This can be attributed to the wFC requiring users to exert an isometric contraction at approximately 70% MVC and maintain it for a few seconds to play the game effectively. Additionally, a positive correlation was observed between pinch force COV and HbO, with higher pinch force COV scores being associated with greater rM1 activity (r=0.494). One thing to take note of is that only the older adults exhibited higher activity in the lM1 in addition to the rM1 during gameplay using the wFC as compared with the woFC. This finding is also consistent with the previous studies mentioned, where older adults generally exhibit more bilateral activation in the motor regions than younger adults performing the same task.

Finally, rPFC in the older adults is recruited to keep up with the higher demands of playing the VR game when using the wFC. Aside from being involved with cognitive functions [51,52] and motor tasks [53], numerous studies also link increasing attentional demands, dual tasking costs, task difficulty, and complexity to the degree of PFC activity [54-57]. The functions of the PFC can also be divided further between the left and right hemispheres, with the rPFC being implicated for both externally and internally cued [58] real-time monitoring of movement and for the retrieval of memory [59,60]. As for this study, in addition to the memory and motor tasks when using the woFC, the introduction of a force control task adds a layer of complexity to the game when using the wFC. This could have caused the older adult group to allocate more attentional and memory resources for adjusting their force output through monitoring of the force bar, as reflected by the similar number of grasps initiated and the higher pinch force COV as compared with the younger adults, while at the same time recalling the order of the food list. fMRI studies [61,62] also found rPFC activity during a precision force control task and cited the visual feedback as a contributing factor to the rPFC activity.

Stronger CS and CR in previous studies reflect an increase in coordinating the senses and motor control function [63] and an additional cognitive load [64] in response to task complexity. In this study’s wFC, the additional requirement of maintaining one’s pinch force, alongside the compensatory mechanism of the aging brain mentioned in the previous sections, may explain higher CS and CR in the older adults during wFC.

The temporal characteristics of the HRF for both groups tend to follow the typical hemodynamic response as those found in the literature [14,17] in which activated brain regions experienced a rise in the concentration of HbO, accompanied by a drop in the concentration of HbR.

When considering the cortical activation maps, differences can be observed between the 2 input systems and between the 2 study groups. Comparing the cortical activation maps of the 2 input systems shows that the wFC activates a larger spatial activity in the brain. fMRI and positron emission tomography studies [50,65-69] have also established a link between the extent of the spatial activity of the brain to the force level and the presence of an isometric contraction during a handgrip task. These studies suggest that recruiting larger areas of the cortex is an adaptation mechanism by the brain for sustaining isometric force tasks. The same process could have also occurred in this study, as gameplay using the wFC requires an isometric pinch as opposed to the woFC system where closing the gap between the index finger and the thumb is enough to play the game properly. On the other hand, a comparison of the cortical activation maps between the 2 study groups reveals that the older adult group experienced more bilateral brain activity irrespective of control systems. fMRI studies [43,70], where subjects were asked to perform finger opposition and hand movements, also obtained comparable results with their older adult groups exhibiting a more bilateral activity in several cortical regions (PMC, SMA, M1, etc) than their younger adult counterparts. The effects of aging on the corpus callosum are cited to be one of the culprits to the reduction of hemispheric lateralization in the older adults.

This study is limited by the following factors: First, the study population consisted of only healthy individuals, so the effects of the developed system on the Hb response of patients with neurological symptoms cannot be determined. Second, the experiment used a fixed block design—5 consecutive wFC gameplay sessions followed by 5 consecutive woFC sessions—to minimize the time interval between conditions. This design was chosen to reduce delays, as removing the sensor was quicker than reapplying it midexperiment, which could have introduced systematic bias. Third, the single game difficulty level was used in the study, as different game difficulty levels [71] can directly affect the Hb responses of the brain. Fourth, the long-term training or rehabilitation effects of this system cannot be determined in this study as this study was a one-time intervention. Fifth, the usage of a single sensor in the system, although transposable, limits it to only accepting inputs between a single thumb and opposing finger pair, and modification to the system needs to be done to let it accept inputs from different thumb and opposing finger pairs. And finally, the limited number of source detector pairs used in fNIRS data acquisition may not be enough to cover the whole motor area region of the brain.

To address these limitations, a long-term follow-up study containing training sessions, adjustable difficulty levels, modified experiment block design, and so forth, can be performed to give more insight regarding the effectiveness of adding a force control component to rehabilitation outcomes of VR-based programs.

In this study, the effects between the regular hand-tracking system of VR head-mounted displays and the novel VR input system on brain activity were compared. To the author’s best knowledge, this is the first system specifically designed for training or rehabilitation in the literature to incorporate a force control component to a VR hand-tracking system. The fNIRS results from this study suggest that the addition of a force control component to VR hand-tracking systems could make them more effective for training or rehabilitation by inducing greater and more widespread activity in the cortex, thus making it better for promoting neural plasticity. Additionally, the results from this study can be used as a reference for the development of new VR-based training or rehabilitation systems, which incorporate force control as a gameplay component.