Published on in Vol 10, No 2 (2022): Apr-Jun

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/35869, first published .
User Experience and Usability of Neumorphism and Gamification User Interface Designs in an HIV Self-Test Referral Program for Men Who Have Sex With Men: Prospective Open-Label Parallel-Group Randomized Controlled Trial

User Experience and Usability of Neumorphism and Gamification User Interface Designs in an HIV Self-Test Referral Program for Men Who Have Sex With Men: Prospective Open-Label Parallel-Group Randomized Controlled Trial

User Experience and Usability of Neumorphism and Gamification User Interface Designs in an HIV Self-Test Referral Program for Men Who Have Sex With Men: Prospective Open-Label Parallel-Group Randomized Controlled Trial

Original Paper

Stanley Ho Centre for Emerging Infectious Diseases, The Chinese University of Hong Kong, Hong Kong, China (Hong Kong)

Corresponding Author:

Shui Shan Lee, FRCP

Stanley Ho Centre for Emerging Infectious Diseases

The Chinese University of Hong Kong

Room 207, Postgraduate Education Centre

Prince of Wales Hospital

Hong Kong

China (Hong Kong)

Phone: 852 22528812

Email: sslee@cuhk.edu.hk


Background: Digital interventions have been applied for promoting HIV prevention and care among men who have sex with men (MSM). As user interface (UI) design plays a role in determining usability and user experience (UX), the intervention outcome could be affected.

Objective: In this study, we hypothesized that 2 UI design styles, namely gamification and neumorphism, could impact usability and be differentially preferred by distinct groups of MSM.

Methods: A prospective parallel-group open-label randomized controlled trial was conducted in Hong Kong. Eligible participants were adult MSM recruited by the research team or referred by enrolled participants, who followed instructions for performing an HIV self-test and promoted its use within their social network. Participants were randomized in a 1:1 ratio into either a gamification or neumorphism arm, with primarily visual differences in the UI only. The primary outcome was usability measured by the System Usability Scale (SUS) between the 2 arms. Distinct characteristics of promoters in the 2 arms who gave an SUS score of 80 or above were identified.

Results: Of 463 MSM registered in the study, 232 and 231 were randomized to the gamification and neumorphism arms, respectively. Excluding those who did not request a self-test kit, data from 218 and 216 participants in the gamification and neumorphism arms, respectively, were analyzed (totally 434 participants). With a median SUS score of 80 overall, participants in the neumorphism arm gave a higher score (P<.001), with a higher proportion giving a promoter-level SUS score (P=.002). Promoters used social media for sex networking (P=.02), used pre-exposure prophylaxis in the preceding year (P=.006), had higher satisfaction in UI design (P<.001), and had made a self-test referral (P=.04). In general, higher usability was recorded among participants who were confident in performing the HIV self-test (P<.001), and this was associated with a promoter-level SUS score in both arms. While no other personal characteristics were associated with promoters in the neumorphism arm, those in the gamification arm had higher HIV-related knowledge (P=.01), preferred a specific partner body image type (P=.03), and progressed toward peer referral by completing online training (P=.04).

Conclusions: Both gamified and neumorphic UI designs were well-accepted by MSM. UX and satisfaction of UI were both crucial in influencing the willingness of MSM to promote the application by referring their peers in the community to participate. The simplistic visual design of neumorphism conferred a more general acceptance in the community, whereas gamification was preferred in certain MSM subcommunities. Appropriate UI/UX design should be considered when developing digital interventions targeting the MSM community.

Trial Registration: ClinicalTrials.gov NCT04379206; https://clinicaltrials.gov/ct2/show/NCT04379206

JMIR Serious Games 2022;10(2):e35869

doi:10.2196/35869

Keywords



Digital technology has been leveraged as an innovative approach to improve health in different settings, especially in HIV care and prevention. It was adopted for promoting HIV testing through the use of specially designed apps targeting the men who have sex with men (MSM) community [1], monitoring adherence of medication intake [2], and providing HIV prevention materials to the community [3]. A qualitative study showed that ease of operation is an important factor for MSM to use and keep using such apps [4]. This is in line with concepts of the theoretical framework in the technology acceptance model stating that perceived ease of use and usefulness contribute to one’s intention to use technology [5]. Perception of ease of use is underpinned by user interface (UI) design, such that poor visual design could negatively affect the user experience (UX) and create usage challenges [6]. Therefore, UI design is one of the key elements influencing the usability of a digital intervention [7].

There are many models and styles in UI design, and a popular one is gamification. Gamification is the application of video game elements in nongaming contexts to enhance UX [8]. It has been applied for improving drug adherence in rheumatoid arthritis patients [9] and for supporting clinical decision-making [10]. The effectiveness of gamification has been shown in the promotion of health among MSM. A previous study found that a gamified mobile app could improve HIV knowledge and confidence in adherence to HIV medications among these individuals [11]. The positive effect of gamification was markedly higher in regular users [12]. With rewards like badges, credit points, and actual prizes, and social elements, such as leader boards, the elements in the gamified experience were not just acceptable to MSM, but could also motivate them to engage in promotional activities [13]. On the other hand, a new UI design style, neumorphism or soft UI, emerged in late 2019, which was introduced by Alexander Plyuto [14]. It is a simplistic version of skeuomorphism, which was widely used in the past to imitate a real-world physical object in the UI. Neumorphism simplifies skeuomorphism by making use of light and dark shadows to accentuate the UI element from its background with the same color while capturing the realistic form of the object [15]. The combination of realistic element and the simplistic design can offer an interactive and tactile UX [16].

The 2 UI design styles of gamification and neumorphism were visually distinctive, leading us to hypothesize that different groups of MSM may be in favor of either design. Against these backgrounds, we aimed to assess the differential impact of gamification and neumorphism UI design styles on UX and usability in a web application developed for promoting HIV self-testing among MSM.


Study Design and Participants

This was a prospective parallel-group open-label randomized controlled study. The study was conducted in Hong Kong, where MSM accounted for a majority of new HIV infections in the past decade [17]. Through a social network approach, enrolled eligible participants were men who ever had sex with other men, who were aged 18 years or above, and who were either referred by another participant in the study or recruited by the research team as seeds. The exclusion criterion was inability to communicate in written English or Chinese. Block randomization was adopted with a block size of 4 and an allocation ratio of 1:1, using a computer-generated list concealed in server-side codes in the web application. With a sample size of 400, a mean difference in the acceptance score of 1% could be detected at an assumed standard deviation of 10% between 2 arms.

Intervention

The participants were randomized into either the neumorphism or gamification UI arm, with the unified objective of undertaking a free HIV self-test using oral fluid or blood at their will. The differences in the 2 arms were related to the UI only (primarily visual). In the neumorphism arm, in terms of form elements, the buttons and frames were in the neumorphism style, while checkboxes and radio buttons remained browser default to balance readability and identification of form elements (Multimedia Appendix 1). A monochromatic color scheme was adopted in compliance with the neumorphism design style. To enrich a gamified UX, the process table in the neumorphic design was replaced by a badge board, and a story background was created (Figure 1). Each of the pages in the gamification arm had a unique backdrop simulating different places in a role-playing game. The profile and questionnaire pages appeared as a forest in daytime and at night, respectively. The test kit request and result upload pages depicted a sword to be picked from the soil and items to be devoted in a shrine, respectively. While training was illustrated as a magical book, the backdrop on the referral page was a tavern. The incentive redemption page was assimilated as a reward bestowed by the “crown.” The study procedures remained the same between the 2 arms, namely the questionnaire, test kit request, test result upload, online training, and peer referral.

Figure 1. The profile pages of the neumorphism (upper panel) and gamification (lower panel) designs.
View this figure

Data Collection

The questionnaire contained questions on demographics; sex networking characteristics, including self-perceived body image types and preferences of sex partners’ body image types [18,19], history of sexual behavior, sex networking, and HIV testing; preferences of self-tests; and 17 questions on HIV-related knowledge. The primary outcome of the study was the usability of the web application measured by the System Usability Scale (SUS) [20] and the Single Ease Question (SEQ) [21], as well as the satisfaction of the UI designs of the 2 styles. The SEQ was used after each task throughout the study, whereas the SUS was only displayed after test result upload and peer referral. Participants were free to fill out these scales without obstruction to the main study procedure. Akin to the SUS, an additional question, “I am satisfied with the system interface design,” with a scale ranging from 1 (strongly disagree) to 5 (strongly agree), was used to assess participants’ acceptance of the UI (UI score). For individual SUS items, a score of at least 4 out of 5 was considered a positive response, whereas for the SEQ, a score of 5 or higher on the scale of 7 was regarded as positive. Overall, the SUS score was calculated according to the standard by inversing responses in even number questions and then multiplying the scores by 2.5. Previous research has shown that an SUS score of 68 is average [22], 71 is acceptable [23] or “good” [24], and 80 indicates a user is willing to promote the product or system to a friend [21]. In the analysis, we adopted the latter 2 thresholds as the definitions of acceptance and participation as a promoter of the system, respectively. Two subscale scores were derived from the SUS by separating items 4 and 10 from the remainder to form the learnability and usability scores, respectively, and the same threshold was used.

Analysis

The scores of the UI, individual SUS items, SUS main scale and subscales, and SEQ at different stages were compared between the 2 arms, and other participant attributes and sex networking characteristics were compared using the Mann-Whitney U test. With an overarching aim to examine the willingness of referring peers to undergo self-testing, the SUS scores were dichotomized by the promoter-level threshold, and associated factors were identified per arm by univariable and multivariable logistic regression models. The analyses and reporting of the results followed the CONSORT-eHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) guidelines [25] (Multimedia Appendix 2). This study has been registered at ClinicalTrials.gov (ID: NCT04379206). All analyses were conducted in R software (R Project for Statistical Computing).

Ethics Approval

This study was approved by the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee and the Ethics Committee of the Department of Health (CREC reference number 2020.087).


Overall, 463 MSM registered in the study between March 1, 2021, and May 12, 2021, of whom, 232 and 231 were assigned to the gamification and neumorphism arms, respectively (Figure 2). After excluding 21 subjects who had not completed the questionnaire and 8 who failed to request a self-test kit, data from 434 participants were analyzed. Overall, the participants’ median age was 28 years (IQR 24-33 years) (Table 1). Moreover, 76.5% (332/434) and 43.3% (188/434) of participants had used location-based MSM social networking apps and social media apps, respectively, to seek male sex partners in the past year, whereas patronizing physical venues, such as bars (33/434, 7.6%), saunas (56/434, 12.9%), and sex parties (26/434, 6.0%), for sex networking amidst the COVID-19 pandemic when social distancing policies were in force was rare. In terms of HIV-related knowledge, 59.0% (256/434) of participants got 11 correct answers and 21.0% (91/434) gave at the most 3 wrong answers out of 17 question items. Overall, 340 (78.3%) had tested for HIV before, of whom, 139 (40.9%) never used an HIV self-test product. More participants were confident to collect sufficient oral fluid (269/434, 62.0%) than blood (197/434, 45.4%) for the self-test. In addition, 66.8% (290/434) were confident in performing self-tests and 70.3% (305/434) were confident in interpreting the test results. A similar proportion (282/434, 65.0%) did not prefer assistance in self-tests.

Figure 2. Study flow.
View this figure
Table 1. Participants’ characteristics.
CharacteristicOverall (N=434)Gamification arm (N=218)Neumorphism arm (N=216)
Age (years), median (IQR)28 (24-33)28 (24-32)29 (24-34)
Sex networking in the past 1 year,n (%)



Used location-based MSMa social networking apps332 (76.5)163 (74.8)169 (78.2)

Used social media apps188 (43.3)90 (41.3)98 (45.4)

Patronized local bars33 (7.6)18 (8.3)15 (6.9)

Patronized local saunas56 (12.9)35 (16.1)21 (9.7)

Attended local sex parties26 (6.0)18 (8.3)8 (3.7)
Gave a correct answer in at least 11 out of 17 questions on HIV-related knowledge, n (%)256 (59.0)130 (59.6)126 (58.3)
Gave a correct answer in at least 14 out of 17 questions on HIV-related knowledge, n (%)91 (21.0)46 (21.1)45 (20.8)
Ever tested for HIV, n (%)340 (78.3)166 (76.1)174 (80.6)
Ever HIV self-tested (N=340), n (%)201 (59.1)101 (60.8)100 (57.5)
Confidence in self-test procedures, n (%)



Collecting sufficient oral fluidb269 (62.0)135 (61.9)134 (62.0)

Collecting sufficient bloodb197 (45.4)90 (41.3)107 (49.5)

Correctly performing the HIV self-testb290 (66.8)142 (65.1)148 (68.5)

Interpreting the self-test resultb305 (70.3)146 (67.0)159 (73.6)

Preferring no assistance for the self-test282 (65.0)143 (65.6)139 (64.4)

aMSM: men who have sex with men.

bGave a score of 9 or above on a scale of 0 to 10.

The median SUS score given at test result upload was 80.0 (IQR 70.0-90.0), with median learnability and usability scores of 87.5 (IQR 62.5-100.0) and 78.1 (IQR 68.8-90.6), respectively. Higher learnability was scored by MSM who used social media for sex networking (P=.02), preferred sex partners to be of the “cute” body image type (P=.009), never had sex with a female individual (P=.04), and gave more correct answers in HIV-related knowledge (P=.005) (Table 2). They also gave higher confidence scores in sample collection for both oral fluid (P=.002) and blood (P=.004), performance of the self-test (P<.001), and result interpretation (P<.001). A higher usability score was given by participants who used social media for sex networking (P=.002), preferred sex partners with the body image type of “cute” (P=.03) or “decent” (P=.02) but not “mature” (P=.04), were confident in using the self-test (P<.01), and returned the self-test result within 24 hours upon receiving the self-test kit (P=.04) (Table 3).

Table 2. Differential patterns of learnability scores and associated participant characteristics (N=351).
VariableLearnability score when the variable is false, median (IQR)Learnability score when the variable is true, median (IQR)P value
Used social media for sex networking in the past 1 year87.5 (62.5-100.0)87.5 (75.0-100.0).02
Preferred sex partners to be of the “cute” body image type87.5 (62.5-100.0)87.5 (75.0-100.0).009
Ever had sex with a female individual87.5 (75.0-100.0)75.0 (62.5-100.0).04
Gave at least 14 correct answers out of 17 questions on HIV-related knowledge87.5 (62.5-100.0)87.5 (75.0-100.0).004
Confidence in self-test procedures



Collecting sufficient oral fluida75.0 (62.5-100.0)87.5 (75.0-100.0).002

Collecting sufficient blooda75.0 (62.5-100.0)87.5 (75.0-100.0)<.001

Correctly performing the HIV self-testa75.0 (62.5-100.0)87.5 (75.0-100.0)<.001

Interpreting the self-test resulta75.0 (62.5-87.5)87.5 (75.0-100.0)<.001

Preferring no assistance for the self-test75.0 (50.0-100.0)87.5 (75.0-100.0)<.001

aGave a score of 9 or above on a scale of 0 to 10.

Table 3. Differential patterns of usability scores and associated participant characteristics (N=351).
VariableUsability score when the variable is false, median (IQR)Usability score when the variable is true, median (IQR)P value
Used social media for sex networking in the past 1 year75.0 (68.8-87.5)84.4 (71.9-93.8).002
Preferred body image type of sex partners



“Cute” type78.1 (68.8-90.6)81.3 (75.0-95.3).03

“Decent” type75.0 (68.8-87.5)78.1 (71.9-93.8).02

“Mature” type81.3 (71.9-93.8)75.0 (68.8-87.5).04
Returned the self-test result within 24 hours upon receiving the self-test kit78.1 (65.6-89.8)78.1 (71.9-93.8).04
Confidence in self-testing procedures



Collecting sufficient oral fluida75.0 (68.8-87.5)81.3 (71.9-93.8).001

Collecting sufficient blooda75.0 (65.6-87.5)84.4 (75.0-93.8)<.001

Correctly performing the HIV self-testa75.0 (65.6-84.4)84.4 (71.9-93.8)<.001

Interpreting the self-test resulta75.0 (65.6-84.4)81.3 (71.9-93.8)<.001

aGave a score of 9 or above on a scale of 0 to 10.

Promoters giving at least 80 points in the SUS scale at test result upload (184/351, 52.4%) were more likely to have passed the online training, made a peer referral, used social media for sex networking, used pre-exposure prophylaxis (PrEP) for HIV prevention when having condomless anal intercourse (CLAI) with known male sex partners, and given a higher UI score (P<.05). They were also more confident in using the self-test (P<.001), while preferring no assistance for the self-test (P=.001), and had requested an oral fluid self-test (P=.04) (Table 4). While promoters did not give more correct answers in HIV-related knowledge, nonpromoters were more likely to wrongfully consider blood donation an appropriate way for HIV screening in the HIV-related knowledge assessment (odds ratio [OR] 2.57, 95% CI 1.33-4.95; P=.01). They also gave a higher SEQ score after the test kit request, result upload, online training, and peer referral (P<.01).

Table 4. Characteristics of participants giving a System Usability Scale score of 80 or above (promoters) (N=351).
VariablePromoters (N=184), n (%)Nonpromoters (N=167), n (%)ORa (95% CI)P value
Used social media for sex networking in the past 1 year90 (48.9)61 (36.5)1.66 (1.09-2.55).02
Used PrEPb in the past 1 year (N=165c)59 (70.2)40 (49.4)2.42 (1.28-4.58).006
UId score of 4 or above on a scale of 1-5174 (94.6)119 (71.3)7.02 (3.42-14.42)<.001
Confidence in self-test procedures




Collecting sufficient oral fluide130 (70.7)89 (53.3)2.11 (1.36-3.27)<.001

Collecting sufficient bloode102 (55.4)53 (31.7)2.68 (1.73-4.14)<.001

Correctly performing the HIV self-teste139 (75.5)90 (53.9)2.64 (1.68-4.16)<.001

Interpreting the self-test resulte147 (79.9)97 (58.1)2.87 (1.79-4.60)<.001

Preferring no assistance for the self-test132 (71.7)92 (55.1)2.07 (1.33-3.22).001
Requested for an oral fluid self-test118 (64.1)89 (53.3)1.57 (1.02-2.40).04
SEQf ≥6 on a scale of 1-7 at test kit request (N=348)147 (80.8)107 (64.5)2.32 (1.42-3.77)<.001
SEQ ≥6 on a scale of 1-7 at result upload177 (96.2)123 (73.7)9.05 (3.94-20.75)<.001
SEQ ≥6 on a scale of 1-7 at online training (N=213)53 (44.5)24 (25.5)2.34 (1.30-4.22).004
SEQ ≥6 on a scale of 1-7 at peer referral (N=119)43 (61.4)18 (36.7)2.74 (1.29-5.83).008
Passed online training115 (62.5)83 (49.7)1.69 (1.10-2.58).02
Made a peer referral66 (35.9)43 (25.8)1.61 (1.02-2.55).04

aOR: odds ratio.

bPrEP: pre-exposure prophylaxis.

cAmong participants who had condomless anal intercourse with previously acquired male sex partners.

dUI: user interface.

eGave a score of 9 or above on a scale of 0 to 10.

fSEQ: Single Ease Question.

Participants in the neumorphism arm gave a higher SUS score than those in the gamification arm (median 82.5 vs 77.5; “Excellent” vs “Good” using the adjective rating scale [24]; P<.001), and more participants in the neumorphism arm had an acceptance level (P<.001) and a promoter level (P=.002) (Table 5). The learnability score was similar between the 2 arms; therefore, the differences in the scores of SUS items 4 and 10 were insignificant. On the other hand, the usability score was higher in the neumorphism arm (P<.001). The proportions of participants who were satisfied with the UI (P=.01) and agreed to all positively worded statements in the SUS, except for item 9 on confidence in system use, were higher in the neumorphism arm (P<.01).

Table 5. Comparison of individual and overall System Usability Scale items, learnability scores, and usability scores between the gamification and neumorphism arms (N=351).
Statement/variableGamification arm (N=176), median (IQR)Neumorphism arm (N=175), median (IQR)P valueGamification arm (N=176), n (%)aNeumorphism arm (N=175), n (%)aORb (95% CI)P value
I am satisfied with the system interface design4 (4-5)4 (4-5).01138 (78.4)155 (88.6)0.47 (0.26-0.84).01
SUSc #1: I think that I would like to use this system frequently4 (4-5)4 (4-5).007136 (77.3)158 (90.3)0.37 (0.20-0.67).01
SUS #2: I found the system unnecessarily complex2 (1-3)2 (1-2).0123 (13.1)17 (9.7)1.40 (0.72-2.72).32
SUS #3: I thought the system was easy to use4 (4-5)4 (4-5).008145 (82.4)162 (92.6)0.38 (0.19-0.74).004
SUS #4: I think that I would need the support of a technical person to be able to use this system1 (1-2)1 (1-2).4014 (8.0)9 (5.1)1.59 (0.67-3.79).29
SUS #5: I found the various functions in this system were well integrated4 (3-4)4 (4-5)<.001103 (58.5)145 (82.9)0.29 (0.18-0.48)<.001
SUS #6: I thought there was too much inconsistency in this system2 (1-3)1 (1-2)<.0019 (5.1)8 (4.6)1.13 (0.42-2.99).81
SUS #7: I would imagine that most people would learn to use this system very quickly4 (4-5)4 (4-5).005137 (77.8)156 (89.1)0.43 (0.24-0.78).004
SUS #8: I found the system very awkward to use2 (1-2)1 (1-2).0029 (5.1)9 (5.1)0.99 (0.39-2.57).99
SUS #9: I felt very confident using the system4 (4-5)4 (4-5).09149 (84.7)159 (90.9)0.56 (0.29-1.07).08
SUS #10: I needed to learn a lot of things before I could get going with this system2 (1-3)2 (1-2).0615 (8.5)14 (8.0)1.07 (0.50-2.29).86
SUS acceptance level (≥71)N/AdN/AN/A109 (61.9)147 (84.0)0.31 (0.19-0.51)<.001
SUS promoter level (≥80)N/AN/AN/A78 (44.3)106 (60.6)0.52 (0.34-0.79).002
SUS overall score77.5 (65.6-89.4)82.5 (75.0-92.5)<.001N/AN/AN/AN/A
SUS learnability score87.5 (62.5-100.0)87.5 (75.0-100.0).11N/AN/AN/AN/A
SUS usability score75.0 (65.6-87.5)84.4 (75.0-93.8)<.001N/AN/AN/AN/A

aPercentage refers to the proportion of participants giving a score of 4 or above on a scale of 5 for the respective statement.

bOR: odds ratio.

cSUS: System Usability Scale.

dN/A: not applicable.

To identify the distinct characteristics associated with a promoter-level SUS score with gamified and neumorphic designs, subgroup analyses were performed (Figure 3). Promoters in the gamification arm were more likely to prefer sex partners to be of “meaty,” “hairy,” or “bear” body image types, request for an oral fluid test, and give a higher SEQ score at peer referral (P<.05). They were also less likely to give a wrong answer in some HIV-related knowledge questions (P<.05) and more likely to use PrEP when having CLAI with known sex partners (P=.009) and pass the online training (P=.04). Neumorphism arm promoters were more likely to give a higher SEQ score at test kit request (P<.001) and be referred from channels other than instant messaging apps (P=.02). The multivariable logistic regression model showed that, other than high UI (P<.001) and SEQ (P=.02) scores, promoters in the gamification arm were more likely to request for an oral fluid test (P=.001), prefer a “bear” body image type partner (P=.02), and be confident in collecting blood samples (P=.003), while those in the neumorphism arm gave a higher SEQ score at result upload (P<.001) and were confident in performing the self-test (P<.001).

Figure 3. Factors associated with promoters in the gamification and neumorphism arms. Shaded variables are mutually associated with both arms. Values on the link are odds ratios (95% CIs). CLAI: condomless anal intercourse; PrEP: pre-exposure prophylaxis; SEQ: single ease question; UI: user interface.
View this figure

The social network–based HIV self-test web application was well-accepted by MSM, with an overall median SUS score of 80. Higher system usability was associated with confidence in using the self-test kit and completing the self-test quicker, indicating that this application had linked the user experiences of online activities with offline ones. MSM encountering issues in learning to use the system were less knowledgeable about HIV and more likely to be bisexual. This echoed the results from a previous study showing that bisexual men had lower HIV-related knowledge, who had a higher level of stigma or discrimination associated with the infection [26]. Online social networking tools could be leveraged to promote health behaviors [27], and users gave higher usability and learnability scores. Together with the participants’ distinct preferences in sex partners’ characteristics, our results informed the need of addressing bisexual MSM and the choice of key opinion leaders of different body image types in future HIV-related promotion campaigns through social media.

The study results confirmed that participants who were satisfied with the system and with a high perceived ease of use throughout the engagement, were more likely to promote it to their peers [21]. Promoters were confident in using the self-test without assistance. They tended to have used PrEP for HIV prevention and possessed adequate HIV knowledge; therefore, they were competent to share their positive self-test experiences and knowledge in HIV, and access to HIV prevention services with their peers, which could in turn invite them to get self-tested. On the other hand, knowledge deficit may have hindered some from referring their peers to engage in prevention programs. This showed that a satisfactory UX, adequate HIV knowledge, and peer referral and support could facilitate HIV testing promotion programs [28].

Participants in the 2 arms were satisfied with the system and the interface design. Compared with the findings in the gamification arm, higher SUS and UI scores were observed among participants assigned to the neumorphism arm, with over half giving a promoter-level score. The impact of UI on UX was only reflected in the usability facet, but not the learnability facet. In reference to the 10 usability heuristics of UI design [29], “esthetic and minimalist design” was found to be of particular importance in contrasting the 2 designs in this study. Neumorphism adopts a simplistic visual design conforming with the heuristics, whereas the colorful graphics in the gamification interface may have distracted users from the primary goal and subsequently weakened its usability. Nevertheless, no matter which arm participants were assigned to, almost all users considered the system consistent and agreed that they did not need to learn a lot of things or ask for support to use the system. These were underpinned by the fourth and sixth heuristics, which were “consistency and standards” and “recognition rather than recall,” respectively.

Preferences in UI design style may be related to one’s personality, age, gender, and education [30,31]. Shared characteristics among promoters in the 2 arms included higher SEQ and UI scores at the same time when the promoter-level SUS score was given, and confidence in performing the self-test without assistance. No personal attributes were associated with promoters in the neumorphism arm, but those in the gamification arm were featured with partner body image type preference, PrEP use, and HIV-related knowledge. This highlighted that the gamified UI was favored by selected MSM, whereas the neumorphic design was more universally accepted. Our finding provides an insight into the future selection of art styles for designing promotion materials. Although physical appearance traits may differ contextually, a previous study in the United States has suggested that the bear community could be a unique group characterized by body image type in the MSM population, which may be at higher HIV risk than others [32]. If a campaign is launched to target them and their potential partners, a gamified approach could be of use. A previous study also showed that gamification could make HIV and sexually transmitted infection testing less stressful [33]. On the contrary, a neumorphic style would be more suitable to reach the general MSM community.

This study has several limitations. First, the use of sensitive sexual behavioral questions might have given rise to social desirability bias in responders, which we have minimized by adopting a self-administered approach. We also used a short recall period to reduce recall bias. Second, in a randomized parallel-group design, we were unable to capture participants’ preferences by presenting them with both UI designs. Instead, we compared the scores between 2 arms and conducted subgroup analyses to identify associating factors with scores reaching the promoter level. The SUS scores used in this study were collected at test result upload; therefore, all participants who did not upload their results were excluded. As the primary goal of the study was to have participants refer their peers for self-testing, uploading a test result was a necessary step, and thus, only those who had uploaded a test result were considered to have completed the study procedure and were included in the analyses. Third, HIV knowledge was not assessed with a conventional validated scale, as such a scale may have been outdated with the recent development of HIV medicine and PrEP. The knowledge questionnaire we designed could however only serve the purpose of identifying a knowledge gap in the community, which was relevant to the theme of the study. Fourth, generalization to other settings with different cultural backgrounds could be limited, particularly for body image types. Finally, neumorphism has an inherent weakness in accessibility due to the lack of contrast in color. We did not enquire about users’ color vision; hence, we were unable to consider the impact of color vision deficiency on the usability of the 2 UI designs.

In conclusion, we demonstrated the effective use of a web application that could link users from online activities to offline engagement for the purpose of promoting HIV testing through the collection of self-test kits. The experience of the offline activity could impact subsequent online engagements, which involve peer referral for extending the network of MSM who could go for HIV self-testing as a health promotion strategy. We also identified MSM’s preferences in and possible implementation scenarios of gamified and neumorphic UI designs, laying the scientific foundation for future UI and UX designs for internet interventions targeting the MSM community.

Acknowledgments

We thank the Li Ka Shing Institute of Health Sciences for providing technical support. This study was supported by AIDS Trust Fund (MSS 324 R).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Screenshots of the pages in both the gamification and neumorphism arms.

PDF File (Adobe PDF File), 319 KB

Multimedia Appendix 2

CONSORT-eHEALTH checklist (V 1.6.1).

PDF File (Adobe PDF File), 1202 KB

  1. Veronese V, Ryan KE, Hughes C, Lim MS, Pedrana A, Stoové M. Using digital communication technology to increase HIV testing among men who have sex with men and transgender women: Systematic review and meta-analysis. J Med Internet Res 2020 Jul 28;22(7):e14230 [FREE Full text] [CrossRef] [Medline]
  2. Chai P, Mohamed Y, Bustamante M, Goodman GR, Najarro J, Castillo-Mancilla J, et al. DigiPrEP: A pilot trial to evaluate the feasibility, acceptability, and accuracy of a digital pill system to measure PrEP adherence in men who have sex with men who use substances. J Acquir Immune Defic Syndr 2022 Feb 01;89(2):e5-e15 [FREE Full text] [CrossRef] [Medline]
  3. Simoni JM, Kutner BA, Horvath KJ. Opportunities and challenges of digital technology for HIV treatment and prevention. Curr HIV/AIDS Rep 2015 Dec 28;12(4):437-440 [FREE Full text] [CrossRef] [Medline]
  4. Mitchell JW, Torres MB, Joe J, Danh T, Gass B, Horvath KJ. Formative work to develop a tailored HIV testing smartphone app for diverse, at-risk, HIV-negative men who have sex with men: A focus group study. JMIR Mhealth Uhealth 2016 Nov 16;4(4):e128 [FREE Full text] [CrossRef] [Medline]
  5. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 1989 Sep;13(3):319. [CrossRef]
  6. Portz JD, Bayliss EA, Bull S, Boxer RS, Bekelman DB, Gleason K, et al. Using the technology acceptance model to explore user experience, intent to use, and use behavior of a patient portal among older adults with multiple chronic conditions: Descriptive qualitative study. J Med Internet Res 2019 Apr 08;21(4):e11604 [FREE Full text] [CrossRef] [Medline]
  7. Hentati A, Forsell E, Ljótsson B, Kaldo V, Lindefors N, Kraepelien M. The effect of user interface on treatment engagement in a self-guided digital problem-solving intervention: A randomized controlled trial. Internet Interv 2021 Dec;26:100448 [FREE Full text] [CrossRef] [Medline]
  8. Deterding S, Sicart M, Nacke L, O'Hara K, Dixon D. Gamification: using game-design elements in non-gaming contexts. In: CHI '11 Extended Abstracts on Human Factors in Computing Systems. 2011 Presented at: Conference on Human Factors in Computing Systems; May 7-12, 2011; Vancouver, BC, Canada p. 2425-2428. [CrossRef]
  9. Sciascia S, Radin M, Cecchi I, Di Nunzio P, Buccarano N, Di Gregorio F, et al. Tailoring tofacitinib oral therapy in rheumatoid arthritis: The TuTORApp-A usability study. Int J Environ Res Public Health 2020 May 15;17(10):3469 [FREE Full text] [CrossRef] [Medline]
  10. Melnick ER, Hess EP, Guo G, Breslin M, Lopez K, Pavlo AJ, et al. Patient-centered decision support: Formative usability evaluation of integrated clinical decision support with a patient decision aid for minor head injury in the emergency department. J Med Internet Res 2017 May 19;19(5):e174 [FREE Full text] [CrossRef] [Medline]
  11. Hightow-Weidman L, Muessig K, Knudtson K, Srivatsa M, Lawrence E, LeGrand S, et al. A gamified smartphone app to support engagement in care and medication adherence for HIV-positive young men who have sex with men (AllyQuest): Development and pilot study. JMIR Public Health Surveill 2018 Apr 30;4(2):e34 [FREE Full text] [CrossRef] [Medline]
  12. Hightow-Weidman L, Muessig KE, Egger JR, Vecchio A, Platt A. Epic Allies: A gamified mobile app to improve engagement in HIV care and antiretroviral adherence among young men who have sex with men. AIDS Behav 2021 Aug 19;25(8):2599-2617. [CrossRef] [Medline]
  13. Andrade-Romo Z, Chavira-Razo L, Buzdugan R, Bertozzi E, Bautista-Arredondo S. Hot, horny and healthy-online intervention to incentivize HIV and sexually transmitted infections (STI) testing among young Mexican MSM: a feasibility study. Mhealth 2020 Jul;6:28 [FREE Full text] [CrossRef] [Medline]
  14. Plyuto A. Skeuomorph Mobile Banking. Dribbble.   URL: https://dribbble.com/shots/7994421-Skeuomorph-Mobile-Banking [accessed 2022-06-10]
  15. Sharma K. Neumorphism (Soft UI) in User interface design - Tutorial. UX Planet. 2019.   URL: https://uxplanet.org/neumorphism-in-user-interface-tutorial-c353698ac5c0
  16. Osinusi K. The New Trend in UI Design: An Overview of Neumorphism. Toptal.   URL: https://www.toptal.com/designers/ui/neumorphic-ui-design [accessed 2022-06-10]
  17. Kwan TH, Wong NS, Lui GCY, Chan KCW, Tsang OTY, Leung WS, et al. Incorporation of information diffusion model for enhancing analyses in HIV molecular surveillance. Emerg Microbes Infect 2020 Jan 30;9(1):256-262 [FREE Full text] [CrossRef] [Medline]
  18. Kwan TH, Lee SS. Predictors of HIV testing and their influence on PrEP acceptance in men who have sex with men: A cross-sectional study. AIDS Behav 2018 Apr 10;22(4):1150-1157. [CrossRef] [Medline]
  19. Kwan TH, Lee SS. Bridging awareness and acceptance of pre-exposure prophylaxis among men who have sex with men and the need for targeting chemsex and HIV testing: Cross-sectional survey. JMIR Public Health Surveill 2019 Jul 03;5(3):e13083 [FREE Full text] [CrossRef] [Medline]
  20. Brooke J. SUS: A 'Quick and Dirty' Usability Scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. Boca Raton, Florida: CRC Press; 1996.
  21. Sauro J, Lewis J. Standardized usability questionnaires. In: Quantifying the User Experience. Burlington, Massachusetts: Morgan Kaufmann; 2016:185-248.
  22. Lewis J, Sauro J. Item benchmarks for the system usability scale. Journal of Usability Studies 2018;13(3):158-167. [CrossRef]
  23. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 2008 Jul 30;24(6):574-594. [CrossRef]
  24. Bangor A, Kortun P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies 2009;4(3):114-123. [CrossRef]
  25. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [CrossRef] [Medline]
  26. Liao M, Wang M, Shen X, Huang P, Yang X, Hao L, et al. Bisexual Behaviors, HIV Knowledge, and Stigmatizing/Discriminatory Attitudes among Men Who Have Sex with Men. PLoS One 2015 Jun 29;10(6):e0130866 [FREE Full text] [CrossRef] [Medline]
  27. Kamal N, Fels S, Ho K. Online social networks for personal informatics to promote positive health behavior. In: WSM '10: Proceedings of the Second ACM SIGMM Workshop on Social Media. 2010 Presented at: Second ACM SIGMM Workshop on Social Media; October 25, 2010; Firenze, Italy p. 47-52. [CrossRef]
  28. Liu Y, Sun X, Qian H, Yin L, Yan Z, Wang L, et al. Qualitative assessment of barriers and facilitators of access to HIV testing among men who have sex with men in China. AIDS Patient Care STDS 2015 Sep;29(9):481-489 [FREE Full text] [CrossRef] [Medline]
  29. Nielsen J. Enhancing the explanatory power of usability heuristics. In: CHI '94: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1994 Presented at: SIGCHI Conference on Human Factors in Computing Systems; April 24-28, 1994; Boston, MA p. 152-158. [CrossRef]
  30. Aktivia R, Djatna T, Nurhadryani Y. Visual usability design for mobile application based on user personality. 2014 Presented at: International Conference on Advanced Computer Science and Information System; October 18-19, 2014; Jakarta, Indonesia. [CrossRef]
  31. Arambepola N, Munasinghe L. Empirical analysis of user factors that affect the user interface design in mobile applications. 2020 Presented at: 20th International Conference on Advances in ICT for Emerging Regions (ICTer); November 4-7, 2020; Colombo, Sri Lanka. [CrossRef]
  32. Quidley-Rodriguez N, De Santis JP. A literature review of health risks in the bear community, a gay subculture. Am J Mens Health 2017 Nov 30;11(6):1673-1679 [FREE Full text] [CrossRef] [Medline]
  33. McCoy SI, Buzdugan R, Grimball R, Natoli L, Mejia CM, Klausner JD, et al. Stick To It: pilot study results of an intervention using gamification to increase HIV screening among young men who have sex with men in California. Mhealth 2018;4:40 [FREE Full text] [CrossRef] [Medline]


CLAI: condomless anal intercourse
MSM: men who have sex with men
OR: odds ratio
PrEP: pre-exposure prophylaxis
SEQ: Single Ease Question
SUS: System Usability Scale
UI: user interface
UX: user experience


Edited by N Zary; submitted 22.12.21; peer-reviewed by V Verma, K Chigudu; comments to author 14.04.22; revised version received 21.04.22; accepted 13.05.22; published 22.06.22

Copyright

©Tsz Ho Kwan, Denise Pui Chung Chan, Shui Shan Lee. Originally published in JMIR Serious Games (https://games.jmir.org), 22.06.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.