Background: The success rate of tuberculosis (TB) treatment in Malaysia remains below the recommended World Health Organization target of 90% despite the implementation of directly observed therapy, short-course, a physical drug monitoring system, since 1994. With increasing numbers of patients with TB in Malaysia defaulting on treatment, exploring another method to improve TB treatment adherence is vital. The use of gamification and real-time elements via video-observed therapies in mobile apps is one such method expected to induce motivation toward TB treatment adherence.
Objective: This study aimed to document the process of designing, developing, and validating the gamification, motivation, and real-time elements in the Gamified Real-time Video Observed Therapies (GRVOTS) mobile app.
Methods: The modified nominal group technique via a panel of 11 experts was used to validate the presence of the gamification and motivation elements inside the app, which were assessed based on the percentage of agreement among the experts.
Results: The GRVOTS mobile app, which can be used by patients, supervisors, and administrators, was successfully developed. For validation purposes, the gamification and motivation features of the app were validated as they achieved a total mean percentage of agreement of 97.95% (SD 2.51%), which was significantly higher than the minimum agreement score of 70% (P<.001). Further, each component of gamification, motivation, and technology was also rated at 70% or more. Among the gamification elements, fun received the lowest scores, possibly because the nature of serious games does not prioritize the fun element and because the perception of fun varies by personality. The least popular element in motivation was relatedness, as stigma and discrimination hinder interaction features, such as leaderboards and chats, in the mobile app.
Conclusions: It has been validated that the GRVOTS mobile app contains gamification and motivation elements, which are intended to encourage medication adherence to TB treatment.
More than 10 million cases of tuberculosis (TB) are reported each year globally . In Malaysia, from 2010 to 2015, TB cases increased from 68.4 to 79.6 cases per 100,000 people [ ]. In addition, according to a model projection, the observed and projected TB incidence in Malaysia will reach 300,000 cases in 2030 [ ].
In Malaysia, directly observed therapy, short-course (DOTS) is a method to ensure medication compliance by having a trained health care worker or other designated individual provide the prescribed TB drugs and watch the patient take every dose. DOTS, which was implemented in 1994, has resulted in a treatment success rate from 76% in 2013 to 81% in 2017 . However, this rate has remained below the recommended World Health Organization target of 90%. In addition, there is an increase in the prevalence of TB treatment default in Malaysia, which has ranged from 4% in 2010 to 4.8% in 2015 and to 5.6% in the latest study [ ]. The increasing number of TB cases indicates that there are still issues and challenges that need to be addressed at all levels.
Specifically, the problems currently facing DOTS can be categorized according to the 3 main stakeholders. For patients with TB, the compulsory need for daily DOTS monitoring results in stigma by the public and absence from home or work responsibilities [- ]. Health workers and policy makers are hesitant to fully implement DOTS, as there are inadequate human resources, increased TB management costs, less participation from lower management, and a lack of public awareness [ , ].
To address these challenges, the World Health Organization recommends the use of digital technologies to promote TB medication adherence . Video directly observed therapy (VDOT) was introduced to replace physical DOTS and has proven to significantly reduce the cost of managing TB, improve patients’ access to doctors, and be less disruptive to patients’ work and family life [ , - ]. According to many studies, VDOT is a more cost-effective method that significantly increases patient treatment adherence compared to conventional DOTS [ - ].
Although data on VDOT are becoming increasingly robust, the system has yet to be rigorously evaluated within low- and middle-income countries, especially regarding its feasibility . This is quite worrying, as the implementation of VDOT requires complete access to hardware and internet connectivity, which some countries cannot afford [ ]. However, in Malaysia, the number of smartphone users is growing and expected to reach over 33 million by 2024, with an 87.36% smartphone penetration, which suggests that VDOT can be implemented in Malaysia [ ].
Making VDOT available via mobile apps could make drug monitoring more convenient and effective . However, with the large numbers of mobile health apps in existence, the problem is often about the sustainability of their use [ ]. As a solution, integrating gamification elements inside mobile apps can positively impact health and well-being, improve health behaviors and patient engagement, decrease health care use, and empower patients to self-manage their disease [ , ]. In addition, the use of real-time elements such as virtual reality and augmented reality has proven to increase learning effectiveness and behavior modification, correct medication identification, correct self-administration of medication, and support patient counseling practices [ - ].
Thus, in our Gamified Real-time Video Observed Therapy (GRVOTS) mobile app, the integration of gamification and real-time elements is expected to increase patient motivation. The purpose of this study was to validate the gamification, motivation, and real-time element in GRVOTS, a mobile app for VDOT, from the perspective of the service provider (expert review).
The mobile app prototype was developed from February 2021 to May 2022. The developmental process as a whole, including content and prototype development as well as content validation (nominal group technique [NGT]), was performed using the design science research process model .
Content development of the prototype involved a few stages of literature review, mapping, and justification of the new framework. From the literature review, we identified 3 frameworks that could be used as features of app: gamification framework, video reality and motivation framework, and technology feature framework. The gamification component, as the foundation of the proposed gamified mobile app GRVOTS, will be based on the validated framework for the gamification of diabetes self-management called The Wheel of Sukr . The framework consists of 8 components: fun, esteem, growth, motivation, sustainability, socializing, self-representation, and self-management. For the video reality and motivation framework, we used the attention, relevance, confidence, and satisfaction (ARCS) model of motivational design [ ]. In terms of the mapping procedure, the ARCS model of motivational design can be combined with the gamification elements to foster motivation [ ]. The dynamic nature of gamification, such as self-management, self-representation, and fun, can be equal to satisfaction in the ARCS model. Further, elements of gamification, such as esteem, reward, growth, and socializing, can be equal to the components of confidence in the same model. Subsequently, these 2 frameworks can be integrated where gamification elements are added to the categories of confidence and satisfaction that are based on the ARCS model. shows the categories and subcategories of the proposed model in matrix form.
Game dynamics can improve user desire and motivation by establishing rules that encourage users to explore and learn about the apps . shows a screenshot of the main function of the GRVOTS mobile app and its relation to our intended gamification, motivation, and real-time elements from the framework.
The GRVOTS mobile app is designed for 3 users—patients, supervisors, and administrators—where they interact with one another via the internet. All data inputted by the patients will be automatically collected by the server and viewed by the specific supervisor (health care worker) and TB management team to help them with clinical interventions. The proposed model presented inis based on the development of mobile apps for smartphones only.
|The Wheel of Sukr (gamification)||—b||—|
aARCS: attention, relevance, confidence, and satisfaction.
Content Validation via the NGT (Expert Review)
The NGT is a structured variation of a small-group discussion to reach consensus. Through the agreement of the description of the elements, the NGT was used in this research as a validation tool to evaluate the presence of the gamification and motivation components intended to be used in the app.
The NGT is a small-group technique suited to panel sizes of more than 10 people [, ]. Therefore, there were 11 experts involved in this NGT session. A panel of experts was involved to validate the gamification elements in the GRVOTS app using physical meetings in 3 different settings.
Study Population Flow
This study was conducted iteratively in 3 meetings for a more comprehensive evaluation, as illustrated in.
The criteria of the experts involved in the group were different according to each group. For the first group, IT experts were experienced and involved in mobile app development for at least 2 years and well versed in the gamification features of mobile apps. The second and third groups were composed of administrative and health care workers who were directly involved in managing patients with TB in the outpatient environment, respectively.
The 3 NGT meetings were performed in 3 settings: at the university for the meeting with the IT expert panel, at the district health office for the meeting session with the administrative team, and at the TB clinic for the meeting with clinical experts.
Instrument for NGT
The instrument used in the NGT was a questionnaire, and the items were generated from a literature review based on 3 existing models as per the previous mapping. These models were developed into a panel expert checklist, and the questionnaire had 2 parts: Part A asked about the gamification-motivation–rea-time theory, The Wheel of Sukr , and the ARCS model, whereas Part B concerned the technology features by Anderson et al [ ].
Implementation of NGT
The implementation of NGT involved experts who were selected according to the scope of the study. The workshop was conducted in a face-to-face meeting by a moderator . The NGT workshop lasted approximately 2 hours. Before the workshop, the experts were given a week to use the app. Some of them experienced the task as a supervisor and some as a patient. They were required to send 3-4 VDOT videos. shows the basic steps to carry out the NGT process, and shows the 5 steps of data analysis for the NGT.
The data analysis process for the NGT was based on the percentage of agreement where an element is accepted when the percentage of agreement is 70% or more . The 1-sample, 2-tailed t test was used to determine whether the mean (SD) percentage of agreement result on the gamification and motivation elements in this app was significantly higher than the percentage of agreement of 70%, with a level of significance (α error) less than .05. The software used at this stage was SPSS (version 28.0; IBM Corp) and Microsoft Excel.
|1. Introduction to problem statement and explanation||Moderator will brief the participants regarding the flow of the session.|
|2. Silent generation of ideas in writing||According to the checklist and GRVOTSa mobile app, participants were asked to answer the questionnaire using a Likert scale.|
|3. Round robin phase: sharing ideas||Participants were invited to share their answers in the round robin manner.|
|4. Discussion of ideas||Participants were asked to justify the need for the least prominent gamification and virtual reality elements in the GRVOTS mobile app prototype.|
|5. Voting and ranking||The voting was done by marking responses on a Likert scale from 1 (totally disagree) to 5 (totally agree). The calculation was done, and the elements were ranked accordingly.|
aGRVOTS: Gamified Real-time Video Observed Therapies.
|1||Ensuring the number of participants (experts) involved in the study|
|2||The formation and calculation of score value is based on the template data analysis of the NGT|
|3||Convert score values into percentage form to obtain the percentage of agreement:|
where A=the total number of experts and B=Likert scale used, ie, 5 points
|4||Determine the acceptance of components and elements based on the percentage of agreement|
|5||Determine the positions of the elements according to the percentage of agreement|
Ethical approval was obtained from the Universiti Teknologi MARA Ethical Board and Medical Research and Ethics Committee, Malaysia: NMRR-21-1016-58994 (IIR).
No informed consent was taken in this study as the data were only retrieved retrospectively from the database and no identifiers were collected for this study.
Mobile app development with the integration of gamification and real-time elements was performed via a literature review, and the content validation of the component was performed via the modified NGT.
The final beta version of the mobile app used Android as the platform. The prototype was designed to be used by 3 users—patients, supervisors, and TB managers—and each user had a different role in the program. This mobile app’s main function is to provide patients with a way to record daily DOTS intake via video (VDOT) as well as medication side effect reporting. After patients log in, they are directed to set up their personal profiles and learn how to use the VDOT medication reminder, which is the main activity. After each VDOT session is uploaded, it is followed by pop-up motivational quotes as well as the movement of the progress meter indicator.
The accumulated points collected from the progress meter will be translated into badges in the progress report theme. Subsequently, a daily pop-up message will also be the main reminder for the next medication, and patient will be asked if they noticed any adverse reactions to previous medication, with a selection of options concerning their symptoms. The ability to report adverse effects gives patients access to their own medication diaries, which can be reviewed during medical visits. Throughout the TB treatment journey, the progress report theme will help patients track their journey and redeem the internal and external rewards offered.
Every VDOT report and any side effects noted by patients will be verified by the supervisor as a feedback interaction. This feature enables supervisors to regularly check and understand the patients’ progress as well. In addition, patients can always go to other main theme of “knowledge” to continuously learn more about TB treatment and nutrition.
The app provides different users with access to different functions and main menus. In the patients’ main menu, there are 4 main themes for VDOT. Information regarding the app, progress report, and side effect reporting is shown in. For the supervisors’ main menu, there are 2 menus, one to validate the VDOT report and another to validate the side effects, as shown in .
NGT Output for Content Validation
A panel of 11 experts were involved in the modified NGT, including 2 gamification experts, 2 public health experts, 1 chest physician, 2 medical officers, 3 medical assistants, and 1 staff nurse. They were asked to rank the app’s elements based on the expert panel checklist.- show the results and rankings of gamification, motivation, and technology feature elements via the NGT.
The total percentage of agreement among experts was 97.95% (SD 2.51%), which was significantly higher than the minimum of 70%, with a difference of 27.947% (95% CI 26.74%-29.15%; P<.001). The t test confirmed the validity of all gamification and motivation components in the app.
|No||Main element||Explanation element||Percentage of agreement, mean (SD)||Acceptance result||Ranking|
|2||Self-representative (avatar)||97 (0.00)||Accepted||6|
|6||Sustainability (trigger)||97 (0.00)||Accepted||7|
aVDOT: video directly observed therapy.
|No||Main element||Explanation element||Percentage of agreement, mean (SD)||Acceptance result||Ranking|
|4||Satisfaction (achievement)||99 (1.361)||Accepted||2|
aVDOT: video directly observed therapy.
|No||Main elements||Explanation element||Percentage of agreement, mean (SD)||Acceptance result||Ranking|
|2||Ease of use (automation)||100 (0.00)||Accepted||1|
|3||Real-time feature||89 (0.00)||Accepted||7|
|4||Self-guided video||99 (1.361)||Accepted||3|
|7||Tactile feedback||98 (0.00)||Accepted||4|
aVDOT: video directly observed therapy.
This comprehensive GRVOTS mobile app was developed based on the integration of gamification and real-time motivational elements of autonomy, competence, relatedness, attention, relevance, self-control, confidence, and satisfaction. These components were successfully validated and significant.
Mobile App Developed
Our GRVOTS mobile app was able to motivate patients toward medication adherence using the ARCS model of motivation, which was translated via its gamification and motivation features. First, the interactive VDOT feature can stimulate the patient’s feelings of attention and arousal. Second, by linking the previous experience of medication intake physically with the experience of using GRVOTS, the element of “relevance” can also be instilled in the patient. When patients perceive a high sense of relatedness, they are more likely to exhibit higher engagement with a program . By providing goals along the journey toward medication adherence, the app can instill confidence and self-belief, especially when the patients can digitally see their previous VDOT report and a progress meter of treatment success. As they use GRVOTS to monitor their medication intake, patients experience more freedom of choice and self-control, and the results of treatment adherence will subsequently reinforce the app’s value. The last element of “satisfaction” can also be realized when patients comply with the use of GRVOTS and receive rewards in the form of badges and progress meters.
Content Validation via the Modified NGT
The results of the modified NGT showed that all the components obtained from the literature and related model were validated by the experts during the NGT session, as shown in- . Based on the results, the elements were prioritized based on the percentage of acceptance. Considering these rankings, some of the GRVOTS functions were improved to provide a better mobile app for the next pilot study.
According to the results for the gamification elements, the growth element received the highest scores, whereas fun received the lowest scores. By definition, “fun” aspects within the gamification elements in this GRVOTS app are exemplified by a progress bar, a progress meter, inspiration quotes, and badge rewards. The fun element was voted as a less visible element in this app, perhaps due to many factors, such as the perception of fun, which varies for different people, as well as the nature of serious games, which does not prioritize the fun element. A number of studies have described that the perception of gamification features differs according to the gamer type, gender, and personality of the player. For instance, extroverts like rewards and leaderboards, which appear to be more entertaining, but introverts prefer badges and feedback [- ].
Health apps are also considered serious games that are played for purposes other than pure entertainment . In this case, having a fun component is not a priority. According to a study regarding serious games, other elements, such as explicit learning tasks, instruction, and support built into the game or added by teachers, may be more important than having fun while playing [ ].
For the motivation elements of the ARCS model, the highest ranked element was confidence and the lowest was relevance. It is suggested that when patients perceive a high sense of relatedness, they are more likely to exhibit higher engagement with the program . In mobile apps, relatedness is seen as a feature of leaderboards or chats that encourages engagement and collaboration to achieve a particular objective [ ]. However, in this GRVOTS mobile app, the relevance element is evidenced by allowing daily communication between patients and supervisors via the VDOT and adverse event reporting components without the leaderboard or chat features. This is because most of our patients with TB refused to disclose their condition and communicate with other patients due to the stigma associated with the disease.
For the technology feature elements, the highest ranked was ease of use and the lowest was real-time features. There are 2 type of VDOT: live VDOT, also known as synchronous VDOT, in which patients and providers interact in real time [, ]; as well as asynchronous technologies that record, upload, and digitally store videos for future review [ , , ]. Synchronous VDOT has the advantage of human interaction but is not a feasible option, as patients and workers need to find time to meet. Asynchronous VDOT is more flexible but can be manipulated easily by sending the same recorded video. Thus, apps with real-time features ensure the originality of the video and simultaneously generate a greater degree of user engagement [ - ]. Although the real-time feature in our app provides details concerning the time and date of the video, feedback from the experts indicated that we should time stamp the videos so that they can be identified more easily later. Since users found it challenging to determine if their VDOT session was successfully uploaded or not, the addition of an upload bar was also recommended.
Strengths and Limitations
The GRVOTS mobile app can benefit users in many ways. For patients with TB, the app can help patients gain self-control, boost their self-esteem, and motivate them to take their medications. For health care workers, this app made it easier for TB system management to detect DOTS defaulters and manage them accordingly. As a portable device, mobile apps enable monitoring that can be done anywhere, saving money and time and boosting patient engagement with the DOTS program. In summary, this app can also initiate a patient self-care system and reduce dependency on health care providers such as doctors and nurses. The limitation of this study is that the mobile app prototype is only being developed for the Android platform because of time and logistics. The app does not have a virtual reality feature, and improvements are needed in the future. This study conducted the validation of the gamification and motivation elements only from the perspective of the experts and not from the perspective of patients, which may limit the review’s validity; an analysis regarding its usability among patients will be conducted in the future.
In the future, GRVOTS should also be available on other platforms, especially the iPhone Operating System. The language options should also include English, Chinese, and Tamil, as these languages are frequently used in multiethnic communities in Malaysia. The use of various languages will expand the benefits to more users. This will enhance knowledge transfer and improve users’ understanding. Next, a usability study to access the user experience will be conducted, followed by an effectiveness study via a single-arm intervention study, in which patients will use the app for DOTS during intensive phase up to 2 months monitoring, followed by an assessment at 3 time intervals to evaluate their medication adherence, motivation, and the usability of the app.
More comprehensive and efficient TB system management via VDOT mobile app monitoring is a way to improve patient treatment adherence. According to the literature review, gamification elements can motivate patients; thus, by integrating the uniqueness of gamification and motivation elements in an app, gamification will increase patient motivation, ensure the sustainability of use, and ultimately increase patient adherence. In addition, our GRVOTS mobile app connects up to 3 users (eg, patients, supervisors, and administrators) remotely and enable DOTS monitoring to be performed from anywhere. Based on the study findings, the GRVOTS mobile app has been validated by the expert panel as having the intended elements of gamification, real time, and motivation. Next, a usability study of the GRVOTS mobile app will be conducted to measure the user experience among patients, followed by a single-arm intervention study to assess the app’s effectiveness in increasing patient motivation and medication adherence in TB treatment.
The author would like to thank to the Director General of Health Malaysia for his permission to publish this study. We give many thanks also to the Universiti Teknologi MARA.
This study is funded by the Joint TDR/World Health Organization Western Pacific Region Small Grant Scheme for Implementation Research.
All data supporting the study findings are within the manuscript. Additional details information and raw data are available from the corresponding author on reasonable request.
SAA and NI conceived the project. SAA contributed to data collection. SAA, NI, and YZ contributed to data analysis. SAA, II, and NHMZ contributed to mapping the apps architecture. NI supervised the study. The manuscript was written mainly by SAA with the approval of all authors. All authors read and approved the final version of manuscript for publication.
Conflicts of Interest
- Global tuberculosis report 2020. World Health Organization. 2020 Oct 15. URL: https://www.who.int/publications/i/item/9789240013131 [accessed 2023-02-28]
- National strategic plan for tuberculosis control (2016-2020). Ministry of Health Malaysia. 2016. URL: https://tinyurl.com/msn622a3 [accessed 2023-02-28]
- Ismail N. Mathematical modelling of tuberculosis transmission and impact of isoniazid preventive therapy in Malaysia [thesis]. University of Malaya. 2017. URL: http://studentsrepo.um.edu.my/10364/4/nurhuda.pdf [accessed 2023-02-28]
- Liew SM, Khoo EM, Ho BK, Lee YK, Mimi O, Fazlina MY, et al. Tuberculosis in Malaysia: predictors of treatment outcomes in a national registry. Int J Tuberc Lung Dis 2015 Jul 01;19(7):764-771. [CrossRef] [Medline]
- Karumbi J, Garner P. Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2015 May 29;2015(5):CD003343 [FREE Full text] [CrossRef] [Medline]
- Zhang H, Ehiri J, Yang H, Tang S, Li Y. Impact of community-based DOT on tuberculosis treatment outcomes: a systematic review and meta-analysis. PLoS One 2016 Feb 5;11(2):e0147744 [FREE Full text] [CrossRef] [Medline]
- Sharani ZZ, Ismail N, Yasin SM, Zakaria Y, Razali A, Demong NAR, et al. Characteristics and determinants of loss to follow-up among tuberculosis (TB) patients who smoke in an industrial state of Malaysia: a registry-based study of the years 2013-2017. BMC Public Health 2022 Apr 01;22(1):638 [FREE Full text] [CrossRef] [Medline]
- Garfein RS, Collins K, Muñoz F, Moser K, Cerecer-Callu P, Raab F, et al. Feasibility of tuberculosis treatment monitoring by video directly observed therapy: a binational pilot study. Int J Tuberc Lung Dis 2015 Sep 01;19(9):1057-1064 [FREE Full text] [CrossRef] [Medline]
- Rahman NHA, Mokhtar KS. Challenges of national TB control program implementation: the Malaysian experience. Procedia - Social and Behavioral Sciences 2015 Jan;172:578-584. [CrossRef]
- Handbook for the use of digital technologies to support tuberculosis medication adherence. World Health Organization. 2017. URL: https://apps.who.int/iris/handle/10665/259832 [accessed 2023-02-28]
- Hoffman JA, Cunningham JR, Suleh AJ, Sundsmo A, Dekker D, Vago F, et al. Mobile direct observation treatment for tuberculosis patients: a technical feasibility pilot using mobile phones in Nairobi, Kenya. Am J Prev Med 2010 Jul;39(1):78-80. [CrossRef] [Medline]
- Chuck C, Robinson E, Macaraig M, Alexander M, Burzynski J. Enhancing management of tuberculosis treatment with video directly observed therapy in New York City. Int J Tuberc Lung Dis 2016 May 01;20(5):588-593. [CrossRef] [Medline]
- Choun K, Achanta S, Naik B, Tripathy JP, Thai S, Lorent N, et al. Using mobile phones to ensure that referred tuberculosis patients reach their treatment facilities: a call that makes a difference. BMC Health Serv Res 2017 Aug 22;17(1):575 [FREE Full text] [CrossRef] [Medline]
- Beeler Asay GR, Lam CK, Stewart B, Mangan JM, Romo L, Marks SM, et al. Cost of tuberculosis therapy directly observed on video for health departments and patients in New York City; San Francisco, California; and Rhode Island (2017-2018). Am J Public Health 2020 Nov;110(11):1696-1703. [CrossRef] [Medline]
- Guo P, Qiao W, Sun Y, Liu F, Wang C. Telemedicine technologies and tuberculosis management: a randomized controlled trial. Telemed J E Health 2020 Sep 01;26(9):1150-1156. [CrossRef] [Medline]
- Ravenscroft L, Kettle S, Persian R, Ruda S, Severin L, Doltu S, et al. Video-observed therapy and medication adherence for tuberculosis patients: randomised controlled trial in Moldova. Eur Respir J 2020 Aug 07;56(2):2000493 [FREE Full text] [CrossRef] [Medline]
- Story A, Aldridge RW, Smith CM, Garber E, Hall J, Ferenando G, et al. Smartphone-enabled video-observed versus directly observed treatment for tuberculosis: a multicentre, analyst-blinded, randomised, controlled superiority trial. Lancet 2019 Mar 23;393(10177):1216-1224 [FREE Full text] [CrossRef] [Medline]
- Truong CB, Tanni KA, Qian J. Video-observed therapy versus directly observed therapy in patients with tuberculosis. Am J Prev Med 2022 Mar;62(3):450-458. [CrossRef] [Medline]
- Holzman SB, Atre S, Sahasrabudhe T, Ambike S, Jagtap D, Sayyad Y, et al. Use of smartphone-based video directly observed therapy (vDOT) in tuberculosis care: single-arm, prospective feasibility study. JMIR Form Res 2019 Aug 27;3(3):e13411 [FREE Full text] [CrossRef] [Medline]
- Number of smartphone users in Malaysia from 2010 to 2020 and a forecast up to 2025. Statista. URL: https://www.statista.com/statistics/494587/smartphone-users-in-malaysia/ [accessed 2023-02-28]
- Stoner MCD, Maragh-Bass AC, Sukhija-Cohen AC, Saberi P. Digital directly observed therapy to monitor adherence to medications: a scoping review. HIV Res Clin Pract 2022 Jul 18;23(1):47-60. [Medline]
- Peng W, Kanthawala S, Yuan S, Hussain SA. A qualitative study of user perceptions of mobile health apps. BMC Public Health 2016 Nov 14;16(1):1158 [FREE Full text] [CrossRef] [Medline]
- Miller AS, Cafazzo JA, Seto E. A game plan: gamification design principles in mHealth applications for chronic disease management. Health Informatics J 2016 Jun 01;22(2):184-193 [FREE Full text] [CrossRef] [Medline]
- Johnson D, Deterding S, Kuhn KA, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv 2016 Nov;6:89-106 [FREE Full text] [CrossRef] [Medline]
- Ventola CL. Virtual reality in pharmacy: opportunities for clinical, research, and educational applications. P T 2019 May;44(5):267-276 [FREE Full text] [Medline]
- Nuffer W, Smith SM, Trinkley K. Integrating virtual reality to expand the public health role of community pharmacists. J Am Pharm Assoc (2003) 2013;53(2):152-158. [CrossRef] [Medline]
- Khan A, Khusro S. Smart assist: smartphone-based drug compliance for elderly people and people with special needs. In: Khan F, Jan M, Alam M, editors. Applications of Intelligent Technologies in Healthcare. Cham, Switzerland: Springer; 2019:99-108.
- Peffers K, Tuunanen T, Gengler CE, Rossi M, Hui W, Virtanen V, et al. Design science research process: a model for producing and presenting information systems research. arXiv. Preprint posted online on June 4, 2020. [CrossRef]
- AlMarshedi A, Wills G, Ranchhod A. Guidelines for the gamification of self-management of chronic illnesses: multimethod study. JMIR Serious Games 2017 May 12;5(2):e12 [FREE Full text] [CrossRef] [Medline]
- Keller JM. Motivational Design for Learning and Performance. New York, NY: Springer; 2010.
- Hamzah WMAFW, Ali NH, Mohd Saman MY, Yusoff MH, Yacob A. Enhancement of the ARCS model for gamification of learning. 2014 Presented at: 2014 3rd International Conference on User Science and Engineering (i-USEr); September 2-5, 2014; Shah Alam, Malaysia p. 287-291. [CrossRef]
- Alsawaier RS. The effect of gamification on motivation and engagement. International Journal of Information and Learning Technology 2018 Jan 02;35(1):56-79. [CrossRef]
- Cantrill JA, Sibbald B, Buetow S. The Delphi and nominal group technique in health services research. Int J Pharm Pract 1996 Jun;4(2):67-74. [CrossRef]
- Yahaya N, Rasul MS, Mohamed Yasin R, Sulaiman M. Nominal group technique application towards the formation of social skills and values elements for apprenticeship competency models. Journal of Technical Education and Training 2020 Jan 19;12(1):38-48 [FREE Full text]
- AlMarshedi A, Wills GB, Ranchhod A. The Wheel of Sukr: a framework for gamifying diabetes self-management in Saudi Arabia. Procedia Comput Sci 2015;63:475-480. [CrossRef]
- Anderson K, Burford O, Emmerton L. Mobile health apps to facilitate self-care: a qualitative study of user experiences. PLoS One 2016 May 23;11(5):e0156164 [FREE Full text] [CrossRef] [Medline]
- Deslandes SF, Mendes CHF, Pires TDO, Campos DDS. Use of the nominal group technique and the Delphi method to draw up evaluation indicators for strategies to deal with violence against children and adolescents in Brazil. Rev Bras Saude Mater Infant 2010 Nov;10(suppl 1):s29-s37. [CrossRef]
- Kerner C, Goodyear VA. The motivational impact of wearable healthy lifestyle technologies: a self-determination perspective on Fitbits with adolescents. Am J Health Educ 2017 Jul 28;48(5):287-297. [CrossRef]
- von Janta Lipinski A, Weber H, Kölle R, Mandl T. Gamification elements and their perceptions by different gamer types: a case study for a project management software. In: Everything Changes, Everything Stays the Same? Understanding Information Spaces. 2016 Presented at: 15th International Symposium of Information Science (ISI 2017); March 13-15, 2017; Berlin, Germany p. 131-144 URL: https://edoc.hu-berlin.de/bitstream/handle/18452/2103/lipinski.pdf
- Codish D, Ravid G. Personality based gamification: how different personalities percive gamification. Research Gate. 2014 Jan. URL: https://www.researchgate.net/publication/286518104_Personality_based_gamification_How_different_personalities_percive_gamification [accessed 2023-02-28]
- Denden M, Tlili A, Essalmi F, Jemni M, Chen N, Burgos D. Effects of gender and personality differences on students’ perception of game design elements in educational gamification. Int J Hum Comput Stud 2021 Oct;154:102674. [CrossRef]
- Wattanasoontorn V, Boada I, García R, Sbert M. Serious games for health. Entertain Comput 2013 Dec;4(4):231-247. [CrossRef]
- Iten N, Petko D. Learning with serious games: is fun playing the game a predictor of learning success? Br J Educ Technol 2014 Nov 11;47(1):151-163. [CrossRef]
- Mirsaeidi M, Farshidpour M, Banks-Tripp D, Hashmi S, Kujoth C, Schraufnagel D. Video directly observed therapy for treatment of tuberculosis is patient-oriented and cost-effective. Eur Respir J 2015 Sep 18;46(3):871-874 [FREE Full text] [CrossRef] [Medline]
- Garfein RS, Liu L, Cuevas-Mota J, Collins K, Muñoz F, Catanzaro DG, et al. Tuberculosis treatment monitoring by video directly observed therapy in 5 health districts, California, USA. Emerg Infect Dis 2018 Oct;24(10):1806-1815 [FREE Full text] [CrossRef] [Medline]
|ARCS: attention, relevance, confidence, and satisfaction|
|DOTS: directly observed therapy, short-course|
|GRVOTS: Gamified Real-time Video Observed Therapies|
|NGT: nominal group technique|
|VDOT: video directly observed therapy|
Edited by G Eysenbach; submitted 28.09.22; peer-reviewed by Y Jeem, T Turk; comments to author 20.12.22; revised version received 20.01.23; accepted 23.01.23; published 07.03.23Copyright
©Siti Aishah Abas, Nurhuda Ismail, Yuslina Zakaria, Ismassabah Ismail, Nurul Hidayah Mat Zain, Siti Munira Yasin, Khalid Ibrahim, Asmah Razali, Mas Ahmad Sherzkawee Mohd Yusoff, Norliza Ahmad, Thilaka Chinnayah. Originally published in JMIR Serious Games (https://games.jmir.org), 07.03.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.