This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.
Because of the COVID-19 pandemic, a doctoral-level public health qualitative research methods course was moved to a web-based format. One module originally required students to conduct in-person observations within the community, but the curriculum was adapted using a web-based video game exercise.
This study sought to evaluate students’ perceptions of this adaptation and determine whether the new pilot format successfully met the module’s original learning objectives.
Recorded footage of a video game session was used for students to observe, take field notes, and compare the results. Qualitative methods were used to evaluate student feedback on the curriculum and determine whether the original learning objectives were met. Data were analyzed using a directed content analysis.
The findings demonstrate that all the learning objectives of this adapted qualitative observational research assignment using a web-based video game exercise were successfully met; namely, the students learned how to compare and contrast the observational notes of peers and to evaluate how personal bias and environmental factors can affect qualitative data collection. The assignment was also positively received by the students.
The results align with the constructivist learning theory and other successful COVID-19 implementations. Our study demonstrates that the learning objectives of a qualitative observational assignment can be addressed given that there are proper forethought and delivery when the assignment is adapted to a web-based context using a video game exercise.
On March 11, 2020, because of the COVID-19 pandemic, the University of Florida (UF) temporarily adapted all in-person courses to a web-based format. By mid-March, all classes were permanently moved to the web for the remainder of the spring semester and continued to be held this way throughout the summer semester. Courses traditionally held in person were mandated to quickly adapt to a web-based format. A brief COVID-19 timeline for UF is presented in
Brief University of Florida COVID-19 timeline. FL: Florida; UF: University of Florida.
Good observation, interviewing, and communication skills are crucial for producing high-quality data for qualitative research that are reliable, trustworthy, and rigorous. These skills require a great amount of interpersonal ingenuity and are best modeled to students through in-person synchronous courses [
Teaching practices in web-based qualitative research methods have become an important topic of inquiry as opportunities for web-based and distance education increase [
There is currently minimal direction regarding the adaptation of qualitative courses to web-based formats despite qualitative research methods comprising a core part of the curriculum for doctoral programs [
Innovative curriculums have been created using pedagogical models designed to better incorporate technology and media into the classroom, such as the ASSURE (Assure Learners, State Standards & Objectives, Select Strategies, Technology, Media & Materials, Utilize Technology, Media & Materials, Require Learner Participation, and Evaluate & Revise) model and Four Component Instructional Design model [
Developing web-based qualitative courses focuses on maintaining the same rigorous approaches and strategies as those followed in traditional in-person courses [
Researchers have shown that video games can facilitate significant learning and can be a useful pedagogical tool for instructors and educators in various academic settings [
In other research settings, instructional videos for teaching in higher education have also been shown to be an effective content delivery and pedagogical tool and cost-effective [
Modifying or adapting video games for educational purposes is a more recent phenomenon in the educational setting, one that continues to gain traction given that it is less expensive and resource intensive and provides instructors with a game environment that can be controlled and modified to fit the course and learning needs to students. It is important to expand the knowledge base of instructional technologies as pedagogical tools in higher education, including the modification or adaptation of video games such as video game recordings. This is especially valuable for fields that have not garnered much research attention but are central to many curriculums.
This study does not examine a qualitative research methods course in its entirety, as it was not developed for web-based learning. Instead, our pilot study examines how a traditional qualitative assignment that could have been assigned in person or on the web but would have been impossible to complete in person due to the COVID-19 pandemic. After considering the COVID-19 pandemic–related constraints, a web-based version of the assignment was developed using a recorded video gaming session to cultivate the competencies that the students would have acquired had the assignment been in person. The objectives of this study were to explore whether the pilot adaptation successfully met the learning objectives of the module and to explore students’ perceptions of the adapted assignment. Perceptions have been operationally defined as the students’ understandings and interpretations of the COVID-19–adapted assignment. Overall, this study hopes to add to the body of literature to support other instructors in adapting their qualitative courses to a web-based format.
This study aimed to answer two research questions: (1) How were the learning objectives met after the assignment was adapted during the COVID-19 pandemic? and (2) What were the students’ perceptions of the assignment?
Data were collected from a doctoral-level qualitative research methods course that included social and behavioral science public health students and rehabilitation science students. Due to the course being an upper-level doctoral course, student registration was limited to allow for deep discussions and engagements. A total of 7 students were enrolled in the study. Data were collected from an observational research assignment and students’ overall course feedback. Qualitative methods were used to evaluate students’ (N=7) course feedback and determine whether the learning objectives were met. Data were analyzed using a directed content analysis [
The research process and methodological decision-making.
We obtained institutional review board (IRB) approval from the UF IRB (IRB202002357). Students completed course evaluations and an evaluation of this assignment while they were enrolled in the course, and there was no research intent to use the data. Once the course was completed, the instructor applied for IRB approval to use feedback in these evaluations. Our study was approved as exempt, and informed consent was not required.
A content analysis was chosen, as we sought to “provide knowledge and understanding of the phenomenon under study” [
Quantitative research measures such as reliability and validity are not used for qualitative studies. Rather, qualitative research relies on the concept of “trustworthiness” instead of reliability and validity [
Triangulation is the use of “multiple methods or data sources in qualitative research to develop a comprehensive understanding of a phenomena” [
During the traditional, face-to-face qualitative research methods course, students learned about different qualitative research techniques (eg, observation, interviews, and focus groups). Observation research is a qualitative technique in which researchers observe and record participants’ ongoing behavior in a natural situation. A central method used in observation research is creating observational field notes, which are descriptive, reflective, and evaluative information documented by the researcher to record observations [
In the pre–COVID-19 pandemic assignment, to teach students about observational research and personal bias, they were asked to pair up with another student, go out into the community, and observe humans in their natural environment for a fixed period, for example, going to a coffee shop and observing the patrons in the coffee shop or focusing on a specific subset of people in that coffee shop (ie, a couple on a date, a mother and child, etc). Students were asked to observe and record observations separately and then pair up to compare the field notes.
Due to the COVID-19 pandemic and state restrictions, the face-to-face assignment described above was impossible to complete and needed to be adapted to the current circumstances (
This game was chosen for the adapted assignment because it encourages exploration and careful observation. Furthermore, the game environment allowed the instructor to create highly controlled and modifiable content for the course that mirrored the face-to-face assignment in several ways. First, the observations made while watching the prerecorded footage were similar to the observations students would make while in a coffee shop. Second, both assignments required students to take observational field notes of their respective settings. To elicit similar observational experiences among the students, they were instructed to view and take observation notes on the video recording without pausing the video. This instruction was given to illicit what a live or real-time observation would feel like while taking observation notes. Finally, both assignments involved students using environmental clues and dialogue to note what they observed. The instructor had familiarity with
The students were separated into groups of 2 or 3 and asked to observe the main character in the story and how they interacted with the world while taking field notes. The students were instructed to watch the video on their own and answer several questions about what they thought was happening in it. After doing this, the students met with their partners on the web and were asked to compare their approaches to observation and taking field notes and understand how their accounts overlapped and varied. After working with their partners, they were asked to reflect on how their personal biases and current events affected their perceptions of the events in the video. The course assignment is provided in
At the end of the semester, the students were asked to complete an end-of-course open-ended survey and reflect on which assignment, discussion, or activity they enjoyed the most and discuss why they felt that way. The students were also asked to reflect on the aspects of the course they did not enjoy and discuss why they felt that way. This end-of-course survey was similar to the one used in another study that investigated approaches to teaching web-based qualitative research methods [
A directed content analysis was used to investigate whether the modified assignment met the predetermined learning objectives and to understand students’ perceptions of the assignment. Directed content analysis is 1 of the 3 approaches used in qualitative content analysis and is used when the study’s frame provides initial coding categories for the study [
Learning objective 1: compare and contrast the observational field notes of peers who observed similar or the same phenomena.
Learning objective 2: evaluate how personal bias can affect qualitative data collection.
Learning objective 3: evaluate how environmental factors can affect qualitative data collection.
Our directed content analysis resulted in 3 major themes directly related to the learning objectives. Subthemes were also identified during the analysis. The themes and subthemes are listed in
Themes and subthemes from the directed content analysis.
Initial themes | Frequency, n |
Learning objective 1: compare and contrast the observational field notes of peers who observed similar or the same phenomena | 26 |
Subtheme 1.1: the web-based observational experiencea | 13 |
Learning objective 2: evaluate how personal bias can affect qualitative data collection | 20 |
Subtheme 2.1: understanding one’s role in the research processb | 6 |
Learning objective 3: evaluate how environmental factors can affect qualitative data collection | 16 |
aOperational definition: interpretation and feedback from the students after the completion of the observational experience.
bOperational definition: responses that helped the students identify their role as a researcher and challenged their assumptions as they worked through the observational research assignment.
There were 26 responses addressing learning objective 1. To accomplish this learning objective, the students observed the video game and documented field notes alone and then compared their field notes with those of a peer. After working in pairs, their observations were posted on the web on the designated e-learning discussion board, so they could compare their findings with those of other groups. By doing so, the students demonstrated an understanding of how people can observe and interpret a phenomenon in similar and different ways depending on various factors. For example, the excerpt below depicts how the perspectives of 2 students on environmental cues can overlap with one another yet also provide a new perspective, even when the assignment variables were held constant:
Accounts overlap in that we were very descriptive of environmental cues, car descriptions, etc. While another student was more narrative in [their] notes and was able to capture the audio conversations more than Student 1. Student 1 accounted more for situations surrounding the houses to account for the physical environment. Student 1 noted more subtle indications of location versus Student 6 noted final, solid indicators at the end.
It is definitely interesting to see the different school of thoughts on exposure to the same video. Just as Student 1, I could only take short bullet point notes at a time and did not elaborate as much when typing. It is also interesting that Student 6 associated the event with rapture as I also had that as a possibility of what happened in the town.
Students also discussed their enjoyment of the observation activity and the benefits of such an activity in a qualitative course. They emphasized the importance of recognizing that people can interpret similar events or situations in various ways:
I also enjoyed this activity as it was beneficial to understand how others interpreted the same video. The discussion solidified the differences between groups and individuals. Lastly, it reinforced the value of a team-based approach and having mechanisms in place to record your data to increase rigor.
Student 2 and I were paired up for the observational fieldnotes assignment, which I enjoyed because it was this common stimulus and we got to experience how two researchers might approach the same dataset with different takeaways, each influenced by our experiences and biases.
This recognition is important for qualitative researchers to ensure reliability and rigor throughout the qualitative process. Furthermore, the students discussed how the exercise of systematically comparing and contrasting their notes with those of a peer was helpful and broadened their perspectives on qualitative observational research:
We thought about participating in research on a team; even for this simple project, it was helpful to talk about what we each interpreted from the scene as that led to richer ideas and discussions. Observational notes are unlikely to be reasonably separated from the perspectives of the observer; thus, the recursive process of qualitative can help here as you can refine your ideas over time with perspectives from other researchers and from the literature.
Overall, the comments indicated that the students perceived collaborating with peers while completing the observational research as a valuable experience, which indicated that the learning objective of comparing and contrasting the field notes of peers who observed a similar phenomenon was met.
A subtheme emerged from learning objective 1 and was termed “web-based observation experience” (
If observing in real time, the observer may miss things when stopping to take notes; the perspective of making observations limits to what you can see, hear.
The above comment could apply to a real-world observation or a web-based observation, whereas the following comments relate more to the ability to replay web-based observations multiple times:
The strategy that I used was pretty similar to yours. I tried to note the key points by jotting down notes in brief then going back later and putting them together in coherent statements. My strategy differed from yours because I treated the video as a recording that I could replay (I watched it twice and I used the pause command the second time through to allow time to jot down detailed notes like the time on the clock tower for example).
Recording would be very helpful in research involving observations. This would give the opportunity to observe multiple times and not have to focus on taking detailed notes in the first viewing.
Both examples relate to how note-taking, whether performed while observing in person or on the web, may not allow a researcher to capture the full observation if the ability to record is not available. As student 4 stated, you “need more than one perspective present to compare and contrast what you observed.” Overall, this demonstrated that students developed a deeper understanding of the strengths and limitations of observational research, especially those related to learning objective 1.
The qualitative analysis of learning objective 2 yielded 20 responses related to personal bias affecting qualitative data collection (
I think my personal bias affected my observations in ways I did not know until my discussion with Student 6. They saw an element of religion and supernatural within the story, predominantly the role and symbolism of the orb. I am not a very religious individual, therefore, I relied on my public health background, which made me omit or neglect any signs of religion. I was only made aware of religion at all when scripture was recited to Jeremy [character in the game]. I think a very obvious bias was my assumption that he was a dad versus a religious leader. I think my background of science and rigor made it hard for me to be creative and open to a fictional situation, so I was trying to make connections within my reality, not the reality of the game.
Several students also commented on their prior experience or lack of prior experience with video games and how that may have impacted the data collection:
I don’t have much exposure to modern video games, so it’s interesting to hear how experience with this type of “world” seems very helpful in navigating/interpreting a video like this. Coupled with Student 2’s discussion of [their] electronics background and how it helped them interpret the video, I think this is a great illustration of how our own experiences and perceptions can influence our observations and interpretations.
Many also commented on how their experiences with movies or books could have also influenced the data collection process:
I think horror movies, sci-fi movies (you mentioned you are reading a sci-fi book), and video games made me think aliens or something supernatural. One aspect that really connected me to aliens was actually the choir voices that were heard throughout [background music within the game]. In the video game “Halo,” a similar ominous choir sound is heard, and it involves aliens. This exercise was a great example to demonstrate how our own personal bias can construe how we observe the world, for me, I think that went along with horror movies (specifically, zombie movies) where characters just need to survive in post-apocalyptic world. Exploring how your backgrounds/personal biases may have impacted how you viewed the video!
I realized my thoughts on what had happened were influenced by my love for sci-fi and horror series and movies. Similar events tend to be portrayed in such movies and I was quick to form theories or assumptions that were highly correlated with the content I had previously seen.
The students demonstrated their ability to critically evaluate how personal biases, such as backgrounds and experiences, affected their qualitative data collection process. Based on their personal biases, the students processed how their approach to data collection varied and how this variation helped their overall comprehension of personal bias in research.
The subtheme understanding one’s role in the research process emerged from our analysis under learning objective 2. This subtheme was operationalized as responses that helped the students identify their role as a researcher and challenge their assumptions as they worked through the assignment (
Several students discussed how they struggled to determine what their role was within this assignment:
We made assumptions at the beginning that we eventually changed, which is seen throughout our notetaking. We struggled to identify who we were in this story, so we continually checked ourselves and our role. Are we part of the story or are we an external observer? We also realized the time-extensive nature of data collection and we only feel like we have a piece of the story. The notes left us with more questions than solutions, which may parallel how research questions are developed.
Student 7 also discussed how they struggled to determine their role in the game:
Like Student 1 mentions, I initially assumed I was Jeremy [character in the game], once it became clear I wasn’t Jeremy, I had the understanding that we are a first-person observer, as you describe. However, even as Jeremy I felt that I was trying to figure out what was happening/had happened in the village. While much of my interpretations of the light and its effect were similar to what you (Student 5) and Student 2 describe, it didn’t occur to me that the light could also be functioning as a guide for the observer.
For our third learning objective, the directed content analysis yielded 16 student responses. Based on their own experiences and other students’ discussions of their respective experiences, the students contemplated whether certain environmental factors affected their data collection process and how this occurred during the observational assignment. Several students discussed how the COVID-19 pandemic directly or indirectly affected their perceptions of this assignment:
Seeing the quarantine signs immediately triggered a parallel to today’s pandemic. My mind went to a virus outbreak and maintained that position through the entire narrative. I think this kept me from deviating to religious or supernatural explanations. Though the pandemic was subtlety influencing my perceptions, I think my personal biases and experiences were more dominant in influence. I was able to make connection such as closed businesses and required quarantines, but the differences in symptoms and apocalyptic state of the community was different enough for me to separate today’s reality from the virtual one.
I definitely think being in the middle of the COVID-19 pandemic impacted my perception of this video; I think this may have heightened my sensitivity to the term “quarantine” and its possible role in the events in the village, particularly in conjunction with the quarantine signs. I initially made the connection between the term quarantine and illness, and I assumed something had “infected” the individuals living in the village. This doesn’t necessarily align with all of the other observations in this video…but it was consistently in the back of my mind while I watched the video and tried to piece together what might have happened.
The ongoing pandemic made it easier for me to conclude that some form of infection spread through the town and led to people quarantining. The town had a number of quarantine signs posted on doors. It was easier to reach this conclusion after experiencing the implications of the emergence of COVID 19, unlike if I had not experienced this.
Student 3 indicated an indirect effect of environmental factors on their data collection experience. At first, they did not think that the COVID-19 pandemic influenced their data collection. However, upon discussing and evaluating other students’ responses to environmental factors, student 3 noted that they were “sensitive to the word ‘quarantine’” throughout the assignment.
In addition to environmental factors affecting qualitative data collection experiences, the students also expressed that relevant environmental factors (ie, the COVID-19 pandemic) impacted their perceptions of the observational assignment:
I think that events surrounding COVID-19 affected the way I perceived what was going on in the video because I felt that the facilitators of this course would piggyback on current events to illustrate a point with regard to this module of instruction. It was a valuable experience in terms of recognizing subjective biases and controlling for them to enhance the likelihood of an objective exploratory exercise.
Students exhibited their abilities to evaluate the influence their immediate environments might have on qualitative data collection. This was especially apparent when students discussed how the COVID-19 pandemic directly affected what they paid attention to in their web-based surroundings, which environmental cues most impacted their interpretations, and how they perceived what was happening through documented observational field notes.
Due to UF’s mandated transition to web-based learning (
Our findings demonstrated that the students learned how to compare and contrast their observational notes with those of their peers who observed the same or similar phenomena through the video game assignment. Students were able to compare not only note-taking styles but also the level of detail recorded. As the students viewed the same prerecorded video game scenes, it was easy for them to identify how their observations were similar and how they differed. The students reported noticing key differences among observers and identifying the importance of comparing and contrasting field notes, showing that learning objective 1 was met through the web-based assignment.
This activity also reinforced the foundational concept of analyzing qualitative data using a team-based approach. This team-based approach begins to teach students the importance of triangulation in qualitative research, which helps ensure the validity of the results [
The students’ observations and notes were not just contained within pairs but also displayed on a discussion board so that the students could compare them among their classmates. This allowed the students to experience data source triangulation, which occurs when data are collected from different individuals with varying experiences and beliefs to gain multiple perspectives [
The findings also demonstrated that learning objective 2 was met through this assignment. Students learned to evaluate how personal biases can affect qualitative data collection. A core aspect of conducting qualitative research is reflexivity. Reflexivity is an awareness of the influence the researcher has on the environment and the people being studied as well as how the research affects the researcher [
There is plenty of research showing how environmental factors affect behaviors, but these factors can also impact data collection and analysis.
At the end of the semester, the students mentioned this assignment as one of their favorite assignments for the course. Not only was it a change of pace for the students, but it also allowed them to work together on the web during a time of social isolation. During the prepandemic period, university students experienced high rates of depression and anxiety [
Although the entirety of this study was not directly informed by the constructivist learning theory, our results suggest that the assignment aligns with the constructivist philosophy and teaching methodology of active learning through real-world experiences (ie, experiential learning) and student-centered approaches [
Constructivist learning strategies also encourage multiple perspectives to represent the topic area of interest. The observation assignment had a discussion portion in addition to the video game portion. Franco and DeLuca [
Although the literature is limited because of the novelty of COVID-19 pandemic, our study and its findings align with the recently published literature concerning COVID-19 and teaching higher education [
Outside the scope of COVID-19, our study adds to the literature on qualitative research. Research is sparse regarding the facilitation and implementation of web-based qualitative research methods courses [
In their rapid communication opinion piece, Neuwirth et al [
Neuwirth et al [
We demonstrated how a web-based method for delivering a qualitative assignment can be successfully implemented in a qualitative research methods course to meet learning objectives, providing additional support for such an assignment. This aligns with a study that investigated the use of
Although our study only focused on only one of the aspects of our course, we would be remiss not to include the lessons learned from our overall experiences with converting this face-to-face qualitative research methods course into a web-based course. As many institutions spend large portions of their curriculums focusing on quantitative research methods, students are often overwhelmed by an entirely new way of critical thinking that encompasses qualitative research. Students desire more synchronous opportunities to discuss readings and assignments and practice data analysis. This interaction can easily be added to a web-based synchronous qualitative course using Zoom videoconferencing (Zoom Video Communications), Microsoft Teams (Microsoft Corp), etc. This study presents numerous opportunities within the realm of qualitative teaching pedagogy, including improvements to the assignment examined in this study. In fact, the instructor has begun to adapt the course from the lessons learned while teaching on the web during the pandemic.
Readings for qualitative research can be dense and difficult to understand, especially when learning qualitative methods first. For future courses, the instructor will post readings and other material on the Perusall learning platform, allowing students to add comments and questions to readings, and see and respond to students’ responses [
Discussion board assignments in this course did not receive positive feedback from the students, except for the observational research assignment described in this study and another assignment that gave the students data to practice coding (qualitative data analysis). Both activities show that the students appreciate discussion boards that allow them to acquire “hands-on” experience with qualitative research methods and then partner with other students to discuss and reflect on what they have learned. Furthermore, the students’ responses suggest that interaction with the instructor and fellow students on discussion boards is important [
The current limitations of this study include its small sample size. This course is an advanced-level doctoral course; therefore, small class sizes are typical. Second, as data were collected from an upper-level doctoral course, the results may not be applicable to qualitative courses whose curriculums are developed for undergraduate students. Although this assignment was developed for doctoral students, we believe that it would also be accessible to undergraduate students with minor adaptations.
Research on the delivery of qualitative research methods courses using web-based learning environments is still in its infancy. We explored whether a pilot web-adapted video game qualitative observational research assignment successfully met the learning objectives of the qualitative research methods course and examined the students’ perceptions of the adapted assignment. While our study provides insights into 1 adaptation of a qualitative assignment, it demonstrates that web-based environments can be used to meet qualitative research learning objectives and adds to our understanding of pedagogical practices for the web-based delivery of qualitative research methods courses. Furthermore, we demonstrate how the assignment aligns with the constructivist learning theory and other successful COVID-19 implementations. With the current pandemic and the resulting digital transformation, it is important to develop methods for an engaging, efficient, and effective delivery of qualitative content and the mastery of qualitative skills without relying on traditional in-person techniques.
COVID-19 pandemic–adapted assignment.
Assure Learners, State Standards & Objectives, Select Strategies, Technology, Media & Materials, Utilize Technology, Media & Materials, Require Learner Participation, and Evaluate & Revise
institutional review board
University of Florida
None declared.