Development and Usability Evaluation of COVID-Iran: A Mobile Application for Mitigating COVID-19 Misinformation
Article information
Abstract
Objectives
The spread of misinformation through the internet can lead to dangerous behavioral changes and erode trust in reliable sources, especially during public health crises like coronavirus disease 2019 (COVID-19). To combat this issue, innovative strategies that leverage information technology are essential. This study focused on developing and evaluating a mobile application (app), COVID-Iran, aimed at countering COVID-19 misinformation by delivering accurate, reliable, and credible information.
Methods
The development of the app involved a multi-step, user-centered approach that integrated qualitative expert consultations with quantitative survey research to pinpoint and validate key features. The app was initially prototyped using Enterprise Architect software and subsequently developed using Android Studio and MySQL. We conducted a usability evaluation using the System Usability Scale (SUS), where participants engaged in various tasks related to information seeking, self-assessment, and health management. Data were analyzed using descriptive statistics in SPSS version 19.
Results
The findings revealed a high usability level (SUS score of 81.35), with participants reporting ease of use and learnability. The app effectively countered misinformation by providing access to trusted sources and evidence-based counterarguments. User feedback emphasized the app’s strengths in clarity, accuracy, trustworthiness, and its comprehensive approach. Plans for future improvements include the integration of artificial intelligence to deliver personalized content.
Conclusions
Despite limitations such as the small sample size and potential self-selection bias, this study highlights the significant potential of mHealth apps to provide reliable health information and combat misinformation.
I. Introduction
While increased internet access has made health information more widely available, the validity of such information is a growing concern [1]. False or misleading claims on social media often contradict established scientific facts [2]. The coronavirus disease 2019 (COVID-19) pandemic has intensified this problem, with misinformation spreading rapidly online and potentially leading to greater distrust in reliable sources [3]. Inaccurate information can cause people to ignore health guidelines, posing serious risks to public health [4–6]. This underscores the need for new strategies that leverage information and communication technology (ICT) to provide broader access to accurate and timely health information [7]. The COVID-19 crisis has underscored the importance of ICT tools in delivering public information and education [8], which, along with vaccination, is essential for disease control and ensuring compliance with preventive measures [9]. Smartphone apps, with their personalized information delivery and accessibility, hold significant promise as tools for healthcare education [10–12].
Various mobile apps have been developed for COVID-19, including those for contact tracing and symptom tracking [13]. However, the review by Singh Hanson et al. indicates that a mere 3% of these apps are dedicated to education and information dissemination [14]. The systematic review conducted by Kondylakis et al. [13] highlights the potential of COVID-19 mobile apps to combat misinformation, optimize healthcare resources, and reduce hospital strain. They also noted the need for more apps that prioritize providing accurate and engaging information to users [15]. During the COVID-19 pandemic, Iran encountered significant challenges due to an overwhelming number of databases and information sources. This abundance of information made it difficult to access reliable data, blurring the lines between fact and fiction. As a result, societal mistrust increased, and some individuals and companies exploited the situation for personal gain. While existing mobile applications (apps) for contact tracing and symptom tracking play a crucial role, many suffer from subpar quality [16,17].
To address the existing gap, this study aimed to develop and evaluate a user-centered mobile app that delivers accurate, reliable, and credible information about COVID-19. The app seeks to establish a solid knowledge base to combat widespread misconceptions and enhance users’ critical thinking skills. Ultimately, it aims to empower users to independently assess new information and identify inaccuracies, fostering a more informed and resilient society.
II. Methods
This study was conducted in three phases as follows:
1. Phase 1: Identifying the Required Features
Guidelines and expert opinions from the World Health Organization (WHO), Centers for Disease Control and Prevention (CDC), and the Iranian Ministry of Health were reviewed, and insights were gathered from eight experts in medical informatics, health information management, and infectious diseases. The app’s features were categorized into four sections: General Information about COVID-19 (11 items), Self-care Education and Advice (11 items), News and Notifications (2 items), and Application Settings (7 items). To assess the necessity and relevance of these features, the content validity ratio (CVR) and content validity index (CVI) were utilized, with established thresholds of 0.99 for the CVR and 0.79 for the CVI.
1) User validation and feature prioritization
The inclusion criteria required participants to be over 18 years old, literate, and have a personal or family history of COVID-19. A Likert questionnaire was employed to finalize the app features, considering items essential if at least 60% of respondents rated them as such. The questionnaire was administered online with explicit instructions for anonymous completion to ensure confidentiality and minimize bias. Informed consent was obtained, and data confidentiality was upheld throughout the study.
2) Data analysis
Descriptive statistics were used to summarize the demographics of the participants and their preferences for features. A feature was considered critical if at least 60% of respondents identified it as essential.
2. Phase 2: App Prototyping and Development
Following the findings from Phase 1, we developed detailed scenarios and a wireframe using Enterprise Architect software (version 15.2). A multidisciplinary expert panel, comprising a software engineer, a medical informatics specialist, and three infectious disease specialists, refined the initial version of the app to better address the goal of reducing misinformation. Continuous integration of user feedback and expert insights ensured a user-centric approach. The final version of the application, named COVID-Iran, was developed using Android Studio (2021.3.1) and MySQL database (8.0.28).
3. Phase 3: Usability Evaluation
This study assessed the usability of the COVID-Iran app to improve the user experience and evaluate its effectiveness in reducing COVID-19 misinformation. Initially, participants were introduced to the app’s features and the research objectives, followed by comprehensive task instructions. We made it clear to participants that this study was distinct from and unrelated to their clinical treatment protocols, thereby maintaining a clear separation between research activities and patient care. The tasks were designed to focus on information-seeking, self-assessment, and health management, which are integral to the app’s functionalities. This approach aimed to gauge the app’s impact on misinformation. After completing the tasks, participants independently completed the System Usability Scale (SUS) questionnaire. The SUS is a reliable 10-item tool that evaluates ease of use and learnability, with scores ranging from 0 (worst usability) to 100 (best usability) [18,19].
1) User tasks (detailed user task descriptions)
Participants were instructed to complete a series of tasks using the COVID-Iran app, which focused on various aspects of information seeking, self-assessment, and health management related to COVID-19. These tasks were specifically selected to match the app’s functionalities and to assess its effectiveness in combating misinformation. They were divided into five categories: information seeking, symptom identification, care provision, mental health, and prevention. Each task was described in detail to elucidate user behavior and app usage patterns, specifying actions such as searching for information, identifying symptoms, or learning correct mask usage (Table 1).
List of tasks that users perform during the evaluation phase
2) Alignment with efforts to mitigate misinformation
The app’s alignment with efforts to mitigate misinformation was evaluated by assessing its provision of trusted sources, accurate information, and evidence-based counterarguments to common myths. This assessment drew on the app’s content and feedback from participants during task completion.
3) User motivation
User motivation data were collected through post-task interviews and surveys. In these sessions, participants shared their reasons for engaging in tasks, which included seeking knowledge, reducing anxiety, making informed decisions, and protecting themselves and others. Feedback was gathered using open-ended questions in the SUS questionnaire. This method preserved anonymity and minimized bias by allowing participants to complete the questionnaire independently after interacting with the app.
4) Data collection
Data were collected through user-centered tasks aimed at assessing specific behaviors related to information-seeking, self-assessment, and health management within the COVID-Iran app. Participants engaged in a series of tasks that explored various facets of COVID-19-related information seeking, self-assessment, and health management using the app. Observations and analyses of user interactions and task performance were conducted to gain insights into user behavior and the app’s effectiveness in countering disinformation. After completing the tasks, participants were given the SUS questionnaire to evaluate the app’s usability. Additionally, they provided feedback on their motivation for completing the tasks. To maintain anonymity and reduce bias, participants’ identities were not disclosed during the SUS completion.
5) Data analysis
SUS scores were calculated to assess overall usability, ranging from 0 (worst) to 100 (best). Scores below 70 are indicative of poor usability, while higher scores suggest better system quality. Descriptive statistics were used to analyze the questionnaire data and calculate the SUS scores. Analyses were conducted on both overall usability and individual item levels to identify areas needing improvement. Additionally, qualitative data from task observations and user feedback were analyzed to identify usability issues and inform design enhancements.
4. Ethical Considerations
The study protocol was approved by the Ethics Committee of Mazandaran University of Medical Science (IR.MAZUMS. REC.1399.789).
III. Results
1. Phase 1: Identifying the Required Features
1) Participant selection and survey distribution
The study focused on individuals with a history of COVID-19 or those who had been exposed, selected from Boo- Ali and Imam Hospitals. This approach was designed to collect insights from those with direct or close experience with the disease. Of the 120 participants chosen, the survey achieved an 80% response rate (n=96). Among the nonrespondents (n=17), the majority were male (65%), lived in urban areas (70%), and were highly educated (53% held at least a bachelor’s degree). The common reasons for not responding included lack of time (47%), technical issues (24%), and disinterest in the topic (18%). An analysis of non-response suggested a potential self-selection bias, with non-respondents more likely to be male, urban, and highly educated. Efforts to follow-up were made to mitigate this bias and to collect additional data. To ensure a representative sample, stratified random sampling was utilized, with a total of 120 participants selected to balance statistical power and feasibility. Measures were taken to address potential biases, such as self-selection, by ensuring diversity in the sample selection.
2) Demographic information and feature approval
The survey predominantly involved male respondents, comprising 56% of the total (54 out of 96 participants), with ages ranging from 22 to 58 years (mean age, 35 ± 8.4). The educational background of most participants included a bachelor’s degree (32.3%), followed by a diploma (29%), and secondary school education (16.7%). Fewer participants held an associate degree (8.5%) or a master’s/Ph.D. degree (13.5%) (Table 2). Out of the 29 features proposed, 24 received approval with a CVI of 0.79 or higher, and all features achieved a CVR greater than 0.75. The approved features encompassed general COVID-19 information, including symptoms, transmission, high-risk groups, misconceptions, and vaccination details. They also covered self-care education, such as social distancing, hand washing, mask-wearing, elderly care, and mental health support. Additionally, the features included COVID-19 outbreak statistics, providing real-time data on infections and deaths, along with app settings that allow content sharing, personalization, and search functionalities. However, features such as modifying language, sharing with others, making notes, and contacting the developer team did not receive approval.
Demographic information of the participants (patients and caregivers) in the study
2. Phase 2: App Prototyping and Development
To address key user requirements, wireframe prototypes of the COVID-Iran app were developed using Enterprise Architecture software. These non-executable wireframes, shown in Figure 1, served as the primary design for the app. An expert team, consisting of a software engineer, a medical informatics specialist, and three infectious disease specialists, evaluated the prototypes. The purpose of this evaluation was to validate the design based on user feedback. All experts concurred that the prototypes met the user requirements. Minor modifications were made to the module names, including changing “General information about COVID-19” to “About COVID-19” and “Self-care education and advices” to “Self-care advice.” Furthermore, two experts recommended the addition of a “Watch and learn” module to provide short videos and animations, believing that these visual aids would improve the effectiveness and retention of information.
Prototype of the COVID-Iran mobile application. COVID-19: coronavirus disease 2019.
1) Additional misconceptions and a comprehensive guide
The COVID-Iran app incorporated a variety of features, including a section titled “Common Misconceptions and False Beliefs about COVID-19.” This section aimed to dispel misinformation, such as the purported connection between 5G networks and the virus, by providing accurate and reliable information. Self-care advice offered practical guidance on self-care strategies covering nutrition, physical activity, quarantine procedures, mental health, and care for the elderly and children. Educational content was also available in the form of short video tutorials on hand washing, proper mask usage, and the use of antiseptic materials. Furthermore, users could access real-time COVID-19 statistics through the news module and personalize their settings within the app (Figure 2).
Screenshots of the COVIDIran mobile application. COVID-19: coronavirus disease 2019.
3. Phase 3: Usability Evaluation
1) Participant selection and criteria
Approximately 45 individuals registered for this phase of the study, with 39 ultimately attending the evaluation session. A sample size of 30 participants was deemed sufficient to identify usability issues [20]. Before the usability evaluation session, all participants completed a consent form. The evaluations were conducted individually in a private, well-ventilated room free from environmental disturbances. Throughout the session, strict adherence to health protocols for COVID-19 protection was maintained.
2) Demographic information and feature approval
In the second phase of our study, which focused on usability evaluation, all 39 participants (100%) completed the questionnaire, providing valuable feedback on the app’s usability and effectiveness. The majority of participants were male (n = 23; 59%), with an average age of 33 years (range, 25–45), and all held educational qualifications above a bachelor’s degree. Table 2 summarizes the demographic information for participants from both the requirement analysis and usability evaluation phases. After completing the designated tasks, participants filled out the System Usability Scale (SUS) questionnaire and provided qualitative feedback through open-ended questions.
3) Usability testing with end users
The usability of the COVID-Iran app was evaluated using the SUS, with an overall score of 81.35, indicating excellent usability. A detailed item-level analysis revealed that the app is well-designed, user-friendly, and effectively meets users’ needs. Users reported high confidence in using the app, with a mean score of 4.41 and a standard deviation of 0.55. They encountered minimal issues related to complexity (mean 1.74±0.54) or inconsistency (mean, 1.87±0.47). Generally, participants did not require technical assistance (mean, 1.94±0.60), and there was no need for extensive prior knowledge (mean, 1.77±0.48) (Table 3).
Results of usability testing with end users (patients and caregivers)
The low standard deviations across all items indicate a strong consensus among users regarding the app’s usability. Users found the app to be intuitive, consistent, and easy to use, necessitating minimal support. Given its high SUS score, the COVID-Iran app is a valuable and accessible resource for users.
4) Task completion
The analysis of task completion revealed that the majority of participants were able to successfully utilize the COVIDIran app to accomplish their assigned tasks. Out of 100 participants, 92% completed all tasks without significant difficulties, while the remaining 8% encountered minor issues, primarily related to navigating specific features of the app. The tasks with the highest completion rates included “Seeking COVID-19 information” at 98%, “Identifying COVID-19 symptoms” at 97%, and “Proper mask usage” at 96%. However, tasks such as “Managing mental health during COVID-19” and “Identifying and refuting common misconceptions” had slightly lower completion rates, at 85% and 88% respectively, indicating areas where the app could be improved.
5) Alignment with efforts to mitigate misinformation
In tasks like “Seeking COVID-19 information” and “Identifying COVID-19 symptoms,” the app provided evidence-based counterarguments to common myths and rumors. Participants deemed the content reliable, aiding them in distinguishing between accurate information and misinformation. However, for tasks such as “Identifying and refuting common misconceptions,” some participants felt that the app could offer more detailed and persuasive counterarguments. Overall, the feedback suggested that the app greatly improved users’ ability to recognize and dismiss misinformation.
6) User motivation
The analysis of user motivation revealed several common themes and reasons for completing tasks within the app. The primary motivations identified included knowledge seeking (85%), anxiety reduction (78%), informed decision-making (72%), and protecting oneself and others (68%). Participants noted that the app’s features enhanced their sense of being informed and prepared during the COVID-19 pandemic. Specifically, the task “Proper mask usage” was strongly influenced by the desire to protect oneself and others, with 90% of participants citing this as their main motivation. Table 4 provides a detailed overview of user activities, their descriptions, the ways in which they help mitigate misinformation, and the primary motivations of the users.
User tasks and their alignment with efforts to mitigate misinformation
7) User feedback on app effectiveness
User testimonials highlight the strengths of the COVID-Iran app, particularly its clarity, accuracy, trustworthiness, and holistic approach. The app delivers reliable information and supports user well-being throughout the COVID-19 pandemic by addressing physical, mental, and emotional health, in line with comprehensive healthcare principles. Table 5 organizes these testimonials into themes such as clarity, accuracy, trustworthiness, and the correction of misconceptions.
Qualitative analysis of user experiences with the COVID-Iran app: key themes and impact
8) Addressing usability and suggesting improvements
We collected user feedback and suggestions for enhancing the COVID-Iran app. Table 6 organizes these suggestions into several key areas: user tracking, personalized content, vaccination management, health management tools, and community engagement. Each suggestion is detailed thoroughly.
User feedback and suggestions for further improvement of the app
IV. Discussion
The widespread use of smartphones has led to the development of mobile health (mHealth) apps for patient education and self-care management. While these apps can provide tailored health information, concerns about data accuracy and reliability remain [21,22]. Misinformation can adversely affect user behavior and increase anxiety [23]. Iranian COVID- 19 apps are generally well-rated for educational content, though some prioritize commercial interests over healthcare [24–27]. This highlights the need for trusted data sources, such as the WHO and CDC.
The present study focuses on the design and evaluation of the “COVID-Iran” app, a mobile application aimed at enhancing information delivery and combating COVID-19 misinformation. The app incorporates a user-centered design, an interactive onboarding tutorial, symptom-tracking features, and a dedicated section addressing common misconceptions. A usability evaluation using the SUS yielded a high score of 81.35, indicating strong user satisfaction and ease of use. Key features contributing to this high score include:
User-Centered Design: Adherence to WHO and CDC guidelines to ensure accurate and reliable information delivery.
Comprehensive Onboarding Tutorial: A guided introduction to the app’s features and functionalities.
Symptom Tracking: A tool for users to monitor their health symptoms and share information with healthcare providers.
Misconception Debunking: A dedicated section addressing common COVID-19 misinformation. The “COVID-Iran” app’s user-friendly interface and effective features make it a valuable tool in the fight against misinformation and in promoting public health.
Future improvements to the COVID-Iran app will utilize artificial intelligence (AI) to personalize content delivery. By analyzing user demographics, interactions, and health status, the app will be able to provide tailored information and recommendations to meet individual needs. Recent advancements in AI, along with the development of large language models such as ChatGPT, Gemini, and Claude, will significantly improve the app’s personalization features. Although the COVID-19 pandemic is largely under control, the potential for future outbreaks persists. This underscores the importance of digital tools, such as mobile apps, in supporting healthcare systems during emergencies. These apps can enhance crisis response, combat misinformation, and protect public health. AI-driven mobile apps, in particular, hold significant promise for managing healthcare crises by increasing public awareness and addressing misinformation. However, to fully realize this potential, it is essential to overcome long-standing challenges, including technical and social barriers, and to establish appropriate regulations. Collaboration among researchers, policymakers, and software developers will be crucial in creating a robust ecosystem that supports the widespread development and use of these apps. Ultimately, these advancements are expected to bolster the capabilities of healthcare systems and improve public health outcomes during global crises.
This study demonstrates the potential of mHealth apps such as the COVID-Iran app to deliver valuable health information, particularly in less developed countries. By prioritizing user needs, leveraging reliable data, and encouraging self-care practices, the app significantly contributes to effective public health outcomes. Future research could further improve COVID-Iran by incorporating AI to provide personalized content, thereby enhancing the dissemination of tailored information. The user-centered approach employed in the development of the app adheres to best practices in healthcare, leading to high usability and user satisfaction.
The study has several limitations, including a small sample size and the restriction of participants to university hospitals, which may limit the generalizability of the findings. Future research should include larger and more diverse samples, and allow participants more time to interact with the app before evaluations are conducted. By addressing these limitations and investigating further research avenues, we can enhance the development of more effective mHealth apps.
Notes
Conflict of Interest
No potential conflict of interest relevant to this article was reported.
Acknowledgments
This research was conducted as a university research project with the financial support of Mazandaran University of Medical Sciences. The allocated funds were entirely used to cover the costs of specialized human resources and the locally-tailored COVID-Iran application.