I. Introduction
Emergency remote teaching (ERT) refers to a temporary shift to online education during crises or disasters [1]. In response to the coronavirus disease 2019 (COVID-19) pandemic, schools were shuttered to curb the virus’s spread, prompting widespread closures of educational systems in several countries [2]. Hodes et al. [1] emphasized the term ‘Emergency’ in ERT, noting that it is a temporary measure distinct from traditional, well-established online learning platforms, and thus should not be directly compared. Consequently, educators faced a substantial increase in preparation time due to the transition to online learning [3].
Several studies have examined educators’ general characteristics that can influence online learning outcomes [4,5]. Kent and Giles found that an educator’s age, highest level of education, and teaching experience impacted their technology self-efficacy in online learning environments. Furthermore, the effectiveness of online learning has been the subject of previous research. Roach and Lemasters [6] assessed both the content and delivery methods of online education, as well as the ease of locating educational materials, and how these factors might affect learners’ satisfaction with online learning. Shelton et al. [7] identified the interaction between teachers and students as a key factor influencing student satisfaction with online education. Hung [8] investigated technology self-efficacy to determine instructors’ preparedness for online teaching and the adequacy of institutional support for online education.
Apart from research on online learning, there are also studies focusing on nursing education during the COVID-19 pandemic. Wild et al. [9] conducted a comparison between simulation-based and traditional face-to-face nursing education in the United States. Meanwhile, Leigh et al. [10] suggested various learning strategies, such as scenario-based, virtual, and self-directed learning, for undergraduate nursing education in the United Kingdom.
Since ERT differs from well-planned online learning, it is believed that the challenges faced by nursing educators are unique. The primary research question of the current study is, “How have nursing educators managed the implementation of nursing education using ERT in various countries during the pandemic?” The secondary research questions are as follows: “What teaching methods have been employed in response to the current pandemic?” “How do educators perceive the status of nursing education?” and “What factors influence satisfaction with ERT?” The aim of this study is to determine how the factors derived in previous online learning research influence ERT satisfaction and to identify the current circumstances and challenges faced by nursing educators, with the goal of proposing enhancements to nursing education and its curriculum.
II. Methods
1. Study Design
This incorporated an international cross-sectional survey and a parallel mixed-method study. We conducted a mixed-method study that involved quantitative and qualitative analysis using natural language processing to gather a wide range of opinions from educators. Additionally, we conducted an international cross-sectional study to gain insights into various current conditions, avoiding bias toward any specific country. Qualitative analysis provides a richer understanding, but is time-intensive, labor-intensive, and prone to bias for researchers [11]. However, a previous study used computerized analyses such as natural language processing— a new form of qualitative analysis that applies algorithms to analyze text data—to compensate for some of the limitations of qualitative research [12]. Thus, after the qualitative responses were subjected to natural language processing, the researchers reviewed the outcomes and subjected them to further analysis.
2. Questionnaire
The questionnaire was developed through iterative discussions among 11 experts from the International Medical Informatics Association–Student and Emerging Professionals Group–Nursing Informatics (IMIA-SEP-NI). It was translated into seven languages: Arabic, English, Korean, Indonesian, Portuguese, Spanish, and Traditional Chinese. For each language, two bilingual nurses or health informaticians with a background in nursing informatics (NI) carried out the translation. This was then verified by at least two other bilingual experts and revised until a final version was achieved.
The questionnaire consisted of 21 questions (see Supplement A). The first seven questions gathered demographic information about the educators, including age, country and city of residence, current position, level of education, educational background, experience in nursing education, and the type of subject taught [4–6,13,14]. The eighth question inquired about the teaching methods used during the COVID-19 pandemic, specifically face-to-face versus online methods. Questions #9 to #18 required participants to rate their responses on a 5-point Likert scale, ranging from strongly disagree to strongly agree. These questions also invited open-ended comments to elaborate on the reasons behind their ratings. The topics addressed in these questions pertained to ERT and included overall satisfaction, content depth and breadth, the delivery of pre-prepared educational materials, teacher–student interactions, perceived burden, technological self-efficacy, and support. The support-related items specifically asked about the ease of accessing digital/online material support and institutional backing for the transition to digital/online resources. Additionally, participants were asked about their willingness to continue using these methods in the future. The final three questions (#19–#21) were open-ended. They prompted respondents to reflect on their experiences with ERT during the COVID-19 pandemic by discussing lessons learned, expectations for the advancement of nursing education, and suggestions for how NI organizations could bolster nursing education. This study received approval from an Institutional Review Board of Hoseo University (No. 1041231-200825-HR-114-01).
3. Participants and Data Collection
Any educator with experience in nursing education during the COVID-19 pandemic, dating back to January 2020, was eligible to participate. We employed the convenience snowball sampling method, which relies on networking and referrals [15]. Initially, the authors emailed the IMIA-SEP-NI group to serve as the starting point for global data collection, distributing Google Forms that included consent forms and the questionnaire (see Supplement A). Subsequently, participants who consented were asked to share the Google Forms link through their academic networks, such as their country’s NI or professional nursing groups, via email or social network services (e.g., Twitter and KakaoTalk). Data collection spanned from November 2020 to January 2021.
4. Analysis
The analytical process used in this study is shown in Figure 1. Authors from each participating country translated the survey responses from their respective national languages into English. The English-translated data were then collectively analyzed. Initially, a descriptive analysis of the participants’ characteristics and the teaching methods referenced in questions #1–#7 was conducted. Subsequently, means and standard deviations were calculated for the scores of questions #9–#18. The responses to question #14, which pertained to burden, were reverse-coded. Lastly, the open-ended responses to questions #9–#21 were subjected to a separate analysis. This involved the use of natural language processing and latent Dirichlet allocation (LDA) topic modeling via Python [16,17], as well as qualitative analysis and the extraction of meaningful insights. These tasks were undertaken by the first and corresponding authors (EJJ and JL), both of whom are NI researchers.
The descriptive analysis was conducted using Python (3.6.9) with the numpy (1.19.4), pandas (1.1.5), and sklearn (0.22.2) libraries. Spearman correlation coefficients and the chi-square test were used to determine the impact of educators’ characteristics.
To analyze qualitative data from open-ended question responses, two researchers initially categorized all comments for each question as either positive or negative. This was followed by the commencement of the pre-processing stage. For natural language preprocessing, we utilized Python’s regular expressions to remove special characters and double spaces. Data were then tokenized using the NLTK (3.2.5) to-kenizer available in Python. During part-of-speech tagging, we included adjectives and verbs, specifically tagging “Adjective (JJ),” “Adjective, comparative (JJR),” “Adjective, superlative (JJS),” “Adverb (RB),” “Adverb, comparative (RBR),” “Adverb, superlative (RBS),” “Verb, base form (VB),” “Verb, past tense (VBD),” “Verb, gerund or present participle (VBG),” “Verb, past participle (VBN),” “Verb, non-3rd person singular present (VBP),” and “Verb, 3rd person singular present (VBZ).” Following this, English stop words from the NLTK library were removed. Finally, lemmatization was performed using the WordNet Lemmatizer of the Python NLTK library (e.g., “have,” “has,” “had” → “have”).
Next, LDA topic modeling was performed using the Python Gensim library (version 3.6.0) to extract core topics. LDA operates by classifying or categorizing text into documents and words per topic, which are modeled based on Dirichlet distributions and processes [18]. Additionally, the optimal number of topics for LDA is determined by the CV coherence score value. This metric evaluates a topic’s semantic consistency, with higher values indicating greater semantic consistency.
Two NI researchers independently reviewed all comments and 20 representative words from each topic to gain a better understanding of the participants’ responses and meanings. They conducted a qualitative analysis to categorize the topics, merging or adding categories as necessary. Each researcher independently read and classified the responses according to the topics identified through LDA, and they introduced new thematic labels to account for comments that did not fit into the existing topics. Any inconsistencies in the categorization of comments and the introduction of new themes were resolved through discussion.
III. Results
1. General Characteristics of Respondents and Teaching Methods
A total of 328 respondents from 18 countries participated in the survey: Brazil (128, 39.02%), Indonesia (57, 17.38%), South Korea (33, 10.06%), Jordan (27, 8.23%), the Philippines (22, 6.71%), Taiwan (17, 5.18%), Argentina (11, 3.35%), Canada (8, 2.44%), Finland (6, 1.83%), the US (6, 1.83%), Mexico (5, 1.52%), Italy (2, 0.61%), India (1, 0.3%), Portugal (1, 0.3%), Saudi Arabia (1, 0.3%), Scotland (1, 0.3%), Switzerland (1, 0.3%), and Türkiye (1, 0.3%).
Table 1 shows the respondents’ characteristics. Most respondents were 30–39 years of age (36.28%), and 26.22% were assistant professors.
Question #8, on online teaching methods, allowed multiple selections. The most used were video conference systems (e.g., Zoom and Microsoft Teams; n = 281, 28.18%), followed by teaching management platforms (e.g., Google Classroom and Moodle; n = 197, 19.76%), social networking technology (e.g., Facebook, Google+, WhatsApp, and Twitter; n = 147, 14.74%), video sharing platforms (e.g., YouTube and Flipgrid; n = 145, 14.54%), the institution’s online platform (n = 99, 9.93%), game-based teaching platforms (e.g., Kahoot; n = 47, 4.71%), blogs or websites (n = 36, 3.61%), massive open online courses (e.g., Coursera and Canvas network;n = 29, 2.91%), and virtual learning systems (e.g., SecondLife and vSim; n = 16, 1.60%). In analyzing the relationship between the number of software platforms used and other variables (Questions #9–#18), no statistically significant correlations were found.
2. ERT Experience: Quantitative Data Analysis
The results for ERT status (questions #9–#18) are shown in Table 2. Regarding the positive aspects of ERT, support from the institution had the highest score, followed by technological self-efficacy, willingness to use ERT in the future, support for using digital/online material, overall satisfaction with ERT, and the range of the content.
The results for the influence of educators’ characteristics on ERT experiences are shown in Table 3. The average satisfaction level was 4.00 ± 0.88 out of 5 for professors, 3.58 ± 0.98 for associate professors, 3.71 ± 0.72 for assistant professors, and 3.49 ± 0.82 for instructor/lecturers. Thus, more highly-ranked positions were generally associated with higher satisfaction with ERT. The country of the educators also showed a significant correlation with ERT experiences. Results showing substantial average differences between countries that provided five or more responses (for questions #9, #11, #12, #13, #14, and #18) are illustrated in Figure 2.
3. Detailed Comments on ERT Experience: Qualitative Data Analysis
Table 4 presents the number of topics selected for each question. More specifically, Supplement B shows topics using words extracted by LDA for each question. Two researchers qualitatively analyzed participants’ responses to the open-ended comments sections of the ERT questionnaire, and the results are presented in Supplement C.
4. Lessons and Expectations
Table 5 shows the results for questions #19–#21 of the questionnaire. The respondents reported that they had learned the following lessons by applying ERT in nursing education: “Effective in theory but inadequate in practice class,” “One must use ERT because it is essential in the pandemic era, but there is room for improvements,” and “Online teaching is a future trend, so it should be continuously improved and utilized.” They also believed that ERT has the advantages of accessibility (i.e., availability anytime and anywhere), repeatability, and self-learning. In contrast, educators commented on the lack of suitability of the ERT for clinical practice in nursing education.
The respondents also provided the following suggestions to improve nursing education: “We need to develop a practical or hybrid course suitable for the pandemic era,” and “Online teaching should be developed.” They also expressed a need for further support for high-quality instruction in ERT through improvements in related laws and guidelines.
Respondents offered several recommendations for how NI organizations can support nursing education, including “support for online teaching and for the simulation of clinical practice,” “provide curricula and standards for online teaching methods,” and “education on freeware teaching tools, and reliable online resources, among others.” Hence, they recognized the need for organizations to provide standards and structures for ERT in nursing.
IV. Discussion
This study is meaningful because it employed both quantitative and qualitative analyses, utilizing computer programs and human researchers, respectively, to investigate the ERT experiences of 328 nursing educators from 18 different countries.
Overall, the participants reported a moderate level of satisfaction with ERT in nursing, as indicated by an average score of 3.59 out of 5. This finding is consistent with survey results from previous research, which indicated that 62.9% of faculty members were satisfied with online teaching in the United Arab Emirates during the COVID-19 pandemic [19]. Similarly, a study on ERT satisfaction in the United States revealed that 68.7% of academic faculty were content with ERT, giving it a score of 3.71 out of 5 [20]. Despite the additional workload involved in preparing for ERT, faculty members reportedly developed online self-efficacy through this experience and adapted accordingly [21]. This may correspond to the higher score observed in this study for the “willingness to use in the future” item, which exceeded satisfaction levels with a score of 3.76 out of 5. Despite the challenges faced by educators, digital technology has proven indispensable for maintaining educational relationships [22]. Moreover, respondents expressed the opinion that ERT, as an emerging trend in education, is expected to improve over time.
Nonetheless, our study and previous studies confirmed that the shift to ERT was burdensome for faculty members. Factors contributing to dissatisfaction included a higher work-load and the need for extended preparation time [19]. Al-leviating this burden will require well-organized IT technical support, collaborative teams, and established standards for online learning [20,23]. Furthermore, there should be clear standards for online learning to enable professors to develop a range of strategies for teaching in this modality [20].
Respondents also described the disadvantages of ERT, including issues such as content depth, delivery, and teacher– student interaction. These challenges are similar to the limitations of online learning. However, is not appropriate to compare the sudden shift to online learning due to COVID-19 with existing well-designed online learning, and doing so may mislead and spread the stigma and general perception that online education is of low quality [24]. Instead, we suggest that the negative aspects were highlighted to a greater extent due to the rapid transition to ERT. Therefore, well-planned online learning is likely to lead to higher satisfaction and growth potential.
Brazil (n = 129) had the lowest score for delivery, with an average of 2.34 ± 1.21 out of 5. Respondents noted a lack of dynamic, real-time interaction for practice and expressed a preference for face-to-face education, emphasizing the importance of eye-to-eye contact with students. In contrast, South Korea (n = 34) reported satisfaction with the delivery of online learning, which was executed as planned. Respondents highlighted the use of various tools and, with a satisfaction score of 4 or more points, many comments mentioned the employment of diverse software such as Google Classroom and Zoom.
Interestingly, the technological self-efficacy score was high. Previous studies have shown that, due to online learning, professors have acquired skills in online technology [20], and this high self-efficacy has facilitated an efficient transition from face-to-face to virtual formats [21]. Additionally, many respondents highlighted the importance of online learning infrastructure and IT support. However, there were instances where the necessary equipment for online learning, such as computers and monitors, was not available, or internet speeds were too slow to conduct lectures effectively. The COVID-19 pandemic has exposed a global educational and technological divide, posing a threat to digital equity [25]. A worldwide survey of higher education students also reported issues with internet connectivity [26]. As technology is rapidly integrated, the disparity in access to the technology and devices required for ERT among educators and students may need to be addressed. In planning for future online learning, it is essential to consider digital equity.
Although 57 topics were identified via LDA, it was sometimes challenging to clearly discern the topics. This difficulty arose partly because the top 20 representative words for each topic were not always sufficient to convey the full scope of the topic, given LDA’s limitations in capturing pragmatic and semantic nuances [11]. For example, topics such as ‘individual level contact’ or ‘not effective’ required a review of the actual responses to fully understand their implications. As a result, this combined approach of computational and human analysis identified an additional 55 distinct themes, underscoring the importance of human interpretation in complementing computer-generated results. In light of the limitations identified in our study, we propose recommendations for future research. Our methodology involved convenience snowball sampling, and we did not collect participants’ email addresses, which would have helped in recognizing duplicate responses. Consequently, our final sample had an uneven geographical representation. Specifically, Brazil accounted for 39% of the responses, European countries had a low response rate, and we received no responses from educators in China. In future studies, we plan to nationalize portions of the research and collect email addresses to better understand international perspectives on the topic.
In conclusion, the findings and implications of our study related to technological challenges and solutions in NI education are as follows. Our results showed that nursing educators had adapted well to ERT, even though they initially found it burdensome to use ERT during the COVID-19 pandemic. This study’s findings underscore the importance of nursing education stakeholders being prepared to apply hybrid education drawing on the experiences gained from ERT. These should include standardized, student-centered classes and strengthened support to promote digital equity. Our findings also highlight the significant role of special interest groups or academic associations of NI in providing feasible software solutions and practical simulation training based on real clinical data, along with reliable online materials. Furthermore, it is essential to organize forums to share training courses and ERT experiences on these ever-changing online teaching platforms.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Full text via PMC
Download Citation
Supplement1
