Healthc Inform Res Search


Healthc Inform Res > Volume 24(1); 2018 > Article
Lee and Park: Effects of Flipped Learning Using Online Materials in a Surgical Nursing Practicum: A Pilot Stratified Group-Randomized Trial



This study examined the effect of flipped learning in comparison to traditional learning in a surgical nursing practicum.


The subjects of this study were 102 nursing students in their third year of university who were scheduled to complete a clinical nursing practicum in an operating room or surgical unit. Participants were randomly assigned to either a flipped learning group (n = 51) or a traditional learning group (n = 51) for the 1-week, 45-hour clinical nursing practicum. The flipped-learning group completed independent e-learning lessons on surgical nursing and received a brief orientation prior to the commencement of the practicum, while the traditional-learning group received a face-to-face orientation and on-site instruction. After the completion of the practicum, both groups completed a case study and a conference. The student's self-efficacy, self-leadership, and problem-solving skills in clinical practice were measured both before and after the one-week surgical nursing practicum.


Participants' independent goal setting and evaluation of beliefs and assumptions for the subscales of self-leadership and problem-solving skills were compared for the flipped learning group and the traditional learning group. The results showed greater improvement on these indicators for the flipped learning group in comparison to the traditional learning group.


The flipped learning method might offer more effective e-learning opportunities in terms of self-leadership and problem-solving than the traditional learning method in surgical nursing practicums.

I. Introduction

With the rise of student-centered learning environments in the field of education, various alternative instructional strategies are now being studied, such as small-group problem solving, student presentations, and group discussions [1]. Recently, new innovative technology and online e-learning platforms (e.g., Blackboard Inc.) have been integrated into higher education, and this provides support for both professors and students. In the past, such online e-learning platforms have been used to implement supplementary strategies to upload class materials; however, the flipped classroom approach utilizes these platforms more actively and efficiently [234].
A typical flipped classroom approach is composed of two parts: (1) students watch pre-recorded lectures online before class, which cover learning content, quizzes, and/or module assignments and (2) students actively participate in interactive and higher-order learning activities during class, such as discussions and problem-solving, which aim to empower students to apply what they have learned pre-class [56]. Thus, the term ‘flipped classroom’ is derived from the idea that “which is traditionally done in class is now done at home, and that which is traditionally done as homework is now completed in class” (p. 13) [6].

1. Comparison of Flipped Learning and Traditional Learning

Despite the benefits for both professors and students, the flipped classroom approach has been criticized as not being entirely novel, as the traditional classroom has long been utilizing new technology and software, such as Blackboard, for lesson content and assignments. However, Strayer [7] insisted that “the regular and systematic use of interactive technology” is the core of the flipped classroom approach. In other words, passive and supplementary use of technology is a limitation of previous flipped classrooms that replaced inclass lectures with videos and homework with regular class work. The systematic use of interactive technology means that it facilitates two-way interaction between students and teachers, in contrast to the previous unidirectional lecture style [7]. In addition, the flipped classroom approach includes differentiated education, in contrast to the onesize- fits-all lecture style used by professors in traditional classrooms [1].

2. Flipped Learning in University Education and Nursing Education

The flipped classroom approach has been implemented in various fields of education [123]. Recently, the flipped classroom approach has been implemented in basic and advanced medical training and education [4]. In the field of nursing education, several studies have employed the flipped learning approach to investigate its effects [89]. However, there has been little evidence demonstrating the effectiveness of the flipped learning method among undergraduate nursing students in clinical practicum courses. In addition, few researchers have detailed the principles of the flipped classroom approach when designing their studies [7], so the intervention may not have shown significant benefits for students and professors. Therefore, the purpose of this study was to determine the effect of flipped learning in comparison to traditional learning for nursing students in a clinical practicum program.

II. Methods

1. Study Design and Participants

This study used a two-arm parallel, stratified group-randomized trial. The study was conducted between September 1, 2016 and November 30, 2016 at a university in Korea. The subjects were 102 nursing students in their third year of university who were scheduled to complete a clinical nursing practicum in either an operating room or general surgical unit. The eligibility criteria for the participants was that (1) they could use a computer or a mobile phone and (2) had home Internet access. All 102 participants provided written informed consent prior to their participation in the study. Furthermore, this study was approved by the Institutional Review Board of Kyungpook National University (No. KNU-2016-90).

2. Randomization

During one semester, 10 teams consisting of five or six students each participated in a practicum in an operating room, and the other 10 teams consisting of the same numbers of students participated in a practicum in a general surgical unit. Randomization was conducted on the 10 teams within each clinical practicum unit. In the operating room practicum, 5 teams were designated as flipped learning groups, and the other 5 as traditional learning groups. The conditions were the same for the general surgical unit practicum. An independent statistician carried out the randomization using the SAS random number generator (SAS ver. 9.4 PROC PLAN; SAS Institute Inc., Cary, NC, USA). A random number was allocated to each of the 10 teams. The teams were placed in numerical order by the numbers allocated. Then, the independent statistician generated a random allocation sequence to designate the teams as either flipped earning group or traditional learning group. The 20 teams (n = 102) were randomly assigned (1:1) to either of the two groups; as a result, the number of students in each group was 51. After written consent had been received from all participants and the baseline survey had been administered, the randomization was completed. The students in each group were formally informed of their allocation to one of the learning styles and whether they were in the experimental or control group. However, the allocation could not be completely blind because students were in the same academic year.

3. Intervention

The flipped learning program for the surgical nursing practicum was designed based on the nine principles of flipped learning design [1] (Table 1). The 1-week, 45-hour clinical nursing practicum included the instructor's orientation before the clinical training, on-site instruction during the clinical practicum, and a case conference after the clinical practicum was finished. The flipped learning group and the traditional learning group received a brief orientation on the operating room or general surgical unit through a faceto- face meeting with the professor. In addition, the flipped learning group was instructed to complete the independent e-learning lessons on surgical nursing before the week of the practicum. Before the week of the clinical practicum, the flipped learning group studied the e-learning contents (i.e., 20 pages of PowerPoint materials and images, and 10-minute videos) on surgical nursing independently using the Smart Learning tool serviced by the Center for Teaching and Learning of the university. On average, students reported that the pre-learning took 40–50 minutes. The Smart Learning tool was accessed via the university's homepage. During the on-site instruction, the flipped learning group had time for questions and answers on the previous e-learning contents, and practiced making nursing diagnoses based on what they had learned from the e-learning contents and their knowledge from the ongoing clinical nursing practicum.
During the on-site instruction, the traditional learning group received face-to-face instruction from the professor on the same content that was provided to the flipped learning group through e-learning; they were also given time for questions and answers. On average, the on-site instruction took 35 minutes. The students were informed that pre-learning was mandatory and that the professor could check who had accessed the material through a function of the Smart Learning tool. The professor confirmed that most students accessed the pre-learning. Qualitatively, the student's responsibility for pre-learning was ensured because evaluation using quizzes was reflected in the grade. The flipped learning group conducted a case study and had a conference based on it in the same manner as the control group (Figure 1). During the on-site instruction, the professor led essential learning activities face-to-face. Furthermore, the professor checked the students' understanding and provided assistance to those experiencing difficulties, and encouraged them to apply the nursing process to the patient case and identify the missing parts during the process. During the case conference, summative feedback was given to each student.

4. Main Outcomes

A baseline survey was conducted during the orientation prior to the commencement of the practicum, and a post-intervention survey was conducted after the case conference.

1) Self-efficacy in clinical performance

Self-efficacy in clinical performance measured the students' confidence during the clinical nursing practicum. Self-efficacy in clinical performance was measured using a 37-item self-efficacy in clinical performance scale, which was divided into the following five subscales: assessment, diagnosis, planning, implementation, and evaluation [10]. The assessment subscale consisted of 12 items, diagnosis consisted of 3 items, planning consisted of 6 items, implementation consisted of 10 items, and evaluation consisted of 6 items. The self-efficacy in clinical performance scale was assessed on an 11-point Likert scale, ranging from 1 (does not correspond at all) to 11 (corresponds exactly), with higher scores indicating higher self-efficacy in clinical performance. Cheraghi et al. [10] reported acceptable validity and reliability for the self-efficacy in clinical performance questionnaire. Cronbach's alpha for the original instrument was 0.96, and in the present study, it was 0.98.

2) Self-leadership

Self-leadership refers to a self-influence perspective concerning leading oneself toward the performance of naturally motivating tasks as well as motivating oneself to do work that must be done but is not naturally motivating [11]. The revised self-leadership questionnaire consisted of 35 items based on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), with higher scores indicating higher self-leadership. The measure consisted of three dimensions of behavior-focused strategies, natural reward strategies, and constructive thought pattern strategies [12]. The dimension of behavior-focused strategies consisted of independent goal setting, self-reward, self-punishment, self-observation, and self-cueing. The dimension of natural reward strategies consisted of one subscale of focusing thoughts on natural rewards, and the dimension of constructive thought pattern strategies consisted of visualizing successful performance, self-talk, and evaluating beliefs and assumptions. The original study verified the validity and reliability of the problem-solving questionnaire [13]. The original instrument has the range of 0.74 to 0.93 for Cronbach's alpha for each subscale; in the present study, it was estimated to be 0.76 to 0.90.

3) Problem solving

Problem solving refers to the process of problem solving as it occurs in the natural environment or ‘real world’ [13]. The revised Social Problem-Solving Inventory is a 52-item, 5-point Likert-type inventory ranging from 1 (not at all true of me) to 5 (extremely true of me), and it consists of the five different dimensions of positive problem orientation, negative problem orientation, rational problem-solving style, impulsive/careless style, and avoidance style. High scores indicate a ‘good’ social problem-solving ability. The original study for testing the validity and reliability of the problemsolving questionnaire found it to be valid and reliable [13]. The original instrument has a Cronbach's alpha range of 0.67 to 0.92 for each subscale and a total score of 0.95; in the present study, it was estimated to be 0.69 to 0.88.
Independent validity studies for the Korean versions of the self-efficacy in clinical performance, self-leadership, and problem-solving scales were not conducted. Thus, we evaluated the face validity of the Korean version with two professors in nursing, one professor in education, and two undergraduate students. They found that it was appropriate to use. The scales that were used in the current study were translated from the English versions into Korean based on the guidelines for translation of self-report measures [14].

5. Statistical Analyses

The characteristics of the participants were analyzed as descriptive statistics (i.e., means, standard deviations [SDs], and frequencies). An independent t-test and chi-square test were used to test the homogeneity of the two groups at preintervention.
Group differences in changes in self-efficacy in clinical performance, self-leadership, and problem solving between pre- and post-intervention were identified using the t-test controlling for baseline scores, and the effect sizes for each p-value were calculated. SAS ver. 9.4 was used for all statistical analyses. A p-value of <0.05 was considered statistically significant, and in all analyses, two-sided p-values were applied.

III. Results

1. Demographics

The characteristics of the study participants are described in Table 2. The mean age was 22.5 years in both the traditional and flipped learning groups (p = 0.999). More than half of the participants had good to very good interpersonal relationship scores (p = 0.053), and most participants were satisfied with their choice of major (mean, 3.5; range 1–5) (p = 0.710). Therefore, there were no significant differences between sample characteristics of the traditional and flipped learning groups.

2. Homogeneity in Outcomes between the Traditional and Flipped Learning Groups on the Pre-test

The homogeneity of the outcomes of the two groups is displayed in Table 3. Almost all the outcome variables showed no significant difference pre-test, except the evaluating beliefs and assumptions indicators (t = 2.64, p = 0.009), which belonged to the subcategory of constructive thought pattern strategies of self-leadership.

3. Self-Efficacy in Clinical Performance

All subscales of self-efficacy in clinical performance showed improvement on the post-test for both groups but no statistically significant differences between groups in the preand post-test were found. For example, assessment was improved to 3.8 (SD = 13.4) for the traditional learning group and 4.8 (SD = 14.6) for the flipped learning group (p = 0.719, effect size [ES] = 0.06) after the intervention (Table 4).

4. Self-Leadership

Among the subscales of the behavior-focused strategies, the difference between the changes in the two groups concerning self-goal setting was significant (p = 0.043, ES = 0.27); the traditional learning group improved by 0.1 (SD = 1.1), and the flipped learning group improved by 0.9 (SD = 2.9) after the intervention.
Among the three subscales of the constructive thought pattern strategies, the evaluating beliefs and assumptions subscale improved significantly for the flipped learning group (mean difference = 0.8, SD = 2.5), while it decreased for the traditional learning group (mean difference = −0.2, SD = 1.3) (p = 0.023, ES = 0.38) after the intervention (Table 4).

5. Problem Solving

Among the subscales of problem solving, the flipped learning group (mean difference = 2.4, SD = 7.5) showed a significant improvement in rational problem solving in comparison to the traditional learning group (mean difference = 0.2, SD = 2.4) (p = 0.048, ES = 0.47). In addition, impulsive/careless style decreased for the flipped learning group (mean difference = −0.7, SD = 3.9), while it increased for the traditional learning group (mean difference = 0.4, SD = 3.8) after the intervention; however, no statistical significance was found (p = 0.152, ES = 0.43) (Table 4).

IV. Discussion

The purpose of this study was to evaluate the effect of the flipped learning approach on a surgical nursing practicum designed to enhance the self-efficacy performance, selfleadership, and problem-solving capacity of nursing students. The results of this study showed greater improvement in the goal-setting abilities, self-leadership beliefs, and capacity for rational problem solving of the flipped learning group in comparison to the traditional learning group. There is ample evidence that an active e-learning approach is positively associated with cognitive skills, including critical thinking [151617]. Several recent experimental studies in nursing have suggested that content knowledge and cognitive skills [18], student confidence in performing core nursing skills [19], student satisfaction and achievement [2021], and other learning outcomes [8] improved after participation in flipped learning nursing courses.
The goal-setting abilities of the flipped learning group improved more than did those of the traditional learning group. The time duration of exposure to the e-learning contents was longer for the flipped learning group than the traditional learning group because the e-learning activities of the flipped learning group started prior to the clinical practicum. This means that the flipped learning group was exposed to the e-learning objectives earlier than the traditional learning group. Because the flipped learning group already had the e-learning goals in their minds prior to attending the practicum classes, the flipped learning group may have been better focused on the course objectives during the limited class hours of the clinical practicum. Awareness of the lesson objectives during the e-learning activity is achieved through students' cognitive processes [22]. Flipped learning activities may increase student recognition of learning objectives through increased interaction with team members, curiosity and interest, and feedback-seeking behaviors to improve their academic performance. In addition, more opportunities for trial and error and clear feedback might have allowed the students in the flipped learning group to better recognize the e-learning objectives in comparison to the traditional learning group. The increased interaction and self-directed learning in the flipped learning group may have improved the cognitive understanding process, and it also might have encouraged the students to relate the current learning activity to positive thinking concerning their future goals.
The flipped learning group improved their rational problem-solving skills and reconsidered their own assumptions and beliefs. These abilities may have been improved by the integration of more team-based learning, which was permitted via the prior learning completed online by the flipped learning group [21]. When encountering situations that conflict with one's established knowledge, reconsidering one's own assumptions and beliefs is important in developing relative thinking skills and knowledge by reasonably accepting others' thoughts. When there is a disagreement with others, through developing relative thinking skills, students may try to evaluate whether their thinking or judgment is correct. In clinical practicums, relative thinking skills are important because the situations and conditions of patients are complex and diverse, and nursing students must use their problem-solving skills for such complex cases. Students in the flipped learning group were better able to participate in team-based learning activities during the on-site instruction because they had already completed the pre-learning, which also allowed them to more actively participate in discussions during the on-site instruction. They shared their prior learning and problems in knowledge establishment with their peers and professor during the onsite instruction. Through the team-based learning activity, the students in the flipped learning group continued the self-evaluation process and might have modified their assumptions and beliefs. Specifically, during the question and answer period during the on-site instruction, the students found that clinical scenarios do not have one definite answer. Rather, there are several potential answers that require more integrated and critical thought. Therefore, it is possible that the flipped learning method is more suitable for this characteristic of clinical nursing practicums focused on case scenarios, as this method is superior in strengthening the cognitive e-learning process and skills such as critical thinking [18]. From the perspective of informatics, the e-learning environments that were provided to the flipped learning group had many advantages. An e-learning environment is more flexible and accessible than face-to-face education [2223]. It is easier to create a customized learning environment that facilitates practice and self-directed learning through interactive feedback and that allows for a sufficient thinking process. Therefore, students in the flipped learning group might have more positive outcomes.
Although the current study makes a unique contribution to the field of clinical nursing education, it is not without limitations. First, the nursing students were recruited from only one university. This means that its findings are not generalizable. Thus, future studies in different nursing courses and different settings should address this issue. Second, although our surgical nursing practicum course was planned based on the suggested nine rules of design for flipped learning, it is possible that there are diverse application points of the rules according to the characteristics of different courses. Focused activities in the flipped learning method might differ for various courses. Therefore, our study represents a trial for surgical nursing practicums, while other trials may be required for other nursing courses. Finally, the overall statistical significance was weak; therefore, more statistically robust findings from further randomized controlled trials are required to confirm the differences between the effects of the flipped learning and traditional learning methods for surgical nursing practicums.
In conclusion, this study suggested that the flipped learning method in surgical nursing practicums could offer more effective e-learning opportunities concerning the aspects of goal setting, accepting others' thoughts and modifying thinking, rational problem-solving skills, and deliberation in the e-learning process in comparison to the traditional learning method. Flipped learning in surgical nursing practicums could provide the benefits of allowing the sharing of prior learning and experienced problems for knowledge establishment due to prior learning and interactivity as well as prompt feedback through team-based learning. Together, these findings suggest that the application of the flipped learning method might differ for various nursing courses.


This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MIST) (No. NRF-2017R1A1A1A05001045 and 2016R1C1B2013649).


Conflict of Interest: No potential conflict of interest relevant to this article was reported.


1. Kim MK, Kim SM, Khera O, Getman J. The experience of three flipped classrooms in an urban university: an exploration of design principles. Internet High Educ 2014;22:37-50.
2. Christiansen MA, Nadelson L, Etchberger L, Cuch M, Kingsford TA, Woodward LO. Flipped learning in synchronously-delivered, geographically-dispersed general chemistry classrooms. J Chem Educ 2017;94(5):662-667.
3. Rossi RD. ConfChem conference on flipped classroom: improving student engagement in organic chemistry using the inverted classroom model. J Chem Educ 2015;92(9):1577-1579.
4. Tolks D, Schafer C, Raupach T, Kruse L, Sarikas A, Gerhardt-Szep S, et al. An introduction to the inverted/flipped classroom model in education and advanced training in medicine and in the healthcare professions. GMS J Med Educ 2016;33(3):Doc46PMID: 27275511.
crossref pmid pmc
5. Davies RS, Dean DL, Ball N. Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educ Technol Res Dev 2013;61(4):563-580.
6. Bergmann J, Sams A. Flip your classroom: reach every student in every class every day. Arlington (VA): International Society for Technology in Education; 2012.
7. Strayer JF. How learning in an inverted classroom influences cooperation, innovation and task orientation. Learn Environ Res 2012;15(2):171-193.
8. Presti CR. The Flipped learning approach in nursing education: a literature review. J Nurs Educ 2016;55(5):252-257. PMID: 27115451.
crossref pmid
9. Njie-Carr VP, Ludeman E, Lee MC, Dordunoo D, Trocky NM, Jenkins LS. An integrative review of flipped classroom teaching models in nursing education. J Prof Nurs 2017;33(2):133-144. PMID: 28363388.
crossref pmid
10. Cheraghi F, Hassani P, Yaghmaei F, Alavi-Majed H. Developing a valid and reliable Self-Efficacy in Clinical Performance scale. Int Nurs Rev 2009;56(2):214-221. PMID: 19646171.
crossref pmid
11. Stewart GL, Courtright SH, Manz CC. Self-leadership: a multilevel review. J Manag 2011;37(1):185-222.
12. Houghton JD, Neck CP. The revised self-leadership questionnaire: testing a hierarchical factor structure for self-leadership. J Manag Psychol 2002;17(8):672-691.
13. D'Zurilla TJ, Nezu AM, Maydeu-Olivares A. Problem-solving Inventory Revised (SPSI-R). North Tonawanda (NY): Multi-Health Systems; 2002.
14. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25(24):3186-3191. PMID: 11124735.
crossref pmid
15. Burke SM. Cultivating critical thinking using virtual interactive case studies. J Pediatr Nurs 2017;33:94-96. PMID: 28139403.
crossref pmid
16. Choi J, Kim EG. Developing a teaching-learning model for flipped learning for institutes of technology and a case of operation of a subject. J Eng Educ Res 2015;18(2):77-88.
17. Martyn J, Terwijn R, Kek MY, Huijser H. Exploring the relationships between teaching, approaches to learning and critical thinking in a problem-based learning foundation nursing course. Nurse Educ Today 2014;34(5):829-835. PMID: 23706962.
crossref pmid
18. Park EO, Park JH. Quasi-experimental study on the effectiveness of a flipped classroom for teaching adult health nursing. Jpn J Nurs Sci 2017 8 18 [Epub]. PMID: 10.1111/jjns.12176.
crossref pmc
19. Kim HS, Kim MY, Cho MK, Jang SJ. Effectiveness of applying flipped learning to clinical nursing practicums for nursing students in Korea: a randomized controlled trial. Int J Nurs Pract 2017;23(5):e12574.
20. El-Banna MM, Whitlow M, McNelis AM. Flipping around the classroom: accelerated Bachelor of Science in Nursing students' satisfaction and achievement. Nurse Educ Today 2017;56:41-46. PMID: 28654815.
crossref pmid
21. Kim H, Jang Y. Flipped learning with simulation in undergraduate nursing education. J Nurs Educ 2017;56(6):329-336. PMID: 28585980.
crossref pmid
22. Lee MK. Effects of mobile phone-based app learning compared to computer-based web learning on nursing students: pilot randomized controlled trial. Healthc Inform Res 2015;21(2):125-133. PMID: 25995965.
crossref pmid pmc
23. Barak M, Rafaeli S. On-line question-posing and peerassessment as means for web-based knowledge sharing in learning. Int J Hum Comput Stud 2004;61(1):84-103.
Figure 1

Processes of the surgical nursing practicum for the traditional learning and flipped learning groups.

Table 1

Surgical nursing practicum based on the nine design principles of flipped learning

Table 2

Characteristics of the study participants


Values are presented as mean ± standard deviation or number (%).

aFisher exact test.

Table 3

Homogeneity of outcomes of the traditional learning and flipped learning groups on the pre-test


Values are presented as mean ± standard deviation.

Table 4

Group differences between changes in outcomes for the traditional learning and flipped learning groups from pre- to post-test


Values are presented as mean ± standard deviation.

An effect size of 0 to 0.3 is small. If it is between 0.3 and 0.6, it is moderate. An effect size greater than 0.6 is large.

Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 16 Web of Science
  • 21 Crossref
  • 21   Scopus
  • 5,065 View
  • 84 Download
Related articles in Healthc Inform Res


Browse all articles >

Editorial Office
1618 Kyungheegung Achim Bldg 3, 34, Sajik-ro 8-gil, Jongno-gu, Seoul 03174, Korea
Tel: +82-2-733-7637, +82-2-734-7637    E-mail:                

Copyright © 2024 by Korean Society of Medical Informatics.

Developed in M2community

Close layer
prev next