dissertation blended learning

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Psychol Res Behav Manag

A Systematic Review of Systematic Reviews on Blended Learning: Trends, Gaps and Future Directions

Muhammad azeem ashraf.

1 Research Institute of Education Science, Hunan University, Changsha, People’s Republic of China

Meijia Yang

Yufeng zhang, mouna denden.

2 Research Laboratory of Technologies of Information and Communication & Electrical Engineering (LaTICE), Tunis Higher School of Engineering (ENSIT), Tunis, Tunisia

Ahmed Tlili

3 Smart Learning Institute, Beijing Normal University, Beijing, People’s Republic of China

4 School of Professional Studies, Columbia University, New York City, NY, USA

Ronghuai Huang

Daniel burgos.

5 Research Institute for Innovation & Technology in Education (UNIR iTED), Universidad Internacional de La Rioja (UNIR), Logroño, 26006, Spain

Blended Learning (BL) is one of the most used methods in education to promote active learning and enhance students’ learning outcomes. Although BL has existed for over a decade, there are still several challenges associated with it. For instance, the teachers’ and students’ individual differences, such as their behaviors and attitudes, might impact their adoption of BL. These challenges are further exacerbated by the COVID-19 pandemic, as schools and universities had to combine both online and offline courses to keep up with health regulations. This study conducts a systematic review of systematic reviews on BL, based on PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, to identify BL trends, gaps and future directions. The obtained findings highlight that BL was mostly investigated in higher education and targeted students in the first place. Additionally, most of the BL research is coming from developed countries, calling for cross-collaborations to facilitate BL adoption in developing countries in particular. Furthermore, a lack of ICT skills and infrastructure are the most encountered challenges by teachers, students and institutions. The findings of this study can create a roadmap to facilitate the adoption of BL. The findings of this study could facilitate the design and adoption of BL which is one of the possible solutions to face major health challenges, such as the COVID-19 pandemic.

Introduction

Blended Learning (BL) is one of the most frequently used approaches related to the application of Information and Communications Technology (ICT) in education. 1 In its simplest definition, BL aims to combine face-to-face (F2F) and online settings, resulting in better learning engagement and flexible learning experiences, with rich settings way further the use of a simple online content repository to support the face-to-face classes. 2 , 3 Researchers and practitioners have used different terms to refer to the blended learning approach, including “brick and click” instruction, 4 hybrid learning, 4 dual-mode instruction, 5 blended pedagogies, 4 HyFlex learning, 6 targeted learning, 4 multimodal learning and flipped learning. 3

Researchers and practitioners have pointed out that designing BL experiences could be complex, as several features need to be considered, including the quality of learning experiences, learning instruction, learning technologies/tools and applied pedagogies. 7–9 Therefore, they have focused on investigating different BL perspectives since 2000. 10 Despite this 21-year investigation and research, there are still several challenges and unanswered questions related to BL, including the quality of the designed learning materials 9 , 11 , 12 applied learning instructions, 9 the culture of resisting this approach, 13 , 14 and teachers being overloaded when applying BL. 15 The COVID-19 pandemic has further highlighted the challenges associated with BL. Specifically, international universities and schools worldwide had to take several actions with respect to health regulations, such as reducing classroom sizes. 16 Therefore, they combined online and offline learning to maintain their courses for both on-campus and off-campus experiences. 16 For instance, as a response to the effort made by the government of Indonesia to carry out physical distancing during the COVID-19 pandemic, in all domains including education, some elementary schools used BL with Moodle platform to ensure the continuity of learning. 17 In this context, several teachers raised concerns about implementing BL experiences, such as the lack of infrastructure and competencies to do so, calling for further investigation in this regard. Several international organizations, such as UNESCO and ILO, claimed that teacher professional development for online and blended learning is one of the priorities for building resilient education systems for the future. 18

Based on the background above, it is seen that there is still room for discussion of designing and implementing effective BL. Researchers have suggested that conducting literature reviews can help identify challenges and solutions in a given domain. 19–21 Review papers may serve the development of new theories and also shape future research studies, as well as disseminate knowledge to promote scientific discussion and reflection about concepts, methods and practices. However, several BL systematic reviews were conducted in the literature which are of variable quality, focus and geographical region. This made the BL literature fragmented, where no study provides a comprehensive summary that could be a reference for different stakeholders to adopt BL. In this context, Smith et al mentioned that a logical and appropriate next step is to conduct a systematic review of reviews of the topic under consideration, allowing the findings of separate reviews to be compared and contrasted, thereby providing comprehensive and in-depth findings for different stakeholders. 22 As BL is becoming the new normal, 23 this study takes a step further beyond simply conducting a systematic review and conducts a systematic review of systematic reviews on BL. By systematically examining high-quality published literature review articles, this study reveals the reported BL trends and challenges, as well as research gaps and future paths. These findings could help different stakeholders (eg, policy makers, teachers, instructional designers, etc.) to facilitate the design and adoption of BL worldwide. Although several systematic reviews of literature reviews have been conducted in different fields, such as engineering, 24 healthcare 25 and tourism, 26 no one was conducted on blended learning, to the best of our knowledge. It should be noted that one study was conducted in this context, but it mainly focused on the transparency of the systematic reviews that were conducted 27 and was not about the BL field itself.

Guided by the technology-based learning model (see Figure 1 ), this study aims to answer the following six research questions:

Blended learning model.

RQ1. What are the trends of blended learning research in terms of: publication year, geographic region and publication venue?

RQ2. What are the covered subject areas in blended learning research?

RQ3. Who are the covered participants (stakeholders) in blended learning research?

RQ4. What are the most frequently used research methods (in systematic reviews) in blended learning research?

RQ5. How blended learning was designed in terms of the used learning models and technologies?

RQ6. What are the learning outcomes of blended learning, as well as the associated challenges?

The findings of this study could help to analyze the behaviors and attitudes of different stakeholders from different BL contexts, hence draw a comprehensive understanding of BL and its impact from different international perspectives. This can promote cross-country collaboration and more open BL design that more worldwide universities could be involved in. The findings could also facilitate the design (eg, in terms of the used learning models and technologies) and adoption of BL which is one of the possible solutions to face major health challenges, such as the COVID-19 pandemic.

Methodology

This study presents a systematic review of systematic review papers on BL. In particular, this review follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. 28 PRISMA provides a standard peer-accepted methodology that uses a guideline checklist, which was strictly followed for this study, to contribute to the quality assurance of the revision process and to ensure its replicability. A review protocol was developed, describing the search strategy and article selection criteria, quality assessment, data extraction and data analysis procedures.

Search Strategy and Selection Criteria

To deal with this topic, an extensive search for research articles was undertaken in the most common and highly valued electronic databases: Web of Science, Scopus and Google Scholar, 29 using the following search strings.

Search string: ((blending learning substring) AND (literature review substring))

Blended learning substring: “Blended learning” OR “blended education” OR “hybrid learning” OR “flipped classroom” OR “flipped learning” OR “inverted classroom” OR “mixed-mode instruction” OR “HyFlex learning”

Literature review substring: “Review” OR “Systematic review” OR “state-of-art” OR “state of the art” OR “state of art” OR “meta-analysis” OR “meta analytic study” OR “mapping stud*” OR “overview”

Databases were searched separately by two of the authors. After searching the relevant databases, the two authors independently analyzed the retrieved papers by titles and abstracts, and papers that clearly were not systematic reviews, such as empirical, descriptive and conceptual papers, were excluded. Then, the two authors independently performed an eligibility assessment by carefully screening the full texts of the remaining papers, based on the inclusion and exclusion criteria described in Table 1 . During this phase, disagreements between the authors were resolved by discussion or arbitration from a third author. Specifically, to provide high-quality papers, this study was restricted to papers published in journals.

Inclusion and Exclusion Criteria

This research yielded a total of 972 articles. After removing duplicated papers, 816 papers remained. 672 papers were then removed based on the screening of titles and abstracts. The remaining 144 papers were considered and assessed as full texts. 85 of these papers did not pass the inclusion criteria. Thus, as a total number, 57 eligible research studies remained for inclusion in the systematic review. Figure 2 presents the study selection process as recommended by the PRISMA group. 28

Flowchart of the systematic review process.

Quality Assessment

For methodological quality evaluation, the AMSTAR assessment tool was used. AMSTAR is widely used as a valuable tool to evaluate the quality of systematic reviews conducted in any academic field. 30 It consists of 11 items that evaluate whether the review was guided by a protocol, whether there was duplicate study selection and data extraction, the comprehensiveness of the search, the inclusion of grey literature, the use of quality assessment, the appropriateness of data synthesis and the documentation of conflicts of interest. Specifically, two authors independently assessed the methodological quality of the included reviews using the AMSTAR checklist. Items were evaluated as “Yes” (meaning the item has been properly handled, 1 point), “No” (indicating the possibility that the item did not perform well, 0 points) or “Not applicable” (in the case of performance failure because the item was not applied, 0 points). Disagreements regarding the AMSTAR score were resolved by discussion or by a decision made by a third author.

Appendix 1 presents the results of the quality assessment of the 57 systematic reviews based on the AMSTAR tool. 19 were rated as being low quality (AMSTAR score 0–4), 30 as being moderate quality (score 5–8), and eight as being high quality (score 9–11). Specifically, no study has acknowledged the conflict of interest in both the systematic review and the included studies. Also, few studies provided the list of the included and excluded studies (3 out of 57), and reported the method used to combine the findings of the studies (13 out of 57). About half of the included studies assessed the scientific quality of the included studies (25 out of 57), but all the studies fulfilled at least one quality criterion.

Data Extraction

This study adapted the technology-based learning model, 31 which has been used in BL contexts, 32 , 33 as shown in Figure 1 . This model is based on six factors: subject area, learning models, participants, outcomes and issues, research methods and adopted technologies. The current study adopted most of the schemes from this model but made slight adjustments according to the features of different models in blended learning. Table 2 presents a detailed description of the coding scheme that was used in this study to answer the aforementioned research questions.

The Coding Scheme for Analyzing the Collected Papers

Results and Discussion

Blended learning trends.

Figure 3 shows that the first two systematic reviews on BL were conducted in 2012. The first, by Keengwe and Kang, 34 investigated the effectiveness of BL practices in the teacher education field. The second was by Rowe et al, 35 which investigated how to incorporate BL in clinical settings and health education. These findings show an early interest in providing teachers with the necessary competencies and skills to use BL, as well as in enhancing health education, where students need more practical knowledge and skills that could be facilitated through BL (eg, simulation health videos, virtual labs, etc.). The number of literature reviews conducted has since increased, showing an increased interest in BL over the years. Specifically, the highest peak of literature reviews conducted on blended learning was in 2020 (16 studies). This might be due to the COVID-19 pandemic, which forced most institutions worldwide to implement BL (online merged with offline) to accommodate the needs of learners in this disruptive time. 18 This has encouraged many institutions to make their own attempts to practice BL and thus furthered the research interest in examining the best practices of BL.

Distribution of studies by publication year.

Additionally, according to the authors’ affiliation countries (see Figure 4 ), China and the United States have the highest number of publications, with nine and seven studies respectively. This could be explained by the continuous rapid evolution of the technological education industry in both China and the United States, 36 which has made researchers and educators innovate to provide more flexible learning experiences by combining both online and offline environments. 37 This could also be explained by the number of blended learning policies that have been issued in these two countries to facilitate blended learning adoption. 38 , 39

Distribution of studies per country.

Interestingly, while several studies are from Europe (eg, Belgium, the UK, Italy, etc.), there are very few studies from the African and Arab regions. Similarly, in BL contexts, Birgili et al 40 conducted a systematic review about flipped learning between 2012 and 2018; they found very few studies coming from Africa. This indicates a trend where countries with more sufficient educational resources and infrastructure are exposed to more chances to develop BL environments and experiences. These findings call for more cross-country collaboration to facilitate the implementation of BL in the countries that have limited knowledge or infrastructure related to BL. For instance, such a collaboration could cover BL policies, ICT trainings and the development of educational resources and technologies.

Finally, the 57 reviews were published in 44 journals. Figure 5 shows the journals that have at least two publications. Education and Information Technologies has the highest number of publications (six studies), followed by Interactive Learning Environments (four studies) and Nurse Education Today (four studies). These journals are mostly from the educational technology and health fields.

Distribution of studies by publication venue.

Subject Area

Figure 6 shows that most of the literature review studies (n = 21) did not mention the covered subject area and discussed BL in general. For example, Wang et al proposed a complex adaptive systems framework to conduct analysis on BL literature. 41 This shows that, despite the popularity of BL, which has existed for a decade, educators and researchers are still finding it to be a complex concept that needs further investigation regardless of the subject. 2

Distribution of studies by subject area.

Other studies considered BL as being context-dependent, 42 investigating it from different subject areas, namely health (14 studies), STEM (five studies) and language (three studies). This could be explained by these three subjects requiring a lot of practical knowledge, such as communication and pronunciation, programming or physical treatments, where the BL concept could provide teachers with a chance to be more innovative and offer students the possibility of practicing this practical knowledge online by using virtual labs or online virtual programming emulators, for instance. Walker et al 43 and Yeonja et al 44 found that BL is considered to be crucial for health students, and health educators have tried to integrate a wide range of advanced technology and learning tools to enhance their skill acquisition.

From these findings, it can be deduced that more research should be conducted to investigate how BL is conducted in other subject areas that are considered crucial for student performance assessment, such as mathematics. This could help researchers and practitioners compare the different BL design and assessment approaches in different subjects and come up with personalized guidelines that could help educators implement their BL in a specific subject. In this context, studies have pointed out that teachers are willing to implement BL in their courses but do not know how. 45 Additionally, as shown in Figure 6 , most of the conducted literature reviews covered limited number of studies (less than 50). Therefore, the future literature reviews on BL should cover more studies (more than 50) to have an in-depth and broad view of how BL is being implemented in different contexts by different researchers.

Participants

As Figure 7A shows, the most targeted participants by the review studies were students (n = 42) followed by teachers (n = 13) and then working adults, health professionals and researchers (one study for each). This analysis shows that none of the review studies have targeted major players in the adoption of BL, such as policy makers. Owston stated that policies on different levels (eg, institutions, faculties, technology use, data collection procedures, learning support, etc.) are crucial to advancing the adoption of BL for future education. 38 Therefore, to advance BL adoption worldwide, more reviews about BL policies and the focus of these policies – including copyright, privacy and data protection, and others, 46 , 47 – should be investigated to develop a BL policy framework to which everyone could refer.

( A ) Distribution by educational level. ( B ) Distribution by participants.

Figure 7B , on the other hand, shows that most of the review studies (n = 33) focused mainly on higher education, followed by K–12 (six studies) and teacher education (five studies). Interestingly, these findings are in line with two older studies that were conducted in 2012 (Halverson et al) 48 and 2013 (Drysdale et al), 49 where they found that BL is mostly applied in higher education. These findings clearly show that, despite the long period of time since 2012, the research setting of BL application has not changed, which calls for more serious efforts and research about BL design in other contexts, such as K–12. Especially since younger students might lack the appropriate self-regulation skills compared to older students that can help them adopt BL, 50 more support should also be provided accordingly. Additionally, as few studies focused on teacher education, more research should investigate how to harness the power of BL for teacher professional development. There are limited empirical findings on BL for teacher professional development, 34 , 51–53 calling for more investigation in this context.

Research Method

Table 3 shows that most reviews conducted were systematic reviews (n = 47). As researchers note, systematic literature reviews are usually composed with a clearly defined objective, a research question, a research approach and inclusion and exclusion criteria. 54–56 Through systematic review, researchers can come to a qualitative conclusion in response to their research question. Only seven reviews conducted meta-analysis to assess the effect size and variability of BL and to identify potential causal factors that can inform practice and further research. Finally, three studies used both systematic reviews and meta-analysis in their studies, which can quantitatively synthesize the results in an even more comprehensive way. For instance, Liu et al 57 first reviewed the literature of the effectiveness of knowledge acquisition in health-subject learners and then conducted a meta-analysis to show that BL had a significant larger pooled effect size than non-BL health-subject learners. In this way, researchers are able to address the extent to which BL is truly effective in the learning. 57 Considering that only three review papers conducted both systematic review and meta-analysis, we must again address the usefulness of quantitative analysis. There are still many unanswered questions that could be addressed in a better way using quantitative analysis. Therefore, future research should consider conducting more meta-analysis in order to provide a better understanding of the nuanced effects of BL.

Distribution of Studies by Research Method and Subject Area

Design (Learning Models and Technologies)

Figure 8 shows that the majority of review studies (33 out of 57) discussed BL as a generic concept and did not mention any specific model. Additionally, the flipped model was the most frequently implemented model, mentioned by 27 review studies. This model is designed based on three stages: pre-class, in-class and post-class (optional). In the pre-class stage, the students engage with the course content through online resources, so that they spend in-class time doing practical activities and having discussions. Then, in the post-class stage, teachers can assess the students’ perceptions and performance in the flipped course. 32

Frequency of usage of blended learning models.

The second most frequently used models were the station rotation model and the flex model (each mentioned by three studies). In the station rotation model, the student can rotate at fixed points of time (on a fixed schedule or at the teacher’s discretion) between different stations, at least one of which is an online learning station). 58 For instance, the students can rotate between face-to-face (F2F) instruction, online instruction and collaborative activities. The flex model, on the other hand, relies entirely on online materials and student self-study, with the availability of F2F teachers when needed. 59

Two review studies mentioned the self-blend (also known as the “à la carte” model) and the enriched virtual model. The first model allows students to take fully online courses with online teachers, in addition to other F2F courses. 60 In the second model, students are asked to be able to conduct F2F sessions with instructors and then can complete their assignments online, but they are not required to attend F2F classes. 60

Finally, only one study applied the mixed model, supplemental model and online practicing model. Specifically, in the mixed model, content delivery and practical activities occur both F2F and online. In the supplemental model, both content delivery and practical activities take place F2F. In contrast, in the online practicing model, students can practice activities through a specific online learning environment. In particular, the reported BL models were implemented differently in many domains. It should be noted that some studies investigated more than one BL model. For instance, Alammary investigated flipped, mixed, flex, supplemental, and online-practicing models. 59

Table 4 presents the distribution of the reviewed studies by BL models and subject areas. 22 studies (seven multiple courses and 15 uncategorized) have focused on the design of BL in general or in multiple courses. This might be explained by the fact that teachers have limited knowledge about BL models that is why they always face challenges on how to design their blended courses and mix the offline and online settings. 58 This blended learning design problem was further emphasized during the COVID-19 pandemic, where several teachers raised concerns about this matter. 61 Therefore, more BL design trainings should be provided for teachers to help them efficiently design their blended courses.

Distribution of Studies by Blended Learning Models and Subject Area

Additionally, the flipped model was frequently used in health (seven studies), followed by STEM (five studies). This may be explained by health and STEM subjects requiring many hands-on practices to promote skill acquisition and long-term retention by the student. 62 , 63 In line with this, the flipped model enables teachers to reduce the in-class time by teaching all the courses online (in the pre-class stage) and counterbalance the students’ workload, so that the class time can be reserved for practical exercises instead of traditional lectures. For instance, in the health domain, the flipped model is applied by explaining the basic concepts of the course using different learning strategies in the pre-class stage, such as online learning platforms, instructional videos, animation, PowerPoint presentations and 3D virtual gaming situations. Also, students can use social media platforms such as Facebook for online discussions. In-class activities include instructor-led training, discussion of issues, practice or doing exercises (eg, assignments or quizzes), clinical teaching (eg, nursing diagnosis training) or lab teaching. In this context, several learning technologies were used, such as traditional computers and projectors, medical or teaching equipment and simulation teaching aids. Finally, in the post-class stage, some teachers used assessment methods to evaluate students’ perception of the applied model using peer evaluation, post-class evaluation and surveys. Similarly, in STEM subjects, the in-class time was reserved for more practice, including complex exercises where students can interact with each other and with the instructor (collaborative group assignments), active learning exercises rather than lectures, gaming activities, examinations and peer instruction.

Furthermore, as Table 4 shows, the mixed, flex, supplemental and online practicing models were also applied in STEM, specifically in programming courses. 59 This may be explained by the fact that STEM subjects – and programming courses in particular – allow for flexibility; combined with emerging technologies, this enables the teaching of this course in different ways, fully online or F2F. 64 For instance, in the mixed model, students received the course content and practical coding exercises in both F2F and online sessions, reserving most of the in-class time for practical exercises and discussion. In this context, in addition to the classical learning strategies such as online self-paced learning, online collaboration and online instructor-led learning, online programming tools were also used for coding and problem solving in online sessions. In the flex model, both course content and practical coding exercises take place online, but students are required to attend F2F sessions from time to time to check their progress or be provided with feedback. In the supplemental model, both course content and practical coding exercises take place F2F. However, online supplemental activities are added to the course to increase students’ engagement with course content. In the online practicing model, an online programming environment is used as the backbone of students’ learning. It allows students to practice programming and problem solving and provides them with immediate feedback about their programming solutions. The delivery of the course content is achieved through lectures and/or self-based online resources. In some cases, online resources are integrated within the online programming environment.

Outcomes and Challenges

Figure 9 presents the different learning outcomes investigated in the 57 review studies based on two categories: psychological and behavioural outcomes. 65 The majority of studies (49 studies) focused on investigating the psychological outcomes within the reviewed studies. Specifically, students’ self-regulation toward learning was the most investigated psychological outcome (10 studies), followed by satisfaction (nine studies) and engagement (eight studies). According to Van Laer and Elen, blended learning design includes attributes that support self-regulation, including authenticity, personalization, learner control, scaffolding and interaction. 66 The 10 studies found that students’ self-regulation was improved. Additionally, BL was found to improve students’ satisfaction and engagement in different domains, especially in health (seven studies). For instance, Li et al 67 and Presti 68 found that flipped learning enhanced students’ engagement and satisfaction in nursing education. Moreover, motivation, attitude, high-order thinking, social interaction and self-efficacy were found to be improved using BL.

Distribution of learning outcomes based on the number of studies addressing them.

The most investigated behavioural outcome is academic performance (26 studies), followed by skill progression and cooperation. In particular, the 26 studies showed that BL supports learning performance in different subject areas, including health, language and STEM. BL can also enhance students’ skills, such as clinical skills in the health domain, 35 , 69 and speaking skills in the language domain. 70 Additionally, its design may include several collaborative learning assignments (online or F2F) that encourage cooperation with students. 71 It should be noted that some studies investigated more than one type of learning outcomes. For instance, Atmacasoy and Aksu investigated students’ engagement with, collaboration in, participation in and academic performance with the blended learning course. 72

Despite the many advantages that BL offers, it also comes with several challenges. Figure 10 presents the most encountered challenges in the 57 review studies. Specifically, the lack of ICT skills is the most mentioned challenge (seven studies), followed by infrastructure issues (six studies) such as internet-related problems and lack of personal computers, course preparation time (three studies), design model and cost of technologies (two studies for each) and course quality content, student engagement and student isolation (one study for each). It should be noted that 47 studies did not mention any challenges and nine studies mentioned more than one challenge each. For instance, Rasheed et al found that students, teachers and institutions may face different challenges in BL, such as students’ isolation, lack of ICT skills for teachers and students and technological provision challenges (eg, cost of online learning technologies) for institutions. 73

Distribution of blended learning challenges.

Both teachers and students from different domains might lack ICT skills, which can negatively influence their adoption of BL. For instance, Atmacasoy and Aksu stated that teachers with low ICT skills may not have positive attitudes toward using BL since it is based on technology use. 72 Teachers might find difficulties in the ease of use of some technologies while creating a BL course, such as in recording videos, uploading videos and using online learning platforms. 73 Additionally, students may face some technological complexity challenges, such as accessing online educational resources or uploading their materials to the online learning environment. 73

ICT infrastructure is also a crucial layer for facilitating and implementing blended courses; however, it is still a major problem for several universities, especially in developing countries 74 and rural areas. 75 For instance, a lack of basic technologies, including internet, computers and projectors can limit the implementation of blended courses. Therefore, it is very important to improve institutes’ ICT infrastructure in order to improve education in general and enable teachers to teach using BL, which has proven to be efficient in many subject areas (see sections above).

In addition to issues with ICT skills and infrastructure, teachers may lack knowledge about designing BL models and hence face difficulties in selecting the appropriate design for their courses, 58 and they may also spend too much time preparing the blended course. 75 , 76 Moreover, some challenges of online learning, such as engagement and students’ isolation, may be faced in BL. In this context, teachers may integrate online collaborative assignments to solve the problem of isolation 77 and integrate new approaches, such as gamification, into the online learning environment in order to make students motivated and engaged while learning online. 78 , 79 In this context, Ekici found that gamified flipped learning enhanced students’ motivation and engagement while learning. 80

This study conducted a systematic review of systematic reviews on BL. It revealed several findings according to each research question: (1) the first two systematic reviews on BL were conducted in 2012, and this number rapidly increased over the years, reflecting a massive interest in BL. Additionally, more cross-country collaboration should be established to facilitate BL implementation in countries that lack, for instance, infrastructure or the needed BL competencies; (2) despite that several studies focused on specific subject area such as health or STEM, most studies did not discuss BL from a specific subject area; (3) most of the studies targeted students as stakeholders, and neglected major key players for BL adoption, such as policy makers; (4) most of the studies conducted a systematic review with qualitative analysis. Therefore, future research should follow a more quantitative approach through meta-analysis in order to provide a better understanding of the nuanced effects of BL; (5) the majority of studies discussed BL as a generic construct and did not focus on the learning models of BL. However, the flipped model was the most frequently implemented model in the papers that focused on learning models specifically in health and STEM ; and (6) BL can affect students’ psychological and behavioural outcomes. In terms of psychological outcomes, it can enhance students’ self-regulation toward learning, satisfaction and engagement while learning in different domains, especially in health. In terms of behavioural outcomes, BL supported students’ academic performance in different subject areas. Additionally, a lack of ICT skills and infrastructure are the most encountered challenges by teachers, students and institutions.

The findings of this study can help create a roadmap about future research on BL. This could facilitate BL adoption worldwide and thus contribute to achieving the UN Sustainable Development Goals (SDGs), especially SDG #4 – equity and high-quality education for all – which works as a backbone for some other SDGs, such as good health (#3), economic Growth (#8) and reduced inequality (#10). Despite the importance of the revealed findings, this study has several limitations that should be acknowledged. For instance, this study used a limited number of search keywords within specific electronic databases.

Future research might focus on: (1) dealing with these limitations; (2) investigating different BL models with specific application domains to test their impacts on students’ psychological and behavioural outcomes; (3) enhancing students’ motivation and engagement in online sessions by integrating new motivational concepts such as gamification in online learning platforms; and (4) dealing with BL challenges by providing some solutions to enhance the learning experience. For instance, for the challenge of a lack of ICT skills, research might work to provide ICT trainings for teachers and students to enhance their skills with technology.

Acknowledgments

The study was supported by the National Natural Science Foundation of China (The Research Fund for International Young Scientists; Grant No. 71950410624). However, any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Natural Science Foundation of China.

The authors report no conflicts of interest in this work.

CURRICULUM, INSTRUCTION, AND PEDAGOGY article

Students' perceptions of a blended learning environment to promote critical thinking.

• School of Foreign Languages, Northeast Normal University, Changchun, China

Critical thinking is considered as one of the indispensable skills that must be possessed by the citizens of modern society, and its cultivation with blended learning has drawn much attention from researchers and practitioners. This study proposed the construction of a blended learning environment, where the pedagogical, social, and technical design was directed to fostering critical thinking. The purpose of the study was to find out students' perceptions of the learning environment concerning its design and its influence on their critical thinking. Adopting the mixed method, the study used questionnaire and interview as the instruments for data collection. The analysis of the data revealed that the students generally held positive perceptions of the environment, and they believed that the blended learning environment could help promote their critical thinking in different aspects.

Introduction

The development of critical thinking has drawn attention of the education ministries and institutions of different levels in countries all over the world. In the last two decades, researchers and practitioners have been exploring the ways to integrate critical thinking cultivation into the instruction of different disciplines, proposing strategies and interventions to promote critical thinking, among which blended learning has been widely recognized (e.g., Van Gelder and Bulker, 2000 ; Gilbert and Dabbagh, 2005 ; Yukawa, 2006 ). Blended learning is proposed as focusing on optimizing achievement of learning objectives by applying the “right” personal learning technologies to the “right” person at the “right” time and “right” place ( Singh, 2003 ). A blended learning environment, integrating the advantages of the e-learning method and traditional method, is believed to be more effective than a face-to-face or online learning environment alone ( Kim and Bonk, 2006 ; Watson, 2008 ; Yen and Lee, 2011 ). Studies have been conducted to construct blended learning environments to improve students' critical thinking. Most of them, however, adopted standardized tests or coding schemes to examine the effectiveness of the learning environments on students' critical thinking ( Chou et al., 2018 ), paying less attention to students' perceptions and attitudes. Therefore, the purpose of the current study is to address this gap.

Critical Thinking

There are a vast number of definitions of critical thinking in the literature (e.g., Paul, 1992 ; Ennis, 1996 ; Fisher and Scriven, 1997 ). Despite the emphasis on different aspects, the core of critical thinking entails taking charge of one's thinking to improve it. Paul and Elder's definition and model of critical thinking were adopted in the study. According to Elder and Paul (1994) , critical thinking refers to “the ability of individuals to take charge of their own thinking and develop appropriate criteria and standards for analyzing their own thinking” (p. 34). They proposed that critical thinking is composed of three dimensions: elements of thinking, intellectual standards, and intellectual traits. People demonstrate critical thinking when they use intellectual standards (clarity, precision, accuracy, importance, relevance, sufficiency, logic, fairness, breadth, depth) to measure elements of thinking (purposes, assumptions, questions, points of view, information, implications, concepts, inferences) ( Paul and Elder, 1999 ).

Critical Thinking Cultivation With Information Communication Technology Tools

Studies applying ICT tools to cultivate critical thinking have been increasingly emerging in the literature. The systematic review conducted by Chou et al. (2018) analyzed and reported the trends and features of critical thinking studies with ICT tools. According to the findings of the review, the most often used tools include online discussion (e.g., Cheong and Cheung, 2008 ), coding or game design or Wikibooks creation (e.g., Yang and Chang, 2013 ), and concept or argument maps (e.g., Rosen and Tager, 2014 ). As for the method involved, the studies adopted both quantitative and qualitative research methods (e.g., Shamir et al., 2008 ; Yang, 2008 ; Yang and Chou, 2008 ; Butchart et al., 2009 ; de Leng et al., 2009 ; Yeh, 2009 ). Data from various measurements revealed overall positive results of using ICT tools in critical thinking cultivation (e.g., Yang, 2008 ; Allaire, 2015 ; Shin et al., 2015 ; Huang et al., 2017 ). The findings of the systematic review showed that the critical thinking-embedded activities using ICT tools were more effective than face-to-face activities in developing students' critical thinking ( Guiller et al., 2008 ; Adam and Manson, 2014 ; Eftekhari et al., 2016 ). However, students' prescriptions of the learning design or critical thinking development have not been fully addressed in the literature.

Blended Learning Environment

The concept of blended learning has been defined by several researchers and scholars. For instance, Singh and Reed (2001) defined blended learning as a learning program where more than one delivery mode is being used to optimize the learning outcome and cost of program delivery. According to Thorne (2003) , blended learning is a way of “meeting the challenges of tailoring learning and development to the needs of individuals by integrating the innovative and technological advances offered by online learning with the interaction and participation offered in the best of traditional learning” (p. 2). The above definitions indicate that blended learning can combine the advantages of both traditional face-to-face learning and e-learning and avoid the drawbacks of the two learning modes. The effectiveness of blended learning has been demonstrated by many studies, for example, the findings of a meta-analysis have shown that blended learning brings more positive impact on students learning than online and face-to-face learning ( BatdÄ, 2014 ). Despite the merits of blended learning itself, the effectiveness is determined by the proper design. How to achieve the equilibrium between e-learning and face-to-face modes is crucial to the success of the blended learning environment ( Osguthorpe and Graham, 2003 ).

This study applied the PST model developed by Wang (2008) as the framework for the environment design. As Kirschner et al. (2004) pointed out, an educational system is a unique combination of pedagogical, social, and technological components. PST model thus consists of three key components: pedagogy, social interaction, and technology. According to Wang (2008) , the pedagogical design involves the selection of appropriate content, activities, and the way to use the resources; the social design refers to the construction of a safe and comfortable environment where learners can share and communicate; the technical design provides learners with a technical space of availability, easy access and attractiveness. In any learning environment, the three components play different roles. The technical design offers a basic condition for pedagogical and social design, while the pedagogical and social design is considered as the most important factor that influences the effectiveness of learning ( Wang, 2008 ).

Perceptions of Blended Learning Environment

It has been acknowledged that students' perceptions and satisfaction are important for determining the quality of blended learning environment ( Naaj et al., 2012 ). Studies have been conducted to examine students' views regarding a blended learning environment and factors influencing it. For example, Bendania (2011) study found that students hold positive attitudes toward the blended learning environment and the influencing factors mainly include experience, confidence, enjoyment, usefulness, intention to use, motivation, and whether students had ICT skills. The positive view was also reported in the study done by Akkoyunlu and Yilmaz (2006) , and it was found to be closely related to students' participation in the online discussion forum. Findings from other studies (e.g., Dziuban et al., 2006 ; Owston et al., 2006 ) also revealed students' positive attitudes toward the blended learning environment, and the satisfaction could be attributed to features like flexibility, convenience, reduced travel time, and face-to-face interaction. Some studies, however, reported some negative perceptions of the blended learning environment. For example, the results of the study of Smyth et al. (2012) showed that the delayed feedback from the teacher and poor connectivity of the internet were perceived as major drawbacks of the environment. In another study conducted by Stracke (2007) , lack of reciprocity between traditional and online modes, no use of printed books for reading and writing, and use of the computer as a medium of instruction was considered as major reasons for students withdraw from the blended course. These findings indicate that students' negative attitudes toward the blended learning environment mainly come from the inadequate design ( Sagarra and Zapata, 2008 ).

The review of the above studies indicates that applying ICT tools to cultivate critical thinking has gained much popularity and produced positive results. Few studies, however, focus on students' perceptions of a learning environment designed to promote critical thinking despite the fact that many studies have been conducted to explore students' perceptions of a blended learning environment in general. Therefore, the purpose of the current research is to investigate students' perceptions of a blended learning environment with the orientation of critical thinking development.

Research Design

Research questions.

By adopting the mixed method, this study aims to answer the following two questions:

1. What are students' perceptions of the blended learning environment to promote critical thinking?

2. How do students perceive the impact of the blended learning environment on the development of their critical thinking?

Context and Participants

The study was carried out in the course of Practical English Writing which is a branch of the comprehensive English course for first-year non-English majors at a Normal University in mainland China. The 6-week course adopted a mixed learning mode of classroom face-to-face and online learning. The face-to-face class ran once a week and each class was 90 min. The e-learning tasks were assigned either before or after the class. Six independent learning centers with networked computers were available for students to use and the whole campus was covered with Wi-Fi signal.

The participants of the study involved a total of 90 non-English major students (33 males and 57 females) aging from 18 to 20 in 2020. The students were allocated into classes of Level A after the placement test of English proficiency, which means their English was about higher intermediate level. Adopting the International Critical Thinking Reading and Writing Test ( Paul and Elder, 2006 ), which was developed from Paul and Elders' thinking model, the study assessed students' critical thinking level at the beginning of the course and found that the students' overall critical thinking level was at the lower medium level. But their information literacy level was sufficient to cope with the online platform and the software in the blended learning environment. Before the implementation of the course, the instructor informed the students about the study, and the consent forms were signed by the students.

Environment Design

For the learning environment to achieve the purpose of developing learners' critical thinking, its structural components should be designed to provide favorable conditions for critical thinking cultivation. A systematic review conducted by Lu (2018) has identified a series of favoring conditions that could promote the students' critical thinking, which include (a) critical thinking as one of the teaching objectives, (b) tasks involving the operation of ideas, (c) authentic context, (d) rich and diversified resources, (e) interaction and collaboration, (f) scaffolding and guidance, (g) communicative tools. These conditions were mapped to the design of the components of the PST learning environment model and the designing strategies were generalized from the instruction practice to guide the detailed design of the environmental components.

Pedagogical Design

In terms of the pedagogical design, the thinking skills that can be cultivated were first decided according to the particular learning content. Aiming at promoting the thinking skills, the learning tasks which mostly introduced problems in the “real” context and involve the operation of ideas were designed. Furthermore, rich and diversified resources were provided to the students. The specific strategies of pedagogical design are listed in Table 1 .

Table 1 . Strategies for pedagogical design.

When designing the learning objectives of the activities, the basic concepts and frameworks of critical thinking were introduced to the students, making them aware of its meaning and significace. Furthermore, students were informed of the thinking skills targeted and their importance. When students associated the thinking skills with the tasks, they would try to use the skills to accomplish them.

In order to enable tasks to involve more operations of ideas, writing, discussion, and evaluation activities were given the priority to provide more opportunities for students to communicate with each other and reflect upon their ideas. Besides, the topics of these activities were chosen to induce more collision of ideas. For example, in learning to write complaint letters, students were assigned the roles of customers who made the complaints and the managers who responded to the complaints. In such an activity, students could realize the existence of different perspectives and think more adequately and deeply.

The creation of a relatively real context drew on the following two strategies: One is to provide sufficient details. In the case of the job application writing, details such as the information about the potential employer were provided to the students so that they could consider themselves as “real” potential employees. The other strategy is to create interesting situations. The contexts described were usually attractive to the students, which helped arouse students' interest in completing the tasks.

With the purpose of collecting sufficient and diversified resources, both traditional and online media were included. Since the materials in the textbook are rather limited, the relevant online resources would make complementation for students to have sufficient resources to deal with. To meet the multi-angle nature of resources, the information collected came from different positions and perspectives. For instance, the students were introduced to the websites both for job hunting and recruitment so that they could read information from the perspectives of both employers and potential employees. To help students conduct resource searches by themselves, online resources such as the Online Writing Lab of Purdue University were presented to them to conduct searches. The search was usually directed by a clear question or a problem, and students needed to accurately identify the target source. Some search engines were also introduced to the students, enabling them to compare and select the relevant resources. Students needed to first define what their search objectives were, then assess the search and query results one by one, and finally synthesize the resources to make a reasonable decision.

Social Design

With the purpose of cultivating students' critical thinking in the environment, interactions and collaborations of different types were stressed in the design (see Table 2 ). Furthermore, the scaffold and guidance from the teacher and the peer were designed to provide support to the students.

Table 2 . Strategies for social design.

In designing interaction and collaboration-rich community, the strategies were applied to target both student-student and student-teacher communities. In terms of student-student community, students were grouped according to their levels and the requirements of the activities. Specifically, in a demanding task, students of different academic levels were grouped to ensure the implementation. In a relatively free discussion, students were grouped according to their own will so that they could feel more comfortable sharing their ideas. Also, various types of interactions such as information exchange, discussion, debate were designed. With the change of partners, roles, and tasks, different critical thinking skills were trained. As for the student-teacher community, the student-teacher communication was facilitated through various forms of teacher-student interaction, such as teachers' feedback, office hour, and communications on Tencent QQ, which were necessary to keep students on the right track of developing thinking skills. With various opportunities of communicating with the teacher, students would not feel powerless or frustrated when facing difficult tasks, thus ensuring the achievement of the learning objectives.

Four strategies were employed when designing the scaffolding and guidance. First, the process of thinking was highlighted. When the focus fell on critical thinking processes such as establishing viewpoints, making assumptions, and evaluating information, students had examples to follow when they conducted these activities independently. Second, the role of peers was given full play. In many cases, the demonstration of peers was more direct and effective for the students to develop critical thinking skills. Third, the teacher consciously created a “democratic” classroom and online atmosphere, where students could express their opinions without fearing judgment from the “authority” or other people. Fourth, the teacher established awarding incentives to encourage students to take the initiative to meet challenges and develop thinking. For example, if one student's feedback to others' work was deeper and more thorough, the instructor gave the student more marks and demonstrated the work to the whole class with their permission.

Technological Design

Moodle (Modular Object-Oriented Dynamic Learning Environment) was the main platform of the e-learning environment. A composition reviewing and grading software TRP (Teaching Resources Platform) was also used to facilitate teachers' grading of the compositions. TRP mainly focuses on the mistakes related to language and grammar, which could help direct teachers' attention to the composition's structure, logic, coherence, and other aspects. In addition, Tencent QQ, a social networking software frequently used by students, was selected to send messages and notices to students.

As shown in Table 3 , both synchronous and asynchronous instruments were applied to provide sufficient communication among students in designing communicative tools. When designing the synchronous instruments, the instructor used the Tencent QQ, which could conveniently support the simultaneous real-time communication between learners and encourage group members to fully communicate with each other. The discussion board of Moodle was used as asynchronous tools, and sufficient time was given to the students to respond to other people's opinions or solve problems. The students could use the time to find information, consult others and translate complex ideas into words.

Table 3 . Strategies for technological design.

Research Instruments

Learning environment questionnaire.

The questionnaire adapted from the Web-Based Learning Environment Instrument (WEBLEI) was used to elicit the information of students' perception of the learning environment. The original WEBLEI questionnaire was first created and subsequently modified by Chang and Fisher for investigating online learning environments in University settings. The primary purpose of the questionnaire was to capture “students' perception of web-based learning environments” ( Chang and Fisher, 2003 , p. 9). The questions in the WEBLEI questionnaire are able to cover the three elements of the PST learning environment model. The researcher modified the questionnaire according to the context of the current study. The Cronbach alpha coefficients indicated the acceptable reliability of the modified questionnaire (see Table 4 ).

Table 4 . Cronbach alpha coefficients for modified WEBLEI.

In order to explore students' perceived improvement of critical thinking and the in-depth reasons behind students' perceptions of the learning environment and critical thinking instruction, interviews were conducted after the administration of the adapted WEBLEI questionnaire. Eight students were randomly chosen and invited to the interview one by one. The interviews lasted about 30 min and were audio-recorded with the participants' approval.

Data Analysis

Both quantitative and qualitative data were collected for this study. In terms of quantitative analysis, descriptive statistics were used to describe the means, standard deviations. As for qualitative data, the recordings of the interviews were transcribed for content analysis. The content about the perceptions of the environment was categorized with the outline of the learning environment components. Regarding the development of students' critical thinking, the “elements of thinking” from Paul and Elder's thinking model formed the framework for data analysis. The relevant script was examined and coded according to the framework by the researcher and her collegue to generalize the aspects of critical thinking improvement.

Results and Discussion

Students' perceptions of the environment, students' perception of the pedagogical design.

The means and standard deviation scores of students' perception of the pedagogical design are listed in Table 5 . The overall mean score was 3.86 (SD = 0.79), suggesting that students were generally satisfied with the pedagogical design. Item 1 (M = 3.98, SD = 0.80) (The learning objectives are clearly stated), Item 4 ( M = 3.93, SD = 0.83) (Expectations of assignments are clearly stated), and Item 5 (M = 4, SD = 1.00) (Activities are planned carefully) got particularly high scores, which indicates that students were aware of the careful design of the activities, content, and context.

Table 5 . Students' Perceptions of the Environment.

The students' positive attitude toward the pedagogical design was also revealed from the interview, in which they expressed their satisfaction with the design of tasks and contexts. For example, Student A expressed that the course was designed in the way that they needed to “find solutions to the problems” by themselves most of the time and he also enjoyed the discussions in class. Student C recognized the relative authentic contexts of the tasks, which helped her devote herself to the tasks. She mentioned that in learning to write a CV, the teacher asked the students to imagine the situation in which they were about to graduate and hunt a job. “I felt the topic was very relevant to me, so I was motivated to do this task well.” She told the interviewer.

Apart from the positive opinions, some students expressed their concern about the pedagogical design. For example, Student H said, “The online learning added to our workload. Sometimes I was scared of all the online assignments we had to finish after class.” And student G had difficulty adapting to this learning approach. “It seemed that we were learning by ourselves. I am not sure whether I have learned enough knowledge. I would rather learn how to write from the teacher.”

Students' Perception of Social Design

As seen from Table 5 , the overall mean score of the social design was 3.90 ( M = 0.82), indicating students' generally positive attitude toward the social design. The data gathered from the students' interviews also suggested that students were satisfied with the social design. For example, student B mentioned that she always received encouragement and help when dealing with difficult tasks. Item 11 ( M = 4.07, SD = 0.65) (Other students respond promptly to my request), Item 12 ( M = 4.09, SD = 0.91) (The teachers give me quick comments on my work) and Item 14 ( M = 4.07, SD = 0.58) (I was supported by a positive attitude from my teacher and my classmates) scored higher than Item 9 ( M = 3.47, SD = 1.01) (I can ask my teacher what I do not understand) and Item 10 ( M = 3.79, SD = 0.78) (I can ask other classmates what I do not understand). This finding reveals that in the environment, both students and teachers responded to others promptly, but students had considerations when they needed to consult others.

When asked the reason for this, the students suggested that the teacher and the environment did provide them with the opportunity to seek help, but sometimes they felt reluctant to trouble others. Student E mentioned when he found something he failed to understand, he would prefer to figure it out by himself first and then seek help from the teacher and classmates. He told the interviewer: “I thought the teacher was busy, and my classmates were also busy, so I would prefer to figure it out by myself.”

Students' Perceptions of Technical Design

As for the technical design (see Table 5 ), the average score is 3.73 (SD = 0.85), which suggests that the environment provided relatively sufficient technological support to the students. Item 16 ( M = 3.93, SD = 0.92) (The online material is available at locations suitable for me) and Item 19 ( M = 4, SD = 0.97) (I decide when I want to learn) got higher scores, which indicates that students could enjoy the convenience of “anywhere” and “anytime” in the learning environment.

This positive attitude was demonstrated in the interview data collected from Student F who expressed his appreciation for the freedom and the sense of control brought by asynchronous discussion. He said, “I could finish the task at the time that is convenient for me as long as I did not miss the deadline. I like it.”

One thing worth noticing is that the mean score of Item 20 (Using blended learning allowed me to explore the interest of my own) is 3.18 (SD = 0.68), which falls toward the middle of the 1–5 scale. This score reveals that students did not think the resources of the blended learning environment play an important role in exploring their own areas of interest. In the interview, student D expressed that he did not find the resources very interesting, for the range of the topics was rather limited, and he was not attracted by the resources provided.

In sum, students' ratings on different dimensions of the questionnaire suggest that students perceived the productiveness of the learning environment in a generally positive way. This result is consistent with the studies exploring students' perceptions of the blending learning environment in general (e.g., Akkoyunlu and Yilmaz, 2006 ; Dziuban et al., 2006 ; Owston et al., 2006 ; Bendania, 2011 ; Wang and Huang, 2018 ). In the study conducted by Wang and Huang (2018) , a blended environment was also constructed from the pedagogical, social, and technical perspectives. The findings of the study reveal that students are generally positive toward the design of the learning environment. This may suggest that students would perceive the learning environment positively if the elements of the blended learning environment are carefully designed. Despite the generally positive attitudes toward the learning environment, some students expressed their concern about the workload and adaptation to the way of learning in the interview. In study Stracke (2007) , the way of learning was also found to make the students withdraw from the blended course. The findings indicate that some students may need more time to adapt to more student-centered learning.

Students' Perceived Impact of the Blended Learning Environment on the Development of Their Critical Thinking

Drawing mainly on Paul and Elder's framework of thinking elements, the following themes emerged as to the students' perceived improvement of critical thinking after data analysis and are elucidated through students' quotations.

Gaining a Deeper Understanding of the Concept of “Critical Thinking”

In the interview, students talked about their improvement in understanding the concept of critical thinking. For example, Student D expressed that the environment helped him clarify the concept of critical thinking. He used to consider the concept as closely related to “criticizing” because of its Chinese translation and came to realize that it was closer to the concept of “rational thinking.”

Some students also expressed that the course helped them realize the importance of critical thinking. As the teacher clearly informed the students of the specific critical thinking skills each task aimed to cultivate, students realized that “critical thinking is not an abstract concept, but concrete ways of guiding people to solve problems” (Student B).

Using Facts and Evidence to Support One's Own Opinion

In the interview, students also talked about the change they experienced when forming and supporting their opinion. They started to recognize the importance of facts and evidence in their writing. Student E told the interviewer that he learned that supporting ideas were very important to make one's opinion accepted. He said: “In accomplishing the writing tasks of the course, I gradually learned to provide arguments with further explanations, examples and,… maybe some data.”

Some students also suggested that facts and evidence were important for them to convince others in the discussions. Student B said: “In the past when someone disagreed with me, I usually felt sad and angry. I would either remain silent or quarrel with them. In this course, I learned that if I wanted others to accept my opinion, I needed to convince them with evidence such as facts and information.” She also felt excited that her well-presented opinions were accepted several times during the discussion with her team members.

Thinking From Multiple Perspectives

Another perceived effect is thinking from multiple perspectives, which was mentioned by many students. For example, Student A described how a particular activity helped him recognize the importance of different perspectives and how his own writing benefited from a particular activity in the course. “The teacher asked some of us to play the role of employer and I was assigned this role. When I thought from the employer's perspective, I knew what kind of employee I needed… When I wrote my job application letter, I had a very clear idea what to include in my letter.” (Student A) Student F also mentioned that recognizing different perspectives helped him finish writing the complaint letter well. According to him, he not only mentioned the dissatisfaction in the complaint letter but also stated the potential negative impact on the company to which he sent the letter.

Exploring and Clarifying the Purpose Behind the Texts or Behaviors

The interviewees also mentioned that they learned to explore and clarify the purpose behind the texts or behaviors. Some students explained how they started to consider purpose as an important component in their writing. Student H told the interviewer that when the teacher started to teach a new genre, she always asked the students to discuss under what circumstances they could meet or use this type of writing, and why they needed it in the daily life. “In this way, I understand that there should be a clear purpose behind each writing. And… and when I tried to finish my own writing task, I also put the writing purpose into my consideration.” said Student H.

Some students also told the interviewer that they gradually learned to avoid distraction and stick to the purpose when they conducted a discussion. According to student G, the students tended to talk about irrelevant things when they had discussions at the beginning of the course. With the instructors' constant reminding, they could realize whether they strayed from the point and returned to the right track in time at the end of the semester.

In summary, the data from the interview suggest that students could perceive their critical thinking development in different thinking dimensions. Furthermore, according to the students' opinion, their development in critical thinking was also manifested in their writing and even transferred to other activities. As for the promoting factors of the development, the students recognized the importance of learning environment design, especially the pedagogical design and the social design. For example, students attributed their deeper understanding of the concept to the instructor's deliberate introduction of critical thinking and focus on the development of thinking skills in the activity design. Also, they believed that the teachers' guidance and peers' scaffold enabled them to realize the importance of multiple perspectives. These factors were also found to promote students' critical thinking in the systematic review conducted by Chou et al. (2018) . This suggests that designing the elements of the learning environment to provide favorable conditions for critical thinking development could bring positive effects.

Limitations and Implications

This study proposed the construction of a blended learning environment to promote critical thinking in terms of pedagogical, social, and technical design and explored students' perceptions of the environment design and their perceived impact on the improvement of critical thinking. The results of the study suggests that students are generally satisfied with the design of the learning environment, and students considered the learning environment helpful in improving critical thinking. Even though the study made a contribution to the instructional design aiming at critical thinking promotion in a blended learning environment, some limitations should be duly noted. First, because the participants of the study were 90 students in the same University, the relative homogeneity of the context may present a possible connection with the result. Therefore, replication is recommended with larger and more diverse samples. Second, the study was not able to present the relationship between environmental design and critical thinking development quantitively. Further study could focus on the correlation between design strategies and the improvement of specific thinking skills, or the predictive capability of elements design for the promotion of critical thinking.

This study also has some implications for critical thinking cultivation in the instruction of specific disciplines. On the one hand, the cultivation of students' critical thinking requires the detailed design of the blended learning environment. Special attention needs to be paid to pedagogical, social, and technical design covering factors such as learning objectives, student interaction, and ICT tools. On the other hand, students' troubles and challenges such as the extra workload and emotional factors should be taken into consideration when designing the learning environment.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by School of Foreign Languages, Northeast Normal University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

DL designed and implemented the learning environment, collected and analyzed the data, and wrote the article.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Adam, A., and Manson, T. M. (2014). Using a pseudoscience activity to teach critical thinking. Teach. Psychol. 41, 130–134. doi: 10.1177/0098628314530343

CrossRef Full Text | Google Scholar

Akkoyunlu, B., and Yilmaz, S. M. (2006). A study on students' views on blended learning environment. Turk. Online J. Distance Educ. 7, 43–56. doi: 10.17718/TOJDE.25211

Allaire, J. L. (2015). Assessing critical thinking outcomes of dental hygiene students utilizing virtual patient simulation: a mixed methods study. J. Dent. Educ. 79, 1082–1092. doi: 10.1002/j.0022-0337.2015.79.9.tb06002.x

PubMed Abstract | CrossRef Full Text | Google Scholar

BatdÄ, V. (2014). The effect of blended learning environments on academic success of students: a meta-analysis study. Cankiri Karatekin Univ. J. Inst. Soc. Sci. 5, 287–302. Retrieved from: https://dergipark.org.tr/en/pub/jiss/issue/25892/272867 (accessed June 11, 2021).

Bendania, A. (2011). Teaching and learning online: King Fahd University of Petroleum and Minerals (KFUPM) Saudi Arabia, case study. Int. J. Arts Sci. 4, 223–241. Retrieved from: https://www.academia.edu/5017206/INSTRUCTORS_AND_LEARNERS_ATTITUDES_TOWARD_TEACHING_AND_LEARNING_ONLINE_KING_FAHD_UNIVERSITY_OF_PETROLEUM_AND_MINERALS_KFUPM_SAUDI_ARABIA_CASE_STUDY (accessed June 11, 2021).

Google Scholar

Butchart, S., Bigelow, J., Oppy, G., Korb, K., and Gold, I. (2009). Improving critical thinking using web-based argument mapping exercises with automated feedback. Australas. J. Educ. Technol. 25, 268–291. doi: 10.14742/ajet.1154

Chang, V., and Fisher, D. (2003). “The validation and application of a new learning environment instrument for online learning in higher education,” in Technology-Rich Learning Environments: A Future Perspective , eds M. S. Khine and D. Fisher (Singapore: World Scientific Publishing Co. Pte. Ltd.), 1–20.

Cheong, C. M., and Cheung, W. S. (2008). Online discussion and critical thinking skills: a case study in a Singapore secondary school. Australas. J. Educ. Technol. 24, 556–573. doi: 10.14742/ajet.1191

Chou, T. L., Wu, J. J., and Tsai, C. C. (2018). Research trends and features of critical thinking studies in E-Learning environments: a review. J. Educ. Comput. Res. 57, 1038–1077. doi: 10.1177/0735633118774350

de Leng, B. A., Dolmans, D. H., Jo bsis, R., Muijtjens, A. M., and van der Vleuten, C. P. (2009). Exploration of an e-learning model to foster critical thinking on basic science concepts during work placements. Comput. Educ. 53, 1–13. doi: 10.1016/j.compedu.2008.12.012

Dziuban, C., Hartman, J., Juge, F., Moskal, P., and Sorg, S. (2006). “Blended learning enters the mainstream,” in Handbook of Blended Learning: Global Perspectives, Local Designs , eds C. J. Bonk and C. R. Graham (San Francisco, CA: Pfeiffer), 195–206. Retrieved from: https://www.researchgate.net/publication/284688507_Blended_learning_enters_the_mainstream (accessed June 11, 2021).

Eftekhari, M., Sotoudehnama, E., and Marandi, S. S. (2016). Computer-aided argument mapping in an EFL setting: does technology precede traditional paper and pencil approach in developing critical thinking? Educ. Technol. Res. Dev. 64, 339–357. doi: 10.1007/s11423-016-9431-z

Elder, L., and Paul, R. (1994). Critical thinking: why we must transform our teaching. J. Dev. Educ. 18, 34–35.

Ennis, R. H. (1996). Critical Thinking . New York, NY: Prentice Hall.

Fisher, A., and Scriven, M. (1997). Critical Thinking: Its Definition and Assessment . Norwich: Center for Research in Critical Thinking.

Gilbert, P. K., and Dabbagh, N. (2005). How to structure online discussion of meaningful discourse: a case study. Br. J. Educ. Technol. 36, 5–18. doi: 10.1111/j.1467-8535.2005.00434.x

Guiller, J., Durndell, A., and Ross, A. (2008). Peer interaction and critical thinking: face-to-face or online discussion? Learn. Instruct. 18, 187–200. doi: 10.1016/j.learninstruc.2007.03.001

Huang, T. C., Jeng, Y. L., Hsiao, K. L., and Tsai, B. R. (2017). SNS collaborative learning design: enhancing critical thinking for human computer interface design. Univ. Access Inf. Soc. 16, 303–312. doi: 10.1007/s10209-016-0458-z

Kim, K., and Bonk, C. J. (2006). The future of online teaching and learning in higher education: the survey says. Educ. Q. 29, 22–30. Retrieved from: https://er.educause.edu/articles/2006/1/the-future-of-online-teaching-and-learning-in-higher-education-the-survey-says (accessed June 11, 2021).

Kirschner, P., Strijbos, J. W., Kreijns, K., and Beers, P. J. (2004). Designing electronic collaborative learning environments. Educ. Technol. Res. Dev. 52, 47–66. doi: 10.1007/BF02504675

Lu, D. (2018). Research on the design and application of blended learning environment with the orientation of critical thinking: a case of college practical English writing course (Unpublished doctoral dissertation), Northeast Normal University, Changchun, China.

Naaj, M. A., Nachouki, M., and Ankit, A. (2012). Evaluating student satisfaction with blended learning in a gender-segregated environment. J. Inf. Technol. Educ. Res. 11, 185–200. doi: 10.28945/1692

Osguthorpe, R. T., and Graham, C. R. (2003). Blended learning environments: definitions and directions. Q. Rev. Distance Educ. 4, 227–233. Retrieved from: https://web.b.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=84d5540c-01b3-4b86-8c21-962ff82af437%40sessionmgr103 (accessed June 11, 2021).

Owston, R. D., Garrison, D. R., and Cook, K. (2006). “Blended learning at Canadian universities: issues and practices,” in The Handbook of Blended Learning: Global Perspectives, Local Designs , eds C. J. Bonk and C. R. Graham (San Francisco, CA: Pfeiffer), 338–350.

Paul, R. (1992). Critical thinking: what, why and how? New Dir. Community Coll. 1992, 3–24. doi: 10.1002/cc.36819927703

Paul, R., and Elder, L. (1999). Critical thinking: teaching students to seek the logic of things. J. Dev. Educ. 23, 34–36.

Paul, R., and Elder, L. (2006). The International Critical Thinking Reading and Writing Test . Dillon Beach, CA: The Foundation for Critical Thinking.

Rosen, Y., and Tager, M. (2014). Making student thinking visible through a concept map in computer-based assessment of critical thinking. J. Educ. Comput. Res. 50, 249–270. doi: 10.2190/EC.50.2.f

Sagarra, N., and Zapata, G. C. (2008). Blending classroom instruction with online homework: a study of student perceptions of computer-assisted L2 learning. ReCALL 20, 208–224. doi: 10.1017/S0958344008000621

Shamir, A., Zion, M., and Spector_Levi, O. (2008). Peer tutoring, metacognitive processes and multimedia problem-based learning: the effect of mediation training on critical thinking. J. Sci. Educ. Technol. 17, 384–398. doi: 10.1007/s10956-008-9108-4

Shin, H., Ma, H., Park, J., Ji, E. S., and Kim, D. H. (2015). The effect of simulation courseware on critical thinking in undergraduate nursing students: multi-site pre-post study. Nurse Educ. Today 35, 537–542. doi: 10.1016/j.nedt.2014.12.004

Singh, H. (2003). Building effective blended learning programs. Educ. Technol. 43, 51–54. Retrieved from: http://www.jstor.org/stable/44428863 (accessed June 11, 2021).

Singh, H., and Reed, C. (2001). A White Paper: Achieving Success With Blended Learning . Retrieved from: http://www.leerbeleving.nl/wbts/wbt2014/blend-ce.pdf (accessed June 11, 2021).

Smyth, S., Houghton, C., Cooney, A., and Casey, D. (2012). Students' experiences of blended learning across a range of postgraduate programmes. Nurse Educ. Today 32, 464–468. doi: 10.1016/j.nedt.2011.05.014

Stracke, E. (2007). A road to understanding: a qualitative study into why learners drop out of a blended language learning (BLL) environment. ReCALL 19, 57–78. doi: 10.1017/S0958344007000511

Thorne, K. (2003). Blended Learning: How to Integrate Online and Traditional Learning . London: Kogan Page.

Van Gelder, T., and Bulker, A. (2000). “Reason! improving informal reasoning skills,” in Proceedings of the Australian Computers in Education Conference (Melbourne, VIC). Retrieved from: https://pdfs.semanticscholar.org/ce84/ec799ae2dc15d56939fd0a6e46123e88112e.pdf (accessed April 20, 2020).

Wang, Q., and Huang, C. (2018). Pedagogical, social and technical designs of a blended synchronous learning environment. Br. J. Educ. Technol. 49, 451–462. doi: 10.1111/bjet.12558

Wang, Q. Y. (2008). A generic model for guiding the integration of ICT into teaching and learning. Innov. Educ. Teach. Int. 45, 411–419. doi: 10.1080/14703290802377307

Watson, J. (2008). Blended Learning: The Convergence of Online and Face-to-Face Education. North American Council for Online Learning report . Retrieved from: https://files.eric.ed.gov/fulltext/ED509636.pdf (accessed June 11, 2021).

Yang, Y. T. C. (2008). A catalyst for teaching critical thinking in a large University class in Taiwan: asynchronous online discussions with the facilitation of teaching assistants. Educ. Technol. Res. Dev. 56, 241–264. doi: 10.1007/s11423-007-9054-5

Yang, Y. T. C., and Chang, C. H. (2013). Empowering students through digital game authorship: enhancing concentration, critical thinking, and academic achievement. Comput. Educ. 68, 334–344. doi: 10.1016/j.compedu.2013.05.023

Yang, Y. T. C., and Chou, H. A. (2008). Beyond critical thinking skills: investigating the relationship between critical thinking skills and dispositions through different online instructional strategies. Br. J. Educ. Technol. 39, 666–684. doi: 10.1111/j.1467-8535.2007.00767.x

Yeh, Y. C. (2009). Integrating e-learning into the direct-instruction model to enhance the effectiveness of critical-thinking instruction. Instruct. Sci. 37, 185–203. doi: 10.1007/s11251-007-9048-z

Yen, J. -C., and Lee, C. -Y. (2011). Exploring problem solving patterns and their impact on learning achievement in a blended learning environment. Comput. Educ. 56, 138–145. doi: 10.1016/j.compedu.2010.08.012

Yukawa, J. (2006). Co-reflection in online learning: collaborative critical thinking as narrative. Int. J. Comput. Suppor. Collab. Learn. 1, 203–228. doi: 10.1007/s11412-006-8994-9

Keywords: students' perceptions, blended learning environment, critical thinking, design, survey

Citation: Lu D (2021) Students' Perceptions of a Blended Learning Environment to Promote Critical Thinking. Front. Psychol. 12:696845. doi: 10.3389/fpsyg.2021.696845

Received: 18 April 2021; Accepted: 31 May 2021; Published: 25 June 2021.

Reviewed by:

Copyright © 2021 Lu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Dan Lu, lud090@nenu.edu.cn

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Research article
• Open access
• Published: 15 February 2018

Blended learning: the new normal and emerging technologies

• Charles Dziuban 1 ,
• Charles R. Graham 2 ,
• Patsy D. Moskal   ORCID: orcid.org/0000-0001-6376-839X 1 ,
• Anders Norberg 3 &
• Nicole Sicilia 1  

International Journal of Educational Technology in Higher Education volume  15 , Article number:  3 ( 2018 ) Cite this article

557k Accesses

391 Citations

117 Altmetric

Metrics details

This study addressed several outcomes, implications, and possible future directions for blended learning (BL) in higher education in a world where information communication technologies (ICTs) increasingly communicate with each other. In considering effectiveness, the authors contend that BL coalesces around access, success, and students’ perception of their learning environments. Success and withdrawal rates for face-to-face and online courses are compared to those for BL as they interact with minority status. Investigation of student perception about course excellence revealed the existence of robust if-then decision rules for determining how students evaluate their educational experiences. Those rules were independent of course modality, perceived content relevance, and expected grade. The authors conclude that although blended learning preceded modern instructional technologies, its evolution will be inextricably bound to contemporary information communication technologies that are approximating some aspects of human thought processes.

Introduction

Blended learning and research issues.

Blended learning (BL), or the integration of face-to-face and online instruction (Graham 2013 ), is widely adopted across higher education with some scholars referring to it as the “new traditional model” (Ross and Gage 2006 , p. 167) or the “new normal” in course delivery (Norberg et al. 2011 , p. 207). However, tracking the accurate extent of its growth has been challenging because of definitional ambiguity (Oliver and Trigwell 2005 ), combined with institutions’ inability to track an innovative practice, that in many instances has emerged organically. One early nationwide study sponsored by the Sloan Consortium (now the Online Learning Consortium) found that 65.2% of participating institutions of higher education (IHEs) offered blended (also termed hybrid ) courses (Allen and Seaman 2003 ). A 2008 study, commissioned by the U.S. Department of Education to explore distance education in the U.S., defined BL as “a combination of online and in-class instruction with reduced in-class seat time for students ” (Lewis and Parsad 2008 , p. 1, emphasis added). Using this definition, the study found that 35% of higher education institutions offered blended courses, and that 12% of the 12.2 million documented distance education enrollments were in blended courses.

The 2017 New Media Consortium Horizon Report found that blended learning designs were one of the short term forces driving technology adoption in higher education in the next 1–2 years (Adams Becker et al. 2017 ). Also, blended learning is one of the key issues in teaching and learning in the EDUCAUSE Learning Initiative’s 2017 annual survey of higher education (EDUCAUSE 2017 ). As institutions begin to examine BL instruction, there is a growing research interest in exploring the implications for both faculty and students. This modality is creating a community of practice built on a singular and pervasive research question, “How is blended learning impacting the teaching and learning environment?” That question continues to gain traction as investigators study the complexities of how BL interacts with cognitive, affective, and behavioral components of student behavior, and examine its transformation potential for the academy. Those issues are so compelling that several volumes have been dedicated to assembling the research on how blended learning can be better understood (Dziuban et al. 2016 ; Picciano et al. 2014 ; Picciano and Dziuban 2007 ; Bonk and Graham 2007 ; Kitchenham 2011 ; Jean-François 2013 ; Garrison and Vaughan 2013 ) and at least one organization, the Online Learning Consortium, sponsored an annual conference solely dedicated to blended learning at all levels of education and training (2004–2015). These initiatives address blended learning in a wide variety of situations. For instance, the contexts range over K-12 education, industrial and military training, conceptual frameworks, transformational potential, authentic assessment, and new research models. Further, many of these resources address students’ access, success, withdrawal, and perception of the degree to which blended learning provides an effective learning environment.

Currently the United States faces a widening educational gap between our underserved student population and those communities with greater financial and technological resources (Williams 2016 ). Equal access to education is a critical need, one that is particularly important for those in our underserved communities. Can blended learning help increase access thereby alleviating some of the issues faced by our lower income students while resulting in improved educational equality? Although most indicators suggest “yes” (Dziuban et al. 2004 ), it seems that, at the moment, the answer is still “to be determined.” Quality education presents a challenge, evidenced by many definitions of what constitutes its fundamental components (Pirsig 1974 ; Arum et al. 2016 ). Although progress has been made by initiatives, such as, Quality Matters ( 2016 ), the OLC OSCQR Course Design Review Scorecard developed by Open SUNY (Open SUNY n.d. ), the Quality Scorecard for Blended Learning Programs (Online Learning Consortium n.d. ), and SERVQUAL (Alhabeeb 2015 ), the issue is by no means resolved. Generally, we still make quality education a perceptual phenomenon where we ascribe that attribute to a course, educational program, or idea, but struggle with precisely why we reached that decision. Searle ( 2015 ), summarizes the problem concisely arguing that quality does not exist independently, but is entirely observer dependent. Pirsig ( 1974 ) in his iconic volume on the nature of quality frames the context this way,

“There is such thing as Quality, but that as soon as you try to define it, something goes haywire. You can’t do it” (p. 91).

Therefore, attempting to formulate a semantic definition of quality education with syntax-based metrics results in what O’Neil (O'Neil 2017 ) terms surrogate models that are rough approximations and oversimplified. Further, the derived metrics tend to morph into goals or benchmarks, losing their original measurement properties (Goodhart 1975 ).

Information communication technologies in society and education

Blended learning forces us to consider the characteristics of digital technology, in general, and information communication technologies (ICTs), more specifically. Floridi ( 2014 ) suggests an answer proffered by Alan Turing: that digital ICTs can process information on their own, in some sense just as humans and other biological life. ICTs can also communicate information to each other, without human intervention, but as linked processes designed by humans. We have evolved to the point where humans are not always “in the loop” of technology, but should be “on the loop” (Floridi 2014 , p. 30), designing and adapting the process. We perceive our world more and more in informational terms, and not primarily as physical entities (Floridi 2008 ). Increasingly, the educational world is dominated by information and our economies rest primarily on that asset. So our world is also blended, and it is blended so much that we hardly see the individual components of the blend any longer. Floridi ( 2014 ) argues that the world has become an “infosphere” (like biosphere) where we live as “inforgs.” What is real for us is shifting from the physical and unchangeable to those things with which we can interact.

Floridi also helps us to identify the next blend in education, involving ICTs, or specialized artificial intelligence (Floridi 2014 , 25; Norberg 2017 , 65). Learning analytics, adaptive learning, calibrated peer review, and automated essay scoring (Balfour 2013 ) are advanced processes that, provided they are good interfaces, can work well with the teacher— allowing him or her to concentrate on human attributes such as being caring, creative, and engaging in problem-solving. This can, of course, as with all technical advancements, be used to save resources and augment the role of the teacher. For instance, if artificial intelligence can be used to work along with teachers, allowing them more time for personal feedback and mentoring with students, then, we will have made a transformational breakthrough. The Edinburg University manifesto for teaching online says bravely, “Automation need not impoverish education – we welcome our robot colleagues” (Bayne et al. 2016 ). If used wisely, they will teach us more about ourselves, and about what is truly human in education. This emerging blend will also affect curricular and policy questions, such as the what? and what for? The new normal for education will be in perpetual flux. Floridi’s ( 2014 ) philosophy offers us tools to understand and be in control and not just sit by and watch what happens. In many respects, he has addressed the new normal for blended learning.

Literature of blended learning

A number of investigators have assembled a comprehensive agenda of transformative and innovative research issues for blended learning that have the potential to enhance effectiveness (Garrison and Kanuka 2004 ; Picciano 2009 ). Generally, research has found that BL results in improvement in student success and satisfaction, (Dziuban and Moskal 2011 ; Dziuban et al. 2011 ; Means et al. 2013 ) as well as an improvement in students’ sense of community (Rovai and Jordan 2004 ) when compared with face-to-face courses. Those who have been most successful at blended learning initiatives stress the importance of institutional support for course redesign and planning (Moskal et al. 2013 ; Dringus and Seagull 2015 ; Picciano 2009 ; Tynan et al. 2015 ). The evolving research questions found in the literature are long and demanding, with varied definitions of what constitutes “blended learning,” facilitating the need for continued and in-depth research on instructional models and support needed to maximize achievement and success (Dringus and Seagull 2015 ; Bloemer and Swan 2015 ).

Educational access

The lack of access to educational technologies and innovations (sometimes termed the digital divide) continues to be a challenge with novel educational technologies (Fairlie 2004 ; Jones et al. 2009 ). One of the promises of online technologies is that they can increase access to nontraditional and underserved students by bringing a host of educational resources and experiences to those who may have limited access to on-campus-only higher education. A 2010 U.S. report shows that students with low socioeconomic status are less likely to obtain higher levels of postsecondary education (Aud et al. 2010 ). However, the increasing availability of distance education has provided educational opportunities to millions (Lewis and Parsad 2008 ; Allen et al. 2016 ). Additionally, an emphasis on open educational resources (OER) in recent years has resulted in significant cost reductions without diminishing student performance outcomes (Robinson et al. 2014 ; Fischer et al. 2015 ; Hilton et al. 2016 ).

Unfortunately, the benefits of access may not be experienced evenly across demographic groups. A 2015 study found that Hispanic and Black STEM majors were significantly less likely to take online courses even when controlling for academic preparation, socioeconomic status (SES), citizenship, and English as a second language (ESL) status (Wladis et al. 2015 ). Also, questions have been raised about whether the additional access afforded by online technologies has actually resulted in improved outcomes for underserved populations. A distance education report in California found that all ethnic minorities (except Asian/Pacific Islanders) completed distance education courses at a lower rate than the ethnic majority (California Community Colleges Chancellor’s Office 2013 ). Shea and Bidjerano ( 2014 , 2016 ) found that African American community college students who took distance education courses completed degrees at significantly lower rates than those who did not take distance education courses. On the other hand, a study of success factors in K-12 online learning found that for ethnic minorities, only 1 out of 15 courses had significant gaps in student test scores (Liu and Cavanaugh 2011 ). More research needs to be conducted, examining access and success rates for different populations, when it comes to learning in different modalities, including fully online and blended learning environments.

Framing a treatment effect

Over the last decade, there have been at least five meta-analyses that have addressed the impact of blended learning environments and its relationship to learning effectiveness (Zhao et al. 2005 ; Sitzmann et al. 2006 ; Bernard et al. 2009 ; Means et al. 2010 , 2013 ; Bernard et al. 2014 ). Each of these studies has found small to moderate positive effect sizes in favor of blended learning when compared to fully online or traditional face-to-face environments. However, there are several considerations inherent in these studies that impact our understanding the generalizability of outcomes.

Dziuban and colleagues (Dziuban et al. 2015 ) analyzed the meta-analyses conducted by Means and her colleagues (Means et al. 2013 ; Means et al. 2010 ), concluding that their methods were impressive as evidenced by exhaustive study inclusion criteria and the use of scale-free effect size indices. The conclusion, in both papers, was that there was a modest difference in multiple outcome measures for courses featuring online modalities—in particular, blended courses. However, with blended learning especially, there are some concerns with these kinds of studies. First, the effect sizes are based on the linear hypothesis testing model with the underlying assumption that the treatment and the error terms are uncorrelated, indicating that there is nothing else going on in the blending that might confound the results. Although the blended learning articles (Means et al. 2010 ) were carefully vetted, the assumption of independence is tenuous at best so that these meta-analysis studies must be interpreted with extreme caution.

There is an additional concern with blended learning as well. Blends are not equivalent because of the manner on which they are configured. For instance, a careful reading of the sources used in the Means, et al. papers will identify, at minimum, the following blending techniques: laboratory assessments, online instruction, e-mail, class web sites, computer laboratories, mapping and scaffolding tools, computer clusters, interactive presentations and e-mail, handwriting capture, evidence-based practice, electronic portfolios, learning management systems, and virtual apparatuses. These are not equivalent ways in which to configure courses, and such nonequivalence constitutes the confounding we describe. We argue here that, in actuality, blended learning is a general construct in the form of a boundary object (Star and Griesemer 1989 ) rather than a treatment effect in the statistical sense. That is, an idea or concept that can support a community of practice, but is weakly defined fostering disagreement in the general group. Conversely, it is stronger in individual constituencies. For instance, content disciplines (i.e. education, rhetoric, optics, mathematics, and philosophy) formulate a more precise definition because of commonly embraced teaching and learning principles. Quite simply, the situation is more complicated than that, as Leonard Smith ( 2007 ) says after Tolstoy,

“All linear models resemble each other, each non nonlinear system is unique in its own way” (p. 33).

This by no means invalidates these studies, but effect size associated with blended learning should be interpreted with caution where the impact is evaluated within a particular learning context.

Study objectives

This study addressed student access by examining success and withdrawal rates in the blended learning courses by comparing them to face-to-face and online modalities over an extended time period at the University of Central Florida. Further, the investigators sought to assess the differences in those success and withdrawal rates with the minority status of students. Secondly, the investigators examined the student end-of-course ratings of blended learning and other modalities by attempting to develop robust if-then decision rules about what characteristics of classes and instructors lead students to assign an “excellent” value to their educational experience. Because of the high stakes nature of these student ratings toward faculty promotion, awards, and tenure, they act as a surrogate measure for instructional quality. Next, the investigators determined the conditional probabilities for students conforming to the identified rule cross-referenced by expected grade, the degree to which they desired to take the course, and course modality.

Student grades by course modality were recoded into a binary variable with C or higher assigned a value of 1, and remaining values a 0. This was a declassification process that sacrificed some specificity but compensated for confirmation bias associated with disparate departmental policies regarding grade assignment. At the measurement level this was an “on track to graduation index” for students. Withdrawal was similarly coded by the presence or absence of its occurrence. In each case, the percentage of students succeeding or withdrawing from blended, online or face-to-face courses was calculated by minority and non-minority status for the fall 2014 through fall 2015 semesters.

Next, a classification and regression tree (CART) analysis (Brieman et al. 1984 ) was performed on the student end-of-course evaluation protocol ( Appendix 1 ). The dependent measure was a binary variable indicating whether or not a student assigned an overall rating of excellent to his or her course experience. The independent measures in the study were: the remaining eight rating items on the protocol, college membership, and course level (lower undergraduate, upper undergraduate, and graduate). Decision trees are efficient procedures for achieving effective solutions in studies such as this because with missing values imputation may be avoided with procedures such as floating methods and the surrogate formation (Brieman et al. 1984 , Olshen et al. 1995 ). For example, a logistic regression method cannot efficiently handle all variables under consideration. There are 10 independent variables involved here; one variable has three levels, another has nine, and eight have five levels each. This means the logistic regression model must incorporate more than 50 dummy variables and an excessively large number of two-way interactions. However, the decision-tree method can perform this analysis very efficiently, permitting the investigator to consider higher order interactions. Even more importantly, decision trees represent appropriate methods in this situation because many of the variables are ordinally scaled. Although numerical values can be assigned to each category, those values are not unique. However, decision trees incorporate the ordinal component of the variables to obtain a solution. The rules derived from decision trees have an if-then structure that is readily understandable. The accuracy of these rules can be assessed with percentages of correct classification or odds-ratios that are easily understood. The procedure produces tree-like rule structures that predict outcomes.

The model-building procedure for predicting overall instructor rating

For this study, the investigators used the CART method (Brieman et al. 1984 ) executed with SPSS 23 (IBM Corp 2015 ). Because of its strong variance-sharing tendencies with the other variables, the dependent measure for the analysis was the rating on the item Overall Rating of the Instructor , with the previously mentioned indicator variables (college, course level, and the remaining 8 questions) on the instrument. Tree methods are recursive, and bisect data into subgroups called nodes or leaves. CART analysis bases itself on: data splitting, pruning, and homogeneous assessment.

Splitting the data into two (binary) subsets comprises the first stage of the process. CART continues to split the data until the frequencies in each subset are either very small or all observations in a subset belong to one category (e.g., all observations in a subset have the same rating). Usually the growing stage results in too many terminate nodes for the model to be useful. CART solves this problem using pruning methods that reduce the dimensionality of the system.

The final stage of the analysis involves assessing homogeneousness in growing and pruning the tree. One way to accomplish this is to compute the misclassification rates. For example, a rule that produces a .95 probability that an instructor will receive an excellent rating has an associated error of 5.0%.

Implications for using decision trees

Although decision-tree techniques are effective for analyzing datasets such as this, the reader should be aware of certain limitations. For example, since trees use ranks to analyze both ordinal and interval variables, information can be lost. However, the most serious weakness of decision tree analysis is that the results can be unstable because small initial variations can lead to substantially different solutions.

For this study model, these problems were addressed with the k-fold cross-validation process. Initially the dataset was partitioned randomly into 10 subsets with an approximately equal number of records in each subset. Each cohort is used as a test partition, and the remaining subsets are combined to complete the function. This produces 10 models that are all trained on different subsets of the original dataset and where each has been used as the test partition one time only.

Although computationally dense, CART was selected as the analysis model for a number of reasons— primarily because it provides easily interpretable rules that readers will be able evaluate in their particular contexts. Unlike many other multivariate procedures that are even more sensitive to initial estimates and require a good deal of statistical sophistication for interpretation, CART has an intuitive resonance with researcher consumers. The overriding objective of our choice of analysis methods was to facilitate readers’ concentration on our outcomes rather than having to rely on our interpretation of the results.

Institution-level evaluation: Success and withdrawal

The University of Central Florida (UCF) began a longitudinal impact study of their online and blended courses at the start of the distributed learning initiative in 1996. The collection of similar data across multiple semesters and academic years has allowed UCF to monitor trends, assess any issues that may arise, and provide continual support for both faculty and students across varying demographics. Table  1 illustrates the overall success rates in blended, online and face-to-face courses, while also reporting their variability across minority and non-minority demographics.

While success (A, B, or C grade) is not a direct reflection of learning outcomes, this overview does provide an institutional level indication of progress and possible issues of concern. BL has a slight advantage when looking at overall success and withdrawal rates. This varies by discipline and course, but generally UCF’s blended modality has evolved to be the best of both worlds, providing an opportunity for optimizing face-to-face instruction through the effective use of online components. These gains hold true across minority status. Reducing on-ground time also addresses issues that impact both students and faculty such as parking and time to reach class. In addition, UCF requires faculty to go through faculty development tailored to teaching in either blended or online modalities. This 8-week faculty development course is designed to model blended learning, encouraging faculty to redesign their course and not merely consider blended learning as a means to move face-to-face instructional modules online (Cobb et al. 2012 ; Lowe 2013 ).

Withdrawal (Table  2 ) from classes impedes students’ success and retention and can result in delayed time to degree, incurred excess credit hour fees, or lost scholarships and financial aid. Although grades are only a surrogate measure for learning, they are a strong predictor of college completion. Therefore, the impact of any new innovation on students’ grades should be a component of any evaluation. Once again, the blended modality is competitive and in some cases results in lower overall withdrawal rates than either fully online or face-to-face courses.

The students’ perceptions of their learning environments

Other potentially high-stakes indicators can be measured to determine the impact of an innovation such as blended learning on the academy. For instance, student satisfaction and attitudes can be measured through data collection protocols, including common student ratings, or student perception of instruction instruments. Given that those ratings often impact faculty evaluation, any negative reflection can derail the successful implementation and scaling of an innovation by disenfranchised instructors. In fact, early online and blended courses created a request by the UCF faculty senate to investigate their impact on faculty ratings as compared to face-to-face sections. The UCF Student Perception of Instruction form is released automatically online through the campus web portal near the end of each semester. Students receive a splash page with a link to each course’s form. Faculty receive a scripted email that they can send to students indicating the time period that the ratings form will be available. The forms close at the beginning of finals week. Faculty receive a summary of their results following the semester end.

The instrument used for this study was developed over a ten year period by the faculty senate of the University of Central Florida, recognizing the evolution of multiple course modalities including blended learning. The process involved input from several constituencies on campus (students, faculty, administrators, instructional designers, and others), in attempt to provide useful formative and summative instructional information to the university community. The final instrument was approved by resolution of the senate and, currently, is used across the university. Students’ rating of their classes and instructors comes with considerable controversy and disagreement with researchers aligning themselves on both sides of the issue. Recently, there have been a number of studies criticizing the process (Uttl et al. 2016 ; Boring et al. 2016 ; & Stark and Freishtat 2014 ). In spite of this discussion, a viable alternative has yet to emerge in higher education. So in the foreseeable future, the process is likely to continue. Therefore, with an implied faculty senate mandate this study was initiated by this team of researchers.

Prior to any analysis of the item responses collected in this campus-wide student sample, the psychometric quality (domain sampling) of the information yielded by the instrument was assessed. Initially, the reliability (internal consistency) was derived using coefficient alpha (Cronbach 1951 ). In addition, Guttman ( 1953 ) developed a theorem about item properties that leads to evidence about the quality of one’s data, demonstrating that as the domain sampling properties of items improve, the inverse of the correlation matrix among items will approach a diagonal. Subsequently, Kaiser and Rice ( 1974 ) developed the measure of sampling adequacy (MSA) that is a function of the Guttman Theorem. The index has an upper bound of one with Kaiser offering some decision rules for interpreting the value of MSA. If the value of the index is in the .80 to .99 range, the investigator has evidence of an excellent domain sample. Values in the .70s signal an acceptable result, and those in the .60s indicate data that are unacceptable. Customarily, the MSA has been used for data assessment prior to the application of any dimensionality assessments. Computation of the MSA value gave the investigators a benchmark for the construct validity of the items in this study. This procedure has been recommended by Dziuban and Shirkey ( 1974 ) prior to any latent dimension analysis and was used with the data obtained for this study. The MSA for the current instrument was .98 suggesting excellent domain sampling properties with an associated alpha reliability coefficient of .97 suggesting superior internal consistency. The psychometric properties of the instrument were excellent with both measures.

The online student ratings form presents an electronic data set each semester. These can be merged across time to create a larger data set of completed ratings for every course across each semester. In addition, captured data includes course identification variables including prefix, number, section and semester, department, college, faculty, and class size. The overall rating of effectiveness is used most heavily by departments and faculty in comparing across courses and modalities (Table  3 ).

The finally derived tree (decision rules) included only three variables—survey items that asked students to rate the instructor’s effectiveness at:

Helping students achieve course objectives,

Creating an environment that helps students learn, and

Communicating ideas and information.

None of the demographic variables associated with the courses contributed to the final model. The final rule specifies that if a student assigns an excellent rating to those three items, irrespective of their status on any other condition, the probability is .99 that an instructor will receive an overall rating of excellent. The converse is true as well. A poor rating on all three of those items will lead to a 99% chance of an instructor receiving an overall rating of poor.

Tables  4 , 5 and 6 present a demonstration of the robustness of the CART rule for variables on which it was not developed: expected course grade, desire to take the course and modality.

In each case, irrespective of the marginal probabilities, those students conforming to the rule have a virtually 100% chance of seeing the course as excellent. For instance, 27% of all students expecting to fail assigned an excellent rating to their courses, but when they conformed to the rule the percentage rose to 97%. The same finding is true when students were asked about their desire to take the course with those who strongly disagreed assigning excellent ratings to their courses 26% of the time. However, for those conforming to the rule, that category rose to 92%. When course modality is considered in the marginal sense, blended learning is rated as the preferred choice. However, from Table  6 we can observe that the rule equates student assessment of their learning experiences. If they conform to the rule, they will see excellence.

This study addressed increasingly important issues of student success, withdrawal and perception of the learning environment across multiple course modalities. Arguably these components form the crux of how we will make more effective decisions about how blended learning configures itself in the new normal. The results reported here indicate that blending maintains or increases access for most student cohorts and produces improved success rates for minority and non-minority students alike. In addition, when students express their beliefs about the effectiveness of their learning environments, blended learning enjoys the number one rank. However, upon more thorough analysis of key elements students view as important in their learning, external and demographic variables have minimal impact on those decisions. For example college (i.e. discipline) membership, course level or modality, expected grade or desire to take a particular course have little to do with their course ratings. The characteristics they view as important relate to clear establishment and progress toward course objectives, creating an effective learning environment and the instructors’ effective communication. If in their view those three elements of a course are satisfied they are virtually guaranteed to evaluate their educational experience as excellent irrespective of most other considerations. While end of course rating protocols are summative the three components have clear formative characteristics in that each one is directly related to effective pedagogy and is responsive to faculty development through units such as the faculty center for teaching and learning. We view these results as encouraging because they offer potential for improving the teaching and learning process in an educational environment that increases the pressure to become more responsive to contemporary student lifestyles.

Clearly, in this study we are dealing with complex adaptive systems that feature the emergent property. That is, their primary agents and their interactions comprise an environment that is more than the linear combination of their individual elements. Blending learning, by interacting with almost every aspect of higher education, provides opportunities and challenges that we are not able to fully anticipate.

This pedagogy alters many assumptions about the most effective way to support the educational environment. For instance, blending, like its counterpart active learning, is a personal and individual phenomenon experienced by students. Therefore, it should not be surprising that much of what we have called blended learning is, in reality, blended teaching that reflects pedagogical arrangements. Actually, the best we can do for assessing impact is to use surrogate measures such as success, grades, results of assessment protocols, and student testimony about their learning experiences. Whether or not such devices are valid indicators remains to be determined. We may be well served, however, by changing our mode of inquiry to blended teaching.

Additionally, as Norberg ( 2017 ) points out, blended learning is not new. The modality dates back, at least, to the medieval period when the technology of textbooks was introduced into the classroom where, traditionally, the professor read to the students from the only existing manuscript. Certainly, like modern technologies, books were disruptive because they altered the teaching and learning paradigm. Blended learning might be considered what Johnson describes as a slow hunch (2010). That is, an idea that evolved over a long period of time, achieving what Kaufmann ( 2000 ) describes as the adjacent possible – a realistic next step occurring in many iterations.

The search for a definition for blended learning has been productive, challenging, and, at times, daunting. The definitional continuum is constrained by Oliver and Trigwell ( 2005 ) castigation of the concept for its imprecise vagueness to Sharpe et al.’s ( 2006 ) notion that its definitional latitude enhances contextual relevance. Both extremes alter boundaries such as time, place, presence, learning hierarchies, and space. The disagreement leads us to conclude that Lakoff’s ( 2012 ) idealized cognitive models i.e. arbitrarily derived concepts (of which blended learning might be one) are necessary if we are to function effectively. However, the strong possibility exists that blended learning, like quality, is observer dependent and may not exist outside of our perceptions of the concept. This, of course, circles back to the problem of assuming that blending is a treatment effect for point hypothesis testing and meta-analysis.

Ultimately, in this article, we have tried to consider theoretical concepts and empirical findings about blended learning and their relationship to the new normal as it evolves. Unfortunately, like unresolved chaotic solutions, we cannot be sure that there is an attractor or that it will be the new normal. That being said, it seems clear that blended learning is the harbinger of substantial change in higher education and will become equally impactful in K-12 schooling and industrial training. Blended learning, because of its flexibility, allows us to maximize many positive education functions. If Floridi ( 2014 ) is correct and we are about to live in an environment where we are on the communication loop rather than in it, our educational future is about to change. However, if our results are correct and not over fit to the University of Central Florida and our theoretical speculations have some validity, the future of blended learning should encourage us about the coming changes.

Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher Education Edition . Austin: The New Media Consortium.

Google Scholar  

Alhabeeb, A. M. (2015). The quality assessment of the services offered to the students of the College of Education at King Saud University using (SERVQUAL) method. Journal of Education and Practice , 6 (30), 82–93.

Allen, I. E., & Seaman, J. (2003). Sizing the opportunity: The quality and extent of online education in the United States, 2002 and 2003. Retrieved from http://files.eric.ed.gov/fulltext/ED530060.pdf

Allen, I. E., Seaman, J., Poulin, R., & Straut, T. T. (2016). Online report card: Tracking online education in the United States, 1–4. Retrieved from http://onlinelearningsurvey.com/reports/onlinereportcard.pdf

Arum, R., Roksa, J., & Cook, A. (2016). Improving quality in American higher education: Learning outcomes and assessments for the 21st century . San Francisco: Jossey-Bass.

Aud, S., Hussar, W., Planty, M., Snyder, T., Bianco, K., Fox, M. A., & Drake, L. (2010). The condition of education - 2010. Education, 4–29. https://doi.org/10.1037/e492172006-019

Balfour, S. P. (2013). Assessing writing in MOOCs: Automated essay scoring and calibrated peer review. Research and Practice in Assessment , 2013 (8), 40–48.

Bayne, S., Evans, P., Ewins, R.,Knox, J., Lamb, J., McLeod, H., O’Shea, C., Ross, J., Sheail, P. & Sinclair, C, (2016) Manifesto for teaching online. Digital Education at Edinburg University. Retrieved from https://onlineteachingmanifesto.wordpress.com/the-text/

Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research , 79 (3), 1243–1289. https://doi.org/10.3102/0034654309333844 .

Article   Google Scholar  

Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education , 26 (1), 87–122.

Bloemer, W., & Swan, K. (2015). Investigating informal blending at the University of Illinois Springfield. In A. G. Picciano, C. D. Dziuban, & C. R. Graham (Eds.), Blended learning: Research perspectives , (vol. 2, pp. 52–69). New York: Routledge.

Bonk, C. J., & Graham, C. R. (2007). The handbook of blended learning: Global perspectives, local designs . San Francisco: Pfeiffer.

Boring, A., Ottoboni, K., & Stark, P.B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. EGERA.

Brieman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees . New York: Chapman & Hall.

California Community Colleges Chancellor’s Office. (2013). Distance education report.

Cobb, C., deNoyelles, A., & Lowe, D. (2012). Influence of reduced seat time on satisfaction and perception of course development goals: A case study in faculty development. The Journal of Asynchronous Learning , 16 (2), 85–98.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika , 16 (3), 297–334 Retrieved from http://psych.colorado.edu/~carey/courses/psyc5112/readings/alpha_cronbach.pdf .

Article   MATH   Google Scholar  

Dringus, L. P., and A. B. Seagull. 2015. A five-year study of sustaining blended learning initiatives to enhance academic engagement in computer and information sciences campus courses. In Blended learning: Research perspectives. Vol. 2. Edited by A. G. Picciano, C. D. Dziuban, and C. R. Graham, 122-140. New York: Routledge.

Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin , 81(6), 358. https://doi.org/10.1037/h0036316 .

Dziuban, C., Hartman, J., Cavanagh, T., & Moskal, P. (2011). Blended courses as drivers of institutional transformation. In A. Kitchenham (Ed.), Blended learning across disciplines: Models for implementation , (pp. 17–37). Hershey: IGI Global.

Chapter   Google Scholar  

Dziuban, C., & Moskal, P. (2011). A course is a course is a course: Factor invariance in student evaluation of online, blended and face-to-face learning environments. The Internet and Higher Education , 14 (4), 236–241.

Dziuban, C., Moskal, P., Hermsdorfer, A., DeCantis, G., Norberg, A., & Bradford, G., (2015) A deconstruction of blended learning. Presented at the 11 th annual Sloan-C blended learning conference and workshop

Dziuban, C., Picciano, A. G., Graham, C. R., & Moskal, P. D. (2016). Conducting research in online and blended learning environments: New pedagogical frontiers . New York: Routledge, Taylor & Francis Group.

Dziuban, C. D., Hartman, J. L., & Moskal, P. D. (2004). Blended learning. EDUCAUSE Research Bulletin , 7 , 1–12.

EDUCAUSE. (2017) 2017 key issues in teaching & learning. Retrieved from https://www.EDUCAUSE.edu/eli/initiatives/key-issues-in-teaching-and-learning

Fairlie, R. (2004). Race and the digital divide. The B.E. Journal of Economic Analysis & Policy , 3 (1). https://doi.org/10.2202/1538-0645.1263 .

Fischer, L., Hilton, J., Robinson, T. J., & Wiley, D. (2015). A Multi-institutional Study of the Impact of Open Textbook Adoption on the Learning Outcomes of Post-secondary Students . Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-015-9101-x .

Floridi, L. (2008). A defence of informational structural realism. Synthese , 161 (2), 219–253.

Article   MathSciNet   Google Scholar  

Floridi, L. (2014). The 4th revolution: How the infosphere is reshaping human reality . Oxford: Oxford University Press.

Garrison, D. R., & Vaughan, N. D. (2013). Blended learning in higher education , (1st ed., ). San Francisco: Jossey-Bass Print.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education , 7 , 95–105.

Goodhart, C.A.E. (1975). “Problems of monetary management: The U.K. experience.” Papers in Monetary Economics. Reserve Bank of Australia. I.

Graham, C. R. (2013). Emerging practice and research in blended learning. In M. G. Moore (Ed.), Handbook of distance education , (3rd ed., pp. 333–350). New York: Routledge.

Guttman, L. (1953). Image theory for the structure of quantitative variates. Psychometrika , 18 , 277–296.

Article   MathSciNet   MATH   Google Scholar  

Hilton, J., Fischer, L., Wiley, D., & Williams, L. (2016). Maintaining momentum toward graduation: OER and the course throughput rate. International Review of Research in Open and Distance Learning , 17 (6) https://doi.org/10.19173/irrodl.v17i6.2686 .

IBM Corp. Released (2015). IBM SPSS statistics for windows, version 23.0 . Armonk: IBM Corp.

Jean-François, E. (2013). Transcultural blended learning and teaching in postsecondary education . Hershey: Information Science Reference.

Book   Google Scholar  

Jones, S., Johnson-Yale, C., Millermaier, S., & Pérez, F. S. (2009). U.S. college students’ internet use: Race, gender and digital divides. Journal of Computer-Mediated Communication , 14 (2), 244–264 https://doi.org/10.1111/j.1083-6101.2009.01439.x .

Kaiser, H. F., & Rice, J. (1974). Little Jiffy, Mark IV. Journal of Educational and Psychological Measurement , 34(1), 111–117.

Kaufmann, S. (2000). Investigations . New York: Oxford University Press.

Kitchenham, A. (2011). Blended learning across disciplines: Models for implementation . Hershey: Information Science Reference.

Lakoff, G. (2012). Women, fire, and dangerous things: What categories reveal about the mind . Chicago: The University of Chicago Press.

Lewis, L., & Parsad, B. (2008). Distance education at degree-granting postsecondary institutions : 2006–07 (NCES 2009–044) . Washington: Retrieved from http://nces.ed.gov/pubs2009/2009044.pdf .

Liu, F., & Cavanaugh, C. (2011). High enrollment course success factors in virtual school: Factors influencing student academic achievement. International Journal on E-Learning , 10 (4), 393–418.

Lowe, D. (2013). Roadmap of a blended learning model for online faculty development. Invited feature article in Distance Education Report , 17 (6), 1–7.

Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record , 115 (3), 1–47.

Means, B., Toyama, Y., Murphy, R., Kaia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning . Washington: US Department of Education.

Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? The Internet and Higher Education , 18 , 15–23.

Norberg, A. (2017). From blended learning to learning onlife: ICTs, time and access in higher education (Doctoral dissertation, Umeå University).

Norberg, A., Dziuban, C. D., & Moskal, P. D. (2011). A time-based blended learning model. On the Horizon , 19 (3), 207–216. https://doi.org/10.1108/10748121111163913 .

Oliver, M., & Trigwell, K. (2005). Can ‘blended learning’ be redeemed? e-Learning , 2 (1), 17–25.

Olshen, Stone , Steinberg , and Colla (1995). CART classification and regression trees. Tree-structured nonparametric data analysis. Statistical algorithms. Salford systems interface and documentation. Salford Systems .

O'Neil, C. (2017). Weapons of math destruction: How big data increases inequality and threatens democracy . Broadway Books.

Online Learning Consortium. The OLC quality scorecard for blended learning programs. Retrieved from https://onlinelearningconsortium.org/consult/olc-quality-scorecard-blended-learning-programs/

Open SUNY. The OSCQR course design review scorecard. Retrieved from https://onlinelearningconsortium.org/consult/oscqr-course-design-review/

Picciano, A. G. (2009). Blending with purpose: The multimodal model. Journal of Asynchronous Learning Networks , 13 (1), 7–18.

Picciano, A. G., Dziuban, C., & Graham, C. R. (2014). Blended learning: Research perspectives , (vol. 2). New York: Routledge.

Picciano, A. G., & Dziuban, C. D. (2007). Blended learning: Research perspectives . Needham: The Sloan Consortium.

Pirsig, R. M. (1974). Zen and the art of motorcycle maintenance: An inquiry into values . New York: Morrow.

Quality Matters. (2016). About Quality Matters. Retrieved from https://www.qualitymatters.org/research

Robinson, T. J., Fischer, L., Wiley, D. A., & Hilton, J. (2014). The Impact of Open Textbooks on Secondary Science Learning Outcomes . Educational Researcher. https://doi.org/10.3102/0013189X14550275 .

Ross, B., & Gage, K. (2006). Global perspectives on blended learning: Insight from WebCT and our customers in higher education. In C. J. Bonk, & C. R. Graham (Eds.), Handbook of blended learning: Global perspectives, local designs , (pp. 155–168). San Francisco: Pfeiffer.

Rovai, A. P., & Jordan, H. M. (2004). Blended learning and sense of community: A comparative analysis with traditional and fully online graduate courses. International Review of Research in Open and Distance Learning , 5 (2), 1–13.

Searle, J. R. (2015). Seeing things as they are: A theory of perception . Chicago: Oxford University Press.

Sharpe, R., Benfield, G., Roberts, G., & Francis, R. (2006). The undergraduate experience of blended learning: A review of UK literature and research. The Higher Education Academy, (October 2006).

Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A national study of community college students. Computers and Education , 75 , 103–111 https://doi.org/10.1016/j.compedu.2014.02.009 .

Shea, P., & Bidjerano, T. (2016). A National Study of differences between distance and non-distance community college students in time to first associate degree attainment, transfer, and dropout. Online Learning , 20 (3), 14–15.

Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology , 59 (3), 623–664.

Smith, L. A. (2007). Chaos: a very short introduction . Oxford: Oxford University Press.

Star, S. L., & Griesemer, J. R. (1989). Institutional ecology, translations and boundary objects: Amatuers and professionals in Berkely’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science , 19 (3), 387–420.

Stark, P. & Freishtat, R. (2014). An evaluation of course evaluations. ScienceOpen. Retrieved from https://www.stat.berkeley.edu/~stark/Preprints/evaluations14.pdf .

Tynan, B., Ryan, Y., & Lamont-Mills, A. (2015). Examining workload models in online and blended teaching. British Journal of Educational Technology , 46 (1), 5–15.

Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation , 54 , 22–42.

Williams, J. (2016). College and the new class divide. Inside Higher Ed July 11, 2016.

Wladis, C., Hachey, A. C., & Conway, K. (2015). Which STEM majors enroll in online courses, and why should we care? The impact of ethnicity, gender, and non-traditional student characteristics. Computers and Education , 87 , 285–308 https://doi.org/10.1016/j.compedu.2015.06.010 .

Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record , 107 (8), 1836–1884. https://doi.org/10.1111/j.1467-9620.2005.00544.x .

Download references

Acknowledgements

The authors acknowledge the contributions of several investigators and course developers from the Center for Distributed Learning at the University of Central Florida, the McKay School of Education at Brigham Young University, and Scholars at Umea University, Sweden. These professionals contributed theoretical and practical ideas to this research project and carefully reviewed earlier versions of this manuscript. The Authors gratefully acknowledge their support and assistance.

Author information

Authors and affiliations.

University of Central Florida, Orlando, Florida, USA

Charles Dziuban, Patsy D. Moskal & Nicole Sicilia

Brigham Young University, Provo, Utah, USA

Charles R. Graham

Campus Skellefteå, Skellefteå, Sweden

Anders Norberg

You can also search for this author in PubMed   Google Scholar

Contributions

The Authors of this article are listed in alphabetical order indicating equal contribution to this article. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Patsy D. Moskal .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Student Perception of Instruction

Instructions: Please answer each question based on your current class experience. You can provide additional information where indicated.

All responses are anonymous. Responses to these questions are important to help improve the course and how it is taught. Results may be used in personnel decisions. The results will be shared with the instructor after the semester is over.

Please rate the instructor’s effectiveness in the following areas:

Organizing the course:

Excellent b) Very Good c) Good d) Fair e) Poor

Explaining course requirements, grading criteria, and expectations:

Communicating ideas and/or information:

Showing respect and concern for students:

Stimulating interest in the course:

Creating an environment that helps students learn:

Giving useful feedback on course performance:

Helping students achieve course objectives:

Overall, the effectiveness of the instructor in this course was:

What did you like best about the course and/or how the instructor taught it?

What suggestions do you have for improving the course and/or how the instructor taught it?

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Dziuban, C., Graham, C.R., Moskal, P.D. et al. Blended learning: the new normal and emerging technologies. Int J Educ Technol High Educ 15 , 3 (2018). https://doi.org/10.1186/s41239-017-0087-5

Download citation

Received : 09 October 2017

Accepted : 20 December 2017

Published : 15 February 2018

DOI : https://doi.org/10.1186/s41239-017-0087-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Blended learning
• Higher education
• Student success
• Student perception of instruction

A Qualitative Case Study Exploring Teachers' Perceptions of Blended Learning and the Impact on Student Achievement

Journal title, journal issn, volume title.

Technology's rapid development is changing how educators plan lessons and incorporate these digital resources into classrooms with the expectation of impacting student academic performance. The problem is that teachers need professional development related to blended learning implementation to improve student achievement in elementary (An, 2021; Hung et al., 2020; Lockee, 2021). The purpose of this qualitative study was to identify the professional development needed to implement best practice strategies in blended learning environments in third through fifth classrooms in a suburban school district in Northwest Atlanta, Georgia. Although previous research studies using diffusion of innovation and self-determination theory exist, the studies focus on teachers' perceptions of blended learning in secondary and postsecondary environments. Data were collected utilizing purposeful sampling to select 15 third to fifth-grade teachers. An open-ended question interview was used to collect the data. Thematic analysis suggested that teachers perceive blended learning professional development as beneficial to their instructional choices. Additionally, teachers expressed that the support received from onsite personnel was more meaningful than generalized professional development. The findings suggest teachers have positively impacted student achievement using blended learning best practices. The results addressed the gap in the literature and suggested that professional development in blended learning best practices impacts teachers' perceptions and instructional choices for students. Furthermore, targeted professional development benefited teacher pedagogy and improved student achievement.

Description

Item.page.type, item.page.format, collections.

Unveiling the Dynamics of Blended Learning A Comprehensive Exploration into Adaptability Levels and Influential Factors among Students in Small Rural Schools

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, index terms.

Applied computing

Recommendations

A study of undergraduate students' learning satisfaction and influencing factors in blended learning: ——a case study of a university in west china.

The "Internet + Education" has brought a new ecology to education and teaching. The blended teaching born in the context of this era brings new opportunities and challenges to the development of teaching and learning in university classrooms. Based on ...

Impact of smart classrooms combined with student-centered pedagogies on rural students’ learning outcomes: Pedagogy and duration as moderator variables

Rural students' lack of access to high-quality educational resources leads to lagging academic achievement and a deficiency in 21st-century skills. Smart classrooms integrate advanced technology to provide equitable digital learning and ...

• SC-SCP enhances rural students' achievement, collaboration, and problem-solving.
• Problem-solving is most influenced by PjBL, followed by PBL and TBL.
• Fading novelty effects of technology are overcome by SC-SCP.
• Improved student ...

Embedded blended learning within an Algebra classroom: a multimedia capture experiment

This two-group, pretest-posttest, quasi-experimental study compared secondary students' learning of Algebra II materials over a 4-week period when identical instruction by the same teacher was delivered through either embedded blended learning treatment ...

Information

Published in.

Association for Computing Machinery

New York, NY, United States

Publication History

Permissions, check for updates, author tags.

• Blended Learning
• Educational Equity
• Learning Adaptation
• Rural Education
• Small rural school
• Research-article
• Refereed limited

Funding Sources

• The research result of the general subject of education in 2018 of the 13th Five-Year Plan of the National Social Science Foundation of China

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View Options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

View options.

View or Download as a PDF file.

View online with eReader .

Share this Publication link

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Author Biographies
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Digital learning orientation and entrepreneurial competencies in graduates: is blended learning sustainable.

1. Introduction

2. theoretical background, 2.1. digitalization and blended learning, 2.2. digital learning orientation and entrepreneurial competencies, 2.3. blended learning and the development of entrepreneurial competencies, 2.4. digital learning orientation and entrepreneurial competencies in a blended learning context, 3. methodology, 3.1. context and data collection, 3.2. measures, 3.3. descriptive statistics, 3.4. statistical examination and hypothesis testing, 3.5. measurement model, 3.6. common method bias, 3.7. structural model, 3.7.1. coefficient of determination (r 2 ), 3.7.2. structural equation modeling.

• H1: DLO positively impacts the development of ECs—Confirmed [path coefficient = 0.739; t-value = 11.034; p -value = 0.000; f 2 = 0.488].
• H2: BL positively influences the development of ECs—Not Confirmed [path coefficient = 0.127; t-value = 6.723; p -value = 0.068; f 2 = 0.354].

3.7.3. Moderator Analysis

• H1a: BL mediates the relation between DLO and the development of ECs. Confirmed [path coefficient = 0.369; t-value = 5.148; p -value = 0.003].

4. Results and Discussion

5. conclusions, limitations and future research, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Nambisan, S.; Wright, M.; Feldman, M. The digital transformation of innovation and entrepreneurship: Progress, challenges and key themes. Res. Policy 2019 , 48 , 103773. [ Google Scholar ] [ CrossRef ]
• Wessel, L.; Baiyere, A.; Ologeanu-Taddei, R.; Cha, J.; Blegind-Jensen, T. Unpacking the difference between digital transformation and IT-enabled organizational transformation. J. Assoc. Inf. Syst. 2021 , 22 , 102–129. [ Google Scholar ] [ CrossRef ]
• Fernández, A.; Gómez, B.; Binjaku, K.; Meçe, E.K. Digital transformation initiatives in higher education institutions: A multivocal literature review. Educ. Inf. Technol. 2023 , 28 , 12351–12382. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aboobaker, N.; Ka, Z. Digital learning orientation and innovative behavior in the higher education sector: Effects of organizational learning culture and readiness for change. Int. J. Educ. Manag. 2021 , 35 , 1030–1047. [ Google Scholar ] [ CrossRef ]
• Rasheed, R.A.; Kamsin, A.; Abdullah, N.A. Challenges in the online component of blended learning: A systematic review. Comput. Educ. 2019 , 144 , 103701. [ Google Scholar ] [ CrossRef ]
• Xu, Z.; Zhao, Y.; Liew, J.; Zhou, X.; Kogut, A. Synthesizing research evidence on self-regulated learning and academic achievement in online and blended learning environments: A scoping review. Educ. Res. Rev. 2023 , 39 , 100510. [ Google Scholar ] [ CrossRef ]
• Macaruso, P.; Wilkes, S.; Prescott, J.E. An investigation of blended learning to support reading instruction in elementary schools. Educ. Technol. Res. Dev. 2020 , 68 , 2839–2852. [ Google Scholar ] [ CrossRef ]
• Nee, C.K. The effect of educational networking on students’ performance in biology. Int. J. Integr. Technol. Educ. 2014 , 3 , 21–41. [ Google Scholar ]
• Su, F.; Zou, D.; Wang, L.; Kohnke, L. Student engagement and teaching presence in blended learning and emergency remote teaching. J. Comput. Educ. 2023 , 11 , 445–470. [ Google Scholar ] [ CrossRef ]
• Wilkes, S.; Kazakoff, E.R.; Prescott, J.E.; Bundschuh, K.; Hook, P.E.; Wolf, R.; Hurwitz, L.B.; Macaruso, P. Measuring the impact of a blended learning model on early literacy growth. J. Comput. Assist. Learn. 2020 , 36 , 595–609. [ Google Scholar ] [ CrossRef ]
• Gelgoot, E.S.; Bulakowski, P.F.; Worrell, F.C. Flipping a classroom for academically talented students. J. Adv. Acad. 2020 , 31 , 451–469. [ Google Scholar ] [ CrossRef ]
• Tse, W.S.; Choi, L.Y.; Tang, W.S. Effects of video-based flipped class instruction on subject reading motivation. Br. J. Educ. Technol. 2019 , 50 , 385–398. [ Google Scholar ] [ CrossRef ]
• Banerjee, G. Blended environments: Learning effectiveness and student satisfaction at a small college in transition. J. Asynchronous Learn. Netw. 2011 , 15 , 8–19. [ Google Scholar ] [ CrossRef ]
• Henderson, M.; Selwyn, N.; Aston, R. What works and why? Student perceptions of ‘useful’digital technology in university teaching and learning. Stud. High. Educ. 2017 , 42 , 1567–1579. [ Google Scholar ] [ CrossRef ]
• Kondakci, Y.; Zayim Kurtay, M.; Caliskan, O. Antecedents of continuous change in educational organizations. Int. J. Educ. Manag. 2019 , 33 , 1366–1380. [ Google Scholar ] [ CrossRef ]
• Dhawan, S. Online learning: A panacea in the time of COVID-19 crisis. J. Educ. Technol. Syst. 2020 , 49 , 5–22. [ Google Scholar ] [ CrossRef ]
• Grand-Clement, S. Digital Learning: Education and Skills in the Digital Age ; RAND Europe: Cambridge, UK, 2017. [ Google Scholar ]
• Orrensalo, T.; Brush, C.; Nikou, S. Entrepreneurs’ information-seeking behaviors in the digital age–A systematic literature review. J. Small Bus. Manag. 2022 , 62 , 892–937. [ Google Scholar ] [ CrossRef ]
• Young, R.; Wahlberg, L.; Davis, E.; Abhari, K. Towards a theory of digital entrepreneurship mindset: The role of digital learning aptitude and digital literacy. In Proceedings of the 26th Americas Conference on Information Systems 2020, Virtual, 15–17 August 2020; pp. 1–10. [ Google Scholar ]
• Kim, H.J.; Hong, A.J.; Song, H.D. The relationships of family, perceived digital competence and attitude, and learning agility in sustainable student engagement in higher education. Sustainability 2018 , 10 , 4635. [ Google Scholar ] [ CrossRef ]
• Ng, W. Can we teach digital natives digital literacy? Comput. Educ. 2012 , 59 , 1065–1078. [ Google Scholar ] [ CrossRef ]
• Mohammadyari, S.; Singh, H. Understanding the effect of e-learning on individual performance: The role of digital literacy. Comput. Educ. 2015 , 82 , 11–25. [ Google Scholar ] [ CrossRef ]
• McLoughlin, C. What ICT-related skills and capabilities should be considered central to the definition of digital literacy? In EdMedia + Innovate Learning ; Association for the Advancement of Computing in Education (AACE): Asheville, NC, USA, 2011; pp. 471–475. [ Google Scholar ]
• Ogbari, M.E.; Olokundun, M.A.; Uzuegbunam, J.; Isiavwe, D.T.; Ilogho, J.E.; Obi, J.N.; Moses, C.L. Data on entrepreneurship education and entrepreneurial performance of aspiring entrepreneurs in selected Nigerian universities. Data Brief 2018 , 20 , 108–112. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Haneberg, D.H.; Aadland, T. Learning from venture creation in higher education. Ind. High. Educ. 2020 , 34 , 121–137. [ Google Scholar ] [ CrossRef ]
• Pinto Borges, A.; Lopes, J.M.; Carvalho, C.; Vieira, B.M.; Lopes, J. Education as a key to provide the growth of entrepreneurial intentions. Educ. Train. 2021 , 63 , 809–832. [ Google Scholar ] [ CrossRef ]
• Shahid, S.M.; Alarifi, G. Social entrepreneurship education: A conceptual framework and review. Int. J. Manag. Educ. 2021 , 19 , 100533. [ Google Scholar ]
• United Nations Educational Scientific and Cultural Organization (UNESCO). Education for Sustainable Development Goals: Learning Objectives ; UNESCO: Paris, France, 2017; Available online: http://unesdoc.unesco.org/images/0024/002474/247444e.pdf (accessed on 18 April 2024).
• Vidal, D.D.; Pittz, T.G.; Hertz, G.; White, R. Enhancing entrepreneurial competencies through intentionally-designed podcasts. Int. J. Manag. Educ. 2021 , 19 , 100537. [ Google Scholar ]
• Konstantinidis, S.; Leonardini, L.; Stura, C.; Richter, P.; Tessari, P.; Winters, M.; Balagna, O.; Farrina, R.; van Berlo, A.; Schlieter, H.; et al. Digital soft skills of healthcare workforce–identification, prioritization and digital training. In Proceedings of the International Conference on Interactive Collaborative Learning, Dresden, Germany, 22–24 September 2021; Springer International Publishing: Cham, Switzerland, 2021; pp. 1118–1129. [ Google Scholar ]
• Crupi, A.; Liu, S.; Liu, W. The top-down pattern of social innovation and social entrepreneurship. Bricolage and agility in response to COVID-19: Cases from China. RD Manag. 2022 , 52 , 313–330. [ Google Scholar ] [ CrossRef ]
• Secundo, G.; Mele, G.; Del Vecchio, P.; Elia, G.; Margherita, A.; Ndou, V. Threat or opportunity? A case study of digital-enabled redesign of entrepreneurship education in the COVID-19 emergency. Technol. Forecast. Soc. Chang. 2021 , 166 , 120565. [ Google Scholar ] [ CrossRef ]
• Viebig, C. Blended learning in entrepreneurship education: A systematic literature review. Educ. Train. 2022 , 64 , 533–558. [ Google Scholar ] [ CrossRef ]
• Almahdi, H.K. Promotion and participation of Saudi universities towards the development of entrepreneurial leadership—An empirical study in Saudi Arabian context. J. Entrep. Educ. 2019 , 22 , 1–14. [ Google Scholar ]
• Vision-2030. 2016. Available online: https://www.vision2030.gov.sa/media/rc0b5oy1/saudi_vision203.pdf (accessed on 18 April 2024).
• Mohammed Nasser Hassan Ja’ashan, M. The challenges and prospects of using e-learning among EFL students in Bisha University. Arab World Engl. J. (AWEJ) 2020 , 11 , 124–137. [ Google Scholar ]
• Crompton, H.; Burke, D. The use of mobile learning in higher education: A systematic review. Comput. Educ. 2018 , 123 , 53–64. [ Google Scholar ] [ CrossRef ]
• Amjad, T.; Rani, S.H.B.A.; Sa’Atar, S.B. Entrepreneurship development and pedagogical gaps in entrepreneurial marketing education. Int. J. Manag. Educ. 2020 , 18 , 100379. [ Google Scholar ] [ CrossRef ]
• Crown, D.; Faggian, A.; Corcoran, J. Foreign-Born graduates and innovation: Evidence from an Australian skilled visa program✰,✰✰,★,★★. Res. Policy 2020 , 49 , 103945. [ Google Scholar ] [ CrossRef ]
• Liguori, E.; Winkler, C. From offline to online: Challenges and opportunities for entrepreneurship education following the COVID-19 pandemic. Entrep. Educ. Pedagog. 2020 , 3 , 346–351. [ Google Scholar ] [ CrossRef ]
• Bakker, A.B.; Demerouti, E.; Sanz-Vergel, A.I. Burnout and work engagement: The JD–R approach. Annu. Rev. Organ. Psychol. Organ. Behav. 2014 , 1 , 389–411. [ Google Scholar ] [ CrossRef ]
• Burisch, R.; Wohlgemuth, V. Blind spots of dynamic capabilities: A systems theoretic perspective. J. Innov. Knowl. 2016 , 1 , 109–116. [ Google Scholar ] [ CrossRef ]
• Vogel-Heuser, B.; Bauernhansl, T.; Ten Hompel, M. Handbuch Industrie 4.0 Bd. 4. In Allgemeine Grundlagen ; Spriger: Berlin/Heidelberg, Germany, 2017; Volume 2. [ Google Scholar ]
• Gomis-Porqueras, P.; Meinecke, J.; Rodrigues-Neto, J.A. New technologies in higher education: Lower attendance and worse learning outcomes? Agenda 2011 , 18 , 69–83. [ Google Scholar ] [ CrossRef ]
• Bonk, C.; Graham, C.R. (Eds.) The Handbook of Blended Learning: Global Perspectives, Local Designs ; Pfeiffer Publishing: San Francisco, CA, USA, 2006. [ Google Scholar ]
• Alexander, C.K. Shock, Fear, and Fatalism: As Coronavirus Prompts Colleges to Close, Students Grapple with Uncertainty. Available online: https://www.chronicle.com/article/Shock-FearFatalism-As/248240 (accessed on 12 March 2020).
• Van Alten, D.C.; Phielix, C.; Janssen, J.; Kester, L. Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educ. Res. Rev. 2019 , 28 , 100281. [ Google Scholar ] [ CrossRef ]
• Garrison, D.R.; Vaughan, N.D. Blended Learning in Higher Education: Framework, Principles and Guidelines , 1st ed.; Jossey-Bass: San Francisco, CA, USA, 2008. [ Google Scholar ]
• Heubeck, T. Managerial capabilities as facilitators of digital transformation? Dynamic managerial capabilities as antecedents to digital business model transformation and firm performance. Digit. Bus. 2023 , 3 , 100053. [ Google Scholar ] [ CrossRef ]
• Zhou, X.; Cai, Z.; Tan, K.H.; Zhang, L.; Du, J.; Song, M. Technological innovation and structural change for economic development in China as an emerging market. Technol. Forecast. Soc. Chang. 2021 , 167 , 120671. [ Google Scholar ] [ CrossRef ]
• Duval-Couetil, N. Assessing the impact of entrepreneurship education programs: Challenges and approaches. J. Small Bus. Manag. 2013 , 51 , 394–409. [ Google Scholar ] [ CrossRef ]
• Boudreaux, C.J.; Nikolaev, B.N.; Klein, P. Socio-cognitive traits and entrepreneurship: The moderating role of economic institutions. J. Bus. Ventur. 2019 , 34 , 178–196. [ Google Scholar ] [ CrossRef ]
• Chell, E. Review of skill and the entrepreneurial process. Int. J. Entrep. Behav. Res. 2013 , 19 , 6–31. [ Google Scholar ] [ CrossRef ]
• Man, T.W.; Lau, T.; Snape, E. Entrepreneurial competencies and the performance of small and medium enterprises: An investigation through a framework of competitiveness. J. Small Bus. Entrep. 2008 , 21 , 257–276. [ Google Scholar ] [ CrossRef ]
• Satar, M.S.; Alarifi, G.; Alrubaishi, D. Exploring the entrepreneurial competencies of E-commerce entrepreneurs. Int. J. Manag. Educ. 2023 , 21 , 100799. [ Google Scholar ] [ CrossRef ]
• Abelha, M.; Fernandes, S.; Mesquita, D.; Seabra, F.; Ferreira-Oliveira, A.T. Graduate employability and competence development in higher education—A systematic literature review using PRISMA. Sustainability 2020 , 12 , 5900. [ Google Scholar ] [ CrossRef ]
• Atef, T.M.; Al-Balushi, M. Entrepreneurship as a means for restructuring employment patterns. Tour. Hosp. Res. 2015 , 15 , 73–90. [ Google Scholar ] [ CrossRef ]
• Neumeyer, X.; Santos, S.C.; Garrell, A. Poverty, entrepreneurship, and digitalization: A review and future research agenda. In Proceedings of the 2021 IEEE International Conference on Technology Management, Operations and Decisions (ICTMOD), Marrakech, Morocco, 24–26 November 2021; IEEE: New York, NY, USA, 2021; pp. 1–6. [ Google Scholar ]
• Hylton, K.; Levy, Y.; Dringus, L.P. Utilizing webcam-based proctoring to deter misconduct in online exams. Comput. Educ. 2016 , 92 , 53–63. [ Google Scholar ] [ CrossRef ]
• Ip, C.Y. Effect of digital literacy on social entrepreneurial intentions and nascent behaviours among students and practitioners in mass communication. Humanit. Soc. Sci. Commun. 2024 , 11 , 34. [ Google Scholar ] [ CrossRef ]
• Satar, M.S.; Alharthi, S.; Alarifi, G.; Omeish, F. Does Digital Capabilities Foster Social Innovation Performance in Social Enterprises? Mediation by Firm-Level Entrepreneurial Orientation. Sustainability 2024 , 16 , 2464. [ Google Scholar ] [ CrossRef ]
• Nambisan, S. Digital entrepreneurship: Toward a digital technology perspective of entrepreneurship. Entrep. Theory Pract. 2017 , 41 , 1029–1055. [ Google Scholar ] [ CrossRef ]
• Abdelaliem, S.M.F.; Elzohairy, M.H.S. The relationship between nursing students’ readiness and attitudes for E-learning: The mediating role of self-leadership: An online survey (comparative study). J. Prof. Nurs. 2023 , 46 , 77–82. [ Google Scholar ] [ CrossRef ]
• Ghafar, A. Convergence between 21st century skills and entrepreneurship education in higher education institutes. Int. J. High. Educ. 2020 , 9 , 218–229. [ Google Scholar ] [ CrossRef ]
• Erhel, S.; Jamet, E. Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness. Comput. Educ. 2013 , 67 , 156–167. [ Google Scholar ] [ CrossRef ]
• Kozlinska, I.; Rebmann, A.; Mets, T. Entrepreneurial competencies and employment status of business graduates: The role of experiential entrepreneurship pedagogy. J. Small Bus. Entrep. 2020 , 35 , 724–761. [ Google Scholar ] [ CrossRef ]
• Benner, M.J.; Tushman, M.L. Reflections on the 2013 Decade Award—“Exploitation, exploration, and process management: The productivity dilemma revisited” ten years later. Acad. Manag. Rev. 2015 , 40 , 497–514. [ Google Scholar ] [ CrossRef ]
• Cronje, J. Towards a new definition of blended learning. Electron. J. e-Learn. 2020 , 18 , 114–121. [ Google Scholar ] [ CrossRef ]
• Kudrik, Y.; Lahn, L.C.; Mørch, A.I. Technology-enhanced workplace learning: Blended learning in insurance company. In Proceedings of the 17th International Conference on Computers in Education, Hong Kong, China, 30 November–4 December 2009; Asia-Pacific Society for Computers in Education: Jhongli, Taiwan, 2009. [ Google Scholar ]
• Dziuban, C.; Graham, C.R.; Moskal, P.D.; Norberg, A.; Sicilia, N. Blended learning: The new normal and emerging technologies. Int. J. Educ. Technol. High. Educ. 2018 , 15 , 3. [ Google Scholar ] [ CrossRef ]
• Linkous, H.M. Self-Directed Learning and Self-Regulated Learning: What’s the Difference? A Literature Analysis. In Proceedings of the American Association for Adult and Continuing Education 2020 Conference, Online, 27–30 October 2020. [ Google Scholar ]
• Rusko, R. Interactive online learning: Solution to learn entrepreneurship in sparsely populated areas? Int. J. Online Pedagog. Course Des. (IJOPCD) 2017 , 7 , 54–69. [ Google Scholar ] [ CrossRef ]
• Harichandran, R.S.; Erdil, N.O.; Carnasciali, M.I.; Nocito-Gobel, J.; Li, C. Developing an Entrepreneurial Mindset in Engineering Students Using Integrated E-Learning Modules. Adv. Eng. Educ. 2018 , 7 , n1. [ Google Scholar ]
• Harima, A.; Gießelmann, J.; Göttsch, V.; Schlichting, L. Entrepreneurship? Let us do it later: Procrastination in the intention–behavior gap of student entrepreneurship. Int. J. Entrep. Behav. Res. 2021 , 27 , 1189–1213. [ Google Scholar ] [ CrossRef ]
• Müller, C.; Mildenberger, T.; Steingruber, D. Learning effectiveness of a flexible learning study programme in a blended learning design: Why are some courses more effective than others? Int. J. Educ. Technol. High. Educ. 2023 , 20 , 10. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Müller, C.; Poandl, E.; Glinik, M. Developing a viable business model for start-ups at the Gruendungsgarage. J. Bus. Models 2019 , 7 , 47–56. [ Google Scholar ]
• Bellotti, F.; Berta, R.; De Gloria, A.; Lavagnino, E.; Antonaci, A.; Dagnino, F.; Ott, M.; Romero, M.; Usart, M.; Mayer, I. Serious games and the development of an entrepreneurial mindset in higher education engineering students. Entertain. Comput. 2014 , 5 , 357–366. [ Google Scholar ] [ CrossRef ]
• Boore, J.; Porter, S. Education for entrepreneurship in nursing. Nurse Educ. Today 2011 , 31 , 184–191. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bandera, C.; Collins, R.; Passerini, K. Risky business: Experiential learning, information and communications technology, and risk-taking attitudes in entrepreneurship education. Int. J. Manag. Educ. 2018 , 16 , 224–238. [ Google Scholar ] [ CrossRef ]
• Waghid, Z.; Oliver, H. Cultivating social entrepreneurial capacities in students through film: Implications for social entrepreneurship education. Educ. Res. Soc. Chang. 2017 , 6 , 76–100. [ Google Scholar ] [ CrossRef ]
• Pisoni, G. Strategies for pan-european implementation of blended learning for innovation and entrepreneurship (I & E) education. Educ. Sci. 2019 , 9 , 124. [ Google Scholar ] [ CrossRef ]
• Kazakeviciute, A.; Urbone, R.; Petraite, M. Curriculum development for technology-based entrepreneurship education: A cross-disciplinary and cross-cultural approach. Ind. High. Educ. 2016 , 30 , 202–214. [ Google Scholar ] [ CrossRef ]
• Dickfos, J.; Cameron, C.; Hodgson, C. Blended learning: Making an impact on assessment and self-reflection in accounting education. Educ. Train. 2014 , 56 , 190–207. [ Google Scholar ] [ CrossRef ]
• Treanor, L.; Noke, H.; Marlow, S.; Mosey, S. Developing entrepreneurial competences in biotechnology early career researchers to support long-term entrepreneurial career outcomes. Technol. Forecast. Soc. Chang. 2021 , 164 , 120031. [ Google Scholar ] [ CrossRef ]
• Wong, T.A.; Cheah, W.C. Triple-blend of classroom instruction, online facilitation and external exposure: Outcomes and challenges. Ind. High. Educ. 2022 , 36 , 151–162. [ Google Scholar ] [ CrossRef ]
• Stefanic, I.; Campbell, R.K.; Russ, J.S.; Stefanic, E. Evaluation of a blended learning approach for cross-cultural entrepreneurial education. Innov. Educ. Teach. Int. 2020 , 57 , 242–254. [ Google Scholar ] [ CrossRef ]
• Hua, M.; Wang, L.; Li, J. The impact of self-directed learning experience and course experience on learning satisfaction of university students in blended learning environments: The mediating role of deep and surface learning approach. Front. Psychol. 2024 , 14 , 1278827. [ Google Scholar ] [ CrossRef ]
• Halverson, L.R.; Graham, C.R. Learner engagement in blended learning environments: A conceptual framework. Online Learn. 2019 , 23 , 145–178. [ Google Scholar ] [ CrossRef ]
• Van Dorresteijn, C.; Fajardo Tovar, D.; Pareja Roblin, N.; Cornelissen, F.; Meij, M.; Voogt, J.; Volman, M. What Factors Contribute to Effective Online and Blended Education? (Summary) ; Universiteit van Amsterdam: Amsterdam, The Netherlands, 2020. [ Google Scholar ]
• Nabi, G.; Holden, R.; Walmsley, A. Graduate career-making and business start-up: A literature review. Educ. Train. 2006 , 48 , 373–385. [ Google Scholar ] [ CrossRef ]
• Levy, Y.; Ramim, M.M. The e-learning skills gap study: Initial results of skills desired for persistence and success in online engineering and computing courses. In Proceedings of the 12th Chais Conference for the Study of Innovation and Learning Technologies: Learning in the Technological Era, Raanana, Israel, 14–15 February 2017; pp. 57–68. [ Google Scholar ]
• Mittal, P.; Raghuvaran, S. Entrepreneurship education and employability skills: The mediating role of e-learning courses. Entrep. Educ. 2021 , 4 , 153–167. [ Google Scholar ] [ CrossRef ]
• Abdelwahed, N.A.A.; Al Doghan, M.A.; Saraih, U.N.; Soomro, B.A. Green entrepreneurship in Saudi Arabia: Shaping the landscape of the greener economy. J. Small Bus. Enterp. Dev. 2023 , 30 , 1352–1376. [ Google Scholar ] [ CrossRef ]
• Bullen, M.; Morgan, T.; Qayyum, A. Digital learners in higher education: Generation is not the issue. Can. J. Learn. Technol. Rev. Can. L’apprentissage Technol. 2011 , 37 , 1–24. [ Google Scholar ] [ CrossRef ]
• Lai, K.W.; Hong, K.S. Technology use and learning characteristics of students in higher education: Do generational differences exist? Br. J. Educ. Technol. 2015 , 46 , 725–738. [ Google Scholar ] [ CrossRef ]
• Satar, M.S. Compassion, Value Creation and Digital Learning Orientation in Social Entrepreneurs, Management Decision. 2024; unpublished work . [ Google Scholar ]
• Ballouk, R.; Mansour, V.; Dalziel, B.; McDonald, J.; Hegazi, I. Medical students’ self-regulation of learning in a blended learning environment: A systematic scoping review. Med. Educ. Online 2022 , 27 , 2029336. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nikolopoulou, K.; Zacharis, G. Blended learning in a higher education context: Exploring university students’ learning behavior. Educ. Sci. 2023 , 13 , 514. [ Google Scholar ] [ CrossRef ]
• Ahmad, N.H. A Cross Cultural Study of Entrepreneurial Competencies and Entrepreneurial Success in SMEs in Australia and Malaysia. Ph.D. Thesis, University of Adelaide, Adelaide, Australia, 2007. [ Google Scholar ]
• Man, W.Y.T. Entrepreneurial Competencies and the Performance of Small and Medium Enterprises in the Hong Kong Services Sector. Ph.D. Thesis, Hong Kong Polytechnic University, Hong Kong, China, 2001. [ Google Scholar ]
• Naman, J.L.; Slevin, D.P. Entrepreneurship and the concept of fit: A model and empirical tests. Strateg. Manag. J. 1993 , 14 , 137–153. [ Google Scholar ] [ CrossRef ]
• Hindle, K.; Yencken, J. Public research commercialisation, entrepreneurship and new technology based firms: An integrated model. Technovation 2004 , 24 , 793–803. [ Google Scholar ] [ CrossRef ]
• Smith, B.; Morse, E. Entrepreneurial competencies: Literature review and best practices. In Small Business Policy Branch ; Industry Canada: Ottawa, ON, Canada, 2005. [ Google Scholar ]
• Chandler, G.N.; Hanks, S.H. Founder competence, the environment, and venture performance. Entrep. Theory Pract. 1994 , 18 , 77–89. [ Google Scholar ] [ CrossRef ]
• Hair, J.F.; Hult, G.T.M.; Ringle, C.M.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM) , 2nd ed.; Sage Publication: Thousand Oaks, CA, USA, 2017. [ Google Scholar ]
• Henseler, J.; Dijkstra, T.K.; Sarstedt, M.; Ringle, C.M.; Diamantopoulos, A.; Straub, D.W.; Ketchen, D.J., Jr.; Hair, J.F.; Hult, G.T.M.; Calantone, R.J. Common beliefs and reality about PLS: Comments on Rönkkö and Evermann (2013). Organ. Res. Methods 2014 , 17 , 182–209. [ Google Scholar ] [ CrossRef ]
• Satar, M.S.; Alshibani, S.M.; Alarifi, G. Effects of Firm-Level Entrepreneurship Orientation on Digital Transformation in SMEs: The Moderating Role of Strategic Agility. Entrep. Res. J. 2024 . [ Google Scholar ] [ CrossRef ]
• Chin, W.W.; Marcolin, B.L.; Newsted, P.R. A partial least squares latent variable modeling approach for measuring interaction effects: Results from a Monte Carlo simulation study and an electronic-mail emotion/adoption study. Inf. Syst. Res. 2003 , 14 , 189–217. [ Google Scholar ] [ CrossRef ]
• Hair, J.F.; Hult, G.T.M.; Ringle, C.M.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM) ; Sage: Thousand Oaks, CA, USA, 2013. [ Google Scholar ]
• Fornell, C.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981 , 18 , 39–50. [ Google Scholar ] [ CrossRef ]
• Hair, J.F.; Risher, J.J.; Sarstedt, M.; Ringle, C.M. When to use and how to report the results of PLS-SEM. Eur. Bus. Rev. 2019 , 31 , 2–24. [ Google Scholar ] [ CrossRef ]
• Harman, H.H. Modern Factor Analysis ; University of Chicago Press: Chicago, IL, USA, 1976. [ Google Scholar ]
• Shrout, P.E.; Bolger, N. Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychol. Methods 2002 , 7 , 422. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tenenhaus, M.; Vinzi, V.E.; Chatelin, Y.M.; Lauro, C. PLS path modeling. Comput. Stat. Data Anal. 2005 , 48 , 159–205. [ Google Scholar ] [ CrossRef ]
• MacKenzie, S.B.; Podsakoff, P.M.; Jarvis, C.B. The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. J. Appl. Psychol. 2005 , 90 , 710–730. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Annarelli, A.; Battistella, C.; Nonino, F.; Parida, V.; Pessot, E. Literature review on digitalization capabilities: Co-citation analysis of antecedents, conceptualization and consequences. Technol. Forecast. Soc. Chang. 2021 , 166 , 120635. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Satar, M.S.; Alharthi, S.; Omeish, F.; Alshibani, S.M.; Saqib, N. Digital Learning Orientation and Entrepreneurial Competencies in Graduates: Is Blended Learning Sustainable? Sustainability 2024 , 16 , 7794. https://doi.org/10.3390/su16177794

Satar MS, Alharthi S, Omeish F, Alshibani SM, Saqib N. Digital Learning Orientation and Entrepreneurial Competencies in Graduates: Is Blended Learning Sustainable? Sustainability . 2024; 16(17):7794. https://doi.org/10.3390/su16177794

Satar, Mir Shahid, Sager Alharthi, Fandi Omeish, Safiya Mukhtar Alshibani, and Natasha Saqib. 2024. "Digital Learning Orientation and Entrepreneurial Competencies in Graduates: Is Blended Learning Sustainable?" Sustainability 16, no. 17: 7794. https://doi.org/10.3390/su16177794

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• My Shodhganga
• Receive email updates
• Edit Profile

Shodhganga : a reservoir of Indian theses @ INFLIBNET

• Shodhganga@INFLIBNET
• University of Mumbai
• Education, Pillai College of Education and Research

Items in Shodhganga are licensed under Creative Commons Licence Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).