higher order thinking skills research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

A Systematic Review of Higher-Order Thinking by Visualizing its Structure Through HistCite and CiteSpace Software

1 College of Education, Capital Normal University, Beijing, China

2 College of Teacher Education, Capital Normal University, Beijing, China

3 Office of International Cooperation and Exchange, Dongguan University of Technology, Dongguan, China

4 College of Resource Environment and Tourism, Capital Normal University, Beijing, China

With the continuous reform of the education curriculum, the focus of the curriculum is to provide personal and social needs-related knowledge. In the teaching process for this type of knowledge, the cultivation of students' higher-order thinking has aroused widespread concern in the education field. In this paper, CiteSpace and HistCite bibliometric methods are utilized to analyse the higher-order thinking literature in the core collection of Web of Science. Through an analysis of the number of articles, the research topic words and the countries from which the papers were sent, the overall characteristics of international higher-order thinking research are summarized. The primary finding of this research is that the field of higher-order thinking is on the rise. Secondary findings are the focus of higher-order thinking research, which can be divided into four aspects: higher-order thinking ability, instruction of higher-order thinking, the curriculum and sections of higher-order thinking, and higher-order thinking learning. In short, this research helps researchers understand the development of higher-order thinking, provides a theoretical foundation and an entry point for research directions, and not only provides meaningful and valuable references for follow-up related research but also promotes the development of higher-order thinking practice.

Introduction

In the early part of the 20th century, education focused on the acquisition of basic literacy skills: reading, writing, and calculating. Complex understanding was thought to occur only by the accumulation of basic, prerequisite learning. Therefore, most schools did not teach students to think and read critically or to solve complex problems. Textbooks were filled with facts that students were expected to memorize, and most tests assessed students’ ability to remember these facts. The main role of teachers was perceived as transmitting information to students (Bransford et al., 2000 ). Learning objectives were sequenced to progress from simple, lower-order cognitive tasks to more complex ones. With the unprecedented growth of information and knowledge, the meaning of “knowing” is changing from the ability to remember and repeat information to the ability to discover and use information effectively. People not only learn simple and low-level knowledge but also develop a non-algorithmic, complex thinking mode, which is higher-order thinking. Such thinking involves uncertainty, the application of multiple criteria, reflection, and self-regulation (Resnick, 1987 ). Creativity and innovation are important keys to success in any field in this era of rapid development. The generation of ideas involves activities for higher-order thinking skills that require high-level creative thinking and action. However, it is difficult to generate good ideas. Idea generation occurs in the human brain through cognitive, metacognitive, chemical and biological processes. Hence, complex thinking skills such as problem solving, creating, analysing, and evaluating are needed to process collected information to generate an idea. Furthermore, higher-order thinking challenges us to interpret, analyse or manipulate information. With higher-order thinking, an individual can utilize new information or prior knowledge and manipulate information to obtain a reasonable response to new situations. Consequently, creative ideas can only be generated through higher-order thinking rather than through low-level thinking through the application of knowledge learned in daily life. Therefore, in the changing world, it is becoming increasingly important to develop people's higher-order thinking. With the continuous development of education, the goal of cultivating learners’ higher-order thinking has been emphasized in most educational practices. The focus of teaching has gradually shifted from traditional textbook-based rote learning to inquiry learning in the real world to better develop students' higher-order thinking.

Currently, there are many studies on the theory and practice of higher-order thinking. A large number of studies apply the development of higher-order thinking to the classroom and assess the development of students’ higher-order thinking through testing. For example, Zohar and Peled ( 2008 ) tested the learning status of low achievers in high-level teaching and learning processes through four different experiments. Miri et al., ( 2007 ) applied the longitudinal case study method to explore the relationship between purposefully promoting the teaching of higher-order thinking skills and improving students’ critical thinking ability. Mcloughlin and Mynard ( 2009 ) conducted research on online forums to promote higher-order thinking tools. Nonetheless, in the context of teaching higher-order thinking, the classic conceptual distinction made in the literature between pedagogical content knowledge and general pedagogical knowledge is fuzzy and unclear. It is difficult for teachers to combine their knowledge of teaching about thinking with the commonly applied concepts in the literature. This issue is related to scholars' debate about whether thinking strategies are general or specific. According to the method of inculcation, the teaching of thinking is combined with the teaching of specific content; thinking skills have general elements, while other elements are related to specific content. Thus, a systematic review of higher-order thinking education is important and appropriate to examine current applications and trends in this field.

This study applies an innovative approach to conduct the literature review through a systematic review with the support of HistCite and CiteSpace software. The research tools used in this study are HistCite and CiteSpace software because they are able to link and visualize the citation history and citation structure of articles in graphical form (Chaomei et al., 2012 ; Chen, 2006 ; Garfield, 2009 ; Lucio-Arias & Leydesdorff, 2008 ). The combined application and support of this software can increase time efficiency and supplement the judgement and understanding demands involved in systematic review analysis. HistCite is free software. It can generate a chronological map of bibliography collections generated by topic, author, institution or source journal searches of the ISI science network. Web of Science (WoS) export files are created in which all cited references for each source document are captured. This provides an evidence map from listed journals that can guide and reinforce researchers' decisions and interpretations. CiteSpace software divides the network into clusters by identifying rapidly growing subject areas and discovering citation hotspots in the field of publications. CiteSpace software then simplifies WoS data for understanding and interpreting the chronological structure and connection with past research patterns and automatically marks clusters using citation terms, geospatial collaboration patterns and unique international collaboration fields. This free software with Java applications supports a unique type of co-citation network analysis—progressive network analysis—based on a time slicing strategy. A series of individual network snapshots defined on continuous time slice nodes are synthesized. These nodes play a key role in the evolution of the network and are turning points of knowledge. A systematic review is an orderly way to review and summarize research. The Cochrane Collaboration noted that systematic review provides a high-level overview of preliminary research on a specific research problem. This study attempts to identify, select, synthesize and evaluate all relevant high-quality research evidence to answer this question. Tsai ( 2013 ) noted that the purpose of the assessment of the education system is to obtain a clearer understanding of the recent situation. Furthermore, systematic analysis procedures are important in identifying associations between instructional design and the theoretical characteristics and best practices of research study. However, systematic evaluation in education has received little attention. This is due to the labour-intensive nature of scientific education methods and the need for researchers to judge and understand the potential value of research quality and results. Therefore, this study aims to utilize established procedures, identify connected structures, and address research gaps or incompleteness in this area for future studies. It is appropriate and worthwhile to use HistCite and CiteSpace software to systematically review research on higher-order thinking.

This study presents a systematic review of the literature related to higher-order thinking. In this way, it reveals the evolutionary path, research focus and research frontier of higher-order thinking research and provides a reference for research in related fields of higher-order thinking. Scholars reveal the development of practice from different theoretical perspectives and systematically pay attention to these issues when returning to these documents: What is the current research status of higher-order thinking? What is its evolution process? What research highlights does it include? When will higher-order thinking fully develop? In view of this, this paper employs a bibliometric method to analyse the development trends of higher-order thinking publications for the period of 1984–2020. Through an in-depth discussion of the number of papers and references about higher-order thinking, this paper can provide the basis for research on higher-order thinking and education.

Methodology

This study focuses on previous research studies on higher-order thinking, and the main source of journal exploration is the WoS database. In the WoS database, “higher-order thinking” or “higher order thinking” was used as the subject search term to retrieve all articles about higher-order thinking from 1984 to 2020. With the support of HistCite and CiteSpace software, the research process was divided into two main parts. First, HistCite analysis was conducted to identify the universe of articles and their citation links to central studies. Second, higher-order thinking studies were identified and isolated through HistCite and CiteSpace analysis.

Part 1—HistCite Analysis Procedures

In the Wos database, 1349 articles from 1984 to 2020 were retrieved by using “higher-order thinking” or “higher order thinking” as the subject search term. After confirming the publication year from 1984 to 2020 and deleting the literature with uncertain publication years, a total of 1320 articles were selected to meet the research requirements. The structure of the studies and relationships among the 1,320 papers identified in the WoS can be analysed by HistCite software (Fig.  1 ). HistCite software was applied to the journal lists to generate chronological historiographs (i.e., a time-based network diagram) based on the relationship of the cited works, which is the number of citations of a paper within the collection. This was followed by a HistCite analysis of higher-order thinking to identify the citation patterns and select related studies for further in-depth document analysis. General information about the results is provided on the first line of the first page (Fig.  1 ) of the HistCite file that includes higher-order thinking. The abbreviation LCS stands for Local Citation Score and is the number of times a paper is cited by other papers in the local collection; GCS stands for Global Citation Score and shows the Citation frequency based on the total count in the Web of Science; LCR stands for Local Cited References and shows the number of references citing local papers; and CR stands for the number of cited references and shows the number of all citing references (for the use of the HistCite methods for algorithmic historiography, see (Garfield, 2004 ; Garfield et al., 2003 )). Due to the subscription history of the library and the coverage of WoS, the collection spans from 1984 to 2020. The timeline for the growth of higher-order thinking is exhibited by a HistCite presentation of the ranked citation index of 1,320 research articles within 706 journals by 3,042 authors and with 36,868 cited references.

The HistCite file of article publications linked to the field of higher-order thinking

Figure  2 shows the HistCite graphmaker display of the historiograph of higher-order thinking that was generated based on the LCS. Figure  2 shows the complete citation relationship of the top 30 most valuable articles from 1984 to 2020, and the time span of these 30 articles is from 1990 to 2017. This includes the relationship of cited works with circle diameters proportional to the LCS as well as an arrow exhibiting the citation direction. The citation direction arrows show that 20 articles were either cited by others or cited other works within the WoS collection and illustrate the relationship of citations between papers. The larger the diameter of the circle, the more often this literature has been cited and the more important this article is in the field of higher order thinking. The number next to the circle indicates the serial number of the article among the 1320 articles. Figure  2 shows that the larger diameter of the circles corresponding to the 91st, 143rd, and 8th papers means that these three papers have been cited the most. The 91st is Higher Order Thinking Skills and Low-Achieving Students: Are They Mutually Exclusive? published by Anat Zohar et al. in 2003 . The 143rd is Purposely Teaching for the Promotion of Higher-order Thinking Skills: A Case of Critical Thinking published by Barak Miri et al. in 2007. The 8th is higher-order thinking in teaching social studies: a rationale for the assessment of classroom thoughtfulness published by Fred M. Newmann in 1990. Of these, the arrow pointing to the 91st paper has the highest number of citations, representing the fact that this paper has been cited most often by other papers in the same field, indicating that this paper has significant scholarly influence in the field of higher-order thinking.

HistCite graphmaker display of the historiographs of the top 30 higher-order thinking studies

Part 2—CiteSpace Analysis Procedures

The identified higher-order thinking database was submitted to CiteSpace analysis to report the cluster terms and search for other important articles through the cited references or bibliographic collections. The purpose of this study was to search for important higher-order thinking articles that are frequently cited within the community. The CiteSpace analysis data originated from 1320 articles published between 1984 and 2020. Figure  3 presents a timeline visualization of the clusters with automatically created labels (only highly cited papers in major clusters are shown). The cluster labels shown are useful for understanding the research scope or direction of higher-order thinking because these terms are frequently applied within the community. Furthermore, these terms are very useful for conducting research related to higher-order thinking development, the results, discussion, conclusions, and suggestions. Next, a narrative was generated for the analysis of the largest cluster. The automatically chosen cluster labels of the 14 largest clusters along with their size, identity number and silhouette value in brackets are presented. The largest cluster of higher-order thinking (#0) was educational robotics, which was active from 1992 to 2020. The second largest cluster (#1) was teachers’ belief, which was active from 1993 to 2020. The third largest cluster was socio- scientific issues (#2). The fourth to tenth largest clusters were project management learning (#3), deep learning (#4), executive functioning (#5), digital learning game (#6), web-based hypermedia resource (#7), social competence (#8) and processing deficit (#9).

Cluster labels and terms generated from 1984 to 2020

Results and Analysis

Research status and analysis of evolutionary characteristics.

As shown in Fig.  4 , the research literature on higher-order thinking in the WoS core database is counted by time distribution. From the perspective of the total amount and increment of documents, international research on higher-order thinking presents two trends. First, the number of research papers in the field of higher-order thinking remained stable from 1984 to 2007. With no more than 20 papers published each year, international research on higher-order thinking is in the exploratory stage. The number of papers published in 2008 was 21, which marks the turning point in higher-order thinking research. After 2008, the number of higher-order thinking research papers increased year by year. Since 2014, the number of papers has increased sharply, reaching a peak of 180 in 2018. Research on higher-order thinking has entered the outbreak period, showing a vigorous development trend and the new vitality of international higher-order thinking research. Less research on higher-order thinking was published in 2019 than in 2018. It is presumed that the reason for its decline may be influenced by COVID-19. Some experiments and teaching practices in higher-order thinking research do not have a corresponding environment. One possible reason for the decline in the number of papers in 2020 is due to the limited speed of updates to the WoS database, and some papers published in 2020 are still not included in the database. Another may be due to the bottleneck in higher-order thinking after research has gone through theoretical exploration, coupled with the impact of the epidemic and the inability to conduct experiments. Therefore, this paper studies the number of published papers combined with high-order thinking, draws the development trend of different periods in the form of arrows, and predicts the trend of the number of published papers in 2020 with the dotted line.

Annual distribution of the number of higher-order thinking research documents

Higher-Order Thinking Research Highlights and Themes

An analysis of higher-order thinking through CiteSpace software was conducted to study the keyword co-word network knowledge map, as shown in Fig.  5 . As shown in the figure, “critical thinking” was the most influential core of the network. Terms such as skills, instruction, achievement and metacognition have great influence, reflecting hot subjects in higher-order thinking research. The top 30 high-frequency keywords according to the summary of the software analysis results are shown. These keywords represent important research content in the current study of higher-order thinking and reflect the research focus in the field to some extent. Among these keywords, the frequency of “critical thinking” is 55, so “critical thinking” is the core of the research field. In addition, skills, instruction and achievement appear relatively frequently, which indicates that they are hotspots of higher-order thinking research. Overall, combined with the high-frequency keywords of higher-order thinking research and the content of the classical literature, hot topics in higher-order thinking research in recent years include higher-order thinking ability, the instruction of higher-order thinking, the curriculum of higher-order thinking, and higher-order thinking learning.

Co-word network atlas of higher-order thinking research (1984–2020)

As shown in Table ​ Table1, 1 , the first aspect concerns research on higher-order thinking ability. High-frequency keywords included in the research hotspots are critical thinking (55), skill (54), higher-order thinking skill (29), metacognition (26), higher-order thinking skill (17), perception (14), literacy (14) and problem solving (13). For instance, Hwang et al. ( 2017 ) consider higher-order thinking skills to include problem solving, critical thinking, teamwork, communication, and creative thinking. Apino and Retnawati ( 2017 ) consider skills that involve students analysing, synthesising and evaluating and creating. In addition, critical, systematic and creative thinking (Miri et al., 2007 ) and metacognitive processes (Hmelo & Ferrari, 1997 ) are all part of higher-order thinking.

List of the top 30 high-frequency keywords of research on higher-order thinking

The second aspect involves research on instruction in higher-order thinking. Higher-order thinking requires good teaching practices and exercise. High-frequency keywords contained in the topic include instruction (31), assessment (31), performance (30), achievement (30), design (28), strategy (22), technology (21), blooms taxonomy (21), model (20), impact (18), pedagogy (17), feedback (16), argumentation (16), and environment (15). For example, Zohar and Dori ( 2003 ), Tony ( 2008 ) and Mahoney and Harris-Reeves ( 2019 ) argued that higher-order thinking stems from Bloom's categories, in which analysis, synthesis and evaluation are higher-order thinking. For another example, Hwang et al. ( 2019 ) thought that the development of students’ higher-order thinking skills can be facilitated through innovative flipped learning strategies and support systems. In addition, approaches such as game design thinking-based instruction (Mohamad et al., 2019 ) and peer assessment (Hadzhikoleva et al., 2019 ) are conducive to developing learners' higher-order thinking in teaching and learning. As another example, Chiong and Lim ( 2020 ) explored the important role of family in the development of higher-order thinking skills under the influence of social policies and found that family factors have a significant role in influencing the development of learners' higher-order thinking.

The third aspect involves research on the curriculum and sections of higher-order thinking. High-frequency keywords contained in this research hotspot include science (47), curriculum (21), higher education (20), mathematics (16), children (15), science education (14) and elementary school (13). For instance, Nicely ( 1985 ) explored higher-order thinking as it is addressed in mathematics materials in primary and secondary schools. Miri et al. ( 2007 ) discussed whether students' critical thinking can be enhanced through purposeful teaching aimed at improving higher-order thinking skills within the framework of science education. FitzPatrick and Schulz ( 2015 ) fand that the importance of higher-order thinking is widely recognized in science education, where higher-order thinking is required.

The fourth aspect involves research on higher-order thinking learning. High-frequency keywords contained in this research hotspot include attitude (14) and active learning (14). For example, Lee and Choi’s ( 2017 ) study confirmed that learners' attitudes towards technology use are related to higher-order thinking. Wu et al. ( 2019 ) found that learners' learning styles and motivation are related to the development of higher-order thinking; Saud et al. ( 2017 ) believed active learning can develop students' higher-order thinking skills.

Hot Countries/Regions Analysis of Higher-Order Thinking Research

Figure  6 shows the hot research countries/regions generated by CiteSpace software. This figure sets “topnperslice” to 50; that is, 50 countries/regions with the highest reference frequency are selected every year to obtain a hot research country/region chart composed of 84 nodes and 138 connections. The statistical table of the top 10 countries/regions is selected, as shown in Table ​ Table2. 2 . As seen in Fig.  6 and Table ​ Table2, 2 , the hot spots of higher-order thinking research show a regional imbalance: researchers mainly concentrated in North America, represented by the USA and Canada. This is followed by Asia, represented by Malaysia, Taiwan Province of China, Israel, Turkey and Indonesia. In the third place is Oceania represented by Australia. Last, Europe is represented by England and Spain. There are 138 connections between countries/regions, which shows that there is considerable exchange and cooperation between countries/regions in research on higher-order thinking. It is necessary to further strengthen cross-regional academic exchanges and cooperation to promote the sustainable development of higher-order thinking and to promote the enrichment and development of innovative teaching modes to adapt to the information society.

Collaborative network of countries/regions for higher-order thinking research

Top 10 countries/regions in total frequency of higher-order thinking research

Discussion of the Results

All findings in this study are derived from the results of the above analysis and from previous research. The primary finding of this research is that in terms of research progress, international higher-order thinking research has generally shown an upward trend, especially after 2014, when research related to higher-order thinking entered a period of rapid development. This may be because higher-order thinking has long been identified as a key predictor of success (Lee & Choi, 2017 ). In the knowledge economy, higher-order thinking is even considered a fundamental competency requirement for the 21st century (Permatasari et al., 2018 ; Wu et al., 2019 ; Yao, 2012 ). This requires educational reforms in many countries or regions (Fensham & Bellocchi, 2013 ; Singh et al., 2018 ) and education to focus more on the development and development of students’ higher-order competencies (Kurniawati, 2019 ). Therefore, influenced by educational policies and developments, an increasing number of scholars have launched research on higher-order thinking. Furthermore, the hotspot regions for research on higher-order thinking are mostly in countries such as the Americas, Asia and Europe.

The secondary finding of this study is that the hotspots of higher-order thinking research can be summarized in four areas: higher-order thinking ability, instruction of higher-order thinking, the curriculum and sections of higher-order thinking, and higher-order thinking learning. One explanation for this may be that higher-order thinking, as opposed to lower-order thinking (Abdullah et al., 2017 ), requires a high level of thinking that encompasses multiple complex abilities (Apino & Retnawati, 2017 ; Hwang et al., 2017 ; Miri et al., 2007 ; Zohar, 2013). Another possible explanation is that researchers believe that education is the main driving force promoting the development of learners’ higher-order thinking, which requires specific methods, strategies, and media to be cultivated (Rubini et al., 2019 ), as technology plays an important role in developing learners’ higher-order thinking (Duan & Ieee, 2012 ; Lee & Choi, 2017 ). Moreover, studies have found that the development of higher-order thinking is influenced by students’ own factors (Lee & Choi, 2017 ; Wu et al., 2019 ).

Implications

The significance of this study is manifested in the following two aspects. First, this study adopts new bibliometric methods and tools (HistCite and CiteSpace) and provides a more comprehensive tracking and visualization of research results in the field of higher-order thinking with more accurate data support and a more diversified analysis perspective, breaking through the limitations of traditional methods of literature content analysis. Second, this study uses the visualization method to quickly locate the key research results in the field of higher-order thinking, clarify the development history of higher-order thinking research, and more accurately identify the most active research frontiers and development trends, which provides a global knowledge map and literature basis for subsequent researchers to carry out in-depth research on higher-order thinking and has certain significance for the research and practice of higher-order thinking.

Limitations and Directions for Future Research

This study has some limitations. First, the literature data in the field of higher-order thinking are from WOS core journal papers, which are highly representative but still have some articles not included in time due to the limitation of the accession time, and not all data are available for the year 2020. In future studies, we will continue to track the literature of 2020 and expand the number and types of literature, such as adding conference papers, education policies, and other types of literature. Second, we cannot rule out the case that individual literature was not retrieved because it did not use a uniform formulation of higher-order thinking, but it does not affect the results of this study too much.

Author Contributions

JL and YX designed the research; YM performed the literature research and data analysis; JL, YX, YM, and XS wrote the paper; ZZ reviewed and edited the paper.

Declarationss

The authors declare no conflict of interest.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Jun Liu, Email: moc.liamxof@uncnujuil .

Yue Ma, Email: [email protected] .

Xue Sun, Email: moc.liamxof@yrrehsnuseux .

Ziqi Zhu, Email: moc.liamxof@tugdiqizuhz .

Yanhua Xu, Email: moc.liamxof@udeuxauhnay .

• Abdullah AH, Mokhtar M, Abd Halim ND, Ali DF, Tahir LM, Kohar UHA. Mathematics teachers’ level of knowledge and practice on the implementation of higher-order thinking skills (HOTS) Eurasia Journal of Mathematics Science and Technology Education. 2017; 13 (1):3–17. doi: 10.12973/eurasia.2017.00601a. [ CrossRef ] [ Google Scholar ]
• Apino E, Retnawati H. Developing instructional design to improve mathematical higher order thinking skills of students. In: Hidayat T, Nandiyanto ABD, Jupri A, Suwarma IR, Riza LS, Kusumawati D, Suhendi D, Munawaroh HSH, Aisyah S, Iriani M, editors. International seminar on mathematics, science, and computer science education. IOP Publishing; 2017. [ Google Scholar ]
• Bransford J, Brown A, Cocking R. How people learn: brain, mind, experience, and school. Expanded edition. National Academy Press; 2000. [ Google Scholar ]
• Chen C, Hu Z, Liu S, Tseng H. Emerging trends in regenerative medicine: A scientometric analysis in CiteSpace. Expert Opinion on Biological Therapy. 2012; 12 (5):593–608. doi: 10.1517/14712598.2012.674507. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chen C. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology. 2006; 57 (3):359–377. doi: 10.1002/asi.20317. [ CrossRef ] [ Google Scholar ]
• Chiong C, Lim L. Seeing families as policy actors: Exploring higher-order thinking reforms in Singapore through low-income families’ perspectives. Journal of Education Policy. 2020 doi: 10.1080/02680939.2020.1777468. [ CrossRef ] [ Google Scholar ]
• Duan, J. J., & Ieee. (2012). Research about technology enhanced higher-order thinking. In Proceedings of 2012 7th international conference on computer science & education, Vols I-Vi (pp. 687–689) IEEE.
• Fensham PJ, Bellocchi A. Higher order thinking in chemistry curriculum and its assessment. Thinking Skills and Creativity. 2013; 10 :250–264. doi: 10.1016/j.tsc.2013.06.003. [ CrossRef ] [ Google Scholar ]
• FitzPatrick Beverly, Schulz Henry. Do curriculum outcomes and assessment activities in science encourage higher order thinking. Canadian Journal of Science, Mathematics and Technology Education. 2015; 15 (2):136–154. doi: 10.1080/14926156.2015.1014074. [ CrossRef ] [ Google Scholar ]
• Garfield E. Historiographic mapping of knowledge domains literature. Journal of Information Science. 2004; 30 (2):119–145. doi: 10.1177/0165551504042802. [ CrossRef ] [ Google Scholar ]
• Garfield E. From the science of science to scientometrics visualizing the history of science with HistCite software. Journal of Informetrics. 2009; 3 (3):173–179. doi: 10.1016/j.joi.2009.03.009. [ CrossRef ] [ Google Scholar ]
• Garfield E, Pudovkin AI, Istomin VS. Why do we need algorithmic historiography? Journal of the American Society for Information Science & Technology. 2003; 54 (5):400–412. doi: 10.1002/asi.10226. [ CrossRef ] [ Google Scholar ]
• Hadzhikoleva S, Hadzhikolev E, Kasakliev N. Using peer assessment to enhance higher order thinking skills. Tem Journal-Technology Education Management Informatics. 2019; 8 (1):242–247. doi: 10.18421/tem81-34. [ CrossRef ] [ Google Scholar ]
• Hmelo CE, Ferrari M. The problem-based learning tutorial: Cultivating higher order thinking skills. Journal for the Education of the Gifted. 1997; 20 (4):401–422. doi: 10.1177/016235329702000405. [ CrossRef ] [ Google Scholar ]
• Hwang G-J, Lai C-L, Liang J-C, Chu H-C, Tsai C-C. A long-term experiment to investigate the relationships between high school students’ perceptions of mobile learning and peer interaction and higher-order thinking tendencies. Educational Technology Research and Development. 2017; 66 (1):75–93. doi: 10.1007/s11423-017-9540-3. [ CrossRef ] [ Google Scholar ]
• Hwang GJ, Yin CJ, Chu HC. The era of flipped learning: Promoting active learning and higher order thinking with innovative flipped learning strategies and supporting systems. Interactive Learning Environments. 2019; 27 (8):991–994. doi: 10.1080/10494820.2019.1667150. [ CrossRef ] [ Google Scholar ]
• Kurniawati T. Improving students’ higher order-thinking skills through problem-based learning in introduction to microeconomics course. In: Hermawan A, Si M, editors. International conference on economics, education, business and accounting. KnE Social Sciences; 2019. pp. 9–20. [ Google Scholar ]
• Lee J, Choi H. What affects learner’s higher-order thinking in technology-enhanced learning environments? The effects of learner factors. Computers & Education. 2017; 115 :143–152. doi: 10.1016/j.compedu.2017.06.015. [ CrossRef ] [ Google Scholar ]
• Lucio-Arias D, Leydesdorff L. Main-path analysis and path-dependent transitions in HistCite TM -based historiograms. Journal of the American Society for Information Science and Technology. 2008; 59 (12):1948–1962. doi: 10.1002/asi.20903. [ CrossRef ] [ Google Scholar ]
• Mahoney JW, Harris-Reeves B. The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education. 2019; 20 (1):25–37. doi: 10.1177/1469787417723243. [ CrossRef ] [ Google Scholar ]
• McLoughlin D., Mynard J. An analysis of higher order thinking in online discussions. Innovations in Education and Teaching International. 2009; 46 (2):147–160. doi: 10.1080/14703290902843778. [ CrossRef ] [ Google Scholar ]
• Miri B, David B-C, Uri Z. Purposely teaching for the promotion of higher-order thinking skills: A case of critical thinking. Research in ScienCe Education. 2007; 37 (4):353–369. doi: 10.1007/s11165-006-9029-2. [ CrossRef ] [ Google Scholar ]
• Mohamad FS, Morini L, Minoi J-L, Arnab S. Game-based learning to teach higher order thinking in rural schools: Case studies in Sarawak Borneo. Interaction Design and Architectures. 2019; 41 :78–86. doi: 10.55612/s-5002-041-006. [ CrossRef ] [ Google Scholar ]
• Nicely RF. Higher-order thinking skills in mathematics textbooks. Educational Leadership. 1985; 42 (7):26–30. [ Google Scholar ]
• Permatasari A, Wartono, & Kusairi, S. Identification of students difficulties in terms of the higher order thinking skills on the subject of work and energy. In: Suparmi A, Nugraha DA, editors. International conference on science and applied science. AIP Publishing LLC; 2018. [ Google Scholar ]
• Resnick LB. Education and learning to think. National Academy Press; 1987. [ Google Scholar ]
• Rubini B, Septian B, Permana I. Enhancing critical thinking through the science learning on using interactive problem based module. In: Abdullah AG, Nandiyanto ABD, Permana I, Agustin RR, Sutarno, Saprudin, editors. International conference on mathematics and science education. IOP Publishing; 2019. [ Google Scholar ]
• Saud MS, Kamin Y, Abd Latib A, Amin NF. A conceptual model of scenario based learning for developing higher order thinking skills in engineering education. Advanced Science Letters. 2017; 23 (1):194–196. doi: 10.1166/asl.2017.7284. [ CrossRef ] [ Google Scholar ]
• Singh RKA, Singh CKS, Tunku MTM, Mostafa NA, Singh TSM. A review of research on the use of higher order thinking skills to teach writing. International Journal of English Linguistics. 2018; 8 (1):86–93. doi: 10.5539/ijel.v8n1p86. [ CrossRef ] [ Google Scholar ]
• Tony T. Mathematics teachers’ interpretation of higher-order thinking in Bloom’s taxonomy. International Electronic Journal of Mathematics Education. 2008; 3 (2):96–109. doi: 10.29333/iejme/221. [ CrossRef ] [ Google Scholar ]
• Tsai SC. Integrating english for specific purposes courseware into task-based learning in a context of preparing for international trade fairs. Australasian Journal of Educational Technology. 2013; 29 (1):111–127. doi: 10.14742/ajet.49. [ CrossRef ] [ Google Scholar ]
• Wu D, Xing DX, Lu C. The effects of learner factors on higher-order thinking in the smart classroom environment. Journal of Computers in Education. 2019; 6 (4):483–498. doi: 10.1007/s40692-019-00146-4. [ CrossRef ] [ Google Scholar ]
• Yao, K. J. (2012). Using modern educational technology promote learners’ higher-order thinking skill. In Z. Zhang & T. B. Zhang (Eds.), 2012 Third International Conference on Education and Sports Education (Vol. 5, pp. 455–458).
• Zohar Anat. Challenges in wide scale implementation efforts to foster higher order thinking (HOT) in science education across a whole school system. Thinking Skills Creativity. 2013; 10 :233–249. doi: 10.1016/j.tsc.2013.06.002. [ CrossRef ] [ Google Scholar ]
• Zohar A, Dori YJ. Higher order thinking skills and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences. 2003; 12 (2):145–181. doi: 10.1207/s15327809jls1202_1. [ CrossRef ] [ Google Scholar ]
• Zohar A, Peled B. The effects of explicit teaching of metastrategic knowledge on low- and high-achieving students. Learning & Instruction. 2008; 18 (4):337–353. doi: 10.1016/j.learninstruc.2007.07.001. [ CrossRef ] [ Google Scholar ]

• Research article
• Open access
• Published: 08 January 2021

Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment

• Kaili Lu 1 ,
• Harrison H. Yang   ORCID: orcid.org/0000-0003-4836-835X 1 , 2 ,
• Yinghui Shi 1 &
• Xuan Wang 1  

International Journal of Educational Technology in Higher Education volume  18 , Article number:  1 ( 2021 ) Cite this article

18k Accesses

107 Citations

9 Altmetric

Metrics details

To understand the development of students’ higher-order thinking skills (HOTS) in the smart classroom environment, a structural equation modeling analysis was used to examine the relationships between key factors that influence students’ learning and their HOTS within a smart classroom environment. A sample of 217 first-year Chinese college students, who studied in a smart classroom environment for one semester, completed a survey that measures their smart classroom preferences, learning motivation, learning strategy, peer interaction, and HOTS. The results indicated that peer interaction and learning motivation had a direct impact on students’ HOTS. Furthermore, indirect effects were found between students’ learning strategy and HOTS through the mediator peer interaction, and between smart classroom preferences and HOTS through the following: learning motivation, the combination of learning strategy and peer interaction, and the combination of learning motivation, learning strategy and peer interaction. Based on these findings, this study recommends that instructors teaching in a smart learning environment should focus on improving peer interaction and learning motivation, as well as smart classroom preferences and learning strategy, to hone students’ HOTS.

Introduction

Smart classrooms have gained the attention of scholars and educators worldwide. In 2017, the EDUCAUSE Center for Analysis and Research identified technology-enabled learning environments as a strategic investment for colleges and universities (Brooks 2017 ; Lee et al. 2019 ). It also predicted that the smart classroom would become widely utilized by 2022.

The term ‘smart classroom’ refers to a physical classroom that integrates advanced forms of educational technology. Such an environment provides opportunities for student learning and participation in formal educational learning experiences that exceed what traditional classrooms can offer (Macleod et al. 2018 ). Li et al. ( 2015 ) identify four features of the smart classroom. First, the smart classroom is a technology-rich learning environment that combines physical and virtual spaces. Second, the smart classroom provides information and communication technology tools, learning resources, and interaction support for various teaching and learning activities, including personalized learning, group learning, inquiry-based learning, collaborative learning, mobile learning, and virtual learning. Third, the smart classroom is capable of storing, collecting, computing, and analyzing learners’ data in order to make optimized pedagogical decisions. Fourth, the smart classroom is an open environment that brings the learners to an authentic learning context. Previous studies have also indicated that the smart classroom environment can stimulate students’ learning motivation, promote active learning, and improve academic performance (Jena 2013 ; Liu et al. 2011 ). However, the impact of the smart classroom environment on students’ higher-order thinking skills (HOTS) is less clear.

The importance of HOTS has been emphasized by policymakers, educators, researchers, and the general public (Abosalem 2016 ; Elfeky 2019 ; Lu et al. in press). Upon conducting an analysis of previous studies, Hwang et al. ( 2017 ) identified three HOTS: problem-solving, critical thinking, and creativity. Problem-solving refers to the ability to identify a problem, collect and analyze relevant information, select and implement a relevant solution. Critical thinking refers to the ability to analyze information objectively, think clearly and rationally, and make a reasoned judgment. Creativity refers to being able to create new objects and develop innovative ideas and methods by elaborating upon, refining, analyzing, and evaluating existing ones.

Researchers argue that HOTS fall under the umbrella of twenty-first-century skills, which comprise the essential skills that youth need to prepare for the future (Ananiadou and Claro 2009 ; Collins 2014 ). Therefore, it is important for educators to identify and use learning environments that stimulate the development of students’ HOTS.

Purpose of this research

To date, few studies have been done from the students’ perspective on the relationships between the key factors influencing students’ learning and students’ HOTS, when they are instructed in a smart classroom context. Only one recent study by Wu et al. ( 2019 ) has attempted to investigate the effects of students’ learning motivation, learning style, and internet attitude on their HOTS in the smart classroom. Their study examined data from 784 students in primary schools and found that students’ learning style and internet attitude had a direct impact on the students’ HOTS, but learning motivation did not have a significant impact on their HOTS. Furthermore, the influence of other key learning factors on HOTS in the smart classroom has remained unclear. Particularly, very few studies have explored the relationship between the key factors influencing college students’ learning and their HOTS within the smart classroom environment. Understanding the factors that influence HOTS can help educators and curriculum designers develop more rigorous learning opportunities and assessment tools.

Thus, this study aims to fill the existing research gap by investigating the following research question:

What are the relationships of the key factors influencing student learning and college students’ HOTS in a smart classroom environment?

Research framework

Previous studies have explored the various factors that are associated with student achievements in terms of skills and knowledge in other learning environments. In general, the key factors that influence student learning include classroom preference (Moore 1989 ; Tsai 2008 ), learning motivation (Pintrich 1999 ), learning strategy (Garcia and Pintrich 1992 ), and peer interaction (Hwang et al. 2017 ; Osman et al. 2011 ).

Learning environment preferences

Learning environment preferences refer to students’ perceptions of a specific learning environment (Fraser 1998 ). Accordingly, students’ smart classroom preferences (SCP) are about their perceptions of the smart classroom (MacLeod et al. 2018 ). Students’ preferences toward a certain learning environment have increasingly drawn attention from educators; it is believed that if educators know about their students’ perspectives of their learning environment, they can make the necessary adjustments (Chuang and Tsai 2005 ).

Previous studies indicate that classroom preferences affect students’ learning in different educational environments. For example, Chang et al. ( 2010 ) found that students’ learning outcomes were in alignment with their environment preferences in a classroom setting where student-centered and teacher-centered instructional approaches coexisted. Furthermore, Hwang et al. ( 2017 ) found that student preferences toward the mobile learning environment were related to HOTS.

• Learning motivation

Learning motivation (LM) prompts individuals to take actions that will help them achieve a goal, or fulfill a need or expectation in the learning process (Gopalan et al. 2017 ). Although there is no consensus on the matter, a prominent study conducted by Pintrich et al. ( 1991 ), which identifies the three general motivational constructs: value, expectancy, and affect.

Previous studies demonstrated that students’ LM is a fundamental link between student performance and achievement in various learning environments. For example, Roberts and Dyer ( 2005 ) found that students’ learning motivation could be associated with their critical thinking, an aspect of HOTS, in an online learning environment. Similarly, Gong et al. ( 2020 ) stated that students’ learning motivation had a direct impact on their computational thinking skills, which includes creativity, algorithmic thinking, cooperation, critical thinking, and problem-solving in a flipped classroom setting. Conversely, Wu et al. ( 2019 ) reported that students’ learning motivation did not affect HOTS in the smart classroom environment.

• Learning strategy

Learning strategy (LS) refers to “a set of processes or steps that can facilitate the acquisition, storage, and/or utilization of information” (Dansereau 1985 ). A key study of LS by Pintrich et al. ( 1991 ) identifies cognitive, metacognitive, and resource management strategies as the main components of LS.

Researchers have determined that LS positively influences student skills and knowledge development in learning environments. Mayer ( 1998 ) argued that the cognitive and metacognitive components of LS had an important influence on successful problem-solving in traditional academic settings. This view has been supported by Gong et al. ( 2020 ) who reported that students’ learning strategy had a direct impact on certain HOTS such as creativity, critical thinking, and problem-solving in a flipped classroom environment. Moreover, Wilgis and Mcconnell ( 2008 ) and Soltis et al. ( 2015 ) found that specific learning strategies such as process-oriented, guided inquiry and concept mapping could significantly improve students’ HOTS in a blended learning environment.

• Peer interaction

Peer interaction (PI) is “a form of cooperative learning that enhances the value of student-to-student interaction and results in different advantages of learning outcomes” (Christudason n. d.). Hwang et al. ( 2017 ) have indicated that PI includes collaboration and communication skills. Specifically, collaboration is the ability of two or more people to work together and share their perspectives and ideas with respect to achieving learning goals or completing learning tasks. Communication refers to the ability to ‘‘articulate thoughts and ideas effectively by using oral, written and nonverbal communication skills in a variety of forms and contexts’’ (Frazier and Reynolds 2012 ).

Previous studies point out that PI is an important factor influencing students’ learning outcomes. For example, Tsai et al. ( 2011 ) found that students’ PI could be a predictor of HOTS in a constructivist context-aware ubiquitous learning environment. Hwang et al. ( 2017 ) also verified that collaboration and communication were positively related to HOTS in a mobile learning environment.

The relationships between learning environment preferences, LM, LS, and PI

Previous studies have explored the relationship between learning environment preferences, LM, LS, and PI in various educational contexts (Al-Khaldi and Al-Jabri 1998 ; Houle 1996 ; Hwang et al. 2017 ; Tsai 2008 ). For example, Houle ( 1996 ), as well as Al-Khaldi and Al-Jabri ( 1998 ), argued that students’ classroom preferences could affect students’ learning motivation in the technology-supported classroom. Moreover, Tsai ( 2008 ) found that students’ preferences toward the constructivist Internet-based learning environment were related to their learning strategies and outcomes.

In addition, existing research shows that both learning motivation and learning strategy have positive influences on peer interaction (King 1991 ; Yang and Chang 2011 ). For instance, Yang and Chang ( 2011 ) found that learning motivation was positively related to students’ peer interactions in an interactive blogging learning environment. Besides, Tsuei ( 2011 ) found that a peer-assisted learning strategy was positively related to peer interaction in a computer-supported collaborative learning environment. Furthermore, existing research has shown that motivation has a critical effect on strategy choices (Ellis 1994 ; Gong et al. 2020 ). For example, Gong et al. ( 2020 ) reported that there was a positive relationship between LM and LS in the flipped classroom instruction environment.

The relational model and hypotheses

Based on our review of related studies, we believe that students’ learning environment preferences, LM, LS, and PI can influence student achievements related to skills and knowledge in various learning environments (Ananiadou and Claro 2009 ; Pintrich et al. 1991 ; Tsai et al. 2011 ). Therefore, as shown in Fig.  1 , we assume that SCP, LM, LS, and PI may influence students’ HOTS when they are taught in a smart classroom. Our hypotheses are as follows:

Proposed research model and hypotheses

Hypothesis 1 (H1):

The level of SCP will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 2 (H2):

The level of LM will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 3 (H3):

The level of LS will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 4 (H4):

The level of PI will be positively related to the degree of college students’ HOTS within a smart classroom environment.

Hypothesis 5 (H5):

The level of SCP will be positively related to the degree of college students’ LM within a smart classroom environment.

Hypothesis 6 (H6):

The level of SCP will be positively related to the degree of college students’ LS within a smart classroom environment.

Hypothesis 7 (H7):

The level of LM will be positively related to the degree of college students’ LS within a smart classroom environment.

Hypothesis 8 (H8):

The level of SCP will be positively related to the degree of college students’ PI within a smart classroom environment.

Hypothesis 9 (H9):

The level of LM will be positively related to the degree of college students’ PI within a smart classroom environment.

Hypothesis 10 (H10):

The level of LS will be positively related to the degree of college students’ PI within a smart classroom environment.

Participants

To investigate the research question, this study used a total number of 217 students enrolled in the Ideological and Moral Cultivation and Legal Basis (IMCLB) course at a university in central China. Both the course and the university were purposely selected for two reasons. Firstly, the course is a compulsory general course for all first-year students at the university. As such, the number of students taking the course allowed us to collect a sufficient number of participants from different disciplines. Secondly, the university attaches great importance to information technology and has built several smart classrooms. All university instructors are provided with training opportunities to learn how to use smart classroom technologies, and were encouraged to conduct their instructional practices in the smart classroom. Particularly, most instructors of the IMCLB course at this university have taught its content in the smart classroom for 3 years.

At this university, the IMCLM course is a semester (12 weeks) long. Instructors and students meet one or two times per week. All classes refer to the same learning materials and facilities in the smart classrooms. Participating students were organized into groups for learning activities. Each group had 4–5 students, organized in a cluster seating arrangement, allowing them to interact with one another and easily work together.

Instruments

The survey adopted elements of the Collaboration, Communication, Critical Thinking, Problem-solving and Creativity Awareness questionnaire (4C1PA), the Preference Instrument of Smart Classroom Learning Environments (PI-SCLE), and the Motivated Strategies for Learning Questionnaire (MSLQ) to measure students’ HOTS, PI, SCP, LM, and LS.

The 4C1PA was developed by Hwang et al. ( 2017 ), and consists of five dimensions capturing students’ HOTS and their tendency to engage in PI. The HOTS tendency subscale (alpha = 0.888) consists of a three-dimensional construct: problem-solving, critical thinking, and creativity. Each dimension has three items. One representative item of this scale is: “I like to observe something I haven’t seen before and understand it in detail.” The Peer Interaction subscale (alpha = 0.858) consists of a two-dimensional construct: collaboration and communication. Each dimension has three items. One representative item of this scale is: “I try to provide useful and sufficient information when I conduct collaborative learning.” All items of 4C1PA were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).

The PI-SCLE was developed by MacLeod et al. ( 2018 ), and includes eight distinct dimensions: student negotiation, inquiry-based learning, reflective thinking, functional design, connectedness, ease of use, perceived usefulness, and multiple sources (alpha = 0.951). Each dimension has three items. All items were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). ‘‘In the smart classroom learning environment, I prefer that I can get the chance to talk to other students’’ is one representative item of the student negotiation dimension.

The MSLQ was developed by Pintrich et al. ( 1991 ), and includes LM (alpha = 0.833) and LS (alpha = 0.863). LM has three dimensions: value component (8 items), expectancy component (5 items), and affective component (3 items). LS has two dimensions: cognitive and metacognitive strategy (14 items) and resource management strategy (11 items). All items were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). One representative item of LM is “In a class like this, I prefer course material that really challenges me so I can learn new things.”

Data collection and analysis

Data were collected at the end of the semester. Before the survey was administered, permission was granted by the university to conduct the research. All 217 participants were introduced to the purpose of the research by a researcher of this study during their instructor’s absence. Participants were informed that their information would only be used for educational research and that their survey results would not affect their grades in the course. All responses were both anonymous and given voluntarily. The survey was issued during a mid-class break, then imported into SPSS 22.0 and SmartPLS 3.2.8 for data analysis. A structural equation modeling analysis was conducted to analyze the relationships between the key influencing factors and students’ HOTS.

Results and discussion

The partial least square (PLS) method was used to verify the proposed research model. PLS was appropriate for the sample size of this study (Chin 1998 ; Gefen et al. 2000 ) and well-suited for testing theories in the early stages of development (Fornell and Bookstein 1982 ). Hair et al. ( 2014 ) introduced the Standardized Root Mean Square Residual (SRMR) as a goodness of fit measure for PLS-SEM that can be used to avoid model misspecification. In general, a value less than 0.08 is considered a good fit (Hu and Bentler 1998 ). The value of SRMR of the model in this study was 0.06, thus, the goodness of fit for the proposed model was verified as acceptable.

Confirming the measurement model

The measurement model was assessed by the reliability of measures, convergent validity, and discriminant validity. As shown in Table 1 , the average variance extracted (AVE) values for all factors were over 0.6, which suggested adequate convergent validity (Fornell and Larcker 1981 ). The reliability of the measurement model was examined using the composite reliability and Cronbach’s alpha.

Findings indicated that the composite reliability (CR) coefficients were over 0.8, which demonstrated satisfactory reliability (Nunnally and Bernstein 1994 ). Cronbach’s alpha was over 0.8 and within acceptable limits (Helmstadter 1964 ). Furthermore, to evaluate the discriminant validity, the square roots of AVE were compared to correlations among latent variables (Fornell and Larcker 1981 ), in which all latent correlations were less than the corresponding AVE square roots. Table 1 shows the results of the measurement model. In sum, the adequacy of the measurement model indicates that all the items were reliable indicators of the hypotheses they were purposed to measure.

Structural equation modeling analysis

A structural model was used to test the hypotheses using path coefficients (β value), R 2 value, and t-value bootstrapping (500 resamples) (Cohen 1988 ). The PLS path modeling estimation for this study is shown in Fig.  2 . Path coefficients along with the associated t-values are provided and the variance given is explained.

The structural model for HOTS

Findings reflect that H2, H4, H5, H6, H7 and H10 are supported, while H1, H3, H8, and H9 are rejected. LM (β = 0.244, p < 0.001) and PI (β = 0.665, p < 0.001) were positively related to HOTS, collectively accounting for 58.3% of R 2 . In addition, SCP (β = 0.286, p < 0.001) and LM (β = 0.505, p < 0.001) positively impacted LS, accounting for 48.4% of R 2 . SCP (β = 0.510, p < 0.001) had a significant positive effect on LM as well, accounting for 26% of R 2 . Furthermore, LS (β = 0.298, p < 0.01) had a significantly positive effect on peer interaction, accounting for 16.9% of R 2 .

Analysis of indirect and total effects among key factors

Further, the direct and indirect effects of factors in each hypothesis were examined (Ullman and Bentler 2003 ). As shown in Fig.  2 and Table 2 , SCP had both direct and indirect influences on LS, and LM mediated the indirect influence. Furthermore, although SCP had no direct influence on HOTS, there were three indirect paths leading from SCP towards HOTS, where LM, LS, and PI acted as partial mediators. In addition, LM had both direct and indirect influence on HOTS. An indirect effect is reflected in the path from LM to HOTS through LS and PI, which suggests that the combination of LS and PI can also mediate the relationships between LM and HOTS.

Discussion of results

This study revealed that PI and LM were directly related to students’ HOTS in the smart classroom environment. This result may be explained by the fact that the smart classroom is a student-centered learning environment. Unlike the traditional teacher-centered classroom, a student-centered classroom is a place where the students are actively involved in the learning process (Utecht 2003 ). In a student-centered class, students no longer only rely on their instructor to give them instructions. Instead, students actively communicate, collaborate, and learn from each other, as well as apply and improve their critical thinking, problem-solving, and creativity skills (Jones 2007 ). It can thus be confirmed that PI and LM are two primary factors to students’ HOTS in a smart classroom. This finding suggests that instructors should endeavor to enhance students’ PI and LM, in order to promote students’ HOTS in the smart classroom environment. For instance, instructors should provide the opportunity for students to engage in self-directed learning, explore high-interest topics and ideas, work collaboratively on projects, and share in decision-making during the learning process (Jones 2007 ; Yang 2001 ; Yang et al. 2000 ).

It is interesting to note that although students’ SCP and LS had no direct effect on HOTS, both did have a significant and positive indirect effect on HOTS. First, the finding that SCP had no direct effect on HOTS was consistent with the previous study (Hwang et al. 2017 ), which found that students’ preferences toward the mobile learning environment had an indirect effect on HOTS via students’ interaction. This finding may be explained by the fact that HOTS were used to describe students’ learning outcomes, which are directly related to cognitive activities. Additionally, learning environment preferences were used to describe students’ perceptions of their learning environment. Second, different from a previous study (Gong et al. 2020 ), this study found that LS had no direct effect on HOTS. This result could be explained by the fact that this study was conducted in the smart classroom environment, which is different from previously used learning environments. In the smart classroom, students were more engaged in the self-directed learning activities. Students’ LS may directly reflect on PI, thus, LS had indirect effect on HOTS via PI.

This study found that SCP positively impacted LM and LS, and LS had a significantly positive effect on PI. Furthermore, the association between SCP and HOTS was mediated by learning motivation, the combination of learning motivation, learning strategy and peer interaction, and also by the combination of learning strategy and peer interaction. Meanwhile, LS had a significant and positive indirect effect on HOTS via the mediating factor peer interaction. This finding suggests that, in order to develop students’ HOTS in the smart classroom environment, instructors and instructional designers should also take SCP and LS into account. For instance, to best meet students’ learning needs, interests, strategies, and abilities, instructors and instructional designers should better incorporate the constructs of the smart classroom environment and technology into the learning process. These constructs include student negotiation, inquiry learning, reflective thinking, ease of use, perceived usefulness, multiple sources, connectedness, and functional design (MacLeod et al. 2018 ).

Conclusions and future research

Given the importance of HOTS and the prevalence of smart classrooms in higher education, it is critical to understand the relationships between students’ HOTS and the key influencing factors, when learning in a smart classroom environment. This study proposed a research model and used a survey to collect data from 217 college students who had learning experience within a smart classroom environment. A structural equation modeling analysis method was used to explore the relationships between the four key factors influencing student learning (SCP, LM, LS, and PI) and students’ HOTS. The results of this study expand our knowledge of these key factors affecting students’ problem-solving, critical thinking, and creativity skills. These results can be used to inform educational processes and pedagogy, which will improve students’ HOTS in the smart classroom environment. The most significant findings of this study indicate that students’ PI and LM directly impact students’ HOTS in the smart classroom. In contrast, SCP and LS do not directly impact HOTS. This study supports the work of other studies, suggesting that peer interaction and learning motivation positively affects students’ ability to learn knowledge and skills in their learning environments (Gong et al. 2020 ; Hwang et al. 2017 ; Roberts and Dyer 2005 ; Tsai et al. 2011 ). In summary, the results of this study indicate that, in order to develop students’ HOTS, instructors should consider students’ learning motivation, peer interaction, learning strategy, and preferences toward the smart classroom when analyzing, designing, developing, implementing, and evaluating learning activities in a smart classroom environment.

While the present research has important implications, it still has limitations. It should be noted that we have only examined four important factors that influence student learning via a structural equation modeling analysis method. Moreover, the context was limited to one subject area conducted in the smart classroom environment. Future research is encouraged to involve more subject areas and more related factors, such as students’ learning styles and approaches to studying, and teaching methods and strategies. Particularly, future studies should extend to different subject areas with other related factors and employ a mixed-methods approach, like adding follow-up interviews or qualitative answers to capture the opinion of the students, to support the triangulation of quantitative results.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

• Higher-order thinking skills
• Smart classroom preferences

Partial least square

Composite reliability

Average variance extracted

Abosalem, Y. (2016). Assessment techniques and students’ higher-order thinking skills. International Journal of Secondary Education, 4 (1), 1–11. https://doi.org/10.11648/j.ijsedu.20160401.11 .

Article   Google Scholar  

Al-Khaldi, M. A., & Al-Jabri, I. M. (1998). The relationship of attitudes to computer utilization: New evidence from a developing nation. Computers in Human Behavior, 14 (1), 23–42. https://doi.org/10.1016/s0747-5632(97)00030-7 .

Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries. OECD Education Working Papers . https://doi.org/10.1787/218525261154 .

Brooks, D. C. (2017, March). Active learning classrooms: The top strategic technology for 2017. EDUCAUSE Center for Analysis and Research report. Retrieved May 7, 2020, from https://er.educause.edu/blogs/2017/3/active-learning-classrooms-the-top-strategic-technology-for-2017 .

Chang, C., Hsiao, C., & Chang, Y. (2010). Science learning outcomes in alignment with learning environment preferences. Journal of Science Education and Technology, 20 (2), 136–145. https://doi.org/10.1007/s10956-010-9240-9 .

Chin, W. W. (1998). The partial least squares approach to structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). New Jersey: Lawrence Erlbaum Associates.

Google Scholar  

Christudason, A. (n. d.). What is peer interaction/learning. Retrieved June 11, 2020, from https://www.igi-global.com/dictionary/peer-interaction-learning/22184 .

Chuang, S., & Tsai, C. (2005). Preferences toward the constructivist Internet-based learning environments among high school students in Taiwan. Computers in Human Behavior, 21 (2), 255–272. https://doi.org/10.1016/j.chb.2004.02.015 .

Cohen, J. (1988). Statistical power analysis for the behavioural sciences . Hillsdale, New Jersey: Erlbaum.

MATH   Google Scholar  

Collins, R. (2014). Skills for the 21st century: Teaching higher-order thinking. Curriculum & Leadership Journal, 12 (14). Retrieved July 6, 2020, from http://www.curriculum.edu.au/leader/teaching_higher_order_thinking,37431.html .

Dansereau, D. F. (1985). Learning strategy research. Thinking and Learning Skills, 1, 209–239.

Elfeky, A. I. M. (2019). The effect of personal learning environments on participants’ higher order thinking skills and satisfaction. Innovations in Education and Teaching International, 56 (4), 505–516. https://doi.org/10.1080/14703297.2018.1534601 .

Ellis, R. (1994). The study of second language acquisition . Oxford, UK: Oxford University Press.

Fornell, C., & Bookstein, F. L. (1982). Two structural equation models: LISREL and PLS applied to consumer exit-voice theory. Journal of Marketing Research, 19 (4), 440–452. https://doi.org/10.2307/3151718 .

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18 (1), 39–50. https://doi.org/10.2307/3151312 .

Fraser, B. J. (1998). Science learning environments: Assessment, effects and determinants. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (pp. 527–561). Dordrecht: Kluwer.

Chapter   Google Scholar  

Frazier, K., & Reynolds, E. (2012). Power up your creative mind . Illinois: Pieces of Learning.

Garcia, T., & Pintrich, P. R. (1992). Critical thinking and its relationship to motivation, learning strategies and classroom experience. In The 100th Annual Meeting of the American Psychological Association. Office of Educational Research and Improvement.

Gefen, D., Straub, D. W., & Boudreau, M. C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the Association for Information Systems, 4 (1), 7.

Gong, D., Yang, H. H., & Cai, J. (2020). Exploring the key influencing factors on college students’ computational thinking skills through flipped-classroom instruction. International Journal of Educational Technology in Higher Education . https://doi.org/10.1186/s41239-020-00196-0 .

Gopalan, V., Bakar, J. A. A., Zulkifli, A. N., Alwi, A., & Mat, R. C. (2017, October). A review of the motivation theories in learning. AIP Conference Proceedings, 1891 (1), 020043. https://doi.org/10.1063/1.5005376 .

Hair, J. F., Henseler, J., Dijkstra, T. K., & Sarstedt, M. (2014). Common beliefs and reality about partial least squares: Comments on Rönkkö and Evermann.  Faculty Publications. 3666 . https://digitalcommons.kennesaw.edu/facpubs/3666 .

Helmstadter, G. C. (1964). Principles of psychological measurement . New York: Appleton-Century-Crofts.

Houle, P. A. (1996). Toward understanding student differences in a computer skills course. Journal of Educational Computing Research, 14 (1), 25–48. https://doi.org/10.2190/c06x-g9uq-6bub-ypty .

Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological methods, 3 (4), 424–453.

Hwang, G., Lai, C., Liang, J., Chu, H., & Tsai, C. (2017). A long-term experiment to investigate the relationships between high school students’ perceptions of mobile learning and peer interaction and higher-order thinking tendencies. Educational Technology Research and Development, 66 (1), 75–93. https://doi.org/10.1007/s11423-017-9540-3 .

Jena, P. C. (2013). Effect of smart classroom learning environment on academic achievement of rural high achievers and low achievers in science. International Letters of Social and Humanistic Sciences, 3 , 1–9. https://doi.org/10.18052/www.scipress.com/ilshs.3.1 .

Jones, L. (2007). The student-centered classroom . Cambridge: Cambridge University Press.

King, A. (1991). A strategy for enhancing peer interaction and learning during teacher training sessions. Teacher Education Quarterly, 18 (1), 15–28.

Lee, Y., Yang, H. H., Macleod, J., & Dai, J. (2019). Developing the rotational synchronous teaching (RST) model: Examination of the connected classroom climate. Australasian Journal of Educational Technology . https://doi.org/10.14742/ajet.4010 .

Li, B., Kong, S. C., & Chen, G. (2015). Development and validation of the smart classroom inventory. Smart Learning Environments . https://doi.org/10.1186/s40561-015-0012-0 .

Liu, M., Horton, L., Olmanson, J., & Toprac, P. (2011). A study of learning and motivation in a new media enriched environment for middle school science. Educational Technology Research and Development, 59 (2), 249–265. https://doi.org/10.1007/s11423-011-9192-7 .

Lu, K., Yang, H. H., & Xue, H. (in press). Investigating the four-level inquiry continuum on college students’ higher order thinking and peer interaction tendencies. International Journal of Innovation and Learning .

Macleod, J., Yang, H. H., Zhu, S., & Li, Y. (2018). Understanding students’ preferences toward the smart classroom learning environment: Development and validation of an instrument. Computers & Education, 122, 80–91. https://doi.org/10.1016/j.compedu.2018.03.015 .

Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26, 49–63. https://doi.org/10.1023/A:1003088013286 .

Moore, W. S. (1989). The learning environment preferences: Exploring the construct validity of an objective measure of the Perry scheme of intellectual development. Journal of College Student Development, 30 (6), 504–514.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory . New York: McGraw-Hill.

Osman, G., Duffy, T. M., Chang, J., & Lee, J. (2011). Learning through collaboration: Student perspectives. Asia Pacific Education Review, 12 (4), 547–558. https://doi.org/10.1007/s12564-011-9156-y .

Pintrich, P. R. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31 (6), 459–470. https://doi.org/10.1016/s0883-0355(99)00015-4 .

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the motivated strategies for learning questionnaire (MSLQ) . Ann Arbor, MI: The University of Michigan.

Roberts, T. G., & Dyer, J. E. (2005). The relationship of self-efficacy, motivation, and critical thinking disposition to achievement and attitudes when an illustrated web lecture is used in an online learning environment. Journal of Agricultural Education, 46 (2), 12–23. https://doi.org/10.5032/jae.2005.02012 .

Soltis, R., Verlinden, N., Kruger, N., Carroll, A., & Trumbo, T. (2015). Process-oriented guided inquiry learning strategy enhances students’ higher level thinking skills in a pharmaceutical sciences course. American Journal of Pharmaceutical Education . https://doi.org/10.5688/ajpe79111 .

Tsai, C. (2008). The preferences toward constructivist Internet-based learning environments among university students in Taiwan. Computers in Human Behavior, 24 (1), 16–31. https://doi.org/10.1016/j.chb.2006.12.002 .

Article   MathSciNet   Google Scholar  

Tsai, P., Tsai, C., & Hwang, G. (2011). Developing a survey for assessing preferences in constructivist context-aware ubiquitous learning environments. Journal of Computer Assisted Learning, 28 (3), 250–264. https://doi.org/10.1111/j.1365-2729.2011.00436.x .

Tsuei, M. (2011). Development of a peer-assisted learning strategy in computer-supported collaborative learning environments for elementary school students. British Journal of Educational Technology, 42 (2), 214–232. https://doi.org/10.1111/j.1467-8535.2009.01006.x .

Ullman, J. B., & Bentler, P. M. (2003). Structural equation modeling. Handbook of psychology , 607–634. Retrieved March 22, 2020, from https://onlinelibrary.wiley.com/doi/pdf/10.1002/9781118133880.hop202023 .

Utecht, J. R. (2003). Problem-based learning in the student centered classroom. Digital Media. Retrieved May 13, 2020, from http://jeffutecht.com/docs/PBL.pdf .

Wilgis, M., & Mcconnell, J. (2008). Concept mapping: An educational strategy to improve graduate nurses’ critical thinking skills during a hospital orientation program. The Journal of Continuing Education in Nursing, 39 (3), 119–126. https://doi.org/10.3928/00220124-20080301-12 .

Wu, D., Xing, D., & Lu, C. (2019). The effects of learner factors on higher-order thinking in the smart classroom environment. Journal of Computers in Education, 6 (4), 483–498. https://doi.org/10.1007/s40692-019-00146-4 .

Yang, C., & Chang, Y. (2011). Assessing the effects of interactive blogging on student attitudes towards peer interaction, learning motivation, and academic achievements. Journal of Computer Assisted Learning, 28 (2), 126–135. https://doi.org/10.1111/j.1365-2729.2011.00423.x .

Yang, H. H. (2001). Mission possible: Project-based learning preparing graduate students for technology. In Society for Information Technology & Teacher Education International Conference (pp. 2855–2857). Association for the Advancement of Computing in Education (AACE).

Yang, H., Shindler, J., & Keen, A. (2000). Minds on, hands on: The linear-nonlinear problem-solving approach to a multimedia and Internet course. In Society for Information Technology & Teacher Education International Conference (pp. 738–743). Association for the Advancement of Computing in Education (AACE).

Download references

Acknowledgements

The authors would like to thank Dr. Ping Mei, Dr. Hui Xue, Dr. Bingguo Xu, and all the participating students for their support in this study.

This work is partially supported by the National Natural Science Foundation of China (Grant No. 61907019).

Author information

Authors and affiliations.

National Engineering Research Center for E-Learning, Central China Normal University, Wuhan, China

Kaili Lu, Harrison H. Yang, Yinghui Shi & Xuan Wang

School of Education, State University of New York at Oswego, 232 Wilber Hall, Oswego, NY, 13126, USA

Harrison H. Yang

You can also search for this author in PubMed   Google Scholar

Contributions

KL undertook the research, collected the data, and prepared the initial manuscript. HY provided the intellectual input, revised and finalized the manuscript. YS participated in the preparation of the manuscript. XW analyzed the data. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Harrison H. Yang .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Lu, K., Yang, H.H., Shi, Y. et al. Examining the key influencing factors on college students’ higher-order thinking skills in the smart classroom environment. Int J Educ Technol High Educ 18 , 1 (2021). https://doi.org/10.1186/s41239-020-00238-7

Download citation

Received : 15 July 2020

Accepted : 26 November 2020

Published : 08 January 2021

DOI : https://doi.org/10.1186/s41239-020-00238-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Smart classroom

Enhancing Students’ Higher Order Thinking Skills with Problem-Based Learning in a Regression Analysis Course

• Conference paper
• First Online: 26 August 2022
• Cite this conference paper

• Minh-Trang Vo Nguyen   ORCID: orcid.org/0000-0002-6940-3209 11 &
• Jane Lu Hsu   ORCID: orcid.org/0000-0002-0685-5814 12  

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13449))

Included in the following conference series:

• International Conference on Innovative Technologies and Learning

1298 Accesses

The concept of Higher-Order Thinking Skills (HOTS) is one of the highlighted aspects in producing human capital of high quality. However, how to enhance students’ HOTS is a challenge. Meanwhile, abilities in data analyzing have become an advantage qualification to employers. Therefore, the main focus in this study is to investigate the potential role of using problem-based learning in a regression analysis course to increase students’ level of HOTS. Data collected from this study is through an assessment giving to students before (pre) and after (post) them taking the regression analysis course. Results from this study conclude that students’ HOTS including looking for data from open sources, using a statistical software, analyzing data, and offering solutions could be enhanced from using problem-based learning. Moreover, there is a statistically difference between pre and post in students’ data analysis level, confidence in applying statistics in practice and the meaningfulness of statistics in life.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Avella, J.T., Kebritchi, M., Nunn, S.G., Kanai, T.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learn. 20 (2), 13–29 (2016)

Google Scholar  

Sin, K., Muthu, L.: Application of big data in education data mining and learning analytics–a literature review. ICTACT J. Soft Comput. 5 (4) (2015)

Alismaiel, O.A.: Adaptation of Big Data: an empirical investigation for sustainability of education. Entrepreneurship Sustain. Issues 9 (1), 590 (2021)

Article   Google Scholar  

Anderson, L.W., et al.: A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Outcomes: Complete edition. Longman, New York (2001)

Aisyah, A., Salehuddin, K., Aman, I., Yasin, R.M., Mimiko, N.: Eliciting elements of higher order thinking skills in the higher secondary examination question structure in Japan and Malaysia. In: Mohamad Noor, M.Y., Ahmad, B.E., Ismail, M.R., Hashim, H., Abdullah Baharum, M.A. (eds.) Proceedings of the Regional Conference on Science, Technology and Social Sciences (RCSTSS 2016), pp. 455–464. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-0203-9_42

Chapter   Google Scholar  

Al-Balushi, S.M., Al-Aamri, S.S.: The effect of environmental science projects on students’ environmental knowledge and science attitudes. Int. Res. Geogr. Environ. Educ. 23 (3), 213–227 (2014)

Harun, Y.: Project-Based Learning Handbook Educating the Millennial Learner. Educational Technology Division Ministry of Education, Kuala Lumpur (2006)

Download references

This study is supported by Ministry of Education in Taiwan (#PBM1101163).

Author information

Authors and affiliations.

Department of Applied Economics, National Chung Hsing University, Taichung, 402202, Taiwan

Minh-Trang Vo Nguyen

Department of Marketing, National Chung Hsing University, Taichung, 402202, Taiwan

Jane Lu Hsu

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Minh-Trang Vo Nguyen or Jane Lu Hsu .

Editor information

Editors and affiliations.

National Cheng Kung University, Tainan City, Taiwan

Yueh-Min Huang

Southern Taiwan University of Science and Technology, Tainan City, Taiwan

Shu-Chen Cheng

University of Trás-os-Montes and Alto Douro, Vila Real, Portugal

João Barroso

OsloMet – Oslo Metropolitan University, Oslo, Norway

Frode Eika Sandnes

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Nguyen, MT.V., Hsu, J.L. (2022). Enhancing Students’ Higher Order Thinking Skills with Problem-Based Learning in a Regression Analysis Course. In: Huang, YM., Cheng, SC., Barroso, J., Sandnes, F.E. (eds) Innovative Technologies and Learning. ICITL 2022. Lecture Notes in Computer Science, vol 13449. Springer, Cham. https://doi.org/10.1007/978-3-031-15273-3_34

Download citation

DOI : https://doi.org/10.1007/978-3-031-15273-3_34

Published : 26 August 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-15272-6

Online ISBN : 978-3-031-15273-3

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Higher Order Thinking: Bloom’s Taxonomy

Many students start college using the study strategies they used in high school, which is understandable—the strategies worked in the past, so why wouldn’t they work now? As you may have already figured out, college is different. Classes may be more rigorous (yet may seem less structured), your reading load may be heavier, and your professors may be less accessible. For these reasons and others, you’ll likely find that your old study habits aren’t as effective as they used to be. Part of the reason for this is that you may not be approaching the material in the same way as your professors. In this handout, we provide information on Bloom’s Taxonomy—a way of thinking about your schoolwork that can change the way you study and learn to better align with how your professors think (and how they grade).

Why higher order thinking leads to effective study

Most students report that high school was largely about remembering and understanding large amounts of content and then demonstrating this comprehension periodically on tests and exams. Bloom’s Taxonomy is a framework that starts with these two levels of thinking as important bases for pushing our brains to five other higher order levels of thinking—helping us move beyond remembering and recalling information and move deeper into application, analysis, synthesis, evaluation, and creation—the levels of thinking that your professors have in mind when they are designing exams and paper assignments. Because it is in these higher levels of thinking that our brains truly and deeply learn information, it’s important that you integrate higher order thinking into your study habits.

The following categories can help you assess your comprehension of readings, lecture notes, and other course materials. By creating and answering questions from a variety of categories, you can better anticipate and prepare for all types of exam questions. As you learn and study, start by asking yourself questions and using study methods from the level of remembering. Then, move progressively through the levels to push your understanding deeper—making your studying more meaningful and improving your long-term retention.

Level 1: Remember

This level helps us recall foundational or factual information: names, dates, formulas, definitions, components, or methods.

Level 2: Understand

Understanding means that we can explain main ideas and concepts and make meaning by interpreting, classifying, summarizing, inferring, comparing, and explaining.

Level 3: Apply

Application allows us to recognize or use concepts in real-world situations and to address when, where, or how to employ methods and ideas.

Level 4: Analyze

Analysis means breaking a topic or idea into components or examining a subject from different perspectives. It helps us see how the “whole” is created from the “parts.” It’s easy to miss the big picture by getting stuck at a lower level of thinking and simply remembering individual facts without seeing how they are connected. Analysis helps reveal the connections between facts.

Level 5: Synthesize

Synthesizing means considering individual elements together for the purpose of drawing conclusions, identifying themes, or determining common elements. Here you want to shift from “parts” to “whole.”

Level 6: Evaluate

Evaluating means making judgments about something based on criteria and standards. This requires checking and critiquing an argument or concept to form an opinion about its value. Often there is not a clear or correct answer to this type of question. Rather, it’s about making a judgment and supporting it with reasons and evidence.

Level 7: Create

Creating involves putting elements together to form a coherent or functional whole. Creating includes reorganizing elements into a new pattern or structure through planning. This is the highest and most advanced level of Bloom’s Taxonomy.

Pairing Bloom’s Taxonomy with other effective study strategies

While higher order thinking is an excellent way to approach learning new information and studying, you should pair it with other effective study strategies. Check out some of these links to read up on other tools and strategies you can try:

• Study Smarter, Not Harder
• Simple Study Template
• Using Concept Maps
• Group Study
• Evidence-Based Study Strategies Video
• Memory Tips Video
• All of our resources

Other UNC resources

If you’d like some individual assistance using higher order questions (or with anything regarding your academic success), check out some of your UNC resources:

• Academic Coaching: Make an appointment with an academic coach at the Learning Center to discuss your study habits one-on-one.
• Office Hours : Make an appointment with your professor or TA to discuss course material and how to be successful in the class.

Works consulted

Anderson, L. W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Wittrock, M.C (2001). A taxonomy of learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman.

“Bloom’s Taxonomy.” University of Waterloo. Retrieved from https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/planning-courses-and-assignments/course-design/blooms-taxonomy

“Bloom’s Taxonomy.” Retrieved from http://www.bloomstaxonomy.org/Blooms%20Taxonomy%20questions.pdf

Overbaugh, R., and Schultz, L. (n.d.). “Image of two versions of Bloom’s Taxonomy.” Norfolk, VA: Old Dominion University. Retrieved from https://www.odu.edu/content/dam/odu/col-dept/teaching-learning/docs/blooms-taxonomy-handout.pdf

If you enjoy using our handouts, we appreciate contributions of acknowledgement.

Make a Gift