the current emphasis on the development of critical thinking

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Conversations on critical thinking: can critical thinking find its way forward as the skill set and mindset of the century.

1. Introduction

Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness. It entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-issue; assumptions; concepts; empirical grounding; reasoning leading to conclusions; implications and consequences; objections from alternative viewpoints; and frame of reference…the development of critical thinking skills and dispositions is a life-long endeavor. The development of critical thinking is included in most conversations related to the development of so-called “21st century skills”. This manuscript is the initial investigation of the discourses around the notion of critical thinking as reflected in four diverse global contexts. It seeks to investigate the current perceptions of critical thinking in the relevant education policies of these different cultural environments if they exist, and, if possible the degree to which critical thinking is articulated. Further study intends to research how, if at all, the rhetoric of critical thinking is actually realized in educational practices and to what degree the development of critical thinking skills can contribute to informed global citizenship. This writing presents the initial investigation of the policyscape in each of the four contexts included in the study. These are captured in the vignettes developed by the participants with academic educational experiences in dissimilar backgrounds. The vignettes are followed by a discussion of the major theories defined and developed by scholars of critical thinking in western tradition and a suggested framework for the possible identification of, and the potential success of teaching and learning around the cognitive capacities of critical thinking in the formal, educational contexts of the countries of those represented by the collaborative team associated with this research project.

2. Research Context

3. purpose of the research, 4. historical perspectives of critical thinking.

Reinforcement fosters the repetition of what gets reinforced, regardless of the acting subject’s understanding of the problem that was posed, and of the inherent logic that distinguishes solutions from inadequate responses (p. 17).

…certain actions are reinforced as a result of their outcomes, so learning follows action. And what is learnt is action: the cognitive element is small [ 28 ].

5. Current Perspectives of Critical Thinking

6. methodology.

• What contextual influences may impact on educational policy statements related to developing students’ critical thinking skills?
• What, if any, do the statements in educational policy, ministerial declarations, curriculum and syllabi indicate about perceptions of the nature and importance of students in schools developing the cognitive capacities of critical thinking?

7. Critical Thinking in the Pakistani Educational Context

7.1. history and tradition contexts, 7.2. perceptions and importance of developing critical thinking skills.

There are some cultural barriers, mentioned by the participants, which discourage critical thinking, especially for women. The common social norm expects females to be quiet and this silent trait makes them more appealing to the proposition of marriage. In addition, people feel unease when questions about religion are being asked, not only for Islamiat, but also for Christianity, as this challenges their beliefs.

Promote higher order thinking skills that develop the capacity for self-directed learning, a spirit of inquiry, critical thinking, reasoning and teamwork [ 51 ] (p. 31).

8. Critical Thinking in the Educational Context of Australia

• goals and purpose;
• questions that lead to the proposition or proposal;
• information, data and experience gleaned;
• inferences and conclusions made;
• concepts and ideas evoked;
• assumptions;
• implications and consequences;
• viewpoints and perspectives.
• Inquiring, identifying, exploring and organising information and ideas : pose questions; identify and clarify information and ideas; organise and process information;
• Generating ideas, possibilities and actions: imagine possibilities and connect ideas; consider alternatives; seek solutions and put ideas into action;
• Reflecting on thinking and processes: think about thinking (metacognition); reflect on processes; transfer knowledge into new contexts;
• Analysing, synthesising and evaluating reasoning and procedures : apply logic and reasoning; draw conclusions and design a course of action; evaluate procedures and outcomes.

9. Critical Thinking in the Educational Context of Vietnam

Educational documentation.

• Logical dimension: thinking is an inference process;
• Psychological dimension: thinking is a psychological process;
• Semiotic dimension: thinking is a process of expressing stored thoughts via language;
• Sociopolitical dimension: thinking is under the influences of the contextual factors;
• Methodological dimension: thinking process employs different strategies and principles to be operated;
• Educational dimension: thinking is a process to develop universal intellectual traits (intellectual humility, intellectual autonomy, intellectual integrity, intellectual courage, intellectual perseverance, confidence in reason, intellectual empathy, and fair-mindedness).

10. Critical Thinking in the Educational Context of India

We need critical thinkers—Times of India, 13 June 2011; Can India have a future without critical thinkers—Hindustan Times, 26 June 2016; The elephant in the room—Indian Express, 28 July 2017; Critical thinking a post-truth remedy—The Hindu, 30 Jan 2017.

Educational Policy and Documentation

11. results, 11.1. cultural, social, religious and political sensitivities which impact on purposes for teaching critical thinking, 11.2. the nature of critical thinking.

We understand critical thinking to be purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. [ 101 ]

Collaborative problem solving (CPS) is a critical and necessary skill used in education and in the workforce. While problem solving, as defined in PISA 2012 (OECD, 2010), relates to individuals working alone on resolving problems where a method of solution is not immediately obvious, in CPS, individuals pool their understanding and effort and work together to solve these problems. Collaboration has distinct advantages over individual problem solving because it allows for: (i) an effective division of labour (ii) the incorporation of information from multiple perspectives, experiences and sources of knowledge [ 99 ] enhanced creativity and quality of solutions stimulated by the ideas of other group members.

11.3. Pedagogies of Critical Thinking

11.4. pedagogies to support the development of critical thinking, 12. conclusions and implications.

• Q3. Where critical thinking and its authentic culturally based counterparts appear in educational documents, is there any evidence of classroom-based practices that articulate the policies?
• Q4. If critical thinking pedagogies in any form are evidenced in school- based practices, what implications may these have for culturally authentic, global citizenship?

Author Contributions

Conflicts of interest.

• Foundation for Young Australians. The New Work Minset. Available online: https://www.fya.org.au/wp-content/uploads/2016/11/The-New-Work-Mindset.pdf (accessed on 16 November 2018).
• Paul, R.; Elder, L. Defining Critical Thinking. Available online: http://www.criticalthinking.org/pages/defining-critical-thinking/766 (accessed on 16 November 2018).
• Connell, R. The neoliberal cascade and education: An essay on the market agenda and its consequences. Crit. Stud. Educ. 2013 , 54 , 99–112. [ Google Scholar ] [ CrossRef ]
• Gary, K. Neoliberal education for work versus liberal education for leisure. Stud. Philos. Educ. 2017 , 36 , 83–94. [ Google Scholar ] [ CrossRef ]
• Robinson, K. Out of Our Minds ; Capstone Publishing Co.: West Sussex, UK, 2011. [ Google Scholar ]
• Zhao, Y. World Class Learners ; Coewin: Thousand Oaks Calif, CA, USA, 2012. [ Google Scholar ]
• Steger, M.; Roy, R. Neoliberalism: A Very Short Introduction ; Oxford University Press: New York, NY, USA, 2010. [ Google Scholar ]
• One World Nations. First, Second and Third Worlds. Available online: http://www.nationsonline.org/oneworld/third_world_countries.htm (accessed on 16 November 2018).
• Boli, J.; Ramirez, F.O.; Meyer, J.W. Explaining the origins and expansion of mass education. Comp. Educ. Rev. 1985 , 29 , 145–170. [ Google Scholar ] [ CrossRef ]
• DeMarrais, K.; LeCompte, M. The Way Schools Work: A Sociological Analysis of Education , 2nd ed.; White Longman: Plains, NY, USA, 1995. [ Google Scholar ]
• Tait, G. Making Sense of Mass Education ; Cambridge University Press: Melbourne, Australia, 2013. [ Google Scholar ]
• Kincheloe, J.; Steinberg, S. A tentative description of post-formal thinking: The critical confrontation with cognitive theory. Harv. Educ. Rev. 1993 , 63 , 296–320. [ Google Scholar ] [ CrossRef ]
• Brown, P.; Lauder, H. Education, globalization and economic development. J. Educ. Policy 1996 , 11 , 1–25. [ Google Scholar ] [ CrossRef ]
• Singh, P. Globalization and education. Educ. Theory 2004 , 54 , 103–115. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Dale, R. Globalization and education: Demonstrating a “common world educational culture” or locating a “globally structured educational agenda”? Educ. Theory 2000 , 50 , 427–448. [ Google Scholar ] [ CrossRef ]
• Rizvi, F. Postcolonialism and Globalization in Education. Cult. Stud. Crit. Method 2007 , 7 , 256–263. [ Google Scholar ] [ CrossRef ]
• Gidley, J. Prospective youth visions through imaginative education. Futures 1998 , 30 , 395–408. [ Google Scholar ] [ CrossRef ]
• Akbari, R. Reflections on reflection: A critical appraisal of reflective practices in L2 teacher education. System 2007 , 35 , 192–207. [ Google Scholar ] [ CrossRef ]
• Levitt, R. Freedom and empowerment: A transformative pedagogy of educational reform. Educ. Stud. 2008 , 44 , 47–61. [ Google Scholar ] [ CrossRef ]
• Schmoker, M. What money can’t buy: Powerful, overlooked opportunities for learning. Phi Delta Kappan 2009 , 90 , 524–527. [ Google Scholar ] [ CrossRef ]
• Gidley, J. Beyond homogenisation of global education: Do alternative pedagogies such as Steiner education have anything to offer an emergent globalising world? In Alternative Educational Futures: Pedagogies for an Emergent World ; Inayatullah, S., Bussey, M., Milojevicm, I., Eds.; Sense Publications: Rotterdam, The Netherlands, 2008; pp. 253–268. [ Google Scholar ]
• Gidley, J. Postformal Education: A Philosophy for Complex Futures ; Sternberg, S., Ed.; Springer: Cham, Switzerland, 2016. [ Google Scholar ]
• The Oxford Encyclopedia of Ancient Greece and Rome ; Gagren, M. (Ed.) Oxford University Press: Oxford, UK, 2010. [ Google Scholar ]
• Marmura, M. Ghazali and ash’arism revisited. Arab. Sci. Philos. 2002 , 12 , 91–110. [ Google Scholar ] [ CrossRef ]
• Wiktorowicz, Q. Anatomy of the Salafi Movement. Stud. Confl. Terror. 2006 , 29 , 207–239. [ Google Scholar ] [ CrossRef ]
• Amr, S.; Tbakhi, A. Abu Bakr Muhammad Ibn Zakariya Al Razi (Rhazes): Philosopher, physician and alchemist. Ann. Saudi Med. 2007 , 27 , 305–307. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Freire, P. Pedagogy of the Oppressed ; Continuum: New York, NY, USA, 1970. [ Google Scholar ]
• Skemp, R. Intelligence, Learning and Action ; Wiley: New York, NY, USA, 1979. [ Google Scholar ]
• Ernest, P. Constructig Mathematical Knowledge: Epistimology and Mathematics Education ; Falmer Press: London, UK, 1994. [ Google Scholar ]
• Von Glasersfeld, E. Radical Constructivism: A Way of Knowing and Learning ; Falmer Press: London, UK, 1995. [ Google Scholar ]
• Ennis, R. A concept of critical thinking. Harv. Educ. Rev. 1962 , 32 , 81–111. [ Google Scholar ]
• Lipman, M. Critical Thinking: What can it be? Anal. Teach. 1998 , 8 , 1–12. [ Google Scholar ]
• Paul, R. The state of critical thinking today. New Dir. Community Coll. 2005 , 130 , 27–38. [ Google Scholar ] [ CrossRef ]
• Scriven, M.; Paul, R. Defining Critical Thinking. Available online: http://www.criticalthinking.org/aboutCT/define_critical_thinking.cfm (accessed on 7 November 2018).
• Facione, P. Critical Thinking: What It Is and Why It Counts ; Pearson: Upper Saddle River, NJ, USA, 2011. [ Google Scholar ]
• Mc Peck, J. Critical Thinking and Education ; Routledge: Oxford, UK, 2016. [ Google Scholar ]
• Pithers, R.; Soden, R. Critical thinking in education: A review. Educ. Res. 2000 , 42 , 237–249. [ Google Scholar ] [ CrossRef ]
• Garrison, D.R. E-Learning in the 21st Century: A Framework for Theory and Practice ; Routledge: Oxford, UK, 2011. [ Google Scholar ]
• Pusey, M. Economic Rationalism in Canberra ; Cambridge University Press: Melboune, Australia, 1991. [ Google Scholar ]
• Melbourne Declaration on Educational Goals for Young Australians. Available online: http://www.curriculum.edu.au/verve/_resources/National_Declaration_on_the_Educational_Goals_for_Young_Australians.pdf (accessed on 7 November 2018).
• Connell, R. Why do market ‘reforms’ persistently increase inequality? Discourse Stud. Cult. Polit. Educ. 2013 , 34 , 279–285. [ Google Scholar ] [ CrossRef ]
• Meltzer, L. Understanding Executive Function ; Meltzer, L., Ed.; Guildford: New York, NY, USA, 2007. [ Google Scholar ]
• McPherson, S. The ‘New’ Basics and How People are Learning Them. Available online: http://www.fya.org.au/2017/06/29/new-basics-young-people-learning/ (accessed on 7 November 2018).
• Sellars, M. Intrapersonal Intelligence, Executive Function and Stage Three Students. Available online: https://researchbank.acu.edu.au/theses/320/ (accessed on 7 November 2018).
• Baars, B.; Gage, N. Cognition, Brain and Consciousness: An Introduction to Cognitive Neurocience ; Elsevier: Laguna Hills, CA, USA, 2010. [ Google Scholar ]
• Bowen, G. Document analysis as a qualitative research method. Qual. Res. J. 2009 , 9 , 27–40. [ Google Scholar ] [ CrossRef ]
• Ali, N. From Hallaj to Heer: Poetic knowledge and the Muslim tradition. J. Narrat. Polit. 2016 , 3 , 2–26. [ Google Scholar ]
• Marsden, M. Living Islam: Muslim Religious Experience in Pakistan’s North-West Frontier. Available online: https://journals.openedition.org/samaj/215 (accessed on 7 November 2017).
• DFID in 2009–2010 Response to the International Development (Reporting and Transparency) Act 2006. Available online: https://reliefweb.int/report/world/dfid-2009-10-response-international-development-reporting-and-transparency-act-2006 (accessed on 16 November 2018).
• Ahmad, I. Islam, Democracy and Citizenship Education: An Examination of the Social Studies Curriculum in Pakistan. Curr. Issue. Comp. Educ. 2004 , 7 , 39–49. [ Google Scholar ]
• A National Framework for Professional Standards for Teaching. Available online: http://www.curriculum.edu.au/verve/_resources/national_framework_file.pdf (accessed on 7 November 2018).
• Australian Curriculum and Reporting Authority. General Capabilities in the Australian Curriculum ; ACARA: Sydney, Australia, 2013.
• Paul, R.; Elder, L. Critical thinking: Teaching students how to study and learn (part I). J. Dev. Educ. 2002 , 26 , 36. [ Google Scholar ]
• Baumfield, V.; Hall, E.; Wall, K. Action Research in Education: Learning Through Practitioner Enquiry ; Sage: London, UK, 2017. [ Google Scholar ]
• Harvey, L.; Moon, S.; Geall, V.; Bower, R. Graduates’ Work: Organisational Change and Students’ Attributes ; Centre for Research into Quality, University of Central England: Birmingham, UK, 1997. [ Google Scholar ]
• Australian Curriculum and Reporting Authority. The Australian Curriculum: History (Version 5.2) ; ACARA: Sydney, Australisa, 2013.
• Bui, L.T. Nang cao suc canh tranh cho sinh vien vietnam tren thu truong lao dong trong nuoc va quoc te (Improving vietnamese students’ competitiveness in the domestic and international labour force). Tap Chi Phat Trien va Hoi Nhap 2013 , 6 , 55–60. [ Google Scholar ]
• Atkinson, D. A critical approach to critical thinking in TESOL. TESOL Q. 1997 , 31 , 71–94. [ Google Scholar ] [ CrossRef ]
• Fox, H. Listening to the World: Cultural Issues in Academic Writing. Available online: https://eric.ed.gov/?id=ED373331 (accessed on 7 November 2018).
• Bureau of Naval Personel UN. Confucianism in Vietnam. Available online: http://www.sacred-texts.com/asia/rsv/rsv06.htm (accessed on 7 November 2018).
• Hofstede, G. Cultural differences in teaching and learning. Int. J. Intercult. Relat. 1986 , 10 , 301–320. [ Google Scholar ] [ CrossRef ]
• Nisbett, R.E. The Geography of Thought: How Asians and Westerners Think Differently—and Why ; Free Press: New York, NY, USA, 2003. [ Google Scholar ]
• Nguyen, K.K. Introduction to Vietnamese Culture (National Report) ; United Nations Educational, Scientific and Cultural Organisation (UNESCO): Paris, France, 1960. [ Google Scholar ]
• Nguyen, Q.K.; Nguyen, Q.C. Education in Vietnam: Development History, Challenges, and Solutions. Available online: https://openknowledge.worldbank.org/handle/10986/6424 (accessed on 16 November 2018).
• Huynh, N.T. Tiep xuc van hoa phuong tay va su hoi nhap van hoa cua thanh pho ho chi minh trong qua trinh phat trien (Ho Chi Minh City in the Process of Western Interaction and Cultural Integration). Available online: http://www.tapchicongsan.org.vn/Home/Thong-tin-ly-luan/2013/24828/Anh-huong-cua-van-hoa-nuoc-ngoai-den-van-hoa-Viet-Nam.aspx (accessed on 16 November 2018).
• Bodewig, C.; Badiani-Magnusson, R. Skilling up Vietnam: Preparing the Workforce for a Modern Market Economy (Annual Report). Available online: https://openknowledge.worldbank.org/handle/10986/18778 (accessed on 16 November 2018).
• Halpern, D.F. Thought and Knowledge: An Introduction to Critical Thinking , 5th ed.; Psychology Press: East Sussex, UK, 2014. [ Google Scholar ]
• Le, T.A. Anh huong cua van hoa nuoc ngoai den van hoa Viet Nam hien nay (Influences of Foreign Cultures to Contemporary Vietnam). Available online: http://www.tapchicongsan.org.vn/Home/Thong-tin-ly-luan/2013/24828/Anh-huong-cua-van-hoa-nuoc-ngoai-den-van-hoa-Viet-Nam.aspx (accessed on 7 November 2018).
• EIU. Educational Outcomes for College Students in Business Administration Department ; Eastern International University: Thủ Dầu Một, Vietnam, 2012. [ Google Scholar ]
• TDTU. Educational Outcomes for Master Degree in Business Administration ; Ton Duc Thang University: Ho Chi Minh City, Vietnam, 2015. [ Google Scholar ]
• To, H.P. Educational Outcomes for Students in Business Administration Department ; Hoa Sen University: Ho Chi Minh City, Vietnam, 2009. [ Google Scholar ]
• Do, T.K. Nhung giai phap nham dinh hinh mot phong cach tu duy phan bien (Solutions for Developing Students’ Critical Thinking). Tap chi Phat trien va Hoi nhap 2013 , 4 , 65–67. [ Google Scholar ]
• Duong, T.H.H. Ban chat cua hoat dong doc van va viec day doc van ban van hoc trong nha truong (The Essence of Literature Reading Activity and Reading-Comprehension Teaching in Secondary Schools). Tap Chi Khoa Hoc 2014 , 56 , 48. [ Google Scholar ]
• Le THCG. Understanding about Critical Thinking: Institute of Educational Research. Available online: http://www.criticalthinking.org/pages/center-for-critical-thinking/401 (accessed on 7 November 2018).
• Ennis, R. The logical basis for measuring CT skills. Educ. Leadersh. 1985 , 43 , 44–48. [ Google Scholar ]
• Ennis, R. Critical thinking assessment. Theory Pract. 1993 , 2 , 179–186. [ Google Scholar ] [ CrossRef ]
• Paul, R.; Elder, L. Critical thinking: The nature of critical and creative thought. J. Dev. Educ. 2006 , 30 , 34–35. [ Google Scholar ]
• Paul, R. Critical Thinking: What Every Person Needs to Survive in a Rapidly Changing World ; Willson, J., Binker, A.J.A., Eds.; Foundation for Critical Thinking: Tomales, CA, USA, 2012. [ Google Scholar ]
• Bui, L.T. Day va ren luyen ky nang tu duy phan bien cho sinh vien (Teaching Critical Thinking for University Students). Tap Chi Phat Trien va Hoi Nhap 2013 , 7 , 76–81. [ Google Scholar ]
• Phung, T.H. A Pilot Comprehensive Critical Thinking Education Framework in TESOL. In Frontiers of Language and Teaching: Proceedings of the 2010 International Online Language Conference (IOLC 2010) ; Shafaei, A., Ed.; Universal Publishers: Boca Raton, FL, USA, 2010; pp. 124–134. [ Google Scholar ]
• Barber, M.; Whelan, F.; Clark, M. McKinsey & Company: Our Insights. Available online: http://mckinseyonsociety.com/capturing-the-leadership-premium/ (accessed on 7 November 2018).
• Nguyen, V.T. Tim hieu mot so thuat ngu trong van kien dai hoc XI cua Dang (Terms using in documents of the 10th national congress of the Communist Party of Vietnam) ; Chinh tri quoc gia Press: Haboi, Vietnam, 2011. [ Google Scholar ]
• Niemierko, B. Taxonomies of educational goals as a lead into creative teacher training. Pol. J. Soc. Sci. 2009 , 4 , 93–106. [ Google Scholar ]
• MOET. Huong dan bien soan de kiem tra (Guidelines for Designing Tests for General Education) ; Ministry of Education and Training: Hanoi, Vietnam, 2010.
• Government of Vietnam. Project on Curriculum and Textbook Renovation for General Education ; Government of Vietnam: Hanoi, Vietnam, 2017.
• Kamii, C. Toward autonomy: The importance of critical thinking and choice making. Sch. Psychol. Rev. 1991 , 20 , 382–388. [ Google Scholar ]
• Bailin, S.; Case, R.; Coombs, J.R.; Daniels, L.B. Common misconceptions of critical thinking. J. Curric. Stud. 1999 , 31 , 269–283. [ Google Scholar ] [ CrossRef ]
• Hirst, J.C. A questioning approach: Learning from Shankara’s pedagogic techniques. Contemp. Educ. Dialogue 2005 , 2 , 137–169. [ Google Scholar ] [ CrossRef ]
• Vaidya, A. Does critical thinking and logic education have a Western bias? The case of the Nyāya School of Classical Indian Philosophy. J. Philos. Educ. 2017 , 51 , 132–160. [ Google Scholar ] [ CrossRef ]
• Nambissan, G.B.; Ball, S.J. Advocacy networks, choice and private schooling of the poor in India. Glob. Netw. 2010 , 10 , 324–343. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Kumar, K. What is Worth Teaching , 3rd ed.; Orient Longman: Hyderabad, India, 2004. [ Google Scholar ]
• Venkatesh, K. Negotiating the ‘Social’ in Elementary School Social Science. Econ. Polit. Wkly. 2017 , 52 , 169–172. [ Google Scholar ]
• National Council of Educational Research and Training. National Curriculum Framework 2005 ; NCERT: New Delhi, India, 2005.
• Government of India. The Right of Children to Free and Compulsory Education Act ; Ministry of Law and Justice, Ed.; Government of India: New Delhi, India, 2009.
• Kumar, K. Political Agenda of Education: A Study of Colonialist and Nationalist Ideas , 2nd ed.; Sage Publications: New Delhi, India, 2005. [ Google Scholar ]
• Chopra, R.; Jeffrey, P. Educational Regimes in Contemporary India ; Sage Publications: New Delhi, India, 2005. [ Google Scholar ]
• Sarangapani, P.M. Constructing School Knowledge: An Ethnography of Learning in an Indian Village ; Sage Publications: New Delhi, India, 2003. [ Google Scholar ]
• Sriprakash, A. Pedagogies for Development: The Politics and Practice of Child Centered Education in India ; Springer: Dordrecht, The Netherlands, 2012. [ Google Scholar ]
• Ennis, R. A taxonomy of critical thinking dispositions and abilities. In Teaching Thinking Skills: Theory and Practice ; Baron, J., Sternberg, R., Eds.; W.H. Freeman: New York, NY, USA, 1987; pp. 9–26. [ Google Scholar ]
• Anderson, L.; Krathwohl, D. Taxonomy of Teaching and Learning: A Revision of Bloom’s Taxonomy of Educational Objectives ; Longman: New York, NY, USA, 2000. [ Google Scholar ]
• Facione, P. The Ideal Critical Thinker. Available online: https://www.insightassessment.com/Resources/Importance-of-Critical-Thinking/Expert-Consensus-on-Critical-Thinking/Delphi-Expert-Consensus-Table-1-The-Ideal-Critical-Thinker (accessed on 9 November 2018).
• Cummins, J. Transformative multiliteracies pedagogy: School-based strategies for closing the achievement gap. Mult. Voice Ethn. Divers. Except. Learn. 2009 , 11 , 38–56. [ Google Scholar ]
• Burgh, G.; Field, T.; Freakley, M. Ethics and the Community of Enquiry: An Approach to Ethics Education ; Thomson Social Science Press: Melbourne, Australia, 2005. [ Google Scholar ]
• Zhao, Y. Students as change partners: A proposal for educational change in the age of globalization. J. Educ. Chang. 2011 , 12 , 267–279. [ Google Scholar ] [ CrossRef ]
• Counting What Counts: Reframing Educational Outcomes ; Zhao, Y. (Ed.) Hawker Brownlow: Victoria, Australia, 2017. [ Google Scholar ]
• Paul, R.; Elder, L. The Miniature Guide to Critical Thinking. Available online: https://www.criticalthinking.org/files/Concepts_Tools.pdf (accessed on 9 November 2018).
• McGregor, S. Transformative education grief and growth. In Narrating Transformative Learning in Education ; Gardner, M., Kelly, U., Eds.; Palgrave Macmillan: New York, NY, USA, 2008. [ Google Scholar ]
• Lave, J.; Wenger, E. Situated Learning: Legitimate Periferal Participation ; Cambridge University Press: Cambridge, UK, 1991. [ Google Scholar ]
• Smith, M. Communities of Practice, the Encyclopedia of Informal Education. Available online: www.infed.org/biblio/communities_of_practice.htm (accessed on 9 November 2018).
• Silova, I.; Steiner-Khamsi, G. How NGOs React: Globalization and Education Reform in the Caucasus, Central Asia and Mongolia ; Kumarian Press: Bloomfield, CT, USA, 2008. [ Google Scholar ]
• Miller, J. The Holistic Curriculum ; University of Toronto Press: Toronto, ON, Canada, 2007. [ Google Scholar ]
• Haberman, M. 11 consequences of failing to address the ‘Pedagogy of Poverty’. Phi Delta Kappan 2010 , 92 , 45. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Sellars, M.; Fakirmohammad, R.; Bui, L.; Fishetti, J.; Niyozov, S.; Reynolds, R.; Thapliyal, N.; Liu-Smith, Y.-L.; Ali, N. Conversations on Critical Thinking: Can Critical Thinking Find Its Way Forward as the Skill Set and Mindset of the Century? Educ. Sci. 2018 , 8 , 205. https://doi.org/10.3390/educsci8040205

Sellars M, Fakirmohammad R, Bui L, Fishetti J, Niyozov S, Reynolds R, Thapliyal N, Liu-Smith Y-L, Ali N. Conversations on Critical Thinking: Can Critical Thinking Find Its Way Forward as the Skill Set and Mindset of the Century? Education Sciences . 2018; 8(4):205. https://doi.org/10.3390/educsci8040205

Sellars, Maura, Razia Fakirmohammad, Linh Bui, John Fishetti, Sarfaroz Niyozov, Ruth Reynolds, Nisha Thapliyal, Yu-Ling Liu-Smith, and Nosheen Ali. 2018. "Conversations on Critical Thinking: Can Critical Thinking Find Its Way Forward as the Skill Set and Mindset of the Century?" Education Sciences 8, no. 4: 205. https://doi.org/10.3390/educsci8040205

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 11 September 2019

Inquiry and critical thinking skills for the next generation: from artificial intelligence back to human intelligence

• Jonathan Michael Spector   ORCID: orcid.org/0000-0002-6270-3073 1 &
• Shanshan Ma 1  

Smart Learning Environments volume  6 , Article number:  8 ( 2019 ) Cite this article

35k Accesses

64 Citations

32 Altmetric

Metrics details

Along with the increasing attention to artificial intelligence (AI), renewed emphasis or reflection on human intelligence (HI) is appearing in many places and at multiple levels. One of the foci is critical thinking. Critical thinking is one of four key 21st century skills – communication, collaboration, critical thinking and creativity. Though most people are aware of the value of critical thinking, it lacks emphasis in curricula. In this paper, we present a comprehensive definition of critical thinking that ranges from observation and inquiry to argumentation and reflection. Given a broad conception of critical thinking, a developmental approach beginning with children is suggested as a way to help develop critical thinking habits of mind. The conclusion of this analysis is that more emphasis should be placed on developing human intelligence, especially in young children and with the support of artificial intelligence. While much funding and support goes to the development of artificial intelligence, this should not happen at the expense of human intelligence. Overall, the purpose of this paper is to argue for more attention to the development of human intelligence with an emphasis on critical thinking.

Introduction

In recent decades, advancements in Artificial Intelligence (AI) have developed at an incredible rate. AI has penetrated into people’s daily life on a variety of levels such as smart homes, personalized healthcare, security systems, self-service stores, and online shopping. One notable AI achievement was when AlphaGo, a computer program, defeated the World Go Champion Mr. Lee Sedol in 2016. In the previous year, AlphaGo won in a competition against a professional Go player (Silver et al. 2016 ). As Go is one of the most challenging games, the wins of AI indicated a breakthrough. Public attention has been further drawn to AI since then, and AlphaGo continues to improve. In 2017, a new version of AlphaGo beat Ke Jie, the current world No.1 ranking Go player. Clearly AI can manage high levels of complexity.

Given many changes and multiple lines of development and implement, it is somewhat difficult to define AI to include all of the changes since the 1980s (Luckin et al. 2016 ). Many definitions incorporate two dimensions as a starting point: (a) human-like thinking, and (b) rational action (Russell and Norvig 2009 ). Basically, AI is a term used to label machines (computers) that imitate human cognitive functions such as learning and problem solving, or that manage to deal with complexity as well as human experts.

AlphaGo’s wins against human players were seen as a comparison between artificial and human intelligence. One concern is that AI has already surpassed HI; other concerns are that AI will replace humans in some settings or that AI will become uncontrollable (Epstein 2016 ; Fang et al. 2018 ). Scholars worry that AI technology in the future might trigger the singularity (Good 1966 ), a hypothesized future that the development of technology becomes uncontrollable and irreversible, resulting in unfathomable changes to human civilization (Vinge 1993 ).

The famous theoretical physicist Stephen Hawking warned that AI might end mankind, yet the technology he used to communicate involved a basic form of AI (Cellan-Jones 2014 ). This example highlights one of the basic dilemmas of AI – namely, what are the overall benefits of AI versus its potential drawbacks, and how to move forward given its rapid development? Obviously, basic or controllable AI technologies are not what people are afraid of. Spector et al. 1993 distinguished strong AI and weak AI. Strong AI involves an application that is intended to replace an activity performed previously by a competent human, while weak AI involves an application that aims to enable a less experienced human to perform at a much higher level. Other researchers categorize AI into three levels: (a) artificial narrow intelligence (Narrow AI), (b) artificial general intelligence (General AI), and (c) artificial super intelligence (Super AI) (Siau and Yang 2017 ; Zhang and Xie 2018 ). Narrow AI, sometimes called weak AI, refers to a computer that focus on a narrow task such as AlphaZero or a self-driving car. General AI, sometimes referred to as strong AI, is the simulation of human-level intelligence, which can perform more cognitive tasks as well as most humans do. Super AI is defined by Bostrom ( 1998 ) as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills” (p.1).

Although the consequence of singularity and its potential benefits or harm to the human race have been intensely debated, an undeniable fact is that AI is capable of undertaking recursive self-improvement. With the increasing improvement of this capability, more intelligent generations of AI will appear rapidly. On the other hand, HI has its own limits and its development requires continuous efforts and investment from generation to generation. Education is the main approach humans use to develop and improve HI. Given the extraordinary growth gap between AI and HI, eventually AI can surpass HI. However, that is no reason to neglect the development and improvement of HI. In addition, in contrast to the slow development rate of HI, the growth of funding support to AI has been rapidly increasing according to the following comparison of support for artificial and human intelligence.

The funding support for artificial and human intelligence

There are challenges in comparing artificial and human intelligence by identifying funding for both. Both terms are somewhat vague and can include a variety of aspects. Some analyses will include big data and data analytics within the sphere of artificial intelligence and others will treat them separately. Some will include early childhood developmental research within the sphere of support for HI and others treat them separately. Education is a major way of human beings to develop and improve HI. The investments in education reflect the efforts put on the development of HI, and they pale in comparison with investments in AI.

Sources also vary from governmental funding of research and development to business and industry investments in related research and development. Nonetheless, there are strong indications of increased funding support for AI in North America, Europe and Asia, especially in China. The growth in funding for AI around the world is explosive. According to ZDNet, AI funding more than doubled from 2016 to 2017 and more than tripled from 2016 to 2018. The growth in funding for AI in the last 10 years has been exponential. According to Venture Scanner, there are approximately 2500 companies that have raised $60 billion in funding from 3400 investors in 72 different countries (see https://www.slideshare.net/venturescanner/artificial-intelligence-q1-2019-report-highlights ). Areas included in the Venture Scanner analysis included virtual assistants, recommendation engines, video recognition, context-aware computing, speech recognition, natural language processing, machine learning, and more.

The above data on AI funding focuses primarily on companies making products. There is no direct counterpart in the area of HI where the emphasis is on learning and education. What can be seen, however, are trends within each area. The above data suggest exponential growth in support for AI. In contrast, according to the Urban Institute, per-student funding in the USA has been relatively flat for nearly two decades, with a few states showing modest increases and others showing none (see http://apps.urban.org/features/education-funding-trends/ ). Funding for education is complicated due to the various sources. In the USA, there are local, state and federal sources to consider. While that mixture of funding sources is complex, it is clear that federal and state spending for education in the USA experienced an increase after World War II. However, since the 1980s, federal spending for education has steadily declined, and state spending on education in most states has declined since 2010 according to a government report (see https://www.usgovernmentspending.com/education_spending ). This decline in funding reflects the decreasing emphasis on the development of HI, which is a dangerous signal.

Decreased support for education funding in the USA is not typical of what is happening in other countries, according to The Hechinger Report (see https://hechingerreport.org/rest-world-invests-education-u-s-spends-less/ ). For example, in the period of 2010 to 2014, American spending on elementary and high school education declined 3%, whereas in the same period, education spending in the 35 countries in the OECD rose by 5% with some countries experiencing very significant increases (e.g., 76% in Turkey).

Such data can be questioned in terms of how effectively funds are being spent or how poorly a country was doing prior to experiencing a significant increase. However, given the performance of American students on the Program for International Student Assessment (PISA), the relative lack of funding support in the USA is roughly related with the mediocre performance on PISA tests (see https://nces.ed.gov/surveys/pisa/pisa2015/index.asp ). Research by Darling-Hammond ( 2014 ) indicated that in order to improve learning and reduce the achievement gap, systematic government investments in high-need schools would be more effective if the focus was on capacity building, improving the knowledge and skills of educators and the quality of curriculum opportunities.

Though HI could not be simply defined by the performance on PISA test, improving HI requires systematic efforts and funding support in high-need areas as well. So, in the following section, we present a reflection on HI.

Reflection on human intelligence

Though there is a variety of definitions of HI, from the perspective of psychology, according to Sternberg ( 1999 ), intelligence is a form of developing expertise, from a novice or less experienced person to an expert or more experienced person, a student must be through multiple learning (implicit and explicit) and thinking (critical and creative) processes. In this paper, we adopted such a view and reflected on HI in the following section by discussing learning and critical thinking.

What is learning?

We begin with Gagné’s ( 1985 ) definition of learning as characterized by stable and persistent changes in what a person knows or can do. How do humans learn? Do you recall how to prove that the square root of 2 is not a rational number, something you might have learned years ago? The method is intriguing and is called an indirect proof or a reduction to absurdity – assume that the square root of 2 is a rational number and then apply truth preserving rules to arrive at a contradiction to show that the square root of 2 cannot be a rational number. We recommend this as an exercise for those readers who have never encountered that method of learning and proof. (see https://artofproblemsolving.com/wiki/index.php/Proof_by_contradiction ). Yet another interesting method of learning is called the process of elimination, sometimes accredited to Arthur Conan Doyle’s ( 1926 ) in The Adventure of the Blanched Soldier – Sherlock Holmes says to Dr. Watson that the process of elimination “starts upon the supposition that when you have eliminated all which is impossible, that whatever remains, however improbable, must be the truth ” (see https://www.dfw-sherlock.org/uploads/3/7/3/8/37380505/1926_november_the_adventure_of_the_blanched_soldier.pdf ).

The reason to mention Sherlock Holmes early in this paper is to emphasize the role that observation plays in learning. The character Sherlock Holmes was famous for his observation skills that led to his so-called method of deductive reasoning (a process of elimination), which is what logicians would classify as inductive reasoning as the conclusions of that reasoning process are primarily probabilistic rather than certain, unlike the proof of the irrationality of the square root of 2 mentioned previously.

In dealing with uncertainty, it seems necessary to make observations and gather evidence that can lead one to a likely conclusion. Is that not what reasonable people and accomplished detectives do? It is certainly what card counters do at gambling houses; they observe high and low value cards that have already been played in order to estimate the likelihood of the next card being a high or low value card. Observation is a critical process in dealing with uncertainty.

Moreover, humans typically encounter many uncertain situations in the course of life. Few people encounter situations which require resolution using a mathematical proof such as the one with which this article began. Jonassen ( 2000 , 2011 ) argued that problem solving is one of the most important and frequent activities in which people engage. Moreover, many of the more challenging problems are ill-structured in the sense that (a) there is incomplete information pertaining to the situation, or (b) the ideal resolution of the problem is unknown, or (c) how to transform a problematic situation into an acceptable situation is unclear. In short, people are confronted with uncertainty nearly every day and in many different ways. The so called key 21st century skills of communication, collaboration, critical thinking and creativity (the 4 Cs; see http://www.battelleforkids.org/networks/p21 ) are important because uncertainty is a natural and inescapable aspect of the human condition. The 4 Cs are interrelated and have been presented by Spector ( 2018 ) as interrelated capabilities involving logic and epistemology in the form of the new 3Rs – namely, re-examining, reasoning, and reflecting. Re-examining is directly linked to observation as a beginning point for inquiry. The method of elimination is one form of reasoning in which a person engages to solve challenging problems. Reflecting on how well one is doing in the life-long enterprise of solving challenging problems is a higher kind of meta-cognitive activity in which accomplished problem-solvers engage (Ericsson et al. 1993 ; Flavell 1979 ).

Based on these initial comments, a comprehensive definition of critical thinking is presented next in the form of a framework.

A framework of critical thinking

Though there is variety of definitions of critical thinking, a concise definition of critical thinking remains elusive. For delivering a direct understanding of critical thinking to readers such as parents and school teachers, in this paper, we present a comprehensive definition of critical thinking through a framework that includes many of the definitions offered by others. Critical thinking, as treated broadly herein, is a multi-dimensioned and multifaceted human capability. Critical thinking has been interpreted from three perspectives: education, psychology, and epistemology, all of which are represented in the framework that follows.

In a developmental approach to critical thinking, Spector ( 2019 ) argues that critical thinking involves a series of cumulative and related abilities, dispositions and other variables (e.g., motivation, criteria, context, knowledge). This approach proceeds from experience (e.g., observing something unusual) and then to various forms of inquiry, investigation, examination of evidence, exploration of alternatives, argumentation, testing conclusions, rethinking assumptions, and reflecting on the entire process.

Experience and engagement are ongoing throughout the process which proceeds from relatively simple experiences (e.g., direct and immediate observation) to more complex interactions (e.g., manipulation of an actual or virtual artifact and observing effects).

The developmental approach involves a variety of mental processes and non-cognitive states, which help a person’s decision making to become purposeful and goal directed. The associated critical thinking skills enable individuals to be likely to achieve a desired outcome in a challenging situation.

In the process of critical thinking, apart from experience, there are two additional cognitive capabilities essential to critical thinking – namely, metacognition and self-regulation . Many researchers (e.g., Schraw et al. 2006 ) believe that metacognition has two components: (a) awareness and understanding of one’s own thoughts, and (b) the ability to regulate one’s own cognitive processes. Some other researchers put more emphasis on the latter component. For example, Davies ( 2015 ) described metacognition as the capacity to monitor the quality of one’s thinking process, and then to make appropriate changes. However, the American Psychology Association (APA) defines metacognition as an awareness and understanding of one’s own thought with the ability to control related cognitive processes (see https://psycnet.apa.org/record/2008-15725-005 ).

Although the definition and elaboration of these two concepts deserve further exploration, they are often used interchangeably (Hofer and Sinatra 2010 ; Schunk 2008 ). Many psychologists see the two related capabilities of metacognition and self-regulation as being closely related - two sides on one coin, so to speak. Metacognition involves or emphasizes awareness, whereas self-regulation involves and emphasizes appropriate control. These two concepts taken together enable a person to create a self-regulatory mechanism, which monitors and regulates the corresponding skills (e.g., observation, inquiry, interpretation, explanation, reasoning, analysis, evaluation, synthesis, reflection, and judgement).

As to the critical thinking skills, it should be noted that there is much discussion about the generalizability and domain specificity of them, just as there is about problem-solving skills in general (Chi et al. 1982 ; Chiesi et al. 1979 ; Ennis 1989 ; Fischer 1980 ). The research supports the notion that to achieve high levels of expertise and performance, one must develop high levels of domain knowledge. As a consequence, becoming a highly effective critical thinker in a particular domain of inquiry requires significant domain knowledge. One may achieve such levels in a domain in which one has significant domain knowledge and experience but not in a different domain in which one has little domain knowledge and experience. The processes involved in developing high levels of critical thinking are somewhat generic. Therefore, it is possible to develop critical thinking in nearly any domain when the two additional capabilities of metacognition and self-regulation are coupled with motivation and engagement and supportive emotional states (Ericsson et al. 1993 ).

Consequently, the framework presented here (see Fig. 1 ) is built around three main perspectives about critical thinking (i.e., educational, psychological and epistemological) and relevant learning theories. This framework provides a visual presentation of critical thinking with four dimensions: abilities (educational perspective), dispositions (psychological perspective), levels (epistemological perspective) and time. Time is added to emphasize the dynamic nature of critical thinking in terms of a specific context and a developmental approach.

Critical thinking often begins with simple experiences such as observing a difference, encountering a puzzling question or problem, questioning someone’s statement, and then leads, in some instances to an inquiry, and then to more complex experiences such as interactions and application of higher order thinking skills (e.g., logical reasoning, questioning assumptions, considering and evaluating alternative explanations).

If the individual is not interested in what was observed, an inquiry typically does not begin. Inquiry and critical thinking require motivation along with an inquisitive disposition. The process of critical thinking requires the support of corresponding internal indispositions such as open-mindedness and truth-seeking. Consequently, a disposition to initiate an inquiry (e.g., curiosity) along with an internal inquisitive disposition (e.g., that links a mental habit to something motivating to the individual) are both required (Hitchcock 2018 ). Initiating dispositions are those that contribute to the start of inquiry and critical thinking. Internal dispositions are those that initiate and support corresponding critical thinking skills during the process. Therefore, critical thinking dispositions consist of initiating dispositions and internal dispositions. Besides these factors, critical thinking also involves motivation. Motivation and dispositions are not mutually exclusive, for example, curiosity is a disposition and also a motivation.

Critical thinking abilities and dispositions are two main components of critical thinking, which involve such interrelated cognitive constructs as interpretation, explanation, reasoning, evaluation, synthesis, reflection, judgement, metacognition and self-regulation (Dwyer et al. 2014 ; Davies 2015 ; Ennis 2018 ; Facione 1990 ; Hitchcock 2018 ; Paul and Elder 2006 ). There are also some other abilities such as communication, collaboration and creativity, which are now essential in current society (see https://en.wikipedia.org/wiki/21st_century_skills ). Those abilities along with critical thinking are called the 4Cs; they are individually monitored and regulated through metacognitive and self-regulation processes.

The abilities involved in critical thinking are categorized in Bloom’s taxonomy into higher order skills (e.g., analyzing and synthesizing) and lower level skills (e.g., remembering and applying) (Anderson and Krathwohl 2001 ; Bloom et al. 1956 ).

The thinking process can be depicted as a spiral through both lower and higher order thinking skills. It encompasses several reasoning loops. Some of them might be iterative until a desired outcome is achieved. Each loop might be a mix of higher order thinking skills and lower level thinking skills. Each loop is subject to the self-regulatory mechanism of metacognition and self-regulation.

But, due to the complexity of human thinking, a specific spiral with reasoning loops is difficult to represent. Therefore, instead of a visualized spiral with an indefinite number of reasoning loops, the developmental stages of critical thinking are presented in the diagram (Fig. 1 ).

Besides, most of the definitions of critical thinking are based on the imagination about ideal critical thinkers such as the consensus generated from the Delphi report (Facione 1990 ). However, according to Dreyfus and Dreyfus ( 1980 ), in the course of developing an expertise, students would pass through five stages. Those five stages are “absolute beginner”, “advanced beginner”, “competent performer”, “proficient performer,” and “intuitive expert performer”. Dreyfus and Dreyfus ( 1980 ) described the five stages the result of the successive transformations of four mental functions: recollection, recognition, decision making, and awareness.

In the course of developing critical thinking and expertise, individuals will pass through similar stages which are accompanied with the increasing practices and accumulation of experience. Through the intervention and experience of developing critical thinking, as a novice, tasks are decomposed into context-free features which could be recognized by students without the experience of particular situations. For further improving, students need to be able to monitor their awareness, and with a considerable experience. They can note recurrent meaningful component patterns in some contexts. Gradually, increased practices expose students to a variety of whole situations which enable the students to recognize tasks in a more holistic manner as a professional. On the other hand, with the increasing accumulation of experience, individuals are less likely to depend simply on abstract principles. The decision will turn to something intuitive and highly situational as well as analytical. Students might unconsciously apply rules, principles or abilities. A high level of awareness is absorbed. At this stage, critical thinking is turned into habits of mind and in some cases expertise. The description above presents a process of critical thinking development evolving from a novice to an expert, eventually developing critical thinking into habits of mind.

We mention the five-stage model proposed by Dreyfus and Dreyfus ( 1980 ) to categorize levels of critical thinking and emphasize the developmental nature involved in becoming a critical thinker. Correspondingly, critical thinking is categorized into 5 levels: absolute beginner (novice), advanced beginner (beginner), competent performer (competent), proficient performer (proficient), and intuitive expert (expert).

Ability level and critical thinker (critical thinking) level together represent one of the four dimensions represented in Fig. 1 .

In addition, it is noteworthy that the other two elements of critical thinking are the context and knowledge in which the inquiry is based. Contextual and domain knowledge must be taken into account with regard to critical thinking, as previously argued. Besides, as Hitchcock ( 2018 ) argued, effective critical thinking requires knowledge about and experience applying critical thinking concepts and principles as well.

Critical thinking is considered valuable across disciplines. But except few courses such as philosophy, critical thinking is reported lacking in most school education. Most of researchers and educators thus proclaim that integrating critical thinking across the curriculum (Hatcher 2013 ). For example, Ennis ( 2018 ) provided a vision about incorporating critical thinking across the curriculum in higher education. Though people are aware of the value of critical thinking, few of them practice it. Between 2012 and 2015, in Australia, the demand of critical thinking as one of the enterprise skills for early-career job increased 125% (Statista Research Department, 2016). According to a survey across 1000 adults by The Reboot Foundation 2018 , more than 80% of respondents believed that critical thinking skills are lacking in today’s youth. Respondents were deeply concerned that schools do not teach critical thinking. Besides, the investigation also found that respondents were split over when and how to teach critical thinking, clearly.

In the previous analysis of critical thinking, we presented the mechanism of critical thinking instead of a concise definition. This is because, given the various perspectives of interpreting critical thinking, it is not easy to come out with an unitary definition, but it is essential for the public to understand how critical thinking works, the elements it involves and the relationships between them, so they can achieve an explicit understanding.

In the framework, critical thinking starts from simple experience such as observing a difference, then entering the stage of inquiry, inquiry does not necessarily turn the thinking process into critical thinking unless the student enters a higher level of thinking process or reasoning loops such as re-examining, reasoning, reflection (3Rs). Being an ideal critical thinker (or an expert) requires efforts and time.

According to the framework, simple abilities such as observational skills and inquiry are indispensable to lead to critical thinking, which suggests that paying attention to those simple skills at an early stage of children can be an entry point to critical thinking. Considering the child development theory by Piaget ( 1964 ), a developmental approach spanning multiple years can be employed to help children develop critical thinking at each corresponding development stage until critical thinking becomes habits of mind.

Although we emphasized critical thinking in this paper, for the improvement of intelligence, creative thinking and critical thinking are separable, they are both essential abilities that develop expertise, eventually drive the improvement of HI at human race level.

As previously argued, there is a similar pattern among students who think critically in different domains, but students from different domains might perform differently in creativity because of different thinking styles (Haller and Courvoisier 2010 ). Plus, students have different learning styles and preferences. Personalized learning has been the most appropriate approach to address those differences. Though the way of realizing personalized learning varies along with the development of technologies. Generally, personalized learning aims at customizing learning to accommodate diverse students based on their strengths, needs, interests, preferences, and abilities.

Meanwhile, the advancement of technology including AI is revolutionizing education; students’ learning environments are shifting from technology-enhanced learning environments to smart learning environments. Although lots of potentials are unrealized yet (Spector 2016 ), the so-called smart learning environments rely more on the support of AI technology such as neural networks, learning analytics and natural language processing. Personalized learning is better supported and realized in a smart learning environment. In short, in the current era, personalized learning is to use AI to help learners perform at a higher level making adjustments based on differences of learners. This is the notion with which we conclude – the future lies in using AI to improve HI and accommodating individual differences.

The application of AI in education has been a subject for decades. There are efforts heading to such a direction though personalized learning is not technically involved in them. For example, using AI technology to stimulate critical thinking (Zhu 2015 ), applying a virtual environment for building and assessing higher order inquiry skills (Ketelhut et al. 2010 ). Developing computational thinking through robotics (Angeli and Valanides 2019 ) is another such promising application of AI to support the development of HI.

However, almost all of those efforts are limited to laboratory experiments. For accelerating the development rate of HI, we argue that more emphasis should be given to the development of HI at scale with the support of AI, especially in young children focusing on critical and creative thinking.

In this paper, we argue that more emphasis should be given to HI development. Rather than decreasing the funding of AI, the analysis of progress in artificial and human intelligence indicates that it would be reasonable to see increased emphasis placed on using various AI techniques and technologies to improve HI on a large and sustainable scale. Well, most researchers might agree that AI techniques or the situation might be not mature enough to support such a large-scale development. But it would be dangerous if HI development is overlooked. Based on research and theory drawn from psychology as well as from epistemology, the framework is intended to provide a practical guide to the progressive development of inquiry and critical thinking skills in young children as children represent the future of our fragile planet. And we suggested a sustainable development approach for developing inquiry and critical thinking (See, Spector 2019 ). Such an approach could be realized through AI and infused into HI development. Besides, a project is underway in collaboration with NetDragon to develop gamified applications to develop the relevant skills and habits of mind. A game-based assessment methodology is being developed and tested at East China Normal University that is appropriate for middle school children. The intention of the effort is to refocus some of the attention on the development of HI in young children.

Availability of data and materials

Not applicable.

Abbreviations

Artificial Intelligence

Human Intelligence

L.W. Anderson, D.R. Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives (Allyn & Bacon, Boston, 2001)

Google Scholar  

Angeli, C., & Valanides, N. (2019). Developing young children’s computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy. Comput. Hum. Behav. Retrieved from https://doi.org/10.1016/j.chb.2019.03.018

B.S. Bloom, M.D. Engelhart, E.J. Furst, W.H. Hill, D.R. Krathwohl, Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain (David McKay Company, New York, 1956)

Bostrom, N. (1998). How long before superintelligence? Retrieved from https://nickbostrom.com/superintelligence.html

R. Cellan-Jones, Stephen hawking warns artificial intelligence could end mankind. BBC. News. 2 , 2014 (2014)

M.T.H. Chi, R. Glaser, E. Rees, in Advances in the Psychology of Human Intelligence , ed. by R. S. Sternberg. Expertise in problem solving (Erlbaum, Hillsdale, 1982), pp. 7–77

H.L. Chiesi, G.J. Spliich, J.F. Voss, Acquisition of domain-related information in relation to high and low domain knowledge. J. Verbal Learn. Verbal Behav. 18 , 257–273 (1979)

Article   Google Scholar  

L. Darling-Hammond, What can PISA tell US about US education policy? N. Engl. J. Publ. Policy. 26 (1), 4 (2014)

M. Davies, in Higher education: Handbook of theory and research . A Model of Critical Thinking in Higher Education (Springer, Cham, 2015), pp. 41–92

Chapter   Google Scholar  

A.C. Doyle, in The Strand Magazine . The adventure of the blanched soldier (1926) Retrieved from https://www.dfw-sherlock.org/uploads/3/7/3/8/37380505/1926_november_the_adventure_of_the_blanched_soldier.pdf

S.E. Dreyfus, H.L. Dreyfus, A five-stage model of the mental activities involved in directed skill acquisition (no. ORC-80-2) (University of California-Berkeley Operations Research Center, Berkeley, 1980)

Book   Google Scholar  

C.P. Dwyer, M.J. Hogan, I. Stewart, An integrated critical thinking framework for the 21st century. Think. Skills Creat. 12 , 43–52 (2014)

R.H. Ennis, Critical thinking and subject specificity: Clarification and needed research. Educ. Res. 18 , 4–10 (1989)

R.H. Ennis, Critical thinking across the curriculum: A vision. Topoi. 37 (1), 165–184 (2018)

Epstein, Z. (2016). Has artificial intelligence already surpassed the human brain? Retrieved from https://bgr.com/2016/03/10/alphago-beats-lee-sedol-again/

K.A. Ericsson, R.T. Krampe, C. Tesch-Römer, The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100 (3), 363–406 (1993)

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction [Report for the American Psychology Association]. Retrieved from https://files.eric.ed.gov/fulltext/ED315423.pdf

J. Fang, H. Su, Y. Xiao, Will Artificial Intelligence Surpass Human Intelligence? (2018). https://doi.org/10.2139/ssrn.3173876

K.W. Fischer, A theory of cognitive development: The control and construction of hierarchies of skills. Psychol. Rev. 87 , 477–431 (1980)

J.H. Flavell, Metacognition and cognitive monitoring: A new area of cognitive development inquiry. Am. Psychol. 34 (10), 906–911 (1979)

R.M. Gagné, The conditions of learning and theory of instruction , 4th edn. (Holt, Rinehart, & Winston, New York, 1985)

I.J. Good, Speculations concerning the first ultraintelligent machine. Adv Comput. 6 , 31-88 (1966)

C.S. Haller, D.S. Courvoisier, Personality and thinking style in different creative domains. Psychol. Aesthet. Creat. Arts. 4 (3), 149 (2010)

D.L. Hatcher, Is critical thinking across the curriculum a plausible goal? OSSA. 69 (2013) Retrieved from https://scholar.uwindsor.ca/ossaarchive/OSSA10/papersandcommentaries/69

Hitchcock, D. (2018). Critical thinking. Retrieved from https://plato.stanford.edu/entries/critical-thinking/

B.K. Hofer, G.M. Sinatra, Epistemology, metacognition, and self-regulation: Musings on an emerging field. Metacogn. Learn. 5 (1), 113–120 (2010)

D.H. Jonassen, Toward a design theory of problem solving. Educ. Technol. Res. Dev. 48 (4), 63–85 (2000)

D.H. Jonassen, Learning to Solve Problems: A Handbook for Designing Problem-Solving Learning Environments (Routledge, New York, 2011)

D.J. Ketelhut, B.C. Nelson, J. Clarke, C. Dede, A multi-user virtual environment for building and assessing higher order inquiry skills in science. Br. J. Educ. Technol. 41 (1), 56–68 (2010)

R. Luckin, W. Holmes, M. Griffiths, L.B. Forcier, Intelligence Unleashed: An Argument for AI in Education (Pearson Education, London, 2016) Retrieved from http://oro.open.ac.uk/50104/1/Luckin%20et%20al.%20-%202016%20-%20Intelligence%20Unleashed.%20An%20argument%20for%20AI%20in%20Educ.pdf

R. Paul, L. Elder, The miniature guide to critical thinking: Concepts and tools , 4th edn. (2006) Retrieved from https://www.criticalthinking.org/files/Concepts_Tools.pdf

J. Piaget, Part I: Cognitive development in children: Piaget development and learning. J. Res. Sci. Teach. 2 (3), 176–186 (1964)

S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach , 3rd edn. (Prentice Hall, Upper Saddle River, 2009) ISBN 978-0-136042594

G. Schraw, K.J. Crippen, K. Hartley, Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Res. Sci. Educ. 36 (1–2), 111–139 (2006)

D.H. Schunk, Metacognition, self-regulation, and self-regulated learning: Research recommendations. Educ. Psychol. Rev. 20 (4), 463–467 (2008)

K. Siau, Y. Yang, in Twelve Annual Midwest Association for Information Systems Conference (MWAIS 2017) . Impact of artificial intelligence, robotics, and machine learning on sales and marketing (2017), pp. 18–19

D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, et al., Mastering the game of Go with deep neural networks and tree search. Nature. 529 (7587), 484 (2016)

J. M. Spector, M. C. Polson, D. J. Muraida (eds.), Automating Instructional Design: Concepts and Issues (Educational Technology Publications, Englewood Cliffs, 1993)

J.M. Spector, Smart Learning Environments: Concepts and Issues . In G. Chamblee & L. Langub (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 2728–2737). (Association for the Advancement of Computing in Education (AACE), Savannah, GA, United States, 2016). Retrieved June 4, 2019 from https://www.learntechlib.org/primary/p/172078/ .

J. M. Spector, Thinking and learning in the anthropocene: The new 3 Rs . Discussion paper presented at the International Big History Association Conference, Philadelphia, PA (2018). Retrieved from http://learndev.org/dl/HLAIBHA2018/Spector%2C%20J.%20M.%20(2018).%20Thinking%20and%20Learning%20in%20the%20Anthropocene.pdf .

J. M. Spector, Complexity, Inquiry Critical Thinking, and Technology: A Holistic and Developmental Approach . In Mind, Brain and Technology (pp. 17–25). (Springer, Cham, 2019).

R.J. Sternberg, Intelligence as developing expertise. Contemp. Educ. Psychol. 24 (4), 359–375 (1999)

The Reboot Foundation. (2018). The State of Critical Thinking: A New Look at Reasoning at Home, School, and Work. Retrieved from https://reboot-foundation.org/wp-content/uploads/_docs/REBOOT_FOUNDATION_WHITE_PAPER.pdf

V. Vinge, The Coming Technological Singularity: How to Survive in the Post-Human Era . Resource document. NASA Technical report server. Retrieved from https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19940022856.pdf . Accessed 20 June 2019.

D. Zhang, M. Xie, Artificial Intelligence’s Digestion and Reconstruction for Humanistic Feelings . In 2018 International Seminar on Education Research and Social Science (ISERSS 2018) (Atlantis Press, Paris, 2018)

X. Zhu, in Twenty-Ninth AAAI Conference on Artificial Intelligence . Machine Teaching: An Inverse Problem to Machine Learning and an Approach toward Optimal Education (2015)

Download references

Acknowledgements

We wish to acknowledge the generous support of NetDragon and the Digital Research Centre at the University of North Texas.

Initial work is being funded through the NetDragon Digital Research Centre at the University of North Texas with Author as the Principal Investigator.

Author information

Authors and affiliations.

Department of Learning Technologies, University of North Texas Denton, Texas, TX, 76207, USA

Jonathan Michael Spector & Shanshan Ma

You can also search for this author in PubMed   Google Scholar

Contributions

The authors contributed equally to the effort. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Jonathan Michael Spector .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Spector, J.M., Ma, S. Inquiry and critical thinking skills for the next generation: from artificial intelligence back to human intelligence. Smart Learn. Environ. 6 , 8 (2019). https://doi.org/10.1186/s40561-019-0088-z

Download citation

Received : 06 June 2019

Accepted : 27 August 2019

Published : 11 September 2019

DOI : https://doi.org/10.1186/s40561-019-0088-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Critical thinking
• Developmental model
• Human intelligence
• Inquiry learning

Classroom Q&A

With larry ferlazzo.

In this EdWeek blog, an experiment in knowledge-gathering, Ferlazzo will address readers’ questions on classroom management, ELL instruction, lesson planning, and other issues facing teachers. Send your questions to [email protected]. Read more from this blog.

Eight Instructional Strategies for Promoting Critical Thinking

• Share article

(This is the first post in a three-part series.)

The new question-of-the-week is:

What is critical thinking and how can we integrate it into the classroom?

This three-part series will explore what critical thinking is, if it can be specifically taught and, if so, how can teachers do so in their classrooms.

Today’s guests are Dara Laws Savage, Patrick Brown, Meg Riordan, Ph.D., and Dr. PJ Caposey. Dara, Patrick, and Meg were also guests on my 10-minute BAM! Radio Show . You can also find a list of, and links to, previous shows here.

You might also be interested in The Best Resources On Teaching & Learning Critical Thinking In The Classroom .

Current Events

Dara Laws Savage is an English teacher at the Early College High School at Delaware State University, where she serves as a teacher and instructional coach and lead mentor. Dara has been teaching for 25 years (career preparation, English, photography, yearbook, newspaper, and graphic design) and has presented nationally on project-based learning and technology integration:

There is so much going on right now and there is an overload of information for us to process. Did you ever stop to think how our students are processing current events? They see news feeds, hear news reports, and scan photos and posts, but are they truly thinking about what they are hearing and seeing?

I tell my students that my job is not to give them answers but to teach them how to think about what they read and hear. So what is critical thinking and how can we integrate it into the classroom? There are just as many definitions of critical thinking as there are people trying to define it. However, the Critical Think Consortium focuses on the tools to create a thinking-based classroom rather than a definition: “Shape the climate to support thinking, create opportunities for thinking, build capacity to think, provide guidance to inform thinking.” Using these four criteria and pairing them with current events, teachers easily create learning spaces that thrive on thinking and keep students engaged.

One successful technique I use is the FIRE Write. Students are given a quote, a paragraph, an excerpt, or a photo from the headlines. Students are asked to F ocus and respond to the selection for three minutes. Next, students are asked to I dentify a phrase or section of the photo and write for two minutes. Third, students are asked to R eframe their response around a specific word, phrase, or section within their previous selection. Finally, students E xchange their thoughts with a classmate. Within the exchange, students also talk about how the selection connects to what we are covering in class.

There was a controversial Pepsi ad in 2017 involving Kylie Jenner and a protest with a police presence. The imagery in the photo was strikingly similar to a photo that went viral with a young lady standing opposite a police line. Using that image from a current event engaged my students and gave them the opportunity to critically think about events of the time.

Here are the two photos and a student response:

F - Focus on both photos and respond for three minutes

In the first picture, you see a strong and courageous black female, bravely standing in front of two officers in protest. She is risking her life to do so. Iesha Evans is simply proving to the world she does NOT mean less because she is black … and yet officers are there to stop her. She did not step down. In the picture below, you see Kendall Jenner handing a police officer a Pepsi. Maybe this wouldn’t be a big deal, except this was Pepsi’s weak, pathetic, and outrageous excuse of a commercial that belittles the whole movement of people fighting for their lives.

I - Identify a word or phrase, underline it, then write about it for two minutes

A white, privileged female in place of a fighting black woman was asking for trouble. A struggle we are continuously fighting every day, and they make a mockery of it. “I know what will work! Here Mr. Police Officer! Drink some Pepsi!” As if. Pepsi made a fool of themselves, and now their already dwindling fan base continues to ever shrink smaller.

R - Reframe your thoughts by choosing a different word, then write about that for one minute

You don’t know privilege until it’s gone. You don’t know privilege while it’s there—but you can and will be made accountable and aware. Don’t use it for evil. You are not stupid. Use it to do something. Kendall could’ve NOT done the commercial. Kendall could’ve released another commercial standing behind a black woman. Anything!

Exchange - Remember to discuss how this connects to our school song project and our previous discussions?

This connects two ways - 1) We want to convey a strong message. Be powerful. Show who we are. And Pepsi definitely tried. … Which leads to the second connection. 2) Not mess up and offend anyone, as had the one alma mater had been linked to black minstrels. We want to be amazing, but we have to be smart and careful and make sure we include everyone who goes to our school and everyone who may go to our school.

As a final step, students read and annotate the full article and compare it to their initial response.

Using current events and critical-thinking strategies like FIRE writing helps create a learning space where thinking is the goal rather than a score on a multiple-choice assessment. Critical-thinking skills can cross over to any of students’ other courses and into life outside the classroom. After all, we as teachers want to help the whole student be successful, and critical thinking is an important part of navigating life after they leave our classrooms.

‘Before-Explore-Explain’

Patrick Brown is the executive director of STEM and CTE for the Fort Zumwalt school district in Missouri and an experienced educator and author :

Planning for critical thinking focuses on teaching the most crucial science concepts, practices, and logical-thinking skills as well as the best use of instructional time. One way to ensure that lessons maintain a focus on critical thinking is to focus on the instructional sequence used to teach.

Explore-before-explain teaching is all about promoting critical thinking for learners to better prepare students for the reality of their world. What having an explore-before-explain mindset means is that in our planning, we prioritize giving students firsthand experiences with data, allow students to construct evidence-based claims that focus on conceptual understanding, and challenge students to discuss and think about the why behind phenomena.

Just think of the critical thinking that has to occur for students to construct a scientific claim. 1) They need the opportunity to collect data, analyze it, and determine how to make sense of what the data may mean. 2) With data in hand, students can begin thinking about the validity and reliability of their experience and information collected. 3) They can consider what differences, if any, they might have if they completed the investigation again. 4) They can scrutinize outlying data points for they may be an artifact of a true difference that merits further exploration of a misstep in the procedure, measuring device, or measurement. All of these intellectual activities help them form more robust understanding and are evidence of their critical thinking.

In explore-before-explain teaching, all of these hard critical-thinking tasks come before teacher explanations of content. Whether we use discovery experiences, problem-based learning, and or inquiry-based activities, strategies that are geared toward helping students construct understanding promote critical thinking because students learn content by doing the practices valued in the field to generate knowledge.

An Issue of Equity

Meg Riordan, Ph.D., is the chief learning officer at The Possible Project, an out-of-school program that collaborates with youth to build entrepreneurial skills and mindsets and provides pathways to careers and long-term economic prosperity. She has been in the field of education for over 25 years as a middle and high school teacher, school coach, college professor, regional director of N.Y.C. Outward Bound Schools, and director of external research with EL Education:

Although critical thinking often defies straightforward definition, most in the education field agree it consists of several components: reasoning, problem-solving, and decisionmaking, plus analysis and evaluation of information, such that multiple sides of an issue can be explored. It also includes dispositions and “the willingness to apply critical-thinking principles, rather than fall back on existing unexamined beliefs, or simply believe what you’re told by authority figures.”

Despite variation in definitions, critical thinking is nonetheless promoted as an essential outcome of students’ learning—we want to see students and adults demonstrate it across all fields, professions, and in their personal lives. Yet there is simultaneously a rationing of opportunities in schools for students of color, students from under-resourced communities, and other historically marginalized groups to deeply learn and practice critical thinking.

For example, many of our most underserved students often spend class time filling out worksheets, promoting high compliance but low engagement, inquiry, critical thinking, or creation of new ideas. At a time in our world when college and careers are critical for participation in society and the global, knowledge-based economy, far too many students struggle within classrooms and schools that reinforce low-expectations and inequity.

If educators aim to prepare all students for an ever-evolving marketplace and develop skills that will be valued no matter what tomorrow’s jobs are, then we must move critical thinking to the forefront of classroom experiences. And educators must design learning to cultivate it.

So, what does that really look like?

Unpack and define critical thinking

To understand critical thinking, educators need to first unpack and define its components. What exactly are we looking for when we speak about reasoning or exploring multiple perspectives on an issue? How does problem-solving show up in English, math, science, art, or other disciplines—and how is it assessed? At Two Rivers, an EL Education school, the faculty identified five constructs of critical thinking, defined each, and created rubrics to generate a shared picture of quality for teachers and students. The rubrics were then adapted across grade levels to indicate students’ learning progressions.

At Avenues World School, critical thinking is one of the Avenues World Elements and is an enduring outcome embedded in students’ early experiences through 12th grade. For instance, a kindergarten student may be expected to “identify cause and effect in familiar contexts,” while an 8th grader should demonstrate the ability to “seek out sufficient evidence before accepting a claim as true,” “identify bias in claims and evidence,” and “reconsider strongly held points of view in light of new evidence.”

When faculty and students embrace a common vision of what critical thinking looks and sounds like and how it is assessed, educators can then explicitly design learning experiences that call for students to employ critical-thinking skills. This kind of work must occur across all schools and programs, especially those serving large numbers of students of color. As Linda Darling-Hammond asserts , “Schools that serve large numbers of students of color are least likely to offer the kind of curriculum needed to ... help students attain the [critical-thinking] skills needed in a knowledge work economy. ”

So, what can it look like to create those kinds of learning experiences?

Designing experiences for critical thinking

After defining a shared understanding of “what” critical thinking is and “how” it shows up across multiple disciplines and grade levels, it is essential to create learning experiences that impel students to cultivate, practice, and apply these skills. There are several levers that offer pathways for teachers to promote critical thinking in lessons:

1.Choose Compelling Topics: Keep it relevant

A key Common Core State Standard asks for students to “write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence.” That might not sound exciting or culturally relevant. But a learning experience designed for a 12th grade humanities class engaged learners in a compelling topic— policing in America —to analyze and evaluate multiple texts (including primary sources) and share the reasoning for their perspectives through discussion and writing. Students grappled with ideas and their beliefs and employed deep critical-thinking skills to develop arguments for their claims. Embedding critical-thinking skills in curriculum that students care about and connect with can ignite powerful learning experiences.

2. Make Local Connections: Keep it real

At The Possible Project , an out-of-school-time program designed to promote entrepreneurial skills and mindsets, students in a recent summer online program (modified from in-person due to COVID-19) explored the impact of COVID-19 on their communities and local BIPOC-owned businesses. They learned interviewing skills through a partnership with Everyday Boston , conducted virtual interviews with entrepreneurs, evaluated information from their interviews and local data, and examined their previously held beliefs. They created blog posts and videos to reflect on their learning and consider how their mindsets had changed as a result of the experience. In this way, we can design powerful community-based learning and invite students into productive struggle with multiple perspectives.

3. Create Authentic Projects: Keep it rigorous

At Big Picture Learning schools, students engage in internship-based learning experiences as a central part of their schooling. Their school-based adviser and internship-based mentor support them in developing real-world projects that promote deeper learning and critical-thinking skills. Such authentic experiences teach “young people to be thinkers, to be curious, to get from curiosity to creation … and it helps students design a learning experience that answers their questions, [providing an] opportunity to communicate it to a larger audience—a major indicator of postsecondary success.” Even in a remote environment, we can design projects that ask more of students than rote memorization and that spark critical thinking.

Our call to action is this: As educators, we need to make opportunities for critical thinking available not only to the affluent or those fortunate enough to be placed in advanced courses. The tools are available, let’s use them. Let’s interrogate our current curriculum and design learning experiences that engage all students in real, relevant, and rigorous experiences that require critical thinking and prepare them for promising postsecondary pathways.

Critical Thinking & Student Engagement

Dr. PJ Caposey is an award-winning educator, keynote speaker, consultant, and author of seven books who currently serves as the superintendent of schools for the award-winning Meridian CUSD 223 in northwest Illinois. You can find PJ on most social-media platforms as MCUSDSupe:

When I start my keynote on student engagement, I invite two people up on stage and give them each five paper balls to shoot at a garbage can also conveniently placed on stage. Contestant One shoots their shot, and the audience gives approval. Four out of 5 is a heckuva score. Then just before Contestant Two shoots, I blindfold them and start moving the garbage can back and forth. I usually try to ensure that they can at least make one of their shots. Nobody is successful in this unfair environment.

I thank them and send them back to their seats and then explain that this little activity was akin to student engagement. While we all know we want student engagement, we are shooting at different targets. More importantly, for teachers, it is near impossible for them to hit a target that is moving and that they cannot see.

Within the world of education and particularly as educational leaders, we have failed to simplify what student engagement looks like, and it is impossible to define or articulate what student engagement looks like if we cannot clearly articulate what critical thinking is and looks like in a classroom. Because, simply, without critical thought, there is no engagement.

The good news here is that critical thought has been defined and placed into taxonomies for decades already. This is not something new and not something that needs to be redefined. I am a Bloom’s person, but there is nothing wrong with DOK or some of the other taxonomies, either. To be precise, I am a huge fan of Daggett’s Rigor and Relevance Framework. I have used that as a core element of my practice for years, and it has shaped who I am as an instructional leader.

So, in order to explain critical thought, a teacher or a leader must familiarize themselves with these tried and true taxonomies. Easy, right? Yes, sort of. The issue is not understanding what critical thought is; it is the ability to integrate it into the classrooms. In order to do so, there are a four key steps every educator must take.

• Integrating critical thought/rigor into a lesson does not happen by chance, it happens by design. Planning for critical thought and engagement is much different from planning for a traditional lesson. In order to plan for kids to think critically, you have to provide a base of knowledge and excellent prompts to allow them to explore their own thinking in order to analyze, evaluate, or synthesize information.
• SIDE NOTE – Bloom’s verbs are a great way to start when writing objectives, but true planning will take you deeper than this.

QUESTIONING

• If the questions and prompts given in a classroom have correct answers or if the teacher ends up answering their own questions, the lesson will lack critical thought and rigor.
• Script five questions forcing higher-order thought prior to every lesson. Experienced teachers may not feel they need this, but it helps to create an effective habit.
• If lessons are rigorous and assessments are not, students will do well on their assessments, and that may not be an accurate representation of the knowledge and skills they have mastered. If lessons are easy and assessments are rigorous, the exact opposite will happen. When deciding to increase critical thought, it must happen in all three phases of the game: planning, instruction, and assessment.

TALK TIME / CONTROL

• To increase rigor, the teacher must DO LESS. This feels counterintuitive but is accurate. Rigorous lessons involving tons of critical thought must allow for students to work on their own, collaborate with peers, and connect their ideas. This cannot happen in a silent room except for the teacher talking. In order to increase rigor, decrease talk time and become comfortable with less control. Asking questions and giving prompts that lead to no true correct answer also means less control. This is a tough ask for some teachers. Explained differently, if you assign one assignment and get 30 very similar products, you have most likely assigned a low-rigor recipe. If you assign one assignment and get multiple varied products, then the students have had a chance to think deeply, and you have successfully integrated critical thought into your classroom.

Thanks to Dara, Patrick, Meg, and PJ for their contributions!

Please feel free to leave a comment with your reactions to the topic or directly to anything that has been said in this post.

Consider contributing a question to be answered in a future post. You can send one to me at [email protected] . When you send it in, let me know if I can use your real name if it’s selected or if you’d prefer remaining anonymous and have a pseudonym in mind.

You can also contact me on Twitter at @Larryferlazzo .

Education Week has published a collection of posts from this blog, along with new material, in an e-book form. It’s titled Classroom Management Q&As: Expert Strategies for Teaching .

Just a reminder; you can subscribe and receive updates from this blog via email (The RSS feed for this blog, and for all Ed Week articles, has been changed by the new redesign—new ones won’t be available until February). And if you missed any of the highlights from the first nine years of this blog, you can see a categorized list below.

• This Year’s Most Popular Q&A Posts
• Race & Racism in Schools
• School Closures & the Coronavirus Crisis
• Classroom-Management Advice
• Best Ways to Begin the School Year
• Best Ways to End the School Year
• Student Motivation & Social-Emotional Learning
• Implementing the Common Core
• Facing Gender Challenges in Education
• Teaching Social Studies
• Cooperative & Collaborative Learning
• Using Tech in the Classroom
• Student Voices
• Parent Engagement in Schools
• Teaching English-Language Learners
• Reading Instruction
• Writing Instruction
• Education Policy Issues
• Differentiating Instruction
• Math Instruction
• Science Instruction
• Advice for New Teachers
• Author Interviews
• Entering the Teaching Profession
• The Inclusive Classroom
• Learning & the Brain
• Administrator Leadership
• Teacher Leadership
• Relationships in Schools
• Professional Development
• Instructional Strategies
• Best of Classroom Q&A
• Professional Collaboration
• Classroom Organization
• Mistakes in Education
• Project-Based Learning

I am also creating a Twitter list including all contributors to this column .

The opinions expressed in Classroom Q&A With Larry Ferlazzo are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Sign Up for EdWeek Update

Edweek top school jobs.

Sign Up & Sign In

• Programs & Services
• Delphi Center

Ideas to Action (i2a)

• What is Critical Thinking?

The ability to think critically calls for a higher-order thinking than simply the ability to recall information.

Definitions of critical thinking, its elements, and its associated activities fill the educational literature of the past forty years. Critical thinking has been described as an ability to question; to acknowledge and test previously held assumptions; to recognize ambiguity; to examine, interpret, evaluate, reason, and reflect; to make informed judgments and decisions; and to clarify, articulate, and justify positions (Hullfish & Smith, 1961; Ennis, 1962; Ruggiero, 1975; Scriven, 1976; Hallet, 1984; Kitchener, 1986; Pascarella & Terenzini, 1991; Mines et al., 1990; Halpern, 1996; Paul & Elder, 2001; Petress, 2004; Holyoak & Morrison, 2005; among others).

After a careful review of the mountainous body of literature defining critical thinking and its elements, UofL has chosen to adopt the language of Michael Scriven and Richard Paul (2003) as a comprehensive, concise operating definition:

Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.

Paul and Scriven go on to suggest that critical thinking is based on: "universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness. It entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-issue, assumptions, concepts, empirical grounding; reasoning leading to conclusions, implication and consequences, objections from alternative viewpoints, and frame of reference. Critical thinking - in being responsive to variable subject matter, issues, and purposes - is incorporated in a family of interwoven modes of thinking, among them: scientific thinking, mathematical thinking, historical thinking, anthropological thinking, economic thinking, moral thinking, and philosophical thinking."

This conceptualization of critical thinking has been refined and developed further by Richard Paul and Linder Elder into the Paul-Elder framework of critical thinking. Currently, this approach is one of the most widely published and cited frameworks in the critical thinking literature. According to the Paul-Elder framework, critical thinking is the:

• Analysis of thinking by focusing on the parts or structures of thinking ("the Elements of Thought")
• Evaluation of thinking by focusing on the quality ("the Universal Intellectual Standards")
• Improvement of thinking by using what you have learned ("the Intellectual Traits")

Selection of a Critical Thinking Framework

The University of Louisville chose the Paul-Elder model of Critical Thinking as the approach to guide our efforts in developing and enhancing our critical thinking curriculum. The Paul-Elder framework was selected based on criteria adapted from the characteristics of a good model of critical thinking developed at Surry Community College. The Paul-Elder critical thinking framework is comprehensive, uses discipline-neutral terminology, is applicable to all disciplines, defines specific cognitive skills including metacognition, and offers high quality resources.

Why the selection of a single critical thinking framework?

The use of a single critical thinking framework is an important aspect of institution-wide critical thinking initiatives (Paul and Nosich, 1993; Paul, 2004). According to this view, critical thinking instruction should not be relegated to one or two disciplines or departments with discipline specific language and conceptualizations. Rather, critical thinking instruction should be explicitly infused in all courses so that critical thinking skills can be developed and reinforced in student learning across the curriculum. The use of a common approach with a common language allows for a central organizer and for the development of critical thinking skill sets in all courses.

• SACS & QEP
• Planning and Implementation
• Why Focus on Critical Thinking?
• Paul-Elder Critical Thinking Framework
• Culminating Undergraduate Experience
• Community Engagement
• Frequently Asked Questions
• What is i2a?

Copyright © 2012 - University of Louisville , Delphi Center

The State of Critical Thinking 2020

November 2020.

Table of Contents

Executive Summary

Introduction Methodology Lessons Learned Conclusion Appendix

Introduction

In 2018, the Reboot Foundation released a first-of-its-kind survey looking at the public’s attitudes toward critical thinking and critical thinking education. The report found that critical thinking skills are highly valued, but not taught or practiced as much as might be hoped for in schools or in public life.

The survey suggested that, despite recognizing the importance of critical thinking, when it came to critical thinking practices—like seeking out multiple sources of information and engaging others with opposing views—many people’s habits were lacking. Significant numbers of respondents reported relying on inadequate sources of information, making decisions without doing enough research, and avoiding those with conflicting viewpoints.

In late 2019, the Foundation conducted a follow up survey in order to see how the landscape may have shifted. Without question, the stakes surrounding better reasoning have increased. The COVID-19 pandemic requires deeper interpretive and analytical skills. For instance, when it comes to news about a possible vaccine, people need to assess how it was developed in order to judge whether it will actually work.

Misinformation, from both foreign and domestic sources, continues to proliferate online and, perhaps most disturbingly, surrounding the COVID-19 health crisis. Meanwhile, political polarization has deepened and become more personal . At the same time, there’s both a growing awareness and divide over issues of racism and inequality. If that wasn’t enough, changes to the journalism industry have weakened local civic life and incentivized clickbait, and sensationalized and siloed content.

Part of the problem is that much of our public discourse takes place online, where cognitive biases can become amplified, and where groupthink and filter bubbles proliferate. Meanwhile, face-to-face conversations—which can dissolve misunderstandings and help us recognize the shared humanity of those we disagree with—go missing. 

Critical thinking is, of course, not a cure-all, but a lack of critical thinking skills across the population exacerbates all these problems. More than ever, we need skills and practice in managing our emotions, stepping back from quick-trigger evaluations and decisions, and over-relying on biased or false sources of information. 

To keep apprised of the public’s view of critical thinking, the Reboot Foundation conducted its second annual survey in late 2019. Unfortunately, the COVID-19 pandemic forced a delay in the release of the results. Nevertheless, this most recent survey dug deeper than our 2018 poll, and looked especially into how the public understands the state of critical thinking education. For the first time, our team also surveyed teachers on their views on teaching critical thinking.

General Findings

Support for critical thinking skills remains high, but there is also clearly skepticism that individuals are getting the help they need to acquire improved reasoning skills.

A very high majority of people surveyed (94 percent) believe that critical thinking is “extremely” or “very important.” But they generally (86 percent) find those skills lacking in the public at large. Indeed, 60 percent of the respondents reported not having studied critical thinking in school. And only about 55 percent reported that their critical thinking skills had improved since high school, with almost a quarter reporting that those skills had deteriorated. 

There is also broad support among the public and teachers for critical thinking education, both at the K-12 and collegiate levels. For example, 90 percent think courses covering critical thinking should be required in K-12. 

Many respondents (43 percent) also encouragingly identified early childhood as the best age to develop critical thinking skills. This was a big increase from our previous survey (just 20 percent) and is consistent with the general consensus among social scientists and psychologists. 

There are worrisome trends—and promising signs—in critical thinking habits and daily practices. In particular, individuals still don’t do enough to engage people with whom they disagree. 

Given the deficits in critical thinking acquisition during school, we would hope that respondents’ critical thinking skills continued to improve after they’ve left school. But only about 55 percent reported that their critical thinking skills had improved since high school, with almost a quarter reporting that their skills had actually deteriorated since then. 

Questions about respondents’ critical thinking habits brought out some encouraging information. People reported using more than one source of information when making a decision at a high rate (around 77 percent said they did this “always” or “often”) and giving reasons for their opinions (85 percent). These numbers were, in general, higher than in our previous survey (see “Comparing Survey Results” below).

In other areas of critical thinking, responses were more mixed. Almost half of respondents, for example, reported only “sometimes,” “rarely,” or “never” seeking out people with different opinions to engage in discussion. Many also reported only “sometimes,” “rarely,” or “never” planning where (35 percent) or how (36 percent) to get information on a given topic. 

These factors are tied closely together. Critical thinking skills have been challenged and devalued at many different levels of society. There is, therefore, no simple fix. Simply cleansing the internet of misinformation, for example, would not suddenly make us better thinkers. Improving critical thinking across society will take a many-pronged effort.

Comparing Survey Results 

Several interesting details emerged in the comparison of results from this survey to our 2018 poll. First, a word of caution: there were some demographic differences in the respondents between the two surveys. This survey skewed a bit older: the average age was 47, as opposed to 36.5. In addition, more females responded this time: 57 percent versus 46 percent.

That said, there was a great deal of consistency between the surveys on participants’ general views of critical thinking. Belief in the importance of critical thinking remains high (94 percent versus 96 percent), as does belief that these skills are generally lacking in society at large. Blame, moreover, was spread to many of the same culprits. Slightly more participants blamed technology this time (29 versus 27 percent), while slightly fewer blamed the education system (22 versus 26 percent). 

Respondents were also generally agreed on the importance of teaching critical thinking at all levels. Ninety-five percent thought critical thinking courses should be required at the K-12 level (slightly up from 92 percent); and 91 percent thought they should be required in college (slightly up from 90 percent). (These questions were framed slightly differently from year to year, which could have contributed to the small increases.)

One significant change came over the question of when it is appropriate to start developing critical thinking skills. In our first survey, less than 20 percent of respondents said that early childhood was the ideal time to develop critical thinking skills. This time, 43 percent of respondents did so. As discussed below, this is an encouraging development since research indicates that children become capable of learning how to think critically at a young age. 

In one potentially discouraging difference between the two surveys, our most recent survey saw more respondents indicate that they did less critical thinking since high school (18 percent versus just 4 percent). But similar numbers of respondents indicated their critical thinking skills had deteriorated since high school (23 percent versus 21 percent).

Finally, encouraging points of comparison emerged in responses to questions about particular critical thinking activities. Our most recent survey saw a slight uptick in the number of respondents reporting engagement in activities like collaborating with others, planning on where to get information, seeking out the opinions of those they disagree with, keeping an open mind, and verifying information. (See Appendix 1: Data Tables.)

These results could reflect genuine differences from 2018, in either actual activity or respondents’ sense of the importance of these activities. But demographic differences in age and gender could also be responsible. 

There is reason to believe, however, that demographic differences are not the main factor, since there is no evident correlation between gender and responses in either survey. Meanwhile, in our most recent survey older respondents reported doing these activities less frequently . Since this survey skewed older, it might have been anticipated that respondents would report doing these activities less. But the opposite is the case.

Findings From Teacher Survey

Teachers generally agree with general survey respondents about the importance of critical thinking. Ninety-four percent regard critical thinking as “extremely” or “very important.” 

Teachers, like general survey participants, also share concerns that young people aren’t acquiring the critical thinking skills they need. They worry, in particular, about the impact of technology on their students’ critical thinking skills. In response to a question about how their school’s administration can help them teach critical thinking education more effectively, some teachers said updated technology (along with new textbooks and other materials) would help, but others thought laptops, tablets, and smartphones were inhibiting students’ critical thinking development. 

This is an important point to clarify if we are to better integrate critical thinking into K-12 education. Research strongly suggests that critical thinking skills are best acquired in combination with basic facts in a particular subject area. The idea that critical thinking is a skill that can be effectively taught in isolation from basic facts is mistaken.

Another common misconception reflected in the teacher survey involves critical thinking and achievement. Although a majority of teachers (52 percent) thought all students benefited from critical thinking instruction, a significant percentage (35) said it primarily benefited high-ability students.

At Reboot, we believe that all students are capable of critical thinking and will benefit from critical thinking instruction. Critical thinking is, after all, just a refinement of everyday thinking, decision-making, and problem-solving. These are skills all students must have. The key is instilling in our young people both the habits and subject-area knowledge needed to facilitate the improvement and refinement of these skills.

Teachers need more support when it comes to critical thinking instruction. In the survey, educators repeatedly mentioned a lack of resources and updated professional development. In response to a question about how administrators could help teachers teach critical thinking more effectively, one teacher asked for “better tools and materials for teaching us how to teach these things.”

Others wanted more training, asking directly for additional support in terms of resources and professional training. One educator put it bluntly: “Provide extra professional development to give resources and training on how to do this in multiple disciplines.”

Media literacy is still not being taught as widely as it should be. Forty-four percent of teachers reported that media literacy courses are not offered at their schools, with just 31 percent reporting required media literacy courses.

This is despite the fact that teachers, in their open responses, recognized the importance of media literacy, with some suggesting it should be a graduation requirement. Many organizations and some governments, notably   Finland’s , have recognized the media literacy deficit and taken action to address it, but the U.S. education system has been slow to act.

Thinking skills have been valuable in all places and at all times. But with the recent upheavals in communication, information, and media, particularly around the COVID-19 crisis, such skills are perhaps more important than ever. 

Part of the issue is that the production of information has been democratized—no longer vetted by gatekeepers but generated by anyone who has an internet connection and something to say. This has undoubtedly had positive effects, as events and voices come to light that might have previously not emerged. The recording of George Floyd’s killing is one such example. But, at the same time, finding and verifying good information has become much more difficult. 

Technological changes have also put financial pressures on so-called “legacy media” like newspapers and television stations, leading to sometimes precipitous drops in quality, less rigorous fact-checking (in the original sense of the term), and the blending of news reports and opinion pieces. The success of internet articles and videos is too often measured by clicks instead of quality. A stable business model for high-quality public interest journalism remains lacking. And, as biased information and propaganda fills gaps left by shrinking newsrooms, polarization worsens. (1)

Traditional and social media both play into our biases and needs for in-group approval. Online platforms have proven ideal venues for misinformation and manipulation. And distractions abound, damaging attention spans and the quality of debate.

Many hold this digital upheaval at least partially responsible for recent political upheavals around the world. Our media consumption habits increasingly reinforce biases and previously held beliefs, and expose us to only the worst and most inflammatory views from the other side. Demagogues and the simple, emotion-driven ideas they advance thrive in this environment of confusion, isolation, and sensationalism. 

It’s not only our public discourse that suffers. Some studies have suggested that digital media may be partially responsible for rising rates of depression and other mood disorders among the young. (2)

Coping with this fast-paced, distraction-filled world in a healthy and productive manner requires better thinking and better habits of mind, but the online world itself tends to encourage the opposite. This is not to suggest our collective thinking skills were pristine before the internet came along, only that the internet presents challenges to our thinking that we have not seen before and have not yet proven able to meet.

There are some positive signs, with more attention and resources being devoted to neglected areas of education like civics and media literacy ; organizations trying to address internet-fueled polarization and extremism; and online tools being developed to counter fake news and flawed information.

But we also need to support the development of more general reasoning skills and habits: in other words, “critical thinking.”

Critical thinking has long been a staple of K-12 and college education, theoretically, at least, if not always in practice. But the concept can easily appear vague and merely rhetorical without definite ideas and practices attached to it.

When, for example, is the best age to teach critical thinking? What activities are appropriate? Should basic knowledge be acquired at the same time as critical thinking skills, or separately? Some of these questions remain difficult to answer, but research and practice have gone far in addressing others.

Part of the goal of our survey was to compare general attitudes about critical thinking education—both in the teaching profession and the general public—to what the best and most recent research suggests. If there is to be progress in the development of critical thinking skills across society, it requires not just learning how best to teach critical thinking but diffusing that knowledge widely, especially to parents and educators.

The surveys were distributed through Amazon’s MTurk Prime service. 

For the general survey, respondents answered a series of questions about critical thinking, followed by a section that asked respondents to estimate how often they do certain things, such as consult more than one source when searching for information. The questions in the “personal habit” section appeared in a randomized order to reduce question ordering effects. Demographic questions appeared at the end of the survey.

For the teacher survey, respondents were all part of a teacher panel created by MTurk Prime. They also answered a series of questions on critical thinking, especially focused on the role of critical thinking in their classrooms. After that, respondents answered a series of questions about how they teach—these questions were also randomized to reduce question ordering effects. Finally, we asked questions related to the role of media literacy in their classrooms.

To maintain consistency with the prior survey and to explore relationships across time, many of the questions remained the same from 2018. In some cases, following best practices in questionnaire design , we revamped questions to improve clarity and increase the validity and reliability of the responses.

For all surveys, only completed responses coming from IP addresses located in the U.S. were analyzed. 1152 respondents completed the general survey; 499 teachers completed the teacher survey.

The complete set of questions for each survey is available upon request

Detailed Findings and Discussion

As summarized above, the survey produced a number of noteworthy findings. One central theme that emerged was a general pessimism about the state of critical thinking and uncertainty about how to improve it. That is, despite the near-universal acknowledgment of the importance of critical thinking, respondents generally think society at large is doing a bad job of cultivating critical thinking skills. Respondents were, moreover, divided about what needs to be done.

Almost all the people surveyed (94 percent) believe that critical thinking is “extremely” or “very important.” But they generally (86 percent) find those skills lacking in the public at large. These numbers don’t come as a huge surprise—and they echo the 2018 results—but they do suggest broad public support for initiatives that advance critical thinking skills, both inside and outside of schools.

Respondents also reported deficits in their own critical thinking training and practices. They tended not to think critical thinking had been a point of emphasis in their own education, with a substantial majority of over 63 percent reporting that they had not studied critical thinking in school. Around 20 percent said their schools had provided no background in critical thinking at all, and another 20 percent said the background in critical thinking they gained from school was only slight.

There were significant differences among age groups in these self-reports. Around half of respondents in both the 0-19 and 20-39 age groups reported having studied critical thinking in school. Those numbers dwindled among older groups, bottoming out at 11 percent among 80 to 100-year-olds.

This result is likely in part due to the increased popularity of the phrase “critical thinking”: prior generations may have spent a substantial amount of time on reasoning skills without it coming under the same vocabulary. The young are also closer to school-age, of course, so may simply have sharper memories of critical thinking activities. But the differences in responses might also reflect genuine differences in education.

In any case it’s clear that, even recently, many—if not most—students come out of school feeling as if they have not learned how to think critically, despite the fact that there is broad consensus on the importance of these skills. Only around 25 percent of respondents reported receiving an “extremely” or “very” strong background in critical thinking from their schools.

There are a number of potential causes—technology, social norms, misguided educational priorities—but perhaps the most salient is that, as cognitive scientist Tim van Gelder puts it, “critical thinking is hard.” As van Gelder emphasizes, we don’t naturally think reasonably and rationally; instead we tend to rely on narrative, emotion, and intuition—what feels right. (3)   Teaching students to think critically requires much more guidance and practice, throughout the curriculum, than is currently being provided.

There is broad support among the public and among teachers for critical thinking education, both at the K-12 and collegiate levels. 

Around 90 percent of respondents in the general public said that courses covering critical thinking should be required at the K-12 level, while 94 percent of teachers said critical thinking is important.

And schools usually echo this sentiment as well, citing the phrase “critical thinking” frequently in curricula and other materials. But it remains unclear if, in practice, critical thinking is really the priority it’s made out to be rhetorically.

One problem is a tendency to think critical thinking and reasoning are too complex for younger students to tackle. But research has shown that children start reasoning logically at a very young age. (4)   Critical thinking through activities like open-ended dialogue, weighing opposing perspectives, and backing up opinions with reasoning can have a positive effect even at the K-5 level. For example, philosophy for kids courses have shown some  positive effects on students’ reading and math skills (gains were even more substantial for disadvantaged students). (5)

Our survey respondents generally agreed that critical thinking skills should be taught from an early age. Forty-three percent favored beginning critical thinking instruction during early childhood (another 27 percent favored beginning at ages 6-12). This was more than a twofold increase over the results from 2018’s survey, in which just 20 percent thought it was best to begin instruction in critical thinking before the age of 6. This increase is encouraging since it’s consistent with recent research that understands critical thinking as part of general cognitive development that starts even before children enter school. (6)

Many teachers likewise support critical thinking instruction beginning at a young age. In the open response, for example, one wrote, “Critical thinking should be explicitly taught in earlier grades than late middle school and high school.”

Another wrote: “By the time students get to high school they should have this skill [critical thinking] well tuned. The pressure to meet standards earlier and earlier makes it harder to teach basic skills like critical thinking.” 

Many teachers (55 percent) also thought the emphasis on standardized testing has made it more difficult to incorporate critical thinking instruction in the classroom. For example, one wrote, “Standardized testing has created an environment of quantitative results that don’t always represent qualitative gains.” 

Moreover, a plurality of teachers (25 percent) believe that state standardized tests do not assess critical thinking skills well at all, while just 13 percent believe they assess critical thinking skills extremely well. Teachers generally (52 percent) believe that their own tests do a better job of measuring critical thinking skills.

The survey also found some worrisome trends—as well as some promising signs—in how people evaluated their own critical thinking skills and daily practices. In particular, individuals don’t do enough to engage people with whom they disagree. 

Given the deficits in critical thinking acquisition during school, it might be hoped that respondents’ critical thinking skills continued to improve after they’ve left school. But only about 55 percent reported that their critical thinking skills had improved since high school, with almost a quarter reporting that their skills had actually deteriorated since then. 

This is especially alarming because thinking critically, unlike say learning about calculus or the Russian Revolution, is generally thought to be a lifelong endeavour. We are supposed to become better with age and experience. Research into adult education suggests that it’s never too late to make gains in critical thinking.  (7)

Questions about respondents’ critical thinking habits brought out more detailed information. Some of these responses were encouraging. People reported using more than one source of information when making a decision at a high rate (around 77 percent said they did this “always” or “often”), giving reason for their opinions (85 percent), supporting their decisions with information (84 percent), and listening to the ideas of those they disagree with (81 percent). Participants generally reported engaging in more critical thinking activities this time than in our initial survey. (See “Comparing Survey Results” above.)

In other areas of critical thinking, responses were more mixed. Almost half of respondents, for example, reported only “sometimes,” “rarely,” or “never” seeking out people with different opinions to engage in discussion. Many also reported only “sometimes,” “rarely,” or “never” planning where (35 percent) or how (36 percent) to get information on a given topic.

It’s difficult to totally identify the drivers of these figures. After all, all humans are prone to overestimating the amount and quality of reasoning we do when we come to decisions, solve problems, or research information. But, at the very least, these numbers indicate that people acknowledge that these various critical thinking habits are admirable goals to shoot for. 

At the same time and unsurprisingly, these results suggest a reluctance to engage in the more demanding aspects of critical thinking: difficult or unpleasant tasks like seriously considering the possibility that our opponents might be right or thinking carefully about how to approach information-gathering before we engage in it.

Weaknesses in these areas of critical thinking can be especially easily exploited by emotionalized, oversimplified, and sensationalistic news and rhetoric. If people jump in to information-gathering without even a rough plan or method in mind they’re more likely to get swept up by clickbait or worse.

The current media environment requires a mindful and deliberate approach if it is to be navigated successfully. And one’s own opinions will remain under-nuanced, reactive, and prone to groupthink if they’re influenced by the extreme opinions and caricatures that are often found online and on television instead of by engagement with well-reasoned and well-intentioned perspectives.

Poor media consumption habits can have a distorting effect on our political perceptions, especially. Recent research, for example, has identified wildly inaccurate stereotypes among the general public about the composition of political parties. One study found that “people think that 32% of Democrats are LGBT (versus 6% in reality) and 38% of Republicans earn over $250,000 per year (vs. 2% in reality).” (8) The study also suggested, alarmingly, that “those who pay the most attention to political media may […] also [be] the likeliest to possess the most misinformation about party composition.” (9)

The public is worried about the impact of technology on the acquisition of critical thinking skills. They also blamed deficits in critical thinking on changing societal norms and the education system.

Modern technology was the most cited reason for a lack of critical thinking skills among the general public, with “changing societal norms” coming in a close second. Over 200 respondents also cited the educational system (see chart below).

A number of the teachers also mentioned potential drawbacks of technology in the classroom environment. For example, in the open response portion of the survey, which allowed teachers to voice general concerns, one teacher wrote: “Get rid of the laptops and tablets and bring back pencil and paper because the students aren’t learning anything using technology.” Another said: “Personal Electronic devices need to be banned in schools.”

In our own work at the Reboot Foundation, the research team found evidence of negative correlations between technology use at schools and achievement. For example, an analysis of data from the National Assessment of Education Progress (NAEP) showed that fourth graders using tablets “in all or almost all” classes performed significantly worse (the equivalent of a full grade level) than their peers who didn’t use them.

Another recent study the foundation supported also suggested students benefited from using pencil and paper as opposed to technology to do math homework. The Organization for Economic Cooperation and Development found similar results a few years ago in their international study of 15-year-olds and computer usage. (10)

There is a great deal the field still doesn’t know about the effects of different kinds of technology on different kinds of learning. But a growing stock of research suggests that schools should be cautious about introducing technology into classrooms and the lives of students in general, especially young students. (11)

It would also be a mistake to slip into simple Luddism though. Technology, obviously, provides benefits as well—making education more accessible, reducing costs, helping teachers to fine-tune instruction to student needs, to name a few. During the coronavirus crisis, moreover, educators have had no choice but to rely and hopefully help improve these tools.

Still, too often in the past schools have turned to technology without properly weighing the costs against the benefits, and without determining whether technology is truly needed or effective. A recent RAND Corporation paper, for example, discussed programs “seeking to implement personalized learning” but without “clearly defined evidence-based models to adopt.” (12)

The Reboot survey suggests that members of the public as well as teachers generally share these concerns, both about educational technology specifically and about the general impact of technology on student learning.

While teachers support critical thinking instruction, they are divided about how to teach it, and some educators have beliefs about critical thinking instruction that conflict with established research.

One central question in the research about how to best instill critical thinking skills in students is whether critical thinking should be taught in conjunction with basic facts and knowledge or separated from it.

Teachers were split on this question, with 41 percent thinking students should engage in critical thinking practice while learning basic facts, while 42 percent thought students should learn basic facts first then engage in critical thinking practice. A further 16 percent believe that basic facts and critical thinking should be taught separately. (However, only about 13 percent of teachers surveyed say that content knowledge either doesn’t matter at all or only matters slightly for critical thinking skills.)

The view that knowledge and critical thinking skills can and should be taught separately is mistaken. There is a common view that since information is so widely accessible today, learning basic facts is no longer important. According to this view, it’s only cognitive skills that matter. But the two cannot be so neatly divorced as is often assumed. (13)

Research in cognitive science strongly suggests that critical thinking is not the type of skill that can be divorced from content and applied generically to all kinds of different contexts. As cognitive scientist Daniel T. Willingham argues, “The ability to think critically […] depends on domain knowledge and practice.” (14)

This means students need to practice critical thinking in many different kinds of contexts throughout the curriculum as they acquire the background knowledge needed to reason in a given context. There are of course general skills and habits that can be extrapolated from these various kinds of practice, but it is very unlikely that critical thinking can be taught as a skill divorced from content. “It […] makes no sense,” Willingham writes, “to try to teach critical thinking devoid of factual content.”

This doesn’t necessarily mean standalone critical thinking courses should be rejected. Students can still gain a lot from learning about formal logic, for example, and from learning about metacognition and the best research practices. But these standalone courses or programs should include acquisition of basic factual knowledge as well, and the skills and habits learned in them must be applied and reinforced in other courses and contexts.

Students, moreover, should be reminded that being “critical” is an empty slogan unless they have the requisite factual knowledge to make a cogent argument in a given domain. They need background knowledge to be able to seek out evidence from relevant sources, to develop reliable and nuanced interpretations of information, and to back the arguments they want to make with evidence.

Reboot also asked teachers about which students they thought benefited from critical thinking instruction. A majority (52 percent) thought it benefits all students, but 35 percent said (with the remaining 13 percent thinking it primarily benefits lower-ability students).

The view that critical thinking instruction is only effective for higher achieving students is another common misconception. Everyone is capable of critical thinking, and even, to a certain extent, engages in critical thinking on their own. The key is for students to develop metacognitive habits and subject-area knowledge so that they can apply critical thought in the right contexts and in the right way. Educators should not assume that lower-achieving students will not benefit from critical thinking instruction.

Teachers need more support when it comes to critical thinking instruction, though at least some teacher training and professional development programs do seem to help.

In the survey, educators repeatedly mentioned a lack of resources and updated professional development. In response to a question about how administrators could help teachers teach critical thinking more effectively, one teacher asked for “better tools and materials for teaching us how to teach these things.”

Another said, “Provide opportunities for teachers to collaborate and cross train across subject areas, as well as providing professional development that is not dry or outdated.” Another characteristic comment: “Provide extra professional development to give resources and training on how to do this in multiple disciplines.”

Overall teachers were relatively satisfied that teacher training and professional development programs were helping them teach critical thinking. Forty-six percent said that their teacher training helped them a lot or a great deal, while 50 percent said professional development programs help them a lot or a great deal.

But other teachers reported burdensome administrative tasks and guidelines were getting in the way of teacher autonomy and critical thinking instruction. For example, one teacher wrote, “Earlier in my career I had much more freedom to incorporate instruction of critical thinking into my lessons.”

Media literacy is still not being taught as widely as it should be. 

In our survey, teachers rightly recognized that media literacy is closely bound up with critical thinking. One said, “I believe that media literacy goes hand in hand with critical thinking skills and should be a requirement […] especially due to the increase in use of technology among our youth.” Another offered that “media literacy should be a graduation requirement like economics or government.”

But schools, at least judging by teachers’ responses in the survey, have been slow in prioritizing media literacy. More than 44 percent reported that media literacy courses are not offered at their schools, and just around 30 percent reported that media literacy courses are required. That said, the majority of teachers did report teaching typical media literacy skills occasionally in their classes.

For example, over 60 percent said that, in at least one class, they “teach students how to distinguish legitimate from illegitimate sources,” and over two-thirds said they “teach students how to find reliable sources.” (15)

Despite the assumption sometimes made that young people (“digital natives”) must be adept navigators of the internet, recent studies have found that students have trouble evaluating the information they consume online. They have problems recognizing bias and misinformation, distinguishing between advertising and legitimate journalism, and verifying information using credible sources.

Our age is one in which unreliable information proliferates; nefarious interests use the internet to influence public opinion; and social media encourages groupthink, emotional thinking, and pile-on. New skills and training are required to navigate this environment. Our schools must adapt.

This means generating and implementing specific interventions that help students learn to identify markers of misinformation and develop healthy information-gathering habits. The Reboot Foundation’s own research suggests that even quick and immediate interventions can have a positive impact. But it also means instilling students with life-long critical thinking habits and skills which they’ll be able to apply to an ever-changing media landscape.

Despite its importance, which is widely acknowledged by the general public, critical thinking remains a somewhat vague and poorly understood concept. Most people realize that it is of vital importance to individual success and educational attainment, as well as to civic life in a liberal democracy. And most seem to realize that 21st-century challenges and changes make acquiring critical thinking skills of even more urgent importance. But when it comes to instilling them in children and developing them in adults, we are, in many ways, still at square one. 

Over the course of the last few decades, K-12 educators have been urged to teach critical thinking, but they have been given conflicting and inconsistent advice on how to do it. There remains a lack of proven resources for them to rely on, a lack of administrative support—and sometimes even a lack of a clear sense of what exactly critical thinking is. Perhaps most importantly, teachers lack the time and freedom within the curriculum to teach these skills.

But there have been a number of insights from cognitive science and other disciplines that suggest a way forward. Perhaps the most important is that critical thinking cannot be understood as a skill on par with learning a musical instrument or a foreign language. It is more complicated than those kinds of skills, involving cognitive development in a number of different areas and integrated with general knowledge learned in other subject areas. Critical thinking courses and interventions that ignore this basic fact may produce some gains, but they will not give students the tools to develop their thinking more broadly and apply critical thought to the world outside of school.

College and continuing education deserve attention too. It should be considered a red flag that only 55 percent of respondents didn’t think they’d made any strides in critical thinking skills since high school. Colleges have long been moving away from a traditional liberal arts curriculum . The critical thinking skills acquired across those disciplines have likely suffered as a result.

In recent years, we’ve seen smart people who should know better time and again exhibit poor judgment online. It is important to remind each other of the importance of stepping back, managing emotions, engaging with others charitably, and seriously considering the possibility that we are wrong. This is especially important when we are searching for information online, an environment that can easily discourage these intellectual virtues. Ramping up media literacy—for both adults and young people—will be a vital part of the solution.

But, ultimately, critical thinking, which touches on so many different aspects of personal and civic life, must be fostered in a multitude of different ways and different domains. A secure, prosperous, and civil future may, quite literally, depend on it.

Appendix 1: Data Tables

[table id=72 /]

[table id=73 /]

[table id=74/]

[table id=75 /]

[table id=76 /]

To download the PDF of this report,

( please click here )

Table of Contents Introduction General Findings Background Methods Detailed Findings and Discussion Conclusion

(1)* W Gandour, R. (2016) A new information environment: How digital fragmentation is shaping the way we produce and consume news. Knight Center for Journalism in the Americas. https://knightcenter.utexas.edu/books/NewInfoEnvironmentEnglishLink.pdf

(2)* Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. Journal of Abnormal Psychology .

(3)*  Gelder, T. V. (2005). Teaching critical thinking: Some lessons from cognitive science. College Teaching , 53 (1), 41-48.

(4)*  Gelman, S. A., & Markman, E. M. (1986). Categories and induction in young children. Cognition, 23 , 183-209.

(5)*  Gorard, S., Siddiqui, N., & See, B. H. (2015). Philosophy for Children: Evaluation report and executive summary. Education Endowment Foundation. https://educationendowmentfoundation.org.uk/public/files/ Projects/Evaluation_Reports/EEF_Project_Report_PhilosophyForChildren.pdf

(6)*  Kuhn, D. (1999). A developmental model of critical thinking. Educational researcher , 28 (2), 16-46.

(7)*  Dwyer, C. P., & Walsh, A. (2019). An exploratory quantitative case study of critical thinking development through adult distance learning. Educational Technology Research and Development, 1-19.

(8)*  Ahler, D. J., & Sood, G. (2018). The parties in our heads: Misperceptions about party composition and their consequences. The Journal of Politics, 80 (3), 964-981. 964.

(9)*  Ibid., 965.

(10)*  Organization for Economic Cooperation and Development. (2015). Students, computers and learning: Making the connection . https://doi.org/10.1787/9789264239555-en

(11)*  Madigan, S., Browne, D., Racine, N., Mori, C., & Tough, S. (2019). Association between screen time and children’s performance on a developmental screening test. JAMA pediatrics, 173(3), 244-250.

(12)*  Pane, J. F. (2018). Strategies for implementing personalized learning while evidence and resources are underdeveloped. RAND Corporation. https://www.rand.org/pubs/perspectives/PE314.html

(13)*  Wexler, N. (2019). The knowledge gap: The hidden cause of America’s broken education system–and how to fix it. Avery.

(14)*  Willingham, D. T. (2007). Critical thinking: Why is it so hard to teach? American Federation of Teachers (Summer 2007) 8-19.

(15)*  Wineburg, S., McGrew, S., Breakstone, J., & Ortega, T. (2016). Evaluating information: The cornerstone of civic online reasoning. Stanford Digital Repository, 8, 2018.

To download the PDF of this survey,

please click here.

Free Critical Thinking Resources​

Subscribe to get updates and news about critical thinking, and links to free resources.

Reboot Foundation, 88 Rue De Courcelles, Paris, France 75008

[email protected], ⓒ 2024 - all rights are reserved, privacy overview.

Our mission is to develop tools and resources to help people cultivate a capacity for critical thinking, media literacy, and reflective thought.

Translate this page from English...

*Machine translated pages not guaranteed for accuracy. Click Here for our professional translations.

Defining Critical Thinking

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PMC10054602

Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education

Branden thornhill-miller.

1 Faculty of Philosophy, University of Oxford, Oxford OX2 6GG, UK

2 International Institute for Competency Development, 75001 Paris, France

Anaëlle Camarda

3 LaPEA, Université Paris Cité and Univ Gustave Eiffel, 92100 Boulogne-Billancourt, France

4 Institut Supérieur Maria Montessori, 94130 Nogent-Sur-Marne, France

Maxence Mercier

Jean-marie burkhardt.

5 LaPEA, Univ Gustave Eiffel and Université Paris Cité, CEDEX, 78008 Versailles, France

Tiffany Morisseau

6 Strane Innovation, 91190 Gif-sur-Yvette, France

Samira Bourgeois-Bougrine

Florent vinchon, stephanie el hayek.

7 AFNOR International, 93210 Saint-Denis, France

Myriam Augereau-Landais

Florence mourey, cyrille feybesse.

8 Centre Hospitalier Guillaume Regnier, Université de Rennes 1, 35200 Rennes, France

Daniel Sundquist

Todd lubart, associated data.

Not Applicable.

This article addresses educational challenges posed by the future of work, examining “21st century skills”, their conception, assessment, and valorization. It focuses in particular on key soft skill competencies known as the “4Cs”: creativity, critical thinking, collaboration, and communication. In a section on each C, we provide an overview of assessment at the level of individual performance, before focusing on the less common assessment of systemic support for the development of the 4Cs that can be measured at the institutional level (i.e., in schools, universities, professional training programs, etc.). We then present the process of official assessment and certification known as “labelization”, suggesting it as a solution both for establishing a publicly trusted assessment of the 4Cs and for promoting their cultural valorization. Next, two variations of the “International Institute for Competency Development’s 21st Century Skills Framework” are presented. The first of these comprehensive systems allows for the assessment and labelization of the extent to which development of the 4Cs is supported by a formal educational program or institution. The second assesses informal educational or training experiences, such as playing a game. We discuss the overlap between the 4Cs and the challenges of teaching and institutionalizing them, both of which may be assisted by adopting a dynamic interactionist model of the 4Cs—playfully entitled “Crea-Critical-Collab-ication”—for pedagogical and policy-promotion purposes. We conclude by briefly discussing opportunities presented by future research and new technologies such as artificial intelligence and virtual reality.

1. Introduction

There are many ways of describing the massive educational challenges faced in the 21st century. With the appearance of computers and digital technologies, new means of interacting between people, and a growing competitiveness on the international level, organizations are now requiring new skills from their employees, leaving educational systems struggling to provide appropriate ongoing training. Indeed, according to the World Economic Forum’s 2020 “Future of Jobs Report”, studying 15 industries in 26 advanced and emerging countries, up to 50% of employees will need some degree of “reskilling” by 2025 ( World Economic Forum 2020 ). Although many national and international educational efforts and institutions now explicitly put the cultivation of new kinds of skills on their educational agendas, practical means of assessing such skills remains underdeveloped, thus hampering the valorization of these skills and the development of guidance for relevant pedagogy ( Care et al. 2018 ; Vincent-Lancrin et al. 2019 ; for overviews and discussion of higher education in global developmental context, see Blessinger and Anchan 2015 ; Salmi 2017 ).

This article addresses some of these challenges and related issues for the future of education and work, by focusing on so-called “21st Century Skills” and key “soft skills” known as the “4Cs” (creativity, critical thinking, communication, and collaboration), more particularly. It begins with a brief discussion of these skills, outlining their conceptual locations and potential roles in the modern educational context. A section on each “C” then follows, defining the C, summarizing research and methods for its scientific assessment at the individual level, and then outlining some means and avenues at the systemic level for fostering its development (e.g., important aspects of curriculum, institutional structure, or of the general environment, as well as pedagogical methods) that might be leveraged by an institution or program in order to promote the development of that C among its students/trainees. In the next section, the certification-like process of “labelization” is outlined and proposed as one of the best available solutions both for valorizing the 4Cs and moving them towards the center of the modern educational enterprise, as well as for benchmarking and monitoring institutions’ progress in fostering their development. The International Institute for Competency Development’s 4Cs Framework is then outlined as an example of such a comprehensive system for assessing and labelizing the extent to which educational institutions and programs support the development of the 4Cs. We further demonstrate the possibility of labelizing and promoting support for the development of the 4Cs by activities or within less formal educational settings, presenting a second framework for assessment of the 4Cs in games and similar training activities. Our discussion section begins with the challenges to implementing educational change in the direction of 21st century skills, focusing on the complex and overlapping nature of the 4Cs. Here, we propose that promoting a “Dynamic Interactionist Model of the 4Cs” not only justifies grouping them together, but it might also assist more directly with some of the challenges of pedagogy, assessment, policy promotion, and ultimately, institutionalization, faced by the 4Cs and related efforts to modernize education. We conclude by suggesting some important future work for the 4Cs individually and also as an interrelated collective of vital skills for the future of education and work.

“21st Century Skills”, “Soft Skills”, and the “4Cs”

For 40 years, so-called “21st century skills” have been promoted as those necessary for success in a modern work environment that the US Army War College ( Barber 1992 ) has accurately described as increasingly “VUCA”—“volatile, uncertain, complex and ambiguous”. Various lists of skills and competencies have been formulated on their own or as part of comprehensive overarching educational frameworks. Although a detailed overview of this background material is outside the scope of this article (see Lamri et al. 2022 ; Lucas 2022 for summaries), one of the first prominent examples of this trend was the Partnership for 21st Century Skills (P21), whose comprehensive “Framework for 21st Century Learning” is presented in Figure 1 ( Battelle for Kids 2022 ). This framework for future-oriented education originated the idea of the “4Cs”, placing them at its center and apex as “Learning and Innovation Skills” that are in need of much broader institutional support at the foundational level in the form of new standards and assessments, curriculum and instructional development, ongoing professional development, and appropriately improved learning environments ( Partnership for 21st Century Skills 2008 ). These points are also consistent with the approach and assessment frameworks presented later in this article.

The P21 Framework for 21st Century Learning. (© 2019, Battelle for Kids. All Rights Reserved. https://www.battelleforkids.org/ ; accessed on 17 January 2023).

Other important organizations such as the World Economic Forum ( 2015 ) have produced similar overarching models of “21st century skills’’ with the 4Cs at their center, but the term “21st century skills’’ has been rightly criticized for a several reasons: the skills referred to are not actually all unique to, or uniquely important to, the 21st century, and it is a term that is often used more as an advertising or promotional label for systems that sometimes conflate and confuse different kinds of skills with other concepts that users lump together ( Lucas 2019 ). Indeed, though there is no absolute consensus on the definition of a “skill”, they are often described as being multidimensional and involve the ability to solve problems in context and to perform tasks using appropriate resources at the right time and in the right combination ( Lamri and Lubart 2021 ). At its simplest, a skill is a “learned capacity to do something useful” ( Lucas and Claxton 2009 ), or an ability to perform a given task at a specified performance level, which develops through practice, experience. and training ( Lamri et al. 2022 ).

The idea of what skills “are’’, however, has also evolved to some extent over time in parallel to the nature of the abilities required to make valued contributions to society. The digital and information age, in particular, has seen the replacement by machines of much traditional work sometimes referred to as “hard skills’’—skills such as numerical calculation or driving, budget-formulating, or copyediting abilities, which entail mastery of fixed sets of knowledge and know-how of standard procedures, and which are often learned on the job. Such skills are more routine, machine-related, or technically oriented and not as likely to be centered on human interaction. In contrast, the work that has been increasingly valued in the 21st century involves the more complex, human interactive, and/or non-routine skills that Whitmore ( 1972 ) first referred to as “soft skills”.

Unfortunately, researchers, educators, and consultants have defined, redefined, regrouped, and expanded soft skills—sometimes labeling them “transversal competencies”, “generic competencies”, or even “life skills” in addition to “21st century skills”—in so many different ways within and across different domains of research and education (as well as languages and national educational systems) that much progress towards these goals has literally been “lost in translation” ( Cinque 2016 ).

Indeed, there is also a long-standing ambiguity and confusion between the terms “competency” (also competence) and “skill” due to their use across different domains (e.g., learning research, education, vocational training, personnel selection) as well as different epistemological backgrounds and cultural specificities ( Drisko 2014 ; Winterton et al. 2006 ; van Klink and Boon 2003 ). The term “competency” is, however, often used as a broader concept that encompasses skills, abilities, and attitudes, whereas, in a narrower sense, the term “skill” has been defined as “goal-directed, well-organized behavior that is acquired through practice and performed with economy of effort” ( Proctor and Dutta 1995, p. 18 ). For example, whereas the command of a spoken language or the ability to write are skills (hard skills, to be precise), the ability to communicate effectively is a competence that may draw on an individual’s knowledge of language, writing skills, practical IT skills, and emotional intelligence, as well as attitudes towards those with whom one is communicating ( Rychen and Hersch 2003 ). Providing high-quality customer service is a competency that relies on listening skills, social perception skills, and contextual knowledge of products. Beyond these potential distinctions, the term “competency” is predominant in Europe, whereas “skill” is more commonly used in the US. Yet it also frequently occurs that both are used as rough synonyms. For example, Voogt and Roblin ( 2012, p. 299 ) examine the “21st century competences and the recommended strategies for the implementation of these skills”, and Graesser et al. ( 2022, p. 568 ) state that twenty-first-century skills “include self-regulated learning, collaborative problem solving, communication (…) and other competencies”. In conclusion, the term “competencies” is often used interchangeably with “skills” (and can have a particularly large overlap with “soft skills”), but it is also often considered in a broader sense as a set of skills, knowledge, and attitudes that, together, meet a complex demand ( Ananiadoui and Claro 2009 ). From this perspective, one could argue that the 4Cs, as complex, “higher-order” soft skills, might best be labeled competencies. For ease and convenience, however, in this text, we consider the two terms interchangeable but favor the term “skills”, only using “competency” in some instances to avoid cumbersome repetition.

Even having defined soft skills as a potentially more narrow and manageable focus, we are still aware of no large-scale study that has employed a comprehensive enough range of actual psychometric measures of soft skills in a manner that might help produce a definitive empirical taxonomy. Some more recent taxonomic efforts have, however, attempted to provide additional empirical grounding for the accurate identification of key soft skills (see e.g., Joie-La Marle et al. 2022 ). Further, recent research by JobTeaser (see Lamri et al. 2022 ) surveying a large, diverse sample of young workers about a comprehensive, systematic list of soft skills as actually used in their professional roles represents a good step towards some clarification and mapping of this domain on an empirical basis. Despite the fact that both these studies necessarily involved assumptions and interpretive grouping of variables, the presence and importance of the 4Cs as higher-order skills is evident in both sets of empirical results.

Various comprehensive “21st century skills” systems proposed in the past without much empirical verification also seem to have been found too complex and cumbersome for implementation. The 4Cs, on the other hand, seem to provide a relatively simple, persuasive, targetable core that has been found to constitute a pedagogically and policy-friendly model by major organizations, and that also now seems to be gaining some additional empirical validity. Gathering support from researchers and industry alike, we suggest that the 4Cs can be seen as highest-level transversal skills—or “meta-competencies”—that allow individuals to remain competent and to develop their potential in a rapidly changing professional world. Thus, in the end, they may also be one of the most useful ways of summarizing and addressing the critical challenges faced by the future of work and education ( National Education Association 2011 ).

Taking them as our focus, we note, however, that the teaching and development of the 4Cs will require a complex intervention and mobilization of educational and socio-economic resources—both a major shift in pedagogical techniques and even more fundamental changes in institutional structures ( Ananiadoui and Claro 2009 ). One very important issue for understanding the 4Cs and their educational implementation related to this, which can simultaneously facilitate their teaching but be a challenge for their assessment, is the multidimensionality, interrelatedness, and transdisciplinary relevance of the 4Cs. Thus, we address the relationships between the Cs in the different C sections and later in our Discussion, we present a “Dynamic Interactionist Model of the 4Cs’’ that we hope will assist in their understanding, in the further development of pedagogical processes related to them, and in their public promotion and related policy. Ultimately, it is partly due to their complexity and interrelationships, we argue, that it is important and expedient that the 4Cs are taught, assessed, and promoted together.

2. The 4Cs, Assessment, and Support for Development

2.1. creativity.

In psychology, creativity is usually defined as the capacity to produce novel, original work that fits with task constraints and has value in its context (for a recent overview, see Lubart and Thornhill-Miller 2019 ). This basic definition, though useful for testing and measurement, is largely incomplete, as it does not contain any information about the individual or groups doing the creating or the nature of physical and social contexts ( Glăveanu 2014 ). Moreover, Corazza ( 2016 ) challenged this standard definition of creativity, arguing that as it focuses solely on the existence of an original and effective outcome, it misses the dynamics of the creative process, which is frequently associated with periods of creative inconclusiveness and limited occasions of creative achievements. To move away from the limitations of the standard definition of creativity, we can consider Bruner’s description of creativity as “figuring out how to use what you already know in order to go beyond what you currently think” (p. 183 in Weick 1993 ). This description echoes the notion of potential, which refers to a latent state that may be put to use if a person has the opportunity.

Creativity is a multifaceted phenomenon that can be approached from many different angles. There are three main frameworks for creativity studies: the 4Ps ( Rhodes 1961 ), the 5As ( Glăveanu 2013 ), and the 7Cs model ( Lubart 2017 ). These frameworks share at least four fundamental and measurable dimensions: the act of creating (process), the outcome of the creative process (product), the characteristics of creative actor(s) enacting the process (person), and the social and physical environment that enable or hinder the creative process (press). Contrary to many traditional beliefs, however, creativity can be trained and taught in a variety of different ways, both through direct, active teaching of creativity concepts and techniques and through more passive and indirect means such as the development of creativity-supporting contexts ( Chiu 2015 ; Thornhill-Miller and Dupont 2016 ). Alongside intelligence, with which it shares some common mechanisms, creativity is now recognized as an indispensable element for the flexibility and adaptation of individuals in challenging situations ( Sternberg 1986 ).

2.1.1. Individual Assessment of Creativity

Drawing upon previous efforts to structure creativity research, Batey ( 2012 ) proposed a taxonomic framework for creativity measurement that takes the form of a three-dimensional matrix: (a) the level at which creativity may be measured (the individual, the team, the organization, and the culture), (b) the facets of creativity that may be assessed (person/trait, process, press, and product), and (c) the measurement approach (objective, self-rating, other ratings). It is beyond the scope of this article to offer a literature review of all these dimensions, but for the purposes of this paper, we address some important aspects of individual-level and institutional-level assessment here.

Assessing creativity at an individual level encompasses two major approaches: (1) creative accomplishment based on production and (2) creative potential. Regarding the first approach focusing on creative accomplishment , there are at least four main assessment techniques (or tools representing variations of assessment techniques): (a) the historiometric approach, which applies quantitative analysis to historically available data (such as the number of prizes won or times cited) in an effort to understand eminent, field-changing creativity ( Simonton 1999 ); (b) the Consensual Assessment Technique (CAT) ( Amabile 1982 ), which offers a method for combining and validating judges’ subjective evaluations of a set of (potentially) creative productions or ideas; (c) the Creative Achievement Questionnaire ( Carson et al. 2005 ), which asks individuals to supply a self-reported assessment of their publicly recognizable achievement in ten different creative domains; and (d) the Inventory of Creative Activities and Achievements (ICAA) ( Jauk et al. 2014 ; Diedrich et al. 2018 ), which includes self-report scales assessing the frequency of engagement in creative activity and also levels of achievement in eight different domains.

The second major approach to individual assessment is based on creative potential, which measures the cognitive abilities and/or personality traits that are important for creative work. The two most popular assessments of creative potential are the Remote Associations Test (RAT) and the Alternative Uses Task (AUT). The RAT, which involves identifying the fourth word that is somehow associated with each of three given words, underscores the role that the ability to convergently associate disparate ideas plays as a key capacity for creativity. In contrast, the AUT, which requires individuals to generate a maximum number of ideas based on a prompt (e.g., different uses for a paperclip), is used to assess divergent thinking capacity. According to multivariate models of creative potential ( Lubart et al. 2013 ), there are cognitive factors (e.g., divergent thinking, mental flexibility, convergent thinking, associative thinking, selective combination), conative factors (openness, tolerance of ambiguity, intuitive thinking, risk taking, motivation to create), and environmental factors that all support creativity. Higher creative potential is predicted by having more of the ingredients for creativity. However, multiple different profiles among a similar set of these important ingredients exist, and their weighting for optimal creative potential varies according to the profession, the domain, and the task under consideration. For example, Lubart and Thornhill-Miller ( 2021 ) and Lubin et al. ( forthcoming ) have taken this creativity profiling approach, exploring the identification and training of the components of creative potential among lawyers and clinical psychologists, respectively. For a current example of this sort of comprehensive, differentiated measurement of creative potential in adults in different domains and professions, see CreativityProfiling.org. For a recent battery of tests that are relevant for children, including domain-relevant divergent-exploratory and convergent-integrative tasks, see Lubart et al. ( 2019 ). Underscoring the growing recognition of the importance of creativity assessment, measures of creative potential for students were introduced internationally for the first time in the PISA 2022 assessment ( OECD 2019a ).

2.1.2. Institutional and Environmental Support for Development of Creativity

The structural support that institutions and programs can provide to promote the development of creativity can be described as coming through three main paths: (1) through design of the physical environment in a manner that supports creativity, (2) through teaching about creativity, the creative process, and creativity techniques, and (3) through training opportunities to help students/employees develop personal habits, characteristics, and other ingredients associated with creative achievement and potential.

Given the multi-dimensionality of the notion of creativity, the environment can positively influence and help develop creative capacities. Studies have shown that the physical environment in which individuals work can enhance their positive emotions and mood and thus their creativity. For example, stimulating working environments might have unusual furniture and spaces that have natural light, windows open to nature, plants and flowers, a relaxing atmosphere and colors in the room (e.g., green and blue), or positive sounds (e.g., calm music or silence), as well as inspiring and energizing colors (e.g., yellow, pink, orange). Furthermore, the arrangement of physical space to promote interpersonal exchange rather than isolation, as well as the presence of tools, such as whiteboards, that support and show the value of exchange, are also important (for reviews, see Dul and Ceylan 2011 ; Samani et al. 2014 ).

Although it has been claimed that “creativity is intelligence having fun” ( Scialabba 1984 ; Reiman 1992 ), for most people, opportunities for fun and creativity, especially in their work environment, appear rather limited. In fact, the social and physical environment often hinders creativity. Corazza et al. ( 2021 )’s theoretical framework concerning the “Space-Time Continuum”, related to support for creativity, suggests that traditional education systems are an example of an environment that is “tight” both in the conceptual “space” it affords for creativity and in the available time allowed for creativity to happen—essentially leaving little room for original ideas to emerge. Indeed, though world-wide data suggest that neither money nor mere time spent in class correlate well with educational outcomes, both policies and pedagogy that direct the ways in which time is spent make a significant difference ( Schleicher 2022 ). Research and common sense suggest that teachers, students, and employees need more space and time to invest energy in the creative process and the development of creative potential.

Underscoring the importance of teaching the creative process and creativity techniques is the demonstration, in a number of contexts, that groups of individuals who generate ideas without a specific method are often negatively influenced by their social environment. For example, unless guarded against, the presence of others tends to reduce the number of ideas generated and to induce a fixation on a limited number of ideas conforming to those produced by others ( Camarda et al. 2021 ; Goldenberg and Wiley 2011 ; Kohn and Smith 2011 ; Paulus and Dzindolet 1993 ; Putman and Paulus 2009 ; Rietzschel et al. 2006 ). To overcome these cognitive and social biases, different variants of brainstorming techniques have shown positive effects (for reviews of methods, see Al-Samarraie and Hurmuzan 2018 ; Paulus and Brown 2007 ). These include: using ( Osborn 1953 ) initial brainstorming rules (which aim to reduce spontaneous self-judgment of ideas and fear of this judgment by others); drawing attention to ideas generated by others by writing them down independently (e.g., the technique known as “brainwriting”); and requiring incubation periods between work sessions by forcing members of a problem-solving group to take breaks ( Paulus and Yang 2000 ; Paulus and Kenworthy 2019 ).

It is also possible to use design methods that are structured to guide the creative process and the exploration of ideas, as well as to avoid settling on uncreative solution paths ( Chulvi et al. 2012 ; Edelman et al. 2022 ; Kowaltowski et al. 2010 ; see Cotter et al. 2022 for a valuable survey of best practices for avoiding the suppression of creativity and fostering creative interaction and metacognition in the classroom). Indeed, many helpful design thinking-related programs now exist around the world and have been shown to have a substantial impact on creative outcomes ( Bourgeois-Bougrine 2022 ).

Research and experts suggest the utility of many additional creativity enhancement techniques (see, e.g., Thornhill-Miller and Dupont 2016 ), and the largest and most rapid effects are often attributed to these more method- or technique-oriented approaches ( Scott et al. 2004 ). More long-term institutional and environmental support for the development of creativity, however, should also include targeted training and understanding of personality and emotional traits associated with the “creative person” (e.g., empathy and exploratory habits that can expand knowledge, as well as increase tolerance of ambiguity, openness, and mental flexibility; see Lubart and Thornhill-Miller 2021 ). Complementing these approaches and focusing on a more systemic level, recent work conducted by the OECD exemplifies efforts aimed to foster creativity (and critical thinking) by focusing simultaneously on curriculum, educational activities, and teacher support and development at the primary, secondary, and higher education levels (see Vincent-Lancrin et al. 2019 ; Saroyan 2022 ).

2.2. Critical Thinking

Researchers, teachers, employers, and public policymakers around the world have long ranked the development of critical thinking (CT) abilities as one of the highest educational priorities and public needs in modern democratic societies ( Ahern et al. 2019 ; Dumitru et al. 2018 ; Pasquinelli et al. 2021 ). CT is central to better outcomes in daily life and general problem solving ( Hitchcock 2020 ), to intelligence and adaptability ( Halpern and Dunn 2021 ), and to academic achievement ( Ren et al. 2020 ). One needs to be aware of distorted or erroneous information in the media, of the difference between personal opinions and proven facts, and how to handle increasingly large bodies of information required to understand and evaluate information in the modern age.

Although much research has addressed both potentially related constructs, such as intelligence and wisdom, and lists of potential component aspects of human thought, such as inductive or deductive reasoning (for reviews of all of these, see Sternberg and Funke 2019 ), reaching a consensus on a definition has been difficult, because CT relies on the coordination of many different skills ( Bellaera et al. 2021 ; Dumitru et al. 2018 ) and is involved in, and sometimes described from the perspective of, many different domains ( Lewis and Smith 1993 ). Furthermore, as a transversal competency, having the skills to perform aspects of critical thinking in a given domain does not necessarily entail also having the metacognitive ability to know when to engage in which of its aspects, or having the disposition, attitude, or “mindset” that motivates one to actually engage in them—all of which are actually required to be a good critical thinker ( Facione 2011 ).

As pointed out by the American Philosophical Association’s consensus definition, the ideal “critical thinker” is someone who is inquisitive, open-minded, flexible, fair-minded, and keeps well-informed, thus understanding different points of view and perspectives ( Facione 1990b ). These characteristics, one might note, are also characteristic of the “creative individual” ( Facione 1990b ; Lai 2011 ), as is the ability to imagine alternatives, which is often cited as a component of critical thinking ability ( Facione 1990b ; Halpern 1998 ). Conversely, creative production in any domain needs to be balanced by critical appraisal and thought at each step of the creative process ( Bailin 1988 ). Indeed, it can be argued that creativity and critical thinking are inextricably linked and are often two sides of the same coin. Representing different aspects of “good thought” that are linked and develop in parallel, it seems reasonable that they should, in practice, be taught and considered together in teaching and learning ( Paul and Elder 2006 ).

Given its complexity, many definitions of critical thinking have been offered. However, some more recent work has helpfully defined critical thinking as “the capacity of assessing the epistemic quality of available information and—as a consequence of this assessment—of calibrating one’s confidence in order to act upon such information” ( Pasquinelli et al. 2021 ). This definition, unlike others proposed in the field (for a review, see: Bellaera et al. 2021 ; Liu et al. 2014 ), is specific (i.e., it limits the use of poorly defined concepts), as well as consensual and operational (i.e., it has clear and direct implications for the education and assessment of critical thinking skills; Pasquinelli et al. 2021 ; Pasquinelli and Bronner 2021 ). Thus, this approach assumes that individuals possess better or worse cognitive processes and strategies that make it possible to judge the reliability of the information received, by determining, for example, what the arguments provided actually are. Are the arguments convincing? Is the source of information identifiable and reliable? Does the information conflict with other information held by the individual?

It should also be noted that being able to apply critical thinking is necessary to detect and overcome the cognitive biases that can constrain one’s reasoning. Indeed, when solving a problem, it is widely recognized that people tend to automate the application of strategies that are usually relevant in similar and analogous situations that have already been encountered. However, these heuristics (i.e., automatisms) can be a source of errors, in particular, in tricky reasoning situations, as demonstrated in the field of reasoning, arithmetic problems ( Kahneman 2003 ) or even divergent thinking tasks ( Cassotti et al. 2016 ; for a review of biases, see Friedman 2017 ). Though some cognitive biases can even be seen as normal ways of thinking and feeling, sometimes shaping human beliefs and ideologies in ways that make it completely normal—and even definitely human— not to be objective (see Thornhill-Miller and Millican 2015 ), the mobilization of cognitive resources such as those involved in critical reasoning on logical bases usually makes it possible to overcome cognitive biases and adjust one’s reasoning ( West et al. 2008 ).

According to Pasquinelli et al. ( 2021 ), young children already possess cognitive functions underlying critical thinking, such as the ability to determine that information is false. However, until late adolescence, studies have demonstrated an underdevelopment of executive functions involved in resistance to biased reasoning ( Casey et al. 2008 ) as well as some other higher-order skills that underlie the overall critical thinking process ( Bloom 1956 ). According to Facione and the landmark American Philosophical Association’s task force on critical thinking ( Facione 1990b ; Facione 2011 ), these components of critical thinking can be organized into six measurable skills: the ability to (1) interpret information (i.e., meaning and context); (2) analyze information (i.e., make sense of why this information has been provided, identify pro and con arguments, and decide whether we can accept the conclusion of the information); (3) make inferences (i.e., determine the implications of the evidence, its reliability, the undesirable consequences); (4) evaluate the strength of the information (i.e., its credibility, determine the trust in the person who provides it); (5) provide explanations (i.e., summarize the findings, determine how the information can be interpreted, and offer verification of the reasoning); (6) self-regulate (i.e., evaluate the strength of the methods applied, determine the conflict between different conclusions, clarify the conclusions, and verify missing elements).

2.2.1. Individual Assessment of Critical Thinking

The individual assessment of critical thinking skills presents a number of challenges, because it is a multi-task ability and involves specific knowledge in the different areas in which it is applied ( Liu et al. 2014 ; Willingham 2008 ). However, the literature provides several tools with which to measure different facets of cognitive functions and skills involved in the overarching critical thinking process ( Lai 2011 ; Liu et al. 2014 ). Most assessments involve multiple-choice questions requiring reasoning within a particular situation based upon a constrained set of information provided. For example, in one of the most widely used tests, the California Critical Thinking Skills Test ( Facione 1990a ), participants are provided with everyday scenarios and have to answer multiple questions targeting the six higher-order skills described previously. Similarly, the Watson–Glaser Critical Thinking Appraisal ( Watson 1980 ; Watson and Glaser 2010 ) presents test takers with passages and scenarios measuring their competencies at recognizing assumptions, evaluating arguments, and drawing conclusions. Although the Watson–Glaser is one of the oldest and most frequently used assessments internationally for hiring and promotion in professional contexts, its construct validity, like many other measures of this challenging topic, has some limitations ( Possin 2014 ).

Less frequently, case study or experiential methods of assessment are also used. This approach may involve asking participants to reflect on past experiences, analyze the situations they faced and the way they behaved or made judgments and decisions and then took action ( Bandyopadhyay and Szostek 2019 ; Brookfield 1997 ). These methods, often employed by teachers or employers on students and employees, usually involve the analysis of qualitative data that can cast doubt on the reliability of the results. Consequently, various researchers have suggested ways to improve analytic methods, and they emphasize the need to create more advanced evaluation methods ( Brookfield 1997 ; Liu et al. 2014 ).

For example, Liu et al. ( 2014 ) reviewed current assessment methods and suggest that future work improves the operational definition of critical thinking, aiming to assess it both in different specific contexts and in different formats. Specifically, assessments could be contextualized within the major areas addressed by education programs (e.g., social sciences, humanities, and/or natural sciences), and the tasks themselves should be as practically connected to the “real world” as possible (e.g., categorizing a set of features, opinions, or facts based on whether or not they support an initial statement). Moreover, as Brookfield ( 1997 ) argues, because critical thinking is a social process that takes place in specific contexts of knowledge and culture, it should be assessed as a social process, therefore, involving a multiplicity of experiences, perceptions, and contributions. Thus, Brookfield makes three recommendations for improving the assessment of critical thinking that are still relevant today: (1) to assess critical thinking in specific situations, so one can study the process and the discourse related to it; (2) to involve students/peers in the evaluation of critical thinking abilities, so that the evaluation is not provided only by the instructor; and (3) to allow learners or participants in an experiment to document, demonstrate, and justify their engagement in critical thinking, because this learning perspective can provide insight into basic dimensions of the critical thinking process.

Finally, another more recent and less widely used form of assessment targets the specific executive functions that underlie logical reasoning and resistance to cognitive biases, as well as the ability of individuals to resist these biases. This form of assessment is usually done through specific experimental laboratory tasks that vary depending on the particular executive function and according to the domain of interest ( Houdé and Borst 2014 ; Kahneman 2011 ; West et al. 2008 ).

2.2.2. Institutional and Environmental Support for Development of Critical Thinking Skills

The executive functions underlying general critical thinking, the ability to overcome bias ( Houdé 2000 ; Houdé and Borst 2014 ), and meta-cognitive processes (i.e., meta information about our cognitive strategies) can all be trained and enhanced by educational programs ( Abrami et al. 2015 ; Ahern et al. 2019 ; Alsaleh 2020 ; Bellaera et al. 2021 ; Uribe-Enciso et al. 2017 ; Popil 2011 ; Pasquinelli and Bronner 2021 ; Yue et al. 2017 ).

Educational programs and institutions can support the development of critical thinking in several different ways. The process of developing critical thinking focuses on the interaction between personal dispositions (attitudes and habits), skills (evaluation, reasoning, self-regulation), and finally, knowledge (general and specific knowledge, as well as experience) ( Thomas and Lok 2015 ). It is specifically in regard to skills and knowledge that institutions are well suited to develop critical thinking through pedagogical elements such as rhetoric training, relevance of information evaluation (e.g., media literacy, where and how to check information on the internet, dealing with “fake news”, etc.), deductive thinking skills, and inductive reasoning ( Moore and Parker 2016 ). A few tools, such as case studies or concept mapping, can also be used in conjunction with a problem-based learning method, both in individual and team contexts and in person or online ( Abrami et al. 2015 ; Carmichael and Farrell 2012 ; Popil 2011 ; Thorndahl and Stentoft 2020 ). According to Marin and Halpern ( 2011 ), training critical thinking should include explicit instruction involving at least the four following components and objectives: (1) working on attitudes and encouraging individuals to think; (2) teaching and practicing critical thinking skills; (3) training for transfer between contexts, identifying concrete situations in which to adopt the strategies learned; and (4) suggesting metacognition through reflection on one’s thought processes. Supporting these propositions, Pasquinelli and Bronner ( 2021 ), in a French national educational report, proposed practical advice for creating workshops to stimulate critical thinking in school classrooms, which appear relevant even in non-school intervention situations. For example, the authors suggest combining concrete examples and exercises with general and abstract explanations, rules and strategies, which can be transferred to other areas beyond the one studied. They also suggest inviting learners to create examples of situations (e.g., case studies) in order to increase the opportunities to practice and for the learner to actively participate. Finally, they suggest making the process of reflection explicit by asking the learner to pay attention to the strategies adopted by others in order to stimulate the development of metacognition.

2.3. Communication

In its most basic definition, communication consists of exchanging information to change the epistemic context of others. In cooperative contexts, it aims at the smooth and efficient exchange of information contributing to the achievement of a desired outcome or goal ( Schultz 2010 ). But human communication involves multiple dimensions. Both verbal and non-verbal communication can involve large quantities of information that have to be both formulated and deciphered with a range of purposes and intentions in mind ( Jones and LeBaron 2002 ). These dimensions of communication have as much to do with the ability to express oneself, both orally and in writing and the mastering of a language (linguistic competences), as with the ability to use this communication system appropriately (pragmatic skills; see Grassmann 2014 ; Matthews 2014 ), and with social skills, based on the knowledge of how to behave in society and on the ability to connect with others, to understand the intentions and perspectives of others ( Tomasello 2005 ).

Like the other 4Cs, according to most authorities, communication skills are ranked by both students and teachers as skills of the highest priority for acquisition in order to be ready for the workforce in 2030 ( OECD 2019b ; Hanover Research 2012 ). Teaching students how to communicate efficiently and effectively in all the new modalities of information exchange is an important challenge faced by all pedagogical organizations today ( Morreale et al. 2017 ). All dimensions of communication (linguistic, pragmatic, and social) are part of what is taught in school curricula at different levels. But pragmatic and social competencies are rarely explicitly taught as such. Work on social/emotional intelligence (and on its role in students’ personal and professional success) shows that these skills are both disparate and difficult to assess ( Humphrey et al. 2007 ). Research on this issue is, however, becoming increasingly rigorous, with the potential to provide usable data for the development of science-based practice ( Keefer et al. 2018 ). Teachers and pedagogical teams also have an important, changing role to play: they also need to master new information and communication technologies and the transmission of information through them ( Zlatić et al. 2014 ).

Communication has an obvious link with the three other Cs. Starting with critical thinking, sound communication implies fostering the conditions for a communicative exchange directed towards a common goal, which is, at least in educational and professional contexts, based on a fair evaluation of reality ( Pornpitakpan 2004 ). Collaboration too has a strong link with communication, because successful collaboration is highly dependent on the quality of knowledge sharing and trust that emerges between group members. Finally, creativity involves the communication of an idea to an audience and can involve high-quality communication when creative work occurs in a team context.

2.3.1. Individual Assessment of Communication

Given the vast field of communication, an exhaustive list of its evaluation methods is difficult to establish. A number of methods have been reported in the literature to assess an individual’s ability to communicate non-verbally and verbally. But although these two aspects are intrinsically linked, they are rarely measured together with a single tool. Moreover, as Spitzberg ( 2003 ) pointed out, communication skills are supported by different abilities, classically conceptualized as motivational functions (e.g., confidence and goal-orientation), knowledge (e.g., content and procedural knowledge), or cognitive and socio-cognitive functions (e.g., theory of mind, verbal cognition, emotional intelligence, and empathy; McDonald et al. 2014 ; Rothermich 2020 ), implying different specific types of evaluations. Finally, producing vs. receiving communication involve different skills and abilities, which can also vary according to the context ( Landa 2005 ).

To overcome these challenges, Spitzberg ( 2003 ) recommends the use of different assessment criteria. These criteria include the clarity of interaction, the understanding of what was involved in the interaction, the satisfaction of having interacted (expected to be higher when communication is effective), the efficiency of the interaction (the more competent someone is, the less effort, complexity, and resources will be needed to achieve their goal), its effectiveness or appropriateness (i.e., its relevance according to the context), as well as criteria relative to the quality of the dialogue (which involves coordination, cooperation, coherence, reciprocity, and mutuality in the exchange with others). Different forms of evaluation are also called for, such as self-reported questionnaires, hetero-reported questionnaires filled out by parents, teachers, or other observers, and tasks involving exposure to role-playing games, scenarios or videos (for a review of these assessment tools, see Cömert et al. 2016 ; Landa 2005 ; Sigafoos et al. 2008 ; Spitzberg 2003 ; van der Vleuten et al. 2019 ). Results from these tools must then be associated with others assessing underlying abilities, such as theory of mind and metacognition.

2.3.2. Institutional and Environmental Support for Development of Communication Skills

Although communication appears to be a key employability skill, the proficiency acquired during studies rarely meets the expectations of employers ( Jackson 2014 ). Communication must therefore become a priority in the training of students, beyond the sectors in which it is already known as essential (e.g., in medicine, nursing, engineering, etc.; Bourke et al. 2021 ; D’Alimonte et al. 2019 ; Peddle et al. 2018 ; Riemer 2007 ), and also through professional development ( Jackson 2014 ). Training programs involving, for example, communication theory classes ( Kruijver et al. 2000 ) and self-assessment tools that can be used in specific situations ( Curtis et al. 2013 ; Rider and Keefer 2006 ) have had convincingly positive results. The literature suggests that interactive approaches in small groups, in which competencies are practiced explicitly in an open and feedback-safe environment, are more effective ( Bourke et al. 2021 ; D’Alimonte et al. 2019 ; AbuSeileek 2012 ; Fryer-Edwards et al. 2006 ). These can take different forms: project-based work, video reviews, simulation or role-play games (see Hathaway et al. 2022 for a review; Schlegel et al. 2012 ). Finally, computer-assisted learning methods can be relevant for establishing a secure framework (especially, for example, when learning another language): anonymity indeed helps to overcome anxiety or social blockages linked to fear of public speaking or showing one’s difficulties ( AbuSeileek 2012 ). Each of these methods tackles one or more dimensions of communication that must then be assessed as such, by means of tools specifically developed and adapted to the contexts in which these skills are expressed (e.g., see the two 4Cs evaluation grids for institutions and for games outlined in Section 4 and Section 5 , below).

2.4. Collaboration

Collaborative problem solving—and more generally, collaboration—has gained increasing attention in national and international assessments (e.g., PISA) as an educational priority encompassing social, emotional, and cognitive skills critical to efficiency, effectiveness, and innovation in the modern global economy ( Graesser et al. 2018 ; OECD 2017 ). Understanding what makes effective collaboration is of crucial importance for professional practice and training ( Détienne et al. 2012 ; Graesser et al. 2018 ), as evidenced by the long line of research on group or team collaboration over the past 40 years (for a review, see e.g., Salas et al. 2004 ; Mathieu et al. 2017 ). Although there is no consensus on a definition of collaboration, scholars often see it as mutual engagement in a coordinated effort to achieve a common goal that involves the sharing of goals, resources, and representations relating to the joint activity of participants; and other important aspects relate to mutual respect, trust, responsibilities, and accountability within situational rules and norms ( Détienne et al. 2012 ).

In the teamwork research literature, skills are commonly described across three classes most often labeled Knowledge, Behavior, and Attitudes (e.g., Cannon-Bowers et al. 1995 ). Knowledge competencies refer to the skills related to elaborating the knowledge content required for the group to process and successfully achieve the task/goal to which they are assigned. Behavior includes skills related to the actualization of actions, coordination, communication, and interactions within the group as well as with any other relevant interlocutors for the task at hand. Note here that effective collaboration involves skills that have also been identified elsewhere as essential competencies, including communication, creativity, and critical thinking. Finally, several attitudes have been evidenced or hypothesized as desirable competencies in the team context, for example, attitude towards teamwork, collective orientation, cohesion/team morale, etc. Another common distinction lies between teamwork and taskwork. Teamwork refers to the collaborative, communicative, or social skills required to coordinate the work within the participants in order to achieve the task, whereas taskwork refers to specific aspects related to solving the task such as using the tools and knowing the procedure, policies, and any other task-related activities ( Salas et al. 2015 ; Graesser et al. 2018 ). Furthermore, collaborative competences can have specific (to a group of people or to a task) and general dimensions (i.e., easily transferable to any group or team situation and to other tasks). For example, skills related to communication, information exchange, conflict management, maintaining attention and motivation, leadership, etc. are present and transferable to a large number of group work situations and tasks (team-generic and task-contingent skills). Other skills can, on the other hand, be more specific to a team or group, such as internal organization, motivation, knowledge of the skills distributed in the team, etc.

2.4.1. Individual Assessment of Collaboration

Assessing collaboration requires capturing the dynamic and multi-level nature of the collaboration process, which is not as easily quantifiable as group/team inputs and outputs (task performance, satisfaction, and changes at group/team and individual level). There are indeed multiple interactions between the context, the collaboration processes, the task processes, and their (various) outcomes ( Détienne et al. 2012 ). The integrative concept of “quality of collaboration” ( Burkhardt et al. 2009 ) encapsulates much of what is currently known about collaborative processes and what constitutes effective collaboration. According to this approach, collaborative processes can be grouped along several dimensions concerning communication processes such as grounding, task-related processes (e.g., exchanges of knowledge relevant for the task at hand), and organization/coordination processes ( Burkhardt et al. 2009 ). Communication processes are most important for ensuring the construction of a common referential within a group of collaborators. Task-related processes relate to how the group resolves the task at hand by sharing and co-elaborating knowledge, by confronting their various perspectives, and by converging toward negotiated solutions. Collaboration also involves group management activities such as: (a) common goal management and coordination activities, e.g., allocation and planning of tasks; (b) meeting/interaction management activities, e.g., ordering and postponing of topics in the meeting. Finally, the ability to pursue reflexive activity, in the sense of reflecting not only on the content of a problem or solution but on one’s collaboration and problem-solving strategies, is critical for the development of the team and supports them in changing and improving their practices. Graesser et al. ( 2018 ) identify collaborative skills based on the combination of these dimensions with a step in the problem-solving process.

A large body of methodology developed to assess collaboration processes and collaborative tools has been focused on quantifying a restricted subset of fine-grained interactions (e.g., number of speakers’ turns; number of words spoken; number of interruptions; amount of grounding questions). This approach has at least two limitations. First, because these categories of analysis are often ad hoc with respect to the considered situation, they are difficult to apply in all situations and make it difficult to compare between studies. Second, quantitative variations of most of these indicators are non-univocal: any increase or decrease of them could signify either an interactive–intensive collaboration or else evidence of major difficulties in establishing and/or maintaining the collaboration ( Détienne et al. 2012 ). Alternatively, qualitative approaches based on multidimensional views of collaboration provide a more elaborated or nuanced view of collaboration and are useful for identifying potential relationships between distinctive dimensions of collaboration and aspects of team performance, in order to identify processes that could be improved. Based on the method of Spada et al. ( 2005 ) in Computer-Supported Collaborative Learning (CSCL) research, Burkhardt et al. ( 2009 ) have proposed a multi-dimensional rating scheme for evaluating the quality of collaboration (QC) in technology-mediated design. QC distinguishes seven dimensions, grouped along five aspects, identified as central for collaboration in a problem-solving task such as design: communication (1, 2), task-oriented processes (3, 4), group-oriented processes (5), symmetry in interaction—an orthogonal dimension—(6), and individual task orientation (7). This method has recently been adapted for use in the context of assessing games as a support to collaborative skills learning.

2.4.2. Institutional and Environmental Support for Development of Collaboration and Collaborative Skills

Support for individuals’ development of collaborative skills provided by institutions and programs can take a variety of forms: (a) through the social impact of the physical structure of the organization, (b) the nature of the work required within the curriculum, (c) content within the curriculum focusing on collaboration and collaborative skills, and (d) the existence and promotion of extracurricular and inter-institutional opportunities for collaboration.

For instance, institutional support for collaboration has taken a variety of forms in various fields such as healthcare, engineering, public participation, and education. Training and education programs such as Interprofessional Education or Team Sciences in the health domain ( World Health Organization 2010 ; Hager et al. 2016 ; O’Carroll et al. 2021 ), Peer-Led Team Learning in chemistry and engineering domains ( Wilson and Varma-Nelson 2016 ), or Collaborative Problem Solving in education ( Peña-López 2017 ; Taddei 2009 ) are notable examples.

Contextual support recently arose from the deployment of online digital media and new mixed realities in the workplace, in the learning environments and in society at large—obviously stimulated and accentuated with the COVID-19 pandemic. This has led many organizations to invest in proposing support for synchronous and asynchronous collaboration (notably remote, between employees, between students and educators or within group members, etc.) in various ways, including the provision of communication hardware and software, computer-supported cooperative work and computer-supported collaborative learning platforms, training and practical guides, etc. Users can collaborate through heterogeneous hybrid collaborative interaction spaces that can be accessed through virtual or augmented reality, but also simple video conferencing or even a voice-only or text-only interface. These new spaces for collaboration are, however, often difficult to use and less satisfactory than face-to-face interactions, suggesting the need for more research on collaborative activities and on how to support them ( Faidley 2018 ; Karl et al. 2022 ; Kemp and Grieve 2014 ; Singh et al. 2022 ; Waizenegger et al. 2020 ).

A substantive body of literature on teams, collaborative learning, and computer-supported technologies provides evidence related to individual, contextual, and technological factors impacting the collaboration quality and efficiency. For example, teacher-based skills that are critical for enhancing collaboration are, among others, the abilities to plan, monitor, support, consolidate, and reflect upon student interaction in group work ( Kaendler et al. 2016 ). Research focuses also on investigating the most relevant tasks and evaluating the possibilities offered by technology to support, to assess (e.g., Nouri et al. 2017 ; Graesser et al. 2018 ), and/or to learn the skills involved in pursuing effective and satisfying collaboration (see e.g., Schneider et al. 2018 ; Doyle 2021 ; Ainsworth and Chounta 2021 ).

3. Labelization: Valorization of the 4Cs and Assessing Support for Their Development

Moving from the nature of the 4Cs and their individual assessment and towards the ways in which institutions can support their development in individuals, we can now address the fundamentally important question of how best to support and promote this 21st century educational mission within and among institutions themselves. This also raises the question of the systemic recognition of educational settings that are conducive to the development of the 4Cs. In response to these questions, the nature and value of labelization is now presented.

A label is “a special mark created by a trusted third party and displayed on a product intended for sale, to certify its origin, to guarantee its quality and to ensure its conformity with the standards of practices in force” ( Renard 2005 ). A label is therefore a way of informing the public about the objective properties and qualities of a product, service, or system. The label is usually easily identifiable and can be seen as a proof that a product or service, a company, or an organization complies with defined criteria. Its effectiveness is therefore closely linked to the choice of requirements set out in its specifications, as well as to the independence and rigor of the body that verifies compliance with the criteria.

3.1. Labeling as a Means of Trust and Differentiation

As a sign of recognition established by a third party, the label or certification can constitute a proof of trust aiming to reassure the final consumer. According to Sutter ( 2005 ), there are different means of signaling trust. First, the brand name of a product or service and its reputation can, in itself, constitute a label when this brand name is recognized on the market. Second, various forms of self-declaration, such as internal company charters, though not statements assessed by a third party, show an internal commitment that can provide reassurance. Finally, there is certification or labeling, which is awarded by an external body and requires a third-party assessment by a qualified expert, according to criteria set out in a specific reference framework. It is this external body, a trusted third party, which guarantees the reliability of the label and constitutes a guarantee of credibility. Its objectivity and impartiality are meant to guarantee that the company, organization, product, or service meets defined quality or reliability criteria ( Jahn et al. 2005 ).

Research on populations around the world (e.g., Amron 2018 ; Sasmita and Suki 2015 ) show that the buying decisions of consumers are heavily influenced by the trust they have in a brand. More specifically, third-party assurances and labelization have been shown to strongly influence customer buying intentions and purchasing behavior (e.g., Kimery and McCord 2002 ; Lee et al. 2004 ). Taking France as an example, research shows that quality certification is seen as “important” or “significant” by 76% of companies ( Chameroy and Veran 2014 ), and decision makers feel more confident and are more willing to invest with the support of third-party approval than if their decision is merely based on the brand’s reputation or its demonstrated level of social responsibility ( Etilé and Teyssier 2016 ). Indeed, French companies with corporate social responsibility labels have been shown to have higher than average growth rates, and the adoption of quality standards is linked with a 7% increase in the share of export turnover ( Restout 2020 ).

3.2. Influence on Choice and Adoption of Goods and Services

Studies diverge in this area, but based on the seminal work of Parkinson ( 1975 ); Chameroy and Veran ( 2014 ), in their research on the effect of labels on willingness to pay, found that in 75% of cases, products with labels are chosen and preferred to those without labels, demonstrating the impact of the label on customer confidence—provided that it is issued by a recognized third party. Thus, brands that have good reputations tend to be preferred over cheaper new brands, because they are more accepted and valued by the individual social network ( Zielke and Dobbelstein 2007 ).

3.3. Process of Labelizing Products and Services

The creation of a label may be the result of a customer or market need, a request from a private sector of activity or from the government. Creating a label involves setting up a working group including stakeholders who are experts in the field, product managers, and a certification body in order to elaborate a reference framework. This is then reviewed by a specialized committee and validated by the stakeholders. The standard includes evaluation criteria that must be clearly defined ( Mourad 2017 ). An audit system is set up by a trusted third party. It must include the drafting of an audit report, a system for making decisions on labeling, and a system for identifying qualified assessors. The validity of the assessment process is reinforced by this double evaluation: a first level of audit carried out by a team of experts according to a clearly defined set of criteria and a second level of decision making assuring that the methodology and the result of the audit are in conformity with the defined reference framework.

3.4. Labelization of 21st Century Skills

The world of education is particularly concerned by the need to develop and assess 21st century skills, because it represents the first link in the chain of skills acquisition, preparing the human resources of tomorrow. One important means of simultaneously offering a reliable, independent assessment of 21st century skills and valorizing them by making them a core target within an educational system (schools, universities, and teaching and training programs of all kinds) is labelization. Two examples of labelization processes related to 21st century skills were recently developed by the International Institute for Competency Development ( 2021 ; see iicd.net; accessed on 20 November 2022) working with international experts, teachers, and researchers from the University of Paris Cité (formerly Université Sorbonne Paris Cité), Oxford University, and AFNOR UK (an accredited certification body and part of AFNOR International, a subsidiary of the AFNOR group, the only standards body in France).

The last two or three decades has seen the simultaneous rise of international ranking systems and an interest in quality assurance and assessment in an increasingly competitive educational market ( Sursock 2021 ). The aim of these labelization frameworks is to assist in the development of “quality culture” in education by offering individual programs, institutions, and systems additional independent, reliable means of benchmarking, charting progress, and distinguishing themselves based on their capacity to support and promote the development of crucial skills. Importantly, the external perspectives provided by such assessment system should be capable of being individually adapted and applied in a manner that can resist becoming rigidly imposed external standards ( Sursock and Vettori 2017 ). Similarly, as we have seen in the literature review, the best approach to understanding and assessing a particular C is from a combination of different levels and perspectives in context. For example, important approaches to critical thinking have been made from educationally, philosophically, and psychologically focused vantage points ( Lai 2011 ). We can also argue that understandings of creativity are also results of different approaches: the major models in the literature (e.g., the “4Ps” and “7Cs” models; see Lubart and Thornhill-Miller 2019 ) explicitly result from and include the objectives of different education-focused, process-focused, and “ingredient” or component-focused approaches.

The two assessment frameworks outlined in the sections that follow were formulated with these different perspectives and objective needs in mind. Given the complexity and very different natures of their respective targets (i.e., one assessing entire formal educational contexts such as institutions or programs, whereas the other targets the less multi-dimensional, informal educational activities represented by games), the assessment of the individual Cs also represents what experts consider a target-appropriate balance of education- and curriculum-focused, process-focused, and component-focused criteria for assessing each different C.

4. The International Institute for Competency Development’s 21st Century Competencies 4Cs Assessment Framework for Institutions and Programs

One comprehensive attempt to operationalize programmatic-level and institutional-level support for the development of the 4Cs is the International Institute for Competency Development’s 4Cs Assessment Framework ( International Institute for Competency Development 2021 ). Based upon expert opinion and a review of the available literature, this evaluation grid is a practical tool that divides each of the 4Cs into three “user-friendly” but topic-covering components (see Table 1 and definitions and further discussion in the sections that follow). Each of these components is then assessed across seven dimensions (see Table 2 , below), designed to cover concisely the pedagogical process and the educational context. Examples for each point level are provided within the evaluation grid in order to offer additional clarity for educational stakeholders and expert assessors.

Three different components of each C in IICD’s 21st Century Skills 4Cs Assessment Framework.

Seven dimensions evaluated for the 3 different components of each C.

* Educational-level dependent and potentially less available for younger students or in some contexts.

The grid itself can be used in several important and different ways by different educational stakeholders: (1) by the institution itself in its self-evaluation and possible preparation for a certification or labelization process, (2) as an explicit list of criteria for external evaluation of the institution and its 4Cs-related programs, and (3) as a potential long-term development targeting tool for the institution or the institution in dialogue with the labelization process.

4.1. Evaluation Grid for Creativity

Dropping the component of “creative person” that is not relevant at the institutional level, this evaluation grid is based on Rhodes’ ( 1961 ) classic “4P” model of creativity, which remains the most concise model today ( Lubart and Thornhill-Miller 2019 ). The three “P” components retained are: creative process , creative environment , and creative product . Creative process refers to the acquisition of a set of tools and techniques that students can use to enhance the creativity of their thinking and work. Creative environment (also called “Press” in earlier literature) is about how the physical and social surroundings of students can help them be more creative. Finally, creative product refers to the evaluation of actual “productions” (e.g., a piece of art, text, speech, etc.) generated through the creative process.

4.2. Evaluation Grid for Critical Thinking

Our evaluation grid divides critical thinking into three main components: critical thinking about the world , critical thinking about oneself (self-reflection), as well as critical action and decision making . The first component refers to having an evidence-based view of the exterior world, notably by identifying and evaluating sources of information and using them to question current understandings and solve problems. Self-reflection refers to thinking critically about one’s own life situation, values, and actions; it presupposes the autonomy of thought and a certain distance as well as the most objective observation possible with regard to one’s own knowledge (“meta-cognition”). The third and final component, critical action and decision making, is about using critical thinking skills more practically in order to make appropriate life decisions as well as to be open to different points of view. This component also addresses soft skills and attitudes such as trusting information.

Our evaluation framework for critical thinking was in part inspired by Barnett’s “curriculum for critical being” (2015), whose model distinguishes two axes: one defined by the qualitative differences in the level of criticality attained and the second comprised of three different domains of application: formal knowledge, the self, and the world. The first two components of our framework (and the seven dimensions on which they are rated) reflect and encompass these three domains. Similar to Barrett’s proposal, our third rubric moves beyond the “skills-plus-dispositions” model of competency implicit in much theorizing about critical thinking and adds the importance of “action”—not just the ability to think critically and the disposition to do so, but the central importance of training and practicing “critical doing” ( Barnett 2015 ). Critical thinking should also be exercised collectively by involving students in collective thinking, facilitating the exchange of ideas and civic engagement ( Huber and Kuncel 2016 ).

4.3. Evaluation Grid for Collaboration

The first component of collaboration skills in the IICD grid is engagement and participation , referring to the active engagement in group work. Perspective taking and openness concerns the flexibility to work with and accommodate other group members and their points of view. The final dimension— social regulation —is about being able to reach for a common goal, notably through compromise and negotiation, as well as being aware of the different types of roles that group members can hold ( Hesse et al. 2015 ; Rusdin and Ali 2019 ; Care et al. 2016 ). (These last two components include elements of leadership, character, and emotional intelligence as sometimes described in other soft-skill and competency-related systems.) Participation, social regulation, and perspective taking have been identified as central social skills in collaborative problem solving ( Hesse et al. 2015 ). Regarding social regulation in this context, recognizing and profiting from group diversity is key ( Graesser et al. 2018 ). When describing an assessment in an educational setting of collaborative problem solving (with a task in which two or more students have to collaborate in order to solve it, each using a different set of resources), two main underpinning skills were described for the assessment: the social skill of audience awareness (“how to adapt one’s own behavior to suit the needs of the task and the partner’s requirements”, Care et al. 2016, p. 258 ) and the cognitive skill of planning and executing (developing a plan to reach for a goal) ( Care et al. 2016 ). The former is included in the perspective taking and openness rubric and the latter in the social regulation component in the IICD grid. Evans ( 2020 ) identified four main collaboration skills consistently mentioned in the scientific literature that are assessed in the IICD grid: the ability to plan and make group decisions (example item from the IICD grid: teachers provide assistance to students to overcome differences and reach a common goal during group work); the ability to communicate about thinking with the group (assessed notably in the meta-reflection strand of the IICD grid); the ability to contribute resources, ideas, and efforts and support group members (included notably in the engagement and participation as well as the social regulation components); and finally, the ability to monitor, reflect, and adapt individual and group processes to benefit the group (example item from the IICD grid: students use perspective-taking tools and techniques in group activities).

4.4. Evaluation Grid for Communication

The evaluation grid for communication is also composed of three dimensions: message formulation, message delivery, and message and communication feedback . Message formulation refers to the ability to design and structure a message to be sent, such as outlining the content of an argument. Message delivery is about effectively transmitting verbal and non-verbal aspects of a message. Finally, message and communication feedback refers to the ability of students and teachers to understand their audience, analyze their social surroundings, and interpret information in context. Other components of communication skills such as theory of mind, empathy, or emotional intelligence are also relevant and included in the process of applying the grid. Thompson ( 2020 ) proposes a four-component operationalized definition of communication for its assessment in students. First, they describe a comprehension strand covering the understanding and selection of adequate information from a range of sources. Message formulation in the IICD grid captures this dimension through its focus on content analysis and generation. Second, the presentation of information and ideas is mentioned in several different modes, adjusted to the intended audience, verbally as well as non-verbally. The message delivery component of the IICD grid focuses on these points. Third, the authors note the importance of communication technology and its advanced use. The IICD grid also covers the importance of technology use in its tools and techniques category, with, for example, an item that reads: students learn to effectively use a variety of formats of communication (social media, make a video, e-mail, letter writing, creating a document). Finally, Thompson ( 2020 ) describes the recognition of cultural and other differences as an important aspect of communication. The IICD grid aims at incorporating these aspects, notably in the meta-reflection category under each of the three dimensions.

5. Assessing the 4Cs in Informal Educational Contexts: The Example of Games

5.1. the 4cs in informal educational contexts.

So far, the focus has been on rather formal ways of nurturing the 4Cs. Although institutions and training programs are perhaps the most significant and necessary avenues of education, they are not the sole context in which 4Cs’ learning and improvement can manifest. One other important potential learning context is game play. Games are activities that are present and participated in throughout human society—by those of all ages, genders, and socio-economic statuses ( Bateson and Martin 2013 ; Huizinga 1949 ; Malaby 2007 ). This informal setting can also provide favorable conditions to help improve the 4Cs ( van Rosmalen et al. 2014 ) and should not be under-appreciated. Games provide a unique environment for learning, as they can foster a space to freely explore possibilities and one’s own potential ( de Freitas 2006 ). We argue that games are a significant potential pathway for the improvement of the 4Cs, and as such, they merit the same attention as more formal ways of learning and developing competencies.

5.2. 4Cs Evaluation Framework for Games

Compared to schools and educational institutions, the focus of IICD’s evaluation framework for games (see International Institute for Competency Development 2021 ) is more narrow. Thus, it is fundamentally different from the institutional grid: games, complex and deep as they can sometimes be, cannot directly be compared to the complexity of a school curriculum and all the programs it contains. The evaluation of a game’s effectiveness for training/improving a given C rests on the following principle: if a game presents affordances conducive to exercising a given skill, engaged playing of that game should help improve that skill.

The game’s evaluation grid is scored based on two criteria. For example, as a part of a game’s rating as a tool for the development of creativity, we determine the game must first meet two conditions. First, whether or not the game allows the opportunity for creativity to manifest itself: if creativity cannot occur in the game, it is obviously not eligible to receive ratings for that C. Second, whether or not creativity is needed in order to perform well in the game: if the players can win or achieve success in the game without needing creativity, this also means it cannot receive a rating for that C. If both conditions are met, however, the game will be considered potentially effective to improve creativity through the practice of certain components of creative behavior. This basic principle applies for all four of the Cs.

As outlined in Table 3 , below, the evaluation grid for each of the four Cs is composed of five components relevant to games that are different for each of the Cs. The grid works as follows: for each of the five components of each C, we evaluate the game on a list of sub-components using two yes/no scales: one for whether it is “possible” for that subcomponent to manifest and one for whether that sub-component is “required for success” in the game. This evaluation is done for all sub-components. After this, each general component is rated on the same two indicators. If 60% (i.e., three out of five) or more sub-components are positively rated as required, the general component is considered required. Then, the game is evaluated on its effectiveness for training and improving each of the 4Cs. If 60% or more components are positively rated as required, the game will be labelized as having the potential to be effective for training and improving the corresponding C.

Five different components evaluated for each C by the 4Cs assessment framework for games.

The evaluation grid for creativity is based on the multivariate model of creative potential (see Section 2.1.1 and Lubart et al. 2013 for more information) and is composed of four cognitive factors and one conative factor: originality , divergent thinking , convergent thinking , mental flexibility , and creative dispositions . Originality refers to the generation of ideas that are novel or unexpected, depending on the context. Divergent thinking corresponds to the generation of multiple ideas or solutions. Convergent thinking refers to the combination of multiple ideas and the selection of the most creative idea. Mental flexibility entails changing perspectives on a given problem and breaking away from initial ideas. Finally, creative dispositions concerns multiple personality-related factors conducive to creativity, such as openness to experience or risk taking.

The evaluation grid for critical thinking echoes Halpern’s ( 1998 ) as well as Marin and Halpern’s ( 2011 ) considerations for teaching this skill, that is, taking into consideration thinking skills, metacognition, and dispositions. The five components of the critical thinking grid are: goal-adequate discernment, objective thinking, metacognition, elaborate reasoning, and uncertainty management. Goal-adequate discernment entails the formulation of inferences and the discernment of contradictions when faced with a problem. Objective thinking corresponds to the suspension of one’s own judgment and the analysis of affirmations and sources in the most objective manner possible. Metacognition, here, is about questioning and reassessing information, as well as the awareness of one’s own cognitive biases. Elaborate reasoning entails reasoning in a way that is cautious, thorough, and serious. Finally, uncertainty management refers to the dispositional propensity to tolerate ambiguity and accept doubt.

The evaluation grid for collaboration is based on the quality of collaboration (QC) method ( Burkhardt et al. 2009 ; see Section 2.4.2 for more details) and is composed of the following five components: collaboration fluidity, well-argued deliberation and consensus-based decision, balance of contribution, organization and coordination, and cognitive syncing, input, and support. Collaboration fluidity entails the absence of speech overlap and the presence of a good flow in terms of turns to speak. Well-argued deliberation and consensus-based decision is about contributing to the discussion and task at hand, as well as participating in discussions and arguments, in order to obtain a consensus. Balance of contribution refers to having equal or equivalent contributions to organization, coordination, and decision making. Organization and coordination refers to effective management of roles, time, and “deadlines”, as well as the attribution of roles depending on participants’ skills. Finally, cognitive syncing, input, and support is about bringing ideas and resources to the group, as well as supporting and reinforcing other members of the group.

The five components used to evaluate communication in games include both linguistic, pragmatic, and social aspects. Linguistic skills per se are captured by the mastery of written and spoken language component. This component assesses language comprehension and the appropriate use of vocabulary. Pragmatic skills are captured by the verbal and non-verbal communication components and refer to the efficient use of verbal and body signals in the context of the game to achieve one’s communicative goals ( Grassmann 2014 ; Matthews 2014 ). Finally, the grid also evaluates social skills with its two last components, social interactions and social cognition, which, respectively, refer to the ability to interact with others appropriately—including by complying with the rules of the game—and to the understanding of other people’ mental states ( Tomasello 2005 ).

6. Discussion and Conclusions

Each of the 4Cs is a broad, multi-faceted concept that is the subject of a tremendous amount of research and discussion by a wide range of stakeholders in different disciplines, professions, and parts of the educational establishment. The development of evaluation frameworks to allow support for the 4Cs to be assessed and publicly recognized, using a label, is an important step for promoting and fostering these skills in educational contexts. As illustrated by IICD’s 4Cs Framework for educational institutions and programs, as well as its games/activities evaluation grid, the specific criteria to detect support for each C can vary depending upon the educational context (e.g., formal and institutional level or informal and at the activity level). Yet considering the 4Cs together highlights some additional observations, current challenges, and opportunities for the future that are worthy of discussion.

6.1. Interrelationships between the 4Cs and a New Model for Use in Pedagogy and Policy Promotion

One very important issue for understanding the 4Cs and their educational implementation that can be simultaneously a help and a hindrance for teaching them—and also a challenge when assessing them—is their multidimensionality and interrelatedness. In other words, the 4Cs are not entirely separate entities but instead, as Figure 2 shows, should be seen as four interlinked basic “elements” for future-oriented education that can help individuals in their learning process and, together, synergistically “bootstrap” the development of their cognitive potentials. Lamri and Lubart ( 2021 ), for example, found a certain base level of creativity was a necessary but not sufficient condition for success in managerial tasks, but that high-level performance required a combination of all four Cs. Some thinkers have argued that one cannot be creative without critical thinking, which also requires creativity, for example, to come up with alternative arguments (see Paul and Elder 2006 ). Similarly, among many other interrelationships, there is no collaboration without communication—and even ostensibly individual creativity is a “collaboration” of sorts with the general culture and precursors in a given field. As a result, it ranges from impossible to suboptimal to teach (or teach towards) one of the 4Cs without involving one or more of the others, and this commingling also underscores the genuine need and appropriateness of assessing them together.

“‘Crea-Critical-Collab-ication’: a Dynamic Interactionist Model of the 4Cs”. (Illustration of the interplay and interpenetration of creativity, critical thinking, collaboration, and communication shown in dimensional space according to their differing cognitive/individual vs. social/interpersonal emphases; (© 2023, Branden Thornhill-Miller. All Rights Reserved. thornhill-miller.com; accessed on 20 January 2023)).

From this perspective, Thornhill-Miller ( 2021 ) proposed a “dynamic interactionist model of the 4Cs” and their interrelated contributions to the future of education and work. Presented in Figure 2 , this model is meant to serve as a visual and conceptual aid for understanding the 4Cs and their interrelationships, thereby also promoting better use and understanding of them in pedagogical and policy settings. In addition to suggesting the portmanteau of “crea-critical thinking” as a new term to describe the overlap of much of the creative and critical thinking processes, the title of this model, “Crea-Critical-Collab-ication”, is a verbal representation of the fluid four-way interrelationship between the 4Cs visually represented in Figure 2 (a title meant to playfully repackage the 4Cs for important pedagogical and policy uses). This model goes further to suggest some dimensional differences in emphases that, roughly speaking, also often exist among the 4Cs: that is to say, the frequently greater emphasis on cognitive or individual elements at play in creativity and critical thinking in comparison to the social and interpersonal aspects more central to communication and collaboration ( Thornhill-Miller 2021 ).

Similarly focused on the need to promote a phase change towards future-oriented education, Lucas ( 2019 ) and colleagues have suggested conflating creative thinking and critical thinking in order to propose “3Cs” (creative thinking, communication, and collaboration) as new “foundational literacies” to symmetrically add to the 3Rs (Reading, wRiting, and aRithmetic) of previous educational eras. Although we applaud these efforts, from our applied research perspective, we believe that the individual importance of, and distinct differences between, creative thinking and critical thinking support preserving them both as separate constructs in order to encourage the greatest development of each of them. Moreover, if only three categories were somehow required or preferable, one could argue that uniting communication and collaboration (as “collab-ication” suggests) might be preferable—particularly also given the fact that substantial aspects of communication are already covered within the 3Rs. In any case, we look forward to more such innovations and collaborations in this vibrant and important area of work at the crossroads between research, pedagogy, and policy development.

6.2. Limitations and Future Work

The rich literature in each of the 4Cs domains shows the positive effects of integrating these dimensions into educational and professional curricula. At the same time, the complexity of their definitions makes them difficult to assess, both in terms of reliability (assessment must not vary from one measurement to another) and of validity (tests must measure that which they are intended to measure). However, applied research in this area is becoming increasingly rigorous, with a growing capacity to provide the necessary tools for evidence-based practice. The development of these practices should involve interdisciplinary teams of teachers and other educational practitioners who are equipped and trained accordingly. Similarly, on the research side, further exploration and clarification of subcomponents of the 4Cs and other related skills will be important. Recent efforts to clarify the conceptual overlap and hierarchical relations of soft skills for the future of education and work, for example, have been helpful and promising (e.g., Joie-La Marle et al. 2022 ; Lamri et al. 2022 ). But the most definitive sort of taxonomy and measurement model that we are currently lacking might only be established based on the large-scale administration of a comprehensive battery of skill-measuring psychometric tests on appropriate cross sections of society.

The rapid development and integration of new technologies will also aid and change the contexts, resources, and implementation of the 4Cs. For example, the recent developments make it clear that the 4Cs will be enhanced and changed by interaction with artificially intelligence, even as 4Cs-related skills will probably, for the same reason, increasingly constitute the core of available human work in the future (see, e.g., Ross 2018 ). Similarly, research on virtual reality and creativity suggest that VR environments assist and expand individual and collaborative creativity ( Bourgeois-Bougrine et al. 2022 ). Because VR technologies offer the possibility of enhanced and materially enriched communication, collaboration, and information availability, they not only allow for the enhancement of creativity techniques but also for similar expansions and improvements on almost all forms of human activity (see Thornhill-Miller and Dupont 2016 )—including the other three Cs.

6.3. Conclusion: Labelization of the 4Cs and the Future of Education and Work

Traditional educational approaches cannot meet the educational needs of our emergent societies if they do not teach, promote, and assess in line with the new learner characteristics and contexts of the 21st century ( Sahin 2009 ). The sort of future-oriented change and development required by this shift in institutional practices, programming, and structure will likely meet with significant resistance from comfortably entrenched (and often outdated) segments of traditional educational and training establishments. Additional external evaluation and monitoring is rarely welcome by workers in any context. We believe, however, that top-down processes from the innovative and competition-conscious administrative levels will be met by bottom-up demands from students and education consumers to support these institutional changes. And we contend that efforts such as labelizing 4C processes will serve to push educators and institutions towards more relevant offerings, oriented towards the future of work and helping build a more successful future for all.

In the end, the 4Cs framework seems to be a manageable, focused model for modernizing education, and one worthy of its growing prevalence in the educational and research marketplace for a number of reasons. These reasons include the complexity and cumbersome nature of larger alternative systems and the 4Cs’ persuasive presence at the core of a number of early and industry-driven frameworks. In addition, the 4Cs have benefitted from their subsequent promotion by organizations such as the OECD and the World Economic Forum, as well as some more direct support from recent empirical research. The promotion, teaching, and assessment of the 4Cs will require a complex social intervention and mobilization of educational resources—a major shift in pedagogy and institutional structures. Yet the same evolving digital technologies that have largely caused the need for these massive, rapid changes can also assist in the implementation of solutions ( van Laar et al. 2017 ). To the extent that future research also converges on such a model (that has already been found pedagogically useful and policy-friendly by so many individuals and organizations), the 4Cs framework has the potential to become a manageable core for 21st century skills and the future of education and work—one that stakeholders with various agendas can already begin building on for a better educational and economic future together.

Funding Statement

This research received no external funding.

Author Contributions

Conceptualization, B.T.-M. and T.L.; writing—original draft preparation, B.T.-M., A.C., M.M., J.-M.B., T.M., S.B.-B., S.E.H., F.V., M.A.-L., C.F., D.S., F.M.; writing—review and editing, B.T.-M., A.C., T.L., J.-M.B., C.F.; visualization, B.T.-M.; supervision, B.T.-M., T.L.; project administration, B.T.-M., T.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Data availability statement, conflicts of interest.

B.T.-M. and T.L. are unpaid academic co-founder and project collaborator for the International Institute for Competency Development, whose labelization frameworks (developed in cooperation with Afnor International and the LaPEA lab of Université Paris Cité and Université Gustave Eiffel) are used as examples in this review. S.E.H. and M.A.-L. are employees of AFNOR International. No funding was received to support this research or article, which reflects the views of the scientists and researchers and not their organizations or companies.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

• Abrami Philip C., Bernard Robert M., Borokhovski Eugene, Waddington David I., Wade C. Anne, Persson Tonje. Strategies for Teaching Students to Think Critically: A Meta-Analysis. Review of Educational Research. 2015; 85 :275–314. doi: 10.3102/0034654314551063. [ CrossRef ] [ Google Scholar ]
• AbuSeileek Ali Farhan. The Effect of Computer-Assisted Cooperative Learning Methods and Group Size on the EFL Learners’ Achievement in Communication Skills. Computers & Education. 2012; 58 :231–39. doi: 10.1016/j.compedu.2011.07.011. [ CrossRef ] [ Google Scholar ]
• Ahern Aoife, Dominguez Caroline, McNally Ciaran, O’Sullivan John J., Pedrosa Daniela. A Literature Review of Critical Thinking in Engineering Education. Studies in Higher Education. 2019; 44 :816–28. doi: 10.1080/03075079.2019.1586325. [ CrossRef ] [ Google Scholar ]
• Ainsworth Shaaron E., Chounta Irene-Angelica. The roles of representation in computer-supported collaborative learning. In: Cress Ulrike, Rosé Carolyn, Wise Alyssa Friend, Oshima Jun., editors. International Handbook of Computer-Supported Collaborative Learning. Springer; Cham: 2021. pp. 353–69. [ CrossRef ] [ Google Scholar ]
• Alsaleh Nada J. Teaching Critical Thinking Skills: Literature Review. [(accessed on 1 November 2022)]; The Turkish Online Journal of Educational Technology. 2020 19 :21–39. Available online: http://files.eric.ed.gov/fulltext/EJ1239945.pdf [ Google Scholar ]
• Al-Samarraie Hosam, Hurmuzan Shuhaila. A Review of Brainstorming Techniques in Higher Education. Thinking Skills and Creativity. 2018; 27 :78–91. doi: 10.1016/j.tsc.2017.12.002. [ CrossRef ] [ Google Scholar ]
• Amabile Teresa M. Social Psychology of Creativity: A Consensual Assessment Technique. Journal of Personality and Social Psychology. 1982; 43 :997–1013. doi: 10.1037/0022-3514.43.5.997. [ CrossRef ] [ Google Scholar ]
• Amron Manajemen Pemasaran. The influence of brand image, brand trust, product quality, and price on the consumer’s buying decision of MPV cars. European Scientific Journal. 2018; 14 :228–39. doi: 10.19044/esj.2018.v14n13p228. [ CrossRef ] [ Google Scholar ]
• Ananiadoui Katerina, Claro Magdalean. 21st Century Skills and Competences for New Millennium Learners in OECD Countries. OECD Publishing; Paris: 2009. OECD Education Working Papers, No. 41. [ CrossRef ] [ Google Scholar ]
• Bailin Sharon. Achieving Extraordinary Ends: An Essay on Creativity. Springer; Dordrecht: 1988. [ CrossRef ] [ Google Scholar ]
• Bandyopadhyay Subir, Szostek Jana. Thinking Critically about Critical Thinking: Assessing Critical Thinking of Business Students Using Multiple Measures. Journal of Education for Business. 2019; 94 :259–70. doi: 10.1080/08832323.2018.1524355. [ CrossRef ] [ Google Scholar ]
• Barber Herbert F. Developing Strategic Leadership: The US Army War College Experience. Journal of Management Development. 1992; 11 :4–12. doi: 10.1108/02621719210018208. [ CrossRef ] [ Google Scholar ]
• Barnett Ronald. The Palgrave Handbook of Critical Thinking in Higher Education. Palgrave Macmillan US; New York: 2015. A Curriculum for Critical Being; pp. 63–76. [ CrossRef ] [ Google Scholar ]
• Bateson Patrick, Martin Paul. Play, Playfulness, Creativity and Innovation. Cambridge University Press; Cambridge: 2013. [ CrossRef ] [ Google Scholar ]
• Batey Mark. The Measurement of Creativity: From Definitional Consensus to the Introduction of a New Heuristic Framework. Creativity Research Journal. 2012; 24 :55–65. doi: 10.1080/10400419.2012.649181. [ CrossRef ] [ Google Scholar ]
• Battelle for Kids Framework for 21st Century Learning Definitions. 2022. [(accessed on 1 November 2022)]. Available online: http://static.battelleforkids.org/documents/p21/P21_Framework_DefinitionsBFK.pdf
• Bellaera Lauren, Weinstein-Jones Yana, Ilie Sonia, Baker Sara T. Critical Thinking in Practice: The Priorities and Practices of Instructors Teaching in Higher Education. Thinking Skills and Creativity. 2021; 41 :100856. doi: 10.1016/j.tsc.2021.100856. [ CrossRef ] [ Google Scholar ]
• Blessinger Patrick, Anchan John P. In: Democratizing Higher Education: International Comparative Perspectives. 1st ed. Blessinger Patrick, Anchan John P., editors. Routledge; London: 2015. [(accessed on 1 November 2022)]. Available online: https://www.routledge.com/Democratizing-Higher-Education-International-Comparative-Perspectives/Blessinger-Anchan/p/book/9781138020955 [ Google Scholar ]
• Bloom Benjamin Samuel., editor. Taxonomy of Educational Objectives: The Classification of Educational Goals: Handbook I, Cognitive Domain. Longmans; New York: 1956. [ Google Scholar ]
• Bourgeois-Bougrine Samira. The Palgrave Encyclopedia of the Possible. Springer International Publishing; Cham: 2022. Design Thinking. [ CrossRef ] [ Google Scholar ]
• Bourgeois-Bougrine Samira, Bonnardel Nathalie, Burkhardt Jean-Marie, Thornhill-Miller Branden, Pahlavan Farzaneh, Buisine Stéphanie, Guegan Jérôme, Pichot Nicolas, Lubart Todd. Immersive Virtual Environments’ Impact on Individual and Collective Creativity: A Review of Recent Research. European Psychologist. 2022; 27 :237–53. doi: 10.1027/1016-9040/a000481. [ CrossRef ] [ Google Scholar ]
• Bourke Sharon L., Cooper Simon, Lam Louisa, McKenna Lisa. Undergraduate Health Professional Students’ Team Communication in Simulated Emergency Settings: A Scoping Review. Clinical Simulation in Nursing. 2021; 60 :42–63. doi: 10.1016/j.ecns.2021.07.004. [ CrossRef ] [ Google Scholar ]
• Brookfield Stephen D. Assessing Critical Thinking. New Directions for Adult and Continuing Education. 1997; 75 :17–29. doi: 10.1002/ace.7502. [ CrossRef ] [ Google Scholar ]
• Burkhardt Jean-Marie, Détienne Françoise, Hébert Anne-Marie, Perron Laurence. Human-Computer Interaction—INTERACT 2009. Springer; Berlin/Heidelberg: 2009. Assessing the ‘Quality of Collaboration’ in Technology-Mediated Design Situations with Several Dimensions; pp. 157–60. [ CrossRef ] [ Google Scholar ]
• Camarda Anaëlle, Bouhours Lison, Osmont Anaïs, Masson Pascal Le, Weil Benoît, Borst Grégoire, Cassotti Mathieu. Opposite Effect of Social Evaluation on Creative Idea Generation in Early and Middle Adolescents. Creativity Research Journal. 2021; 33 :399–410. doi: 10.1080/10400419.2021.1902174. [ CrossRef ] [ Google Scholar ]
• Cannon-Bowers Janis, Tannenbaum Scott I., Salas Eduardo, Volpe Catherine E. Defining team competencies and establishing team training requirements. In: Guzzo Richard A., Salas Eduardo., editors. Team Effectiveness and Decision Making in Organizations. Jossey-Bass; San Francisco: 1995. pp. 333–80. [ Google Scholar ]
• Care Esther, Scoular Claire, Griffin Patrick. Assessment of Collaborative Problem Solving in Education Environments. Applied Measurement in Education. 2016; 29 :250–64. doi: 10.1080/08957347.2016.1209204. [ CrossRef ] [ Google Scholar ]
• Care Esther, Kim Helyn, Vista Alvin, Anderson Kate. Education System Alignment for 21st Century Skills: Focus on Assessment. Brookings Institution; Washington, DC: 2018. [ Google Scholar ]
• Carmichael Erst, Farrell Helen. Evaluation of the Effectiveness of Online Resources in Developing Student Critical Thinking: Review of Literature and Case Study of a Critical Thinking Online Site. Journal of University Teaching and Learning Practice. 2012; 9 :38–55. doi: 10.53761/1.9.1.4. [ CrossRef ] [ Google Scholar ]
• Carson Shelley H., Peterson Jordan B., Higgins Daniel M. Reliability, Validity, and Factor Structure of the Creative Achievement Questionnaire. Creativity Research Journal. 2005; 17 :37–50. doi: 10.1207/s15326934crj1701_4. [ CrossRef ] [ Google Scholar ]
• Casey Betty J., Getz Sarah, Galvan Adriana. The Adolescent Brain. Developmental Review: DR. 2008; 28 :62–77. doi: 10.1016/j.dr.2007.08.003. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cassotti Mathieu, Camarda Anaëlle, Poirel Nicolas, Houdé Olivier, Agogué Marine. Fixation Effect in Creative Ideas Generation: Opposite Impacts of Example in Children and Adults. Thinking Skills and Creativity. 2016; 19 :146–52. doi: 10.1016/j.tsc.2015.10.008. [ CrossRef ] [ Google Scholar ]
• Chameroy Fabienne, Veran Lucien. Immatérialité de La Qualité et Effet Des Labels Sur Le Consentement à Payer. Management International. 2014; 18 :32–44. doi: 10.7202/1025088ar. [ CrossRef ] [ Google Scholar ]
• Chiu Fa-Chung. Improving Your Creative Potential without Awareness: Overinclusive Thinking Training. Thinking Skills and Creativity. 2015; 15 :1–12. doi: 10.1016/j.tsc.2014.11.001. [ CrossRef ] [ Google Scholar ]
• Chulvi Vicente, Mulet Elena, Chakrabarti Amaresh, López-Mesa Belinda, González-Cruz Carmen. Comparison of the Degree of Creativity in the Design Outcomes Using Different Design Methods. Journal of Engineering Design. 2012; 23 :241–69. doi: 10.1080/09544828.2011.624501. [ CrossRef ] [ Google Scholar ]
• Cinque Maria. ‘Lost in Translation’. Soft Skills Development in European Countries. Tuning Journal for Higher Education. 2016; 3 :389–427. doi: 10.18543/tjhe-3(2)-2016pp389-427. [ CrossRef ] [ Google Scholar ]
• Cömert Musa, Zill Jördis Maria, Christalle Eva, Dirmaier Jörg, Härter Martin, Scholl Isabelle. Assessing Communication Skills of Medical Students in Objective Structured Clinical Examinations (OSCE) - A Systematic Review of Rating Scales. PLoS ONE. 2016; 11 :e0152717. doi: 10.1371/journal.pone.0152717. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Corazza Giovanni Emanuele. Potential Originality and Effectiveness: The Dynamic Definition of Creativity. Creativity Research Journal. 2016; 28 :258–67. doi: 10.1080/10400419.2016.1195627. [ CrossRef ] [ Google Scholar ]
• Corazza Giovanni Emanuele, Darbellay Frédéric, Lubart Todd, Panciroli Chiara. Developing Intelligence and Creativity in Education: Insights from the Space–Time Continuum. In: Lemmetty Soila, Collin Kaija, Glăveanu Vlad, Forsman Panu., editors. Creativity and Learning. Springer International Publishing; Cham: 2021. pp. 69–87. [ CrossRef ] [ Google Scholar ]
• Cotter Katherine N., Beghetto Ronald A., Kaufman James C. Creativity in the Classroom: Advice for Best Practices. In: Lubart Todd, Botella Marion, Bourgeois-Bougrine Samira, Caroff Xavier, Guégan Jérôme, Mouchiroud Christohe, Nelson Julien, Zenasni Franck., editors. Homo Creativus. Springer International Publishing; Cham: 2022. pp. 249–64. [ CrossRef ] [ Google Scholar ]
• Curtis J. Randall, Back Anthony L., Ford Dee W., Downey Lois, Shannon Sarah E., Doorenbos Ardith Z., Kross Erin K., Reinke Lynn F., Feemster Laura C., Edlund Barbara, et al. Effect of Communication Skills Training for Residents and Nurse Practitioners on Quality of Communication with Patients with Serious Illness: A Randomized Trial. JAMA: The Journal of the American Medical Association. 2013; 310 :2271. doi: 10.1001/jama.2013.282081. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D’Alimonte Laura, McLaney Elizabeth, Prospero Lisa Di. Best Practices on Team Communication: Interprofessional Practice in Oncology. Current Opinion in Supportive and Palliative Care. 2019; 13 :69–74. doi: 10.1097/SPC.0000000000000412. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• de Freitas Sara. Learning in Immersive Worlds: A Review of Game-Based Learning. JISC; Bristol: 2006. [(accessed on 1 November 2022)]. Available online: http://www.jisc.ac.uk/media/documents/programmes/elearninginnovation/gamingreport_v3.pdf [ Google Scholar ]
• Détienne Françoise, Baker Michael, Burkhardt Jean-Marie. Perspectives on Quality of Collaboration in Design. CoDesign. 2012; 8 :197–99. doi: 10.1080/15710882.2012.742350. [ CrossRef ] [ Google Scholar ]
• Diedrich Jennifer, Jauk Emanuel, Silvia Paul J., Gredlein Jeffrey M., Neubauer Aljoscha C., Benedek Mathias. Assessment of Real-Life Creativity: The Inventory of Creative Activities and Achievements (ICAA) Psychology of Aesthetics, Creativity, and the Arts. 2018; 12 :304–16. doi: 10.1037/aca0000137. [ CrossRef ] [ Google Scholar ]
• Doyle Denise. Creativity in the Twenty First Century. Edited by Anna Hui and Christian Wagner. Springer International Publishing; Cham: 2021. Creative and Collaborative Practices in Virtual Immersive Environments; pp. 3–19. [ CrossRef ] [ Google Scholar ]
• Drisko James W. Competencies and Their Assessment. Journal of Social Work Education. 2014; 50 :414–26. doi: 10.1080/10437797.2014.917927. [ CrossRef ] [ Google Scholar ]
• Dul Jan, Ceylan Canan. Work Environments for Employee Creativity. Ergonomics. 2011; 54 :12–20. doi: 10.1080/00140139.2010.542833. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Dumitru Daniela, Bigu Dragos, Elen Jan, Ahern Aoife, McNally Ciaran, O’Sullivan John. A European Review on Critical Thinking Educational Practices in Higher Education Institutions. UTAD; Vila Real: 2018. [(accessed on 2 November 2022)]. Available online: http://repositorio.utad.pt/handle/10348/8320 [ Google Scholar ]
• Edelman Jonathan, Owoyele Babajide, Santuber Joaquin. Design Thinking in Education. Springer International Publishing; Cham: 2022. Beyond Brainstorming: Introducing Medgi, an Effective, Research-Based Method for Structured Concept Development; pp. 209–32. [ CrossRef ] [ Google Scholar ]
• Etilé Fabrice, Teyssier Sabrina. Signaling Corporate Social Responsibility: Third-Party Certification versus Brands: Signaling CSR: Third-Party Certification versus Brands. The Scandinavian Journal of Economics. 2016; 118 :397–432. doi: 10.1111/sjoe.12150. [ CrossRef ] [ Google Scholar ]
• Evans Carla. Measuring Student Success Skills: A Review of the Literature on Collaboration. National Center for the Improvement of Educational Assessment; Dover: 2020. [ Google Scholar ]
• Facione Peter Arthur. The California Critical Thinking Skills Test–College Level. Technical Report# 1. Experimental Validation and Content Validity. [(accessed on 2 November 2022)]; 1990a Available online: https://files.eric.ed.gov/fulltext/ED327549.pdf
• Facione Peter Arthur. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Research Findings and Recommendations. ERIC, Institute of Education Sciences; Washington, DC: 1990b. [(accessed on 2 November 2022)]. pp. 1–112. Available online: https://eric.ed.gov/?id=ED315423 [ Google Scholar ]
• Facione Peter Arthur. Critical thinking: What it is and why it counts. Insight Assessment. 2011; 2007 :1–23. [ Google Scholar ]
• Faidley Joel. Ph.D. dissertation. East Tennessee State University; Johnson City, TN, USA: 2018. Comparison of Learning Outcomes from Online and Face-to-Face Accounting Courses. [ Google Scholar ]
• Friedman Hershey H. Cognitive Biases That Interfere with Critical Thinking and Scientific Reasoning: A Course Module. SSRN Electronic Journal. 2017:1–60. doi: 10.2139/ssrn.2958800. [ CrossRef ] [ Google Scholar ]
• Fryer-Edwards Kelly, Arnold Robert M., Baile Walter, Tulsky James A., Petracca Frances, Back Anthony. Reflective Teaching Practices: An Approach to Teaching Communication Skills in a Small-Group Setting. Academic Medicine: Journal of the Association of American Medical Colleges. 2006; 81 :638–44. doi: 10.1097/01.ACM.0000232414.43142.45. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Glăveanu Vlad Petre. Rewriting the Language of Creativity: The Five A’s Framework. Review of General Psychology: Journal of Division 1, of the American Psychological Association. 2013; 17 :69–81. doi: 10.1037/a0029528. [ CrossRef ] [ Google Scholar ]
• Glăveanu Vlad Petre. The Psychology of Creativity: A Critical Reading. Creativity Theories Research Applications. 2014; 1 :10–32. doi: 10.15290/ctra.2014.01.01.02. [ CrossRef ] [ Google Scholar ]
• Goldenberg Olga, Wiley Jennifer. Quality, Conformity, and Conflict: Questioning the Assumptions of Osborn’s Brainstorming Technique. The Journal of Problem Solving. 2011; 3 :96–118. doi: 10.7771/1932-6246.1093. [ CrossRef ] [ Google Scholar ]
• Graesser Arthur C., Sabatini John P., Li Haiying. Educational Psychology Is Evolving to Accommodate Technology, Multiple Disciplines, and Twenty-First-Century Skills. Annual Review of Psychology. 2022; 73 :547–74. doi: 10.1146/annurev-psych-020821-113042. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Graesser Arthur C., Fiore Stephen M., Greiff Samuel, Andrews-Todd Jessica, Foltz Peter W., Hesse Friedrich W. Advancing the Science of Collaborative Problem Solving. Psychological Science in the Public Interest. 2018; 19 :59–92. doi: 10.1177/1529100618808244. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grassmann Susanne. The pragmatics of word learning. In: Matthews Danielle., editor. Pragmatic Development in First Language Acquisition. John Benjamins Publishing Company; Amsterdam: 2014. pp. 139–60. [ CrossRef ] [ Google Scholar ]
• Hager Keri, St Hill Catherine, Prunuske Jacob, Swanoski Michael, Anderson Grant, Lutfiyya May Nawal. Development of an Interprofessional and Interdisciplinary Collaborative Research Practice for Clinical Faculty. Journal of Interprofessional Care. 2016; 30 :265–67. doi: 10.3109/13561820.2015.1092951. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Halpern Diane F. Teaching Critical Thinking for Transfer across Domains: Disposition, Skills, Structure Training, and Metacognitive Monitoring. The American Psychologist. 1998; 53 :449–55. doi: 10.1037/0003-066X.53.4.449. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Halpern Diane F., Dunn Dana S. Critical Thinking: A Model of Intelligence for Solving Real-World Problems. Journal of Intelligence. 2021; 9 :22. doi: 10.3390/jintelligence9020022. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hanover Research A Crosswalk of 21st Century Skills. 2012. [(accessed on 15 August 2022)]. Available online: http://www.hanoverresearch.com/wp-content/uploads/2011/12/A-Crosswalk-of-21st-Century-Skills-Membership.pdf
• Hathaway Julia R., Tarini Beth A., Banerjee Sushmita, Smolkin Caroline O., Koos Jessica A., Pati Susmita. Healthcare Team Communication Training in the United States: A Scoping Review. Health Communication. 2022:1–26. doi: 10.1080/10410236.2022.2036439. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hesse Friedrich, Care Esther, Buder Juergen, Sassenberg Kai, Griffin Patrick. A Framework for Teachable Collaborative Problem Solving Skills. In: Griffin Patrick, Care Esther., editors. Assessment and Teaching of 21st Century Skills. Springer Netherlands; Dordrecht: 2015. pp. 37–56. [ Google Scholar ]
• Hitchcock David. Critical Thinking. In: Edward Nouri Zalta., editor. The Stanford Encyclopedia of Philosophy (Fall 2020 Edition) Stanford University; Stanford: 2020. [ Google Scholar ]
• Houdé Olivier. Inhibition and cognitive development: Object, number, categorization, and reasoning. Cognitive Development. 2000; 15 :63–73. doi: 10.1016/S0885-2014(00)00015-0. [ CrossRef ] [ Google Scholar ]
• Houdé Olivier, Borst Grégoire. Measuring inhibitory control in children and adults: Brain imaging and mental chronometry. Frontiers in Psychology. 2014; 5 :616. doi: 10.3389/fpsyg.2014.00616. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Huber Christopher R., Kuncel Nathan R. Does College Teach Critical Thinking? A Meta-Analysis. Review of Educational Research. 2016; 86 :431–68. doi: 10.3102/0034654315605917. [ CrossRef ] [ Google Scholar ]
• Huizinga Johan. Homo Ludens: A Study of the Play-Elements in Culture. Routledge; London: 1949. [ Google Scholar ]
• Humphrey Neil, Curran Andrew, Morris Elisabeth, Farrell Peter, Woods Kevin. Emotional Intelligence and Education: A Critical Review. Educational Psychology. 2007; 27 :235–54. doi: 10.1080/01443410601066735. [ CrossRef ] [ Google Scholar ]
• International Institute for Competency Development 21st Century Skills 4Cs Labelization. 2021. [(accessed on 2 November 2022)]. Available online: https://icd-hr21.org/offers/21st-century-competencies/
• Jackson Denise. Business Graduate Performance in Oral Communication Skills and Strategies for Improvement. The International Journal of Management Education. 2014; 12 :22–34. doi: 10.1016/j.ijme.2013.08.001. [ CrossRef ] [ Google Scholar ]
• Jahn Gabriele, Schramm Matthias, Spiller Achim. The Reliability of Certification: Quality Labels as a Consumer Policy Tool. Journal of Consumer Policy. 2005; 28 :53–73. doi: 10.1007/s10603-004-7298-6. [ CrossRef ] [ Google Scholar ]
• Jauk Emanuel, Benedek Mathias, Neubauer Aljoscha C. The Road to Creative Achievement: A Latent Variable Model of Ability and Personality Predictors. European Journal of Personality. 2014; 28 :95–105. doi: 10.1002/per.1941. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Joie-La Marle Chantal, Parmentier François, Coltel Morgane, Lubart Todd, Borteyrou Xavier. A Systematic Review of Soft Skills Taxonomies: Descriptive and Conceptual Work. 2022. [(accessed on 2 November 2022)]. Available online: [ CrossRef ]
• Jones Stanley E., LeBaron Curtis D. Research on the Relationship between Verbal and Nonverbal Communication: Emerging Integrations. The Journal of Communication. 2002; 52 :499–521. doi: 10.1111/j.1460-2466.2002.tb02559.x. [ CrossRef ] [ Google Scholar ]
• Kaendler Celia, Wiedmann Michael, Leuders Timo, Rummel Nikol, Spada Hans. Monitoring Student Interaction during Collaborative Learning: Design and Evaluation of a Training Program for Pre-Service Teachers. Psychology Learning & Teaching. 2016; 15 :44–64. doi: 10.1177/1475725716638010. [ CrossRef ] [ Google Scholar ]
• Kahneman Daniel. A Perspective on Judgment and Choice: Mapping Bounded Rationality. The American Psychologist. 2003; 58 :697–720. doi: 10.1037/0003-066X.58.9.697. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kahneman Daniel. Thinking, Fast and Slow. Macmillan; New York: 2011. [ Google Scholar ]
• Karl Katherine A., Peluchette Joy V., Aghakhani Navid. Virtual Work Meetings during the COVID-19 Pandemic: The Good, Bad, and Ugly. Small Group Research. 2022; 53 :343–65. doi: 10.1177/10464964211015286. [ CrossRef ] [ Google Scholar ]
• Keefer Kateryna V., Parker James D. A., Saklofske Donald H. The Springer Series on Human Exceptionality. Springer International Publishing; Cham: 2018. Three Decades of Emotional Intelligence Research: Perennial Issues, Emerging Trends, and Lessons Learned in Education: Introduction to Emotional Intelligence in Education; pp. 1–19. [ Google Scholar ]
• Kemp Nenagh, Grieve Rachel. Face-to-Face or Face-to-Screen? Undergraduates’ Opinions and Test Performance in Classroom vs. Online Learning. Frontiers in Psychology. 2014; 5 :1278. doi: 10.3389/fpsyg.2014.01278. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kimery Kathryn, McCord Mary. Third-Party Assurances: Mapping the Road to Trust in E-retailing. The Journal of Information Technology Theory and Application. 2002; 4 :63–82. [ Google Scholar ]
• Kohn Nicholas W., Smith Steven M. Collaborative Fixation: Effects of Others’ Ideas on Brainstorming. Applied Cognitive Psychology. 2011; 25 :359–71. doi: 10.1002/acp.1699. [ CrossRef ] [ Google Scholar ]
• Kowaltowski Doris C. C. K., Bianchi Giovana, de Paiva Valéria Teixeira. Methods That May Stimulate Creativity and Their Use in Architectural Design Education. International Journal of Technology and Design Education. 2010; 20 :453–76. doi: 10.1007/s10798-009-9102-z. [ CrossRef ] [ Google Scholar ]
• Kruijver Irma P. M., Kerkstra Ada, Francke Anneke L., Bensing Jozien M., van de Wiel Harry B. M. Evaluation of Communication Training Programs in Nursing Care: A Review of the Literature. Patient Education and Counseling. 2000; 39 :129–45. doi: 10.1016/S0738-3991(99)00096-8. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lai Emily R. Critical thinking: A literature review. Pearson’s Research Reports. 2011; 6 :40–41. doi: 10.25148/lawrev.11.2.3. [ CrossRef ] [ Google Scholar ]
• Lamri Jérémy, Lubart Todd. Creativity and Its’ Relationships with 21st Century Skills in Job Performance. Kindai Management Review. 2021; 9 :75–91. [ Google Scholar ]
• Lamri Jérémy, Barabel Michel, Meier Olivier, Lubart Todd. Le Défi Des Soft Skills: Comment les Développer au XXIe Siècle? Dunod; Paris: 2022. [ Google Scholar ]
• Landa Rebecca J. Assessment of Social Communication Skills in Preschoolers: Assessing Social Communication Skills in Children. Mental Retardation and Developmental Disabilities Research Reviews. 2005; 11 :247–52. doi: 10.1002/mrdd.20079. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lee Sang M., Choi Jeongil, Lee Sang-Gun. The impact of a third-party assurance seal in customer purchasing intention. Journal of Internet Commerce. 2004; 3 :33–51. doi: 10.1300/J179v03n02_03. [ CrossRef ] [ Google Scholar ]
• Lewis Arthur, Smith David. Defining Higher Order Thinking. Theory into Practice. 1993; 32 :131–37. doi: 10.1080/00405849309543588. [ CrossRef ] [ Google Scholar ]
• Liu Ou Lydia, Frankel Lois, Roohr Katrina Crotts. Assessing Critical Thinking in Higher Education: Current State and Directions for next-Generation Assessment: Assessing Critical Thinking in Higher Education. ETS Research Report Series. 2014; 2014 :1–23. doi: 10.1002/ets2.12009. [ CrossRef ] [ Google Scholar ]
• Lubart Todd. The 7 C’s of Creativity. The Journal of Creative Behavior. 2017; 51 :293–96. doi: 10.1002/jocb.190. [ CrossRef ] [ Google Scholar ]
• Lubart Todd, Thornhill-Miller Branden. Creativity: An Overview of the 7C’s of Creative Thought. Heidelberg: Heidelberg University Publishing. 2019 doi: 10.17885/HEIUP.470.C6678. [ CrossRef ] [ Google Scholar ]
• Lubart Todd, Barbot Baptiste, Besançon Maud. Creative Potential: Assessment Issues and the EPoC Battery/Potencial Creativo: Temas de Evaluación y Batería EPoC. Estudios de Psicologia. 2019; 40 :540–62. doi: 10.1080/02109395.2019.1656462. [ CrossRef ] [ Google Scholar ]
• Lubart Todd, Zenasni Franck, Barbot Baptiste. Creative potential and its measurement. International Journal of Talent Development and Creativity. 2013; 1 :41–51. [ Google Scholar ]
• Lubart Tubart, Thornhill-Miller Branden. Creativity in Law: Legal Professions and the Creative Profiler Approach. In: Masson Antoine, Robinson Gavin., editors. Mapping Legal Innovation: Trends and Perspectives. Springer International Publishing; Cham: 2021. pp. 1–19. [ CrossRef ] [ Google Scholar ]
• Lubin Jeffrey, Hendrick Stephan, Thornhill-Miller Branden, Mercier Maxence, Lubart Todd. Creativity in Solution-Focused Brief Therapy Forthcoming.
• Lucas Bill. Why We Need to Stop Talking about Twenty-First Century Skills. Centre for Strategic Education; Melbourne: 2019. [ Google Scholar ]
• Lucas Bill. Creative Thinking in Schools across the World. The Global Institute of Creative Thinking; London: 2022. [ Google Scholar ]
• Lucas Bill, Claxton Guy. Wider Skills for Learning: What Are They, How Can They Be Cultivated, How Could They Be Measured and Why Are They Important for Innovation? NESTA; London: 2009. [ Google Scholar ]
• Malaby Thomas M. Beyond Play: A New Approach to Games. Games and Culture. 2007; 2 :95–113. doi: 10.1177/1555412007299434. [ CrossRef ] [ Google Scholar ]
• Marin Lisa M., Halpern Diane F. Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity. 2011; 6 :1–13. doi: 10.1016/j.tsc.2010.08.002. [ CrossRef ] [ Google Scholar ]
• Mathieu John E., Hollenbeck John R., van Knippenberg Daan, Ilgen Daniel R. A Century of Work Teams in the Journal of Applied Psychology. The Journal of Applied Psychology. 2017; 102 :452–67. doi: 10.1037/apl0000128. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Matthews Danielle. Pragmatic Development in First Language Acquisition. Amsterdam: John Benjamins Publishing Company. 2014 doi: 10.1075/tilar.10. [ CrossRef ] [ Google Scholar ]
• McDonald Skye, Gowland Alison, Randall Rebekah, Fisher Alana, Osborne-Crowley Katie, Honan Cynthia. Cognitive Factors Underpinning Poor Expressive Communication Skills after Traumatic Brain Injury: Theory of Mind or Executive Function? Neuropsychology. 2014; 28 :801–11. doi: 10.1037/neu0000089. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Moore Brooke Noel, Parker Richard. Critical Thinking. 20th ed. McGraw-Hill Education; New York: 2016. [ Google Scholar ]
• Morreale Sherwyn P., Valenzano Joseph M., Bauer Janessa A. Why Communication Education Is Important: A Third Study on the Centrality of the Discipline’s Content and Pedagogy. Communication Education. 2017; 66 :402–22. doi: 10.1080/03634523.2016.1265136. [ CrossRef ] [ Google Scholar ]
• Mourad Maha. Quality Assurance as a Driver of Information Management Strategy: Stakeholders’ Perspectives in Higher Education. Journal of Enterprise Information Management. 2017; 30 :779–94. doi: 10.1108/JEIM-06-2016-0104. [ CrossRef ] [ Google Scholar ]
• National Education Association . Preparing 21st Century Students for a Global Society: An Educator’s Guide to the “Four Cs”. National Education Association; Alexandria: 2011. [ Google Scholar ]
• Nouri Jalal, Åkerfeldt Anna, Fors Uno, Selander Staffan. Assessing Collaborative Problem Solving Skills in Technology-Enhanced Learning Environments—The PISA Framework and Modes of Communication. International Journal of Emerging Technologies in Learning (IJET) 2017; 12 :163. doi: 10.3991/ijet.v12i04.6737. [ CrossRef ] [ Google Scholar ]
• O’Carroll Veronica, Owens Melissa, Sy Michael, El-Awaisi Alla, Xyrichis Andreas, Leigh Jacqueline, Nagraj Shobhana, Huber Marion, Hutchings Maggie, McFadyen Angus. Top Tips for Interprofessional Education and Collaborative Practice Research: A Guide for Students and Early Career Researchers. Journal of Interprofessional Care. 2021; 35 :328–33. doi: 10.1080/13561820.2020.1777092. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• OECD . PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. OECD Publishing; Paris: 2017. PISA 2015 collaborative problem-solving framework. [ CrossRef ] [ Google Scholar ]
• OECD . Framework for the Assessment of Creative Thinking in PISA 2021: Third Draft. OECD; Paris: 2019a. [(accessed on 2 November 2022)]. Available online: https://www.oecd.org/pisa/publications/PISA-2021-creative-thinking-framework.pdf [ Google Scholar ]
• OECD . Future of Education and Skills 2030: A Series of Concept Notes. OECD Learning Compass; Paris: 2019b. [(accessed on 2 November 2022)]. Available online: https://www.oecd.org/education/2030-project/teaching-and-learning/learning/learning-compass-2030/OECD_Learning_Compass_2030_Concept_Note_Series.pdf [ Google Scholar ]
• Osborn A. F. Applied Imagination. Charles Scribner’s Sons; New York: 1953. [ Google Scholar ]
• Parkinson Thomas L. The Role of Seals and Certifications of Approval in Consumer Decision-Making. The Journal of Consumer Affairs. 1975; 9 :1–14. doi: 10.1111/j.1745-6606.1975.tb00545.x. [ CrossRef ] [ Google Scholar ]
• Partnership for 21st Century Skills . 21st Century Skills Education and Competitiveness: A Resource and Policy Guide. Partnership for 21st Century Skills; Tuscon: 2008. [ Google Scholar ]
• Pasquinelli Elena, Bronner Gérald. Éduquer à l’esprit critique. Bases théoriques et indications pratiques pour l’enseignement et la formation. Ministère de l’Éducation Nationale, de la JEUNESSE et des Sports; Paris: 2021. Rapport du Conseil Scientifique de l’Éducation Nationale. [ Google Scholar ]
• Pasquinelli Elena, Farina Mathieu, Bedel Audrey, Casati Roberto. Naturalizing Critical Thinking: Consequences for Education, Blueprint for Future Research in Cognitive Science. Mind, Brain and Education: The Official Journal of the International Mind, Brain, and Education Society. 2021; 15 :168–76. doi: 10.1111/mbe.12286. [ CrossRef ] [ Google Scholar ]
• Paul Richard, Elder Linda. Critical thinking: The nature of critical and creative thought. Journal of Developmental Education. 2006; 30 :34–35. [ Google Scholar ]
• Paulus Paul B., Yang Huei-Chuan. Idea Generation in Groups: A Basis for Creativity in Organizations. Organizational Behavior and Human Decision Processes. 2000; 82 :76–87. doi: 10.1006/obhd.2000.2888. [ CrossRef ] [ Google Scholar ]
• Paulus Paul B., Kenworthy Jared B. Effective brainstorming. In: Paulus Paul B., Nijstad Bernard A., editors. The Oxford Handbook of Group Creativity and Innovation. Oxford University Press; New York: 2019. [ CrossRef ] [ Google Scholar ]
• Paulus Paul B., Dzindolet Mary T. Social Influence Processes in Group Brainstorming. Journal of Personality and Social Psychology. 1993; 64 :575–86. doi: 10.1037/0022-3514.64.4.575. [ CrossRef ] [ Google Scholar ]
• Paulus Paul B., Brown Vincent R. Toward More Creative and Innovative Group Idea Generation: A Cognitive-Social-Motivational Perspective of Brainstorming: Cognitive-Social-Motivational View of Brainstorming. Social and Personality Psychology Compass. 2007; 1 :248–65. doi: 10.1111/j.1751-9004.2007.00006.x. [ CrossRef ] [ Google Scholar ]
• Peddle Monica, Bearman Margaret, Radomski Natalie, Mckenna Lisa, Nestel Debra. What Non-Technical Skills Competencies Are Addressed by Australian Standards Documents for Health Professionals Who Work in Secondary and Tertiary Clinical Settings? A Qualitative Comparative Analysis. BMJ Open. 2018; 8 :e020799. doi: 10.1136/bmjopen-2017-020799. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Peña-López Ismaël. PISA 2015 Results (Volume V): Collaborative Problem Solving. PISA, OECD Publishing; Paris: 2017. [ Google Scholar ]
• Popil Inna. Promotion of Critical Thinking by Using Case Studies as Teaching Method. Nurse Education Today. 2011; 31 :204–7. doi: 10.1016/j.nedt.2010.06.002. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pornpitakpan Chanthika. The Persuasiveness of Source Credibility: A Critical Review of Five Decades’ Evidence. Journal of Applied Social Psychology. 2004; 34 :243–81. doi: 10.1111/j.1559-1816.2004.tb02547.x. [ CrossRef ] [ Google Scholar ]
• Possin Kevin. Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score. Informal Logic. 2014; 34 :393–416. doi: 10.22329/il.v34i4.4141. [ CrossRef ] [ Google Scholar ]
• Proctor Robert W., Dutta Addie. Skill Acquisition and Human Performance. Sage Publications, Inc.; Thousand Oaks: 1995. [ Google Scholar ]
• Putman Vicky L., Paulus Paul B. Brainstorming, Brainstorming Rules and Decision Making. The Journal of Creative Behavior. 2009; 43 :29–40. doi: 10.1002/j.2162-6057.2009.tb01304.x. [ CrossRef ] [ Google Scholar ]
• Reiman Joey. Success: The Original Handbook. Longstreet Press; Atlanta: 1992. [ Google Scholar ]
• Ren Xuezhu, Tong Yan, Peng Peng, Wang Tengfei. Critical Thinking Predicts Academic Performance beyond General Cognitive Ability: Evidence from Adults and Children. Intelligence. 2020; 82 :101487. doi: 10.1016/j.intell.2020.101487. [ CrossRef ] [ Google Scholar ]
• Renard Marie-Christine. Quality Certification, Regulation and Power in Fair Trade. Journal of Rural Studies. 2005; 21 :419–31. doi: 10.1016/j.jrurstud.2005.09.002. [ CrossRef ] [ Google Scholar ]
• Restout Emilie. Labels RSE: Un décryptage des entreprises labellisées en France. Goodwill Management. 2020. [(accessed on 2 November 2022)]. Available online: https://goodwill-management.com/labels-rse-decryptage-entreprises-labellisees/
• Rhodes Mel. An Analysis of Creativity. The Phi Delta Kappan. 1961; 42 :305–10. [ Google Scholar ]
• Rider Elizabeth A., Keefer Constance H. Communication Skills Competencies: Definitions and a Teaching Toolbox: Communication. Medical Education. 2006; 40 :624–29. doi: 10.1111/j.1365-2929.2006.02500.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Riemer Marc J. Communication Skills for the 21st Century Engineer. Global Journal of Engineering Education. 2007; 11 :89. [ Google Scholar ]
• Rietzschel Eric F., Nijstad Bernard A., Stroebe Wolfgang. Productivity Is Not Enough: A Comparison of Interactive and Nominal Brainstorming Groups on Idea Generation and Selection. Journal of Experimental Social Psychology. 2006; 42 :244–51. doi: 10.1016/j.jesp.2005.04.005. [ CrossRef ] [ Google Scholar ]
• Ross David. Why the Four Cs Will Become the Foundation of Human-AI Interface. 2018. [(accessed on 2 November 2022)]. Available online: https://www.gettingsmart.com/2018/03/04/why-the-4cs-will-become-the-foundation-of-human-ai-interface/
• Rothermich Kathrin. Social Communication Across the Lifespan: The Influence of Empathy [Preprint] SocArXiv. 2020 doi: 10.31235/osf.io/adgmy. [ CrossRef ] [ Google Scholar ]
• Rusdin Norazlin Mohd, Ali Siti Rahaimah. Practice of Fostering 4Cs Skills in Teaching and Learning. International Journal of Academic Research in Business and Social Sciences. 2019; 9 :1021–35. doi: 10.6007/IJARBSS/v9-i6/6063. [ CrossRef ] [ Google Scholar ]
• Rychen Dominique Simone, Hersch Salganik Laura., editors. Key Competencies for a Successful Life and a Well-Functioning Society. Hogrefe and Huber; Cambridge: 2003. [ Google Scholar ]
• Sahin Mehmet Can. Instructional Design Principles for 21st Century Learning Skills. Procedia, Social and Behavioral Sciences. 2009; 1 :1464–68. doi: 10.1016/j.sbspro.2009.01.258. [ CrossRef ] [ Google Scholar ]
• Salas Eduardo, Stagl Kevin C., Burke C. Shawn. International Review of Industrial and Organizational Psychology. John Wiley & Sons, Ltd.; Chichester: 2004. 25 Years of Team Effectiveness in Organizations: Research Themes and Emerging Needs; pp. 47–91. [ CrossRef ] [ Google Scholar ]
• Salas Eduardo, Shuffler Marissa L., Thayer Amanda L., Bedwell Wendy L., Lazzara Elizabeth H. Understanding and Improving Teamwork in Organizations: A Scientifically Based Practical Guide. Human Resource Management. 2015; 54 :599–622. doi: 10.1002/hrm.21628. [ CrossRef ] [ Google Scholar ]
• Salmi Jamil. The Tertiary Education Imperative: Knowledge, Skills and Values for Development. Springer; Cham: 2017. [ Google Scholar ]
• Samani Sanaz Ahmadpoor, Rasid Siti Zaleha Binti Abdul, bt Sofian Saudah. A Workplace to Support Creativity. Industrial Engineering & Management Systems. 2014; 13 :414–20. doi: 10.7232/iems.2014.13.4.414. [ CrossRef ] [ Google Scholar ]
• Saroyan Alenoush. Fostering Creativity and Critical Thinking in University Teaching and Learning: Considerations for Academics and Their Professional Learning. OECD; Paris: 2022. [ CrossRef ] [ Google Scholar ]
• Sasmita Jumiati, Suki Norazah Mohd. Young consumers’ insights on brand equity: Effects of brand association, brand loyalty, brand awareness, and brand image. International Journal of Retail & Distribution Management. 2015; 43 :276–92. doi: 10.1108/IJRDM-02-2014-0024. [ CrossRef ] [ Google Scholar ]
• Schlegel Claudia, Woermann Ulrich, Shaha Maya, Rethans Jan-Joost, van der Vleuten Cees. Effects of Communication Training on Real Practice Performance: A Role-Play Module versus a Standardized Patient Module. The Journal of Nursing Education. 2012; 51 :16–22. doi: 10.3928/01484834-20111116-02. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schleicher Andreas. Why Creativity and Creative Teaching and Learning Matter Today and for Tomorrow’s World. GloCT in Collaboration with OECD CERI; Paris: 2022. Creativity in Education Summit 2022. [ Google Scholar ]
• Schneider Bertrand, Sharma Kshitij, Cuendet Sebastien, Zufferey Guillaume, Dillenbourg Pierre, Pea Roy. Leveraging Mobile Eye-Trackers to Capture Joint Visual Attention in Co-Located Collaborative Learning Groups. International Journal of Computer-Supported Collaborative Learning. 2018; 13 :241–61. doi: 10.1007/s11412-018-9281-2. [ CrossRef ] [ Google Scholar ]
• Schultz David M. Eloquent Science: A course to improve scientific and communication skills; Paper presented at the 19th Symposium on Education; Altanta, GA, USA. January 18–21; 2010. [ Google Scholar ]
• Scialabba George. Mindplay. Harvard Magazine. 1984; 16 :19. [ Google Scholar ]
• Scott Ginamarie, Leritz Lyle E., Mumford Michael D. The Effectiveness of Creativity Training: A Quantitative Review. Creativity Research Journal. 2004; 16 :361–88. doi: 10.1080/10400410409534549. [ CrossRef ] [ Google Scholar ]
• Sigafoos Jeff, Schlosser Ralf W., Green Vanessa A., O’Reilly Mark, Lancioni Giulio E. Communication and Social Skills Assessment. In: Matson Johnny L., editor. Clinical Assessment and Intervention for Autism Spectrum Disorders. Elsevier; Amsterdam: 2008. pp. 165–92. [ CrossRef ] [ Google Scholar ]
• Simonton Dean Keith. Creativity from a Historiometric Perspective. In: Sternberg Robert J., editor. Handbook of Creativity. Cambridge University Press; Cambridge: 1999. pp. 116–34. [ CrossRef ] [ Google Scholar ]
• Singh Pallavi, Bala Hillol, Dey Bidit Lal, Filieri Raffaele. Enforced Remote Working: The Impact of Digital Platform-Induced Stress and Remote Working Experience on Technology Exhaustion and Subjective Wellbeing. Journal of Business Research. 2022; 151 :269–86. doi: 10.1016/j.jbusres.2022.07.002. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Spada Hans, Meier Anne, Rummel Nikol, Hauser Sabine. Proceedings of the 2005 Conference on Computer Support for Collaborative Learning Learning 2005: The next 10 Years!—CSCL’05, Taipei, Taiwan, May 30–June 4. Association for Computational Linguistics; Morristown: 2005. A New Method to Assess the Quality of Collaborative Process in CSCL. [ Google Scholar ]
• Spitzberg Brian H. Methods of interpersonal skill assessment. In: Greene John O., Burleson Brant R., editors. The Handbook of Communication and Social Interaction Skills. Lawrence Erlbaum Associates; Mahwah: 2003. [ Google Scholar ]
• Sternberg Robert. Intelligence, Wisdom, and Creativity: Three Is Better than One. Educational Psychologist. 1986; 21 :175–90. doi: 10.1207/s15326985ep2103_2. [ CrossRef ] [ Google Scholar ]
• Sternberg Robert J., Funke Joachim. The Psychology of Human Thought: An Introduction. Heidelberg University Publishing (heiUP); Heidelberg: 2019. [ CrossRef ] [ Google Scholar ]
• Sursock Andrée. Quality assurance and rankings: Some European lessons. In: Hazelkorn Ellen, Mihut Georgiana., editors. Research Handbook on University Rankings. Edward Elgar Publishing; Cheltenham: 2021. pp. 185–96. [ CrossRef ] [ Google Scholar ]
• Sursock Andrée, Vettori Oliver. Qualitätskultur. Ein Blick in Die Gelebte Praxis der Hochschulen. Agency for Quality Assurance and Accreditation; Vienna: 2017. [(accessed on 2 November 2022)]. Quo vadis, quality culture? Theses from different perspectives; pp. 13–18. Available online: https://www.aq.ac.at/de/ueber-uns/publikationen/sonstige-publikationen.php [ Google Scholar ]
• Sutter Éric. Certification et Labellisation: Un Problème de Confiance. Bref Panorama de La Situation Actuelle. Documentaliste-Sciences de l Information. 2005; 42 :284–90. doi: 10.3917/docsi.424.0284. [ CrossRef ] [ Google Scholar ]
• Taddei François. Training Creative and Collaborative Knowledge-Builders: A Major Challenge for 21st Century Education. OCDE; Paris: 2009. [ Google Scholar ]
• Thomas Keith, Lok Beatrice. Teaching Critical Thinking: An Operational Framework. In: Davies Martin, Barnett Ronald., editors. The Palgrave Handbook of Critical Thinking in Higher Education. Palgrave Macmillan US; New York: 2015. pp. 93–105. [ CrossRef ] [ Google Scholar ]
• Thompson Jeri. Measuring Student Success Skills: A Review of the Literature on Complex Communication. National Center for the Improvement of Educational Assessment; Dover: 2020. [ Google Scholar ]
• Thorndahl Kathrine L., Stentoft Diana. Thinking Critically about Critical Thinking and Problem-Based Learning in Higher Education: A Scoping Review. Interdisciplinary Journal of Problem-Based Learning 14. 2020 doi: 10.14434/ijpbl.v14i1.28773. [ CrossRef ] [ Google Scholar ]
• Thornhill-Miller Branden. ‘Crea-Critical-Collab-ication’: A Dynamic Interactionist Model of the 4Cs (Creativity, Critical Thinking, Collaboration and Communication) 2021. [(accessed on 2 November 2022)]. Available online: http://thornhill-miller.com/newWordpress/index.php/current-research/
• Thornhill-Miller Branden, Dupont Jean-Marc. Virtual Reality and the Enhancement of Creativity and Innovation: Underrecognized Potential Among Converging Technologies? Journal for Cognitive Education and Psychology. 2016; 15 :102–21. doi: 10.1891/1945-8959.15.1.102. [ CrossRef ] [ Google Scholar ]
• Thornhill-Miller Branden, Millican Peter. The Common-Core/Diversity Dilemma: Revisions of Humean Thought, New Empirical Research, and the Limits of Rational Religious Belief. European Journal for Philosophy of Religion. 2015; 7 :1–49. doi: 10.24204/ejpr.v7i1.128. [ CrossRef ] [ Google Scholar ]
• Tomasello Michael. Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press; Cambridge: 2005. [ CrossRef ] [ Google Scholar ]
• Uribe-Enciso Olga Lucía, Uribe-Enciso Diana Sofía, Vargas-Daza María Del Pilar. Pensamiento Crítico y Su Importancia En La Educación: Algunas Reflexiones. Rastros Rostros. 2017; 19 doi: 10.16925/ra.v19i34.2144. [ CrossRef ] [ Google Scholar ]
• van der Vleuten Cees, van den Eertwegh Valerie, Giroldi Esther. Assessment of Communication Skills. Patient Education and Counseling. 2019; 102 :2110–13. doi: 10.1016/j.pec.2019.07.007. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• van Klink Marcel R., Boon Jo. Competencies: The triumph of a fuzzy concept. International Journal of Human Resources Development and Management. 2003; 3 :125–37. doi: 10.1504/IJHRDM.2003.002415. [ CrossRef ] [ Google Scholar ]
• van Laar Ester, Van Deursen Alexander J. A. M., Van Dijk Jan A. G. M., de Haan Jos. The Relation between 21st-Century Skills and Digital Skills: A Systematic Literature Review. Computers in Human Behavior. 2017; 72 :577–88. doi: 10.1016/j.chb.2017.03.010. [ CrossRef ] [ Google Scholar ]
• van Rosmalen Peter, Boyle Elizabeth A., Nadolski Rob, van der Baaren John, Fernández-Manjón Baltasar, MacArthur Ewan, Pennanen Tiina, Manea Madalina, Star Kam. Lecture Notes in Computer Science. Springer International Publishing; Cham: 2014. Acquiring 21st Century Skills: Gaining Insight into the Design and Applicability of a Serious Game with 4C-ID; pp. 327–34. [ CrossRef ] [ Google Scholar ]
• Vincent-Lancrin Stéphan, González-Sancho Carlos, Bouckaert Mathias, de Luca Federico, Fernández-Barrerra Meritxell, Jacotin Gwénaël, Urgel Joaquin, Vidal Quentin. Fostering Students’ Creativity and Critical Thinking: What It Means in School. OECD Publishing; Paris: 2019. [ CrossRef ] [ Google Scholar ]
• Voogt Joke, Roblin Natalie Pareja. A Comparative Analysis of International Frameworks for 21st Century Competences: Implications for National Curriculum Policies. Journal of Curriculum Studies. 2012; 44 :299–321. doi: 10.1080/00220272.2012.668938. [ CrossRef ] [ Google Scholar ]
• Waizenegger Lena, McKenna Brad, Cai Wenjie, Bendz Taino. An Affordance Perspective of Team Collaboration and Enforced Working from Home during COVID-19. European Journal of Information Systems: An Official Journal of the Operational Research Society. 2020; 29 :429–42. doi: 10.1080/0960085X.2020.1800417. [ CrossRef ] [ Google Scholar ]
• Watson Goodwin. Watson-Glaser Critical Thinking Appraisal. Psychological Corporation; San Antonio: 1980. [ Google Scholar ]
• Watson Goodwin, Glaser Edwin M. Technical Manual and User’s Guide. Pearson; Kansas City: 2010. Watson-Glaser TM II critical thinking appraisal. [ Google Scholar ]
• Weick Karl E. The collapse of sensemaking in organizations: The Mann Gulch disaster. Administrative Science Quarterly. 1993; 38 :628–52. doi: 10.2307/2393339. [ CrossRef ] [ Google Scholar ]
• West Richard F., Toplak Maggie E., Stanovich Keith E. Heuristics and Biases as Measures of Critical Thinking: Associations with Cognitive Ability and Thinking Dispositions. Journal of Educational Psychology. 2008; 100 :930–41. doi: 10.1037/a0012842. [ CrossRef ] [ Google Scholar ]
• Whitmore Paul G. What are soft skills; Paper presented at the CONARC Soft Skills Conference; Fort Bliss, TX, USA. December 12–13; 1972. pp. 12–13. [ Google Scholar ]
• Willingham Daniel T. Critical Thinking: Why Is It so Hard to Teach? Arts Education Policy Review. 2008; 109 :21–32. doi: 10.3200/AEPR.109.4.21-32. [ CrossRef ] [ Google Scholar ]
• Wilson Sarah Beth, Varma-Nelson Pratibha. Small Groups, Significant Impact: A Review of Peer-Led Team Learning Research with Implications for STEM Education Researchers and Faculty. Journal of Chemical Education. 2016; 93 :1686–702. doi: 10.1021/acs.jchemed.5b00862. [ CrossRef ] [ Google Scholar ]
• Winterton Jonathan, Deist Françoise Delamare-Le, Stringfellow Emma. Typology of Knowledge, Skills and Competences: Clarification of the Concept and Prototype. Office for Official Publications of the European Communities; Luxembourg: 2006. [ Google Scholar ]
• World Economic Forum . New Vision for Education: Unlocking the Potential of Technology. World Economic Forum; Geneva: 2015. [ Google Scholar ]
• World Economic Forum The Future of Jobs Report 2020. 2020. [(accessed on 2 November 2022)]. Available online: https://www.weforum.org/reports/the-future-of-jobs-report-2020
• World Health Organization . Framework for Action on Interprofessional Education and Collaborative Practice. World Health Organization; Geneva: 2010. No. WHO/HRH/HPN/10.3. [ PubMed ] [ Google Scholar ]
• Yue Meng, Zhang Meng, Zhang Chunmei, Jin Changde. The Effectiveness of Concept Mapping on Development of Critical Thinking in Nursing Education: A Systematic Review and Meta-Analysis. Nurse Education Today. 2017; 52 :87–94. doi: 10.1016/j.nedt.2017.02.018. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zielke Stephan, Dobbelstein Thomas. Customers’ Willingness to Purchase New Store Brands. Journal of Product & Brand Management. 2007; 16 :112–21. doi: 10.1108/10610420710739982. [ CrossRef ] [ Google Scholar ]
• Zlatić Lidija, Bjekić Dragana, Marinković Snežana, Bojović Milevica. Development of Teacher Communication Competence. Procedia, Social and Behavioral Sciences. 2014; 116 :606–10. doi: 10.1016/j.sbspro.2014.01.265. [ CrossRef ] [ Google Scholar ]

When Critical Thinking Is Not Worth It

Personal perspective: should we always share our critical thinking.

Updated September 5, 2024 | Reviewed by Lybi Ma

In a recent post, I discussed social barriers to applying critical thinking . I received interesting feedback on this particular topic and I thought further discussion around this dialogue would be of interest. First and foremost, consider when we should apply critical thinking. As I’ve stated before, it might come as a surprise to readers that someone like me, who places great value on such thought, would suggest that critical thinking doesn’t need to be applied as often as many might think. The reality is that critical thinking is effortful and time-consuming. If we thought critically about every mundane decision we had to make each day, we’d be exhausted before mid-morning. We should only think critically about issues that we care about and that are important to us.

Why would someone even contemplate engaging in critical thinking when they could potentially face negative outcomes for it? It’s because the issue is important to them. But, is that a good enough reason? It depends. For example, I have thought critically about some rather controversial topics (arguably, these are the ones that require the most critical thinking given that what makes them controversial is that so many people care about them, yet have very different views) and I recognise that the conclusions yielded, in light of logic and evidence, may not always be palatable to people in certain contexts. Depending on the situation, I will choose to share my conclusions or choose against them. This, of course, is where we find the fork in the road at the crux of this conversation.

As I mentioned in the aforementioned post, there are arguably two different perspectives on whether or not one should share their critical thinking in environments that might discourage or even punish this thinking, if the conclusions drawn contradict what is deemed acceptable (be it socially, politically, or even legally). First, there is the idealistic, yes, we should always share critical thinking. Second is the practical, ‘know your audience’. Often, staying quiet seems like a practical and prudent move.

With that, such prudence might be seen to contradict what many might view as intellectual integrity; but, on the other hand, it can just as easily be argued that inhibiting such response is appropriate—an act of metacognition (thinking about thinking) about a specific metacognitive process (critical thinking). And so, the intellectually appropriate thing would be to make the best decision you can for the preservation of what or who you care about, such as through this 'meta-metacognition'. Perhaps the key is the question of what’s contextually more important, being right or avoiding punishment ? What is to be gained from speaking out? For me, the only situation I would share my critical thinking, in this context, is if my well-being or that of my family was at risk to the extent that such risk surpasses the impact of the punishment.

To reiterate, context is key here; what I do depends on the situation. Sometimes, having a conclusion is all that is needed. If I have thought critically about a topic to determine what is best for me or my family, why would I have to advertise my decision publicly? I don’t. Sure, I may choose to if I’m in discussion with friends, but I’m not required to do so (of course, this might change in situations where we are ‘forced’ to share our thinking, such as in cases where important decisions are being made for us or when we are specifically asked to infer a conclusion—for example, at work). Moreover, I’m less likely to share if I think it’s going to start a fight or annoyance. Why risk the hassle if there’s nothing real to gain? In both cases, self-regulation is useful. Most of the time, we can simultaneously benefit from engaging in critical thinking and keeping it to ourselves.

Consistent with this perspective, an important aspect of critical thinking is being practical. A practical person would not risk punishment unless they have a genuine chance of positively affecting the issue that they care about. An unfortunate by-product of this, in context, is that many critical thinkers remain quiet on controversial topics presented in the media (particularly if their thinking contradicts the status quo of the moral majority and their value signaling ). Even though you may not be imprisoned for your conclusions (that is, in nations where people enjoy free speech), you might risk other negative outcomes. Sure, we are aware of various sides of the argument; but quite often, we only hear the bias and emotion -based perspectives. Passion is distinct from care in consideration of applying such thinking.

We often hear the emotional callouts of those ‘for’ and ‘against’ particular ideas and movements; but less often do we hear the critical thinking. That’s not to say that the thinking isn’t there; rather, it’s less likely to get the focus because of social mechanisms that thrive when emotion is at play—like ‘they who shout loudest’ or the ‘squeaky wheel gets the grease.’ It could well be the case, in terms of controversial topics, that critical thinkers might actually represent a substantially large, though silent population.

I’m cognisant that some people fear that critical thinking is dying. I don’t think this is necessarily the case; rather, it might be that those not engaging in such thinking are getting louder – not because there are growing numbers of people who lack critical thinking , but because we have so many platforms available for people to spread their messages. I’m not saying that this is harmless and that such people can simply be ignored (for example, uninformed populations can vote other uninformed individuals into positions of power and law-making), but at the same time, we should not overestimate the impact of every erroneous statement made publicly. Give people credit – just because one person posts something silly online, doesn’t mean that the majority agrees with them. With that, some errors are more influential than others. Avoid stressing over the ones that don’t affect you. Be concerned about the ones that do and evaluate whether it is in your interest to share your thinking in those situations. Engage critical thinking but be practical; and don’t get baited into discourses with people who haven’t thought critically, are not open-minded to other perspectives, and not willing to change their mind.

Christopher Dwyer, Ph.D., is a lecturer at the Technological University of the Shannon in Athlone, Ireland.

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• International
• New Zealand
• South Africa
• Switzerland
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Self Tests NEW
• Therapy Center
• Diagnosis Dictionary
• Types of Therapy

It’s increasingly common for someone to be diagnosed with a condition such as ADHD or autism as an adult. A diagnosis often brings relief, but it can also come with as many questions as answers.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

What Is Critical Thinking? 6 Things You Should Know

• The Speaker Lab
• September 6, 2024

Table of Contents

Ever found yourself at a crossroads, weighing the pros and cons of a decision that could change everything? That’s critical thinking in action. Critical thinking isn’t exclusively the realm of academics or intellectuals; it’s a practical ability we employ daily. In fact, it’s one that you’re using right now as you decide whether to keep reading.

Critical thinking is defined as the objective analysis and evaluation of an issue in order to form a judgment. This process isn’t about what you think but how you think. In this article, we’re going to take a closer look at critical thinking, from its origins to its modern-day implications and applications. Join in, and you’ll learn how to work through those complex problems with ease. So what are you waiting for? Let’s get started.

Understanding Critical Thinking

At first, critical thinking just sounds like a fancy term we’ve all heard a million times. But when you get down to it, it’s actually our secret weapon for navigating a world chock-full of information and decisions.

When you think critically, you are engaging in an intellectually disciplined process where you skillfully conceptualize, apply, analyze, synthesize, and/or evaluate information gathered from observation, experience, reflection, researching, or communication. Instead of just taking things at face-value, you do the math yourself to make sure the claims others are making are actually logical conclusions.

To be clear, critical thinking is not about doubting everything under the sun. It’s more like being that detective in a mystery novel who sifts through clues carefully in order to make connections. It’s about looking at info and saying, “Okay, but why?” or “Says who?” before making up your mind. Every bit of info is potential evidence leading towards smarter decisions—that’s what critical thinking is all about. Embracing this approach can help you navigate decisions like what products to buy or what news sources to trust without relying on gut feelings alone.

The Intellectual Roots of Critical Thinking

Although it may sound strange to think about critical thinking having a history, it does. Ancient Greek philosophers like Socrates, Plato, and Aristotle were the early pioneers. Far from merely draping themselves in togas and mulling over philosophical concepts, they laid the groundwork for a culture of perpetual inquiry.

Fast forward to the 20th century, and thinkers like John Dewey and Edward Glaser took this baton further. They evolved critical thinking into what we know today. In fact, Edward Glaser gave us a clear definition of critical thinking, something that has proved difficult due to contesting views of critical thinking . According to Glaser, critical thinking is “a persistent effort to examine any…supposed form of knowledge in the light of the evidence that supports it.” (To read Glaser’s whole definition, find the full quote from his book here .)

Riding on his coattails was Richard Paul, who believed in reasoning through problems systematically—a disciplined process if you will—that involves skillfully conceptualizing, analyzing, and evaluating information gathered from observation or experience. While Paul’s definition does not contradict Glaser’s, it does provide more specifics.

Today, our ability to solve complex puzzles in everyday life is all thanks to the groundwork laid down by these intellectuals.

Find Out Exactly How Much You Could Make As a Paid Speaker

Use The Official Speaker Fee Calculator to tell you what you should charge for your first (or next) speaking gig — virtual or in-person! 

The Process of Critical Thinking

Like any other skill, critical thinking is one that can improve with practice. But before you can get started, it’s helpful to break this exercise down into steps. Let’s take a look.

• Identify the problem or question: This step is about understanding exactly what’s on your plate.
• Gather relevant information: Now we’re talking detective work. Dive into research like Sherlock Holmes and collect those facts.
• Analyze the evidence: Time to put on your thinking cap. Look at that info you gathered and start connecting dots.
• Consider alternative perspectives: What would your opponent say? In other words, are there other ways of looking at this situation? Considering different viewpoints is key here.
• Draw conclusions based on evidence and reasoning: Once you’ve evaluated the situation, it’s time to make your judgement call.

No matter what kind of decisions you encounter, critical thinking will always prove helpful. And it’s not just for the academic or analytically-minded either. It’s for anyone who’s willing to go beyond surface-level and challenge their own assumptions. It’s a little like having a superpower, but instead of leaping tall buildings, you’re navigating complex issues with grace and precision. So, next time life throws a curveball, you know exactly how to hit it out of the park.

The Role of Open-Mindedness in Critical Thinking

Building critical thinking skills is all about asking questions and challenging assumptions—yours included. By being open to new ideas, we can dodge biases that blindside us and embrace richer understanding instead. Diverse perspectives don’t just add color; they help us solve problems more creatively and make decisions more wisely. By making room at the table for everyone, you’ll find that better solutions bubble up from unexpected places.

If the idea of questioning your own assumptions makes you nervous, then know it doesn’t have to be an anxiety-inducing experience. As Aristotle once quipped, “It is the mark of an educated mind to be able to entertain a thought without accepting it.” In other words, you don’t have to completely dismantle your belief system every time you consider an opposing idea. The idea here is that you’re just hearing the other side out. After all, who knows? They may have a point you’ve never considered before.

So next time you catch yourself shutting down an unfamiliar idea or perspective before giving it due consideration, pause. Ask yourself: what might I discover if I listen? What bridges could this build? Becoming truly great at critical thinking requires this openness. It allows our thoughts to avoid echo chambers that stifle growth. So let’s stay curious together because by welcoming different angles, we enrich not only ourselves but also those around us.

Improving Your Critical Thinking Skills

Once you have a handle on what critical thinking looks like practically, you’re ready to start improving your critical thinking skills. To get started, consider the steps below.

• Question everything: This doesn’t mean doubting every single thing around you but being curious about why things are the way they are.
• Dive into diverse perspectives: Listen to podcasts and read books from authors that challenge your views. Growth lives in discomfort.
• Analyze, then analyze some more: When faced with information or an argument, break it down. What’s being said? Why? By whom? Investigate the proof and critically evaluate its trustworthiness.
• Solve problems systematically: If you have a problem, approach it step by step. Define it clearly first; brainstorm solutions next; choose one option after weighing options carefully; implement; and review results. Rinse and repeat as necessary.
• Meditate on mistakes: Reflection is key. Don’t just move past errors—learn what went wrong to avoid repeating history.

Becoming better at critical thinking doesn’t happen overnight. Think of it as building muscle: consistent effort pays off big time. Keep pushing and you’ll have your critical thinking skills sharpened in no time.

Using Creativity and Critical Thinking to Problem Solve

The more you hone your critical thinking skills, the more you’ll find yourself thinking outside the box. The result? A healthy creativity. When combined, these two powerhouse skills prove exceptionally effective for solving complex problems. It’s not just about finding a solution; it’s about crafting the smartest, most innovative one.

Imagine you’re faced with a gnarly problem. If you try to use a conventional approach, there seems to be no way forward. But with a little creativity, you gain a fresh perspective. To achieve this fresh perspective, consider the steps below.

• Step back: Sometimes, we’re too close to see clearly. Taking a step back lets our creative juices flow.
• Mix things up: Applying ideas from different domains can spark unique solutions no one saw coming.
• Rethink assumptions: What if the “problem” isn’t really the problem? Questioning what we’ve always believed could unexpectedly simplify our journey ahead.

This blend of critical thinking skills, like analysis and evaluation, with a dash of creative zest doesn’t just solve problems systematically—it does so with flair. Creativity nudges us to “what if?” until obstacles don’t look so daunting anymore. So next time you’re facing down a beast of an issue, know that with creativity and critical thought on your side, you’re unstoppable.

Demonstrating Your Critical Thinking in Professional Settings

Once you’ve honed those critical thinking skills, it’s time to put them to work. Specifically, we’re talking about putting them on your resume. Potential employers love seeing critical thinking skills in their job candidates, because it means you can problem solve and think outside the box. However, if you can’t showcase these abilities on your resume, it’s like they don’t even exist to potential employers. Let’s fix that.

Crafting a Resume That Highlights Your Skills

To get started, you’re going to need to include key words on your resume . In the current job market, this is an essential step for any resume. Why? Because it’s necessary for getting past the applicant tracking systems (ATS). Increasingly common, these machines process applications before a human even lays eyes on them. To sort through resumes, the machine is given key words to look for, and any resume that lacks these key words is culled. If you want to make it past the ATS, consider these steps.

• Analyze This: Under each job title on your resume, use bullet points to highlight how you’ve used analytical skills to solve problems or make decisions at past jobs.
• Evidence-Based Wins: Employers love results. Showcase instances where your critical analysis led to measurable success for projects or teams.
• Vocab Matters: Use words like “analyzed,” “evaluated,” and “implemented” to describe your problem-solving prowess.
• Show Don’t Tell: Toss out vague claims of being a “team player.” Instead, detail specific scenarios where you collaborated effectively using clear logic and reasoning.

To really stand out, tailor each application by weaving relevant terms found in the job description into your narrative. Promote your skills like this, and watch as interview invites start rolling in more than ever before.

Evaluating Sources for Credibility

A big part of critical thinking involves discerning the credibility of sources. After all, in a world overflowing with information, not everything you stumble upon is going to be the golden truth. That’s why sound evidence and sharp research skills aren’t just nice-to-haves; they’re your armor in the battle against misinformation. By dispelling falsehoods and exposing faulty sources, you can uncover the truth of the matter. This skill is particularly helpful when you’re writing a persuasive speech or a research essay.

• Analyze the source: Look at where your info is coming from. Is it reputable? What’s their track record like?
• Check for bias: Every story has two sides. Make sure you’re getting both angles to avoid falling into an echo chamber.
• Cross-reference facts: One source says one thing; another says something slightly different. Who’s right? Cross-check those details across multiple reliable sources.

Whether you’re a university student writing a term paper or a mom shopping for washable couch covers, being able to find a source you can trust is important. So arm yourself with these critical analysis skills because knowing how to evaluate credibility isn’t just about winning debates or acing papers—it’s about making informed decisions in every aspect of life.

Ready to Get Your First (Or Next) Paid Speaking Gig?

Download our free 26-page guide and get the 14 exact steps you can follow to book a paid speaking gig right now!

Together, we’ve navigated the intricate waters of critical thinking, exploring its core, history, methodology, and its relevance today. It’s not just an academic term gathering dust in textbooks; it’s the silent engine powering decisions big and small. Because when life throws puzzles your way, critical thinking is there to walk you through them. And when you pair it with creativity? You can problem solve in a way that will leave your employers impressed.

So remember, critical thinking isn’t just for stuffy academics. It’s for everyone, helping make life richer, work smarter, arguments sounder, and even making us all-around sharper individuals.

• Last Updated: September 11, 2024

Explore Related Resources

Learn How You Could Get Your First (Or Next) Paid Speaking Gig In 90 Days or Less

We receive thousands of applications every day, but we only work with the top 5% of speakers .

Book a call with our team to get started — you’ll learn why the vast majority of our students get a paid speaking gig within 90 days of finishing our program .

If you’re ready to control your schedule, grow your income, and make an impact in the world – it’s time to take the first step. Book a FREE consulting call and let’s get you Booked and Paid to Speak ® .

About The Speaker Lab

We teach speakers how to consistently get booked and paid to speak.  Since 2015, we’ve helped thousands of speakers find clarity, confidence, and a clear path to make an impact.

Get Started

Let's connect.

[email protected]

Copyright ©2023 The Speaker Lab. All rights reserved.

• Table of Contents
• New in this Archive
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Critical Thinking

Critical thinking is a widely accepted educational goal. Its definition is contested, but the competing definitions can be understood as differing conceptions of the same basic concept: careful thinking directed to a goal. Conceptions differ with respect to the scope of such thinking, the type of goal, the criteria and norms for thinking carefully, and the thinking components on which they focus. Its adoption as an educational goal has been recommended on the basis of respect for students’ autonomy and preparing students for success in life and for democratic citizenship. “Critical thinkers” have the dispositions and abilities that lead them to think critically when appropriate. The abilities can be identified directly; the dispositions indirectly, by considering what factors contribute to or impede exercise of the abilities. Standardized tests have been developed to assess the degree to which a person possesses such dispositions and abilities. Educational intervention has been shown experimentally to improve them, particularly when it includes dialogue, anchored instruction, and mentoring. Controversies have arisen over the generalizability of critical thinking across domains, over alleged bias in critical thinking theories and instruction, and over the relationship of critical thinking to other types of thinking.

2.1 Dewey’s Three Main Examples

2.2 dewey’s other examples, 2.3 further examples, 2.4 non-examples, 3. the definition of critical thinking, 4. its value, 5. the process of thinking critically, 6. components of the process, 7. contributory dispositions and abilities, 8.1 initiating dispositions, 8.2 internal dispositions, 9. critical thinking abilities, 10. required knowledge, 11. educational methods, 12.1 the generalizability of critical thinking, 12.2 bias in critical thinking theory and pedagogy, 12.3 relationship of critical thinking to other types of thinking, other internet resources, related entries.

Use of the term ‘critical thinking’ to describe an educational goal goes back to the American philosopher John Dewey (1910), who more commonly called it ‘reflective thinking’. He defined it as

active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends. (Dewey 1910: 6; 1933: 9)

and identified a habit of such consideration with a scientific attitude of mind. His lengthy quotations of Francis Bacon, John Locke, and John Stuart Mill indicate that he was not the first person to propose development of a scientific attitude of mind as an educational goal.

In the 1930s, many of the schools that participated in the Eight-Year Study of the Progressive Education Association (Aikin 1942) adopted critical thinking as an educational goal, for whose achievement the study’s Evaluation Staff developed tests (Smith, Tyler, & Evaluation Staff 1942). Glaser (1941) showed experimentally that it was possible to improve the critical thinking of high school students. Bloom’s influential taxonomy of cognitive educational objectives (Bloom et al. 1956) incorporated critical thinking abilities. Ennis (1962) proposed 12 aspects of critical thinking as a basis for research on the teaching and evaluation of critical thinking ability.

Since 1980, an annual international conference in California on critical thinking and educational reform has attracted tens of thousands of educators from all levels of education and from many parts of the world. Also since 1980, the state university system in California has required all undergraduate students to take a critical thinking course. Since 1983, the Association for Informal Logic and Critical Thinking has sponsored sessions in conjunction with the divisional meetings of the American Philosophical Association (APA). In 1987, the APA’s Committee on Pre-College Philosophy commissioned a consensus statement on critical thinking for purposes of educational assessment and instruction (Facione 1990a). Researchers have developed standardized tests of critical thinking abilities and dispositions; for details, see the Supplement on Assessment . Educational jurisdictions around the world now include critical thinking in guidelines for curriculum and assessment. Political and business leaders endorse its importance.

For details on this history, see the Supplement on History .

2. Examples and Non-Examples

Before considering the definition of critical thinking, it will be helpful to have in mind some examples of critical thinking, as well as some examples of kinds of thinking that would apparently not count as critical thinking.

Dewey (1910: 68–71; 1933: 91–94) takes as paradigms of reflective thinking three class papers of students in which they describe their thinking. The examples range from the everyday to the scientific.

Transit : “The other day, when I was down town on 16th Street, a clock caught my eye. I saw that the hands pointed to 12:20. This suggested that I had an engagement at 124th Street, at one o'clock. I reasoned that as it had taken me an hour to come down on a surface car, I should probably be twenty minutes late if I returned the same way. I might save twenty minutes by a subway express. But was there a station near? If not, I might lose more than twenty minutes in looking for one. Then I thought of the elevated, and I saw there was such a line within two blocks. But where was the station? If it were several blocks above or below the street I was on, I should lose time instead of gaining it. My mind went back to the subway express as quicker than the elevated; furthermore, I remembered that it went nearer than the elevated to the part of 124th Street I wished to reach, so that time would be saved at the end of the journey. I concluded in favor of the subway, and reached my destination by one o’clock.” (Dewey 1910: 68-69; 1933: 91-92)

Ferryboat : “Projecting nearly horizontally from the upper deck of the ferryboat on which I daily cross the river is a long white pole, having a gilded ball at its tip. It suggested a flagpole when I first saw it; its color, shape, and gilded ball agreed with this idea, and these reasons seemed to justify me in this belief. But soon difficulties presented themselves. The pole was nearly horizontal, an unusual position for a flagpole; in the next place, there was no pulley, ring, or cord by which to attach a flag; finally, there were elsewhere on the boat two vertical staffs from which flags were occasionally flown. It seemed probable that the pole was not there for flag-flying.

“I then tried to imagine all possible purposes of the pole, and to consider for which of these it was best suited: (a) Possibly it was an ornament. But as all the ferryboats and even the tugboats carried poles, this hypothesis was rejected. (b) Possibly it was the terminal of a wireless telegraph. But the same considerations made this improbable. Besides, the more natural place for such a terminal would be the highest part of the boat, on top of the pilot house. (c) Its purpose might be to point out the direction in which the boat is moving.

“In support of this conclusion, I discovered that the pole was lower than the pilot house, so that the steersman could easily see it. Moreover, the tip was enough higher than the base, so that, from the pilot's position, it must appear to project far out in front of the boat. Morevoer, the pilot being near the front of the boat, he would need some such guide as to its direction. Tugboats would also need poles for such a purpose. This hypothesis was so much more probable than the others that I accepted it. I formed the conclusion that the pole was set up for the purpose of showing the pilot the direction in which the boat pointed, to enable him to steer correctly.” (Dewey 1910: 69-70; 1933: 92-93)

Bubbles : “In washing tumblers in hot soapsuds and placing them mouth downward on a plate, bubbles appeared on the outside of the mouth of the tumblers and then went inside. Why? The presence of bubbles suggests air, which I note must come from inside the tumbler. I see that the soapy water on the plate prevents escape of the air save as it may be caught in bubbles. But why should air leave the tumbler? There was no substance entering to force it out. It must have expanded. It expands by increase of heat, or by decrease of pressure, or both. Could the air have become heated after the tumbler was taken from the hot suds? Clearly not the air that was already entangled in the water. If heated air was the cause, cold air must have entered in transferring the tumblers from the suds to the plate. I test to see if this supposition is true by taking several more tumblers out. Some I shake so as to make sure of entrapping cold air in them. Some I take out holding mouth downward in order to prevent cold air from entering. Bubbles appear on the outside of every one of the former and on none of the latter. I must be right in my inference. Air from the outside must have been expanded by the heat of the tumbler, which explains the appearance of the bubbles on the outside. But why do they then go inside? Cold contracts. The tumbler cooled and also the air inside it. Tension was removed, and hence bubbles appeared inside. To be sure of this, I test by placing a cup of ice on the tumbler while the bubbles are still forming outside. They soon reverse” (Dewey 1910: 70–71; 1933: 93–94).

Dewey (1910, 1933) sprinkles his book with other examples of critical thinking. We will refer to the following.

Weather : A man on a walk notices that it has suddenly become cool, thinks that it is probably going to rain, looks up and sees a dark cloud obscuring the sun, and quickens his steps (1910: 6–10; 1933: 9–13).

Disorder : A man finds his rooms on his return to them in disorder with his belongings thrown about, thinks at first of burglary as an explanation, then thinks of mischievous children as being an alternative explanation, then looks to see whether valuables are missing, and discovers that they are (1910: 82–83; 1933: 166–168).

Typhoid : A physician diagnosing a patient whose conspicuous symptoms suggest typhoid avoids drawing a conclusion until more data are gathered by questioning the patient and by making tests (1910: 85–86; 1933: 170).

Blur : A moving blur catches our eye in the distance, we ask ourselves whether it is a cloud of whirling dust or a tree moving its branches or a man signaling to us, we think of other traits that should be found on each of those possibilities, and we look and see if those traits are found (1910: 102, 108; 1933: 121, 133).

Suction pump : In thinking about the suction pump, the scientist first notes that it will draw water only to a maximum height of 33 feet at sea level and to a lesser maximum height at higher elevations, selects for attention the differing atmospheric pressure at these elevations, sets up experiments in which the air is removed from a vessel containing water (when suction no longer works) and in which the weight of air at various levels is calculated, compares the results of reasoning about the height to which a given weight of air will allow a suction pump to raise water with the observed maximum height at different elevations, and finally assimilates the suction pump to such apparently different phenomena as the siphon and the rising of a balloon (1910: 150–153; 1933: 195–198).

Diamond : A passenger in a car driving in a diamond lane reserved for vehicles with at least one passenger notices that the diamond marks on the pavement are far apart in some places and close together in others. Why? The driver suggests that the reason may be that the diamond marks are not needed where there is a solid double line separating the diamond line from the adjoining lane, but are needed when there is a dotted single line permitting crossing into the diamond lane. Further observation confirms that the diamonds are close together when a dotted line separates the diamond lane from its neighbour, but otherwise far apart.

Rash : A woman suddenly develops a very itchy red rash on her throat and upper chest. She recently noticed a mark on the back of her right hand, but was not sure whether the mark was a rash or a scrape. She lies down in bed and thinks about what might be causing the rash and what to do about it. About two weeks before, she began taking blood pressure medication that contained a sulfa drug, and the pharmacist had warned her, in view of a previous allergic reaction to a medication containing a sulfa drug, to be on the alert for an allergic reaction; however, she had been taking the medication for two weeks with no such effect. The day before, she began using a new cream on her neck and upper chest; against the new cream as the cause was mark on the back of her hand, which had not been exposed to the cream. She began taking probiotics about a month before. She also recently started new eye drops, but she supposed that manufacturers of eye drops would be careful not to include allergy-causing components in the medication. The rash might be a heat rash, since she recently was sweating profusely from her upper body. Since she is about to go away on a short vacation, where she would not have access to her usual physician, she decides to keep taking the probiotics and using the new eye drops but to discontinue the blood pressure medication and to switch back to the old cream for her neck and upper chest. She forms a plan to consult her regular physician on her return about the blood pressure medication.

Candidate : Although Dewey included no examples of thinking directed at appraising the arguments of others, such thinking has come to be considered a kind of critical thinking. We find an example of such thinking in the performance task on the Collegiate Learning Assessment (CLA+), which its sponsoring organization describes as

a performance-based assessment that provides a measure of an institution’s contribution to the development of critical-thinking and written communication skills of its students. (Council for Aid to Education 2017)

A sample task posted on its website requires the test-taker to write a report for public distribution evaluating a fictional candidate’s policy proposals and their supporting arguments, using supplied background documents, with a recommendation on whether to endorse the candidate.

Immediate acceptance of an idea that suggests itself as a solution to a problem (e.g., a possible explanation of an event or phenomenon, an action that seems likely to produce a desired result) is “uncritical thinking, the minimum of reflection” (Dewey 1910: 13). On-going suspension of judgment in the light of doubt about a possible solution is not critical thinking (Dewey 1910: 108). Critique driven by a dogmatically held political or religious ideology is not critical thinking; thus Paulo Freire (1968 [1970]) is using the term (e.g., at 1970: 71, 81, 100, 146) in a more politically freighted sense that includes not only reflection but also revolutionary action against oppression. Derivation of a conclusion from given data using an algorithm is not critical thinking.

What is critical thinking? There are many definitions. Ennis (2016) lists 14 philosophically oriented scholarly definitions and three dictionary definitions. Following Rawls (1971), who distinguished his conception of justice from a utilitarian conception but regarded them as rival conceptions of the same concept, Ennis maintains that the 17 definitions are different conceptions of the same concept. Rawls articulated the shared concept of justice as

a characteristic set of principles for assigning basic rights and duties and for determining… the proper distribution of the benefits and burdens of social cooperation. (Rawls 1971: 5)

Bailin et al. (1999b) claim that, if one considers what sorts of thinking an educator would take not to be critical thinking and what sorts to be critical thinking, one can conclude that educators typically understand critical thinking to have at least three features.

• It is done for the purpose of making up one’s mind about what to believe or do.
• The person engaging in the thinking is trying to fulfill standards of adequacy and accuracy appropriate to the thinking.
• The thinking fulfills the relevant standards to some threshold level.

One could sum up the core concept that involves these three features by saying that critical thinking is careful goal-directed thinking. This core concept seems to apply to all the examples of critical thinking described in the previous section. As for the non-examples, their exclusion depends on construing careful thinking as excluding jumping immediately to conclusions, suspending judgment no matter how strong the evidence, reasoning from an unquestioned ideological or religious perspective, and routinely using an algorithm to answer a question.

If the core of critical thinking is careful goal-directed thinking, conceptions of it can vary according to its presumed scope, its presumed goal, one’s criteria and threshold for being careful, and the thinking component on which one focuses As to its scope, some conceptions (e.g., Dewey 1910, 1933) restrict it to constructive thinking on the basis of one’s own observations and experiments, others (e.g., Ennis 1962; Fisher & Scriven 1997; Johnson 1992) to appraisal of the products of such thinking. Ennis (1991) and Bailin et al. (1999b) take it to cover both construction and appraisal. As to its goal, some conceptions restrict it to forming a judgment (Dewey 1910, 1933; Lipman 1987; Facione 1990a). Others allow for actions as well as beliefs as the end point of a process of critical thinking (Ennis 1991; Bailin et al. 1999b). As to the criteria and threshold for being careful, definitions vary in the term used to indicate that critical thinking satisfies certain norms: “intellectually disciplined” (Scriven & Paul 1987), “reasonable” (Ennis 1991), “skillful” (Lipman 1987), “skilled” (Fisher & Scriven 1997), “careful” (Bailin & Battersby 2009). Some definitions specify these norms, referring variously to “consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends” (Dewey 1910, 1933); “the methods of logical inquiry and reasoning” (Glaser 1941); “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication” (Scriven & Paul 1987); the requirement that “it is sensitive to context, relies on criteria, and is self-correcting” (Lipman 1987); “evidential, conceptual, methodological, criteriological, or contextual considerations” (Facione 1990a); and “plus-minus considerations of the product in terms of appropriate standards (or criteria)” (Johnson 1992). Stanovich and Stanovich (2010) propose to ground the concept of critical thinking in the concept of rationality, which they understand as combining epistemic rationality (fitting one’s beliefs to the world) and instrumental rationality (optimizing goal fulfillment); a critical thinker, in their view, is someone with “a propensity to override suboptimal responses from the autonomous mind” (2010: 227). These variant specifications of norms for critical thinking are not necessarily incompatible with one another, and in any case presuppose the core notion of thinking carefully. As to the thinking component singled out, some definitions focus on suspension of judgment during the thinking (Dewey 1910; McPeck 1981), others on inquiry while judgment is suspended (Bailin & Battersby 2009), others on the resulting judgment (Facione 1990a), and still others on the subsequent emotive response (Siegel 1988).

In educational contexts, a definition of critical thinking is a “programmatic definition” (Scheffler 1960: 19). It expresses a practical program for achieving an educational goal. For this purpose, a one-sentence formulaic definition is much less useful than articulation of a critical thinking process, with criteria and standards for the kinds of thinking that the process may involve. The real educational goal is recognition, adoption and implementation by students of those criteria and standards. That adoption and implementation in turn consists in acquiring the knowledge, abilities and dispositions of a critical thinker.

Conceptions of critical thinking generally do not include moral integrity as part of the concept. Dewey, for example, took critical thinking to be the ultimate intellectual goal of education, but distinguished it from the development of social cooperation among school children, which he took to be the central moral goal. Ennis (1996, 2011) added to his previous list of critical thinking dispositions a group of dispositions to care about the dignity and worth of every person, which he described as a “correlative” (1996) disposition without which critical thinking would be less valuable and perhaps harmful. An educational program that aimed at developing critical thinking but not the correlative disposition to care about the dignity and worth of every person, he asserted, “would be deficient and perhaps dangerous” (Ennis 1996: 172).

Dewey thought that education for reflective thinking would be of value to both the individual and society; recognition in educational practice of the kinship to the scientific attitude of children’s native curiosity, fertile imagination and love of experimental inquiry “would make for individual happiness and the reduction of social waste” (Dewey 1910: iii). Schools participating in the Eight-Year Study took development of the habit of reflective thinking and skill in solving problems as a means to leading young people to understand, appreciate and live the democratic way of life characteristic of the United States (Aikin 1942: 17–18, 81). Harvey Siegel (1988: 55–61) has offered four considerations in support of adopting critical thinking as an educational ideal. (1) Respect for persons requires that schools and teachers honour students’ demands for reasons and explanations, deal with students honestly, and recognize the need to confront students’ independent judgment; these requirements concern the manner in which teachers treat students. (2) Education has the task of preparing children to be successful adults, a task that requires development of their self-sufficiency. (3) Education should initiate children into the rational traditions in such fields as history, science and mathematics. (4) Education should prepare children to become democratic citizens, which requires reasoned procedures and critical talents and attitudes. To supplement these considerations, Siegel (1988: 62–90) responds to two objections: the ideology objection that adoption of any educational ideal requires a prior ideological commitment and the indoctrination objection that cultivation of critical thinking cannot escape being a form of indoctrination.

Despite the diversity of our 11 examples, one can recognize a common pattern. Dewey analyzed it as consisting of five phases:

• suggestions , in which the mind leaps forward to a possible solution;
• an intellectualization of the difficulty or perplexity into a problem to be solved, a question for which the answer must be sought;
• the use of one suggestion after another as a leading idea, or hypothesis , to initiate and guide observation and other operations in collection of factual material;
• the mental elaboration of the idea or supposition as an idea or supposition ( reasoning , in the sense on which reasoning is a part, not the whole, of inference); and
• testing the hypothesis by overt or imaginative action. (Dewey 1933: 106–107; italics in original)

The process of reflective thinking consisting of these phases would be preceded by a perplexed, troubled or confused situation and followed by a cleared-up, unified, resolved situation (Dewey 1933: 106). The term ‘phases’ replaced the term ‘steps’ (Dewey 1910: 72), thus removing the earlier suggestion of an invariant sequence. Variants of the above analysis appeared in (Dewey 1916: 177) and (Dewey 1938: 101–119).

The variant formulations indicate the difficulty of giving a single logical analysis of such a varied process. The process of critical thinking may have a spiral pattern, with the problem being redefined in the light of obstacles to solving it as originally formulated. For example, the person in Transit might have concluded that getting to the appointment at the scheduled time was impossible and have reformulated the problem as that of rescheduling the appointment for a mutually convenient time. Further, defining a problem does not always follow after or lead immediately to an idea of a suggested solution. Nor should it do so, as Dewey himself recognized in describing the physician in Typhoid as avoiding any strong preference for this or that conclusion before getting further information (Dewey 1910: 85; 1933: 170). People with a hypothesis in mind, even one to which they have a very weak commitment, have a so-called “confirmation bias” (Nickerson 1998): they are likely to pay attention to evidence that confirms the hypothesis and to ignore evidence that counts against it or for some competing hypothesis. Detectives, intelligence agencies, and investigators of airplane accidents are well advised to gather relevant evidence systematically and to postpone even tentative adoption of an explanatory hypothesis until the collected evidence rules out with the appropriate degree of certainty all but one explanation. Dewey’s analysis of the critical thinking process can be faulted as well for requiring acceptance or rejection of a possible solution to a defined problem, with no allowance for deciding in the light of the available evidence to suspend judgment. Further, given the great variety of kinds of problems for which reflection is appropriate, there is likely to be variation in its component events. Perhaps the best way to conceptualize the critical thinking process is as a checklist whose component events can occur in a variety of orders, selectively, and more than once. These component events might include (1) noticing a difficulty, (2) defining the problem, (3) dividing the problem into manageable sub-problems, (4) formulating a variety of possible solutions to the problem or sub-problem, (5) determining what evidence is relevant to deciding among possible solutions to the problem or sub-problem, (6) devising a plan of systematic observation or experiment that will uncover the relevant evidence, (7) carrying out the plan of systematic observation or experimentation, (8) noting the results of the systematic observation or experiment, (9) gathering relevant testimony and information from others, (10) judging the credibility of testimony and information gathered from others, (11) drawing conclusions from gathered evidence and accepted testimony, and (12) accepting a solution that the evidence adequately supports (cf. Hitchcock 2017: 485).

Checklist conceptions of the process of critical thinking are open to the objection that they are too mechanical and procedural to fit the multi-dimensional and emotionally charged issues for which critical thinking is urgently needed (Paul 1984). For such issues, a more dialectical process is advocated, in which competing relevant world views are identified, their implications explored, and some sort of creative synthesis attempted.

If one considers the critical thinking process illustrated by the 11 examples, one can identify distinct kinds of mental acts and mental states that form part of it. To distinguish, label and briefly characterize these components is a useful preliminary to identifying abilities, skills, dispositions, attitudes, habits and the like that contribute causally to thinking critically. Identifying such abilities and habits is in turn a useful preliminary to setting educational goals. Setting the goals is in its turn a useful preliminary to designing strategies for helping learners to achieve the goals and to designing ways of measuring the extent to which learners have done so. Such measures provide both feedback to learners on their achievement and a basis for experimental research on the effectiveness of various strategies for educating people to think critically. Let us begin, then, by distinguishing the kinds of mental acts and mental events that can occur in a critical thinking process.

• Observing : One notices something in one’s immediate environment (sudden cooling of temperature in Weather , bubbles forming outside a glass and then going inside in Bubbles , a moving blur in the distance in Blur , a rash in Rash ). Or one notes the results of an experiment or systematic observation (valuables missing in Disorder , no suction without air pressure in Suction pump )
• Feeling : One feels puzzled or uncertain about something (how to get to an appointment on time in Transit , why the diamonds vary in frequency in Diamond ). One wants to resolve this perplexity. One feels satisfaction once one has worked out an answer (to take the subway express in Transit , diamonds closer when needed as a warning in Diamond ).
• Wondering : One formulates a question to be addressed (why bubbles form outside a tumbler taken from hot water in Bubbles , how suction pumps work in Suction pump , what caused the rash in Rash ).
• Imagining : One thinks of possible answers (bus or subway or elevated in Transit , flagpole or ornament or wireless communication aid or direction indicator in Ferryboat , allergic reaction or heat rash in Rash ).
• Inferring : One works out what would be the case if a possible answer were assumed (valuables missing if there has been a burglary in Disorder , earlier start to the rash if it is an allergic reaction to a sulfa drug in Rash ). Or one draws a conclusion once sufficient relevant evidence is gathered (take the subway in Transit , burglary in Disorder , discontinue blood pressure medication and new cream in Rash ).
• Knowledge : One uses stored knowledge of the subject-matter to generate possible answers or to infer what would be expected on the assumption of a particular answer (knowledge of a city’s public transit system in Transit , of the requirements for a flagpole in Ferryboat , of Boyle’s law in Bubbles , of allergic reactions in Rash ).
• Experimenting : One designs and carries out an experiment or a systematic observation to find out whether the results deduced from a possible answer will occur (looking at the location of the flagpole in relation to the pilot’s position in Ferryboat , putting an ice cube on top of a tumbler taken from hot water in Bubbles , measuring the height to which a suction pump will draw water at different elevations in Suction pump , noticing the frequency of diamonds when movement to or from a diamond lane is allowed in Diamond ).
• Consulting : One finds a source of information, gets the information from the source, and makes a judgment on whether to accept it. None of our 11 examples include searching for sources of information. In this respect they are unrepresentative, since most people nowadays have almost instant access to information relevant to answering any question, including many of those illustrated by the examples. However, Candidate includes the activities of extracting information from sources and evaluating its credibility.
• Identifying and analyzing arguments : One notices an argument and works out its structure and content as a preliminary to evaluating its strength. This activity is central to Candidate . It is an important part of a critical thinking process in which one surveys arguments for various positions on an issue.
• Judging : One makes a judgment on the basis of accumulated evidence and reasoning, such as the judgment in Ferryboat that the purpose of the pole is to provide direction to the pilot.
• Deciding : One makes a decision on what to do or on what policy to adopt, as in the decision in Transit to take the subway.

By definition, a person who does something voluntarily is both willing and able to do that thing at that time. Both the willingness and the ability contribute causally to the person’s action, in the sense that the voluntary action would not occur if either (or both) of these were lacking. For example, suppose that one is standing with one’s arms at one’s sides and one voluntarily lifts one’s right arm to an extended horizontal position. One would not do so if one were unable to lift one’s arm, if for example one’s right side was paralyzed as the result of a stroke. Nor would one do so if one were unwilling to lift one’s arm, if for example one were participating in a street demonstration at which a white supremacist was urging the crowd to lift their right arm in a Nazi salute and one were unwilling to express support in this way for the racist Nazi ideology. The same analysis applies to a voluntary mental process of thinking critically. It requires both willingness and ability to think critically, including willingness and ability to perform each of the mental acts that compose the process and to coordinate those acts in a sequence that is directed at resolving the initiating perplexity.

Consider willingness first. We can identify causal contributors to willingness to think critically by considering factors that would cause a person who was able to think critically about an issue nevertheless not to do so (Hamby 2014). For each factor, the opposite condition thus contributes causally to willingness to think critically on a particular occasion. For example, people who habitually jump to conclusions without considering alternatives will not think critically about issues that arise, even if they have the required abilities. The contrary condition of willingness to suspend judgment is thus a causal contributor to thinking critically.

Now consider ability. In contrast to the ability to move one’s arm, which can be completely absent because a stroke has left the arm paralyzed, the ability to think critically is a developed ability, whose absence is not a complete absence of ability to think but absence of ability to think well. We can identify the ability to think well directly, in terms of the norms and standards for good thinking. In general, to be able do well the thinking activities that can be components of a critical thinking process, one needs to know the concepts and principles that characterize their good performance, to recognize in particular cases that the concepts and principles apply, and to apply them. The knowledge, recognition and application may be procedural rather than declarative. It may be domain-specific rather than widely applicable, and in either case may need subject-matter knowledge, sometimes of a deep kind.

Reflections of the sort illustrated by the previous two paragraphs have led scholars to identify the knowledge, abilities and dispositions of a “critical thinker”, i.e., someone who thinks critically whenever it is appropriate to do so. We turn now to these three types of causal contributors to thinking critically. We start with dispositions, since arguably these are the most powerful contributors to being a critical thinker, can be fostered at an early stage of a child’s development, and are susceptible to general improvement (Glaser 1941: 175)

8. Critical Thinking Dispositions

Educational researchers use the term ‘dispositions’ broadly for the habits of mind and attitudes that contribute causally to being a critical thinker. Some writers (e.g., Paul & Elder 2006; Hamby 2014; Bailin & Battersby 2016) propose to use the term ‘virtues’ for this dimension of a critical thinker. The virtues in question, although they are virtues of character, concern the person’s ways of thinking rather than the person’s ways of behaving towards others. They are not moral virtues but intellectual virtues, of the sort articulated by Zagzebski (1996) and discussed by Turri, Alfano, and Greco (2017).

On a realistic conception, thinking dispositions or intellectual virtues are real properties of thinkers. They are general tendencies, propensities, or inclinations to think in particular ways in particular circumstances, and can be genuinely explanatory (Siegel 1999). Sceptics argue that there is no evidence for a specific mental basis for the habits of mind that contribute to thinking critically, and that it is pedagogically misleading to posit such a basis (Bailin et al. 1999a). Whatever their status, critical thinking dispositions need motivation for their initial formation in a child—motivation that may be external or internal. As children develop, the force of habit will gradually become important in sustaining the disposition (Nieto & Valenzuela 2012). Mere force of habit, however, is unlikely to sustain critical thinking dispositions. Critical thinkers must value and enjoy using their knowledge and abilities to think things through for themselves. They must be committed to, and lovers of, inquiry.

A person may have a critical thinking disposition with respect to only some kinds of issues. For example, one could be open-minded about scientific issues but not about religious issues. Similarly, one could be confident in one’s ability to reason about the theological implications of the existence of evil in the world but not in one’s ability to reason about the best design for a guided ballistic missile.

Critical thinking dispositions can usefully be divided into initiating dispositions (those that contribute causally to starting to think critically about an issue) and internal dispositions (those that contribute causally to doing a good job of thinking critically once one has started) (Facione 1990a: 25). The two categories are not mutually exclusive. For example, open-mindedness, in the sense of willingness to consider alternative points of view to one’s own, is both an initiating and an internal disposition.

Using the strategy of considering factors that would block people with the ability to think critically from doing so, we can identify as initiating dispositions for thinking critically attentiveness, a habit of inquiry, self-confidence, courage, open-mindedness, willingness to suspend judgment, trust in reason, wanting evidence for one’s beliefs, and seeking the truth. We consider briefly what each of these dispositions amounts to, in each case citing sources that acknowledge them.

• Attentiveness : One will not think critically if one fails to recognize an issue that needs to be thought through. For example, the pedestrian in Weather would not have looked up if he had not noticed that the air was suddenly cooler. To be a critical thinker, then, one needs to be habitually attentive to one’s surroundings, noticing not only what one senses but also sources of perplexity in messages received and in one’s own beliefs and attitudes (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Habit of inquiry : Inquiry is effortful, and one needs an internal push to engage in it. For example, the student in Bubbles could easily have stopped at idle wondering about the cause of the bubbles rather than reasoning to a hypothesis, then designing and executing an experiment to test it. Thus willingness to think critically needs mental energy and initiative. What can supply that energy? Love of inquiry, or perhaps just a habit of inquiry. Hamby (2015) has argued that willingness to inquire is the central critical thinking virtue, one that encompasses all the others. It is recognized as a critical thinking disposition by Dewey (1910: 29; 1933: 35), Glaser (1941: 5), Ennis (1987: 12; 1991: 8), Facione (1990a: 25), Bailin et al. (1999b: 294), Halpern (1998: 452), and Facione, Facione, & Giancarlo (2001).
• Self-confidence : Lack of confidence in one’s abilities can block critical thinking. For example, if the woman in Rash lacked confidence in her ability to figure things out for herself, she might just have assumed that the rash on her chest was the allergic reaction to her medication against which the pharmacist had warned her. Thus willingness to think critically requires confidence in one’s ability to inquire (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Courage : Fear of thinking for oneself can stop one from doing it. Thus willingness to think critically requires intellectual courage (Paul & Elder 2006: 16).
• Open-mindedness : A dogmatic attitude will impede thinking critically. For example, a person who adheres rigidly to a “pro-choice” position on the issue of the legal status of induced abortion is likely to be unwilling to consider seriously the issue of when in its development an unborn child acquires a moral right to life. Thus willingness to think critically requires open-mindedness, in the sense of a willingness to examine questions to which one already accepts an answer but which further evidence or reasoning might cause one to answer differently (Dewey 1933; Facione 1990a; Ennis 1991; Bailin et al. 1999b; Halpern 1998, Facione, Facione, & Giancarlo 2001). Paul (1981) emphasizes open-mindedness about alternative world-views, and recommends a dialectical approach to integrating such views as central to what he calls “strong sense” critical thinking.
• Willingness to suspend judgment : Premature closure on an initial solution will block critical thinking. Thus willingness to think critically requires a willingness to suspend judgment while alternatives are explored (Facione 1990a; Ennis 1991; Halpern 1998).
• Trust in reason : Since distrust in the processes of reasoned inquiry will dissuade one from engaging in it, trust in them is an initiating critical thinking disposition (Facione 1990a, 25; Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001; Paul & Elder 2006). In reaction to an allegedly exclusive emphasis on reason in critical thinking theory and pedagogy, Thayer-Bacon (2000) argues that intuition, imagination, and emotion have important roles to play in an adequate conception of critical thinking that she calls “constructive thinking”. From her point of view, critical thinking requires trust not only in reason but also in intuition, imagination, and emotion.
• Seeking the truth : If one does not care about the truth but is content to stick with one’s initial bias on an issue, then one will not think critically about it. Seeking the truth is thus an initiating critical thinking disposition (Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001). A disposition to seek the truth is implicit in more specific critical thinking dispositions, such as trying to be well-informed, considering seriously points of view other than one’s own, looking for alternatives, suspending judgment when the evidence is insufficient, and adopting a position when the evidence supporting it is sufficient.

Some of the initiating dispositions, such as open-mindedness and willingness to suspend judgment, are also internal critical thinking dispositions, in the sense of mental habits or attitudes that contribute causally to doing a good job of critical thinking once one starts the process. But there are many other internal critical thinking dispositions. Some of them are parasitic on one’s conception of good thinking. For example, it is constitutive of good thinking about an issue to formulate the issue clearly and to maintain focus on it. For this purpose, one needs not only the corresponding ability but also the corresponding disposition. Ennis (1991: 8) describes it as the disposition “to determine and maintain focus on the conclusion or question”, Facione (1990a: 25) as “clarity in stating the question or concern”. Other internal dispositions are motivators to continue or adjust the critical thinking process, such as willingness to persist in a complex task and willingness to abandon nonproductive strategies in an attempt to self-correct (Halpern 1998: 452). For a list of identified internal critical thinking dispositions, see the Supplement on Internal Critical Thinking Dispositions .

Some theorists postulate skills, i.e., acquired abilities, as operative in critical thinking. It is not obvious, however, that a good mental act is the exercise of a generic acquired skill. Inferring an expected time of arrival, as in Transit , has some generic components but also uses non-generic subject-matter knowledge. Bailin et al. (1999a) argue against viewing critical thinking skills as generic and discrete, on the ground that skilled performance at a critical thinking task cannot be separated from knowledge of concepts and from domain-specific principles of good thinking. Talk of skills, they concede, is unproblematic if it means merely that a person with critical thinking skills is capable of intelligent performance.

Despite such scepticism, theorists of critical thinking have listed as general contributors to critical thinking what they variously call abilities (Glaser 1941; Ennis 1962, 1991), skills (Facione 1990a; Halpern 1998) or competencies (Fisher & Scriven 1997). Amalgamating these lists would produce a confusing and chaotic cornucopia of more than 50 possible educational objectives, with only partial overlap among them. It makes sense instead to try to understand the reasons for the multiplicity and diversity, and to make a selection according to one’s own reasons for singling out abilities to be developed in a critical thinking curriculum. Two reasons for diversity among lists of critical thinking abilities are the underlying conception of critical thinking and the envisaged educational level. Appraisal-only conceptions, for example, involve a different suite of abilities than constructive-only conceptions. Some lists, such as those in (Glaser 1941), are put forward as educational objectives for secondary school students, whereas others are proposed as objectives for college students (e.g., Facione 1990a).

The abilities described in the remaining paragraphs of this section emerge from reflection on the general abilities needed to do well the thinking activities identified in section 6 as components of the critical thinking process described in section 5 . The derivation of each collection of abilities is accompanied by citation of sources that list such abilities and of standardized tests that claim to test them.

Observational abilities : Careful and accurate observation sometimes requires specialist expertise and practice, as in the case of observing birds and observing accident scenes. However, there are general abilities of noticing what one’s senses are picking up from one’s environment and of being able to articulate clearly and accurately to oneself and others what one has observed. It helps in exercising them to be able to recognize and take into account factors that make one’s observation less trustworthy, such as prior framing of the situation, inadequate time, deficient senses, poor observation conditions, and the like. It helps as well to be skilled at taking steps to make one’s observation more trustworthy, such as moving closer to get a better look, measuring something three times and taking the average, and checking what one thinks one is observing with someone else who is in a good position to observe it. It also helps to be skilled at recognizing respects in which one’s report of one’s observation involves inference rather than direct observation, so that one can then consider whether the inference is justified. These abilities come into play as well when one thinks about whether and with what degree of confidence to accept an observation report, for example in the study of history or in a criminal investigation or in assessing news reports. Observational abilities show up in some lists of critical thinking abilities (Ennis 1962: 90; Facione 1990a: 16; Ennis 1991: 9). There are items testing a person’s ability to judge the credibility of observation reports in the Cornell Critical Thinking Tests, Levels X and Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). Norris and King (1983, 1985, 1990a, 1990b) is a test of ability to appraise observation reports.

Emotional abilities : The emotions that drive a critical thinking process are perplexity or puzzlement, a wish to resolve it, and satisfaction at achieving the desired resolution. Children experience these emotions at an early age, without being trained to do so. Education that takes critical thinking as a goal needs only to channel these emotions and to make sure not to stifle them. Collaborative critical thinking benefits from ability to recognize one’s own and others’ emotional commitments and reactions.

Questioning abilities : A critical thinking process needs transformation of an inchoate sense of perplexity into a clear question. Formulating a question well requires not building in questionable assumptions, not prejudging the issue, and using language that in context is unambiguous and precise enough (Ennis 1962: 97; 1991: 9).

Imaginative abilities : Thinking directed at finding the correct causal explanation of a general phenomenon or particular event requires an ability to imagine possible explanations. Thinking about what policy or plan of action to adopt requires generation of options and consideration of possible consequences of each option. Domain knowledge is required for such creative activity, but a general ability to imagine alternatives is helpful and can be nurtured so as to become easier, quicker, more extensive, and deeper (Dewey 1910: 34–39; 1933: 40–47). Facione (1990a) and Halpern (1998) include the ability to imagine alternatives as a critical thinking ability.

Inferential abilities : The ability to draw conclusions from given information, and to recognize with what degree of certainty one’s own or others’ conclusions follow, is universally recognized as a general critical thinking ability. All 11 examples in section 2 of this article include inferences, some from hypotheses or options (as in Transit , Ferryboat and Disorder ), others from something observed (as in Weather and Rash ). None of these inferences is formally valid. Rather, they are licensed by general, sometimes qualified substantive rules of inference (Toulmin 1958) that rest on domain knowledge—that a bus trip takes about the same time in each direction, that the terminal of a wireless telegraph would be located on the highest possible place, that sudden cooling is often followed by rain, that an allergic reaction to a sulfa drug generally shows up soon after one starts taking it. It is a matter of controversy to what extent the specialized ability to deduce conclusions from premisses using formal rules of inference is needed for critical thinking. Dewey (1933) locates logical forms in setting out the products of reflection rather than in the process of reflection. Ennis (1981a), on the other hand, maintains that a liberally-educated person should have the following abilities: to translate natural-language statements into statements using the standard logical operators, to use appropriately the language of necessary and sufficient conditions, to deal with argument forms and arguments containing symbols, to determine whether in virtue of an argument’s form its conclusion follows necessarily from its premisses, to reason with logically complex propositions, and to apply the rules and procedures of deductive logic. Inferential abilities are recognized as critical thinking abilities by Glaser (1941: 6), Facione (1990a: 9), Ennis (1991: 9), Fisher & Scriven (1997: 99, 111), and Halpern (1998: 452). Items testing inferential abilities constitute two of the five subtests of the Watson Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), two of the four sections in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), three of the seven sections in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), 11 of the 34 items on Forms A and B of the California Critical Thinking Skills Test (Facione 1990b, 1992), and a high but variable proportion of the 25 selected-response questions in the Collegiate Learning Assessment (Council for Aid to Education 2017).

Experimenting abilities : Knowing how to design and execute an experiment is important not just in scientific research but also in everyday life, as in Rash . Dewey devoted a whole chapter of his How We Think (1910: 145–156; 1933: 190–202) to the superiority of experimentation over observation in advancing knowledge. Experimenting abilities come into play at one remove in appraising reports of scientific studies. Skill in designing and executing experiments includes the acknowledged abilities to appraise evidence (Glaser 1941: 6), to carry out experiments and to apply appropriate statistical inference techniques (Facione 1990a: 9), to judge inductions to an explanatory hypothesis (Ennis 1991: 9), and to recognize the need for an adequately large sample size (Halpern 1998). The Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) includes four items (out of 52) on experimental design. The Collegiate Learning Assessment (Council for Aid to Education 2017) makes room for appraisal of study design in both its performance task and its selected-response questions.

Consulting abilities : Skill at consulting sources of information comes into play when one seeks information to help resolve a problem, as in Candidate . Ability to find and appraise information includes ability to gather and marshal pertinent information (Glaser 1941: 6), to judge whether a statement made by an alleged authority is acceptable (Ennis 1962: 84), to plan a search for desired information (Facione 1990a: 9), and to judge the credibility of a source (Ennis 1991: 9). Ability to judge the credibility of statements is tested by 24 items (out of 76) in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) and by four items (out of 52) in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). The College Learning Assessment’s performance task requires evaluation of whether information in documents is credible or unreliable (Council for Aid to Education 2017).

Argument analysis abilities : The ability to identify and analyze arguments contributes to the process of surveying arguments on an issue in order to form one’s own reasoned judgment, as in Candidate . The ability to detect and analyze arguments is recognized as a critical thinking skill by Facione (1990a: 7–8), Ennis (1991: 9) and Halpern (1998). Five items (out of 34) on the California Critical Thinking Skills Test (Facione 1990b, 1992) test skill at argument analysis. The College Learning Assessment (Council for Aid to Education 2017) incorporates argument analysis in its selected-response tests of critical reading and evaluation and of critiquing an argument.

Judging skills and deciding skills : Skill at judging and deciding is skill at recognizing what judgment or decision the available evidence and argument supports, and with what degree of confidence. It is thus a component of the inferential skills already discussed.

Lists and tests of critical thinking abilities often include two more abilities: identifying assumptions and constructing and evaluating definitions.

In addition to dispositions and abilities, critical thinking needs knowledge: of critical thinking concepts, of critical thinking principles, and of the subject-matter of the thinking.

We can derive a short list of concepts whose understanding contributes to critical thinking from the critical thinking abilities described in the preceding section. Observational abilities require an understanding of the difference between observation and inference. Questioning abilities require an understanding of the concepts of ambiguity and vagueness. Inferential abilities require an understanding of the difference between conclusive and defeasible inference (traditionally, between deduction and induction), as well as of the difference between necessary and sufficient conditions. Experimenting abilities require an understanding of the concepts of hypothesis, null hypothesis, assumption and prediction, as well as of the concept of statistical significance and of its difference from importance. They also require an understanding of the difference between an experiment and an observational study, and in particular of the difference between a randomized controlled trial, a prospective correlational study and a retrospective (case-control) study. Argument analysis abilities require an understanding of the concepts of argument, premiss, assumption, conclusion and counter-consideration. Additional critical thinking concepts are proposed by Bailin et al. (1999b: 293), Fisher & Scriven (1997: 105–106), and Black (2012).

According to Glaser (1941: 25), ability to think critically requires knowledge of the methods of logical inquiry and reasoning. If we review the list of abilities in the preceding section, however, we can see that some of them can be acquired and exercised merely through practice, possibly guided in an educational setting, followed by feedback. Searching intelligently for a causal explanation of some phenomenon or event requires that one consider a full range of possible causal contributors, but it seems more important that one implements this principle in one’s practice than that one is able to articulate it. What is important is “operational knowledge” of the standards and principles of good thinking (Bailin et al. 1999b: 291–293). But the development of such critical thinking abilities as designing an experiment or constructing an operational definition can benefit from learning their underlying theory. Further, explicit knowledge of quirks of human thinking seems useful as a cautionary guide. Human memory is not just fallible about details, as people learn from their own experiences of misremembering, but is so malleable that a detailed, clear and vivid recollection of an event can be a total fabrication (Loftus 2017). People seek or interpret evidence in ways that are partial to their existing beliefs and expectations, often unconscious of their “confirmation bias” (Nickerson 1998). Not only are people subject to this and other cognitive biases (Kahneman 2011), of which they are typically unaware, but it may be counter-productive for one to make oneself aware of them and try consciously to counteract them or to counteract social biases such as racial or sexual stereotypes (Kenyon & Beaulac 2014). It is helpful to be aware of these facts and of the superior effectiveness of blocking the operation of biases—for example, by making an immediate record of one’s observations, refraining from forming a preliminary explanatory hypothesis, blind refereeing, double-blind randomized trials, and blind grading of students’ work.

Critical thinking about an issue requires substantive knowledge of the domain to which the issue belongs. Critical thinking abilities are not a magic elixir that can be applied to any issue whatever by somebody who has no knowledge of the facts relevant to exploring that issue. For example, the student in Bubbles needed to know that gases do not penetrate solid objects like a glass, that air expands when heated, that the volume of an enclosed gas varies directly with its temperature and inversely with its pressure, and that hot objects will spontaneously cool down to the ambient temperature of their surroundings unless kept hot by insulation or a source of heat. Critical thinkers thus need a rich fund of subject-matter knowledge relevant to the variety of situations they encounter. This fact is recognized in the inclusion among critical thinking dispositions of a concern to become and remain generally well informed.

Experimental educational interventions, with control groups, have shown that education can improve critical thinking skills and dispositions, as measured by standardized tests. For information about these tests, see the Supplement on Assessment .

What educational methods are most effective at developing the dispositions, abilities and knowledge of a critical thinker? Abrami et al. (2015) found that in the experimental and quasi-experimental studies that they analyzed dialogue, anchored instruction, and mentoring each increased the effectiveness of the educational intervention, and that they were most effective when combined. They also found that in these studies a combination of separate instruction in critical thinking with subject-matter instruction in which students are encouraged to think critically was more effective than either by itself. However, the difference was not statistically significant; that is, it might have arisen by chance.

Most of these studies lack the longitudinal follow-up required to determine whether the observed differential improvements in critical thinking abilities or dispositions continue over time, for example until high school or college graduation. For details on studies of methods of developing critical thinking skills and dispositions, see the Supplement on Educational Methods .

12. Controversies

Scholars have denied the generalizability of critical thinking abilities across subject domains, have alleged bias in critical thinking theory and pedagogy, and have investigated the relationship of critical thinking to other kinds of thinking.

McPeck (1981) attacked the thinking skills movement of the 1970s, including the critical thinking movement. He argued that there are no general thinking skills, since thinking is always thinking about some subject-matter. It is futile, he claimed, for schools and colleges to teach thinking as if it were a separate subject. Rather, teachers should lead their pupils to become autonomous thinkers by teaching school subjects in a way that brings out their cognitive structure and that encourages and rewards discussion and argument. As some of his critics (e.g., Paul 1985; Siegel 1985) pointed out, McPeck’s central argument needs elaboration, since it has obvious counter-examples in writing and speaking, for which (up to a certain level of complexity) there are teachable general abilities even though they are always about some subject-matter. To make his argument convincing, McPeck needs to explain how thinking differs from writing and speaking in a way that does not permit useful abstraction of its components from the subject-matters with which it deals. He has not done so. Nevertheless, his position that the dispositions and abilities of a critical thinker are best developed in the context of subject-matter instruction is shared by many theorists of critical thinking, including Dewey (1910, 1933), Glaser (1941), Passmore (1980), Weinstein (1990), and Bailin et al. (1999b).

McPeck’s challenge prompted reflection on the extent to which critical thinking is subject-specific. McPeck argued for a strong subject-specificity thesis, according to which it is a conceptual truth that all critical thinking abilities are specific to a subject. (He did not however extend his subject-specificity thesis to critical thinking dispositions. In particular, he took the disposition to suspend judgment in situations of cognitive dissonance to be a general disposition.) Conceptual subject-specificity is subject to obvious counter-examples, such as the general ability to recognize confusion of necessary and sufficient conditions. A more modest thesis, also endorsed by McPeck, is epistemological subject-specificity, according to which the norms of good thinking vary from one field to another. Epistemological subject-specificity clearly holds to a certain extent; for example, the principles in accordance with which one solves a differential equation are quite different from the principles in accordance with which one determines whether a painting is a genuine Picasso. But the thesis suffers, as Ennis (1989) points out, from vagueness of the concept of a field or subject and from the obvious existence of inter-field principles, however broadly the concept of a field is construed. For example, the principles of hypothetico-deductive reasoning hold for all the varied fields in which such reasoning occurs. A third kind of subject-specificity is empirical subject-specificity, according to which as a matter of empirically observable fact a person with the abilities and dispositions of a critical thinker in one area of investigation will not necessarily have them in another area of investigation.

The thesis of empirical subject-specificity raises the general problem of transfer. If critical thinking abilities and dispositions have to be developed independently in each school subject, how are they of any use in dealing with the problems of everyday life and the political and social issues of contemporary society, most of which do not fit into the framework of a traditional school subject? Proponents of empirical subject-specificity tend to argue that transfer is more likely to occur if there is critical thinking instruction in a variety of domains, with explicit attention to dispositions and abilities that cut across domains. But evidence for this claim is scanty. There is a need for well-designed empirical studies that investigate the conditions that make transfer more likely.

It is common ground in debates about the generality or subject-specificity of critical thinking dispositions and abilities that critical thinking about any topic requires background knowledge about the topic. For example, the most sophisticated understanding of the principles of hypothetico-deductive reasoning is of no help unless accompanied by some knowledge of what might be plausible explanations of some phenomenon under investigation.

Critics have objected to bias in the theory, pedagogy and practice of critical thinking. Commentators (e.g., Alston 1995; Ennis 1998) have noted that anyone who takes a position has a bias in the neutral sense of being inclined in one direction rather than others. The critics, however, are objecting to bias in the pejorative sense of an unjustified favoring of certain ways of knowing over others, frequently alleging that the unjustly favoured ways are those of a dominant sex or culture (Bailin 1995). These ways favour:

• reinforcement of egocentric and sociocentric biases over dialectical engagement with opposing world-views (Paul 1981, 1984; Warren 1998)
• distancing from the object of inquiry over closeness to it (Martin 1992; Thayer-Bacon 1992)
• indifference to the situation of others over care for them (Martin 1992)
• orientation to thought over orientation to action (Martin 1992)
• being reasonable over caring to understand people’s ideas (Thayer-Bacon 1993)
• being neutral and objective over being embodied and situated (Thayer-Bacon 1995a)
• doubting over believing (Thayer-Bacon 1995b)
• reason over emotion, imagination and intuition (Thayer-Bacon 2000)
• solitary thinking over collaborative thinking (Thayer-Bacon 2000)
• written and spoken assignments over other forms of expression (Alston 2001)
• attention to written and spoken communications over attention to human problems (Alston 2001)
• winning debates in the public sphere over making and understanding meaning (Alston 2001)

A common thread in this smorgasbord of accusations is dissatisfaction with focusing on the logical analysis and evaluation of reasoning and arguments. While these authors acknowledge that such analysis and evaluation is part of critical thinking and should be part of its conceptualization and pedagogy, they insist that it is only a part. Paul (1981), for example, bemoans the tendency of atomistic teaching of methods of analyzing and evaluating arguments to turn students into more able sophists, adept at finding fault with positions and arguments with which they disagree but even more entrenched in the egocentric and sociocentric biases with which they began. Martin (1992) and Thayer-Bacon (1992) cite with approval the self-reported intimacy with their subject-matter of leading researchers in biology and medicine, an intimacy that conflicts with the distancing allegedly recommended in standard conceptions and pedagogy of critical thinking. Thayer-Bacon (2000) contrasts the embodied and socially embedded learning of her elementary school students in a Montessori school, who used their imagination, intuition and emotions as well as their reason, with conceptions of critical thinking as

thinking that is used to critique arguments, offer justifications, and make judgments about what are the good reasons, or the right answers. (Thayer-Bacon 2000: 127–128)

Alston (2001) reports that her students in a women’s studies class were able to see the flaws in the Cinderella myth that pervades much romantic fiction but in their own romantic relationships still acted as if all failures were the woman’s fault and still accepted the notions of love at first sight and living happily ever after. Students, she writes, should

be able to connect their intellectual critique to a more affective, somatic, and ethical account of making risky choices that have sexist, racist, classist, familial, sexual, or other consequences for themselves and those both near and far… critical thinking that reads arguments, texts, or practices merely on the surface without connections to feeling/desiring/doing or action lacks an ethical depth that should infuse the difference between mere cognitive activity and something we want to call critical thinking. (Alston 2001: 34)

Some critics portray such biases as unfair to women. Thayer-Bacon (1992), for example, has charged modern critical thinking theory with being sexist, on the ground that it separates the self from the object and causes one to lose touch with one’s inner voice, and thus stigmatizes women, who (she asserts) link self to object and listen to their inner voice. Her charge does not imply that women as a group are on average less able than men to analyze and evaluate arguments. Facione (1990c) found no difference by sex in performance on his California Critical Thinking Skills Test. Kuhn (1991: 280–281) found no difference by sex in either the disposition or the competence to engage in argumentative thinking.

The critics propose a variety of remedies for the biases that they allege. In general, they do not propose to eliminate or downplay critical thinking as an educational goal. Rather, they propose to conceptualize critical thinking differently and to change its pedagogy accordingly. Their pedagogical proposals arise logically from their objections. They can be summarized as follows:

• Focus on argument networks with dialectical exchanges reflecting contesting points of view rather than on atomic arguments, so as to develop “strong sense” critical thinking that transcends egocentric and sociocentric biases (Paul 1981, 1984).
• Foster closeness to the subject-matter and feeling connected to others in order to inform a humane democracy (Martin 1992).
• Develop “constructive thinking” as a social activity in a community of physically embodied and socially embedded inquirers with personal voices who value not only reason but also imagination, intuition and emotion (Thayer-Bacon 2000).
• In developing critical thinking in school subjects, treat as important neither skills nor dispositions but opening worlds of meaning (Alston 2001).
• Attend to the development of critical thinking dispositions as well as skills, and adopt the “critical pedagogy” practised and advocated by Freire (1968 [1970]) and hooks (1994) (Dalgleish, Girard, & Davies 2017).

A common thread in these proposals is treatment of critical thinking as a social, interactive, personally engaged activity like that of a quilting bee or a barn-raising (Thayer-Bacon 2000) rather than as an individual, solitary, distanced activity symbolized by Rodin’s The Thinker . One can get a vivid description of education with the former type of goal from the writings of bell hooks (1994, 2010). Critical thinking for her is open-minded dialectical exchange across opposing standpoints and from multiple perspectives, a conception similar to Paul’s “strong sense” critical thinking (Paul 1981). She abandons the structure of domination in the traditional classroom. In an introductory course on black women writers, for example, she assigns students to write an autobiographical paragraph about an early racial memory, then to read it aloud as the others listen, thus affirming the uniqueness and value of each voice and creating a communal awareness of the diversity of the group’s experiences (hooks 1994: 84). Her “engaged pedagogy” is thus similar to the “freedom under guidance” implemented in John Dewey’s Laboratory School of Chicago in the late 1890s and early 1900s. It incorporates the dialogue, anchored instruction, and mentoring that Abrami (2015) found to be most effective in improving critical thinking skills and dispositions.

What is the relationship of critical thinking to problem solving, decision-making, higher-order thinking, creative thinking, and other recognized types of thinking? One’s answer to this question obviously depends on how one defines the terms used in the question. If critical thinking is conceived broadly to cover any careful thinking about any topic for any purpose, then problem solving and decision making will be kinds of critical thinking, if they are done carefully. Historically, ‘critical thinking’ and ‘problem solving’ were two names for the same thing. If critical thinking is conceived more narrowly as consisting solely of appraisal of intellectual products, then it will be disjoint with problem solving and decision making, which are constructive.

Bloom’s taxonomy of educational objectives used the phrase “intellectual abilities and skills” for what had been labeled “critical thinking” by some, “reflective thinking” by Dewey and others, and “problem solving” by still others (Bloom et al. 1956: 38). Thus, the so-called “higher-order thinking skills” at the taxonomy’s top levels of analysis, synthesis and evaluation are just critical thinking skills, although they do not come with general criteria for their assessment (Ennis 1981b). The revised version of Bloom’s taxonomy (Anderson et al. 2001) likewise treats critical thinking as cutting across those types of cognitive process that involve more than remembering (Anderson et al. 2001: 269–270). For details, see the Supplement on History .

As to creative thinking, it overlaps with critical thinking (Bailin 1987, 1988). Thinking about the explanation of some phenomenon or event, as in Ferryboat , requires creative imagination in constructing plausible explanatory hypotheses. Likewise, thinking about a policy question, as in Candidate , requires creativity in coming up with options. Conversely, creativity in any field needs to be balanced by critical appraisal of the draft painting or novel or mathematical theory.

• Abrami, Philip C., Robert M. Bernard, Eugene Borokhovski, David I. Waddington, C. Anne Wade, and Tonje Person, 2015, “Strategies for Teaching Students to Think Critically: A Meta-analysis”, Review of Educational Research , 85(2): 275–314. doi:10.3102/0034654314551063
• Aikin, Wilford M., 1942, The Story of the Eight-year Study, with Conclusions and Recommendations , Volume I of Adventure in American Education , New York and London: Harper & Brothers. [ Aikin 1942 available online ]
• Alston, Kal, 1995, “Begging the Question: Is Critical Thinking Biased?”, Educational Theory , 45(2): 225–233. doi:10.1111/j.1741-5446.1995.00225.x
• –––, 2001, “Re/Thinking Critical Thinking: The Seductions of Everyday Life”, Studies in Philosophy and Education , 20(1): 27–40. doi:10.1023/A:1005247128053
• American Educational Research Association, 2014, Standards for Educational and Psychological Testing / American Educational Research Association, American Psychological Association, National Council on Measurement in Education , Washington, DC: American Educational Research Association.
• Anderson, Lorin W., David R. Krathwohl, Peter W. Airiasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock, 2001, A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives , New York: Longman, complete edition.
• Bailin, Sharon, 1987, “Critical and Creative Thinking”, Informal Logic , 9(1): 23–30. [ Bailin 1987 available online ]
• –––, 1988, Achieving Extraordinary Ends: An Essay on Creativity , Dordrecht: Kluwer. doi:10.1007/978-94-009-2780-3
• –––, 1995, “Is Critical Thinking Biased? Clarifications and Implications”, Educational Theory , 45(2): 191–197. doi:10.1111/j.1741-5446.1995.00191.x
• Bailin, Sharon and Mark Battersby, 2009, “Inquiry: A Dialectical Approach to Teaching Critical Thinking”, in Juho Ritola (ed.), Argument Cultures: Proceedings of OSSA 09 , CD-ROM (pp. 1–10), Windsor, ON: OSSA. [ Bailin & Battersby 2009 available online ]
• –––, 2016, “Fostering the Virtues of Inquiry”, Topoi , 35(2): 367–374. doi:10.1007/s11245-015-9307-6
• Bailin, Sharon, Roland Case, Jerrold R. Coombs, and Leroi B. Daniels, 1999a, “Common Misconceptions of Critical Thinking”, Journal of Curriculum Studies , 31(3): 269–283. doi:10.1080/002202799183124
• –––, 1999b, “Conceptualizing Critical Thinking”, Journal of Curriculum Studies , 31(3): 285–302. doi:10.1080/002202799183133
• Berman, Alan M., Seth J. Schwartz, William M. Kurtines, and Steven L. Berman, 2001, “The Process of Exploration in Identity Formation: The Role of Style and Competence”, Journal of Adolescence , 24(4): 513–528. doi:10.1006/jado.2001.0386
• Black, Beth (ed.), 2012, An A to Z of Critical Thinking , London: Continuum International Publishing Group.
• Bloom, Benjamin Samuel, Max D. Engelhart, Edward J. Furst, Walter H. Hill, and David R. Krathwohl, 1956, Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , New York: David McKay.
• Casserly, Megan, 2012, “The 10 Skills That Will Get You Hired in 2013”, Forbes , Dec. 10, 2012. Available at https://www.forbes.com/sites/meghancasserly/2012/12/10/the-10-skills-that-will-get-you-a-job-in-2013/#79e7ff4e633d ; accessed 2017 11 06.
• Center for Assessment & Improvement of Learning, 2017, Critical Thinking Assessment Test , Cookeville, TN: Tennessee Technological University.
• Cohen, Jacob, 1988, Statistical Power Analysis for the Behavioral Sciences , Hillsdale, NJ: Lawrence Erlbaum Associates, 2nd edition.
• College Board, 1983, Academic Preparation for College. What Students Need to Know and Be Able to Do , New York: College Entrance Examination Board, ERIC document ED232517.
• Commission on the Relation of School and College of the Progressive Education Association, 1943, Thirty Schools Tell Their Story , Volume V of Adventure in American Education , New York and London: Harper & Brothers.
• Council for Aid to Education, 2017, CLA+ Student Guide . Available at http://cae.org/images/uploads/pdf/CLA_Student_Guide_Institution.pdf ; accessed 2017 09 26.
• Dalgleish, Adam, Patrick Girard, and Maree Davies, 2017, “Critical Thinking, Bias and Feminist Philosophy: Building a Better Framework through Collaboration”, Informal Logic , 37(4): 351–369. [ Dalgleish et al. available online ]
• Dewey, John, 1910, How We Think , Boston: D.C. Heath. [ Dewey 1910 available online ]
• –––, 1916, Democracy and Education: An Introduction to the Philosophy of Education , New York: Macmillan.
• –––, 1933, How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process , Lexington, MA: D.C. Heath.
• –––, 1936, “The Theory of the Chicago Experiment”, Appendix II of Mayhew & Edwards 1936: 463–477.
• –––, 1938, Logic: The Theory of Inquiry , New York: Henry Holt and Company.
• Dominguez, Caroline (coord.), 2018a, A European Collection of the Critical Thinking Skills and Dispositions Needed in Different Professional Fields for the 21st Century , Vila Real, Portugal: UTAD. Available at http://bit.ly/CRITHINKEDUO1 ; accessed 2018 04 09.
• ––– (coord.), 2018b, A European Review on Critical Thinking Educational Practices in Higher Education Institutions , Vila Real: UTAD. Available at http://bit.ly/CRITHINKEDUO2 ; accessed 2018 04 14.
• Dumke, Glenn S., 1980, Chancellor’s Executive Order 338 , Long Beach, CA: California State University, Chancellor’s Office. Available at https://www.calstate.edu/eo/EO-338.pdf ; accessed 2017 11 16.
• Ennis, Robert H., 1958, “An Appraisal of the Watson-Glaser Critical Thinking Appraisal”, The Journal of Educational Research , 52(4): 155–158. doi:10.1080/00220671.1958.10882558
• –––, 1962, “A Concept of Critical Thinking: A Proposed Basis for Research on the Teaching and Evaluation of Critical Thinking Ability”, Harvard Educational Review , 32(1): 81–111.
• –––, 1981a, “A Conception of Deductive Logical Competence”, Teaching Philosophy , 4(3/4): 337–385. doi:10.5840/teachphil198143/429
• –––, 1981b, “Eight Fallacies in Bloom’s Taxonomy”, in C. J. B. Macmillan (ed.), Philosophy of Education 1980: Proceedings of the Thirty-seventh Annual Meeting of the Philosophy of Education Society , Bloomington, IL: Philosophy of Education Society, pp. 269–273.
• –––, 1984, “Problems in Testing Informal Logic, Critical Thinking, Reasoning Ability”. Informal Logic , 6(1): 3–9. [ Ennis 1984 available online ]
• –––, 1987, “A Taxonomy of Critical Thinking Dispositions and Abilities”, in Joan Boykoff Baron and Robert J. Sternberg (eds.), Teaching Thinking Skills: Theory and Practice , New York: W. H. Freeman, pp. 9–26.
• –––, 1989, “Critical Thinking and Subject Specificity: Clarification and Needed Research”, Educational Researcher , 18(3): 4–10. doi:10.3102/0013189X018003004
• –––, 1991, “Critical Thinking: A Streamlined Conception”, Teaching Philosophy , 14(1): 5–24. doi:10.5840/teachphil19911412
• –––, 1996, “Critical Thinking Dispositions: Their Nature and Assessability”, Informal Logic , 18(2–3): 165–182. [ Ennis 1996 available online ]
• –––, 1998, “Is Critical Thinking Culturally Biased?”, Teaching Philosophy , 21(1): 15–33. doi:10.5840/teachphil19982113
• –––, 2011, “Critical Thinking: Reflection and Perspective Part I”, Inquiry: Critical Thinking across the Disciplines , 26(1): 4–18. doi:10.5840/inquiryctnews20112613
• –––, 2013, “Critical Thinking across the Curriculum: The Wisdom CTAC Program”, Inquiry: Critical Thinking across the Disciplines , 28(2): 25–45. doi:10.5840/inquiryct20132828
• –––, 2016, “Definition: A Three-Dimensional Analysis with Bearing on Key Concepts”, in Patrick Bondy and Laura Benacquista (eds.), Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of the Ontario Society for the Study of Argumentation (OSSA), 18–21 May 2016 , Windsor, ON: OSSA, pp. 1–19. Available at http://scholar.uwindsor.ca/ossaarchive/OSSA11/papersandcommentaries/105 ; accessed 2017 12 02.
• –––, 2018, “Critical Thinking Across the Curriculum: A Vision”, Topoi , 37(1): 165–184. doi:10.1007/s11245-016-9401-4
• Ennis, Robert H., and Jason Millman, 1971, Manual for Cornell Critical Thinking Test, Level X, and Cornell Critical Thinking Test, Level Z , Urbana, IL: Critical Thinking Project, University of Illinois.
• Ennis, Robert H., Jason Millman, and Thomas Norbert Tomko, 1985, Cornell Critical Thinking Tests Level X & Level Z: Manual , Pacific Grove, CA: Midwest Publication, 3rd edition.
• –––, 2005, Cornell Critical Thinking Tests Level X & Level Z: Manual , Seaside, CA: Critical Thinking Company, 5th edition.
• Ennis, Robert H. and Eric Weir, 1985, The Ennis-Weir Critical Thinking Essay Test: Test, Manual, Criteria, Scoring Sheet: An Instrument for Teaching and Testing , Pacific Grove, CA: Midwest Publications.
• Facione, Peter A., 1990a, Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction , Research Findings and Recommendations Prepared for the Committee on Pre-College Philosophy of the American Philosophical Association, ERIC Document ED315423.
• –––, 1990b, California Critical Thinking Skills Test, CCTST – Form A , Millbrae, CA: The California Academic Press.
• –––, 1990c, The California Critical Thinking Skills Test--College Level. Technical Report #3. Gender, Ethnicity, Major, CT Self-Esteem, and the CCTST , ERIC Document ED326584.
• –––, 1992, California Critical Thinking Skills Test: CCTST – Form B, Millbrae, CA: The California Academic Press.
• –––, 2000, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill”, Informal Logic , 20(1): 61–84. [ Facione 2000 available online ]
• Facione, Peter A. and Noreen C. Facione, 1992, CCTDI: A Disposition Inventory , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Noreen C. Facione, and Carol Ann F. Giancarlo, 2001, California Critical Thinking Disposition Inventory: CCTDI: Inventory Manual , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Carol A. Sánchez, and Noreen C. Facione, 1994, Are College Students Disposed to Think? , Millbrae, CA: The California Academic Press. ERIC Document ED368311.
• Fisher, Alec, and Michael Scriven, 1997, Critical Thinking: Its Definition and Assessment , Norwich: Centre for Research in Critical Thinking, University of East Anglia.
• Freire, Paulo, 1968 [1970], Pedagogia do Oprimido . Translated as Pedagogy of the Oppressed , Myra Bergman Ramos (trans.), New York: Continuum, 1970.
• Glaser, Edward Maynard, 1941, An Experiment in the Development of Critical Thinking , New York: Bureau of Publications, Teachers College, Columbia University.
• Halpern, Diane F., 1998, “Teaching Critical Thinking for Transfer Across Domains: Disposition, Skills, Structure Training, and Metacognitive Monitoring”, American Psychologist , 53(4): 449–455. doi:10.1037/0003-066X.53.4.449
• –––, 2016, Manual: Halpern Critical Thinking Assessment , Mödling, Austria: Schuhfried. Available at https://drive.google.com/file/d/0BzUoP_pmwy1gdEpCR05PeW9qUzA/view ; accessed 2017 12 01.
• Hamby, Benjamin, 2014, The Virtues of Critical Thinkers , Doctoral dissertation, Philosophy, McMaster University. [ Hamby 2014 available online ]
• –––, 2015, “Willingness to Inquire: The Cardinal Critical Thinking Virtue”, in Martin Davies and Ronald Barnett (eds.), The Palgrave Handbook of Critical Thinking in Higher Education , New York: Palgrave Macmillan, pp. 77–87.
• Haynes, Ada, Elizabeth Lisic, Kevin Harris, Katie Leming, Kyle Shanks, and Barry Stein, 2015, “Using the Critical Thinking Assessment Test (CAT) as a Model for Designing Within-Course Assessments: Changing How Faculty Assess Student Learning”, Inquiry: Critical Thinking Across the Disciplines , 30(3): 38–48. doi:10.5840/inquiryct201530316
• Hitchcock, David, 2017, “Critical Thinking as an Educational Ideal”, in his On Reasoning and Argument: Essays in Informal Logic and on Critical Thinking , Dordrecht: Springer, pp. 477–497. doi:10.1007/978-3-319-53562-3_30
• hooks, bell, 1994, Teaching to Transgress: Education as the Practice of Freedom , New York and London: Routledge.
• –––, 2010, Teaching Critical Thinking: Practical Wisdom , New York and London: Routledge.
• Johnson, Ralph H., 1992, “The Problem of Defining Critical Thinking”, in Stephen P, Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 38–53.
• Kahane, Howard, 1971, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Belmont, CA: Wadsworth.
• Kahneman, Daniel, 2011, Thinking, Fast and Slow , New York: Farrar, Straus and Giroux.
• Kenyon, Tim, and Guillaume Beaulac, 2014, “Critical Thinking Education and Debasing”, Informal Logic , 34(4): 341–363. [ Kenyon & Beaulac 2014 available online ]
• Krathwohl, David R., Benjamin S. Bloom, and Bertram B. Masia, 1964, Taxonomy of Educational Objectives, Handbook II: Affective Domain , New York: David McKay.
• Kuhn, Deanna, 1991, The Skills of Argument , New York: Cambridge University Press. doi:10.1017/CBO9780511571350
• Lipman, Matthew, 1987, “Critical Thinking–What Can It Be?”, Analytic Teaching , 8(1): 5–12. [ Lipman 1987 available online ]
• Loftus, Elizabeth F., 2017, “Eavesdropping on Memory”, Annual Review of Psychology , 68: 1–18. doi:10.1146/annurev-psych-010416-044138
• Martin, Jane Roland, 1992, “Critical Thinking for a Humane World”, in Stephen P. Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 163–180.
• Mayhew, Katherine Camp, and Anna Camp Edwards, 1936, The Dewey School: The Laboratory School of the University of Chicago, 1896–1903 , New York: Appleton-Century. [ Mayhew & Edwards 1936 available online ]
• McPeck, John E., 1981, Critical Thinking and Education , New York: St. Martin’s Press.
• Nickerson, Raymond S., 1998, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises”, Review of General Psychology , 2(2): 175–220. doi:10.1037/1089-2680.2.2.175
• Nieto, Ana Maria, and Jorge Valenzuela, 2012, “A Study of the Internal Structure of Critical Thinking Dispositions”, Inquiry: Critical Thinking across the Disciplines , 27(1): 31–38. doi:10.5840/inquiryct20122713
• Norris, Stephen P., 1985, “Controlling for Background Beliefs When Developing Multiple-choice Critical Thinking Tests”, Educational Measurement: Issues and Practice , 7(3): 5–11. doi:10.1111/j.1745-3992.1988.tb00437.x
• Norris, Stephen P. and Robert H. Ennis, 1989, Evaluating Critical Thinking (The Practitioners’ Guide to Teaching Thinking Series), Pacific Grove, CA: Midwest Publications.
• Norris, Stephen P. and Ruth Elizabeth King, 1983, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1984, The Design of a Critical Thinking Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland. ERIC Document ED260083.
• –––, 1985, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1990a, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• –––, 1990b, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• Obama, Barack, 2014, State of the Union Address , January 28, 2014. [ Obama 2014 available online ]
• OCR [Oxford, Cambridge and RSA Examinations], 2011, AS/A Level GCE: Critical Thinking – H052, H452 , Cambridge: OCR. Information available at http://www.ocr.org.uk/qualifications/as-a-level-gce-critical-thinking-h052-h452/ ; accessed 2017 10 12.
• OECD [Organization for Economic Cooperation and Development] Centre for Educational Research and Innovation, 2018, Fostering and Assessing Students’ Creative and Critical Thinking Skills in Higher Education , Paris: OECD. Available at http://www.oecd.org/education/ceri/Fostering-and-assessing-students-creative-and-critical-thinking-skills-in-higher-education.pdf ; accessed 2018 04 22.
• Ontario Ministry of Education, 2013, The Ontario Curriculum Grades 9 to 12: Social Sciences and Humanities . Available at http://www.edu.gov.on.ca/eng/curriculum/secondary/ssciences9to122013.pdf ; accessed 2017 11 16.
• Passmore, John Arthur, 1980, The Philosophy of Teaching , London: Duckworth.
• Paul, Richard W., 1981, “Teaching Critical Thinking in the ‘Strong’ Sense: A Focus on Self-Deception, World Views, and a Dialectical Mode of Analysis”, Informal Logic , 4(2): 2–7. [ Paul 1981 available online ]
• –––, 1984, “Critical Thinking: Fundamental to Education for a Free Society”, Educational Leadership , 42(1): 4–14.
• –––, 1985, “McPeck’s Mistakes”, Informal Logic , 7(1): 35–43. [ Paul 1985 available online ]
• Paul, Richard W. and Linda Elder, 2006, The Miniature Guide to Critical Thinking: Concepts and Tools , Dillon Beach, CA: Foundation for Critical Thinking, 4th edition.
• Payette, Patricia, and Edna Ross, 2016, “Making a Campus-Wide Commitment to Critical Thinking: Insights and Promising Practices Utilizing the Paul-Elder Approach at the University of Louisville”, Inquiry: Critical Thinking Across the Disciplines , 31(1): 98–110. doi:10.5840/inquiryct20163118
• Possin, Kevin, 2008, “A Field Guide to Critical-Thinking Assessment”, Teaching Philosophy , 31(3): 201–228. doi:10.5840/teachphil200831324
• –––, 2013a, “Some Problems with the Halpern Critical Thinking Assessment (HCTA) Test”, Inquiry: Critical Thinking across the Disciplines , 28(3): 4–12. doi:10.5840/inquiryct201328313
• –––, 2013b, “A Serious Flaw in the Collegiate Learning Assessment (CLA) Test”, Informal Logic , 33(3): 390–405. [ Possin 2013b available online ]
• –––, 2014, “Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score”, Informal Logic , 34(4): 393–416. [ Possin 2014 available online ]
• Rawls, John, 1971, A Theory of Justice , Cambridge, MA: Harvard University Press.
• Rousseau, Jean-Jacques, 1762, Émile , Amsterdam: Jean Néaulme.
• Scheffler, Israel, 1960, The Language of Education , Springfield, IL: Charles C. Thomas.
• Scriven, Michael, and Richard W. Paul, 1987, Defining Critical Thinking , Draft statement written for the National Council for Excellence in Critical Thinking Instruction. Available at http://www.criticalthinking.org/pages/defining-critical-thinking/766 ; accessed 2017 11 29.
• Sheffield, Clarence Burton Jr., 2018, “Promoting Critical Thinking in Higher Education: My Experiences as the Inaugural Eugene H. Fram Chair in Applied Critical Thinking at Rochester Institute of Technology”, Topoi , 37(1): 155–163. doi:10.1007/s11245-016-9392-1
• Siegel, Harvey, 1985, “McPeck, Informal Logic and the Nature of Critical Thinking”, in David Nyberg (ed.), Philosophy of Education 1985: Proceedings of the Forty-First Annual Meeting of the Philosophy of Education Society , Normal, IL: Philosophy of Education Society, pp. 61–72.
• –––, 1988, Educating Reason: Rationality, Critical Thinking, and Education , New York: Routledge.
• –––, 1999, “What (Good) Are Thinking Dispositions?”, Educational Theory , 49(2): 207–221. doi:10.1111/j.1741-5446.1999.00207.x
• Simpson, Elizabeth, 1966–67, “The Classification of Educational Objectives: Psychomotor Domain”, Illinois Teacher of Home Economics , 10(4): 110–144, ERIC document ED0103613. [ Simpson 1966–67 available online ]
• Skolverket, 2011, Curriculum for the Compulsory School, Preschool Class and the Recreation Centre , Stockholm: Ordförrådet AB. Available at http://malmo.se/download/18.29c3b78a132728ecb52800034181/pdf2687.pdf ; accessed 2017 11 16.
• Smith, B. Othanel, 1953, “The Improvement of Critical Thinking”, Progressive Education , 30(5): 129–134.
• Smith, Eugene Randolph, Ralph Winfred Tyler, and the Evaluation Staff, 1942, Appraising and Recording Student Progress , Volume III of Adventure in American Education , New York and London: Harper & Brothers.
• Splitter, Laurance J., 1987, “Educational Reform through Philosophy for Children”, Thinking: The Journal of Philosophy for Children , 7(2): 32–39. doi:10.5840/thinking1987729
• Stanovich Keith E., and Paula J. Stanovich, 2010, “A Framework for Critical Thinking, Rational Thinking, and Intelligence”, in David D. Preiss and Robert J. Sternberg (eds), Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development , New York: Springer Publishing, pp 195–237.
• Stanovich Keith E., Richard F. West, and Maggie E. Toplak, 2011, “Intelligence and Rationality”, in Robert J. Sternberg and Scott Barry Kaufman (eds.), Cambridge Handbook of Intelligence , Cambridge: Cambridge University Press, 3rd edition, pp. 784–826. doi:10.1017/CBO9780511977244.040
• Tankersley, Karen, 2005, Literacy Strategies for Grades 4–12: Reinforcing the Threads of Reading , Alexandria, VA: Association for Supervision and Curriculum Development.
• Thayer-Bacon, Barbara J., 1992, “Is Modern Critical Thinking Theory Sexist?”, Inquiry: Critical Thinking Across the Disciplines , 10(1): 3–7. doi:10.5840/inquiryctnews199210123
• –––, 1993, “Caring and Its Relationship to Critical Thinking”, Educational Theory , 43(3): 323–340. doi:10.1111/j.1741-5446.1993.00323.x
• –––, 1995a, “Constructive Thinking: Personal Voice”, Journal of Thought , 30(1): 55–70.
• –––, 1995b, “Doubting and Believing: Both are Important for Critical Thinking”, Inquiry: Critical Thinking across the Disciplines , 15(2): 59–66. doi:10.5840/inquiryctnews199515226
• –––, 2000, Transforming Critical Thinking: Thinking Constructively , New York: Teachers College Press.
• Toulmin, Stephen Edelston, 1958, The Uses of Argument , Cambridge: Cambridge University Press.
• Turri, John, Mark Alfano, and John Greco, 2017, “Virtue Epistemology”, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2017 Edition). URL = < https://plato.stanford.edu/archives/win2017/entries/epistemology-virtue/ >
• Warren, Karen J. 1988. “Critical Thinking and Feminism”, Informal Logic , 10(1): 31–44. [ Warren 1988 available online ]
• Watson, Goodwin, and Edward M. Glaser, 1980a, Watson-Glaser Critical Thinking Appraisal, Form A , San Antonio, TX: Psychological Corporation.
• –––, 1980b, Watson-Glaser Critical Thinking Appraisal: Forms A and B; Manual , San Antonio, TX: Psychological Corporation,
• –––, 1994, Watson-Glaser Critical Thinking Appraisal, Form B , San Antonio, TX: Psychological Corporation.
• Weinstein, Mark, 1990, “Towards a Research Agenda for Informal Logic and Critical Thinking”, Informal Logic , 12(3): 121–143. [ Weinstein 1990 available online ]
• –––, 2013, Logic, Truth and Inquiry , London: College Publications.
• Zagzebski, Linda Trinkaus, 1996, Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge , Cambridge: Cambridge University Press. doi:10.1017/CBO9781139174763

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up this entry topic at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• Association for Informal Logic and Critical Thinking (AILACT)
• Center for Teaching Thinking (CTT)
• Critical Thinking Across the European Higher Education Curricula (CRITHINKEDU)
• Critical Thinking Definition, Instruction, and Assessment: A Rigorous Approach (criticalTHINKING.net)
• Critical Thinking Research (RAIL)
• Foundation for Critical Thinking
• Insight Assessment
• Partnership for 21st Century Learning (P21)
• The Critical Thinking Consortium
• The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions and Abilities , by Robert H. Ennis

abilities | bias, implicit | children, philosophy for | civic education | decision-making capacity | Dewey, John | dispositions | education, philosophy of | epistemology: virtue | logic: informal

Copyright © 2018 by David Hitchcock < hitchckd @ mcmaster . ca >

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2016 by The Metaphysics Research Lab , Center for the Study of Language and Information (CSLI), Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

• Open access
• Published: 11 September 2024

Curriculum, competency development, and assessment methods of MSc and PhD pharmacy programs: a scoping review

• Dana ElKhalifa 1 ,
• Ola Hussein 2 ,
• Abeer Hamid 3 ,
• Nour Al-Ziftawi 4 ,
• Israa Al-Hashimi 5 &
• Mohamed Izham Mohamed Ibrahim 2  

BMC Medical Education volume  24 , Article number:  989 ( 2024 ) Cite this article

Metrics details

Background/objectives

We aim to systematically review and evaluate the current landscape of postgraduate pharmacy education to a) identify current evidence, best practices, challenges, recommendations, and solutions; and b) develop a framework to optimize postgraduate pharmacy programs.

A scoping review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). Electronic databases, including PubMed, Scopus, EMBASE, ProQuest, Web of Science, and Google Scholar were utilized. The search covered studies published from January 2011 to September 2023. Following the principles of Arksey and O'Malley's framework, data charting and extraction were performed using a pre-designed data collection tool, followed by the synthesis and grouping of studies based on common themes.

Of the 5542 articles found, the review included 36 eligible ones focusing on pharmacy postgraduate education (PhD and MSc), grouped into three themes: 1) courses and curriculum; 2) training and skills development; 3) assessment and mentorship methods. Utilized methodologies included descriptive analyses, questionnaires, surveys, trials, and focus groups/interviews. The studies underscored the need for competency-based curricula with regular evaluations, career planning, and diverse course offerings. Identified key skills and competencies in the studies included soft skills, communication, research, desperate skills (e.g., leadership and management), and critical thinking. The studies also emphasized the value of comprehensive evaluation and peer review methods. Challenges included balancing academic and real-world requirements, training, limited resources, time constraints, and faculty workload.

Evidence-based suggestions to improve postgraduate pharmacy education include the implementation of practice-oriented courses, value of tailored/or comprehensive assessments, focus on real-world skills, effectiveness of advanced teaching methods, and mentorship role. The proposed framework can guide program enhancement and highlight the need to improve programs holistically, entailing the three themes.

Peer Review reports

Pharmacy is a dynamic discipline of science, rapidly expanding with a rising number of students pursuing postgraduate studies in the field [ 1 ]. Postgraduate education is pivotal in shaping and advancing pharmacy practice across diverse settings, effectively addressing significant challenges and bridging crucial gaps. Such a specialized knowledge would ultimately contribute to improved patient care and population health outcomes. Further, postgraduate education programs must ensure the provision of teaching across diverse specialized domains. These include, but are not limited to, professional education, drug discovery, medicinal chemistry, pharmaceutics, biotechnology, biochemistry, pharmacogenetics, pharmacokinetics, pharmacognosy, pharmacology, pharmacotherapy, pharmacoepidemiology, pharmacoeconomics, and pharmacoinformatics. Additionally, these programs should aim to contribute to advancing and improving healthcare systems, pharmacy laws and ethics, and proficiency in working with advanced machines and analytical techniques [ 2 , 3 ], all of which have positive impacts for the quality and safety of patient care and the overall health of populations.

Postgraduate pharmacy education faces a range of challenges. These include the surplus of postgraduates in traditional disciplines as compared to available emerging jobs in the market, curricula that fail to align with the demands of pharmaceutical practice settings, maintaining traditional teaching methods despite the dynamic change in the pharmaceutical industry, and advanced global practice and technology [ 4 , 5 , 6 ]. Notably, pharmacy postgraduate education in low- and middle-income nations confronts numerous challenges and gaps related to education, systems, and practice. Further, teaching methods at different universities are diverse [ 7 , 8 , 9 , 10 ]. As a result, it is unclear whether these universities are effectively optimizing and tailoring their educational strategies to meet the current needs of postgraduate students and align with the demands of pharmaceutical industries and healthcare systems [ 3 ]. Nevertheless, institutions offering postgraduate education have a fundamental responsibility to provide high-quality education, necessitating the continuous evaluation and enhancement of their curricula to align with the developing needs of future graduates and prospective employers. This holds particular significance as postgraduate students carry the expectation that their universities have designed high-quality educational programs to fulfill their diverse needs [ 4 ].

There is a noticeable absence of a definitive guide on how universities can effectively address the expanding challenges within pharmacy postgraduate education. This is primarily because accreditation bodies focus predominantly on evaluating and reviewing undergraduate curricula, neglecting the unique challenges of postgraduate education in pharmacy. Furthermore, international experiences and needs in pharmacy education vary significantly between countries. This raises the following research question: what insights, perspectives, challenges, and recommendations can inform the optimization of postgraduate (PhD and MSc) pharmacy programs at universities worldwide? To answer this question, it is essential to conduct this scoping review to systematically chart the available evidence and understand the current body of knowledge about pharmacy postgraduate education. Through this endeavor, our objectives are a) to identify current insights, perspectives, challenges, and recommendations that can assist various postgraduate pharmacy programs in addressing potential gaps within their systems and possibly refining their existing educational structures (e.g., curricula) and approaches (e.g., educational methods) to enhance the overall learning process for their students; and b) to develop a framework to optimize postgraduate pharmacy programs.

Study design

We conducted a scoping review to synthesize and map the available evidence and identify a framework for improving educational programs for postgraduate degrees in pharmacy. Scoping reviews tackle broad subjects and usually aim to recognize research gaps in the existing literature [ 11 ]. While conducting this review, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist, which contains 22 reporting items [ 12 ]. Our filled PRISMA-ScR checklist for this scoping review is included in Supplementary Material 1.

The study protocol was drafted and reviewed using the Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocols (PRISMA-P) checklist [ 13 ]. While it was not previously published, it is available as supplementary material (Supplementary Material 2).

Eligibility criteria

Eligibility criteria for studies included in this scoping review: i) Studies published in peer-reviewed journals; ii) The primary focus of the studies should be on curriculum and education development within MSc and PhD Pharmacy programs; iii) Studies needed to discuss related aspects such as competencies, assessment methods, and courses; iv) They need to be published in English between the period of January 2011- September 2023; v) To encompass various aspects of graduate pharmacy education, studies were included if they employed qualitative, quantitative, or mixed-method study designs.

Conversely, studies were excluded if they: i) Focused on public health, PharmD, diploma, or clinical practice-based master programs that do not include research/thesis, as these programs often have distinct educational objectives compared to research-based postgraduate programs; ii) Addressed initiatives to improve research unrelated to postgraduate programs, because the focus of this review is solely on postgraduate education; iii) Were centered on dual pharmacy (PharmD)/master of public health (MPH), as these programs are mainly interdisciplinary in nature and do not specifically reflect the unique challenges of research-oriented programs; iv) Focused on genetic counseling, because this field has a distinct aim and is not directly related to postgraduate research-based pharmacy education; v) Were associated with other non-pharmacy-related programs, as our focus is on Pharmacy; vi) Focused solely on university facilities, because our target is the educational content; vii) Were categorized as commentaries or review articles, to avoid bias in reporting and prioritize original research content.

Information sources/literature search

The search for relevant studies was conducted on PubMed, EMBASE, Scopus, ProQuest, Web of Science, and Google Scholar to identify relevant studies published between January 2011 and September 2023. The search strategy utilized related keywords: postgraduate, higher education, graduate, PhD, MSc, masters, education, curriculum, courses, syllabus, skills, competencies, assessment, evaluation, pharmacy, and pharmaceutical sciences. Search limits were applied to the title/abstract and English language. Three investigators independently performed the initial screening of the titles and abstracts to identify eligible articles. Discrepancies were resolved through discussion and agreement. Specialized journals were also specifically reached to identify relevant articles, specifically the American Journal of Pharmaceutical Education, Journal of Medical Education and Curricular Development, Currents in Pharmacy Teaching and Learning, Pharmacy Education, European Journal of Education, Journal of Pharmacy Practice and Research, and Health Education Journal. The final search strategy for each database is presented in Table S1 (Supplementary Material 3). Finally, the removal of duplicates, title/abstract screening, and full-text screening were conducted using the Rayyan application [ 14 ].

Studies selection and data charting

Three reviewers independently screened all included citations and full-text articles and agreed on their eligibility. A standardized data extraction tool was created using Microsoft Excel and utilized to chart data from all eligible articles. In addition, the following information was collected independently by three reviewers: authors, year of publication, focus of the study, title, relevant/irrelevant, objectives, country, challenges, recommendations, and conclusion. Discrepancies were resolved through discussion and agreement between the authors.

We followed the framework proposed by Arksey and 'O'Malley for data synthesis and charting [ 11 ]. Eligible studies were grouped based on common themes. Our grouping focused on the following three themes in Table  1 : i) Courses, curriculum, and syllabus; ii) Training, competencies, and skills development; and iii) Assessment, evaluation, and mentorship methods.

Development of the conceptual model

To develop a comprehensive conceptual model guiding the creation of collective, high-quality pharmacy postgraduate (MSc/PhD) programs, we conducted a rigorous literature review focusing on the challenges, recommendations, factors, and successful interventions. To synthesize this information, we employed the Arksey and 'O'Malley framework for data synthesis and charting. The model development process involved the following steps:

Identification of key themes: Based on the literature review, three primary themes emerged as critical for postgraduate pharmacy program development:

Courses, curriculum, and syllabus

Training, competencies, and skill development

Assessment, evaluation, and mentorship methods

Model construction: A conceptual model was constructed around these themes, incorporating essential components, including:

Curriculum design and development, including necessary and optional elements

Competency-based curriculum development

Training and skills development aligned with student, program, and job market needs

Diverse assessment and evaluation methods to measure program effectiveness, student learning, and job market impact

Successful interventions and international experiences

Model enrichment: To ensure comprehensiveness, the model was expanded to include additional factors and emerging trends deemed important to the study team. For instance, under the "courses and curriculum" theme, we incorporated elements like needs assessment, regular evaluation, and program refinement to promote the concept of program sustainability. Additionally, we explored the potential of using advanced tools like artificial intelligence for assessment, evaluation, and mentorship, based on what has been reported within the included studies.

Overall, this systematic approach, grounded in both literature and practical examples, resulted in a robust conceptual model to inform the development and evaluation of collective, high-quality pharmacy postgraduate programs.

Literature search

After removing duplicates, 5542 articles were identified from the different searched databases (Fig.  1 ). After titles and abstracts screening, 5461 citations were excluded because they matched our exclusion criteria, leaving 81 full-text articles to be further assessed for eligibility. Among them, 45 were excluded and summarized with their exclusion reasons in Table S3 in Supplementary Material 3. The remaining articles ( n  = 36) matched our inclusion criteria and were included in this scoping review.

PRISMA flow diagram of the studies selection process

Study characteristics

Based on their primary focus, the included studies were classified into three commonly identified themes, as defined in Table  1 . Study characteristics are summarized in Table  2 , including the study authors, publication year, focus, objectives, place of origin, design, and main findings. Among them, 14 addressed courses, curriculum, and syllabus issues; 9 discussed training, competencies, and skills development; and 13 targeted topics pertaining to assessment, evaluation, and mentorship methods (Fig.  2 ). The studies implemented various designs, including quantitative, qualitative, and mixed-method (Fig.  2 ).

Distribution of the research methods employed in the included studies per identified theme

The included publications on various pharmacy postgraduate educational programs (MSc, PhD, or both) were segregated based on their focus on the three themes (Fig.  3 ). Notably, most research articles concentrated on master’s programs compared to PhD programs, validating that more research is conducted on this program type (Fig.  3 ). In addition, the distribution of research on Master programs across the three themes revealed a larger number of publications focusing on courses, curriculum, and syllabus (Fig.  3 ). In contrast, research on PhD programs disclosed that training, competencies, and skills development garnered the most attention, implying a distinct focus on research efforts and underlining the necessity of competencies/skills development for PhD graduates (Fig.  3 ). Details on the distribution of the articles by country are outlined in Fig.  4 a.

A radar chart comparing the three identified themes based on the postgraduate program type

Distribution of research articles: ( a ) Overall 36 articles by country; ( b ) Articles based on country and identified theme

Courses, curriculum and syllabus

As outlined in Fig.  2 , 14 publications were dedicated to enhancing courses and curricula for various specialized MSc and PhD programs. These originated from nine countries, and as outlined in Fig.  4 b, most of these studies were conducted in the USA ( n  = 5, 35.71%), followed by Australia ( n  = 2, 14.29%). Additional contributing countries encompassed Jordan, Iran, Portugal, Malaysia, the Netherlands, Switzerland, and China (Table  2 ). Five of the identified studies emphasized the importance of specialized courses for improving postgraduate education (Table  2 ), such as research ethics, preparation for future faculty roles, pharmacoinformatics, and laboratory experience [ 15 , 16 , 17 , 18 , 19 ]. In the context of curriculum development, nine studies specifically addressed the design, format, review, and restructuring of postgraduate pharmacy programs to meet students' present and future needs [ 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. Methodologies utilized in these studies included descriptive analyses, cross-sectional questionnaires, and surveys, as well as the incorporation of focus groups and interviews (Table  2 ).

Training, competencies and skills development

Nine published studies evaluated specific skills necessary for inclusion in postgraduate curricula (Fig.  2 ). Most of these studies originated from the USA ( n  = 7, 77.7%), with the remaining two published in India and Poland (Fig.  4 a). Most studies ( n  = 8, 88.9%) primarily focused on PhD students, while only two included MSc students (Fig.  3 ). The studies shed light on the need for developing competencies and skills in research, curriculum development, communication, health administration and leadership, industrial training, and critical thinking and problem-solving (Table  2 ). Study designs were varied and encompassed mixed-methods, cross-sectional surveys, retrospective studies, interviews, and descriptive studies (Table  2 ).

Assessment, evaluation and mentorship methods

The third group of studies ( n  = 13) embraced approaches and criteria for assessing students' performance, methods for delivering certain lectures, and mentorship (Table  2 ). Consistent with the previous themes, the majority of the studies ( n  = 5, 38.46%) were based in the USA, while two articles originated from Jordan ( n  = 2, 15.38%) (Fig.  4 ). The remaining studies originated in the United Kingdom, India, Denmark, China, Australia, and Ukraine. Six studies within this group focused on assessment methods, emphasizing the importance of comprehensive evaluation and peer review (Table  2 ) [ 38 , 39 , 40 , 41 , 42 , 43 ]. Two studies specifically emphasized the significance of evaluating students' performance without merely focusing on testing theoretical understanding [ 41 , 42 ] (Table  2 ). Additionally, four studies advocated the added value of artificial intelligence and computer-based tools in delivering lectures and elucidating certain life sciences concepts [ 44 , 45 , 46 , 47 ] (Table  2 ). Three studies evaluated postgraduate mentorship models, highlighting the positive impact of dual mentors [ 48 , 49 , 50 ] (Table  2 ).

The conceptual model content

To visualize the relationships among the identified themes and relevant key components, we developed a conceptual model (Fig.  5 ). Figure 5  presents the final summary derived from our comprehensive literature review of the core elements proposed for a collective high-quality pharmacy postgraduate (MSc/PhD) educational program. To our knowledge, this is the first model to be developed and published on the specific topic of pharmacy postgraduate educational programs. All the identified insights, perspectives, challenges, and recommendations that can inform the optimization of postgraduate pharmacy programs at universities worldwide have been grouped together to allow for the development of this coherent model outlining the following themes:

A conceptual framework for developing a collective high-quality Pharmacy postgraduate (MSc/PhD) educational program. This illustration was developed based on the recommendations discussed in the literature from the relevant identified themes

This theme encompasses curriculum design and development. Studies investigating the key concept of curriculum design and development, particularly emphasizing the importance of aligning the curriculum to concepts of competency-based learning, job market demands, career planning, collaboration, research, specialization areas, and others. It was noted that a number of concepts are needed to develop a comprehensive postgraduate program that is essential for graduates’ success and meets the needs of the job market.

Training, competencies and skill development

This theme included the main components a program could target to implement effective change in students’ knowledge, skills, attitudes, and competencies required in real-world practice. It was noted that skills relevant to other disciplines, e.g., leadership and management, data science, as well as various non-science communication skills, are highly essential for the modern job market. Additionally, the necessity of providing students with training on additional non-academic career skills was deemed crucial and had a positive influence on students’ skills and competencies, as well as the effectiveness of programs to produce graduates who can meet the demands of employers in the modern job market.

This theme features the role of effective assessment and mentorship in program development and student success. Various assessment strategies, such as mentorship, advanced assessment tools, merged assessment, and peer assessment, were all positive experiences reported in the literature with the potential benefit of conducting successful assessments of students learning and intervention effectiveness, both of which are related to successful program implementation and development.

Principal findings

This scoping review aimed to map the existing literature on postgraduate education in pharmacy systematically. We identified 36 primary studies addressing the curriculum, skills development, and/or assessment approaches of postgraduate (MSc and PhD) pharmacy programs on an international scale. The studies were categorized and will be discussed based on their primary focus into three distinct themes: i) courses, curriculum, and syllabus issues; ii) training, competencies, and skills development; and iii) assessment, evaluation, and mentorship methods.

A group of studies explored the importance of different universal and specific courses within diverse pharmacy postgraduate curricula.

One of the crucial topics explored in the literature for its critical value in pharmacy postgraduate education is research ethics (RE). RE education is an integral component that equips students with the necessary skills to adhere to ethical standards when designing and conducting clinical and biomedical research [ 51 , 52 ]. Unfortunately, the findings indicate that ethics training in postgraduate programs related to pharmacy and medical sciences remains insufficient, particularly in developing countries such as Jordan and Iran [ 17 , 19 , 42 ]. Ahmed et al. also reported that, on a global scale, only 10% of research-based master programs offered standalone research ethics courses, and 40% offered some discussions within their curriculum [ 19 ]. Consequently, it is recommended that postgraduate schools consider integrating comprehensive research ethics training into their curricula, especially in developing countries. Furthermore, there is a need for additional studies in developed countries to explore this aspect further.

While many students pursue higher education to enter academia, postgraduate schools often prioritize research skills over teaching skills [ 53 ]. Teaching assistantships can offer students interested in academia practical teaching experiences [ 54 ]. However, not all students can access such opportunities, necessitating a compromise. For instance, integrating embedded lectures and courses into the curriculum can provide a viable solution. Preparing Future Faculty (PFF) was a valuable course to prepare graduate students and postdoctoral fellows for academic teaching [ 15 ]. The course syllabus included practical teaching experience and lecture delivery under dual supervision [ 15 ]. By the end of the course, many PFF graduates could secure faculty positions, and the course was distinguished as sustainable and valuable for students planning to work in academia [ 15 ]. The same course was taught to doctoral public health students and it showed excellent outcomes [ 55 ]. Similar courses can be adopted in postgraduate programs, especially PhD, as optional electives for students interested in academia.

A study by Fox et al. highlighted the urgent need to include pharmacoinformatics courses in pharmacy master's programs [ 18 ]. In particular, careers in pharmacoinformatics require students to have advanced conceptual knowledge and hands-on experiential education [ 56 ]. Recommended lecture topics encompass drug formulary management, advanced pharmacy and medical informatics, supply chain management, evidence-based medicine, and health policy [ 18 ]. Notably, the study revealed higher expectations for pharmacoinformatics knowledge for MSc graduates compared to their BSc counterparts, emphasizing the necessity for developing a comprehensive postgraduate pharmacoinformatics curriculum [ 18 ].

Another important curricular element is laboratory experience, particularly in basic sciences postgraduate programs. For example, the Non-Stop Lab Week (NSLW) was formed as part of the master's program at the University of Aveiro, Portugal, to equip students with real-life lab experience [ 16 ]. Over 1 week, students independently conducted molecular assay projects in an environment mirroring a real laboratory setting [ 16 ]. Most students found the NSLW's intensity very suitable and beneficial for their careers [ 16 ]. After graduation, they found this exposure to be similar to their experience in their current workplaces [ 16 ]. Often, postgraduate students focus solely on specific skills aligned with their thesis supervisor's area of expertise, potentially missing out on essential skills for future roles in the pharmaceutical industry. Therefore, experiences like the NSLW help expose students to the actual work environment. Likewise, programs may add curricular modules for students to get hands-on exposure to different research projects during their first semester, offering insights into potential future careers and a great scientific breadth while connecting with potential thesis supervisors.

Concentrating on a few specific courses is insufficient, and crafting a comprehensive curriculum poses a complex challenge [ 57 ]. There is a notable shift towards Competency-Based Education (CBE) in contemporary postgraduate and undergraduate pharmacy and medical education systems due to its demonstrated effectiveness [ 58 , 59 , 60 ]. CBE occurs when a curriculum incorporates comprehensive tasks, such as systems of instruction, didactic and experiential courses, and assessments to demonstrate proficiency in taught skills and concepts [ 60 ]. Various studies advocated incorporating CBE when developing postgraduate pharmacy curricula [ 21 , 25 , 26 , 27 ]. Keller et al. suggested some curricular components and building blocks to be included in postgraduate pharmacy education, encompassing the decision on core competencies, foundational concepts, lectures, syllabus, thematic training, research seminars, research integrity, supervision, student feedback, evaluation, assessment, stipends and financial support, and alumni networking [ 25 ]. They also proposed a set of competencies to be taught in PhD health sciences programs, categorized into three domains [ 25 ]:

Scientific knowledge: information literacy, research methods, scientific writing, ethics and integrity, and professional conduct.

Management and Organization: self-management, project management, and teaching.

Leadership and personal: leadership and communication.

Pharmacy education continues to adapt to the evolving needs of diverse pharmacy career paths. Initiatives for developing curriculum recommendations have been undertaken, focusing on equipping graduates with knowledge and skills for future career paths. A key initiative is the American Association of Colleges of Pharmacy (AACP) Research and Graduate Affairs Committee report [ 20 ]. The AACP report was developed based on data from different universities within the USA [ 21 ]. It addressed universal skills applicable to all pharmacy disciplines to be incorporated into different curricula [ 21 ]. These identified skills were grouped into five domains [ 21 ]:

Foundational knowledge.

Scientific communications.

Leadership and management.

Personal and professional development.

The three key proposals endorsed by the report were the need to concentrate on career guidance, external peer review, and preparing students for roles in academia [ 21 ]. In another study by Koster et al., three distinct pharmacy-related master's programs tailored for community or hospital pharmacists were described [ 27 ]. These programs were adapted to pharmacy education based on the CanMEDS framework, which originally describes the required skills for physicians to effectively address the needs of the individuals they serve [ 27 ]. In addition, the authors highlighted the importance of experiential (workplace) education over traditional on-campus education and the need to expose the students to a mixture of both [ 27 ].

A very important curriculum component is career planning and professional skills development. This is particularly important because many postgraduates move into postdoctoral training, even if they do not plan to take a research career path [ 61 ]. Regrettably, career discussions usually happen close to graduation [ 20 ]. Indeed, early career guidance and mentoring, ideally at the program's commencement, would empower students to make more informed decisions about their future career paths. Traditionally, pursuing a PhD was synonymous with academic positions. Still, this perception has evolved in the last decade, necessitating an educational shift to prepare students for broader career options [ 20 ]. The current job market reveals a growing "supply–demand" gap, with limited academic sector vacancies and an increasing number of postgraduate students graduating annually. Therefore, there must be a shift to diversify curriculum content, gearing it towards paths beyond traditional academic careers. For example, in a study by Fuhrmann et al., biomedical PhD students indicated that they were considering various career paths (research and non-research), which underlines the necessity for a comprehensive doctoral curriculum [ 20 ]. To aid students in achieving their career goals, the development of their plans, including career planning and professional skills training, can be encouraged through discussions with program mentors. Moreover, programs may allow flexible mandatory electives where students can select their preferred courses based on their constructed career plans.

It is crucial to have regular curriculum revisions to ensure that the educational content remains current and aligns with the expanding industry requirements and needs. An example of these revisions was published by Allen et al., in which a pharmaceutical medicine curriculum was reviewed at an Australian university based on cross-sectional survey findings to identify required updates to the program [ 24 ]. They developed a two-year, part-time, fully online program with interactive assessments to support students' career goals [ 24 ]. In another study, Barrett et al. presented a qualitative description of an established Master’s program in drug discovery and development [ 22 ]. The program was initially a course that was refined and expanded based on student and market demands, covering various stages of drug development [ 22 ]. The curriculum encompasses topics delivered as courses by different colleges, including epidemiology, nanotechnology, pharmacogenomics, and project management [ 22 ]. The authors reported that most program graduates secured jobs in the pharmaceutical industry upon graduation, emphasizing the significance of regular program evaluations and refinement [ 22 ]. Similarly, Lypson et al. outlined the newly adopted program evaluation process at the University of Michigan Health System, involving dedicated faculty and formal resident members [ 23 ]. This process also benefits from standardization of meetings, content experts, a transition from paper to electronic committee materials, and a focus on continuous improvement efforts for the program [ 23 ].

To uphold the quality of pharmacy postgraduate programs, supervisors must ensure students meet program requirements and graduate efficiently. Incorporating blended learning, which combines online and on-campus classroom experiences, has been suggested as a valuable learning tool [ 62 , 63 ]. Furthermore, introducing dual postgraduate degrees alongside undergraduate education in pharmacy programs can enable students to attain advanced degrees in a shorter duration efficiently. Implementing a hybrid teaching format can also be helpful, particularly for working professionals.

Training, skills and competencies development

Student preparation should extend beyond curricular coursework to encompass practical training and skill development, including cultivating critical thinking skills. Postgraduate pharmacy students must acquire skills and competencies to excel in their future roles. While some skills may be specific to student specialization, others are universally applicable and should be integrated into most specialized medical and pharmaceutical programs. Furthermore, the current job market demands more than traditional scientific research skills. Students may also need to demonstrate disparate skills in business, policy management, and advanced technologies. Therefore, the curriculum should incorporate relevant course content to address these multifaceted requirements. Competencies are frequently defined as meaningful job-related skills, knowledge, attitudes, and abilities essential for competent performance in distinct professions [ 60 ]. Key skills and competencies highlighted in the literature regarding pharmacy postgraduate education encompass research competencies, curriculum development training, communication skills, health administration and leadership training, industrial training, and critical thinking and problem-solving.

In a study published by Poloyac et al., core research competencies for a PhD program were developed in a clinical pharmaceutical sciences curriculum [ 29 ]. Eight major competencies were identified for students to integrate preclinical and clinical evidence into their research successfully [ 29 ]. These competencies included: i) literature review and evaluation; ii) hypothesis generation; iii) research methods and study design; iv) statistical methods and data evaluation; v) grantsmanship; vi) presentation and delivery of oral and written scientific information; vii) ethical conduct of research; viii) leadership, management, and multidisciplinary teamwork [ 29 ]. Each category features subcategories of competencies, and evaluation rubrics were created to assess students' performance [ 29 ]. These competencies provide a valuable framework that can be adapted for other research-based postgraduate programs.

As previously discussed, some students pursue higher education to enter academia, emphasizing the need to acquire essential competencies to excel in their potential roles. Given that curriculum development and revision are ongoing and dynamic processes, training postgraduate pharmacy students on curriculum development becomes invaluable for those aspiring to pursue an academic career in pharmacy. Newton et al. demonstrated the effectiveness of incorporating a faculty simulation of curriculum development seminar for MSc and PhD pharmacy students, offering a practical and successful tool to prepare them for the responsibilities associated with academic roles [ 30 ].

Research, being a multifaceted interdisciplinary field, demands excellent communication skills. Thus, students must undergo training in presentation, negotiation, and conflict management skills. Additionally, employers' appreciation of diverse soft skills highlights the importance of cultivating a broad skill set in graduates [ 64 ]. Studies have highlighted substantial benefits for doctoral pharmacy students who received training to enhance their communication skills, improve confidence in discussing findings, and enhance public speaking abilities [ 31 , 32 ]. Therefore, integrating courses and lectures focused on communication into the curriculum emerges as an invaluable component, aiding students in cultivating and strengthening their personal and interpersonal communication capabilities.

Specialized programs often require students to develop unique skills and competencies relevant to their areas of study. For instance, a master’s degree in health-system pharmacy administration and leadership training (HSPAL) was a novel program developed within the Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill [ 33 ]. That program combined Master’s education with practical HSPAL residency [ 33 ]. The program was designed to provide a balanced curriculum encompassing leadership, management, clinical, administrative, and didactic courses to prepare students for pharmacy administrative positions and leadership careers [ 33 ]. The program indicated attainment of the main core competencies and outcomes by enrolled students and graduates [ 33 ]. Furthermore, supervisors noted a greater likelihood of hiring graduates for administrative positions [ 33 ].

Many pharmacy programs often encompass laboratory components involving traditional basic sciences practice labs. However, there is a recognized need to integrate elements that provide students with the necessary knowledge for pharmaceutical industry practice, particularly those aspiring to work in drug discovery and development. A study revealed that most graduate programs inadequately address industry-related skills, emphasizing the importance of incorporating experiences that better prepare graduates for non-academic careers [ 35 ]. In another study by McLaughlin et al., a qualitative analysis of employers’ expectations for pharmaceutical sciences PhD graduates was conducted to understand the skills sought by employers [ 34 ]. The authors identified themes such as depth and breadth of knowledge, collaboration, communication, adaptability, experiential training, research productivity, and motivation [ 34 ]. Thus, integrating a holistic lab experience throughout the study duration, rather than limiting exposure to technical skills, can add significant value. This could be implemented by placing students in local and international pharmaceutical industries for mandatory practical experiences.

Critical thinking and problem-solving are among the highly desired skills in pharmacy postgraduates and are key to successful research conduct and evaluation of published evidence. However, various barriers may hinder their acquisition, including students' perceptions, limited metacognitive skills, biases, and the need for effortful thinking [ 65 ]. Though challenging, developing and nurturing these skills is not impossible in a thoughtful and encouraging educational environment. Research from India and Poland explored these skills in pharmacy postgraduate education [ 36 , 37 ]. Research from India investigated the performance and perception of students and their instructors regarding utilizing critical appraisal tools [ 37 ]. Both students and instructors reported that journal club (JC) criticism activities were vital in pharmacy postgraduate education, contributing to an enhancement in critical appraisal skills among participating students [ 37 ]. Research from Poland reported that graduates had insufficient knowledge of and attitudes toward evidence-based pharmacy, especially in their critical appraisal of scientific articles and problem-solving skills [ 36 ]. The study suggested that blended learning, combining classroom and online multi-module courses, could enhance the learning experience [ 36 ]. Further research evaluating critical-thinking and problem-solving training in pharmacy postgraduate education across diverse countries is essential to draw wide-ranging conclusions and recommend improvements in relevant curricula.

Assessment, evaluation and mentorship

Competency-based education proves beneficial when students' competence is continually assessed throughout the program [ 66 , 67 ]. A well-defined course syllabus should outline the timeline, assessment approaches, deadlines, and submissions and emphasize feedback and constructive criticism [ 66 , 67 ]. Course instructors should decide on the course objectives and identify potentially relevant embedded assessment tools to achieve these goals [ 40 ]. For instance, a program-level assessment process was developed for an MSc in Pharmaceutical Sciences program using an iterative data collection process, peer evaluation, and discussions [ 40 ]. The main assessment domains were cognitive (knowledge-based), affective (emotion-based), and psychomotor (action-based) [ 68 ]. The assessment was developed utilizing Bloom's taxonomy, which includes cognitive (knowledge-based), affective (emotion-based), and psychomotor (action-based) domains, and can guide the setting of course goals based on complexity and specificity [ 40 , 68 ]. Program success can be evaluated through student evaluations and feedback on course content, format, assessment methods, and suggestions [ 40 , 68 ].

A set of publications explored the performance assessment of pharmacy postgraduate students. In a descriptive study by Robinson et al., a comprehensive competency review assessing postgraduates' competencies was discussed [ 38 ]. Students were required to provide written evidence for each competency, and the instructor would either accept it if found satisfactory or reject it while requesting a rewritten version to ensure the development of the required competencies [ 38 ]. Based on students' performance, the faculty member may suggest elective courses during the program's second half [ 38 ]. Similarly, in a National Institutes of Health (NIH) Grant Application Writing Assessment for pharmacology postgraduate students, grades improved considerably upon resubmission, with survey responses indicating increased student confidence in grant writing capability [ 39 ]. The study emphasized improving writing skills through writing, revision, submission, constructive feedback, rewriting, and resubmitting [ 39 ]. Together, these findings underline the importance of midpoint evaluations for various competencies. While this process may be time-consuming for students and faculty, it is considered a worthwhile investment in time, cost, and effort.

Assessing students' understanding of theoretical concepts alone may not be sufficient; they should also be evaluated based on their actions and practical applications. For instance, a study conducted in Jordan reported a high theoretical understanding of plagiarism among pharmacy postgraduate students. However, when given practical assignments, overall performance was unsatisfactory, revealing a high prevalence of plagiarism [ 41 ]. Therefore, educational institutions should ensure students learn various paraphrasing methods and are educated on useful references for plagiarism checking. In another study conducted in the same country, the adherence rate of postgraduate students to ethical standards related to data confidentiality and informed consent when dealing with human subjects was also inadequate [ 42 ]. Hence, assessing students’ performance in applying theoretical concepts is also recommended to ensure they are highly competent in real-world settings.

The careful selection of tools and methods for delivering lectures is crucial, especially in the context of advancing technologies and artificial intelligence. In postgraduate education, 3D virtual computer simulation methods were deemed advantageous [ 44 , 46 ]. In a randomized controlled study, using 3D technology to demonstrate drug-receptor interactions significantly enhanced students’ understanding and performance compared to traditional 2D graphics [ 44 ]. Similarly, a computer-simulated method in experimental animal modeling in postgraduate pharmacology improved the experimental outcomes and confidence when conducted before an isolated live tissue-based bioassay [ 46 ]. Moreover, a pilot study assessing the value of technology (Lecture Tools) as an active learning method in teaching pharmacokinetics and pharmacodynamics demonstrated a positive experience [ 45 ]. Lecture Tools is a cloud-based system that permits various question designs, student participation, and in-class evaluations [ 45 ]. Students can use any smart device, like laptops, tablets, or mobile phones, and take notes within the same slide of the teaching presentation [ 45 ]. Despite the provision of real-time interactions, there are limitations, including weak faculty preparedness for using this tool, the time required for preparing the lecture slides, and limited lecture time [ 45 ]. Other tools have also proven valuable for undergraduate and postgraduate pharmacy education, offering interactive and easily accessible sessions, such as Coursera and EdX platforms, Socrative, Yammer, and the Lecture Capture System [ 69 , 70 , 71 ].

Postgraduate supervision is crucial to students' success, emphasizing the need for high-quality and sufficient mentorship. Swedish PhD students indicated that poor supervision prolonged their studies and delayed the completion of their thesis projects [ 49 ]. Every student has the right to guarantee that their mentorship is provided by qualified supervisors capable of effectively mentoring postgraduate students. When interviewed, supervisors expressed a need for training regarding the required instructions, guidance, and clarification of their roles as mentors [ 49 ]. In a study by Yue et al., which investigated Master’s mentor competence, it was reported that a mentor’s development can be encouraged via supportive policy, time, and appropriate programs [ 50 ]. Secondly, the mentor’s competencies should be assessed through mentor training and evaluation [ 50 ]. Lastly, mentors should endorse all competencies voluntarily [ 50 ]. The dual-mentorship model is a promising key initiative to improve mentorship in postgraduate education. Soucy et al. advocated for the dual-mentored PhD model, where each student is supervised by two expert mentors from different organizations, leading to superior outcomes [ 48 ]. Graduates of this program demonstrated great success, graduating two years earlier than traditional Ph.D. program students without compromising the outcomes [ 48 ].

After discussing each theme in detail, we employed a visual presentation to summarize the major identified challenges (Fig.  6 ), offering educators and readers an overview of the current potential challenges. Understanding these barriers can ensure that postgraduate pharmacy programs are effective and subject to continuous improvement. Table S4 (Supplementary Material 3) provides a detailed explanation of the identified challenges.

A summary of the major recognized challenges from each of the three identified themes

Study recommendations

Based on the insights driven from this scoping review, we have synthesized and developed a conceptual framework outlining an optimal structure for Pharmacy postgraduate programs (Fig.  5 ). This framework elucidates evidence-based recommendations for universities to improve the educational experience for students and for refining pharmacy postgraduate programs. While implementing this framework, it is noteworthy that research and improvement efforts should be tailored to each program context and capacity.

This model can be utilized by various stakeholders. The use of such a model should be tailored to the specific target audience and the overall context. Several stakeholders could benefit from the model; for example, investigators could focus their research on a theme or a subcategory to develop and examine the effectiveness of an intervention. Likewise, postgraduate students could use this model to identify the key knowledge areas, skills, and competencies they need to master in order for them to stay ahead of the continuously changing demands of the job market. Additionally, educators, management teams, and administrators at postgraduate programs could use the model for the continuous development and refinement of their postgraduate programs.

This scoping review highlights various recommendations to be explored in future research efforts. First, all included studies were observational and descriptive, with only one randomized controlled trial (RCT) and a limited number of mixed-method studies. Thus, we suggest the need for well-designed RCTs and mixed-method research studies evaluating postgraduate programs focusing on the three presented themes. RCTs would provide valuable high-level evidence to support future research and practice applications. At the same time, mixed-method studies can facilitate the collection and evaluation of unique quantitative and qualitative data in individual program contexts. Second, future research should investigate educational programs from the need assessment and/or program objective development stage to the final evaluation of programs and their improvements. Exploring the utilization of well-established frameworks from the literature will allow the development of research and/or program evaluation following a systematic and comprehensive approach. Third, there are few publications on the assessment and evaluation methods. Accordingly, investigating this theme can provide valuable information on the effectiveness of the implemented programs and guide the process of program improvement and development. Fourth, our findings suggest that research involving MSc programs focuses more on courses, curriculum, and syllabus topics and less on training, competencies, and skills development. Given the growing need for competent professionals, investigating training and competencies within MSc programs will assist in preparing competent graduates. On the other hand, there were limited studies exploring courses and curriculum topics in PhD programs. Therefore, studies investigating the development or evaluation of PhD-tailored curricula and courses, particularly those focusing on competency-based education, should be considered in future research efforts. Finally, we have identified a need for additional international research efforts, from both developing and developed countries, to advance postgraduate pharmacy education on a global scale.

Study strengths and limitations

Strengths and limitations inherent to the scoping review.

It is noteworthy that this review is, to our knowledge, the first to systematically synthesize and chart available evidence on pharmacy postgraduate (MSc and PhD) education. Such a systematic approach offered many strengths relevant to conceptual and theoretical aspects and other strengths relevant to the standard methodology utilized in this scoping review. Studies discussed educational program implementation in detail, which provided a comprehensive overview and opportunity to learn from programs at various stages of implementation (i.e., both programs at advanced stages with successful implementation practices and nascent programs with identified improvement opportunities). The details provided in this review and in individual studies could be utilized to inform the implementation and improvement of other programs at the international level. Moreover, the included studies utilized diverse research methodologies and offered valuable insights into the current literature landscape on MSc and PhD pharmacy programs. The review also resulted in the development of an evidence-based conceptual framework for enhancing pharmacy postgraduate education. Further, included herein are outcomes of the examination of the postgraduate pharmacy educational curriculum, competency development, and assessment methods. Another strength of this study is the use of these defined themes to guide the framework constriction, analysis and presentation of findings. Additionally, methodological strengths included a) utilization of standard methodology (i.e., PRISMA-ScR) to conduct this review; b) employment of the framework proposed by Arksey and 'O'Malley for data synthesis and charting; c) utilization of a comprehensive search strategy documented in the supplementary material to increase the transparency and replicability of the search strategy; and d) utilization of major databases and journals relevant to the field of pharmacy education research to ensure comprehensiveness. Despite these strengths, this review has some limitations. Firstly, due to its scoping nature, the outcomes of the studies were not assessed using formal quality assessment tools; thus, interpretation of findings and efforts to implement any intervention or recommendation would require further investigation. However, to ensure the inclusion of high-quality data and to mitigate this limitation, we included articles from peer-reviewed journals only. Secondly, the included studies were descriptive and observational, with only one RCT. Thus, well-designed RCT studies evaluating pharmacy postgraduate programs are recommended for future research efforts. Lastly, the inclusion criteria were limited to studies published in English between 2011 and 2023; this might affect the inclusion of articles published in non-English or before 2011.

Limitations of the conceptual model

Although this model can provide a valuable foundation for developing collective, high-quality pharmacy postgraduate programs, there are some limitations to take into consideration before utilizing or interpreting the information provided, which include:

First, concerning methodological rigor, there is a need for a more robust and structured methodology for developing this model, such as the Delphi method, which would ultimately enhance its robustness. Nevertheless, the development of the model relied on a comprehensive literature review and synthesis. The model utilized a rich dataset originating from primary studies and implementing various research methodologies, e.g., survey research, qualitative interviews, and mixed-methods research. The various methodologies used in the primary studies, the various types of data originated and data qualities, as well as the unique experiences of various postgraduate programs, enriched this model and improved its quality.

Second is the issue of contextual applicability. Developing countries can find it difficult and costly to implement or adapt this model into their educational programs, mainly due to the cost and availability of necessary resources. For instance, certain components of the model could require expensive resources that are not readily affordable in low-income countries. Therefore, these educational programs should customize this model in a cost-effective approach, taking into consideration their available resources.

Finally, the consideration of stakeholders’ perspectives is crucial. Various stakeholders are required to provide their perspectives and input effectively to evaluate this model before its utilization. To elaborate, educators, program administrators, and employers representing the job market may find the model or some aspects of the model not applicable to their scope of interest or resources or may identify additional factors or priorities that are not explicitly addressed in the model.

Conclusions

Postgraduate pharmacy education represents a vital transition from undergraduate learning to unique, practice-oriented knowledge, preparing graduates for exceptional service across diverse pharmacy areas, topics, pursuits, and settings. Therefore, tailored pharmacy programs at higher education institutions must constantly evaluate various aspects of their educational systems with ongoing updates to remain relevant. This scoping review offered a wide breadth of evidence-based suggestions, recommendations, gaps, improvement opportunities, and conclusions pertaining to key areas of a) practice-oriented courses, curricula, and modules; b) performance-based assessments; c) real-world competencies, applied skills, and training; d) diverse tools and methods for teaching and learning; e) programs emphasizing the crucial role of mentorship and support in diverse pharmacy postgraduate topics. This review resulted in developing a conceptual framework, which can serve as a reverence for improving and developing Pharmacy postgraduate educational programs. Various opportunities for further research were also recognized to address various challenges and identified gaps in pharmacy postgraduate education.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Brazeau GA, Meyer SM, Belsey M, Bednarczyk EM, Bilic S, Bullock J, et al. Preparing pharmacy graduates for traditional and emerging career opportunities. Am J Pharm Educ. 2009 Dec 17;73(8):157. Available from: https://pubmed.ncbi.nlm.nih.gov/20221350 .

Scahill SL, Atif M, Babar ZU. Defining pharmacy and its practice: a conceptual model for an international audience. Integr Pharm Res Pract. 2017 May 12;6:121–9. Available from: https://pubmed.ncbi.nlm.nih.gov/29354558 .

Poloyac SM, Block KF, Cavanaugh JE, Dwoskin LP, Melchert RB, Nemire RE, et al. Competency, Programming, and Emerging Innovation in Graduate Education within Schools of Pharmacy: The Report of the 2016–2017 Research and Graduate Affairs Committee. Am J Pharm Educ. 2017 Oct;81(8):S11–S11. Available from: https://pubmed.ncbi.nlm.nih.gov/29200459 .

Doran MR, Lott WB. A duty of care. Trends Biochem Sci. 2013 Jan 1;38(1):1–2. Available from: https://pubmed.ncbi.nlm.nih.gov/23157921/ .

Wu-Pong S, Gobburu J, O’Barr S, Shah K, Huber J, Weiner D, et al. The future of the pharmaceutical sciences and graduate education: recommendations from the AACP Graduate Education Special Interest Group. Am J Pharm Educ. 2013 May 13;77(4):S2–S2. Available from: https://pubmed.ncbi.nlm.nih.gov/23716757 .

Hadi MA, Awaisu A. Postgraduate programs in clinical pharmacy and pharmacy practice: are we heading in the right direction? Am J Pharm Educ. 2010 May 12;74(4):72b-72b. Available from: https://pubmed.ncbi.nlm.nih.gov/20585434/ .

Pillai G, Chibale K, Constable EC, Keller AN, Gutierrez MM, Mirza F, et al. The Next Generation Scientist program: capacity-building for future scientific leaders in low- and middle-income countries. BMC Med Educ. 2018;18(1):233. Available from: https://doi.org/10.1186/s12909-018-1331-y .

Atif M, Razzaq W, Mushtaq I, Malik I, Razzaq M, Scahill S, et al. Pharmacy Services beyond the Basics: A Qualitative Study to Explore Perspectives of Pharmacists towards Basic and Enhanced Pharmacy Services in Pakistan. Int J Environ Res Public Health. 2020 Mar 31;17(7):2379. Available from: https://pubmed.ncbi.nlm.nih.gov/32244475 .

Al-Worafi YM. The challenges of pharmacy education in Yemen. Am J Pharm Educ. 2014 Oct 15;78(8):146. Available from: https://pubmed.ncbi.nlm.nih.gov/25386011 .

Bilal AI, Tilahun Z, Gebretekle GB, Ayalneh B, Hailemeskel B, Engidawork E. Current status, challenges and the way forward for clinical pharmacy service in Ethiopian public hospitals. BMC Health Serv Res. 2017 May 19;17(1):359. Available from: https://pubmed.ncbi.nlm.nih.gov/28526021 .

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005F 1;8(1):19–32.

Article   Google Scholar  

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018O 2;169(7):467–73.

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Rev Esp Nutr Humana y Diet. 2016;20(2):148–60.

Google Scholar  

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev . 2016;5(1):210. Available from: https://doi.org/10.1186/s13643-016-0384-4 .

Medina MS, Tomsek JJ, Bowers-Pippin J. The use of mentors and partnerships in a preparing future faculty program at a Health Sciences Center. Curr Pharm Teach Learn. 2015;7(2):145–50. Available from: https://doi.org/10.1016/j.cptl.2014.11.008 .

Freitas MJ, Silva JV, Korrodi-Gregório L, Fardilha M. Non-stop lab week: A real laboratory experience for life sciences postgraduate courses. Biochem Mol Biol Educ. 2016 May 6;44(3):297–303. Available from: https://doi.org/10.1002/bmb.20947 .

Nikravanfard N, Khorasanizadeh F, Zendehdel K. Research Ethics Education in Post-Graduate Medical Curricula in I.R. Iran. Dev World Bioeth. 2017 Aug 1;17(2):77–83. Available from: https://doi.org/10.1111/dewb.12122 .

Zainal INA, Karim NAA, Soh YC, Suleiman AK, Khan TM, Hameed MA, et al. Key Elements of Pharmacoinformatics for the Degrees of Bachelor and Master of Pharmacy. Ther Innov Regul Sci. 2017 Apr 7;51(4):419–25. Available from: https://doi.org/10.1177/2168479017701977

Ahmed WS, Nebeker C. Assessment of research ethics education offerings of pharmacy master programs in an Arab nation relative to top programs worldwide: A qualitative content analysis. PLoS One. 2021 Feb 19;16(2):e0238755–e0238755. Available from: https://pubmed.ncbi.nlm.nih.gov/33606694 .

Fuhrmann CN, Halme DG, O’Sullivan PS, Lindstaedt B. Improving graduate education to support a branching career pipeline: recommendations based on a survey of doctoral students in the basic biomedical sciences. CBE Life Sci Educ. 2011;10(3):239–49. Available from: https://pubmed.ncbi.nlm.nih.gov/21885820 .

Eddington (Chair) ND, Aubé J, Das SK, Ellingrod VL, Hansen R, Madhavan SS, et al. Report of the 2014–2016 AACP Research and Graduate Affairs Committee. Am J Pharm Educ. 2016 Nov 25;80(9):S21. Available from: http://www.ajpe.org/content/80/9/S21.abstract .

Barrett JE, McGonigle P, Clark JE. Graduate Education in Pharmacology: Addressing the need for specialized training for pharmaceutical and biotechnology careers. Pharmacol Res. 2016;113:327–31. Available from: https://www.sciencedirect.com/science/article/pii/S1043661816308453 .

Lypson ML, Prince MEP, Kasten SJ, Osborne NH, Cohan RH, Kowalenko T, et al. Optimizing the post-graduate institutional program evaluation process. BMC Med Educ. 2016 Feb 17;16:65. Available from: https://pubmed.ncbi.nlm.nih.gov/26887758 .

Allen GM, Chisholm O. Postgraduate Education in Pharmaceutical Medicine in Australia: Evaluation and Evolution to a Global Program Over 20 Years. Ther Innov Regul Sci. 2018 Oct 5;53(5):654–60. Available from: https://doi.org/10.1177/2168479018793129 .

Keller F, Dhaini S, Briel M, Henrichs S, Höchsmann C, Kalbermatten D, et al. How to Conceptualize and Implement a PhD Program in Health Sciences-The Basel Approach. J Med Educ Curric Dev. 2018 Apr 24;5:2382120518771364–2382120518771364. Available from: https://pubmed.ncbi.nlm.nih.gov/29780889 .

Chisholm O. Curriculum Transformation: From Didactic to Competency-Based Programs in Pharmaceutical Medicine. Front Pharmacol. 2019 Mar 21;10:278. Available from: https://pubmed.ncbi.nlm.nih.gov/30949056 .

Koster AS, Mantel-Teeuwisse AK, Woerdenbag HJ, Mulder WMC, Wilffert B, Schalekamp T, et al. Alignment of CanMEDS-based Undergraduate and Postgraduate Pharmacy Curricula in The Netherlands. Pharm (Basel, Switzerland). 2020 Jul 10;8(3):117. Available from: https://pubmed.ncbi.nlm.nih.gov/32664306 .

Gu J, Liu Y, Xu F, Zhang Y, Shao R, Lu T, et al. Development and challenges of professional Master of pharmacy education in China. J Chinese Pharm Sci. 2021;30(1):69–78.

Poloyac SM, Empey KM, Rohan LC, Skledar SJ, Empey PE, Nolin TD, et al. Core competencies for research training in the clinical pharmaceutical sciences. Am J Pharm Educ. 2011 Mar 10;75(2):27. Available from: https://pubmed.ncbi.nlm.nih.gov/21519417 .

Newton GD, Hagemeier NE. A curriculum development simulation in a graduate program. Am J Pharm Educ. 2011 Nov 10;75(9):184. Available from: https://pubmed.ncbi.nlm.nih.gov/22171112 .

Garces H, Black EP. Corporate communication strategies are applicable for teaching non-science communication skills to pharmaceutical sciences PhD students. Curr Pharm Teach Learn. 2015;7(2):265–72. Available from: https://www.sciencedirect.com/science/article/pii/S1877129714001579 .

Ponzio NM, Alder J, Nucci M, Dannenfelser D, Hilton H, Linardopoulos N, et al. Learning Science Communication Skills Using Improvisation, Video Recordings, and Practice, Practice, Practice. J Microbiol Biol Educ. 2018 Mar 30;19(1):19.1.15. Available from: https://pubmed.ncbi.nlm.nih.gov/29904514 .

Colmenares EW, McLaughlin JE, Morbitzer KA, Eckel SF. Development and perceived value of a master’s degree in health-system pharmacy administration training. Am J Heal Pharm. 2021 Jan 1;78(1):74–9. Available from: https://doi.org/10.1093/ajhp/zxaa338 .

McLaughlin JE, Minshew LM, Gonzalez D, Lamb K, Klus NJ, Aubé J, et al. Can they imagine the future? A qualitative study exploring the skills employers seek in pharmaceutical sciences doctoral graduates. PLoS One. 2019 Sep 9;14(9):e0222422–e0222422. Available from: https://pubmed.ncbi.nlm.nih.gov/31498853 .

Lebovitz L, Swaan PW, Eddington ND. Trends in Research and Graduate Affairs in Schools and Colleges of Pharmacy, Part 2: Students. Am J Pharm Educ. 2020 May 1;84(5):7642. Available from: http://www.ajpe.org/content/84/5/7642.abstract .

Panczyk M, Cieålak I, Zarzeka A, Jaworski M, Gotlib J. Effective training of phd students with evidence-based pharmacy ñ the use of online multi-module course. Acta Pol Pharm - Drug Res. 2019;76(1):185–94.

Gajbhiye S, Tripathi R, Parmar U, Khatri N, Potey A. Critical appraisal of published research papers - A reinforcing tool for research methodology: Questionnaire-based study. Perspect Clin Res. 2019/05/14. 2021;12(2):100–5. Available from: https://pubmed.ncbi.nlm.nih.gov/34012907 .

Robinson GFWB, Moore CG, Mctigue KM, Rubio DM, Kapoor WN. Assessing Competencies in a Master of Science in Clinical Research Program: The Comprehensive Competency Review. Clin Transl Sci. 2015;8(6):770–5.

Leak RK, O’Donnell LA, Surratt CK. Teaching Pharmacology Graduate Students how to Write an NIH Grant Application. Am J Pharm Educ. 2015 Nov 25;79(9):138. Available from: https://pubmed.ncbi.nlm.nih.gov/28435165 .

Bloom TJ, Hall JM, Liu Q, Stagner WC, Adams ML. Developing an Assessment Process for a Master’s of Science Degree in a Pharmaceutical Sciences Program. Am J Pharm Educ. 2016 Sep 25;80(7):125. Available from: https://pubmed.ncbi.nlm.nih.gov/27756933 .

Abu Farha R, Mukattash T, Al-Delaimy W. Predictors of Plagiarism Research Misconduct: A Study of Postgraduate Pharmacy Students in Jordan. J Acad Ethics. 2020;(0123456789). Available from: https://doi.org/10.1007/s10805-020-09386-x .

Swedan S, Khabour OF, Alzoubi KH, Aljabali AAA. Graduate students reported practices regarding the issue of informed consent and maintaining of data confidentiality in a developing country. Heliyon. 2020 Sep 19;6(9):e04940–e04940. Available from: https://pubmed.ncbi.nlm.nih.gov/32995624 .

Alcorn SR, Cheesman MJ. Technology-assisted viva voce exams: A novel approach aimed at addressing student anxiety and assessor burden in oral assessment. Curr Pharm Teach Learn. 2022M;14(5):664–70.

Richardson A, Bracegirdle L, McLachlan SIH, Chapman SR. Use of a three-dimensional virtual environment to teach drug-receptor interactions. Am J Pharm Educ. 2013 Feb 12;77(1):11. Available from: https://pubmed.ncbi.nlm.nih.gov/23459131 .

Swanson HI, Piascik MT. A Pilot Study on the Use of Lecture Tools to Enhance the Teaching of Pharmacokinetics and Pharmacodynamics. J Med Educ Curric Dev. 2014 Jan 1;1:JMECD.S19011. Available from: https://doi.org/10.4137/JMECD.S19011 .

Jalgaonkar S V, Joshi SS, Gajbhiye S V, Singh KNM, Sayyed MP. Perception of postgraduate students in pharmacology toward animal simulation model. Indian J Pharmacol. 2019;51(6):400–6. Available from: https://pubmed.ncbi.nlm.nih.gov/32029962 .

Volodymyrovych TY, Tetiana K, Yaroslavovych TB. Experience in Using Peer-to-Peer Platforms and Immersive Technologies in the Training of Future Masters of Pharmacy in Medical Lyceum. Int J Comput Sci Netw Secur. 2022;22(2):23–8.

Soucy K, Fairhurst RM, Lynn GM, Fomalont K, Wynn TA, Siegel RM. Breaking the Mold: Partnering with the National Institutes of Health Intramural Research Program to Accelerate PhD Training. Trends Immunol. 2016;37(12):813–5. Available from: https://doi.org/10.1016/j.it.2016.10.005 .

Raffing R, Jensen TB, Tønnesen H. Self-reported needs for improving the supervision competence of PhD supervisors from the medical sciences in Denmark. BMC Med Educ. 2017 Oct 23;17(1):188. Available from: https://pubmed.ncbi.nlm.nih.gov/29058586 .

Yue J-J, Chen G. An exploration of the structure of pharmaceutical mentors’ competence at universities in Western China. Stud Educ Eval. 2019;63:1–8. Available from: https://www.sciencedirect.com/science/article/pii/S0191491X1930001X .

Masic I, Hodzic A, Mulic S. Ethics in medical research and publication. Int J Prev Med. 2014 Sep;5(9):1073–82. Available from: https://pubmed.ncbi.nlm.nih.gov/25317288 .

Stankovic B, Stankovic M. Educating about biomedical research ethics. Med Heal Care Philos. 2014;17(4):541–8. Available from: https://doi.org/10.1007/s11019-014-9561-1 .

Brownell SE, Tanner KD. Barriers to faculty pedagogical change: lack of training, time, incentives, and..tensions with professional identity? CBE Life Sci Educ. 2012;11(4):339–46. Available from: https://pubmed.ncbi.nlm.nih.gov/23222828 .

Smith CR, Delgado C. Developing a Model of Graduate Teaching Assistant Teacher Efficacy: How Do High and Low Teacher Efficacy Teaching Assistants Compare? CBE Life Sci Educ. 2021 Mar;20(1):ar2–ar2. Available from: https://pubmed.ncbi.nlm.nih.gov/33444107 .

Koblinsky SA, Hrapczynski KM, Clark JE. Preparing future faculty and professionals for public health careers. Am J Public Health. 2015 Mar;105 Suppl(Suppl 1):S125–31. Available from: https://pubmed.ncbi.nlm.nih.gov/25706007 .

Fox BI, Flynn AJ, Fortier CR, Clauson KA. Knowledge, skills, and resources for pharmacy informatics education. Am J Pharm Educ. 2011 Jun 10;75(5):93. Available from: https://pubmed.ncbi.nlm.nih.gov/21829267 .

Khan RA, Spruijt A, Mahboob U, van Merrienboer JJG. Determining “curriculum viability” through standards and inhibitors of curriculum quality: a scoping review. BMC Med Educ. 2019 Sep 5;19(1):336. Available from: https://pubmed.ncbi.nlm.nih.gov/31488128 .

Ten Cate O. Competency-Based Postgraduate Medical Education: Past, Present and Future. GMS J Med Educ . 2017 Nov 15;34(5):Doc69–Doc69. Available from: https://pubmed.ncbi.nlm.nih.gov/29226237 .

Kerdijk W, Snoek JW, van Hell EA, Cohen-Schotanus J. The effect of implementing undergraduate competency-based medical education on students’ knowledge acquisition, clinical performance and perceived preparedness for practice: a comparative study. BMC Med Educ. 2013 May 27;13:76. Available from: https://pubmed.ncbi.nlm.nih.gov/23711403 .

van der Vleuten CPM. Competency-based education is beneficial for professional development. Perspect Med Educ. 2015 Dec;4(6):323–5. Available from: https://pubmed.ncbi.nlm.nih.gov/26553242 .

Council NR. Research Training in the Biomedical, Behavioral, and Clinical Research Sciences. Washington, DC: The National Academies Press; 2011. Available from: https://www.nap.edu/catalog/12983/research-training-in-the-biomedical-behavioral-and-clinical-research-sciences .

Karamizadeh Z, Zarifsanayei N, Faghihi AA, Mohammadi H, Habibi M. The study of effectiveness of blended learning approach for medical training courses. Iran Red Crescent Med J. 2012/01/01. 2012 Jan;14(1):41–4. Available from: https://pubmed.ncbi.nlm.nih.gov/22737553 .

Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res. 2016 Jan 4;18(1):e2–e2. Available from: https://pubmed.ncbi.nlm.nih.gov/26729058 .

Touloumakos AK. Expanded Yet Restricted: A Mini Review of the Soft Skills Literature. Front Psychol. 2020 Sep 4;11:2207. Available from: https://pubmed.ncbi.nlm.nih.gov/33013574 .

Persky AM, Medina MS, Castleberry AN. Developing Critical Thinking Skills in Pharmacy Students. Am J Pharm Educ . 2019 Mar;83(2):7033. Available from: https://pubmed.ncbi.nlm.nih.gov/30962645 .

Chimea T La, Kanji Z, Schmitz S. Assessment of clinical competence in competency-based education. Can J Dent Hyg CJDH = J Can l’hygiene Dent JCHD. 2020 Jun 1;54(2):83–91. Available from: https://pubmed.ncbi.nlm.nih.gov/33240368 .

Gruppen LD, Mangrulkar RS, Kolars JC. The promise of competency-based education in the health professions for improving global health. Hum Resour Health. 2012 Nov 16;10:43. Available from: https://pubmed.ncbi.nlm.nih.gov/23157696 .

Adams NE. Bloom’s taxonomy of cognitive learning objectives. J Med Libr Assoc. 2015 Jul;103(3):152–3. Available from: https://pubmed.ncbi.nlm.nih.gov/26213509 .

Subhi Y, Andresen K, Bojsen SR, Nilsson PM, Konge L. Massive open online courses are relevant for postgraduate medical training. Dan Med J. 2014;61(10):1–5.

Munusamy S, Osman A, Riaz S, Ali S, Mraiche F. The use of Socrative and Yammer online tools to promote interactive learning in pharmacy education. Curr Pharm Teach Learn. 2019;11(1):76–80. Available from: https://www.sciencedirect.com/science/article/pii/S1877129717300709 .

Hussain FN, Al-Mannai R, Diab MI, Agouni A. Investigating the use of a lecture capture system within pharmacy education: Lessons from an undergraduate pharmacy program at Qatar University. Int J Educ Technol High Educ. 2020;17(1):1–14.

Download references

The College of Pharmacy, Qatar University provided the funding for the publication of this article. 

Author information

Authors and affiliations.

Department of Pharmacy, Aspetar Orthopedic and Sports Medicine Hospital, Doha, Qatar

Dana ElKhalifa

College of Pharmacy, QU Health, Qatar University, Doha, Qatar

Ola Hussein & Mohamed Izham Mohamed Ibrahim

Department of Pharmacy, Al-Maarif University College, Ramadi, Iraq

Abeer Hamid

Department of Pharmacy, Aman Hospital, Doha, Qatar

Nour Al-Ziftawi

Clinical Imaging Department, Hamad General Hospital, Doha, Qatar

Israa Al-Hashimi

You can also search for this author in PubMed   Google Scholar

Contributions

DK, OH, & MI worked on developing the study protocol. DK, OH, & NA screened and identified eligible articles. DK, OH, & AH grouped the studies, extracted the required data, and summarized the results. DK drafted the initial manuscript. AH, IA, & MI revised and edited the manuscript. MI supervised all the steps and provided guidance and feedback throughout the whole process of constructing this review. All the authors read and approved the final manuscript.

Corresponding authors

Correspondence to Dana ElKhalifa or Mohamed Izham Mohamed Ibrahim .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’ s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

ElKhalifa, D., Hussein, O., Hamid, A. et al. Curriculum, competency development, and assessment methods of MSc and PhD pharmacy programs: a scoping review. BMC Med Educ 24 , 989 (2024). https://doi.org/10.1186/s12909-024-05820-5

Download citation

Received : 24 April 2024

Accepted : 25 July 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s12909-024-05820-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Postgraduate

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]