Investigating Critical Thinking in ELT Textbooks: A Systematic Literature Review of Textbook Evaluation Studies
Abstract
The pivotal role of critical thinking and its integration in English language education are synthesised through systematic literature reviews and meta-analysis studies in the areas of pedagogical interventions, teaching methods, and assessment methods. However, there is little evidence of systematic literature review on the integration of critical thinking into English language teaching materials such as textbooks. In this study, 41 empirical textbook evaluation studies published between 2010 and 2021 were reviewed from critical thinking perspective. The purpose of this study is to provide an overview of English textbook evaluations on critical thinking integration, to identify critical thinking frameworks applied, and to look for possible research gaps among the studies. An extensive literature search was conducted by applying Xiao and Watson’s (2017) eight-step systematic literature review method. From a collection of 41 empirical studies, this review focuses on such four aspects as applied critical thinking frameworks, findings, recommendations, and the language levels and content areas of the textbooks studied. The results revealed that (1) Bloom’s Taxonomy and Bloom’s Revised Taxonomy were most frequently applied critical thinking frameworks, (2) integration of higher-order thinking skills in textbooks was less common than lower-order thinking skills, (3) textbook assessment occurred primarily at the middle and high school levels more than at other levels, and (4) methodological considerations regarding the reliability and validity of the coding process in textbook content analysis received little attention. This study contributes a synthesised literature background of English textbook evaluation with recommendations for
methodological improvements in future studies.
Keywords
systematic literature review; critical thinking; ELT textbook evaluation; higher order thinking skills; lower order thinking skills
References
References marked with an asterisk indicate studies included in the systematic literature review.
*Abdelrahman, M. (2014). An analysis of the tenth grade English language textbooks questions in Jordan based on the revised edition of Bloom’s taxonomy. Journal of Education and Practice, 5(18), 139–151.
*Adli, N., & Mahmoudi, A. (2017). Reading comprehension questions in EFL textbooks and learners’ levels. Theory and Practice in Language Studies, 7(7), 590–595. https://dx.doi.org/10.17507/tpls.07074
*Akrong, B. S., Chaka, C., & van der Walt, C. (2021). A modified version of Bloom’s taxonomy and critical language awareness: Evaluating EAL textbooks for junior high schools in Ghana through a critical literacy approach. International Journal of Applied Linguistics and English Literature, 10(4), 106–115.
*Alfadda, H., Fatima, M., Ghaffar, A., & Afzaal, M. (2020). Critical thinking perspective in ESL new English textbooks: A case study of Pakistan. International Journal of Applied Linguistics and English Literature, 9(2), 24–31.
*Alfaki, I. (2014). Sudan English language syllabus: Evaluating reading comprehension questions using Bloom’s taxonomy. International Journal of English Language Teaching, 2(3), 53–74.
American Psychological Association. (2020). Publication manual of the American psychological association (7th ed.). American Psychological Association.
Alnofaie, H. (2013). A framework for implementing critical thinking as a language pedagogy in EFL preparatory programmes. Thinking Skills and Creativity, 10, 154–158. http://dx.doi.org/10.1016/j.tsc.2013.09.002
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E, Pintrich, P., Raths, J., & Wittrock, M. C. (2001). Taxonomy of learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (2nd ed.). Addison and Wesley.
Armstrong, D., Gosling, A., Weinman, J., & Marteau, T. (1997). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31(3), 597–606.
*Askaripour, S. A. A. (2014). A textbook evaluation of new version of Top Notch English series. English for Specific Purposes World, 15(44), 1–30.
*Assaly, I. R., & Smadi, O. M. (2015). Using Bloom’s taxonomy to evaluate the cognitive levels of master class textbook’s questions. English Language Teaching, 8(5), 100–110.
*Azizi, M. (2012). Critical thinking in General English course books taught in Iranian Universities. Journal of Language, Culture, and Translation, 1(3), 15–36.
Behar-Horenstein, L. S., & Niu, L. (2011). Teaching critical thinking skills in higher education: A review of the literature. Journal of College Teaching & Learning, 8(2), 25–42.
*Birjandi, P., & Alizadeh, I. (2013). Manifestation of critical thinking skills in the English textbooks employed by language institutes in Iran. International Journal of Research Studies in Language Learning, 2(1), 27–38.
Bloom, B., Englehart, M., Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Longman.
Brown, H. D., & Lee, H. (2015). Teaching by principles. Pearson Education.
Cooper, H. M. (1988). Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society, 1(1), 104–126.
Cottrell, S. (2017). Critical thinking skills: Effective analysis, argument and reflection. Bloomsbury Publishing.
Cummins, J. (1999). BICS and CALP: Clarifying the distinction. Washington, DC: Opinion Papers. (ERIC Document Reproduction Service No. ED 438 551). Retrieved from https://eric.ed.gov/?id=ED438551
*Dabbagh, A., & Safaei, A. (2019). Comparative textbook evaluation: Representation of learning objectives in locally and internationally published ELT textbooks. Issues in Language Teaching, 8(1), 249–277.
Dummett, P., & Hughes, J. (2019). Critical thinking in ELT: A working model for the classroom. National Geographic Learning.
Dummett, P. (2016). How to write critical thinking activities. ELT Teacher 2 Writer.
*Es-Salhi, A., & Elfatihi, M. (2019). Evaluating critical thinking skills in Moroccan EFL textbooks: Gateway to English 2 as a case. Higher Education of Social Science, 17(1), 13–22. http://doi.org/10.3968/11284
Elder, L., & Paul, R. (2012). Critical thinking: Competency standards essential to the cultivation of intellectual skills, Part 4. Journal of Developmental Education, 35(3), 30–31.
Ellis, R. (1997). The empirical evaluation of language teaching materials. ELT Journal, 51(1), 36–42.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115.
Ennis, R. H. (2018). Critical thinking across the curriculum: A vision. Topoi, 37(1), 165–184. https://doi.org/10.1007/s11245-016-9401-4
Facione, P. A. (2011). Critical thinking: What it is and why it counts. Academic Press. Retrieved from https://courseware.e-education.psu.edu/downloads/geog882/Critical%20Thinking%20What%20it%20is%20and%20why%20it%20counts.pdf
Fahim, M., Barjesteh, H., & Vaseghi, R. (2012). Effects of critical thinking strategy training on male/female EFL learners’ reading comprehension. English Language Teaching, 5(1), 140–145.
*Fitriani, D. H., & Kirana, D. P. (2022). Higher order thinking skill (hots) in English language textbook in senior high school. International Journal of Research on English Teaching and Applied Linguistics, 2(2), 55–66.
*Freahat, N. M., & Smadi, O. M. (2014). Lower-order and higher-order reading questions in secondary and university level EFL textbooks in Jordan. Theory & Practice in Language Studies, 4(9), 1804–1813. https://doi.org/10.4304/tpls.4.9.1804-1813
Gardner, H. (1993). Frames of mind: The theory of multiple intelligences. Basic Books.
Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st century. Basic Books.
Golpour, F. (2014). Critical Thinking and EFL learners’ performance on different writing modes. Journal of Pan-Pacific Association of Applied Linguistics, 18(1), 103–119.
*Gordani, Y. (2010). An analysis of English textbooks used at Iranian guidance schools in terms of Bloom’s taxonomy. Journal of Asia TEFL, 7(2), 249–278.
Halpern, D. F. (1999). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 89, 69–74.
Halpern, D. F. (2002). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 1999(80), 69–74.
*Hei, J. X. (2017). Thinking skills in the 12th grade English course book in Timor Leste. Beyond Words, 5(1), 32–45.
Hughes, J. (2014). Critical thinking in the language classroom. Recanati (Italy): ELI. Retrieved from: http://www.elionline.com/eng/teachers-area/critical-thinking
Ilyas, H. P. (2015). Critical thinking: Its representation in Indonesian ELT textbooks and education [Unpublished doctoral thesis]. University of York.
*Igbaria, A. K. (2013). A content analysis of the WH-Questions in the EFL textbook of Horizons. International Education Studies, 6(7), 200–224. https://doi.org/10.5539/ies.v6n7p200
*Jebbour, M. (2019). Exploring the manifestation of critical thinking in the Moroccan textbooks of English: The case of Ticket 2 English. JEES (Journal of English Educators Society), 4(1), 39–45.
Jones, E., Hoffman, S., Moore, L., Ratcliff, G., Tibbetts, S. & Glick, B. (1995). Essential Skills in Writing. Speech and Listening. and Critical Thinking for College Graduates: Perspectives of Faculty, Employers, and Policy Makers. Project summary, U.S. Department of Education, OERI Contract No. R117GI0037. National Center for Post-secondary Teaching, Learning and Assessment. University Park, Pennsylvania State University Press.
*Khodadady, E., & Karami, M. (2017). An evaluation of textbooks designed for advanced English learners within a foreign context: A critical thinking perspective. Porta LINGUARUM, 28, 95–109.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218.
Krippendorff, K. (2019). Content analysis: An introduction to its methodology. Sage.
Lin, Y. (2018). Developing critical thinking in EFL classes. Springer.
Littlejohn, A. (2011). The analysis of language teaching materials: Inside the Trojan Horse. In B. Tomlinson (Ed.), Materials development in language teaching (2nd ed., pp. 179–211). Cambridge University Press.
Lune, H., & Berg, B. L. (2017). Qualitative research methods for the social sciences. Pearson.
*Margana, & Widyantoro, A. (2017). Developing English textbooks oriented to higher order thinking skills for students of vocational high schools in Yogyakarta. Journal of Language Teaching and Research, 8(1), 26–38. http://dx.doi.org/10.17507/jltr.0801.04
McDonough, J., Shaw, C. & Masuhara, H. (2013). Materials and Methods in ELT (3rd ed.). John Wiley & Sons.
McGrath, I. (2002). Materials evaluation and design for language teaching. Edinburgh University Press.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and implementation. Jossey-Bass.
*Mizbani, M., & Chalak, A. (2017). Analyzing reading and writing activities of Iranian EFL textbook Prospect 3 based on Bloom’s revised taxonomy. Journal of Applied Linguistics and Language Research, 4(2), 13–27.
Moher D, Liberati A., Tetzlaff J., Altman D., the PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264–270.
Moore, T. (2013). Critical thinking: Seven definitions in search of a concept. Studies in Higher Education, 38(4), 506–522.
Moore, B. & Parker, R. (2009). Critical thinking (9th ed.). McGraw-Hill Higher Education.
*Muchlis, M. (2015). An analysis of thinking order of reading comprehension questions in English textbook for young foresters of Forestry Vocational School of Samarinda. Jurnal Nalar Pendidikan, 3(1), 314–316.
*Nainggolan, B. R., & Wirza, Y. (2021). Evaluation of an English textbook: Does English in Mind promote critical thinking skills? Advances in Social Science, Education and Humanities Research, 546, 51–59.
Nappi, J. S. (2017). The importance of questioning in developing critical thinking skills. Delta Kappa Gamma Bulletin, 84(1), 30–41.
Nicholls, J. (2003). Methods in school textbook research. International Journal of Historical Learning, Teaching and Research, 3(2), 1–17.
Nilson, L. B. (2021). Infusing critical thinking into your course: A concrete, practical approach. Stylus Publishing.
*Olimat, M. (2015). Analyzing action pack textbooks’ questions according to revised Bloom taxonomy. Journal of Education and Practice, 6(28), 152–159.
Peterson, S. (2008). Master critical thinking for the SAT. Peterson’s.
Potter, W. J., & Levine‐Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258–284.
Puig, B., Blanco-Anaya, P., Bargiela, I. M., & Crujeiras-Pérez, B. (2019). A systematic review on critical thinking intervention studies in higher education across professional fields. Studies in Higher Education, 44(5), 860–869.
*Rahpeyma, A., & Khoshnood, A. (2015). The analysis of learning objectives in Iranian junior high school English textbooks based on Bloom’s revised taxonomy. International Journal of Education & Literacy Studies, 3(2), 44–55. https://doi.org/10.7575/aiac.ijels.v.3n.2p.44
*Raqqad, Y. M. A., & Ismail, H. H. (2018). Analyzing the reading questions of AP 12 textbook according to analyzing the reading questions of ap12 textbook according to Bloom’s taxonomy. International Journal of Education, Psychology and Counselling, 3(22), 84–94.
*Razmjoo, S. A., & Kazempourfard, E. (2012). On the representation of Bloom’s revised taxonomy in Interchange course books. The Journal of Teaching Language Skills (JTLS), 4(1), 171–204.
*Riazi, A. M., & Mosalanejad, N. (2010). Evaluation of learning objectives in Iranian highschool and pre-university English textbooks using Bloom’s taxonomy. TESL-EJ, 13(4)1–16.
*Roohani, A., Taheri, F., & Poorzangeneh, M. (2013). Evaluating Four Corners textbooks in terms of cognitive processes using Bloom’s revised taxonomy. Research in Applied Linguistics, 4(2), 51–67.
*Roohani, A. (2015). Analyzing cognitive processes and Multiple Intelligences in the nop-notch textbooks. Journal of Modern Research in English Language Studies, 2(3), 65–39.
*Sahragard, R., & Zahed Alavi, S. (2016). Investigating the predominant levels of learning objectives in general English books. Journal of English Language Teaching and Learning, 17(8), 93–114.
Sanavi, V. R., & Tarighat, S. (2014). Critical thinking and speaking proficiency: A mixed-method study. Theory and Practice in Language Studies, 4(1), 79–87.
Sheldon, L. E. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42(4), 237–246.
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75.
Shirkhani, S., & Fahim, M. (2011). Enhancing critical thinking in foreign language learners. Procedia-Social and Behavioral Sciences, 29, 111–115. https://doi.org/10.1016/j.sbspro.2011.11.214
*Shuyi, N. S., & Renandya, W. A. (2019). An analysis of the cognitive rigour of questions used in secondary school English language textbooks in Singapore. Asian Journal of English Language Studies (AJELS), 7, 169–188.
*Sirait, R. E. S. R. (2014). Comprehension levels of reading exercises in Look Ahead English coursebooks. International Journal of English Language Education, 2(2), 253–268. http://dx.doi.org/10.5296/ijele.v2i2.6307
Solihati, N., & Hikmat, A. (2018). Critical thinking tasks manifested in Indonesian language textbooks for senior secondary students. Sage Open, 8(3), 1–8. https://doi.org/10.1177/2158244018802164
*Sucipto, S., & Cahyo, S. D. (2019). A content analysis of the reading activities in Bright 2 an English textbook for junior high school students. English Language Teaching Educational Journal, 2(1), 13–21.
*Tabari, M. A., & Tabari, I. A. (2015). Links between Bloom’s taxonomy and Gardener’s Multiple Intelligences: The issue of textbook analysis. Advances in Language and Literary Studies, 6(1), 94–101. https://doi.org/10.7575/aiac.alls.v.6n.1p.94
*Talebinezhad, M. R., & Matou, Z. (2012). EFL reading comprehension textbooks at university level: A critical thinking perspective. Theory & Practice in Language Studies, 2(8), 1746–1754. https://doi.org/doi:10.4304/tpls.2.8.1746-1754
*Tangsakul, P., Kijpoonphol, W., Linh, N. D., & Kimura, L. N. (2017). Using Bloom’s revised taxonomy to analyze reading comprehension questions in Team Up in English 1–3 and grade 9 English o-net tests. International Journal of Research-Grantaalayah, 5(7), 31–41.
*Tayyeh, M. N. (2021). An analysis of reading comprehension questions in English textbook English For Iraq according to revised Bloom’s taxonomy. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(7), 2868–2874.
Thomson, C. (2011). Critical thinking across the curriculum: Process over output. International Journal of Humanities and social science, 1(9), 1–7.
Tomlinson, B. (2012). Materials development for language learning and teaching. Language Teaching, 45(2), 143–179.
Tomlinson, B., & H. Masuhara (2004). Developing language course materials. RELC Portfolio Series.
*Ulum, Ö. G. (2016). A descriptive content analysis of the extent of Bloom’s taxonomy in the reading comprehension questions of the course book Q: Skills for success 4 reading and writing. The Qualitative Report, 21(9), 1674–1683.
*Wu, Y., & Pei, Z. (2018). An investigation of critical thinking manifested in the questions of EFL textbooks for tertiary-level English majors of China. American Journal of Education and Learning, 3(2), 72–84. https://doi.org/10.20448/804.3.2.72.84
Webb, N. L. (2002). Depth of knowledge levels for four content areas [Unpublished manuscript]. Wisconsin Center for Education Research, University of Wisconsin Madison. Retrieved from http://ossucurr.pbworks.com/w/file/fetch/49691156/Norm%20web%20dok%20by%20subject%20area.pdf
Williams, D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37(3), 251–255.
Xiao, Y., & Watson, M. (2017). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93–112. https://doi.org/10.1177/0739456X17723971
Yang, Y. T. C., & Gamble, J. (2013). Effective and practical critical thinking-enhanced EFL instruction. ELT Journal, 67(4), 398–412.
*Zamani, G., & Rezvani, R. (2015). ‘HOTS’ in Iran’s Official Textbooks: Implications for Material Design and Student Learning. Journal of Applied Linguistics and Language Research, 2(5), 138–151.
*Zareian, G., Davoudi, M., Heshmatifar, Z., & Rahimi, J. (2015). An evaluation of questions in two ESP course books based on Bloom’s new taxonomy of cognitive learning domain. International Journal of Education and Research, 3(8), 313–326.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The Copyright Holders of the submitted texts are the Authors. The Reader is granted the rights to use the material available in the TAPSLA websites and pdf documents under the provisions of the Creative Commons 4.0 International License: Attribution - Share Alike (CC BY-SA 4.0). The user is free to copy and redistribute the material in any medium or format, and to remix, transform, and build upon the material for any purpose, even commercially.
1. License
The University of Silesia Press provides immediate open access to journal’s content under the Creative Commons BY-SA 4.0 license (http://creativecommons.org/licenses/by-sa/4.0/). Authors who publish with this journal retain all copyrights and agree to the terms of the above-mentioned CC BY-SA 4.0 license.
2. Author’s Warranties
The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s.
If the article contains illustrative material (drawings, photos, graphs, maps), the author declares that the said works are of his authorship, they do not infringe the rights of the third party (including personal rights, i.a. the authorization to reproduce physical likeness) and the author holds exclusive proprietary copyrights. The author publishes the above works as part of the article under the licence "Creative Commons Attribution-ShareAlike 4.0 International".
ATTENTION! When the legal situation of the illustrative material has not been determined and the necessary consent has not been granted by the proprietary copyrights holders, the submitted material will not be accepted for editorial process. At the same time the author takes full responsibility for providing false data (this also regards covering the costs incurred by the University of Silesia Press and financial claims of the third party).
3. User Rights
Under the CC BY-SA 4.0 license, the users are free to share (copy, distribute and transmit the contribution) and adapt (remix, transform, and build upon the material) the article for any purpose, provided they attribute the contribution in the manner specified by the author or licensor.
4. Co-Authorship
If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.
I hereby declare that in the event of withdrawal of the text from the publishing process or submitting it to another publisher without agreement from the editorial office, I agree to cover all costs incurred by the University of Silesia in connection with my application.
