A Corpus-based Analysis of High School English Textbooks and English University Entrance Exams in Turkey


Abstract

This study aims to explore the missing link between English textbooks used in high schools (9th-12th grades) and English university entrance exams (2010-2019) in Turkey on lexical and syntactic complexity levels by using corpus linguistics tools: AntWordProfiler, TAALED, and the L2 Syntactic Complexity Analyzer (L2SCA). Official textbooks and complementary materials obtained from the Ministry of National Education have been compared against the official university entrance exams from the past decade. The results show that: (i) differences in lexical sophistication level can be observed between the two corpora, the lexical sophistication level of the exam corpus was higher than that of the textbook corpus, (ii) there is a statistically significant difference between the two corpora in terms of lexical diversity, the exam corpus has a significantly higher level of lexical diversity than the textbook corpus, (iii) statistically significant differences also existed between the two corpora regarding the syntactic complexity indices. The syntactic complexity level of the exam corpus was higher than that of the textbook corpus. The findings suggest that Turkish high school students who have to learn English with the official textbooks throughout their high school years will have to tackle low-frequency and more sophisticated words at a higher level of  syntactic complexity at the time of taking the nationwide exam. This, in return, creates a negative backwash effect, distorts their approach to L2, and raises other concerns about the misalignment between the official language education materials and nationwide exams.


Keywords

corpus linguistics; lexical diversity; syntactic complexity

Allen, H. W. (2008). Textbook materials and foreign language teaching: Perspectives from the classroom. The NECTFL Review, 62, 5–28.

Bardel, C., Gudmundson, A., & Lindqvist, C. (2012). Aspects of lexical sophistication in advanced learners’ oral production: Vocabulary acquisition and use in L2 French and Italian. Studies in Second Language Acquisition, 34(2), 269–290. https://doi.org/10.1017/S0272263112000058

Beauchamp, D., & Constantinou, F. (2020). Using corpus linguistic tools to identify instances

of low linguistic accessibility in tests. Research Matters: A Cambridge Assessment publication, 29, 10–16.

Biber, D. (1989). A typology of English texts. Linguistics, 27, 3–43.

Bulté, B., & Roothooft, H. (2020). Investigating the interrelationship between rated L2 proficiency and linguistic complexity in L2 speech, System, 91, 1–16.

Casanave, C. P. (1994). Language development in students’ journals. Journal of second language

writing, 3(3), 179–201.

Choi, I. (2008). The impact of EFL testing on EFL education in Korea. Language Testing, 25(1), 39–62.

Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.

Crossley, S. A., & Salsbury, T. (2010). Using lexical indices to predict produced and not produced words in second language learners. The Mental Lexicon, 5(1), 115–147.

Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011). What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly, 45(1), 182–193.

Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5–43.

Daller, H., Van Hout, R., & Treffers‐Daller, J. (2003). Lexical richness in the spontaneous speech of bilinguals. Applied linguistics, 24(2), 197–222.

Dean, A. C. (2017). Complex Dynamic Systems and Interlanguage Variability: Investigating Topic, Syntactic Complexity, and Accuracy in NS-NNS Written Interaction. Working

Papers in TESOL & Applied Linguistics, 17(1), 56–97.

Du, W. (2019). Analysis on the development of lexical complexity in Chinese science students’ English writing. Noble International Journal of Social Sciences Research, 4(7), 116–120.

Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity, and accuracy in second language narrative writing. Studies in Second Language Acquisition, 26(1), 59–84.

Ertmer, P. A., Bai, H., Dong, C., Khalil, M., Hee Park, S., & Wang, L. (2002). Online professional development: Building administrators’ capacity for technology leadership. Journal of Computing in Teacher Education, 19(1), 5–11.

Fletcher, P. (1985). A child’s learning of English. Blackwell.

Frase, L. T., Faletti, J., Ginther, A., & Grant, L. (1999). Computer analysis of the TOEFL test of written English. Educational Testing Service.

Gençoğlu, C. (2017, October). Republic of Turkey Ministry of National Education. COMCEC, Ankara, Turkey.

Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123–145.

Hatipoğlu, Ç. (2016). The impact of the university entrance exam on EFL education in Turkey:

Pre-service English language teachers’ perspective. Procedia-Social and Behavioral Sciences, 232, 136–144.

Hunt, K. W. (1965). Grammatical structures written at three grade levels. NCTE Research Report No. 3, 2–176.

Hyltenstam, K. (1988). Lexical characteristics of near‐native second‐language learners of

Swedish. Journal of Multilingual & Multicultural Development, 9(1–2), 67–84.

Kalyuga, S. (2006). Rapid assessment of learners’ proficiency: A cognitive load approach. Educational Psychology, 26(6), 735–749. https://doi.org/10.1080/01443410500342674

Kim, J. Y. (2014). Predicting L2 Writing Proficiency Using Linguistic Complexity Measures: A Corpus-Based Study. English Teaching, 69(4), 27–51.

Kirkgoz, Y. (2007). English language teaching in Turkey: Policy changes and their implementations.

RELC Journal, 38(2), 216–228.

Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication [Georgia State University]. http://scholarworks.gsu.edu/alesl_diss/35/

Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using finegrained clausal and phrasal indices. The Modern Language Journal, 102(2), 333–349.

Kyle, K. (2019). Measuring Lexical Richness. In S. Webb (Ed.), The Routledge handbook of vocabulary studies (pp. 454–475). Routledge.

Kwary, D., Artha, A., & Amalia, Y. (2018). Lexical word-class distributions in research articles of four subject areas. Studies about Languages, 33, 108–118.

Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27(4), 590–619.

Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production.

Applied linguistics, 16(3), 307–322.

Lu, X. (2011). A corpus‐based evaluation of syntactic complexity measures as indices of college‐level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.

Lu, X., & Ai, H. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29, 16–27.

Milli Eğitim Bakanlığı. (2018). Ortaöğretim İngilizce Dersi Öğretim Programı. Retrieved from: http://mufredat.meb.gov.tr/ProgramDetay.aspx?PID=342

Mirshojaee, S. B., & Sahragard, R. (2015). Reading comprehension passages of Iranian general English books and MA reading comprehension tests: A corpus analysis. Journal of Modern Research in English Language Studies, 2(2), 77–98.

McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated

approaches to lexical diversity assessment. Behavior research methods, 42(2),

–392.

McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57–86.

Miller, D. P. (1981). The depth/breadth trade-off in hierarchical computer menus. In Proceedings of the Human Factors Society 25th Annual Meeting (pp. 296–300). HFES.

Nelson, N. W., & Van Meter, A. M. (2007). Measuring written language ability in narrative samples. Reading & Writing Quarterly, 23(3), 287–309.

Norrby, C., & Håkansson, G. (2007). The interaction of complexity and grammatical processability:

The case of Swedish as a foreign language. International Review of Applied Linguistics in Language Teaching, 45(1), 45–68.

Nur, S., & Islam, M. (2018). The (Dis)Connection between Secondary English Education

Assessment Policy and Practice: Insights from Bangladesh. International Journal of English Language Education, 6(1), 100–132.

Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college‐level L2 writing. Applied linguistics, 24(4), 492–518.

Park, S.-Y. (2012). A corpus-based study of syntactic complexity measures as development indices of college-level L2 learners’ proficiency in writing. Korean Journal of Applied Linguistics, 28(3), 139–160.

Read, J. (2000). Assessing vocabulary. Cambridge University Press.

Sheldon, L. E. (1998). Evaluating ELT textbooks and materials. ELT Journal, 42(4), 237–246.

Skalicky, S., Duran, N., & Crossley, S. A. (2020). Please, please, just tell me: The linguistic

features of humorous deception. Retrieved from: osf.io/qdjmn

Tai, S., & Chen, H.-J. (2015). Are teachers test-oriented? A comparative corpus-based analysis of the English entrance exam and junior high school English textbooks. In F. Helm,

L. Bradley, M. Guarda, & S. Thouësny (Eds.), Critical CALL – Proceedings of the 2015

EUROCALL Conference, Padova, Italy (pp. 518–522). Research-publishing.net. http://dx.doi.

org/10.14705/rpnet.2015.000386

Thomas, D. (2005). Type-Token Ratios in one teacher’s classroom talk: An investigation of lexical complexity. University of Birmingham.

Torruella, J., & Capsada, R. (2013). Lexical statistics and typological structures: A measure of lexical richness. Procedia-Social and Behavioral Sciences, 95, 447–454.

Underwood, P. (2010). A comparative analysis of MEXT English reading textbooks and Japan’s National Center Test. RELC Journal, 41(2), 165–182.

Vermeer, A. (2004). Vocabulary size in Dutch L1 and L2 children. In P. Bogaards & B. Laufer (Eds.), Vocabulary in a second language: Selection, acquisition, and testing (pp. 173–189). John Benjamins Publishing Company.

Wang, M. D. (1970). The role of syntactic complexity as a determiner of comprehensibility. Journal of Verbal Learning and Verbal Behavior, 9(4), 398–404.

Wang, S., & Slater, T. (2016). Syntactic complexity of EFL Chinese students’ writing. English Language and Literature Studies, 6(1), 81–86.

Wolfe-Quintero, K., Inagaki, S., & Kim. H. Y. (1998). Second language development in writing: Measures of fluency. Accuracy and complexity. University of Hawaii Press.

Yu, X. (2018). Analyses and comparisons of three lexical features in native and nonnative academic English writing [University of Central Florida]. https://stars.library.ucf.edu/etd/6061

Download

Published : 2022-01-28


GedikT., & KolsalY. (2022). A Corpus-based Analysis of High School English Textbooks and English University Entrance Exams in Turkey. Theory and Practice of Second Language Acquisition, 8(1), 157-176. https://doi.org/10.31261/TAPSLA.9152

Tan Arda Gedik  tangedik@gmail.com
Middle East Technical University  Turkey
Yağmur Su Kolsal 




Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The Copyright Holders of the submitted texts are the Authors. The Reader is granted the rights to use the material available in the TAPSLA websites and pdf documents under the provisions of the Creative Commons 4.0 International License: Attribution - Share Alike  (CC BY-SA 4.0). The user is free to copy and redistribute the material in any medium or format, and to remix, transform, and build upon the material for any purpose, even commercially.

1. License

The University of Silesia Press provides immediate open access to journal’s content under the Creative Commons BY-SA 4.0 license (http://creativecommons.org/licenses/by-sa/4.0/). Authors who publish with this journal retain all copyrights and agree to the terms of the above-mentioned CC BY-SA 4.0 license.

2. Author’s Warranties

The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s.

If the article contains illustrative material (drawings, photos, graphs, maps), the author declares that the said works are of his authorship, they do not infringe the rights of the third party (including personal rights, i.a. the authorization to reproduce physical likeness) and the author holds exclusive proprietary copyrights. The author publishes the above works as part of the article under the licence "Creative Commons Attribution-ShareAlike 4.0 International".

ATTENTION! When the legal situation of the illustrative material has not been determined and the necessary consent has not been granted by the proprietary copyrights holders, the submitted material will not be accepted for editorial process. At the same time the author takes full responsibility for providing false data (this also regards covering the costs incurred by the University of Silesia Press and financial claims of the third party).

3. User Rights

Under the CC BY-SA 4.0 license, the users are free to share (copy, distribute and transmit the contribution) and adapt (remix, transform, and build upon the material) the article for any purpose, provided they attribute the contribution in the manner specified by the author or licensor.

4. Co-Authorship

If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.

I hereby declare that in the event of withdrawal of the text from the publishing process or submitting it to another publisher without agreement from the editorial office, I agree to cover all costs incurred by the University of Silesia in connection with my application.