dEjournals Home Page
Central European Journal of Educational Research
Current Archives Guides Templates Submit online Search
Register Login

Central European Journal of Educational Research

Vol. 7 No. 1 (2025): Institutional Contexts and Support Systems in Education: Family, School, and Community Perspectives
Research papers

Investigating the patterns of syntactic complexity predicting high-quality writing: a corpus-based study of the written text production at the B2+ English Language Exam at a Hungarian University

Published:
2025-06-30
DOI Linkhttps://doi.org/10.37441/cejer/2025/7/1/15027
Author
View
PDF
Keywords
Syntactic complexity academic writing language assessment
License

Copyright (c) 2025 by the authors

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How To Cite
Selected Style: APA
Adamova, K. (2025). Investigating the patterns of syntactic complexity predicting high-quality writing: a corpus-based study of the written text production at the B2+ English Language Exam at a Hungarian University. Central European Journal of Educational Research, 7(1). https://doi.org/10.37441/cejer/2025/7/1/15027
Abstract

This pilot study explores the predictive role of syntactic complexity in assessing L2 writing proficiency, with a focus on its potential contribution to validating a high-stakes English language examination. Drawing on prior research that highlights the importance of syntactic complexity in writing evaluation, the study aims to identify specific syntactic measures that reliably distinguish between low-rated and high-rated L2 texts. The analysis is based on a corpus of written texts for the B2+ level for so-called 'Basic' English Language Examination (BLE) administered at a Hungarian university. Although labeled "Basic" the BLE represents a mandatory proficiency examination (B2+ level according to the CEFR) required for academic advancement. Rather than examining inter-rater reliability, the research centers on contribution to validation by investigating linguistic features associated with rated writing quality. Grades assigned by human raters were used solely to group texts and build the corpus for analysis. A total of 60 syntactic complexity indices were extracted using the Multidimensional Analysis Tagger (MAT) (Nini, 2019) and the Coh-Metrix 3.0 software package (Graesser, McNamara & Kulikowich, 2011). These indices include measures of clausal, phrasal, and overall structural complexity. The findings are expected to inform ongoing validation efforts for the BLE and contribute to more robust, evidence-based practices in L2 writing assessment by identifying linguistic patterns that correlate with writing proficiency.

References
  1. Adamova, K. (2021, November 18–19). Patterns of syntactic complexity predicting high-quality writing: A corpus-based study of the written text production at the Basic English Language Exam at the Hungarian University [Conference abstract]. Szeged, Hungary.
  2. Ai, H., & Lu, X. (2013). A corpus-based comparison of syntactic complexity in NNS and NS university students’ writing. In A. Díaz-Negrillo, N. Ballier & P. Thompson (Eds.), Automatic treatment and analysis of learner corpus data (pp. 249–264). John Benjamins.
  3. Alapvizsga writing. (n.d.). Alapvizsga writing: Descriptors for the exam [Manuscript]. Pázmány Péter Catholic University.
  4. Biber, D. (1988). Variation across speech and writing. Cambridge University Press.
  5. Biber, D. (1992). The multi-dimensional approach to linguistic analyses of genre variation: An overview of methodology and findings. Computers and the Humanities, 26(5–6), 331–345.
  6. Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45(1), 5–35. https://doi.org/10.5054/tq.2011.244483
  7. Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37(5), 639–668. https://doi.org/10.1093/applin/amu059
  8. Biber, D., Gray, B., Staples, S., & Egbert, J. (2020). Investigating grammatical complexity in L2 English writing research: Linguistic description versus predictive measurement. Journal of English for Academic Purposes, 46. https://doi.org/10.1016/j.jeap.2020.100869
  9. Bulté, B., & Housen, A. (2012). Defining and operationalising L2 complexity. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 21–46). John Benjamins.
  10. Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65. https://doi.org/10.1016/j.jslw.2014.09.005
  11. Conrad, S., & Biber, D. (2001). Variation in English: Multi-dimensional studies. Pearson.
  12. Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
  13. Crossley, S. A., & McNamara, D. S. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66–79. https://doi.org/10.1016/j.jslw.2014.09.006
  14. Ferris, D. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(3), 414–420. https://doi.org/10.2307/3587446
  15. Graesser, A., McNamara, D., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234. https://doi.org/10.3102/0013189X11413260
  16. Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123–145. https://doi.org/10.1016/S1060-3743(00)00019-9
  17. Greenbaum, S., & Quirk, R. (1995). A student’s grammar of the English language. Longman.
  18. Haegeman, L. (1994). Introduction to government and binding theory (2nd ed.). Blackwell.
  19. Housen, A., Kuiken, F., & Vedder, I. (2012). Complexity, accuracy and fluency: Definitions, measurement and research. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 1–20). John Benjamins.
  20. Lu, X. (2017). Automated measurement of syntactic complexity in corpus-based L2 writing research and implications for writing assessment. Language Testing, 34(4), 493–511. https://doi.org/10.1177/0265532217710675
  21. Mazgutova, D., & Kormos, J. (2015). Syntactic and lexical development in an intensive English for academic purposes programme. Journal of Second Language Writing, 29, 3–15. https://doi.org/10.1016/j.jslw.2015.06.004
  22. McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press. https://doi.org/10.1017/CBO9780511894664
  23. Nini, A. (2014). Multidimensional Analysis Tagger (Version 1.1) [Computer software manual]. https://sites.google.com/site/multidimensionaltagger/
  24. Nini, A. (2019). The Multi-Dimensional Analysis Tagger. In T. Berber Sardinha & M. Veirano Pinto (Eds.), Multi-dimensional analysis: Research methods and current issues (pp. 67–94). Bloomsbury Academic.
  25. Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college‐level L2 writing. Applied Linguistics, 24(4), 492–518. https://doi.org/10.1093/applin/24.4.492
  26. Reményi, A. A. (2024). Evaluating written L2-English learner texts through manual and automated quantitative analysis. In A. Dósa, Á. Magnuczné Godó, R. L. Nagano, & A. Schäffer (Eds.), Crossing boundaries: Space, identity and discourse in Anglophone studies.
  27. Taguchi, N., Crawford, W., & Wetzel, D. Z. (2013). What linguistic features are indicative of writing quality? A case of argumentative essays in a college composition program. TESOL Quarterly, 47(2), 420–430. https://doi.org/10.1002/tesq.91