ساخت آزمون خواندن برای دانش‌آموزان پایه‌ی ششم ابتدایی با استفاده از انگاره‌ی سنجش شناختی تشخیصی

نوع مقاله: مقاله پژوهشی

نویسندگان

1 دانشجوی دکتری سنجش آموزش- دانشگاه تهران

2 دانشیار روان‌شناسی و علوم تربیتی- دانشگاه تهران

3 استاد روان‌شناسی و علوم تربیتی- دانشگاه تهران

4 استادیار روان‌شناسی و علوم تربیتی- دانشگاه تهران

5 استادیار پژوهشگاه مطالعات آموزش و پرورش وابسته به سازمان پژوهش و برنامه ریزی آموزشی

چکیده

پژوهش حاضر با هدف ساخت و رواسازی آزمون خواندن با استفاده از سنجش شناختی­تشخیصی انجام شد. به این منظور ابتدا با دسته‌بندی خصیصه‌های مرتبط با خواندن در تحقیقات و اسناد ملی و بین‌المللی، مهم‌ترین خصیصه‌های خواندن مشخص شد. پس از آن با استفاده از نظر متخصصان زبان و ادبیات فارسی و معلمان ابتدایی آزمون اولیه طراحی شد. پس از اجرای مقدماتی و تحلیل‌های اولیه و حذف پرسش­‌های نامناسب درنهایت آزمون نهایی با 27پرسش طراحی شد که چهار خصیصه‌ی درک اطلاعات صریح متن، استنباط، تلفیق، تفسیر و دانش واژگان را مورد سنجش قرار می‌دهد. در این تحقیق مراحل ساخت این آزمون تشریح شده است. با استفاده از روش نمونه‌گیری خوشه‌ای، آزمون روی 816 نفر از دانش‌آموزان پایه‌ی ششم ابتدایی شهر قم اجرا شد. تحلیل‌های آماری با استفاده از مدل شناختی­تشخیصی­­ افزایشی، انجام شد. شاخص‌های نسبی و مطلق برازش، نشان‌دهنده‌ی برازش مناسب مدل است. همچنین استفاده از شاخص‌های همسانی و دقت طبقه‌بندی نشان‌دهنده‌ی همسانی و دقت مناسب طبقه‌بندی دانش‌آموزان توسط آزمون است. نتایج این تحقیق نشان داد این آزمون می‌تواند نیم‌رخ خصیصه‌های خواندن دانش‌‌آموزان را با استفاده از مدل سنجش شناختی­تشخیصی شناسایی کند.

کلیدواژه‌ها


عنوان مقاله [English]

Developing a Reading Test for Sixth Grade Students Using Cognitive Diagnostic Assessment Model

نویسندگان [English]

  • Hasan Taghiyan 1
  • Ebrahim Khodayee 2
  • Abbas Bazergan 3
  • Ali Moghaddamzadeh 4
  • Masoud Kabiri 5
چکیده [English]

The present research aimed at the construction and validation of a reading test based on cognitive diagnostic assessment model (CDM). In order to construct the test, the most important reading attributes were identified by classifying the attributes relevant to reading in national and international research and documents. Subsequently, the test was designed by considering the viewpoints of experts in the field of Persian language and literature and primary school teachers. After pilot and preliminary analysis and eliminating inappropriate items, the final test was designed with 27 items, which measured four attributes of comprehension of explicit information in the text, inference, integration and interpretation, and vocabulary. In this research, the construction stages of the test were elaborated. Using a cluster sampling method, the test was performed on a sample consisting of 816 sixth grade students in Qom. The statistical analyses were carried out via additive CDM. The relative and absolute indices of fitness indicated that the model is sufficiently fit.  Moreover, use of classification consistency and accuracy indices indicated that the test is capable of classifying students with proper consistency and accuracy. The research results illustrated that this test can identify students’ reading attributes using cognitive diagnostic assessment model.

Extended Abstract:
Evaluating ability of reading specifically in primary school students is of utmost importance. ‌‌‌‌‌‌Though, national and international researches and assessments demonstrate weakness in Iranian students reading, no test for assessing of reading skills for primary school student is yet available.‌‌‌‌ Recent developments of educational assessment is basis to construct tests that diagnose students mastery in different attitudes. ‌‌‌Among them cognitive diagnostic assessment can demonstrate students weakness and strength in different attitudes. ‌Generally, the goal of cognitive diagnostic assessment is promoting assessment for learning versus learning outcome assessment by providing the information needed to modify the teacher's training in the classroom. ‌‌In another word the goal of cognitive diagnostic assessment is obtaining diagnosis and formative feedback by demonstrating students mastery of attitude exactly. The present research aimed at construction and validation of a reading test using cognitive diagnostic assessment‌. So this research has two minor goals: (A) identification of reading attributes and (b) designing a reading test using a cognitive diagnostic model. In order to achieve the first goal, reading theories were first reviewed‌‌‌. Reviewing reading theories suggests that existing theories analyze reading with great complexity and subtlety, making it almost impossible to make a test based on these theories‌‌‌‌‌‌. Therefore, in order to identify reading attributes, first reading attributes in national and international research and documentation, and valid language tests and researches that were performed with the cognitive diagnostic approach to assessing language, were collected and the most important reading attributes were collected‌. The attributes were categorized in several stages with the view of the specialists of Persian language and primary school teachers, and finally, four main attributes for the construction of the test were identified. Subsequently, The test was designed by viewpoints of experts in the field of Persian language and literature and primary school teachers. ‌Though, in the stages of the construction of the initial test, the appropriate coverage of the various combination of the Q-matrix was carefully noted, but in a Focus group, the suitability of the Q matrix was investigated‌‌‌. After pilot and preliminary analysis and eliminating inappropriate items, the final test was designed with 27 items, which assessed four attributes of comprehension of explicit information in the text, inference, integration and interpretation, and vocabulary. The test consists of three types of items‌‌. In a class of items, the student must answer questions about the story content after reading a short text. ‌In the second class, the student should understand the meaning of the words and ancient literature sentences. In third class items, students should be able to recognize the meaning of words in different sentences. In this test, in addition to the standards of educational measurement, a cognitive diagnostic model was used and specific Persian language requirements such as Iranian proverbs, ancient poetry and ancient literature, and some sources of reading errors that are rooted in Persian features (such as different letters with the same pronunciation, differences in the written and spoken word of some words, etc.) has also been considered. In this research, the construction stages of the test were elaborated. ‌‌‌Using a cluster sampling method, the test was performed on a random sample consisting of 816 sixth grade students in Qom.‌ The statistical analyses were carried out using additive cognitive diagnostic model.‌‌‌‌ After the initial analysis and the removal of inappropriate items, the relative and absolute indices of fitness indicate that the model is sufficiently fit. ‌‌‌‌‌Moreover, usage of classification consistency and accuracy indices indicate that the test is capable of classifying students with proper consistency and accuracy. Students' mastery of the four attitudes of reading in this test was between 0.53 and 0.68‌‌‌. Subjects' mastery in the attributes of comprehension of explicit information in the text was 0.68, in inference attributes and vocabulary knowledge, each was 0.58, and in the integration and interpretation attribute was 0.53‌‌‌‌‌‌. Students' mastery in the attributes has been compared with the two minimal criteria and the criterion that Jang (2009) has proposed. According to the minimal criterion, students have mastered all reading attributes. Based on the criterion that Jang (2009) suggests, the minimum level of proficiency required for each attribute is 0.6. Accordingly, students have mastered the comprehension of explicit information in the text, and have not mastered other reading attributes.‌‌‌‌‌ The results of this study showed that this test can identify the profile of reading attributes of students using a diagnostic cognitive model.

کلیدواژه‌ها [English]

  • cognitive diagnostic assessment
  • Reading
  • Sixth Grade
  • additive CDM
پاشاشریفی، حسن، دانش‌پژوه، زهرا. (1383). سنجش ملی پیشرفت تحصیلی زبان فارسی. تعلیم و تربیت. 20 (3)، صص:7-50.

دانای طوسی، مریم. (1390). آموزش خواندن در برنامه­ی درسی زبان ملی برای دوره ابتدایی: شواهدی از کشورهای آمریکا، انگلستان، سنگاپور و ایران. مطالعات برنامه درسی. 6 (21)، صص:11-42.

دانشگر، مریم. (1396). مهارت‌های دانش‌آموختگان نظام متوسطه با محوریت درس زبان و ادبیات فارسی (مطالعه موردی: شهر تهران). جستارهای زبانی، (8)1، صص:231-256.

دانشگر، مریم؛ عابدینی، سهیلا و محمدی ندا. (1394). تحلیل آماری نمره‌های زبان و ادبیات فارسی در کنکور سراسری. مطالعات ملی. 16 (2)، صص: 77-94.

دفتر برنامه‌ریزی و تألیف کتب درسی (1386). راهنمای برنامه درسی زبان و ادبیات فارسی. تهران: سازمان پژوهش و برنامه‌ریزی آموزشی.

رنجبران، فاطمه، علوی، سیدمحمد. (1396). سنجش تشخیصی­شناختی آزمون خواندن و درک مفاهیم برای بازخورد تشخیصی تکوینی. پژوهش‌های زبان‌شناختی در زبان‌های خارجی. 6 (2)، صص: 321-342.

شعاری‌نژاد، علی‌اکبر. (1395). ادبیات کودکان. تهران: انتشارات اطلاعات.

کریمی، عبدالعظیم؛ بخشعلی‌زاده، شهرناز و کبیری، مسعود. (1391). گزارش اجمالی از مهم‌ترین نتایج تیمز و پرلز 2011. تهران: مرکز ملی مطالعات تیمز و پرلز، سازمان پژوهش و برنامه‌ریزی آموزشی.

لیاقت، سمیه. (1395). مدل‌یابی شناختی تشخیصی قابلیت‌های خواندن در نمونه­ی آماری آزمون پرلز با تأکید بر دانش‌آموزان دوزبانه ایرانی، پایان‌نامه دکتری، دانشگاه تهران.

مقدم، اعظم، فلسفی نژاد، محمدرضا، فرخی، نورعلی و استاجی، معصومه (1394). تحلیل تشخیصی سؤالات بخش درک مطلب زبان انگلیسی عمومی آزمون ورودی دوره‌های دکتری با استفاده از مدل غیرجبرانی فیوژن. اندازه‌گیری تربیتی6 (22)، صص: 41-68.

نیساری، سلیم، اکبری شلدره، فریدون، زندی، بهمن، قاسم‌پورمقدم، حسین و نجفی پازکی، معصومه. (1394). راهنمای آموزش زبان فارسی در دوره ابتدایی. تهران: انتشارات مدرسه.

 

Ball, J. (2010). Enhancing learning of children from diverse language backgrounds: Mother tongue-based bilingual or multilingual education in early childhood and early primary school years. Victoria, Canada: Early Childhood Development Intercultural Partnerships, University of Victoria.

Bernhardt, E. (2005). Progress and procrastination in second language reading. Annual review of applied linguistics, 25, 133-150.

Brown, J. D., & Hudson, T. (2002). Criterion-referenced language testing. Cambridge: Cambridge University Press.

Buck, G., Tatsuoka, K., & Kostin, I. (1997). The Subskills of Reading: Rule‐space Analysis of a Multiple‐choice Test of Second Language Reading Comprehension. Language Learning, 47(3), 423-466.

California Standards Tests(2008).Grade 6 English–Language Arts. Retrieved from http://www.msmcclure.com/pdf/STARla6.pdf

Cui, Y., Gierl, M. J., & Chang, H. H. (2012). Estimating classification consistency and accuracy for cognitive diagnostic assessment. Journal of Educational Measurement, 49(1), 19-38.

Danay Tosi, M. (2011). Reading instruction at national language curriculum for elementary level: Evidence from America (New Jersi State), England, Singapore and Iran. Journal of Curriculum Studies. 6 (21), 11-42 [In Persian].

Daneshgar, M. (2017). Literary and Lingual Skills Evaluation (Case Study: Tehran Secondary School Students). Language Related Research. 8 (1), 231-256 [In Persian].

Daneshgar, M., Abedini, S., & Mohammadi, N. (2015). The Statistical analysis of persian language & literature in the university entrance exam. National Studies. 16 (2), 77-94 [In Persian].

Davis, F. B. (1968). Research in comprehension in reading. Reading Research Quarterly,3(4), 499-545.

Goldsmith-Phillips, J. (1989). Word and context in reading development: A test of the interactivecompensatory hypothesis. Journal of Educational Psychology, 81(3), 299-305.

Goldstein, H. (2004). Education for all: the globalization of learning targets. Comparative Education, 40(1), 7-14

Hemmati, S. J., Baghaei, P., & Bemani, M. (2016). Cognitive diagnostic modeling of L2 reading comprehension ability: Providing feedback on the reading performance of Iranian candidates for the university entrance examination. Language Testing, 6(2), 92-100.

Jang, E. E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. doctoral dissertation, University of Illinois at Urbana-Champaign.

Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26, 31–73.

Jang, E. E., Dunlop, M., Wagner, M., Kim, Y. H., & Gu, Z. (2013). Elementary school ELLs' reading skill profiles using cognitive diagnosis modeling: Roles of length of residence and home language environment. Language Learning, 63(3), 400-436.

Jang, E.E. (2008). A framework for cognitive diagnostic assessment. In C. A. Chapelle, Y.-R. Chung, & J. Xu (Eds.), Towards adaptive CALL: Natural Language Processing for Diagnostic Language Assessment (pp. 117-131). Amer, IA: Iowa Scaner University.

Karimi, A. A., Bkhshlyzadh, Sh., & Kabiri, M. (2011). A Brief Report of the Results of the Timss & Pirls, Timss and Pirls Centre National D'etudes. Research and Educational Planning Organization. Tehran. Bita [In Persian].

Kim, A. Y. (2015). Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability. Language Testing, 32, 227–258.

Lei, P. W., & Li, H. (2016). Performance of fit indices in choosing correct cognitive diagnostic models and Q-matrices. Applied Psychological Measurement, 40(6), 405-417.

Leighton, J. P., & Gierl, M. J. (2007). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practices, 26(2), 3–16.

Li, H. (2011). A cognitive diagnostic analysis of the MELAB reading test. Spaan Fellow, 9, 17-46.

Liyaghat, S. (2017). Cognitive Diagnostic Modeling of Reading Abilities in a Pearl Test Sample with Emphasis on Bilingual Iranian Students, doctoral dissertation, University of Tehran. [In Persian].

Moghadam, A., Falsafinejhad, M., Farokhi, N., & Estaji, M. (2016). Diagnostic analysis of general English reading comprehension's items of PhD entrance exam using non-compensatory fusion model. Quarterly of Educational Measurement, 6(22), 41-68 [In Persian].

Mullis, I. V., Martin, M. O., Kennedy, A. M., Trong, K. L., & Sainsbury, M. (2009). PIRLS 2011 Assessment framework. International Association for the Evaluation of Educational Achievement. Herengracht 487, Amsterdam, 1017 BT, The Netherlands.

Nichols, P. D. (1994). A framework for developing cognitively diagnostic assessments. Review of Educational Research, 64(4), 575–603.

Nysari, S., Akbari Seldere, F., Zandi. B., Ghasempour, H., & Najafi Pazouki. M. (2015). A guide to teaching Persian language in elementary school. Tehran: Madrese Press. [In Persian].

OECD (2013), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy, OECD Publishing.

Office of Planning and writing textbooks (2007). Persian Language and Literature Secondary school curriculum guide. [In Persian].

Pasha Shareefee, H., & Daneshpajooh, Z. (2004). National assessment of achievement in Farsi language. Quarterly Journal of Education. 20 (3) [In Persian].

Pearson Longman. )2010(. The Official Guide to Pearson Test of English Academic. Hong Kong. Pearson Education Asia ESL.

Ranjbaran, F., & Alavi, M. (2017). Cognitive Diagnostic Assessment of a Reading Comprehension Test Battery for Formative Diagnostic Feedback. Foreign Language Research Journal. 6 (2), 321-342.

Ravand, H. (2016). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 34(8), 782-799.

Sawaki, Y., Kim, H. J., & Gentile, C. (2009). Q-matrix construction: Defining the link between constructs and test items in large-scale reading and listening comprehension assessments. Language Assessment Quarterly, 6(3), 190-209.

Scarborough, H. S. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory, and practice. In Neuman. S. and Dickinson. D. (Eds.), Handbook for research in early literacy. New York: Guilford Press.

Shoarinejad, A. (2014). Children's Literature. Tehran: Ettelaat Press. [In Persian].

 

Spearritt, D. (1972). Identification of sub-skills of reading comprehension by maximum likelihood factor analysis. ETS Research Report Series, RB-72-4.

Stanovich, K. E. (1980). Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly, 16(1), 32–71.

Taylor, L., & Weir, C. J. (Eds.). (2012). IELTS collected papers 2: Research in reading and listening assessment (Vol. 2). Cambridge University Press.

Tinajero, A. R., & Loizillon, A. (2012). The review of care, education and child development indicators in early childhood. Report, UNESCO, Paris.

von Davier, A. A., & Kong, N. (2005). A unified approach to linear equating for the nonequivalent groups design. Journal of Educational and Behavioral Statistics, 30(3), 313-342.

von Davier, M., & von Davier, A. A. (2007). A unified approach to IRT scale linking and scale transformations. Methodology, 3(3), 115-124.