Developing a Reading Test for Sixth Grade Students Using Cognitive Diagnostic Assessment Model

The present research aimed at the construction and validation of a reading test based on cognitive diagnostic assessment model (CDM). In order to construct the test, the most important reading attributes were identified by classifying the attributes relevant to reading in national and international research and documents. Subsequently, the test was designed by considering the viewpoints of experts in the field of Persian language and literature and primary school teachers. After pilot and preliminary analysis and eliminating inappropriate items, the final test was designed with 27 items, which measured four attributes of comprehension of explicit information in the text, inference, integration and interpretation, and vocabulary. In this research, the construction stages of the test were elaborated. Using a cluster sampling method, the test was performed on a sample consisting of 816 sixth grade students in Qom. The statistical analyses were carried out via additive CDM. The relative and absolute indices of fitness indicated that the model is sufficiently fit.  Moreover, use of classification consistency and accuracy indices indicated that the test is capable of classifying students with proper consistency and accuracy. The research results illustrated that this test can identify students’ reading attributes using cognitive diagnostic assessment model.

Extended Abstract:
Evaluating ability of reading specifically in primary school students is of utmost importance. ‌‌‌‌‌‌Though, national and international researches and assessments demonstrate weakness in Iranian students reading, no test for assessing of reading skills for primary school student is yet available.‌‌‌‌ Recent developments of educational assessment is basis to construct tests that diagnose students mastery in different attitudes. ‌‌‌Among them cognitive diagnostic assessment can demonstrate students weakness and strength in different attitudes. ‌Generally, the goal of cognitive diagnostic assessment is promoting assessment for learning versus learning outcome assessment by providing the information needed to modify the teacher's training in the classroom. ‌‌In another word the goal of cognitive diagnostic assessment is obtaining diagnosis and formative feedback by demonstrating students mastery of attitude exactly. The present research aimed at construction and validation of a reading test using cognitive diagnostic assessment‌. So this research has two minor goals: (A) identification of reading attributes and (b) designing a reading test using a cognitive diagnostic model. In order to achieve the first goal, reading theories were first reviewed‌‌‌. Reviewing reading theories suggests that existing theories analyze reading with great complexity and subtlety, making it almost impossible to make a test based on these theories‌‌‌‌‌‌. Therefore, in order to identify reading attributes, first reading attributes in national and international research and documentation, and valid language tests and researches that were performed with the cognitive diagnostic approach to assessing language, were collected and the most important reading attributes were collected‌. The attributes were categorized in several stages with the view of the specialists of Persian language and primary school teachers, and finally, four main attributes for the construction of the test were identified. Subsequently, The test was designed by viewpoints of experts in the field of Persian language and literature and primary school teachers. ‌Though, in the stages of the construction of the initial test, the appropriate coverage of the various combination of the Q-matrix was carefully noted, but in a Focus group, the suitability of the Q matrix was investigated‌‌‌. After pilot and preliminary analysis and eliminating inappropriate items, the final test was designed with 27 items, which assessed four attributes of comprehension of explicit information in the text, inference, integration and interpretation, and vocabulary. The test consists of three types of items‌‌. In a class of items, the student must answer questions about the story content after reading a short text. ‌In the second class, the student should understand the meaning of the words and ancient literature sentences. In third class items, students should be able to recognize the meaning of words in different sentences. In this test, in addition to the standards of educational measurement, a cognitive diagnostic model was used and specific Persian language requirements such as Iranian proverbs, ancient poetry and ancient literature, and some sources of reading errors that are rooted in Persian features (such as different letters with the same pronunciation, differences in the written and spoken word of some words, etc.) has also been considered. In this research, the construction stages of the test were elaborated. ‌‌‌Using a cluster sampling method, the test was performed on a random sample consisting of 816 sixth grade students in Qom.‌ The statistical analyses were carried out using additive cognitive diagnostic model.‌‌‌‌ After the initial analysis and the removal of inappropriate items, the relative and absolute indices of fitness indicate that the model is sufficiently fit. ‌‌‌‌‌Moreover, usage of classification consistency and accuracy indices indicate that the test is capable of classifying students with proper consistency and accuracy. Students' mastery of the four attitudes of reading in this test was between 0.53 and 0.68‌‌‌. Subjects' mastery in the attributes of comprehension of explicit information in the text was 0.68, in inference attributes and vocabulary knowledge, each was 0.58, and in the integration and interpretation attribute was 0.53‌‌‌‌‌‌. Students' mastery in the attributes has been compared with the two minimal criteria and the criterion that Jang (2009) has proposed. According to the minimal criterion, students have mastered all reading attributes. Based on the criterion that Jang (2009) suggests, the minimum level of proficiency required for each attribute is 0.6. Accordingly, students have mastered the comprehension of explicit information in the text, and have not mastered other reading attributes.‌‌‌‌‌ The results of this study showed that this test can identify the profile of reading attributes of students using a diagnostic cognitive model.