اعتبارِ سازه‌ی معیارنمره‌دهیِ مهارت نوشتن در آزمون جامع فارسی دانشگاه فردوسی برای غیرفارسی‌زبانان (مقاله علمی پژوهشی)

Language assessment is an essential part of each Language teaching syllabus.The effectiveness of language educational systems highly depends on the validity of language assessment methods. This research tries to study the construct validity of the writing scoring rubric in the Ferdowsi University’s Persian Proficiency test. For this purpose the results from one of the official tests performed in the Ferdowsi international center for teaching Persian to non-Persian speakers was analyzed with Rasch and factor analysis models. The results showed that the three constructs which are topic development, quality of Language and organization, all have high factor loads. While organization has the highest loads with 0.98 this amount for the two other constructs are 0.97. In order to score each construct, the writing rubric in this test has a six grades’ scale. The Rasch models showed that all raters used this scale in an appropriate way because the order of the thresholds are well organized and match with order of the scores. Also the Write map indicates that this scale can differentiate well elementary, intermediate and advanced examinees and the grades cover all students with different levels of abilities. The rater reliability for this test is 0.98 which is highly acceptable.

Extended Abstract:
Language assessment is one of the most important part of any language educational system.Much of the effectiveness of Persian language centers depends on the use of precise assessment and evaluation techniques.In fact, Persian language institutions face the fundamental question of how to convert abstract concepts of linguistic knowledge and communication ability into numbers.Understanding learners' progress, identifying their weaknesses, and making accurate decisions about them requires accurate and scientific methods of assessment and evaluation. The present study attempts to evaluate the construct validity of the Writing Skill’s Assessment rubric in the Ferdowsi Persian Language Examination.The test, which is approved by the Ministry of Science, is held twice a year at Ferdowsi University of Mashhad, Iraq, and in some other countries.This research attempts to answer the following three questions:
1. To what extent do the constructs defined in the scoring rubric measure distinct components of the writing skill?
    2. To what extent can the six-point scale distinguish intermediate, weak, and strong test takers?
    3. To what extent do the scorers agree on the use of the scoring criteria?
So far, several scoring rubrics have been presented in Persian language in order to assess non-native Persian language speaker’s writing ability but none of them have been validated by quantitative research methods. The aim of current study was to investigate the construct validity of writing section of Ferdowsi University’s Persian language proficiency test. This test is designed based on the TOEFL theoretical underpinnings. The writing scoring rubric in Ferdowsi University’s Persian language proficiency test consists of three components namely language quality, cohesion, and topic development. Obviously, this scoring rubric is derived from a communicative view of the concept of language, which not only measures test taker’s linguistic knowledge at the level of words, sentence and discourse, but also assesses the ability to perform language tasks.Based on this rubric, in addition to the quality of the language and the cohesion of the text, the evaluators examine to what extent the written text is consistent with the purpose of the language task.
In order to evaluate the construct validity of the scoring rubric of the test, the results of one of the tests held in Ferdowsi International Persian Language Center were analyzed by Rash statistical model and factor analysis.The test was held on July 8, 1397 at the International Center for Persian Language at Ferdowsi University of Mashhad and Strasbourg University in France.The writing section of the test includes two tasks.In the first task, an audio file was first played for the test takers, and then they were asked to write a summary of it.In the second task, the test takers were given a topic to write about it in 200 words.
The participants in this study were 106 students consisting of 30 women and 76 men.Iraq, with 50 participants, and Pakistan with 30, respectively, had the first and second highest participants.The other participants were from India, Indonesia, Lebanon, Syria, and Italy, each with 13, 2, 2, 4 and 5 participants, respectively.In terms of educational background, the Humanities, with 68 members, constitutes the most participants.Engineering with 23 and medicine with 11 were in the next ranks in terms of number of participants.
Factor analysis results showed that the three identified constructs all have high validity.That is, writing skill can be divided into language quality, cohesion, and topic development and can be scored separately.Cohesion with 0.98 had the highest factor load and the other two constructs each with 0.97 had the second highest factor loadings.Based on these statistics, it can be said that writing proficiency can be divided into language quality, cohesion and topic development constructs and each of these components measures a separate construct.
The Ferdowsi Persian Language Proficiency test use a six-point scoring scale. Rash's statistical model showed that each of the scorers was able to use this criterion fairly correctly because the order of thresholds were in accordance with the order of the scores and did not change. On the other hand, the write map indicated that this scoring scale has the ability to distinguish between weak, intermediate, and strong test takers, and the 0 to 5 criteria cover all range of test takers so this scale can measure all candidates with any level of writing ability.On the other hand, the reliability of the scoring was 0.96, which is very well.This result indicates that test evaluators have used scoring criteria in the same way so the test has reasonable scorer reliability.