ارزیابی محتوایی مجموعه فارسی بیاموزیم براساس چارچوب طبقه‌بندی تجدیدنظرشده بلوم

The present study evaluated Farsi Biyamouzim (Let's Learn Persian First Edition) series using Bloom's revised taxonomy. To this aim, all exercises and activities of this series were coded and to ensure the reliability of the coding procedure, inter- and intra-coder reliabilities were estimated. Then, the relative frequency of occurrence of different learning objectives was calculated. Findings revealed that overall, lower-order thinking skills (Remember, Understand, and Apply) were more frequent than higher-order thinking skills (Analyze, Evaluate, and Create) in the series. Nevertheless, A1 (Remember factual knowledge), B1 (Understand factual knowledge), B2 (Understand conceptual knowledge), C1 (Apply factual knowledge), C2 (Apply conceptual knowledge), C3 (Apply procedural knowledge), and A2 (Remember conceptual knowledge) were the most to the least frequent codes of low learning objectives which exhibited a decreasing trend of relative frequency from the first to the fifth volume. On the other hand, higher-order thinking skills, which were F0 (Create using facts, concepts, principles, and procedures), D0 (Analyze using facts, concepts, principles, and procedures), and E0 (Evaluate using facts, concepts, principles, and procedures) from the most to the least frequent, respectively, showed an increasing trend from the first to the fifth volume. In line with this finding, results of the Chi-Square test also indicated statistically significant differences among different volumes in terms of Bloom's revised taxonomy's learning objectives.
Extended Abstract:
Introduction
In the teaching and learning process, instructional books play a very important role in achieving educational goals (Ebadi & Mozafari, 2016). Ebadi and Mozafari believe that having a textbook for learners leads to their purposefulness and seriousness in passing course units. Therefore, the success or failure of any educational program is dependent on textbooks. In fact, textbooks are considered as sources for transferring knowledge to learners and have a significant role in curricula. Hence, evaluating textbooks to examine their impact on higher levels of learning is essential. That is why various frameworks have been introduced for analyzing the content of instructional books. One of the well-known and widely accepted frameworks is Bloom's taxonomy and its revised version (Nejatian Bostani & Miri, 1396). So far, however, only a few researchers have evaluated Persian language textbooks based on Bloom's revised taxonomy. Therefore, it seems that examining the content of these textbooks based on this taxonomy requires researchers' special attention.
Among Persian language textbooks, the book "Let's Learn Persian" (Zolfaghari et al., 1382) has also been specifically analyzed and evaluated based on various frameworks. However, previous research indicates that the only study conducted on this series based on Bloom's revised taxonomy is Ebadi and Mozafari's (2016) study on three lessons of this series, randomly selected from each volume. Therefore, as far as current researchers are aware, no research has been conducted to examine all the exercises and activities of this five-volume series based on Bloom's revised taxonomy. To fill this research gap, the present study examined all the questions and exercises of this series by seeking answers to the following research question:
 
In the activities and exercises of each volume of "Let's Learn Persian", how are different levels of Bloom's revised taxonomy distributed? In terms of the representation of different levels of this taxonomy, is there any statistically significant difference among different volumes?
 
Methodology
Due to lack of comprehensive studies on "Let's Learn Persian" series using Bloom's revised taxonomy, the present researchers selected this series for evaluation. Given that in this study, all lessons of each of the five volumes (i.e., the entire population) were considered, no specific sampling procedure was used. To collect data, all exercises and activities of the five volumes were extracted lesson by lesson and noted in separate tables. Then, they were coded based on the coding scheme developed by Razmjoo and Kazempourfard (2012). Two weeks after coding, the second researcher randomly coded a portion of the data to ensure intra-coder reliability. Additionally, another individual (a graduate of the English language teaching doctoral program who was a faculty member of another university) who was familiar with the taxonomy and had previously used it for coding in her doctoral dissertation, coded a portion of the data randomly selected. The index of the former was 99% while that of the latter was 98%, which were acceptable. It is worth mentioning that the researchers and the independent coder discussed and examined cases where they disagreed with each other in detail during several meetings to reach an agreement. Next, the relative frequency of each learning objective in each volume was calculated. Then, the chi-square test was used to compare different volumes in terms of the distribution of different levels of the taxonomy.
 
Results
In this series, overall, A1 which is the lowest level of Bloom's revised taxonomy was the most frequent code in the five volumes with a total relative frequency of 23.7. On the other hand, apart from codes with zero frequency, the least frequent code was A2, with a relative frequency of 2.2.
 
Table 1. Chi-square test results
 




857.065a


Chi-square




36


df




0.000


Asymp.sig




 
     Additionally, as shown in the above table, there was a statistically significant difference in the use of different levels of the taxonomy among different volumes of this series (Sig.=0.000).
 
Table 2. Relative frequency of use of higher- and lower-level learning objectives in the entire series
 




Higher learning objectives


Lower learning objectives


Learning objectives in "Let's Learn Persian"




7.8


92.2


Relative frequency


Volume 1




15.3


84.8


Relative frequency


Volume 2




12.9


87.1


Relative frequency


Volume 3




31.3


68.6


Relative frequency


Volume 4




42.7


57.4


Relative frequency


Volume 5




 
Overall, it seems that lower-level learning objectives have received more attention in this series. More specifically, compared to higher-level learning objectives, lower-level ones, namely, "Remembering," "Understanding," and "Applying," were more utilized. This finding aligns with the results of numerous previous studies (e.g., Gordani, 2010; Rashidi & Raghnezhad, 2014; Seo et al., 2010) indicating a discrepancy in the presentation order of lower-level learning objectives. On the other hand, however, this finding is consistent with the results of several other studies (e.g., Razmjoo & Kazempourfard, 2012; Sahragard & Zahed Alavi, 2016). However, comparing the relative frequency of higher-level learning objectives in different volumes showed that the use of these levels has an increasing trend, and parallel to the increase in the educational level of the volumes from elementary to advanced, the relative frequency of higher levels also significantly increased (from 25 to 216) (Table 2).
Based on these findings, in response to the research question, the results showed that A1 had the highest relative frequency in the entire series. Considering that exercises and lesson activities involving reading, repetition, and listening are coded A1, learners encountering such exercises are not engaged in any complex cognitive processes such as analysis, evaluation, and creation and may not even need to understand the meaning of the content. In fact, it can be said that learning such materials in the form of parrot-like memorization instead of focusing on the formation of knowledge through higher-level cognitive skills will not lead to the growth and advancement of learners' higher-order thinking skills.
It is worth noting that codes A3, A4, B3, B4, C4, D4, E4, and F4 were totally absent throughout this series. In general, cognitive awareness includes metacognitive knowledge, task-related cognitive knowledge, including appropriate background knowledge, conditional knowledge, and self-knowledge. This finding is consistent with the results of Razmjoo and Kazempourfard's study (2012) who believed that the absence and utilization of this type of knowledge in textbooks may be because this knowledge, as an internal cognitive activity, is like an internal question that every learner must ask themselves in response to a specific lesson question or exercise, and since it is considered an internal cognitive activity, it may not be clearly visible in textbooks, and perhaps for this reason, no instances of these codes were observed in this collection.
 
Conclusion
Data analysis indicated that although, in general, the relative frequency of lower-level learning objectives was higher compared to higher-level ones, higher-level thinking skills were not only well covered but also properly distributed so that these skills increased from Volume 1 to Volume 5. Therefore, we witness learners' progress from lower-level learning objectives to higher-level ones, which is consistent with the learners' language proficiency level. Overall, it can be concluded that the design of activities and exercises in this series aims to increase learners' engagement with cognitive processes, indicating that the authors of the series have made increasing efforts to develop learners' higher-order thinking skills and have tailored their content as much as possible to educational cognitive objectives.
As the series has been revised in recent years, future researchers can analyze and evaluate the revised textbooks using the same taxonomy. Additionally, another research can be conducted to uncover the reflection of this taxonomy in the tests administered based on this series in AZFA centers, the results of which can determine the direction in which the design of these tests is promoting learning objectives.
 
Conflict of Interest
The authors have no conflict of interest.
 
Acknowledgement
We would like to thank JTPSOL anonymous reviewers for providing us with their insifgtful and constructive comments which undoubtedly led to the betterment of this manuscript.