Content Analysis of the Method Section of Research Articles Published in the ‘Journal of Teaching Persian to Speakers of Other Languages’

Method section constitutes the key element of Research Articles (RAs) since it gives the readers sufficient information on the steps taken, instruments used, and the reliability and validity of the study, thus most editors and reviewers pay a great deal of attention to this section. This study investigates the method section of all the 141 RAs published between 2012 and 2020 in the Journal of Teaching Persian to Speakers of Other Languages (TPSOL). To be more exact, it is a content analysis of the research approaches and designs, data collection tools and data sources, number of participants, statistical tests and techniques used in data analysis, and the missing elements of the method section of the said RAs. Analyzing the data through inductive content analysis revealed that almost 57% of the papers followed quantitative approach, followed by 31% mixed-methods approach. Descriptive, comparative, and correlational studies were the most frequent research designs, while case study was the least frequent one. Researchers collected the needed data mostly through questionnaires, achievement tests, pre-test and post-tests, and documents. The number of participants varied according to the research designs and purposes, with few studies employing sample sizes smaller than the standard size. The data of RAs were analyzed through descriptive statistics, content analysis, and t-test. It was also revealed that data analysis section, participants, and instruments were the most problematic sub-sections in the method section. The study ends with several practical suggestions for future researchers, present reviewers, and editors of the journal
Extended Abstract:
In today’s world, publishing research articles (RAs) in English is essential for postgraduate students and university professors in almost all majors (Tikhonova, 2020). However, conducting quality research and publishing RAs in high-ranking and prestigious journals seems to be a very challenging task for most of postgraduate students and novice researchers. As a result, and they cannot graduate even several years after they have written up their theses and dissertations. Thus, novice researchers are under pressure to improve their composition skills through increasing their knowledge of the rhetorical structure, organizational patterns, and needed elements of an RA for their field of study in order to increase their chances of acceptance and publication (Hyland, 2004).
Statement of the problem: Although novice researchers need to increase their knowledge of different sections of an RA, having sufficient information about the method section of RAs is of utmost importance (Kellet, 2004). Method section is the general plan of the research, forms the main section in RAs (Kellet, 2004; Lim, 2006) because it gives the readers sufficient information on the actions taken for performing the research (Musa, Khamis, & Zanariah, 2015), provides the readers with information regarding the validity of the data collection instruments, and is one of the sections to which most reviewers pay a great deal of attention (Kellet, 2004; Lim, 2006). Nonetheless, despite the importance of the method section in writing and evaluation processes of RAs, previous literature examining the methods section of RAs is so little that no universal model has been put forward for its move structure (Behnam & Zamanian, 2013; Kanoksilapatham, 2005; Lim, 2006; Peacock, 2011). While there are several studies conducted on the method section of the RAs in linguistics and applied linguistics since 2000, most of them have investigated the method section from angles different from this research. They have either addressed textual organization and linguistic features (Morales, 2016), rhetorical structure (Fazilatfar & Naseri, 2014; Soodmand Afshar & Ranjbar, 2017; Zhang & Wannaruk, 2016), grammatical complexity and clause types (Seifoori & Fattahi, 2014), RAs in several fields (Peacock, 2011), or genre analysis (Farnia & Baratizade, 2020). There exist far fewer studies which are similar to this study in their aims and procedures (Khany & Tazik, 2017; Lazaraton, 2000; Sahragard & Meihami, 2016; Tazik, Khany, Aliakbari, 2020; Yağız, Aydın, & Akdemir, 2016), among which only Sahragard and Meihami (2016) studied the RAs in the Journal of Teaching Persian to Speakers of Other Languages (TPSOL), which is the focus of the present study.
Aims and data collection process: In order to fill this gap in literature, this study investigates the method section of all the 141 RAs published between 2012 and 2020 in the Journal of TPSOL. To be more exact, it is a detailed content analysis of the research approaches and designs, data collection tools and data sources, number of participants, statistical tests and software used in data analysis, and the main missing elements or problems of the method section of the mentioned RAs. In order to reach these aims, all the published articles between 2012 and 2020 were downloaded. In order to speed up the data collection process, four PhD students (paid research assistants) helped the researcher in the data collection process. They downloaded all the 141 RAs and coded them as 1 to 141. However, the numbering was not done according to the order they were published so that readers cannot guess the codes and find the articles. Then, the researcher read the title, abstract, and method sections of these articles in order to find the needed information for answering the research questions. Since the researcher could not find the needed information in these sections in some cases, he had to read the whole article carefully, looking for the needed information sometimes in the results and conclusion sections.
Data analysis: In order to analyse the data, as mentioned above, the researcher read the articles two times and highlighted the information in the PDF files, leaving comments in sticky notes. Later, the researcher recorded the information for all the research question in a table. In fact, the researcher did not follow any specified and preplanned framework for the content analysis of the data and recorded exactly what was mentioned in the articles, even if they were wrongly mentioned. The researcher followed inductive content analysis to depict a comprehensive and exact picture of the method sections.
Results and conclusion: Analyzing the data revealed that almost 57% of the papers followed quantitative approach, followed by mixed-methods approach (31%). Descriptive, comparative, and correlational studies were the most widely used research designs, while case study was employed the least. Researchers gathered the needed data mainly through questionnaires, achievement tests, pre-test and post-tests, and documents such as textbooks and transcriptions. The number of participants in these studies varied according to the research designs and purposes, with some few studies gathering data from sample sizes smaller than the standard size. The data of these 141 RAs were analyzed through descriptive statistics, content analysis, and t-test. It was also revealed that data analysis section, participants, and instruments were the most problematic sub-sections in the method section. Not specifying the validity and reliability of the data collection instruments, gathering data from low number of participants, not specifying the criteria for scoring or analysis of the data, presenting insufficient data about the raters and inter-rater reliability, and offering no details about the treatment were the most serious problems with the method sections of these RAs. The study ends with several practical suggestions for future researchers, present reviewers, and editors of the journal.