چکیده:
هدف اصلی این پژوهش مقایسه روش آنگوف مبتنی بر IRT و روش بوکمارک در تعیین استاندارد چیرگی آزمون زبان MSRT بود. بدین منظور، یکی از نمونه سوالات آزمون MSRT (آزمون مورخه شهریور 1397) به طور تصادفی انتخاب شد و پاسخ های داده شده به سوالات توسط 596 آزمودنی از وزارت علوم اخذ شد. آزمون MSRT دارای 100 سوال بوده که 30 سوال آن را گرامر، 30 سوال بخش شنیداری و 40 سوال آن را بخش درک مطلب خواندن تشکیل می دهد. دو پنل تخصصی متشکل از 15 متخصص تدریس آزمون تافل تشکیل شد و سپس با استفاده از روش های تعیین استاندارد آنگوف مبتنی بر نظریه سوال- پاسخ، و بوکمارک استانداردهای این آزمون در سه بخش جداگانه (گرامر، شنیداری، درک مطلب خواندن) در سه مرحله ارزیابی مشخص شدند. یافته های تحقیق گویای این بود نمره برش بدست آمده از روش آنگوف مبتنی بر نظریه سوال پاسخ برابر با 66/53 و روش بوکمارک برابر با 27/54 است. هر دو نمره برش بدست آمده از نمره تعیین شده توسط وزارت علوم بالاتر بود. همچنین، یافته های تحقیق نشان دادکه در روش معمول و سنتی وزارت علوم که بر پایه نمره 50 به عنوان نمره قبولی عمل می شود 3/73 درصد از شرکت کنندگان مردود و 7/26 درصد از شرکت کنندگان قبول شده اند اما، اساس روش آنگوف مبتنی بر نظریه سوال- پاسخ آمار مردودی برابر 5/78 درصد و آمار قبولی برابر با 4/23 درصد می شود.
The main purpose of this study was to compare the Angof-based IRT method and the Bookmark method determine standardization of the MSRT language test. For this purpose, one sample of the MSRT test questions (September 1397 test) was randomly selected and the answers to the questions were obtained by 596 subjects from the Ministry of Science. The MSRT test has 100 questions, of which 30 are grammar, 30 are auditory, and 40 are reading comprehension. Two expert panels consisting of 15 TOEFL teaching experts were formed and then standardized using ANGOF-based IRT method, and Bookmark's standards in three separate sections (grammar, listening, Reading comprehension) were identified in three evaluation stages. The findings of the study showed that the cut-off score obtained by Angof's method based on the question-answer theory is 53.66 and the bookmark method is 54.27. Both cut scores were higher than those determined by the Ministry of Science. Also, the findings of the study showed that 73.3% of the participants rejected the traditional and traditional method of the Ministry of Science, which is based on the score of 50, but 26.7% of the participants failed but based on the Angof method. Based on the question-answer theory, rejection rates are 78.5% and acceptance rates are 23.4%. Finally, according to Bookmark, the decline in statistics is 21.5%. The research findings imply that Ministry of Science language test designers need to be revised to determine the standard score of this test. The main purpose of this study was to compare the Angof-based IRT method and the Bookmark method to determine standardization of the MSRT language test. For this purpose, one sample of the MSRT test questions (September 1397 test) was randomly selected and the answers to the questions were obtained by 596 subjects from the Ministry of Science. The MSRT test has 100 questions, of which 30 are grammar, 30 are auditory, and 40 are reading comprehension. Two expert panels consisting of 15 TOEFL teaching experts were formed and then standardized using ANGOF-based IRT method, and Bookmark's standards in three separate sections (grammar, listening, Reading comprehension) were identified in three evaluation stages. The findings of the study showed that the cut-off score obtained by Angof's method based on the question-answer theory is 53.66 and the bookmark method is 54.27. Both cut scores were higher than those determined by the Ministry of Science. Also, the findings of the study showed that 73.3% of the participants rejected the traditional and traditional method of the Ministry of Science, which is based on the score of 50, but 26.7% of the participants failed but based on the Angof method. Based on the question-answer theory, rejection rates are 78.5% and acceptance rates are 23.4%. Finally, according to Bookmark, the decline in statistics is 21.5%. The research findings imply that Ministry of Science language test designers need to be revised to determine the standard score of this test. The main purpose of this study was to compare the Angof-based IRT method and the Bookmark method to determine standardization of the MSRT language test. For this purpose, one sample of the MSRT test questions (September 1397 test) was randomly selected and the answers to the questions were obtained by 596 subjects from the Ministry of Science. The MSRT test has 100 questions, of which 30 are grammar, 30 are auditory, and 40 are reading comprehension. Two expert panels consisting of 15 TOEFL teaching experts were formed and then standardized using ANGOF-based IRT method, and Bookmark's standards in three separate sections (grammar, listening, Reading comprehension) were identified in three evaluation stages. The findings of the study showed that the cut-off score obtained by Angof's method based on the question-answer theory is 53.66 and the bookmark method is 54.27. Both cut scores were higher than those determined by the Ministry of Science. Also, the findings of the study showed that 73.3% of the participants rejected the traditional and traditional method of the Ministry of Science, which is based on the score of 50, but 26.7% of the participants failed but based on the Angof method. Based on the question-answer theory, rejection rates are 78.5% and acceptance rates are 23.4%. Finally, according to Bookmark, the decline in statistics is 21.5%. The research findings imply that Ministry of Science language test designers need to be revised toThe main purpose of this study was to compare the Angof-based IRT method and the Bookmark method to determine standardization of the MSRT language test. For this purpose, one sample of the MSRT test questions (September 1397 test) was randomly selected and the answers to the questions were obtained by 596 subjects from the Ministry of Science. The MSRT test has 100 questions, of which 30 are grammar, 30 are auditory, and 40 are reading comprehension. Two expert panels consisting of 15 TOEFL teaching experts were formed and then standardized using ANGOF-based IRT method, and Bookmark's standards in three separate sections (grammar, listening, Reading comprehension) were identified in three evaluation stages. The findings of the study showed that the cut-off score obtained by Angof's method based on the question-answer theory is 53.66 and the bookmark method is 54.27. Both cut scores were higher than those determined by the Ministry of Science. Also, the findings of the study showed that 73.3% of the participants rejected the traditional and traditional method of the Ministry of Science, which is based on the score of 50, but 26.7% of the participants failed but based on the Angof method. Based on the question-answer theory, rejection rates are 78.5% and acceptance rates are 23.4%. Finally, according to Bookmark, the decline in statistics is 21.5%. The research findings imply that Ministry of Science language test designers need to be revised to determine the standard score of this test. determine the standard score of this test. The main purpose of this study was to compare the Angof-based IRT method and the Bookmark method to determine standardization of the MSRT language test. For this purpose, one sampl of the MSRT test questions (September 1397 test) was randomly selected and the answers to the questions were obtained by 596 subjects from the Ministry of Science. The MSRT test has 100 questions, of which 30 are grammar, 30 are auditory, and 40 are reading comprehension. Two expert panels consisting of 15 TOEFL teaching experts were formed and then standardized using ANGOF-based IRT method, and Bookmark's standards in three separate sections (grammar, listening, Reading comprehension) were identified in three evaluation stages. The findings of the study showed that the cut-off score obtained by Angof's method based on the question-answer theory is 53.66 and the bookmark method is 54.27
خلاصه ماشینی:
دو پنل تخصصي متشکل از ١٥ متخصص تدريس آزمون تافل تشکيل شد و سپس با استفاده از روشهاي تعيين استاندارد آنگوف مبتني بر نظريه سؤال- پاسخ و بوکمارک استانداردهاي اين آزمون در سه بخش جداگانه )گرامر، شنيداري، درک مطلب خواندن( در سه مرحله ارزيابي مشخص شدند.
اولين محدوديت اين روش ناتواني ارزيابان در برآورد معقول از عملکرد آزمودنيهاي داراي حداقل شايستگي در سؤالت آزمون است )٢٠٠٨ ,Williams &Clauser, Harik, Margolis, McManus, Mollon, Chis (.
Response probobility مرحله بازخوردهاي لزم به اعضاي گروه داده ميشود و در نهايت در مرحله سوم يک نمره برش کلي که همان نمره استاندارد آزمون با استفاده از روش آنگوف مبتني بر نظريه سؤال پاسخ است حاصل ميشود.
روش بوکمارک: يکي از روشهاي سؤال محور براي تعيين نمره برش است که در آن سؤالت ابتدا براساس نظريه سؤال پاسخ بهترتيب ضريب دشواري رتبهبندي ميشوند و کتابچه طبقهبندي سؤالت به تفکيک از سؤالت آسان به دشوار تهيه و در اختيار گروهي از ارزيابان متخصص در حوزه مرتبط )براي مثال زبان انگليسي( قرار داده ميشود.
با توجه به يافتههاي پژوهش فعلي که نمره برش آزمون MSRT براي روش آنگوف مبتني بر نظريه سؤال - پاسخ برابر با ٥٣/٦٦، براي روش بوکمارک برابر با ٥٤/٢٢ بهدست آمد و ازآنجاکه نمره برش تعيين شده توسط وزارت علوم براي اين آزمون برابر با ٥١ است.
براي مثال، يافتههاي )٢٠٠٣ ,Wang( دال بر اين بود که نمره برش بهدستآمده از روش بوکمارک در مقايسه با نمرات بهدستآمده از روش آنگوف پايينتر است و يا پژوهشهاي انجام شده توسط )٢٠١٣ ,Peterson(، )٢٠٠٢ ,Plake &Buckendahl, Smith, Impara .