TYPES OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS IN MEDICAL EDUCATION AND THEIR EFFECT ON PASSING RATE OF STUDENTS
Keywords:
MCQs, flawed items, Academic achievements, Medical Education, UndergraduateAbstract
Background: The most popular written exam in health professions education is the multiple-choice style question (MCQ). This study sought to understand the effects of multiple-choice writing errors on students' academic performance in medical education. Methods: This descriptive study was done from December 2017 to June 2019 in Azad Jammu Kashmir (AJK) Medical College, Muzaffarabad. Ten block examinations were included. The item review committee reviewed all MCQs for flaws. Two tests from each class of MBBS were taken. The original tests containing all items were labeled as flawed tests and the result of each flawed test was evaluated. The students were graded into high, moderate and low achievement groups with scores of more than 79.9, between 50–79.9 and less than 50 percent respectively. Flawed items were then removed from the tests by the review committee and the scores of each test (standard test) were determined and compared with flawed tests and its effects were assessed in three achieving groups of students. Optical mark reading (OMR) classic-4 software was used for post-exam analysis and data were analyzed by using SPSS-25. Results: The passing rate ranged from 68.18% to 90.82% in flawed and 75.54% to 93.69% in standard tests. Most standard tests (7) had higher passing rates than flawed tests. Conclusion: Tests containing in-house developed MCQs have frequent item writing flaws and their inclusion in assessment did affect the passing rates of students but were not statistically significant.
Pak J Physiol 2023;19(4):49–52
Downloads
References
Downing SM, Yudkowsky R, (Eds). Assessment in health professions education. New York: Routledge; 2009.p.149–84.
Palmer, EJ, Devitt, PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ 2007;7:49.
Schuwirth LW, van der Vleuten, CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ 2004;38(9):974–9.
DiBattista D, Kurzawa, L. Examination of the quality of multiple-choice items on classroom tests. Can J Scholarsh Teach Learn 2011;2(2):4.
Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. National Board of Medical Examiners: Philadelphia, PA; 1998.
Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ 2002;15(3):309–34.
Downing SM. Construct-irrelevant variance and flawed test questions: Do multiple-choice item-writing principles make any difference? Acad Med 2002;77(10 Suppl):S103–4.
Downing SM. The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Adv Health Sci Educ Theory Pract 2005;10:133–43.
Tarrant M, Ware J. Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Med Educ 2008;42(2):198–206.
Wadi M, Rahim AFA, Yusoff MSB, Baharuddin, KA. The effect of MCQ vetting on students’ examination performance. Educ Med J 2014;6(2):e16–26.
Przymusza?a P, Piotrowska K, Lipski D, Marciniak R, Cerbin-Koczorowska M. Guidelines on writing multiple choice questions: A well-received and effective faculty development intervention. SAGE Open 2020;10(3):2158244020947432.
Fayyaz Khan H, Farooq Danish K, Saeed Awan A, Anwar M. Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions. Pak J Med Sci 2013;29(3):715–8.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Pakistan Journal of Physiology
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Pakistan Journal of Physiology, Pak J Physiol, PJP is FREE for research and academic purposes. It can be freely downloaded and stored, printed, presented, cited and quoted with full reference of, and acknowledgement to the PJP.
