[Each day in October, I analyze one of the 31 item writing rules from Haladyna, Downing and Rodriquez (2002), the super-dominant list of item authoring guidelines.]
Content: Keep vocabulary simple for the group of students being tested.
Hey! I like this one. This is good. Not even any claims of unanimity, when a large portion of their sources don’t even mention it.
Now, I wouldn’t quite say it this way. The issue is not really just about vocabulary, as their explanation shows (i.e., they mention “reading demand” and “simplified language”). Syntax, style, sentence length, paragraph construction and even passage length can be quite relevant. So, we could improve this rule, but it’s very much in the right ballpark.
What is truly great about this rule is that it acknowledges that it matters who are being tested. What is appropriate depends on this group of test takers. They cite testing ELL (English Language Learners) students, which is a natural concern. But this matters across all population, and it also gets to regionalisms and some other dimensions of fairness. We love that this rule acknowledges the existence of diversity and difference among test takers, and that it should shape how we all write, refine and evaluate items.
[Haladyna et al.’s exercise started with a pair of 1989 articles, and continued in a 2004 book and a 2013 book. But the 2002 list is the easiest and cheapest to read (see the linked article, which is freely downloadable) and it is the only version that includes a well formatted one-page version of the rules. Therefore, it is the central version that I am taking apart, rule by rule, pointing out how horrendously bad this list is and how little it helps actual item development. If we are going to have good standardized tests, the items need to be better, and this list’s place as the dominant item writing advice only makes that far less likely to happen.
Haladyna Lists and Explanations
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Routledge.
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. Routledge.
Haladyna, T., Downing, S. and Rodriguez, M. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education. 15(3), 309-334
Haladyna, T.M. and Downing, S.M. (1989). Taxonomy of Multiple Choice Item-Writing Rules. Applied Measurement in Education, 2 (1), 37-50
Haladyna, T. M., & Downing, S. M. (1989). Validity of a taxonomy of multiple-choice item-writing rules. Applied measurement in education, 2(1), 51-78.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied measurement in education, 15(3), 309-333.
]