Multiple choice (MC) items are not merely questions with a bunch of supplied answer options. They do not operate like constructed response (CR) or open-ended question. Unfortunately, too few people understand and appreciate how important answer options are to understanding how people respond to MC items.
While the famous Haladyna, Downing and Rodriquez list of item writing guidelines say that answer options should be placed in a logical order, they do not address the impact of that oder on how test takers work through an item. Sure, answer options could be ordered by length, put in alphabetical order or some sort of chronological order. But whatever rule one follows, it can have unintended impacts on the cognitive path that test takers work down to come to their choice.
Simply compare the cognitive path of placing the correct answer (i.e., the key) in the first position or in the last position.
For items such as mathematical calculations, putting the key first allows the test taker to skip all the other answer options entirely. But if the key is the last answer option, the test taker must consider (and perhaps compare to their answer) each of the other answer options before recognizing the key at last.
But if the item is perhaps less black and white, the test take might have to try to interpret and make sense of answer options, comparing each to their own thinking. If the key is first position, the test taker can quickly come to a sense that it matches their answer, select it and then move on. But if the key is last, the test taker has to figure out whether each distractor (i.e., an incorrect answer option) means what they are looking for, or whether it means something else. As they move through the list, they might lean a little bit more into a “Well, does it kinda mean the same thing?” sort of thinking.
Clearly, the order of answer options can impact how long it takes test takers to work through an item, and the sort of thinking they need to do—even without changing anything about any individual answer option. Different test taker strategies can also influence these, but distractors matter.
Obviously answer option order impacts the how test takers respond to items, right?