Because assessment is an important part of teaching, it is not surprising that content development professionals (CDPs) for large scale assessment (e.g., standardized testing)—the professionals who develop and refine the contents of tests—require many of the same skills, knowledge and expertise as teachers. However, CDPs also need other skills and knowledge to work at a high level.
Unlike teachers, CDPs do not have to worry about classroom management or lesson plan construction. But they do need a teachers’ understanding of the content area and how to think about content and learning paths. Of course, both teachers and CDPs need to understand the cognition of others without unwittingly projecting their thinking on them.
However, CDPs need to think about these things a bit differently than teachers. Large scale assessment does not have as many opportunities to assess students as classroom practice, so it must do so much more efficiently. Teachers can triangulate lots of different information from and about students to figure out whether they understand something, but large scale assessment usually depends on a single assessment to make that inference. Therefore, CDPs need a much more precise view of evidence than teachers’. They need to be able to recognize the ambiguity of evidence so that they can create test items that can elicit more definitive evidence of a test takers’ level of proficiency.
While teachers often focus on how to integrate the knowledge, skills and abilities found in various learning standards into larger lessons and activities, CDPs need to understand how to isolate them while still preserving some amount of authenticity. They need to be particularly mindful of the kinds of mistakes that learners make and how they relate to particular learning goals—recognizing their connection to the targeted cognition of an item.
Like teachers, CDPs need to understand how others think—often others very different than themselves. Teachers have their students in front of them, and can learn more about them over time. CDPs have to imagine test takers, rather than being reminded of them every day by their presence. Moreover, the range of diversity and dimensions of diversity are vastly larger with large scale assessment than a single teacher in a single school must account for. We call this radical empathy because of the amount of variation in background, experience and perspective that CDPs must be able to anticipate.
CDPs also require technical knowledge and skills that teachers do not. CDPs need to know the differences between different item types, how they work, and which are most appropriate to elicit evidence of different sorts of cognition. They need to be able to recognize problems in a multiple choice item and how to make it better elicit evidence of the targeted cognition for the range of typical test takers. The fact is that it is incredibly difficult to create high quality multiple choice items that produce high quality evidence, a fact that makes it more important to take those challenges seriously.
They need to understand the workflows, contributors and collaborations that comprise the test development process. Moreover, they need to have the ability to push back against the various pressures to alter items in ways that compromise they ability to elicit evidence of that targeted cognition for the range of typical test takers—or even omit them entirely from an assessment. Of course, all of this requires understanding the values and thinking of the the many different disciplines that contribute to these collaborations.
I would never suggest that CDP work is more difficult or complex than teacher work; clearly it is not. Working with children—of any age—and being sensitive to their needs is enormously challenging and complex work, made more so the official and unofficial learning goals. All of those challenges are magnified exponentially by the reality of how many children are there at the same time.
However, the work of developing the contents of standardized tests is itself complex and difficult, mostly in ways unappreciated by the public—and even by others involved in large scale assessment. It leans on many areas of skill and knowledge that overlap with teachers’ expertise, but it has different goals and constraints. Therefore, it also requires different expertise—including, but not limited to—expertise in the content area.