We are finally working on the book, Rigorous Test Development: A Practical and Conceptual Guide for Content Development Professionals.
This book is intended for the people who develop standardized tests, on the item level. There is a startling lack of training, certification or professionalized/disciplinary knowledge available for them. There are no virtually no textbooks, journals, courses, degree programs or professional development programs. There is nothing that explains how they fit into the larger test development process, that gives them a conceptual view of their work or tries to treat like intelligent and decimated professionals who want to learn and grow.
This standards in contrast to psychometrics, who can get masters degrees and doctorates, who have countless conferences, journals and online courses available to them.
As in so many areas that we care about, are worried about that quality of the work that we so care about, and want very much to help it to improve. We know how the sausage is made, and we want better processes for making it so that better sausage is available to everyone.
Yeah, test items are the sausage, in this metaphor. But you knew that, right?
We know that low quality items can only produce low quality tests. You simply cannot make chicken salad out of chicken scat. We care enormously about item quality because we know that standardized matter and are not going away.
Which (new school) begs the question: what even is item quality? Well, that’s in the book! The industry doesn’t really have a good definition of item quality — though we do. No one offers ideas about how to think about item quality or how to improve it. All of that is in the book, and we will write abut it a little bit here jin the weeks and months ahead.
The thing that really makes RTD different is that our lens for duding item quality is the interactions of test takers and content. It is a content-focused view. We do not think that item quality can be judged without looking at the contents of the item, and the KSAs (knowledge, skill and/or ability) from the target domain that that the item is targeted. Statistical analyses based on data from field or operational testing can help, but that data comes far too late, far too expensively and never gives any insight as to why an item is performing well or poorly. We offer frameworks for thinking about that.
So, we are seriously working on the book. It is coming. In the weeks and months ahead, we will offer previews of the thinking behind various chapters, and next week we will even offer a preview of perhaps the most important chapter, The RTD Theory of the Item.