Life is about tradeoffs. Work is about tradeoffs. Work-life balance is about tradeoffs.
Nothing is perfect in every possible way. Instead, tradeoffs grounded in external pressure, priorities and/or values aim towards some sort of acceptable balance of different factors and criteria. These tradeoffs really are about values.
There are many different values that we might bring to large scale assessment development, including
trust of educators.
information for parents.
feedback for students.
information for policy makers.
reliability.
validity.
testing/seat time.
cost of development.
operational costs.
scoring costs.
And no doubt many many more.
Some of these potential values have clearly been given lower priority, and some higher priority.
Now, The Standards for Educational and Psychological Testing say (in the second sentence of the very first chapter) that validity is "the most fundamental consideration in developing tests and evaluating tests.” I quote this line all the time. Validity is the alpha and omega of assessment — or it should be.
I am concerned — I have been concerned for a long long time — that validity is not given the priority that our bible says it should be. I am concerned that we optimize for reliability, at the expense of validity. I am concerned that we optimize for time, at the expense of validity. I am concerned that we optimize for cost, at the expense of validity.
Reliance on templated items and various forms of automation can deliver less expensive tests. But they will be worth less because of the increasing costs to validity. If the goal of these new efforts and technologies is to invest those cost savings and time saving back into validity, perhaps they can be worth it. But if the vision is to merely save time and money, they are just another drain on validity, when our tests are far from having the validity to spare.