Imagine that you are in a kitchen and need to measure the volume of some odd solid object, or the difference in volumes between two odd solid objects. But the only real measuring tools are scales (i.e, a kitchen scale and a bathroom scale) and any number of household tape measures, rulers and yard/meter sticks. And the internet is down.
* One approach might be to simply take the mass of the object(s) and figure that most things have a mass of around 1 g/cm3, and go with that. If you need the difference, take the difference.
* Another approach might be to do that Archimedes thing and try displacement. Fill up a cup or larger container to the bring with water, drop the object in the cup and catch all the water that the new object forces out of the cup. That would take a saucer (or serving platter) under the vessel to catch the water. Measure the mass of that saucer (or serving platter) empty and with the water. Eureka! The difference is the volume, so long as you convert the units, right? So clever, that Archimedes.
* The third, and hardest approach would be very much like the second approach, but it departs from the Archimedes version, because these objects are not gold crowns. You’d need to push the object down into the water, making sure that it is entirely submerged—but without putting anything else in the water. Either push it JUST under the water, or use some very very fine tools to hold it down further. Again, calculate the mass of the displaced water and convert the units. That’s the mass of the object, and just subtract the lower mass from the greater if comparing two objects.
The third approach is way more clever than the first two, and is the only one that will actually give you volume. The first approach approximates volumes, but will not work for objects that easily float or sink—signaling a density significantly different than water’s. The first approach just gives you mass. The second approach will work for denser objects, which do entirely submerge in the water, but not for objects that float (i.e., are not entirely submerged). For the former, yes volume. But for the latter it just gives mass again. Not actually as clever as we thought.
(Archimedes’s experiment as a bit different, and he had a whole bunch of spare gold lying around. Neither you nor I have that available for our work.)
I have no doubt that there are many people who think that psychometrics is analogous to the third approach. That it really is clever enough to take the products of limited tools to measure difficult constructs. But what I see is a dependence of limited tools that simply measure something different than the intended construct. And, no, the analysis is not so clever as successfully to convert the results to the intended construct. Disturbingly, it is not that adequate tools are not available, rather it’s the insistence on using unidimensional psychometric models and filters to measure multi-dimensional constructs. There are other models, they just are not favored. Perhaps they are not as easy to use. Perhaps they don’t have the established place in curricula and/or practice. Perhaps it is simply if we've always tended to use a hammer, we tend to redefine problem into problems that can be solved with a hammer.
But the charge of testing is to measure the intended construct, not some other construct that our favored tools are better at measuring.