Without a grasp of concepts like maximum torque, number of cylinders, engine displacement, and horsepower, it would be difficult for someone looking for a new vehicle to compare and contrast different brands and models while searching for their optimal new ride. For instance, understanding the difference between 750 horsepower and 700 horsepower enables the consumer to imagine the added power of the stronger engine. However, torque may be more important than horsepower to a person interested in using their vehicle to tow a substantial weight. Determining exactly what attributes are important for the purchase decision can take a surprising amount of study and consideration. This assertion about the utility of being familiar with product attributes — and the relative importance of each of those attributes for the needs at hand — holds true for just about any product (including personality assessments).
For personality assessments such as those created by Tilt 365, two primary characteristics that convey some of the strengths of the given assessment are “validity” and “reliability.” To fully understand why our company has been rated a Top 20 company in the Training Industry for multiple years, one must have knowledge of these two psychometric concepts. As with most words, these terms can have drastically different meanings when used in everyday conversation as opposed to in more formal settings (e.g., where they are being used by a professor to defend the merits of a particular exam format).
Definitions and Background for Each Concept
An accessible example for illustration purposes is the scale at the doctor’s office. If someone knows that they weigh 150 pounds (perhaps by having recently used numerous other scales and having received that same number each time), but the scale says they weigh 300 pounds, we have pretty good reason to believe that this scale’s measurement is invalid. But if the same person steps on a different scale and it reads 150 pounds, we have less of a reason to doubt that that measure is inaccurate, and thus we are more willing to think that the scale is serving as a valid measurement tool.
Now let’s suppose the same person eats well for a month and doesn’t gain any weight. When they get weighed again after those thirty days and the scale reads “150 pounds,” we have witnessed the scale demonstrating reliability (a synonym for this type of reliability would be “consistent”; in technical language, this is called Test-Retest Reliability). Even if the scale read 300 pounds the first time and then still read 300 pounds after the thirty days of eating well, we could still say it demonstrates reliability (even though the reading is clearly invalid/inaccurate).
This demonstrates an interesting nuance between the two terms: A measurement can be reliable but not necessarily valid (as just discussed above), but a measurement cannot be both valid and unreliable. Continuing our scale example: if one steps on a scale and gets a reading of 150, then steps off and back on and gets a reading of 300, and then off and on again for a reading of -25, we would have a great reason to conclude that the scale is both unreliable (because it doesn’t give the same numbers each time) and, in this case, we can therefore also conclude that the scale is not valid (because a valid scale would accurately show the weight each time, more or less) — despite its having shown the “correct” weight the first time that we tried it.
If we extend this to an academic setting, we want a math test to be written in such a way that someone who understands 95% of the content will approximately get, for instance, a 95 on the test. And if they take the same test a month later, (while not forgetting anything learned and also while not learning any more of the content), we want them to roughly get a 95 again (i.e., we want the test to be valid/accurate and reliable/consistent).
Test-Retest Reliability and Change
In both the weight and the math-test example above, you may have noticed an important stipulation: We are assuming that nothing about the underlying state of the individual (specifically, their weight or their understanding of math content) is changing. As you can imagine, determining whether a measure is reliable (at least in this particular way) is much more tricky when we allow for or even expect changes in the underlying attribute being measured: If we change our diet and exercise routines for several months and then return to the scale, we would be suspicious of the scale’s reliability if it didn’t read a different value. Likewise, we would raise our eyebrows if a diligent student received the same score on the same assessment of math understanding at the beginning and at the end of the (developmentally appropriate) math course on that topic.
