Not All Test Validation Strategies are Equal
Did you know that the majority of employment tests are not criterion validated? Overwhelmingly, content validation strategies are the industry standard. Content Validation is demonstrated by job analysis and analysis of the relationship between test content and job demands, with subject matter expert information being its foundation. This is often documented in the form of a job analysis. Content validation is an integral part in ensuring that an exam is job-related.
While content validation is important in demonstrating that exam content is job-related, it does not provide strong empirical evidence of the relationship between test scores and performance on the job. Criterion Validation is conducted to demonstrate this relationship through data collection and statistical analysis. Criterion validation consists of comparing test scores to job performance measures as a proof of the predictive value of the test. Performance evaluations are the most common criteria used to validate test scores. That is, the objective of the study is to empirically demonstrate that higher test scores are associated with higher performance evaluations from supervisors. Validation through use of quantitative criteria, in addition to content validation, can provide strong defensibility.
It is not always feasible to conduct criterion validation studies. Limitations such as test exposure and cost of the process are a couple of the major barriers to conducting such a study. Test exposure is not acceptable in many organizations since the perception is that these tests will be compromised and create unfair advantages to some test takers. If feasible, it is always better to have both criterion and content validation.
Remember to ask how a test was validated, not just if it is validated.