Exam 8: Criterion-Related Validation

To improve prediction over random selection, the validity coefficient must be at least of ________ magnitude.

Using Cohen's (1988) guidelines to evaluate the magnitude of correlation-based effect sizes, an obtained criterion-related validity coefficient of .52 would be considered ________.

Consider a scenario in which a highly selective school wanted to improve its selection of prospective students. A consultant developed a new predictor with an internal consistency reliability of .75. The consultant then correlated the new measure with a criterion of student success, and found that the resulting correlation was .39. Prior to reporting the results, the consultant considered whether corrections for attenuation due to unreliability should be performed in the predictor, the criterion, neither, or both. Provide a recommendation explaining how the consultant should proceed.

The correction of artifacts impacting the predictor in a criterion-related validity study would likely ________ the estimate of criterion-related validity.

________ criterion-related validation designs typically provide the greatest freedom in obtaining samples.

Construct irrelevant variance is closely tied to ________.

________ criterion-related validation designs require considerably more time before drawing a conclusion about the experimental measure.

A small organization which has too few employees to conduct a full-scale criterion-related validation study may wish to employ the use of ________ validity.

(a) Explain why restriction in range in a predictor can be an issue for demonstrating criterion-related validity. (b) What is the biggest challenge in mathematically correcting for the problem of restriction in range?

Roth et al. (2017) found that the use of job incumbent samples to estimate the amount of variability in predictor scores in the population results in an ________ of the true criterion-related validity.

In a ________ criterion-related validation study, scores on a criterion of interest are collected prior to obtaining test scores.

(a) Identify one advantage of utilizing a concurrent or postdictive criterion-related validation designs compared to predictive criterion-related studies. (b) Identify two disadvantages of utilizing a concurrent or postdictive criterion-related validation study compared to predictive criterion-related studies.

_______ percent of the variance in a criterion would remain unexplained if the criterion-related validity of a test was found to be .30.

Criterion-related validation studies require sufficiently large sample sizes. (a) What common research concerns might lead you to doubt the results of a criterion-related validation study that utilized a small sample? (b) What minimal size sample was suggested by Schmidt, Hunter and Urry (1976) for conducting criterion-related validation studies?

Explain why the choice of a criterion is crucial to criterion-related validation.