Deck 7: Content Validation

Question 1

A first step in any content validation process is ________.

A) recruiting Subject Matter Experts (SMEs)
B) securing funding to pay Subject Matter Experts (SMEs)
C) clearly defining the content domain
D) establishing protocols for combining Subject Matter Expert (SME) ratings.

Answer 1

C

Question 2

In an effort to improve the content validity of a multidimensional measure, the number of items per dimension should be ________.

A) no fewer than 6
B) no more than 20
C) divided equally across dimensions
D) proportionate to the relative importance of each dimension

Answer 2

D

Question 3

Lawshe's (1975) content validity ratio (CVR) is computed by using Subject Matter Experts (SMEs) ratings of the degree to which items are _______ to the operationalization of the construct.

A) at least minimally related
B) essential
C) useful or essential
D) devoid of content unrelated

Answer 3

B

Question 4

When interpreting the size of a Content Validity Ratio (CVR), a value of _______ would indicate that half the Subject Matter Experts (SMEs) provided the highest possible rating.

A) 0
B) )25
C) )50
D) 1.0

Question 5

While researchers continue to debate the minimally acceptable Content Validity Ratio (CVR) value, one point of agreement is that the more _______ used, the better.

A) test developers
B) criteria
C) Subject Matter Experts (SMEs)
D) items

Question 6

Shultz, Whitney, and Zickar (2021) reported that newer approaches to content validation such as those of Colquitt et al (2019) and Hinkin and Tracey (1999) ask individuals to provide Likert-type ratings of the degree to which items are _______.

A) free from bias and cultural influences
B) clearly written
C) likely to distinguish between test-takers of low and high ability
D) consistent with the definition of the intended construct

Question 7

Which of the following individuals would be least appropriate to serve as a Subject Matter Expert (SME) to provide content validation evidence for a job knowledge test?

A) An employee whose performance was rated as satisfactory
B) An employee nearing retirement
C) A newly hired employee
D) A supervisor

Question 8

_______ may result in challenges to the qualifications of those selected to serve as Subject Matter Experts (SMEs).

A) Legal disputes
B) Low criterion-related validity estimates
C) Unstandardized test administration procedures
D) Experts not selected to participate in the test development process

Question 9

Recent approaches to content validation suggest that a minimum criterion for serving as a Subject Matter Expert (SME) would be possession of _______.

A) a high school diploma
B) a college degree
C) sufficient intellectual ability to judge the correspondence between items and theoretical constructs
D) 5 or more years of recognized expertise

Question 10

The content validation process for which of the following measures would likely be easier than the others?

A) A newly developed measure of Openness to Experience
B) A pre-existing measure of Integrity
C) A pre-existing measure of Excitability
D) A newly developed measure of Job Knowledge

Question 11

What's the logic behind content validation? Specifically, why do we believe it's important to have subject matter experts (SMEs) review the items on a test?

Question 12

Given the importance of the judgment of subject matter experts (SMEs) in content validation, what characteristics of the SMEs are essential?

Question 13

(a) Identify the three aspects of a test examined in Barrett's (1996) Content Validation Form (CVF).
(b) What does the CVF address that is unable to be addressed computing the CVR for each item on the test?

Question 14

Why might face validity be considered an important element of the content validation of a measure?

Question 15

Why does content validation remain an important tool for test developers in demonstrating the appropriateness of a measure?