Question 1

Which of the following individuals would be least appropriate to serve as a Subject Matter Expert (SME) to provide content validation evidence for a job knowledge test?

Accepted Answer

A)  An employee whose performance was rated as satisfactory 
B)  An employee nearing retirement 
C)  A newly hired employee 
D)  A supervisor 
A)  An employee whose performance was rated as satisfactory 
B)  An employee nearing retirement 
C)  A newly hired employee 
D)  A supervisor C

Question 2

(a) Identify the three aspects of a test examined in Barrett's (1996) Content Validation Form (CVF).
(b) What does the CVF address that is unable to be addressed computing the CVR for each item on the test?

Accepted Answer

(a) The CVF possess 18 questions to SMEs to assess (i) the test as a whole (ii) an item-by-item analysis, and (iii) symptoms of questionable tests.
(b) Unfortunately, neither the CVR nor the CVI can provide an indication of the degree to which items proportionately represent the relative important of each dimension the test is intended to measure. The CVF would be able to better provide content validation information in this respect since the test as a whole is examined.

Question 3

Shultz, Whitney, and Zickar (2021) reported that newer approaches to content validation such as those of Colquitt et al (2019) and Hinkin and Tracey (1999) ask individuals to provide Likert-type ratings of the degree to which items are _______.

Accepted Answer

A)  free from bias and cultural influences 
B)  clearly written 
C)  likely to distinguish between test-takers of low and high ability 
D)  consistent with the definition of the intended construct 
A)  free from bias and cultural influences 
B)  clearly written 
C)  likely to distinguish between test-takers of low and high ability 
D)  consistent with the definition of the intended construct D

Question 4

Lawshe's (1975) content validity ratio (CVR) is computed by using Subject Matter Experts (SMEs) ratings of the degree to which items are _______ to the operationalization of the construct.

Accepted Answer

A)  at least minimally related 
B)  essential 
C)  useful or essential 
D)  devoid of content unrelated 
A)  at least minimally related 
B)  essential 
C)  useful or essential 
D)  devoid of content unrelated

Question 5

Recent approaches to content validation suggest that a minimum criterion for serving as a Subject Matter Expert (SME) would be possession of _______.

Accepted Answer

A)  a high school diploma 
B)  a college degree 
C)  sufficient intellectual ability to judge the correspondence between items and theoretical constructs 
D)  5 or more years of recognized expertise 
A)  a high school diploma 
B)  a college degree 
C)  sufficient intellectual ability to judge the correspondence between items and theoretical constructs 
D)  5 or more years of recognized expertise

Question 6

_______ may result in challenges to the qualifications of those selected to serve as Subject Matter Experts (SMEs).

Accepted Answer

A)  Legal disputes 
B)  Low criterion-related validity estimates 
C)  Unstandardized test administration procedures 
D)  Experts not selected to participate in the test development process 
A)  Legal disputes 
B)  Low criterion-related validity estimates 
C)  Unstandardized test administration procedures 
D)  Experts not selected to participate in the test development process

Question 7

When interpreting the size of a Content Validity Ratio (CVR), a value of _______ would indicate that half the Subject Matter Experts (SMEs) provided the highest possible rating.

Accepted Answer

A)  0 
B)  0.25 
C)  0.50 
D)  1.0 
A)  0 
B)  0.25 
C)  0.50 
D)  1.0

Question 8

A first step in any content validation process is ________.

Accepted Answer

A)  recruiting Subject Matter Experts (SMEs) 
B)  securing funding to pay Subject Matter Experts (SMEs) 
C)  clearly defining the content domain 
D)  establishing protocols for combining Subject Matter Expert (SM
E)  ratings. 
A)  recruiting Subject Matter Experts (SMEs) 
B)  securing funding to pay Subject Matter Experts (SMEs) 
C)  clearly defining the content domain 
D)  establishing protocols for combining Subject Matter Expert (SM
E)  ratings.

Question 9

The content validation process for which of the following measures would likely be easier than the others?

Accepted Answer

A)  A newly developed measure of Openness to Experience 
B)  A pre-existing measure of Integrity 
C)  A pre-existing measure of Excitability 
D)  A newly developed measure of Job Knowledge 
A)  A newly developed measure of Openness to Experience 
B)  A pre-existing measure of Integrity 
C)  A pre-existing measure of Excitability 
D)  A newly developed measure of Job Knowledge

Question 10

Given the importance of the judgment of subject matter experts (SMEs) in content validation, what characteristics of the SMEs are essential?

Accepted Answer

A number of suggested characteristics of

Question 11

Why does content validation remain an important tool for test developers in demonstrating the appropriateness of a measure?

Accepted Answer

Validation is not a single step, but rat

Question 12

In an effort to improve the content validity of a multidimensional measure, the number of items per dimension should be ________.

Accepted Answer

A)  no fewer than 6 
B)  no more than 20 
C)  divided equally across dimensions 
D)  proportionate to the relative importance of each dimension 
A)  no fewer than 6 
B)  no more than 20 
C)  divided equally across dimensions 
D)  proportionate to the relative importance of each dimension

Question 13

While researchers continue to debate the minimally acceptable Content Validity Ratio (CVR) value, one point of agreement is that the more _______ used, the better.

Accepted Answer

A)  test developers 
B)  criteria 
C)  Subject Matter Experts (SMEs) 
D)  items 
A)  test developers 
B)  criteria 
C)  Subject Matter Experts (SMEs) 
D)  items

Question 14

Why might face validity be considered an important element of the content validation of a measure?

Accepted Answer

Face validity relies on the perception o

Question 15

What's the logic behind content validation? Specifically, why do we believe it's important to have subject matter experts (SMEs) review the items on a test?

Accepted Answer

An infinite number of items could be cre

Which of the following individuals would be least appropriate to serve as a Subject Matter Expert (SME) to provide content validation evidence for a job knowledge test?

(a) Identify the three aspects of a test examined in Barrett's (1996) Content Validation Form (CVF). (b) What does the CVF address that is unable to be addressed computing the CVR for each item on the test?

Shultz, Whitney, and Zickar (2021) reported that newer approaches to content validation such as those of Colquitt et al (2019) and Hinkin and Tracey (1999) ask individuals to provide Likert-type ratings of the degree to which items are _______.

Lawshe's (1975) content validity ratio (CVR) is computed by using Subject Matter Experts (SMEs) ratings of the degree to which items are _______ to the operationalization of the construct.

Recent approaches to content validation suggest that a minimum criterion for serving as a Subject Matter Expert (SME) would be possession of _______.

_______ may result in challenges to the qualifications of those selected to serve as Subject Matter Experts (SMEs).

When interpreting the size of a Content Validity Ratio (CVR), a value of _______ would indicate that half the Subject Matter Experts (SMEs) provided the highest possible rating.

A first step in any content validation process is ________.

The content validation process for which of the following measures would likely be easier than the others?

Given the importance of the judgment of subject matter experts (SMEs) in content validation, what characteristics of the SMEs are essential?

Why does content validation remain an important tool for test developers in demonstrating the appropriateness of a measure?

In an effort to improve the content validity of a multidimensional measure, the number of items per dimension should be ________.

While researchers continue to debate the minimally acceptable Content Validity Ratio (CVR) value, one point of agreement is that the more _______ used, the better.

Why might face validity be considered an important element of the content validation of a measure?

What's the logic behind content validation? Specifically, why do we believe it's important to have subject matter experts (SMEs) review the items on a test?

Introduction and Overview

Statistics Review for PSYC Measurement

Sychological Scaling

Test Preparation and Specification

Reliability Overview: Classical Test Theory

Estimating Reliability

Criterion-Related Validation

Construct Validation

VG Psychometric Meta-Analysis Copy

Tests Bias Unfairness and Equivalence

Developing Tests of Maximal Performance

Classical Test Theory Item Analysis

Scoring Tests

Developing Measures of Typical Performance

Response Biases

Multiple Regression

Exploratory Factor Analysis

Confirmatory Factor Analysis

Item Response Theory

Applications of Item Response Theory: Computer Adaptive Testing and Differential Item Functioning

Generalizability Theory

Filters

Exam 7: Content Validation

Which of the following individuals would be least appropriate to serve as a Subject Matter Expert (SME) to provide content validation evidence for a job knowledge test?

(a) Identify the three aspects of a test examined in Barrett's (1996) Content Validation Form (CVF). (b) What does the CVF address that is unable to be addressed computing the CVR for each item on the test?

Shultz, Whitney, and Zickar (2021) reported that newer approaches to content validation such as those of Colquitt et al (2019) and Hinkin and Tracey (1999) ask individuals to provide Likert-type ratings of the degree to which items are _______.

Lawshe's (1975) content validity ratio (CVR) is computed by using Subject Matter Experts (SMEs) ratings of the degree to which items are _______ to the operationalization of the construct.

Recent approaches to content validation suggest that a minimum criterion for serving as a Subject Matter Expert (SME) would be possession of _______.

_______ may result in challenges to the qualifications of those selected to serve as Subject Matter Experts (SMEs).

When interpreting the size of a Content Validity Ratio (CVR), a value of _______ would indicate that half the Subject Matter Experts (SMEs) provided the highest possible rating.

A first step in any content validation process is ________.

The content validation process for which of the following measures would likely be easier than the others?

Given the importance of the judgment of subject matter experts (SMEs) in content validation, what characteristics of the SMEs are essential?

Why does content validation remain an important tool for test developers in demonstrating the appropriateness of a measure?

In an effort to improve the content validity of a multidimensional measure, the number of items per dimension should be ________.

While researchers continue to debate the minimally acceptable Content Validity Ratio (CVR) value, one point of agreement is that the more _______ used, the better.

Why might face validity be considered an important element of the content validation of a measure?

What's the logic behind content validation? Specifically, why do we believe it's important to have subject matter experts (SMEs) review the items on a test?

Introduction and Overview

Statistics Review for PSYC Measurement

Sychological Scaling

Test Preparation and Specification

Reliability Overview: Classical Test Theory

Estimating Reliability

Criterion-Related Validation

Construct Validation

VG Psychometric Meta-Analysis Copy

Tests Bias Unfairness and Equivalence

Developing Tests of Maximal Performance

Classical Test Theory Item Analysis

Scoring Tests

Developing Measures of Typical Performance

Response Biases

Multiple Regression

Exploratory Factor Analysis

Confirmatory Factor Analysis

Item Response Theory

Applications of Item Response Theory: Computer Adaptive Testing and Differential Item Functioning

Generalizability Theory

Filters