Deck 6: How Do We Gather Evidence of Validity Based on the Content of a Test

Question 1

What are concrete attributes?

A) attributes that can be clearly described in terms of observable and measurable behaviors
B) attributes that are difficult to describe in terms of behaviors as people may disagree on what these behaviors represent
C) attributes that specifically describe the characteristics of an individual's personality
D) attributes for which the psychological test gives a raw score for each test taker

Answer 1

A

Question 2

Which of the following are attributes that are difficult to describe in terms of behaviors because people disagree on what these behaviors represent?

A) abstract
B) absolute
C) concrete
D) comparative

Answer 2

A

Question 3

Defining the testing universe is an important step for ensuring a test has evidence of validity based on its ______.

A) content
B) reliability
C) relations with variables
D) constructs

Answer 3

A

Question 4

Which one of the following statements best describes a practical test?

A) the driving portion of a driving test
B) the math portion of the SAT
C) tests such as temperature or weight measurement
D) a job interview

Question 5

Which of the following would provide evidence of a test's validity based on content?

A) the extent to which test questions are representative of the knowledge domain measured
B) the extent to which a person's test scores can predict his or her current and future behavior
C) the extent to which the test taker recognizes the purpose of each test question
D) a numerical analysis of the test's construct dimensions

Question 6

When test developers choose the type of test they wish to create, such as essay or multiple choice, they are choosing the ______.

A) testing universe
B) test specifications
C) test format
D) content areas

Question 7

When test scores used to select employees are systematically related to a score on a job performance appraisal instrument, the test is demonstrating evidence of validity based on its ______.

A) content
B) relationship with another construct
C) reliability
D) relationship with a criterion

Question 8

Which one of the following types of evidence of validity can be gathered without using correlations?

A) evidence based on relations with a construct
B) evidence based on test content
C) predictive evidence of validity
D) concurrent evidence of validity

Question 9

What do we call a documented plan containing details about a test's content?

A) testing universe
B) content area
C) test specifications
D) test format

Question 10

If a test includes questions that are a representative sample of the material covered in a training course, the test demonstrates evidence of ______.

A) reliability
B) validity based on a relation with other variables
C) face validity
D) validity based on content

Question 11

When we ask, "Are the inferences being made from a set of test scores appropriate?" we are referring to ______.

A) internal consistency
B) face validity
C) reliability
D) validity

Question 12

______ is/are difficult to describe because people disagree on what behaviors represent, whereas ______ is/are clearly described in observable and measurable behaviors.

A) Concrete attributes; abstract attributes
B) Abstract attributes; concrete attributes
C) Concurrent validity; construct validity
D) Construct validity; concurrent validity

Question 13

Which one of the following research studies could yield information about evidence of validity based on the content of a test?

A) a study that involves performing a series of systematic steps during test development
B) a study that involves correlating test scores with a previously identified performance measure
C) a study that involves correlating test scores with another previously identified test
D) a study where test takers provide their opinion on how well test questions relate to the test topic

Question 14

When you ask experts to rate the extent to which individual questions are essential to measure the testing universe, you are gathering evidence of validity based on ______.

A) relations with other variables
B) internal structure
C) test content
D) response processes

Question 15

Evidence of construct validity is most appropriate when a test measures an abstract construct such as ______.

A) mathematical computation
B) English grammar
C) marital satisfaction
D) an IRS tax code

Question 16

Which of the following are considered "traditional" terms that are no longer used in the most recent version of the Standards for Educational and Psychological Testing?

A) reliability and precision
B) internal consistency and test-retest reliability
C) test developer and test publisher
D) content- and criterion-related validity

Question 17

Which one of the following statements about validity is FALSE?

A) Evidence of validity based on a test's relations with other variables involves correlating test scores to a measure of performance or another test.
B) Evidence of validity based on content involves examining the questions on the test to see whether they represent the domain being measured.
C) Evidence of validity based on content involves correlating test scores to a criterion.
D) Evidence of validity based on a test's relations with a criterion can be gathered via the concurrent and predictive methods.

Question 18

The predictive and concurrent methods provide two types of evidence for ______.

A) validity based on relations with a construct
B) face validity
C) validity based on relations with a criterion
D) validity based on content

Question 19

What is the appropriate strategy for gathering validity evidence of a test?

A) gathering every different source of evidence of validity
B) gathering one source of evidence of validity
C) gathering concrete and abstract attributes
D) it depends on the purpose of the test

Question 20

Evidence of validity based on content is most appropriate for what types of tests?

A) achievement tests that measure concrete attributes
B) personality tests that measure absolute attributes
C) concrete tests that measure aptitude and achievement
D) comparative tests that measure personality characteristics

Question 21

Explain the nature and importance of face validity. Give examples.

Question 22

Test specifications are very much like a(n) ______.

A) blueprint
B) matrix
C) table of contents
D) index

Question 23

Which of the following is a benefit of developing competency models?

A) A competency model will describe the specific tasks that are performed on a job.
B) A competency model will focus on the technical skills necessary to perform a job.
C) A competency model can often be used for multiple different jobs within an organization.
D) A competency model is most useful for developing high stakes selection programs for police and firefighters.

Question 24

Job analysis differs from competency modeling in that it ______.

A) is a less technically rigorous process than competency modeling
B) usually focuses on a general family of jobs rather than one specific job
C) cannot be used to develop evidence of validity based on content
D) usually identifies the specific tasks an individual performs on the job

Question 25

The 2014 Standards for Educational and Psychological Testing recognize five sources of evidence of validity. Which one of the following is NOT one of those sources?

A) evidence based on test content
B) evidence based on face validity
C) evidence based on response processes
D) evidence based on relations to other variables

Question 26

A friend tells you that he learned that a test was valid if it measures what it was designed to measure. Provide an explanation and example of why this isn't always true.

Question 27

The process of competency modeling would most likely be used during test development to gather evidence for validity based on a test's ______.

A) relations with a criteria
B) content
C) response processes
D) constructs

Question 28

When experts meet to rate how essential each test question is to the test, their data can be used to provide evidence of validity by calculating a ______.

A) correlation coefficient
B) content validity ratio
C) mean rating
D) frequency table of ratings

Question 29

The content validity ratio is associated with providing evidence of validity based on ______.

A) response processes
B) test content
C) relations with other variables
D) internal structure

Question 30

Content validity ratios can range from ______.

A) −1 to 1
B) −1 to 0
C) 0 to +1
D) −10 to +10

Question 31

Which of the following is not an acceptable means of determining the TRUE validity of a test?

A) face validity
B) content validity
C) construct validity
D) criterion-related validity

Question 32

What is competency modeling? How is it used to provide evidence of validity?

Question 33

Which one of the following concepts is most controversial among testing experts?

A) content validity
B) concurrent validity
C) face validity
D) construct validity

Question 34

Which of the following is concerned with how test takers perceive the attractiveness and appropriateness of a test?

A) content validity
B) construct validity
C) face validity
D) concurrent validity

Question 35

A behavior validation survey is most likely to be used when developing ______.

A) questions to use on a job interview
B) a personality test
C) an intelligence test
D) an aptitude test

Question 36

Discuss and give examples of the five sources of evidence of validity described in the newest Standards for Educational and Psychological Testing.

Question 37

What is test validity? Define and give examples.

Question 38

Which of the following common job interview questions is most likely to be able to demonstrate evidence for validity based on content?

A) What are your greatest strengths?
B) What is your favorite sport to watch on TV?
C) Tell me about a time when you had to work on a team.
D) Tell me about a time when you really enjoyed a movie.

Question 39

Describe how you would demonstrate evidence for validity for questions used in an employment interview.

Question 40

Define and discuss how evidence of validity based on content is obtained and interpreted. Give examples.

Question 41

Describe a quantitative method for obtaining evidence of validity based on content after the test is developed. Give an example.

Question 42

Define and discuss test specifications. What is their purpose? How are they developed and used? Give an example.