Deck 8: How Do We Gather Evidence of Validity Based on a Tests Relation to Constructs

Question 1

Defining or explaining a psychological construct is referred to as ______.

A) construct validity
B) research that bears out a theory's predictions
C) calculating convergent validity
D) construct explication

Answer 1

D

Question 2

Which one of the following coefficients is NOT found in a multitrait-multimethod correlation matrix?

A) reliability coefficient
B) correlation coefficient between the test and a criterion
C) convergent coefficient
D) discriminant coefficient

Answer 2

B

Question 3

In 1954, the American Psychological Association (APA) published recommendations that suggested a new method for establishing validity in which the researcher provides evidence that the test measures behavior predicted by a psychological theory. What did the APA call this evidence?

A) construct validity
B) factor analysis
C) multitrait, multimethod analysis
D) generalizability

Answer 3

A

Question 4

When a researcher chooses three constructs that are unrelated in theory and three types of tests such as objective, projective, and a peer rating to measure each of the constructs, she is conducting a ______.

A) multitrait-multimethod design
B) content validation study
C) generalizability study
D) factor analysis

Question 5

When an underlying theory explains the relation between a predictor and a criterion based on a psychological theory, there is ______.

A) concurrent evidence of validity
B) predictive evidence of validity
C) evidence of construct validity
D) convergent evidence of validity

Question 6

In 1959, Campbell and Fiske designed a method for collecting evidence of reliability, convergent validity, and discriminant validity in one study. They called it the ______.

A) exploratory method of factor analysis
B) confirmatory method of factor analysis
C) discriminatory method
D) multitrait-multimethod design

Question 7

When the scores on a test correlate strongly with scores on other tests that measure the same construct, we say the test has ______.

A) evidence of validity based on content
B) evidence of validity based on relations with external criteria
C) convergent evidence of validity
D) discriminant evidence of validity

Question 8

If the correlation between social studies test scores and statistics test scores is zero (or not statistically significant), then the social studies test has demonstrated ______.

A) convergent evidence of validity
B) concurrent evidence of validity
C) discriminant evidence of validity
D) evidence of face validity

Question 9

If test scores correlate with measures of constructs that an underlying theory says are related, then we say the test has ______.

A) convergent evidence of validity
B) discriminant evidence of validity
C) evidence of validity based on content
D) concurrent evidence of validity

Question 10

Who defines constructs?

A) the American Psychological Association
B) leading researchers in the field
C) can vary from researcher to researcher
D) must not be allowed to vary

Question 11

How did the APA traditionally define construct validity?

A) the strength of the relationship between a test and a criterion
B) the strength of the relationship between the test and itself
C) the extent to which the test represents all parts of the test domain
D) the extent to which the test measures a theoretically based on phenomenon

Question 12

Which one of the following is an abstract construct?

A) job satisfaction
B) job turnover
C) days absent in a 30-day period
D) number of customer complaints in one week

Question 13

When test users accept evidence of construct validity, they must also accept what?

A) underlying definition of the construct used in the validation process
B) evidence of validity for other tests of similar constructs
C) test as valid for all uses
D) validity of test takers' scores on the test

Question 14

When test scores do not correlate with unrelated constructs, there is ______.

A) convergent evidence of validity
B) discriminant evidence of validity
C) evidence of validity based on relations with external criteria
D) evidence of face validity

Question 15

Which one of the following would be most likely to establish that a test is grounded in psychological theory?

A) the correlation between test score and external criteria
B) the correlation between the test scores of the same people at two different points in time
C) a high coefficient alpha from a test of internal consistency
D) evidence of validity based on the test's relationship with other constructs

Question 16

What did theorists challenge about using validity evidence based on test content and/or validity evidence based on relationships with criteria to establish test validity?

A) They challenged the use of validity based on content because it was not based on statistical analysis.
B) They challenged validity based on relationships with criteria because it did not require reliable tests.
C) They challenged both because the test was not directly being linked to an accepted theory of psychological behavior.
D) They challenged both because they did not rely on accepted ethical standards for testing.

Question 17

We can divide the process of establishing evidence of construct validity into two parts which include ______.

A) collecting data and analyzing the data
B) gathering theoretical evidence and gathering psychometric evidence
C) interviewing study participants and testing those participants
D) proposing a hypothesis and proving that

Question 18

Which one of the following evidences for validity the textbook suggests is most associated with the traditional notion of construct validity?

A) evidence based on test content
B) evidence based on internal structure
C) evidence based on relations with other variables
D) evidence based on response processes

Question 19

When a group of test items all share something in common that accounts for the high correlations among them, we can say they all share a common ______.

A) cause
B) theory
C) nomological network
D) factor

Question 20

Which two validation strategies below help us to establish evidence of construct validity?

A) analysis of variance and multitrait, multimethod analysis
B) convergent evidence of validity and discriminant evidence of validity
C) coefficient alpha and test-retest reliability
D) construct explication and nomological networks

Question 21

Evidence of construct validity is provided when convergent validity correlations are ______.

A) higher than discriminant validity correlations
B) lower than discriminant validity correlations
C) the same as are discriminant validity correlations
D) similar as are discriminant validity correlations

Question 22

Explain and give examples for the three steps of construct explication.

Question 23

What is a factor?

A) the correlations between subscales on a test
B) a concept that may explain why group questions on a test are correlated
C) an environmental force that changes the interpretation of test scores
D) evidence that a test has construct validity

Question 24

Which of the following is a statistical procedure whose purpose is to identify the underlying constructs that contribute to an overall test score?

A) analysis of variance
B) multitrait-multimethod design
C) factor analysis
D) multiple regression

Question 25

What was Cronbach's (1989) advice about construct validation?

A) Validation is a risky process likely to bring the young researcher to ruin.
B) Construct validity is vastly more important than any of the other validities.
C) Validation is a lengthy, even endless process.
D) The test user who spurns validation cannot make valid decisions.

Question 26

A multitrait-multimethod correlation matrix is shown below. What is the correlation of Test of Trait 1/Multiple Choice and Test of Trait 2/Multiple Choice and what is its type?

A) .75; monotrait-monomethod
B) .75; heterotrait-monomethod
C) .89; heterotrait-heteromethod
D) .89; montrait-heteromethod

Question 27

Explain the process of establishing evidence of validity based on a test's relationship with constructs, including the various factors that can provide evidence of that validity.

Question 28

Which one of the following does NOT provide evidence of construct validity?

A) standard error of measurement
B) evidence of validity based on relations with external criteria
C) evidence of validity based on content
D) factor analysis

Question 29

When researchers conduct a factor analysis procedure to see whether the factors they proposed in a test really exist in the test?

A) exploratory factor analysis
B) confirmatory factor analysis
C) discriminant factor analysis
D) construct factor analysis

Question 30

What is a confirmatory factor analysis?

A) a process that provides information on test bias and discrimination
B) a process that researchers use to determine what factors underlie a construct
C) a process of conducting a goodness-of-fit test
D) a process that researchers use to demonstrate that the constructs that theory suggests their test should contain actually exist

Question 31

Discuss the roles of confirmatory and exploratory factor analysis in establishing construct validity. Give an example of a study using each type of factor analysis.

Question 32

Explain how Campbell and Fiske's Multitrait-Multimethod Matrix (MTMM) provides evidence of construct validity. Give examples and draw a MTMM matrix.

Question 33

Discuss the concept of a construct and give examples of theoretical constructs.

Question 34

A multitrait-multimethod correlation matrix is shown below. What is the correlation of Test of Trait 2/Essay with Test of Trait 3 /Interview and what is its type?

A) .08; heterotrait-heteromethod
B) .08; heterotrait-monomethod
C) .40; heterotrait-monomethod
D) .40; heterotrait-heteromethod

Question 35

Which one of the following provides an observable and measurable behavior for the theoretical construct of success as an actor?

A) When the actor feels an emotion, the audience feels the same emotion in the same strength and depth.
B) when the actor believes she has the underlying talent and the ability to express that talent
C) when the actor experiences high levels of stage fright
D) when the actor brings in revenue at the box office under all conditions

Question 36

Discuss the types of psychometric data that can provide evidence of construct validity. Define and give an example of each.

Question 37

When test scores from a new test correlate with test scores from other tests that measure related constructs, they provide ______.

A) concurrent evidence of validity
B) convergent evidence of validity
C) evidence of validity based on test content
D) predictive evidence of validity

Question 38

Which one of the following processes requires researchers to first specify a hypothesis?

A) multitrait-multimethod study
B) content validity study
C) exploratory factor analysis
D) confirmatory factor analysis

Question 39

The statistical procedure that researchers use to identify underlying components that may exist in test without first proposing a formal model?

A) a goodness-of-fit test
B) construct explication
C) exploratory factor analysis
D) confirmatory factor analysis

Question 40

When an underlying theory explains the relation between a predictor and a criterion, there is evidence of ______.

A) construct validity
B) criterion-related validity
C) predictive validity
D) concurrent validity