Deck 5: Validity

Question 1

Consider this argument. "This test can't be valid because there were too few females in the standardization program." This argument illustrates confusion between ___.

A) Reliability and validity
B) Reliability and accuracy of norms
C) Validity and accuracy of norms
D) Validity and purpose of the test

Answer 1

C

Question 2

We might think of trait X as having three components: A, B, and C. Our test for trait X measures components B and C but not A. Thus we say that the test shows ________ with respect to component A.

A) unreliable measurement
B) construct under-representation
C) construct over-representation
D) internal consistency

Answer 2

B

Question 3

A test measures, in part, characteristics other than those which we intend it to measure. That part of the measurement is called ___.

A) construct irrelevant variance
B) validity generalization
C) construct underrepresentation
D) criterion contamination

Answer 3

A

Question 4

Which of the following refers to whether a test looks like it measures its target construct?

A) content validity
B) face validity
C) construct validity
D) criterion-related validity

Question 5

Content validity is mostly applicable to ___.

A) intelligence measures
B) objective personality measures
C) educational achievement measures
D) attitude/interest measures

Question 6

The process of developing the list of knowledge and skills required by a job is often called ___.

A) job specification
B) job content
C) job selection
D) job analysis

Question 7

Content validity refers to the relationship between _____.

A) the content of a test and the content of another test of a similar construct
B) the performance on a test and on some other indicator of the construct of interest
C) the performance on a test and membership in a particular group
D) the content of a test and some well-defined domain of knowledge or behavior

Question 8

Instructional validity refers to whether test content _____.

A) covers knowledge and skills required by a particular job
B) fairly matches a well-defined body of content
C) has been adequately taught in a school curriculum
D) matches another indicator of the construct of interest

Question 9

Which of these statements expresses a potential difficulty in establishing content validity?

A) It can be difficult to judge how well test items cover the relevant content.
B) It may be difficult to find a criterion for comparing test performance.
C) Content validity cannot be qualitatively demonstrated.
D) There are no difficulties; establishing content validity is always a simple process.

Question 10

The two main applications of content validity are to _______.

A) intelligence and personality tests
B) intelligence and achievement tests
C) achievement and employment tests
D) employment and attitude tests

Question 11

Which is a well known system for classifying mental processes for purposes of specifying content validity?

A) Freud's theory of the unconscious
B) Hogan's mental chart
C) Spearman's "g" theory
D) Bloom's taxonomy

Question 12

The essential feature of criterion-related validity is establishing the relationship between performance on a test and performance on _____.

A) an unstructured interview
B) the same test after a two week period
C) the same test after five years
D) some other indicator of the construct of interest

Question 13

What statistic would ordinarily be used to show that a college admissions test predicts freshman year GPA?

A) mean
B) standard deviation
C) t-test
D) correlation coefficient

Question 14

When predicting Y from X by means of a regression line, the standard deviation of actual observations (actual Y scores) around the regression line is known as the _______.

A) standard error of measurement
B) standard error of estimate
C) error factor
D) slope

Question 15

The essential difference between concurrent and predictive validity relates to the _____ relationship between test and criterion performance.

A) statistical
B) qualitative
C) linear
D) temporal

Question 16

Establishing the relationship between performance on the test and on some other indicator of the construct of interest is the essential feature of ___.

A) content validity
B) criterion-related validity
C) construct validity
D) face validity

Question 17

Which of the following is NOT a common approach to criterion-related validity? The use of ___.

A) alternate forms of same test
B) some external criterion
C) group contrasts
D) some other valid test

Question 18

Which of the following is NOT a common application of criterion-related validity?

A) use of a realistic criterion defining the construct of interest
B) comparison to another test
C) correlation with test performance at a later date
D) use of group contrasts

Question 19

When utilizing the contrasted groups method for demonstrating criterion-related validity, one wishes to demonstrate that _____.

A) group membership is a poor definition of the criterion
B) the test differentiates one group from another
C) less differentiation between the groups indicates a more valid test
D) statistical significance implies practical difference

Question 20

The correlation between Y and predictors X₁, X₂,...X_k is called a ___.

A) coefficient of multiple correlation
B) factor analysis
C) coefficient of multiple determination
D) coefficient of multiple nondetermination

Question 21

A family of statistical techniques that help identify the common dimensions underlying performance on many different measures is called ___.

A) factor analysis
B) linear regression
C) multiple correlation
D) multitrait-multimethod analysis

Question 22

The technique used to express the relationship between one variable and a combination of two or more other variables is called ___.

A) linear regression
B) factor analysis
C) multitrait-multimethod analysis
D) multiple correlation

Question 23

A crucial technique for determining incremental validity is ___.

A) linear regression
B) multiple correlation
C) factor analysis
D) multitrait-multimethod analysis

Question 24

Factor analysis is a statistical technique that ___.

A) allows us to predict one factor from another
B) creates a correlation matrix
C) identifies variables that go together based on their correlations
D) fits a straight line to data in a bivariate distribution

Question 25

You are trying to predict students' future performance in algebra class. You build your prediction based on an IQ test, teachers' ratings of performance in previous math classes, and a test of motivation to take algebra. This sounds like an ideal situation to use ___.

A) homoscedasticity
B) multiple correlation
C) standard scores
D) z-scores

Question 26

We want to predict success as a salesperson (the criterion) from a combination of a mental ability test, a personality test, and a rating from an interview. This sounds like an ideal opportunity to use ___.

A) analysis of variance
B) factor analysis
C) multiple correlation
D) t-test

Question 27

Which of the following is NOT a reason for establishing the validity of a new test in relation to an old, established test of the same construct?

A) The new test may be shorter or less expensive than the old test.
B) The old test may have poorer norms than the new test.
C) The old test may have better predictive value than the new test.
D) The new test may be a new edition enhanced by computer scoring.

Question 28

The usual method for using several tests to predict status on a criterion is ___.

A) factor analysis
B) partial correlation
C) multiple correlation
D) coefficient alpha

Question 29

An expectancy table is a specific application of ____ validity.

A) content
B) construct
C) consequential
D) criterion-related

Question 30

Consider this statement: The Scranton Anxiety Test and the Scranton Depression Test must be measuring different constructs because one names 'anxiety' and the other names 'depression' as their target constructs." This statement is an example of ___.

A) a self-contradiction
B) the jangle fallacy
C) criterion contamination
D) confusing norms with validity

Question 31

In psychometric language, the term disattenuated means ___.

A) unreliable
B) invalid
C) standard error of measurement
D) corrected for unreliability

Question 32

The validity coefficient for a college admissions test for predicting freshman year GPA is .40. The test's reliability is .80. If we apply the correction for unreliability in this situation, the validity coefficient will ___.

A) decrease
B) increase
C) stay the same
D) become negative

Question 33

The technical term for the limit placed on validity by imperfect reliability is ___.

A) homoscedasticity
B) attenuation
C) heterogeneity
D) nonlinearity

Question 34

When determining the correlation between a test and a criterion, the degree of correlation is affected by ___.

A) reliability of the test
B) reliability of the criterion
C) both a and b
D) neither a nor b

Question 35

Which statement correctly expresses the relationship between the reliability and validity of a test?

A) A test with no reliability can have some validity.
B) A perfectly reliable test always possesses some validity.
C) The imperfect reliability of a test does not affect its validity.
D) The imperfect reliability of a test limits the validity of the test.

Question 36

When establishing criterion-related validity, _____ contamination refers to a situation in which performance on the test influences status on the criterion.

A) criterion
B) performance
C) test
D) validity

Question 37

When criterion contamination occurs, the correlation between test and criterion will ordinarily ___.

A) increase
B) decrease
C) equal zero
D) stay the same

Question 38

A test that shows a high correlation between performance on the test and on some criterion measuring the same construct is said to possess _____ validity.

A) discriminant
B) divergent
C) content
D) convergent

Question 39

We show that our test of anxiety has a low correlation with a test of depression. This low correlation helps to establish the ______ validity of our test.

A) factorial
B) content
C) convergent
D) discriminant

Question 40

A method that compares the validity of tests that purport to measure several different traits using several different methods is called ___.

A) decision theory
B) multitrait-multimethod analysis
C) a multiple correlation
D) cross trait-cross method comparison

Question 41

The multitrait-multimethod matrix is most useful in illustrating ___.

A) predictive and concurrent validity
B) content and construct validity
C) convergent and discriminant validity
D) content and criterion-related validity

Question 42

In the terminology of decision theory as applied to test validity, a case that has the same status on the test as on the criterion is called a ___.

A) hit
B) miss
C) false positive
D) false negative

Question 43

When applying the concepts of hits, false positives, and false negatives, what will be true when the correlation between the test and criterion is perfect (+1.00)?

A) Hits will equal false positives
B) Hits will equal false negatives
C) Hits will equal false positives plus false negatives
D) There will be no false positives

Question 44

When applying the concepts of hits, false positives, and false negatives, what will be true when the correlation between the test and criterion is zero?

A) Hits will equal false positives
B) Hits will equal false negatives
C) Hits will equal false positives plus false negatives
D) There will be no false positives

Question 45

In decision theory, false positives represent a situation in which ___.

A) cases exceed the test cut-score and are successful on the criterion
B) cases exceed the test cut-score but fail on the criterion
C) cases fall below the test cut-score but are successful on the criterion
D) cases fall below the test cut-score and fail on the criterion

Question 46

Cases that exceed the test cut-off score but fail on the criterion are called ___.

A) false positives
B) hits
C) false negatives
D) misses

Question 47

The "base rate" is a special problem when considering which test characteristic?

A) validity
B) reliability
C) norms
D) item analysis

Question 48

When using a test to identify individuals with a particular characteristic, it is easiest to obtain high test validity when the base rate for the characteristic is ___.

A) high
B) moderate
C) low
D) negative

Question 49

Which of the following relationships holds true regarding statistical and clinical predictions?

A) There is no consistent relationship.
B) Clinical predictions usually beat statistical predictions.
C) Clinical judgments are equivalent to statistical predictions.
D) Statistical predictions usually beat clinical predictions.

Question 50

The test's _____ is the extent to which it correctly identifies the criterion group, while the test's _____ is the extent to which it does NOT identify the contrast group.

A) rationale…purpose
B) specificity…sensitivity
C) purpose…rationale
D) sensitivity…specificity

Question 51

A test's specificity refers to the extent to which the test ___.

A) avoids identifying the contrast group
B) correctly identifies the criterion group
C) correctly identifies the contrast group
D) fails to identify the criterion group

Question 52

A test's sensitivity refers to the extent to which the test ___.

A) correctly identifies the criterion group
B) correctly identifies the contrast group
C) avoids identifying the criterion group
D) balances the criterion and contrast groups

Question 53

When applying the concepts of sensitivity and specificity, we would like to have a test that displays ___.

A) low sensitivity and low specificity
B) high sensitivity and high specificity
C) high sensitivity and low specificity
D) low sensitivity and high specificity

Question 54

All of the following are types of evidence that support a test's construct validity EXCEPT ___.

A) internal structure
B) response processes
C) normative data
D) developmental changes

Question 55

What does research show about the degree of use of consequential validity by test authors and publishers?

A) It is basically ignored.
B) It is used with achievement tests but not with other tests.
C) It is used with all types of tests except achievement tests.
D) It is widely used with all types of tests.

Question 56

Judging the relevance of certain studies to a specific anticipated use of a test requires all of the following EXCEPT ___.

A) knowledge of the relevant content area
B) close involvement with test development
C) familiarity with past research with the test and with similar tests
D) understanding of the concepts and procedures involved in determining validity

Question 57

The unitary definition of validity suggests that all validity is really ____ validity.

A) content
B) criterion-related
C) construct
D) face

Question 58

Which type of validity references the test to the implications of its uses and interpretations?

A) face validity
B) incremental validity
C) consequential validity
D) criterion-related validity

Question 59

Suppose the authors of an anxiety test claim that the test (a) accurately measures the degree of anxiety and (b) leads to more effective treatment of the condition. The second claim (b) calls for demonstrating __________ validity.

A) content
B) criterion-related
C) factorial
D) consequential

Question 60

Which of these types of validity evidence emerged most recently in the Standards for Educational and Psychological Testing?

A) content
B) criterion-related
C) construct
D) consequential

Question 61

The process of weighing all the evidence and judging the relevance of existing studies to a specific anticipated use is called ___.

A) construct irrelevant variance
B) validity generalization
C) construct underrepresentation
D) criterion contamination

Question 62

A technique for summarizing the actual statistical information contained in many different studies on a single topic is called ___.

A) multitrait-multimethod analysis
B) factor analysis
C) multiple analysis
D) meta-analysis

Question 63

Consider this statement: "The Scranton Intelligence Scale and the New York Intelligence Scale must be measuring the same construct (intelligence) as indicated by their test titles." This statement is an example of ___.

A) a self-contradiction
B) the jingle fallacy
C) criterion contamination
D) confusing norms with validity

Question 64

You conduct a multiple correlation study, determine R, and the weights for the predictor variables. You then apply the weights in a new sample to see what R will be. This process is known as _____.

A) double weighting
B) cross-validation
C) trial-and-error multiple R
D) double sampling