Deck 4: Validity and Test Development

Question 1

A test has______ validity if it looks valid to test users, examiners, and especially the examinees.

A) content
B) face
C) criterion-related
D) construct

Answer 1

face

Question 2

Most likely, the regression equation Y = .07X + .2 describes

A) a smooth curve
B) a best-fitting straight line
C) estimated IQ
D) the band of error

Answer 2

B

Question 3

Correlations between a new test and existing tests are often cited as evidence of______ validity.

A) content
B) concurrent
C) predictive
D) construct

Answer 3

concurrent

Question 4

The quantification or "common sense" approach to content validity advocated by the text

A) can identify non-existent items that should be added to a test to increase validity
B) helps cull out existing items that are deemed inappropriate by expert raters
C) provides an acceptable substitute for a careful selection of items
D) all the above

Question 5

Traditionally, the different ways of accumulating validity evidence have been grouped into three categories:

A) content, criterion-related, and construct validity
B) face, criterion-related, and construct validity
C) face, content, and construct validity
D) content, concurrent, and predictive validity

Question 6

______validity is particularly relevant for entrance examinations and employment tests.

A) Content
B) Concurrent
C) Predictive
D) Construct

Question 7

What is the relationship between the reliability and the validity of a psychological test?

A) none; these concepts are separate and independent
B) reliability is necessary but not sufficient for validity
C) reliability is necessary and sufficient for validity
D) validity is a necessary precursor to reliability

Question 8

When tests are used for purposes of_______ , it is necessary to develop a regression equation.

A) predicting outcomes
B) validating a theory
C) content validation
D) estimating reliability

Question 9

A test is valid when the inferences made from it are

A) appropriate
B) meaningful
C) useful
D) all of the above

Question 10

The degree to which items on a test are representative of the universe of behavior the test was designed to sample is an index of______ validity.

A) content
B) concurrent
C) predictive
D) construct

Question 11

The only reason for building in______ validity involves public relations.

A) content
B) face
C) criterion-related
D) construct

Question 12

An) _______is any outcome measure against which a test is validated.

A) response
B) behavior
C) criterion
D) index variable

Question 13

In test validation studies, criterion measures should incorporate items that are ________to the test items.

A) highly similar
B) identical
C) completely unrelated
D) none of the above

Question 14

The validity of a test is typically summed up by

A) a single, tidy statistic
B) multiple neat statistical summaries
C) a continuum ranging from weak to strong
D) none of the above

Question 15

The theoretical upper limit of the validity coefficient is constrained by the reliability of

A) the test
B) the criterion
C) both the test and the criterion
D) neither the test nor the criterion

Question 16

The higher the ________, the more accurate the test is in predicting the criterion.

A) validity coefficient
B) reliability coefficient
C) validity curve
D) reliability curve

Question 17

In________ validity, test scores are used to estimate outcome measures obtained at a later date.

A) content
B) concurrent
C) predictive
D) construct

Question 18

When are expert judges needed to determine the content validity of a test?

A) always
B) when the trait being measured is ill-defined
C) when the trait being measured is highly technical
D) all of the above

Question 19

When similar or identical questions are found on both a test measure and a criterion outcome) measure related to the test, what source of error is likely?

A) criterion contamination
B) criterion default
C) criterion dissolution
D) criterion replication

Question 20

Content validity can be quantified if

A) the test developer has access to thousands of subjects
B) two or more judges assess the relevance of all test items
C) the test developer performs a factor analysis on the response matrix
D) all of the above

Question 21

In the multi-trait multi-method matrix, the correlations down the main diagonal are _______coefficients.

A) reliability
B) validity
C) error
D) regression

Question 22

The goal in factor analysis is to find a smaller set of dimensions, called ________, that can account for the observed array of intercorrelations among individual tests.

A) loadings
B) factors
C) rotations
D) iterations

Question 23

Suppose a new measure of extraversion possesses a negligible correlation with intelligence. This would indicate support for_______ validity.

A) convergent
B) discriminant
C) content
D) concurrent

Question 24

Proponents of decision theory make the following fundamental assumptions):

A) the value of various outcomes to the institution can be expressed in terms of a common scale
B) the most generally useful strategy is one which maximizes the average gain or minimizes average loss
C) both a and b
D) neither a nor b

Question 25

____________has to do with accurate identification of patients with a syndrome.

A) Sensitivity
B) Specificity
C) Reliability
D) Loading

Question 26

A multi-trait multi-method matrix provides information about

A) reliability
B) convergent validity
C) discriminant validity
D) all of the above

Question 27

"Interest in and concern for others" defines

A) altruism
B) sociability
C) extraversion
D) social interest

Question 28

The standard error of the estimate is an index of the error of measurement caused by the _______of a test.

A) unreliability
B) imperfect validity
C) sampling restriction
D) standardization sample

Question 29

Suppose the standard error of the estimate for predicted grade point average GPA) is 0.2 grade units. Suppose a student has a predicted GPA of 2.90. Approximately what is the probability that the student will achieve a GPA of 3.3 or higher?

A) .5 percent
B) 2.5 percent
C) 10 percent
D) cannot be computed

Question 30

If a test measures a single construct, then its component items or subtests) likely will be_______ .

A) divergent
B) internally valid
C) heterogeneous
D) homogeneous

Question 31

Factor loadings can vary between

A) 0 and 100
B) -1.0 and +1.0
C) 0.0 and +1.0
D) none of the above

Question 32

Many psychometric theorists regard_______ validity as the unifying concept for all types of validity evidence.

A) content
B) concurrent
C) predictive
D) construct

Question 33

A factor loading is actually(an)_____

A) beta weight
B) correlation
C) eigenvalue
D) all of the above

Question 34

A construct is

A) a theoretical, intangible quality or trait in which individuals differ
B) any variable which permits precise measurement
C) essentially synonymous with a concept
D) all of the above

Question 35

The construct validity of a vocabulary test would be buttressed if scores _______with increasing age.

A) increased
B) decreased
C) remained stable
D) increased and then decreased

Question 36

The validity of a screening test is bolstered to the extent it possesses______ sensitivity and ________specificity.

A) low; low
B) low; high
C) high; low
D) high; high

Question 37

A construct possesses the following characteristics):

A) it cannot be operationally defined
B) a network of predictions can be derived from theory about the construct
C) both a and b
D) neither a nor b

Question 38

In decision theory, persons who are predicted to succeed but who fail are known as

A) False positives
B) False negatives
C) true positives
D) true negatives

Question 39

Internal consistency is roughly synonymous with

A) trait stability
B) homogeneity
C) item-total independence
D) all of the above

Question 40

Formula for the hit rate is

A) misses)/hits + misses)
B) hits)/hits + misses)
C) hits + misses)/misses)
D) hits + misses)/hits)

Question 41

Which of the following scaling methods is considered the "gold standard"?

A) expert rankings
B) Likert scale
C) Guttman scale
D) none of the above

Question 42

What concept is best summed up by the question, "Does use of this test result in better patient outcomes or more efficient delivery of services?"

A) test functionality
B) test utility
C) test generalizability
D) test efficacy

Question 43

A(n) _______scale has a conceptually meaningful zero point.

A) nominal
B) ordinal
C) interval
D) ratio

Question 44

Suppose a test developer has hired a panel of experts to determine the suitability of certain test items for the concept being measured. Most likely, the test developer is concerned with

A) content validity
B) criterion-related validity
C) construct validity
D) reliability

Question 45

An individual takes a new test for a medical disease, and the results indicate that she does not have the disease. In fact, it turns out that she did have the disease. This is an example of a

A) False positive
B) False negative
C) true positive
D) true negative

Question 46

The more powerful and useful statistics should only be used with _______levels of measurement.

A) nominal and ordinal
B) ordinal and interval
C) interval and ratio
D) ordinal and ratio

Question 47

The first step in test construction is

A) defining the test
B) selecting a scaling method
C) constructing the items
D) testing the items

Question 48

What best describes the relationship between sensitivity and specificity?

A) inverse
B) corresponding
C) exponential
D) indiscriminate

Question 49

The method of _______involves ratings of favorability by expert judges.

A) factor analysis
B) expert rankings
C) absolute scaling
D) equal-appearing intervals

Question 50

A form of ranking is found in _______scales.

A) nominal
B) ordinal
C) interval
D) ratio

Question 51

In a ________validation study, test scores and criterion information are obtained simultaneously.

A) content
B) predictive
C) construct
D) concurrent

Question 52

In testing, extra-validity concerns refer to

A) unintended side effects of testing
B) long-term validity effects
C) content, criterion-related, and construct validity
D) none of the above

Question 53

A homogeneous scale is also referred to as

A) internally consistent
B) externally consistent
C) internally discriminant
D) externally discriminant

Question 54

The Glasgow Coma Scale was developed by the method of

A) factor analysis
B) expert rankings
C) absolute scaling
D) equal-appearing intervals

Question 55

Which is the correct order for levels of measurement?

A) ordinal, nominal, interval, ratio
B) ordinal, nominal, ratio, interval
C) nominal, ordinal, ratio, interval
D) nominal, ordinal, interval, ratio

Question 56

According to the functionalist perspective on test validity, a test is valid if

A) inferences made from it are meaningful and appropriate
B) the test serves the purpose for which it is used
C) its screening function is proved effective
D) none of the above

Question 57

The method of_______ equal-appearing intervals was developed by

A) Thurstone
B) Likert
C) Guttman
D) Binet

Question 58

Which of the below is a primary goal of the Kaufman Assessment Battery for Children?

A) to measure intelligence from a strong theoretical and research basis
B) to separate acquired factual knowledge from the ability to solve unfamiliar problems
C) to yield scores that translate to educational intervention
D) all of the above

Question 59

In _______scales, the numbers serve only as category names.

A) nominal
B) ordinal
C) interval
D) ratio

Question 60

Which level of measurement is most commonly employed in current psychological testing instruments?

A) nominal
B) ordinal
C) interval
D) ratio

Question 61

For individually administered tests, the procedure of choice is the

A) short answer objective item
B) multiple-choice question
C) open-ended question
D) essay

Question 62

To help insure that a test contains the desired mixture of cognitive processes and content domains, test developers

A) follow the guidelines specified by Binet
B) use factor analysis on the test items
C) use a table of specifications
D) always consult similar instruments before writing test items

Question 63

The forced-choice methodology is designed to counter the problem of

A) guessing
B) social desirability
C) distractor difficulty
D) all of the above

Question 64

In the method of ________, a set of common test items is administered to two or more age groups.

A) factor analysis
B) expert rankings
C) absolute scaling
D) equal-appearing intervals

Question 65

For true-False items, the optimal level of item difficulty is

A) .5
B) .75
C) 1.0
D) 100

Question 66

In general, the optimal level of item difficulty is

A) 0
B) .5
C) 1.0
D) 100

Question 67

If an item such as "I drink a lot of water" was on a depression scale, the test likely employed the_______ scaling method

A) empirical keying
B) Likert
C) absolute
D) rational

Question 68

It is common practice in test development that the prepublication version of a new instrument might contain ________the number of items desired on the final draft.

A) fewer than
B) double
C) five times
D) ten times

Question 69

On a_______ scale, respondents who endorse one statement also agree with milder statements pertinent to the same underlying continuum.

A) Thurstone
B) Likert
C) Guttman
D) Binet

Question 70

From a psychometric standpoint,________ questions are the weakest.

A) multiple-choice
B) matching
C) true-False
D) fill in the blank

Question 71

The proportion of examinees in a large tryout sample who get a specific item correct is called

A) item-difficulty index
B) item-reliability index
C) item-validity index
D) index of item discrimination

Question 72

The following items are probably an example of a______ scale: )
I occasionally feel sad or blue.
I often feel sad or blue.
I feel sad or blue most of the time.
I always feel sad and I can't stand it.

A) Thurstone
B) Likert
C) Guttman
D) Binet

Question 73

For group-administered tests of intellect or achievement, the technique of choice is the

A) short-answer essay
B) true-False question
C) multiple choice question
D) method of matching

Question 74

Which scale construction method guarantees that all scale items correlate positively with each other and also with the total score for the scale?

A) empirical keying
B) equal-appearing intervals
C) rational scale construction
D) expert rankings

Question 75

An) __________effect is observed when significant numbers of examinees obtain perfect or near-perfect scores.

A) floor
B) ceiling
C) interval
D) absolute

Question 76

The method of _______is entirely atheoretical.

A) empirical keying
B) equal-appearing intervals
C) absolute scaling
D) expert rankings

Question 77

A ________scale presents the examinee with five responses ordered on an agree/disagree or approve/disapprove continuum.

A) Thurstone
B) Likert
C) Guttman
D) Binet

Question 78

A test used to designate children for a remedial-education program should contain many________ items.

A) extremely easy
B) average difficulty
C) moderately difficult
D) extremely difficult

Question 79

A Likert scale is also referred to as a ________.

A) rational scale
B) summative scale
C) classification scale
D) ratio scale

Question 80

The index of item difficulty can vary from

A) 0 to 100
B) -1.0 to +1.0
C) 0.0 to +1.0
D) 0.5 to +1.0