Question 1

If an instrument measures consistently but does not measure what it was designed to measure, the instrument is:

Accepted Answer

A)  valid but not reliable.
B)  reliable but not valid.
C)  standardized but not reliable.
D)  reliable but not normative. B

Question 2

In examining validation evidence, Messick suggests that clinicians should consider the meaning of the score and the :

Accepted Answer

A)  representativeness of the norming group
B)  social value or consequence of using the instrument
C)  magnitude of the validity coefficients
D)  construct's ability to predict accurately B

Question 3

The most comprehensive type of validity that incorporates a gradual accumulation of evidence is __________ validity.

Accepted Answer

A)  concurrent
B)  content
C)  predictive
D)  construct D

Question 4

A statistical procedure used to determine the generalizability of an instrument's validity is:

Accepted Answer

A)  generalizable analysis.
B)  factor analysis.
C)  regression analysis
D)  meta-analysis.

Question 5

An example of gathering evidence based on response processes is:

Accepted Answer

A)  comparing scores on one assessment to a similar assessment.
B)  asking individual's to "think aloud" while completing an inventory.
C)  looking for patterns of answers within an identified assessment.
D)  having individuals rate their perceived accuracy of a completed assessment.

Question 6

Standard error of measurement is to standard error of estimate as:

Accepted Answer

A)  concurrent validity is to predictive validity
B)  content validity is to face validity
C)  a reliability coefficient is to a validity coefficient
D)  a normal distribution is to a skewed distribution

Question 7

An important parameter in item response theory is the slope of the curve, which is an indicator of how well the item discriminates.If there is a ________ slope then the item is probably a good discriminator ,whereas a ________ slope probably indicates the item does not discriminate very well.

Accepted Answer

A)  flat; steep
B)  symmetrical; asymmetrical
C)  steep; flat
D)  asymmetrical; symmetrical Discussion Questions

Question 8

Using decision theory, both expectancy tables and group separation help to determine:

Accepted Answer

A)  differences between the standardization group and the population of interest
B)  whether an instrument differentiates between groups
C)  whether or not counselors should decide to use an instrument based on the groups represented in the norming sample
D)  whether or not males and females should be separated during group test administration

Question 9

A validity coefficient is best illustrated by the correlation between:

Accepted Answer

A)  test and retest scores.
B)  two different tests.
C)  test scores and performance on a criterion.
D)  an item and the total test score.

Question 10

The difference between concurrent and predictive validity is whether the:

Accepted Answer

A)  prediction is made in the current context or in the future.
B)  focus is on the normative sample or the content of the items.
C)  instrument is designed for adults or children.
D)  evidence is analyzed using regression or analysis of variance.

Question 11

When considering evidence of validity and associated social consequences of instruments counselors should consider all of the following except for:

Accepted Answer

A)  use.
B)  interpretation.
C)  cost effectiveness.
D)  social implications.

Question 12

A low, non-significant correlation between an observation-based assessment of symptoms of autism and a teacher-report of ADHD symptoms provides ______________________ of the validity of the autism measure.

Accepted Answer

A)  convergent evidence
B)  multimethod evidence
C)  discriminant evidence
D)  differential evidence

Question 13

An item difficulty index provides:

Accepted Answer

A)  the proportion of individuals who got an item incorrect
B)  the number of individuals who got an item incorrect
C)  the proportion of individuals who got an item correct
D)  the number of individuals who got an item correct

Question 14

In evaluating an instrument's validity, practitioners should:

Accepted Answer

A)  always select an instrument that has the highest validity coefficients.
B)  be convinced by the preponderance of evidence on the appropriate use of that instrument.
C)  not use an instrument unless a factor analysis has been conducted.
D)  select those instruments that have a multitrait-multimethod matrix.

Question 15

If an instructor in an appraisal in counseling course was interested if an exam question was too easy or too difficult, she would want to examine the ________ of that question.

Accepted Answer

A)  internal consistency
B)  item difficulty
C)  item discrimination
D)  item variance

Question 16

The standard error of estimate is an index of the:

Accepted Answer

A)  accuracy of the instrument scores.
B)  content validity of a test.
C)  reliability of an instrument.
D)  probable range of scores on the criterion.

Question 17

A high correlation between a self-report inventory that measures depression and a depression measure completed by the therapist is an example of:

Accepted Answer

A)  convergent evidence
B)  discriminant evidence
C)  reliability
D)  content-related evidence

Question 18

In appraisal, regression equations are most frequently used to:

Accepted Answer

A)  predict test-takers' performance on the criterion.
B)  compute test-retest reliability.
C)  measure developmental changes.
D)  analyze the makeup of the norming group.

Question 19

The difference between the percentage of examinees in the upper group who answer an item correctly and the percentage of examinees in the lower group who answer the item correctly is called the ____________ index.

Accepted Answer

A)  internal consistency
B)  item difficulty
C)  item reliability
D)  item discrimination

Question 20

The higher the item difficulty index , the ________ the item.

Accepted Answer

A)  easier
B)  harder
C)  more reliable
D)  less reliable

Exam 4: Validity and Item Analysis