Deck 10: Item Analysis

Question 1

A loosely structured group of statistics that can be computed for each item in a test is a(n)

A) test analysis
B) item confirmation
C) test validation
D) item analysis

Answer 1

D

Question 2

Test items with p values of 0.0 or 1.0 affect the _______ of a test.

A) reliability
B) mean
C) validity
D) internal consistency

Answer 2

B

Question 3

A test question which everyone answers correctly

A) is a poor item because any item that more than 60% of people answer correctly is too easy
B) is a poor item because it has a low reliability
C) is a poor item because it does not discriminate
D) is NOT a poor item because it is fair to everyone

Answer 3

C

Question 4

If an item analysis suggests that an item in a test is poor, and if that item is removed from the test, the reliability of the shorter test is likely to be

A) lower than the original reliability coefficient
B) reliability would not change, but validity would increase
C) higher than the original reliability coefficient
D) none of the above

Question 5

Item Response Theory provides measures that are

A) sample specific
B) sample indifferent
C) sample invariant
D) sample inversed

Question 6

An item with a p value of _______ guarantees that the item will be a good discriminator.

A) 0.5
B) 1.0
C) 0.0
D) none of the above

Question 7

A good test item should be ______ for people who do not know the correct response.

A) easy
B) difficult
C) logical
D) impossible

Question 8

The probability of choosing the correct answer to an item can be represented graphically via

A) item characteristic curves
B) item classification curves
C) item consistency curves
D) item response curves

Question 9

In order to decide whether each applicant is above or below the 30th percentile, you should use a test that consists of items with p values near

A) 0.2
B) 0.3
C) 0.5
D) 1.0

Question 10

The reliability coefficient tells you something about the effects of _______ on test scores, while _______ can tell you why a test is reliable.

A) variability; validity coefficients
B) measurement error; item analysis
C) measurement error; validity coefficients
D) variability; item analysis

Question 11

A distractor that is chosen significantly fewer times than would be expected by chance

A) lowers the difficulty of the item
B) increases the difficulty of the item
C) does not affect item difficulty
D) reflects an item not in the content domain

Question 12

When everyone answers an item incorrectly the p value will be

A) 0.0
B) 1.0
C) between 0.0 and 1.0
D) -1.0

Question 13

A test with either all extremely difficult items or all extremely easy items will show ______ on individual test scores.

A) little variability
B) much variability
C) extreme range
D) large differences

Question 14

The item-total correlation is a statistic that gives information on

A) item difficulty
B) item discrimination
C) item distraction
D) item complexity

Question 15

Item difficulty refers to

A) item complexity
B) item obscurity
C) the number of people who answer an item correctly
D) all of the above

Question 16

A distractor is

A) an incorrect alternative response to a test question
B) a vague, yet correct alternative to a test question
C) an item that serves as a filler item
D) the correct response to a test item

Question 17

Item difficulty analysis shows

A) relatedness of responses on one test to other test items on the test
B) how many people chose each response for an item
C) how many people chose the correct response to an item
D) the proportion of people who chose each response for each item

Question 18

Suppose that for a question, 15% of people selected response "a", 20% selected response "b", 30% selected response "c" (which is the correct response), and 35% selected response "d". Which item analysis data do you have for that item?

A) item difficulty analysis data
B) item discrimination analysis data
C) item distractor analysis data
D) both a and c

Question 19

The D statistic in item analysis is a measure of

A) item difficulty
B) item discrimination
C) item distraction
D) item complexity

Question 20

When the item-total correlation is ____, the slope of the item characteristic curve is _______.

A) negative; positive
B) negative; negative
C) positive; negative
D) positive; zero

Question 21

If an Item Characteristic Curve has a steep upward slope, this indicates that

A) there is a weak relationship between test scores and responses to the particular item illustrated in the graph
B) an item has negative discriminating power
C) people who did well on the test are less likely to do well on this particular item
D) people who did well on the test are much more likely to answer the item correctly

Question 22

Which approach gives you the best opportunity to obtain information by examining which incorrect response was chosen on a test?

A) classical reliability theory
B) true score theory
C) item response theory
D) response dissimulation theory

Question 23

A test is more precise if the difficulty level of the test corresponds with each individual's ability level. Constructing a test for each individual is

A) too time consuming
B) tailoring the test
C) routinely done for all tests
D) all of the above

Question 24

Item Response Theory does all of the following EXCEPT

A) encourages you to think about why people answer items correctly
B) defines difficulty in terms of the level of ability needed to answer the item correctly
C) links item responses to total test scores rather than to the construct the test is designed to measure
D) has become the dominant theme in modern psychometric research