Question 1

An item analysis splits the "Upper" and "Lower" groups into the top and bottom 50% of cases. Here are the item statistics for four items (21, 22, 23, and 24).
 Which item has the highest p-value?

Accepted Answer

A)  21 
B)  22 
C)  23 
D)  24

Question 2

If you try out 150 items to get the best 20 items for your test using a relatively small sample of cases (N = 30), what process should be used to obtain the true item statistics for the 20 items you have selected?

Accepted Answer

A)  item selection 
B)  cross-validation 
C)  Spearman-Brown procedure 
D)  item analysis

Question 3

What term does the field use for test development procedures that attempt to minimize construct irrelevant variance especially for different subgroups of examinees?

Accepted Answer

A)  CIV design 
B)  for-all design 
C)  universal design 
D)  subgroup-free design

Question 4

Most group administered ability and achievement tests use a _____ response format.

Accepted Answer

A)  true-false 
B)  multiple-choice 
C)  fill-in-the-blank 
D)  essay

Question 5

The first major step in test development is ___.

Accepted Answer

A)  item analysis 
B)  item preparation 
C)  defining the test's purpose 
D)  preliminary design issues

Question 6

Which of these is an advantage of selected-response items over constructed-response items? The selected-response items ___.

Accepted Answer

A)  are less expensive to construct 
B)  are easier to construct 
C)  have better validity 
D)  have better scorer reliability

Question 7

Theta (F1F1F1S1F1F1F10) values usually range from ___.

Accepted Answer

A)  -1.0 to 1.0 
B)  .0 to 1.0 
C)  1.0 to 4.0 
D)  -4.0 to 4.0

Question 8

According to the text, the Binet scales, the Wechsler scales, the MMPI, and the SAT all developed in response to ___.

Accepted Answer

A)  political pressures 
B)  theoretical developments 
C)  economic needs 
D)  practical needs

Question 9

On a given test item, 75% of the students answered correctly, 20% answered incorrectly, and 5% did not answer the item. This item's p-value is -

Accepted Answer

A)  .05 
B)  .20 
C)  .25 
D)  .75

Question 10

In general, test reliability is maximized when item discrimination indexes are -

Accepted Answer

A)  high 
B)  moderate 
C)  low 
D)  a mixture of high and low

Question 11

An easy test will have an average p-value that is ___.

Accepted Answer

A)  from .40 - .60 
B)  high 
C)  low 
D)  equal to the SD

Question 12

As a general rule, the item discrimination index can take on its maximum value when the item p-value is ___.

Accepted Answer

A)  high 
B)  moderate 
C)  low 
D)  negative

Question 13

As used in the world of testing, a portfolio is a ___.

Accepted Answer

A)  collection of a person's work 
B)  set of art prints 
C)  group of essays 
D)  test with little reliability

Question 14

Which of the following is NOT a major advantage of selected-response items?

Accepted Answer

A)  scoring can occur very rapidly 
B)  can complete more items in a given amount of time 
C)  scoring requires little judgment 
D)  opportunity to observe test-taking behavior

Question 15

What are the three phases included within the term item analysis?

Accepted Answer

A)  item tryout, statistical analysis, item selection 
B)  item writing, item tryout, standardization 
C)  standardization, item selection, reliability analysis 
D)  statistical analysis, item review, item writing

Question 16

What appears along the base (i.e., the X-axis or horizontal axis) of the plot of an item characteristic curve?

Accepted Answer

A)  item discrimination 
B)  slope 
C)  theta 
D)  probability of passing

Question 17

The index of discrimination is often symbolized as -

Accepted Answer

A)  D 
B)  DIF 
C)  IOD 
D)  I

Question 18

In _____ scoring, the person scoring an essay rates that essay on several different dimensions.

Accepted Answer

A)  automated 
B)  holistic 
C)  point system 
D)  analytic

Question 19

In the "internal method" of determining item discrimination values, the groups contrasted as having more or less of the trait are defined in terms of ___.

Accepted Answer

A)  total score on the test 
B)  clinical judgment 
C)  score on odd-numbered items 
D)  some other test

Question 20

Which is NOT one of the rules for writing selected-response items?

Accepted Answer

A)  use "all of the above" as an option frequently 
B)  keep the length of options fairly consistent 
C)  place options in logical order 
D)  include the central idea in the stem

Exam 6: Test Development and Item Analysis

An item analysis splits the "Upper" and "Lower" groups into the top and bottom 50% of cases. Here are the item statistics for four items (21, 22, 23, and 24) . Which item has the highest p-value?

If you try out 150 items to get the best 20 items for your test using a relatively small sample of cases (N = 30) , what process should be used to obtain the true item statistics for the 20 items you have selected?