Deck 21: Applications of Item Response Theory: Computer Adaptive Testing and Differential Item Functioning

Question 1

When paper and pencil based tests are transferred to computer without any modification, Shultz, Whitney, and Zickar (2021) refer to them as ___________ tests.

A) computer-adaptive
B) electronically-delivered
C) digitally-administered
D) computer-based

Answer 1

D

Question 2

Computerized adaptive tests are going to be most useful when _____________.

A) item banks are large
B) there are items that span the theta range
C) items in the bank are discriminating
D) all of the above

Answer 2

D

Question 3

Which of the following concepts in computerized adaptive language relates to test security? 

A) item exposure
B) item difficulty
C) item selection randomization
D) item discrimination

Answer 3

A

Question 4

In a computer adaptive test, the first item administered usually should have a __________?

A) Discrimination close to one
B) Difficulty close to zero
C) Difficulty close to -1.0
D) Discrimination closer to zero

Question 5

Which of the following is a reason why some smaller companies may choose not to use computerized adaptive testing?

A) IRT models are not tenable
B) legal defensibility
C) requires a large number of test takers to calibrate
D) test takers may rebel against adaptive tests

Question 6

In IRT, what is the technique used to see if items work the same across different populations?

A) differential item functioning
B) measurement nonequivalence
C) item adaptive functioning
D) Mantel-Hansel

Question 7

Suppose you are conducting a differential item functioning analysis to see if women respond differentially to items that were originally developed on a sample of men. In this example, (using IRT terms) women would be the ________ group.

A) referent
B) baseline
C) comparison
D) focal

Question 8

Which of the following statistical techniques is mentioned by Shultz, Whitney, and Zickar (2021) as a useful technique in understanding item bias?

A) canonical correlation
B) logistic regression
C) exploratory factor analysis
D) discriminant function analysis

Question 9

Shultz, Whitney, and Zickar (2021) mention which of the following as a common problem in differential item functioning studies?

A) Hard to get a large enough sample for the minority group
B) Parameters often vary randomly across populations
C) It is hard to determine which IRT model to use
D) Lack of local independence

Question 10

If the discrimination parameters differ across groups in a differential item functioning (DIF) study, which of the following types of DIF would be exhibited?

A) uniform
B) nonuniform
C) single-group
D) multi-group

Question 11

What are the major advantages of CAT administration over traditional paper-and-pencil test administration?

Question 12

What are key potential challenges with using Computerized Adaptive Testing?

Question 13

In which circumstances, would you recommend Computerized Adaptive Testing to be used?

Question 14

If you are interested in item bias, please explain the various techniques that can be used to understand it.

Question 15

What are strengths and limitations of the IRT-based approach to differential item functioning?