Deck 22: Generalizability Theory

Question 1

Which of the following approaches is best able to quantify multiple sources of error in one analysis?

A) Generalizability Theory
B) Item Response Theory
C) Classical Test Theory
D) Exploratory Factor

Answer 1

A,D

Question 2

Which of the following statistical frameworks is the backbone of Generalizability Theory?

A) Multiple Regression
B) Discriminant Function Analysis
C) Analysis of Variance
D) Exploratory Factor Analysis

Answer 2

C

Question 3

Which of the following Generalizability Theory statistics is used determine which factors are important sources of variance and which factors account for trivial amounts? 

A) Generalizability Coefficient
B) Variance Component Estimate
C) Factor Stability Estimate
D) Multiple R²

Answer 3

B

Question 4

In Generalizability Theory, a(n) ___ design means that measurements have scores on all possible factors?

A) nested
B) orthogonal
C) random
D) crossed

Question 5

Suppose you are conducting a Generalizability Theory study and you want to determine whether type of academic department influences the stability of student evaluations in particular classes? If you choose Finance, Psychology, and Primary Education because those Departments were the ones you were most familiar with, what type of GT Design would that be?

A) Random
B) Cautious
C) Fixed
D) Free

Question 6

In Generalizability Theory, if you wanted to figure out how many interviewers would be needed to achieve a level of reliability, what type of study would you need to conduct?

A) Decision Study
B) Generalizability Study
C) Relative Raters Study
D) Sufficiency Measurement Study

Question 7

In GT notation, when making a decision, if you are choosing based on a particular threshold or cut-off, you are making a(n) ______ decision.

A) relative
B) absolute
C) high-stakes
D) robust

Question 8

Which of the following is recommended by Shultz, Whitney, and Zickar (2021) when conducting a Generalizability Theory analysis?

A) Include a minimum of twenty items
B) Use a minimum of three time points.
C) Measure as many possible sources of variance as possible.
D) Try, if possible, to conduct a relative decisions analysis

Question 9

In Generalizability Theory, if you have a variance component estimate that is extremely small, what does that tell you?

A) That component is not an important source of variance.
B) It would be difficult to generalize across the different levels of that component.
C) You need to collect more data on that component.
D) That component must be fixed.

Question 10

According to Shultz, Whitney, and Zickar (2021) which of the following is a reason why Generalizability Theory studies are infrequently conducted?

A) They require more complex data collections.
B) You need high speed computers to do the analysis.
C) There is still disagreement by researchers on how to interpret the statistics
D) Journal editors are often prejudiced against this methodology.

Question 11

What are the major decisions that need to be done to conduct a Generalizability Theory study?

Question 12

What advantages does Generalizability Theory have over Classical Test Theory?

Question 13

Why has Generalizability Theory been slow to have been adopted?

Question 14

How should Generalizability Theory be used in conjunction with other statistical techniques?