Deck 4: Reliability

Question 1

Alternate form reliability based on simultaneous administration is primarily sensitive to measurement error due to:

A)content sampling.
B)content sampling and temporal instability.
C)practice effects.
D)temporal instability.

Answer 1

content sampling.

Question 2

Test-retest reliability is primarily sensitive to measurement error due to:

A)content sampling.
B)content sampling and temporal instability.
C)factor invariance.
D)temporal instability.

Answer 2

temporal instability.

Question 3

If 6% of test scores' observed variance is due to measurement error,the reliability coefficient of the test would be:

A).06.
B).36.
C).60.
D).94.

Answer 3

.94.

Question 4

The reliability of composite scores is generally ________ the reliability of the individual scores that contributed to the composite.

A)equal to
B)higher than
C)lower than

Question 5

As a general rule,_________ tests produce more reliable scores than ______ tests.

A)brief; lengthy
B)intensive; extensive
C)longer; shorter
D)shorter; longer

Question 6

The uncorrected split-half reliability coefficient __________ the reliability of the full test score.

A)accurately reflects
B)overestimates
C)underestimates
D)provides an indeterminate reflection of

Question 7

The mean of error scores in a population is equal to ________.

A)0.1
B)0
C)1
D)10

Question 8

The reliability coefficient (r_xx)equals true score variance ( $\sigma$ ²_T)divided by the:

A)observed score.
B)measurement error .
C)variance due to measurement error.
D)variance of the total test.

Question 9

Alternate form reliability based on delayed administration is sensitive to measurement error due to:

A)content sampling.
B)content sampling and temporal instability.
C)practice effects.
D)temporal instability.

Question 10

Classical Test Theory focuses our attention on ________ measurement error.

A)random
B)variable
C)standard
D)systematic

Question 11

__________ is sensitive to the heterogeneity of the test content.

A)Alternate-from reliability with delayed administration
B)Coefficient Alpha
C)Split-half reliability
D)Test-retest reliability

Question 12

On a classroom essay test,__________ is a major concern.

A)inter-rater reliability
B)internal consistency reliability
C)split-half reliability
D)test-retest reliability

Question 13

What conclusion could be accurately drawn from a reliability coefficient of .80?

A)18% of test score variance is due to true score variance.
B)20% of test score variance is due to true score variance.
C)64% of test score variance is due to true score variance.
D)80% of test score variance is due to true score variance.

Question 14

There is (a)_______ relationship between an individual's level on a construct and the amount of measurement error impacting their observed score.

A)no
B)weak
C)moderate
D)strong

Question 15

In Classical Test Theory,the X represents _______ and the T represents ________.

A)measurement error,observed score
B)observed score; stable test-taker characteristics
C)observed score; measurement error
D)stable test-taker characteristics; observed score

Question 16

_________ is applicable when test items are scored dichotomously while _________ can be used when test items produce multiple values.

A)Coefficient Alpha; KR 20
B)KR 20; Coefficient Alpha
C)Split-half reliability; test-retest reliability
D)Test-retest reliability; split-half reliability

Question 17

Split-half reliability is primarily sensitive to measurement error due to:

A)content sampling.
B)content sampling and temporal instability.
C)practice effects.
D)temporal instability.

Question 18

_________ refers to the consistency or stability of test scores.

A)Measurement error
B)Reliability
C)Variance
D)Validity

Question 19

________ is/are usually considered the largest source of error in test scores.

A)Administrative errors
B)Clerical errors
C)Content sampling
D)Time sampling

Question 20

On a test comprised of constructed response items,it is important to consider:

A)administrative errors.
B)clerical errors.
C)inter-rater differences.
D)time sampling.

Question 21

As reliability increases,confidence intervals:

A)decrease.
B)do not change.
C)increase.

Question 22

If the reliability coefficient equals .81,the reliability index equals:

A).19.
B).81.
C).90.
D)1.0.

Question 23

The average of all possible split-half coefficients is known as:

A)Coefficient alpha.
B)correlation coefficient.
C)alternate form reliability.
D)Spearman-Brown coefficient.

Question 24

The reliability index reflects the correlation between:

A)true scores and observed scores.
B)true scores and measurement error.
C)observed scores and measurement error.
D)true scores and true scores.

Question 25

Which of the following methods is necessary when estimating the reliability of a test score intended to predict performance at a future time?

A)Alternate form reliability with simultaneous administration
B)Coefficient alpha
C)KR 20
D)Test-rest reliability

Question 26

The Spearman-Brown formula is used to:

A)correct a split-half reliability coefficient.
B)estimate construct reliability.
C)perform a curvilinear transformation of the scores.
D)perform a linear transformation of the scores.

Question 27

The ____________ is an index of the amount of measurement error in test scores and is used in calculating confidence intervals.

A)Standard Error of Estimate
B)Standard Error of Measurement
C)Spearman-Brown Coefficient
D)Skew Coefficient

Question 28

______________________ is a useful index when comparing the reliability of the scores produced by different tests,but when the focus is on interpreting the test scores of individuals,the ________________________ is more practical.

A)Reliability Coefficient; Standard Error of Measurement
B)Standard Error of Measurement; Reliability Coefficient
C)Standard Error of Estimate; Coefficient Alpha
D)Standard Error of Estimate; Reliability Coefficient

Question 29

_________ is a result of transient events in the test taker (fatigue,illness,etc.)and the testing environment (temperature,noise level,etc.).

A)Administration error
B)Content sampling error
C)Temporal instability
D)Systematic measurement error

Question 30

As the reliability of a test score_______ the standard error of measurement _______.

A)decreases; increases
B)decreases; decreases
C)increases; remains the same
D)decreases; remains the same

Question 31

Reliability coefficients based on a homogeneous sample would likely be ________ coefficients based on a heterogeneous sample.

A)equal to
B)larger than
C)smaller than

Question 32

In Item Response Theory,information on the reliability of test scores is typically reported as a:

A)Test Information Function
B)Standard Error of Estimate
C)Skew Coefficient
D)Coefficient of Determination
E)Coefficient of Non-determination

Question 33

Which of the following is a measure of inter-rater agreement that takes into consideration the degree of agreement expected by chance?

A)KR 20
B)Strong-Campbell Beta
C)Cronbach's Coefficient alpha
D)Cohen's kappa

Question 34

Which reliability estimate would be preferred for a score derived from a test with heterogeneous content?

A)Coefficient Aplha
B)KR 20
C)Split-Half Coefficient

Question 35

What happens to the size of confidence intervals as reliability coefficients increase?

A)They decrease.
B)They increase.
C)They remain the same.
D)It is indeterminate - it depends on the construct being measured.

Question 36

A limitation of the test-retest approach to estimating reliability is the influence of:

A)administration effects.
B)content effects.
C)practice effects.
D)temporal effects.

Question 37

Generalizability Theory typically uses which statistical procedure to estimate reliability?

A)Analysis of Variance (ANOVA)
B)Correlation Coefficient
C)Linear Regression
D)Multivariate Analysis of Vvariance (MANOVA)

Question 38

Sally's obtained scored on a statistics exam is 75.The SEM is 2.With what confidence interval would we capture her true score 68% of the time?

A)71 to 79
B)73 to 77
C)69 to 81
D)70 to 80

Question 39

Which method of rating reliability would be appropriate for scores from a speed test?

A)Coefficient Alpha
B)Kuder Richardson 20
C)Test-retest reliability
D)Split-half reliability

Question 40

The reliability of difference scores is typically _______ the reliability of the individual scores.

A)equal to
B)higher than
C)lower than