Deck 3: Reliability

Question 1

What statistical technique is often used to calculate an estimate of reliability?

A)regression
B)analysis of variance
C)factor analysis
D)correlation

Answer 1

correlation

Question 2

If we interpreted a reliability coefficient to indicate that 80% of the variance was true variance to observed variance, what would be the ratio of error variance to observed variance?

A).20
B).40
C).60
D).80

Answer 2

.20

Question 3

A reliability coefficient provides a measure of:

A)systematic error
B)unsystematic error
C)both systematic and unsystematic error
D)the amount of systematic error in each score

Answer 3

unsystematic error

Question 4

According to classical test theory, if the observed variance of a test is 50 and the true variance is 40, what is the estimated reliability of the test?

A).40
B).50
C).80
D).90

Question 5

The statistic that represents the percentage of shared variance between two variables is the:

A)correlation coefficient
B)reliability correlation
C)coefficient of determination
D)coefficient of shared variance

Question 6

For which of the following tests would a test-retest reliability estimate be least appropriate?

A)intelligence tests
B)tests of moment-to-moment mood changes
C)academic achievement tests
D)instruments to measure art aptitude

Question 7

The alternate form method of estimating test reliability, as contrasted to the test-retest method, tends to lessen the influence of:

A)content sampling
B)item variance
C)similar items
D)memory and practice effects

Question 8

With a reliability coefficient of .90, the percentage of total variance of test scores attributable to unsystematic error is:

A)practically zero
B)10
C)81
D)90

Question 9

With a reliability coefficient of .81, using classical test theory we would interpret that the:

A)amount of error variance to observed variance is 81 percent
B)amount of true variance to observed variance is 81 percent
C)instrument has good enough reliability
D)instrument's validity coefficient would be .81²

Question 10

If the correlation between interest in statistics and being a "fun date " was -.70, it would mean that:

A)the higher someone's interest was in statistics, the more likely it would be that he or she is a fun date.
B)the higher someone's interest was in statistics, the less likely it would be that he or she is a fun date.
C)there is a 70% chance that someone with an interest in statistics is a fun date.
D)there is a 30% chance that someone with an interest in statistics is a fun date.

Question 11

Which of the following is NOT one of the assumptions that should be met when establishing the reliability of an instrument using the test-retest method?

A)Test-retest reliability is only valid when measuring situational traits
B)The characteristic or trait measured should be stable over time
C)There should be no differential in practice effect
D)There should be no differential in learning between the test and retest

Question 12

The correlation between IQ scores and grade point average in college is .40.What percent of the variance is explained by this relationship?

A)16
B)40
C)60
D)80

Question 13

According to classical test theory, which of the following statements would be the most accurate interpretation of Obs.= T + E?

A)Observable behavior equals testing conditions plus experimenter influence
B)Observations equal test anxiety plus examiner expectations
C)Observed score equals true score plus error
D)None of the above

Question 14

If there is no evidence of a relationship between two groups of test scores, the correlation between them will be closest to:

A)-1.00
B).00
C).50
D)1.00

Question 15

The most significant difficulty with estimating reliability with the alternate or parallel forms procedure is:

A)calculating the correlation coefficient using two different forms.
B)the effects of remembering specific items in the second testing.
C)developing two sound instruments that are equivalent or parallel.
D)finding two similar sets of test takers.

Question 16

Correlation coefficients range from:

A)-1.0 to 1.0
B)0 to 1.0
C)0 to -1.0
D)-5.0 to 5.0

Question 17

A correlation coefficient is an indicator of:

A)the validity of an assessment
B)the variability of the obtained scores
C)the relationship between two sets of data
D)the fluctuation of an individual's score over time

Question 18

Which of the following correlation coefficients shows the strongest relationship

A).51
B).70
C)-.85
D)-.50

Question 19

When calculating correlations, the most common method used is the:

A)Pearson-Product Moment Correlation Coefficient
B)Correlation Coefficient of Most Common Factors
C)Reliability Coefficient of Correlation
D)Reliability Correlation Coefficient

Question 20

Systematic error (as compared to unsystematic error):

A)significantly lowers the reliability of an instrument.
B)insignificantly lowers the reliability of an instrument.
C)increases the reliability of an instrument.
D)has no effect on the reliability of an instrument.

Question 21

Jennifer took an aptitude test and is interested in whether the score on the verbal aptitude subscale is significantly higher than her mathematical aptitude subscale score.What statistic would you be interested in to answer her question?

A)standard error of difference
B)standard error of estimate
C)standard error of measurement
D)standard deviation

Question 22

The Spearman-Brown formula is used:

A)to correct a test-retest reliability coefficient.
B)to correct a split-half reliability coefficient.
C)when the items are all of the same difficulty level.
D)when the items are of differing levels of difficulty.

Question 23

The ABC Self-Concept Inventory has a split-half reliability coefficient of .90, and the XYZ Self-Concept Inventory has a Spearman-Brown reliability coefficient of .90.If all other factors are equal, which instrument would you choose to use?

A)ABC
B)XYZ
C)either one because they are equal
D)neither one

Question 24

In generalizability or domain sampling theory, the focus is on:

A)using measures of internal consistency.
B)determining the standard error of measurement.
C)identifying specific sources of variation under defined conditions.
D)identifying where an individual's true score would fall.

Question 25

Which of the following is appropriate for determining the reliability of a criterion-referenced instrument?

A)test-retest
B)Kuder-Richardson
C)Spearman-Brown
D)none of the above due to the nature of criterion-referenced instruments

Question 26

As the reliability of an instrument increases, the standard error of measurement _______.

A)decreases
B)increases
C)could either increase or decrease
D)is unaffected

Question 27

In evaluating an instrument's reliability, a counselor should:

A)always select the instrument with the highest reliability coefficients.
B)select instruments where coefficient alphas have been calculated.
C)consider how the instrument is going to be used.
D)never use an instrument where the reliability coefficient is less than .92.

Question 28

If an instrument requires some professional judgments in scoring, then the manual should also include information on:

A)interrater reliability
B)corrections of the reliability coefficients using the Spearman-Brown formula
C)both KR 20s and KR 21s
D)test-retest reliability coefficients

Question 29

The reliability of Test LMN was estimated by three methods: 1) Spearman-Brown, 2) test-retest, and 3) coefficient alpha.Which method probably yielded the lowest reliability coefficient?

A)coefficient alpha
B)test-retest
C)Spearman-Brown
D)the reliability coefficients will all be equal

Question 30

Measurement experts generally suggest that counselors should use ________ in interpreting a client's test score results.

A)stanines
B)standard error measurement
C)variance
D)validity generalization

Question 31

Joe had a score of 72 on the Counseling Aptitude Scale, and the standard error of measurement of the scale is 3.Where would we expect Tom's true score to fall 99.5 percent of the time?

A)71 to 73
B)69 to 75
C)66 to 78
D)63 to 81

Question 32

Standard error of measurement is designed to:

A)tell the clinician if an instrument is reliable.
B)provide an estimate of the probable range of scores for an individual.
C)indicate the percentage of error in the reliability coefficient.
D)determine the statistical significance of the reliability coefficient.

Question 33

The decision to use either the Kuder-Richardson 20 (KR 20) or the Kuder-Richardson 21 (KR 21) is based on whether:

A)the method used to determine reliability is the test-retest or the alternate form method.
B)correlation or regression is going to be used.
C)the items measure a homogeneous or heterogeneous behavior domain.
D)the item format is multiple-choice or true-false.

Question 34

If the reliability coefficient of an instrument is .91, and the standard deviation is 10, then a client's score of 59 could be interpreted that 95 percent of the time his/her true score will fall between _____ and _____ using standard error of measurement

A)56 and 62
B)53 and 65
C)49 and 69
D)-3.00 and 3.00

Question 35

The standard error of measurement is best used for _______ and the reliability coefficient is best used for _______.

A)scores that have a large range; unique circumstances where most scores center around the mean
B)interpreting individual scores; comparing different instruments
C)communicating with other professional counselors; communicating with clients
D)personality and mood inventories; intelligence tests

Question 36

What is the most appropriate way to determine reliability with Likert scales, where different answers receive different weightings?

A)test-retest
B)Spearman-Brown
C)Kuder-Richardson 20
D)coefficient alpha (or Cronbach's Alpha)

Question 37

A different reliability model from the "true score" or "classical" model is the:

A)internal consistency model.
B)performance evaluation model.
C)standard error of measurement model.
D)generalizability or domain sampling model.

Question 38

The standard deviation of a particular subtest of the WISC-IV is 3, and the reliability coefficient is .84.What is the standard error of measurement of that subtest?

A).48
B)1.20
C)2.75
D)3.00

Question 39

In general, the reliability coefficients for instruments designed for infants and young children are _____________ those of instruments designed for adolescents and adults.

A)lower than
B)higher than
C)comparable to
D)exactly the same as

Question 40

According to the Standards for Educational and Psychological Testing, which statistic should be used for the interpretation of group test scores?

A)Standard error of measurement
B)Correlation coefficient
C)Cronbach's alpha
D)Standard error of the observed score means

Question 41

When is name of the meta-analytic method, developed by Vacha-Haase (1998), that explores variability in reliability estimates across studies?

A)universal reliability
B)cross-validation reliability
C)generalizability estimation
D)reliability generalization

Question 42

Reliability is the precursor to:

A)determining the coefficient alpha of an assessment instrument.
B)validity.
C)sharing with clients their scores.
D)fully understanding the utility of an assessment instrument.
Discussion Questions