Deck 5: Reliability

Question 1

The SEM for an achievement test is 2.45.Jonny scores 100 and we assume that 68% of the time his true score falls between ±1 SEM.This means the confidence interval would be between:

A)0 and 102.45
B)2.45 and 100
C)95.10 and 104.90
D)97.55 and 102.45

Answer 1

97.55 and 102.45

Question 2

The test manual reports a reliability coefficient r)of .92,which means:

A)92% of the variance in scores is attributed to error
B)92% of the variance in scores is explained by real differences
C)nothing conclusively because it depends on the method used
D)nothing conclusively because it depends on the source of measurement error

Answer 2

B

Question 3

You are attempting to account for time sampling error and decide to administer the test a second time.In discussing reliability,you report this as what method of estimating reliability?

A)Alternate forms
B)Test-retest
C)Split-half reliability
D)Internal consistency reliability

Answer 3

B

Question 4

Keisha receives a 79 on the test.This is her:

A)true score
B)reported score
C)observed score
D)predicted score

Question 5

In reviewing a newly developed test instrument,the evaluator noticed that some of the items did not appear to reflect the construct being measured.He reported there was:

A)content-sampling error
B)time-sampling error
C)interrater differences error
D)test-taker variables error

Question 6

Jose has developed a test that has poor reliability.He can seek to increase reliability by:

A)increasing the number of test questions
B)decreasing the number of test questions
C)making the test questions more ambiguous
D)starting over and developing a new test

Question 7

A test was administered to a group of students the morning after homecoming.Several of the students appeared tired and some were coughing and sneezing.These factors may result in what type of error:

A)time-sampling error
B)test administration
C)quality of test items
D)test-taker variables

Question 8

Several test takers complained that items on the test were vague and confusing.This creates concern for:

A)time-sampling error
B)interrater differences
C)quality of test items
D)test-taker variables

Question 9

When interviewing test takers who had an achievement test on three different occasions,participants reported that they had remembered some of the answers from the previous test administration.This is known as:

A)practice effect
B)content effect
C)test-retest effect
D)carryover effect

Question 10

A researcher wants to measure content-sampling error and has two versions of an achievement test available.What measure of estimating reliability would be best in this situation?

A)Alternate forms
B)Test-retest
C)Split-Half reliability
D)Internal consistency reliability

Question 11

You are reading about reliability of a test in the test manual and notice that the researchers report using a Spearman-Brown coefficient.You can infer that internal consistency reliability was measured using:

A)the Kuder-Richardson formulas
B)coefficient alpha
C)split-half reliability
D)test-retest

Question 12

The researcher determines that the reliability coefficient is .65.This means the reliability is:

A)generally acceptable because it is over .50
B)generally acceptable for only interrater reliability
C)not generally acceptable
D)not generally acceptable unless it is a new test

Question 13

A researcher is concerned with measuring internal consistency reliability and has decided to use the Kuder- Richardson Formulas with a Likert Scale test.This is a problem because the:

A)test does not have dichotomous test items
B)researcher needs a second test for comparison
C)test does not measure internal consistency reliability
D)researcher is concerned with content sampling error

Question 14

When the researcher interprets the reliability coefficient,the closer the score is to 0,:

A)the greater likelihood that variance in scores is attributed to actual differences among test takers
B)the more reliable the test is
C)the closer the observed scores are to the true score
D)the more the scores represent random error

Question 15

An administrator and the school psychologist were observing a child to assess for behavioral problems.An error may occur in reviewing what the two observers notice.This is reported as:

A)content-sampling error
B)time-sampling error
C)interrater differences
D)test-taker variables

Question 16

If the reliability coefficient of a test is determined to be .27,what percentage is attributed to random chance or error?

A)27%
B)73%
C)2.7%
D)unknown percentage

Question 17

The researcher reports the standard error of measurement SEM).This is:

A)a different term for the standard deviation
B)the spread of scores of a single individual if he/she took a test repeated times
C)the spread of scores of a group of test takers on a single test
D)a measure to use alone as an index of reliability

Question 18

A researcher administers an achievement test to the same group of participants on three different occasions.In reporting the results,he describes the error that occurs from repeatedly testing the same individuals.This is called:

A)content-sampling error
B)time-sampling error
C)interrater differences error
D)test-taker variables error

Question 19

A researcher wants to measure internal consistency in a test that measures two different constructs self-esteem and depression)without subdividing the items into the two construct groupings.Which of the following would be the best method to use in measuring internal consistency?

A)Split-half reliability
B)The Kuder-Richardson formulas
C)Coefficient alpha
D)Any of the above methods would be equally appropriate

Question 20

A researcher wants to measure content-sampling error with a Likert scale test.Which of the following methods would be best?

A)Test-retest
B)Coefficient Alpha
C)The Kuder-Richardson formulas
D)Interrater differences

Question 21

You are a researcher that has developed a test to measure self-esteem.You are concerned about reliability.Think about the various methods used to estimate reliability.Create a plan to estimate the reliability of your test.Explain why you chose specific methods and the sources of measurement error that may be involved in using the method.Finally explain some steps you may take if the test has a low reliability coefficient.

Question 22

You read in a report that time-sampling error is a concern,but you cannot remember what method was used to estimate reliability.What methods may have been used,and why do they create a concern for time-sampling error?

Question 23

A researcher wants to measure internal consistency reliability and is aware that he could use split-half reliability,the Kuder-Richardson formulas,or coefficient alpha.He is unclear when it is best to use one method over another.Help him by providing a brief description of each method,including an explanation of when to use one method over another.Additionally,provide an example of when to use each method.

Question 24

Manuel is taking an achievement test for the second time.He expresses feeling tired after attending a pep rally at school the night before,but is excited about playing in the football game tonight.The heater does not appear to be working properly.It is making a clicking sound,and the room is relatively cold.After taking the test,Manuel reports that some of the questions were easy to answer because he remembers them from the previous test.Classify the three sources of measurement error in this scenario and describe how they may impact reliability.