Deck 4: Reliability and Validity

Question 1

As long as a test has good validity, the reliability is not important.

Answer 1

False

Question 2

Which of the following is not used to measure internal consistency?

A) Kuder-Richardson-20
B) coefficient alpha
C) split-half reliability
D) interrater reliability

Answer 2

D

Question 3

The scores obtained by Dr. Smith on a psychology test were compared and it was determined that students who scored high on the first exam tended to score high on the second exam. Which of the following represents the probable correlation between the two sets of scores?

A) no correlation
B) positive correlation
C) negative correlation
D) valid correlation

Answer 3

B

Question 4

When a teacher gives a test to students on one occasion, repeats the exam on the same students at a later date, and then correlates the scores, the teacher is demonstrating interest in determining which of the following?

A) split-half reliability
B) face validity
C) construct validity
D) test-retest reliability

Question 5

Which of the following refers to the consistency of scores on a specific instrument across time or across items?

A) test variability
B) test validity
C) test error
D) test reliability

Question 6

The term used to ensure that a psychological trait, personality trait, psychological concept, attribute or theoretical characteristic is measured by an instrument.

A) construct validity
B) determined by the purpose of the assessment
C) determined by the examiner according to the criterion
D) test-retest reliability

Question 7

A correlation of +.88 represents a strong negative correlation.

Question 8

Which type of reliability study may yield a lower coefficient, before correction, due to the reduced number of items?

A) split-half reliability
B) interrater reliability
C) coefficient alpha
D) Kuder-Richardson-20

Question 9

Measures of internal reliability are used to determine which of the following?

A) The stability of a trait over time.
B) The stability of a trait across different groups.
C) Consistency of items on an instrument to measure a skill, trait or domain.
D) Consistency of scoring by various examiners.

Question 10

When a preschool test is needed to determine how students will perform in reading, it is important that the test have good predictive reliability.

Question 11

When attempting to analyze error using estimated true score, the estimated score will be closer to the group mean than the obtained score.

Question 12

When a test measures what it was designed to measure and what it purports to measure, this test is said to be reliable.

Question 13

When analyzing the correlation illustrated on a scattergram, the closer the approximation of a straight line, the nearer the correlation is to which of the following?

A) +1.00
B) 0
C) +.50
D) -.50

Question 14

A teacher who administers a wide range achievement test in reading i.e., WRAT) is able to take the score obtained and generalize it across all of the individual domains/skills of reading.

Question 15

The validity of a test should be determined by how consistent the results are over time.

Question 16

Some of the variables of content validity that may influence the manner in which results are obtained and can contribute to bias in testing include presentation format and response mode

Question 17

Standard error of measurement is used to determine confidence intervals.

Question 18

Which of the following may be determined by analyzing the relationship between two sets of scores?

A) standard error
B) reliability
C) deviation
D) percentiles

Question 19

Generally, when tests measure constructs that are affected by developmental changes, the reliability may not be as good for very young ages.

Question 20

Which of the following statements is true concerning the relationship between reliability and standard error of measurement?

A) In general, the higher the reliability, the higher the standard error of measurement.
B) In general, the higher the reliability, the lower the standard error of measurement.
C) There is no way to determine a consistent relationship between the two, most of the time.
D) There is a positive correlation between the standard deviation and the standard error of measurement as well as reliability.

Question 21

Practice effects may be a problem when studying the results of which of the following?

A) split-half reliability
B) content validity
C) test-retest reliability
D) internal consistency

Question 22

Reliability coefficients are expressed:

A) between +.5 and -.5
B) between + 1 and 0
C) between -1 and 0.
D) between -1 and +1.

Question 23

Which of the following is considered a good reliability score?

A) .50
B) .60
C) .70
D) .80

Question 24

Select the strongest relationship.

A) -.59
B) -.47
C) +.28
D) +.57

Question 25

Establishing interrater reliability is especially important when test data are

A) scored by computer
B) subjective
C) scored by students
D) objective

Question 26

How would the use of standard error of measurement help in making educational decisions for students?

Question 27

Many colleges use the SAT as an entrance requirement for college. Why? What type of validity does the SAT purport to have?

Question 28

Select the statement that most likely would represent a negative correlation:

A) the number of hours spent at a party the night before an exam and the exam grade
B) the number of hours spent studying and test grades
C) the number of classes attended and test scores
D) the course section number and the test grade

Question 29

Given the general flow of the direction of the data, these variables are likely to

A) be positively correlated
B) be negatively correlated
C) have no correlation
D) There is no way to tell correlation from visual interpretation of data.

Question 30

This type of reliability checks the consistency across items of an instrument where credit varies across responses.

A) Split-half
B) Kuder-Richardson 20
C) Coefficient Alpha
D) Equivalent Forms

Question 31

When an instrument can predict performance on some other variable, it is considered to have

A) content validity
B) construct validity
C) predicative validity
D) criterion-related validity

Question 32

Comparing the scores of a new assessment instrument with the scores of an older test may help to establish which of the following?

A) content validity
B) internal reliability
C) alternative forms of validity
D) coefficient alpha

Question 33

Correlation is represented by what symbol?

A) ∑
B) µ
C) r
D) α

Question 34

Using the graph below, answer the questions that follow.
What is the name given to this type of graph?

A) Dot graph
B) Scattergram
C) Unfinished line graph
D) Variable comparison graph

Question 35

The type of reliability that looks at internal consistency by using odd and even numbered items to compare performance is called:

A) the odd-even comparison method
B) split-half reliability
C) odd-even halves
D) difficult

Question 36

Discuss the reasons why interrater reliability is important on assessment devices that have open-ended questions, such as essay exams.

Question 37

Administering two instruments within a short period of time is necessary in order to establish which of the following?

A) interrater reliability
B) alternate forms reliability
C) predictive validity
D) content validity

Question 38

A test item is considered to have this if it is answered incorrectly a disproportionate number of times by one group compared to another.

A) Item bias
B) Discriminatory features
C) Reliability
D) Validity

Question 39

Compare and contrast reliability and validity.

Question 40

Discuss the reasons why an educator may select a particular test for young children and a different test with older children even though both tests were designed to be used with both age groups.

Question 41

Describe how the data would appear in a scattergram for a positive, negative and no correlation.

Question 42

Determine a student's true score given the following information:
Mean score of the group = 70
Reliability coefficient = .81
Student's obtained score = 85

Question 43

Identify and define at least two ways to determine internal reliability.

Question 44

Discuss the controversy between the validity of tests and their use.

Question 45

What is construct validity and what type of studies are used to establish it?

Question 46

Determine the standard error of measurement given the following information:
Standard deviation = 3
Reliability coeffiecient = .51

Question 47

What are the three types of reliability? What are the optimal reliability scores?