Deck 4: Reliability and Validity

Question 1

Compare the reliability coefficient obtained by nine-year-olds on the Estimation subtests with the reliability coefficient obtained by seven-year-olds on the same subtest. Which age group performed with more consistency or reliability?

Answer 1

Question 2

Examiner wants to administer a test that allows for examiner judgment.

Answer 2

Internal reliability attempts to measure the consistency among the items within a test. Other types of reliability studies attempt to measure the consistency of the test as a whole instrument.

Question 3

What happens to the standard error of measurement as the reliability increases?

Answer 3

SEM goes down as reliability increases.

Question 4

Educator is concerned with item consistency; items scored with different point values for correct responses.

Question 5

Educator wants to administer the same test twice to measure achievement of objectives.

Question 6

Would you consider purchasing this instrument?

Question 7

The following sets of data are scores on a mathematics ability test and grade level achievement in math for fifth graders. Plot the scores on the scattergram.

Question 8

Using the standard error of measurement found in problems 1 and 2, calculate the ranges for each age level if the obtained scores were both 7. Calculate the ranges for both 68% and 95% confidence intervals.

Question 9

Based on this information, in which grades would you feel that the test would yield more reliable results? Why?

Question 10

What type of reliability is reported in Table 4.1?

Question 11

Look at the reliability reported for age 7. Using fall statistics, compare the reliability coefficient obtained on the Numeration subtest with the reliability coefficient obtained on the Estimation subtest. On which subtest did seven-year-olds perform with more consistency?

Question 12

Complete the scattergrams using the following sets of data. Determine whether the scattergrams illustrate positive, negative, or no correlation.

Question 13

Examiner is concerned with consistency of trait over time.

Question 14

The standard error of measurement for a seven-year-old who was administered the Problem Solving subtest in the fall

Question 15

A twelve-year-old's standard error of measurement for Problem-Solving if the test was administered in the fall

Question 16

Here is an example of a negative correlation between two variables. Plot the scores on the scattergram.

Question 17

What happens to the standard error of measurement as the standard deviation increases?

Question 18

Explain how the concurrent criterion-related validity would have been determined

Question 19

Educator is concerned with item reliability; items are scored as right and wrong.

Question 20

A new academic achievement test assesses elementary-age students' math ability. The test developers found, however, that students in the research group who took the test two times had scores that were quite different upon the second test administration, which was conducted two weeks after the initial administration. It was determined that the test did not have acceptable ___.

Question 21

On a teacher-made test of math, the following items were included: two single-digit addition problems, one single-digit subtraction problem, four problems of multiplication of fractions, and one problem of converting decimals to fractions. This test does not appear to have good ___.

Question 22

Use the following set of data to determine the mean, median, mode, range, variance, standard deviation, standard error of measurement, and possible range for each score assuming 68% confidence. The reliability coefficient is .85.
Data: 50, 75, 31, 77, 65, 81, 90, 92, 76, 74, 88

Question 23

The smaller the standard error of measurement, the more ___ the test.

Question 24

No matter how many times a student may be tested, the student's ___ may never be determined.

Question 25

One would expect that the number of classes a college student attends in a specific course and the final exam grade in that course would have a ___.

Question 26

A test instrument may have good reliability; however, that does not guarantee that the test has ___.

Question 27

When a test measures what it was designed to measure and what it purports to measure, this test is said to be reliable.

Question 28

Estimated true score allows you to calculate the amount of error with reference to the distance of the score from the mean of the group.

Question 29

In order to have a better understanding of a student's true abilities, the concept of ___ must be understood and applied to obtained scores.

Question 30

When calculating the range of possible scores, it is best to use the appropriate standard error of measurement for the student's ___ provided in the test manual.

Question 31

The number of times a student moves during elementary school may likely have a ___ to the student's achievement scores in elementary school.

Question 32

Standard error of measurement is used to determine confidence intervals.

Question 33

The larger the standard error of measurement, the less ___ the test.

Question 34

In studying the relationship between the scores of the administration of one test administration with the second administration of the test, the number .89 represents the ___.

Question 35

A perfect positive correlation would be indicated by a straight line between scores on a scattergram.

Question 36

A college student failed the first test of the new semester. The student hoped that the first test did not have strong ___ about performance on the final exam.

Question 37

The score obtained during the assessment of a student may not be the true score, because all testing situations are subject to chance ___.

Question 38

A closer estimation of the student's best performance can be calculated by using the ___ score.

Question 39

A new test was designed to measure the self-concept of students of middle school age. The test required students to use essay-type responses to answer three questions regarding their feelings about their own self-concept. Two assessment professionals were comparing the students' responses and how these responses were scored by the professionals. On this type of instrument, it is important that the ___ is acceptable.

Question 40

A range of possible scores can then be determined by using the ___ for the specific test.

Question 41

When analyzing the correlation illustrated on a scattergram, the closer the approximation of a straight line (indicating a perfect degree of correlation), the nearer the correlation is to which of the following?

A) +1.00
B) 0
C) +.50
D) −.50

Question 42

Which of the following refers to the consistency of scores on a specific instrument across time or across items?

A) Test variability
B) Test validity
C) Test error
D) Test reliability

Question 43

The validity of a test should be determined by how consistent the results are over time.

Question 44

Measures of internal consistency are used to determine which of the following?

A) The stability of a trait over time
B) The stability of a trait across different groups
C) Consistency of items on an instrument to measure a skill, trait or domain
D) Consistency of scoring by various examiners

Question 45

Generally, when tests measure constructs that are affected by developmental changes, the reliability may not be as good for very young ages.
Multiple Choice

Question 46

Which type of reliability study may yield a lower coefficient, before correction, due to the reduced number of items?

A) Split-half reliability
B) Interrater reliability
C) Coefficient alpha
D) Kuder-Richardson-20

Question 47

Administering two instruments within a short period of time is necessary in order to establish which of the following?

A) Interrater reliability
B) Alternate forms reliability
C) Predictive validity
D) Content validity

Question 48

The scores obtained by Dr. Smith on a psychology test were compared and it was determined that students who scored high on the first exam tended to score high on the second exam. Which of the following represents the probable correlation between the two sets of scores?

A) No correlation
B) Positive correlation
C) Negative correlation
D) Valid correlation

Question 49

A practice effect may be a problem when studying the results of which of the following?

A) Split-half reliability
B) Content validity
C) Test-retest reliability
D) Internal consistency

Question 50

What is the term used to ensure that a psychological trait, personality trait, psychological concept attribute or theoretical characteristic is measured by an instrument?

A) Construct validity
B) Determined by the purpose of the assessment
C) Determined by the examiner according to the criterion
D) Test-retest reliability

Question 51

Which of the following is not used to measure internal consistency?

A) Kuder-Richardson-20
B) Coefficient alpha
C) Split-half reliability
D) Interrater reliability

Question 52

A teacher who administers a wide range achievement test in reading (i.e., WRAT) is able to take the score obtained and generalize it across all of the individual domains and skills associated with reading.

Question 53

Which of the following statements is true concerning the relationship between reliability and standard error of measurement?

A) In general, the higher the reliability, the higher the standard error of measurement.
B) In general, the higher the reliability, the lower the standard error of measurement.
C) There is no way to determine a consistent relationship between the two, most of the time.
D) There is a positive correlation between the standard deviation and the standard error of measurement as well as reliability.

Question 54

When a preschool test is needed to determine how students will perform in reading, it is important that the test have good predictive validity.

Question 55

As long as a test has good validity, the reliability is not important.

Question 56

Some of the variables of content validity that may influence the manner in which results are obtained and which can contribute to bias in testing include presentation format and response mode.

Question 57

When a teacher gives a test to students on one occasion, repeats the exam on the same students at a later date, and then correlates the scores, the teacher is demonstrating interest in determining which of the following?

A) Split-half reliability
B) Face validity
C) Construct validity
D) Test-retest reliability

Question 58

Comparing the scores of a new assessment instrument with the scores of an older test may help to establish which of the following?

A) Content validity
B) Internal reliability
C) Alternative forms of validity
D) Coefficient alpha

Question 59

Which of the following may be determined by analyzing the relationship between two sets of scores?

A) Standard error
B) Reliability
C) Deviation
D) Percentiles

Question 60

Correlation coefficients are expressed as which of the following?

A) Between +.5 and −.5
B) Between + 1 and 0
C) Between − 1 and 0
D) Between −1.00 and +1.00

Question 61

A test item is considered to have this if it is answered incorrectly a disproportionate number of times by one group compared to another.

A) Item bias
B) Discriminatory features
C) Reliability
D) Validity

Question 62

Many colleges use the SAT as an entrance requirement for college. Why? What type of validity does the SAT offer?

Question 63

Describe how the data would appear in a scattergram for a positive, negative and no correlation.

Question 64

Which of the following is considered a good reliability score?

A) .50
B) .60
C) .70
D) .80

Question 65

Determine the standard error of measurement given the following information:
Standard deviation = 3
Reliability coeffiecient = .51

Question 66

Correlation is represented by what symbol?

A) ∑
B) µ
C) r
D) α
Using the graph below, answer the questions that follow.

Question 67

When an instrument can predict performance on some other variable, it is considered to have which of the following?

A) Content validity
B) Construct validity
C) Predicative validity
D) Criterion-related validity
Short Answer

Question 68

The type of reliability that can look at internal consistency by using odd and even numbered items of the same test to compare performance is called which of the following?

A) The odd-even comparison method
B) Split-half reliability
C) Odd-even halves
D) Difficult

Question 69

Discuss the positives and negatives of the implementation of accommodations when assessing a student with special needs.

Question 70

What is the name given to this type of graph?

A) Dot graph
B) Scattergram
C) Unfinished line graph
D) Variable comparison graph

Question 71

Given the general flow of the direction of the data, these variables are likely to be which of the following?

A) Positively correlated
B) Negatively correlated
C) Possess no correlation
D) There is no way to tell correlation from visual interpretation of data.

Question 72

How would the use of standard error of measurement help in making educational decisions for students?

Question 73

Discuss the reasons why interrater reliability is important on assessment devices that have open-ended questions, such as essay exams.

Question 74

Establishing interrater reliability is especially important when test data are which of the following?

A) Scored by computer
B) Subjective
C) Scored by students
D) Objective

Question 75

Discuss some potential challenges associated with the validity of tests and their use.

Question 76

Which of the following options would most likely represent a negative correlation?

A) The number of hours spent at a party the night before an exam and the exam grade
B) The number of hours spent studying and test grades
C) The number of classes attended and test scores
D) The course section number and the test grade

Question 77

What is error in psychoeducational assessment? Does error play into assessing learners with special needs to a greater or lesser extent than typical learners? Discuss.

Question 78

Compare and contrast reliability and validity.

Question 79

Which of the following represents the strongest relationship?

A) −.59
B) −.47
C) +.28
D) +.57

Question 80

This type of reliability checks the consistency across items of an instrument where credit varies across responses.

A) Split-half
B) Kuder-Richardson 20
C) Coefficient Alpha
D) Equivalent Forms