Deck 6: Just the Truth: An Introduction to Understanding Reliability and Validity

Question 1

What is the key to establishing criterion validity?

A)number of criteria
B)quality of criterion
C)type of correlation
D)predictive ability

Answer 1

B

Question 2

In experiment, the treatment is called ______.

A)dependent variable
B)independent variable
C)intervening variable
D)mediating variable

Answer 2

B

Question 3

If you wanted to examine the impact of fast-food consumption on weight gain, your measurement of weight gain would be the _______.

A)dependent variable
B)treatment variable
C)predictor variable
D)independent variable

Answer 3

A

Question 4

If a researcher examined the effect of different reading programs on comprehension, the dependent variable was ______.

A)reading programs
B)comprehension
C)differences
D)researcher

Question 5

Reliability and validity specifically relate to the ______.

A)quality of the data themselves
B)appropriateness of the statistical tests chosen
C)qualitative methodology
D)correlations between variables

Question 6

Political affiliation depends on political orientation.Which variable is political orientation?

A)dependent variable
B)independent variable
C)mediation variable
D)intervening variable

Question 7

If you correlate scores from your test with some other valid measure that assesses the same set of abilities, what type of validity evidence are you collecting?

A)criterion validity
B)content validity
C)construct validity
D)internal consistency

Question 8

If I can say that my weekly statistics quiz fairly assesses the material covered, what source of validity evidence should I have collected?

A)criterion validity
B)content validity
C)construct validity
D)concurrent validity

Question 9

If a researcher examined how comprehension affected test scores, the independent variable was ______.

A)comprehension
B)researcher c, test scores
D)examination

Question 10

Which of the following is concerned with monitoring estimates of future performance?

A)content validity
B)predictive validity
C)concurrent validity
D)construct validity

Question 11

The more violent television programs an individual watches, the more likely the individual develops aggressive behavior.Watching violent television is ______.

A)a dependent variable
B)an independent variable
C)a mediation variable
D)an intervening variable

Question 12

The outcome variable in an analysis is called the _______.

A)predictor variable
B)treatment variable
C)dependent variable
D)independent variable

Question 13

If you want to know that a test measures some underlying psychological construct, what type of validity evidence would you want to collect?

A)content validity
B)construct validity
C)predictive validity
D)concurrent validity

Question 14

What type of validity is concerned with the adequate representation of test items?

A)content validity
B)predictive validity
C)concurrent validity
D)construct validity

Question 15

The EATS test (of culinary skills) has been shown to be correlated with being a fine chef 2 years after culinary school.This is an example of which type of validity?

A)content validity
B)criterion validity
C)concurrent validity
D)construct validity

Question 16

_______ explore(s) the question, "How do I know that the test, scale, instrument, and so on I use works every time I use it?"

A)Reliability
B)Validity
C)Both reliability and validity
D)Neither reliability nor validity

Question 17

Reliability and validity should be considered ______.

A)before you start analyzing data
B)after data have been analyzed
C)when data are being collected
D)in regard to research on human subjects only

Question 18

It is true-men who participate in body contact sports score higher on the TEST (osterone) test of aggression.This is an example of which type of validity?

A)content validity
B)criterion validity
C)concurrent validity
D)construct validity

Question 19

_______ explore(s) the question, "How do I know that the test, scale, instrument, etc.I use measures what it is supposed to?"

A)Reliability
B)Validity
C)Both reliability and validity
D)Neither reliability nor validity

Question 20

If data are not reliable or not valid, the results of any test or hypothesis ______.

A)must be true
B)are inconclusive
C)are valid in only certain situations
D)must be interpreted

Question 21

Cronbach's alpha is used to calculate _______.

A)internal consistency reliability
B)internal consistency validity
C)parallel forms reliability
D)internal validity

Question 22

_______ is based on the judgment of how well a test reflects an underlying idea.

A)Construct validity
B)Criterion validity
C)Internal consistency validity
D)Content validity

Question 23

In terms of the reliability of test scores, there are multiple elements to each person's score.The score that is actually recorded is the _______.

A)error score
B)true score
C)false score
D)observed score

Question 24

Which coefficient reflects the occurrence of perfect reliability?

A)1.00
B).5
C).100
D)2.00

Question 25

_______ is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the present.

A)content validity
B)predictive validity
C)concurrent criterion validity
D)test-retest validity

Question 26

"A measure of how stable a test is over time" is an example of which of the following types of reliability?

A)interrater
B)test-retest
C)parallel forms
D)internal consistency

Question 27

The _______ is the difference between the observed and true scores.

A)score differential
B)score difference
C)error score
D)error of approximation

Question 28

The correlation between scores from Time 1 and Time 2 is called _______.

A)test-retest reliability
B)parallel forms reliability
C)internal consistency reliability
D)similar forms reliability

Question 29

The actual or measured score is called the _______.

A)true score
B)observed score
C)error score
D)real score

Question 30

_______ is established by consultation with an expert on the topic focused upon by your instrument.

A)content validity
B)criterion validity
C)parallel forms validity
D)construct validity

Question 31

If the measures associated with a test are said to be consistent, you might conclude that the measure is _______.

A)standard
B)valid
C)reliable
D)concurrent

Question 32

High school class rank is highly correlated with college GPA.This is an example of what type of validity?

A)concurrent validity
B)predictive validity
C)content validity
D)construct validity

Question 33

The number of agreements between your two raters divided by the total number of possible agreements is the way to calculate _______.

A)parallel forms reliability
B)multiple judges reliability
C)interrater reliability
D)multiple judges validity

Question 34

Which of the following is NOT a measure of reliability?

A)test-retest reliability
B)multiple-forms reliability
C)internal consistency reliability
D)interrater reliability

Question 35

The correlation between two forms of the same test is called _______.

A)parallel forms reliability inverse reliabilityb.test-retest reliability
C)internal consistency reliability

Question 36

_______ is the type of validity that examines how well a test outcome is consistent with a criterion that occurs in the future.

A)content validity
B)test-retest validity
C)concurrent criterion validity
D)predictive validity

Question 37

_______ is the score that you would receive if the test contained no error.

A)Observed score
B)Actual score
C)True score
D)Error score

Question 38

_______ is determined by the association between the test scores and some specified present or future criterion.

A)construct validity
B)criterion validity
C)content validity
D)internal consistency validity

Question 39

Which of the following is NOT a measure of validity?

A)content validity
B)parallel forms validity
C)criterion validity
D)construct validity

Question 40

Two trained professionals observe the behavior of children in a classroom.They each rate observed behaviors using the same form, and the number of items that were rated the same is calculated.This is an example of which type of reliability?

A)parallel forms
B)test-retest
C)interrater
D)internal consistency

Question 41

Test-retest reliability is an example of which of the following?

A)measure of consistency
B)measure of reliability over time
C)measure of agreement
D)measure of equivalence

Question 42

Who is responsible for creating the formula for coefficient alpha?

A)Lee Cronbach
B)Jacob Cohen
C)Karl Pearson
D)R.A.Fisher

Question 43

The maximum level of validity possible is equal to ______.

A)the square of the reliability coefficient
B)the square root of the reliability coefficient
C)the cube of the reliability coefficient
D)the square root of the reliability coefficient, plus one

Question 44

If you want to know if the items on a test assess one and only one dimension, what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 45

What would the interrater reliability be for a 50-item measure in which the number of agreements between Rater 1 and Rater 2 was 45?

A)0.45
B)80%
C)0.90
D)95%

Question 46

If you want to know whether there is consistency in the rating of some outcome, what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 47

For a total of 80 possible agreements, there are 60 times where Teacher A and Teacher B agreed on whether students in college checked their cellphone in the classroom.What is the interrater reliability coefficient?

A)1.33
B).80
C)25
D).75

Question 48

Which of the following is the Greek letter associated with Cronbach's alpha, sometimes known as coefficient alpha?

A)k
B)α
C)Σ
D)s

Question 49

If you want to know if several different forms of a test are reliable, what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 50

For a total of 18 possible agreements, there are 7 times where Duck and Cow agreed on whether students in college fell asleep at their desks or not.What is the interrater reliability coefficient?

A)56
B)0.39
C)3.90
D).003

Question 51

What is the difference between the true score and the observed score?

A)critical score
B)error score
C)expected score
D)reliable score

Question 52

Parallel forms reliability is an example of which of the following?

A)measure of consistency
B)measure of stability
C)measure of agreement
D)measure of equivalence

Question 53

For a total of 10 possible agreements, there are 8 times where Bear and Cat agreed on whether children in ice cream commercials closed their eyes while eating ice cream or not.What is the interrater reliability coefficient?

A)0.80
B)0.08
C)80.0
D)8.0

Question 54

When we calculate reliability, we know the observed score.What are the two unknown components of the reliability equation?

A)method and error scores
B)means and standard deviations
C)test-retest and interrater scores
D)true and error scores

Question 55

If you want to know whether a test is reliable over time, what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 56

Observed scores are commonly exactly the same as true scores in the social and behavioral sciences.

Question 57

Interrater reliability measures consistency ______.

A)over time
B)from form to form
C)across different tests
D)from rater to rater

Question 58

Interrater reliability is an example of which of the following?

A)measure of consistency
B)measure of stability
C)measure of agreement
D)measure of equivalence

Question 59

For high reliability, you want to reliability coefficients to be positive and as close as possible to 1.

Question 60

What would the interrater reliability be for a 100-item measure in which the number of agreements between Rater 1 and Rater 2 was 80?

A)0.45
B)80%
C)0.90
D)95%

Question 61

Criterion validity assesses whether a test reflects a set of abilities in a future setting.

Question 62

Name four types of reliability and what they measure.

Question 63

Explain the formula for understanding observed scores.What is included in any observed score?

Question 64

When should you use construct validity?

Question 65

A test can be valid but not reliable, but it is impossible to have a test that is reliable and not valid.

Question 66

When should we measure internal consistency reliability?

Question 67

What is the difference between reliability and validity?

Question 68

If your test is not reliable, you must try to lower your error.

Question 69

How does error impact reliability? Write the general equation for reliability that demonstrates this concept.

Question 70

Construct validity is the most difficult of all the validities to establish.

Question 71

Observed score equals the true score minus error score.

Question 72

What are the traditional types of validity?

Question 73

How do reliability and validity work together?

Question 74

To use archery practice as a metaphor for reliability and validity, hitting the bull's-eye of a target demonstrates _______, whereas repeatedly hitting the target in the same small area demonstrates _______.

Question 75

Interrater reliability examines the percentage of agreement between raters.

Question 76

Internal consistency reliability tests how each individual item scores correlates with the total score.

Question 77

Why can we not know the true score for a test?