Deck 5: Measuring and Manipulating Variables: Reliability and Validity

Question 1

Which of the following is not an operational definition of "love"?

A) Love is a many-splendored thing.
B) Love is taking a woman home on the first date and kissing her twice on the nose.
C) Love is eating your spouse's cooking.
D) Love is not having to say you're sorry more than twice during any 24 hour period.

Answer 1

Love is a many-splendored thing.

Question 2

Which of the descriptions given below is an operational definition of liking?

A) genuine caring for another
B) positive feelings toward another
C) number of favors done for another
D) selfless dedication to another's well-being

Answer 2

number of favors done for another

Question 3

Which of the following is an operational definition?

A) hunger is 8 hours of food deprivation
B) attachment is the bond between an infant and its mother
C) learning is a relatively permanent change in behavior
D) anger is a negative emotional state

Answer 3

hunger is 8 hours of food deprivation

Question 4

Which of the following is NOT a construct?

A) sadness
B) anger
C) nervousness
D) crying

Question 5

Psychological research can produce objective, scientific data because

A) individual psychologists are objective
B) psychology is a social science
C) psychologists use operational definitions
D) psychologists are cautious individuals

Question 6

By definition, a valid measure is one that

A) is consistent
B) measures what it claims it is measuring
C) has high internal consistency
D) is extremely reliable

Question 7

Reliable measures are _______ ; valid measures are _______.

A) experimental, quasi-experimental
B) consistent, accurate
C) quasi-experimental, experimental
D) accurate, consistent

Question 8

Determining whether a measure is reliable involves determining the extent to which a measure is ______; determining whether a measure is valid involves determining the extent to which a measure is ______.

A) convergent, divergent
B) accurate, consistent
C) consistent, accurate
D) divergent, convergent

Question 9

The extent to which a test measures anything consistently is its

A) normality
B) objectivity
C) reliability
D) validity

Question 10

If a test is designed to measure "love" but measures liking instead, the test is

A) not reliable
B) not valid
C) not objective
D) not standardized

Question 11

Introducing random error into a study

A) reduces reliability
B) reduces validity
C) biases the results
D) both a and b

Question 12

If a measure involves having observers rate behavior, which of the following is the most severe threat to that measure's reliability?

A) observer bias
B) failure to calculate inter-observer reliability coefficients
C) failure to use double-blind techniques
D) random observer error

Question 13

A major advantage that objective measures (e.g., multiple-choice) items generally have over subjectively scored measures is

A) less vulnerability to observer biases
B) less vulnerability to subject biases
C) neither a nor b
D) both a and b

Question 14

Which of the following is not a technique for increasing reliability?

A) using unobtrusive observation
B) using instruments
C) standardizing procedures
D) having raters count rather than judge behavior

Question 15

A test-retest coefficient can range in value from

A) 0 to 100
B) 0 to 1.00
C) 1 to 99
D) -1.00 to +1.00

Question 16

Which measure is the most reliable? Note that all reliabilities refer to test-retest reliabilities where participants were retested 10 weeks later.

A) Measure A which has a .90 test-retest reliability
B) Measure B which has a .85 test-retest reliability
C) Measure C which has a .80 test-retest reliability
D) Cannot tell from the information given

Question 17

Which measure is the most valid? Note that all reliabilities refer to test-retest reliabilities where subjects were retested 10 weeks later.

A) Measure A which has a .90 test-retest reliability coefficient
B) Measure B which has a .85test-retest reliability coefficient
C) Measure C which has a .80 test-retest reliability coefficient
D) Cannot tell from the information given

Question 18

Which reliability coefficient would you expect to be the highest?

A) inter-observer reliability
B) median inter-item correlations
C) test-retest reliability
D) all of the above should be about the same

Question 19

Which reliability coefficient would you expect to be the lowest?

A) inter-observer reliability
B) split-half reliability
C) test-retest reliability
D) can't tell form the information given

Question 20

The extent to which a measure measures what it was designed to measure is the

A) internal consistency
B) reliability
C) standardization
D) validity

Question 21

If a measure is _______, then it is also _______.

A) reliable, valid
B) valid, reliable
C) internally consistent, valid
D) free of observer bias, valid

Question 22

The problem of observers seeing what they expect to see is called

A) observer bias
B) random observer error
C) subject bias
D) demand characteristics

Question 23

Which of the following is the most severe threat to a measure's validity?

A) observer bias
B) failure to calculate inter-observer reliability coefficients
C) failure to use instruments
D) random observer error

Question 24

Participants in a study may be influenced by hints given about what is expected of them. These hints are called

A) placebo effects
B) social desirability bias
C) demand characteristics
D) observer bias

Question 25

Which technique is more effective for reducing observer bias than for reducing random observer error?

A) using instruments
B) using multiple-choice tests
C) making raters blind
D) training raters

Question 26

The case of Clever Hans ("the mathematical horse") is an example of

A) random observer error
B) observer bias
C) subject bias
D) fraud

Question 27

When observing behavior, one way to avoid the effects of subject bias is to:

A) hide the observer
B) take careful notes
C) make friends with the observed
D) record only selected details

Question 28

All but one of the following techniques were mentioned as methods of reducing observer bias. Which is it?

A) using scientific instruments
B) simplifying the observers' task
C) making observers blind
D) using multiple observers

Question 29

There is some evidence to suggest that the pattern of cues evident in a research procedure may convey the researcher's hypothesis to participants, who may react by trying to confirm the hypothesis in the way that they respond to the conditions. The pattern of cues referred to here is known as

A) pretesting effects
B) demand characteristics
C) experimenter bias
D) participant clues

Question 30

Participant biases may be reduced by using ______ observation.

A) unstructured
B) structured
C) unobtrusive
D) obtrusive

Question 31

If participants lie or exaggerate to make themselves look good, the study has been affected by:

A) demand characteristics
B) observer bias
C) social desirability bias
D) random participant error

Question 32

When an investigator correlates scores from a new measurement instrument to an existing instrument of the same construct, the investigator is trying to determine the new measure's

A) reliability
B) convergent validity
C) internal consistency
D) discriminant validity

Question 33

A measure with internal consistency

A) accurately predicts future performance
B) correlates with other tests of the same construct
C) appears to measure what it claims to measure
D) appears to have a high level of split-half reliability

Question 34

If you establish that a measure of lust does not correlate highly with a measure of liking, you have taken a step toward establishing the measure's

A) content validity
B) criterion validity
C) discriminant validity
D) internal consistency

Question 35

If you establish that a measure of lust does not correlate highly with a measure of social desirability, you have taken a step toward establishing the measure's

A) content validity
B) criterion validity
C) discriminant validity
D) internal consistency

Question 36

When students judge a midterm exam, they are looking at the test's

A) convergent validity
B) discriminant validity
C) content validity
D) internal consistency

Question 37

The known groups technique is used to establish a measure's

A) convergent validity
B) discriminant validity
C) content validity
D) internal consistency

Question 38

A professor of a continuing education course called "Becoming rich" is correlating final exam scores with students' income five years after graduation. This professor is trying to establish that the final exam has

A) convergent validity
B) discriminant validity
C) content validity
D) internal consistency

Question 39

A professor of a continuing education course called "Becoming rich" is correlating final exam scores with students' SAT scores. This professor is trying to establish that the final exam has

A) convergent validity
B) discriminant validity
C) content validity
D) internal consistency

Question 40

The broadest type of validity, in that it includes all the others, is ______ validity.

A) convergent
B) construct
C) content
D) discriminant

Question 41

If we are concerned that our measure distinguishes friends from lovers, we are trying to establish:

A) internal consistency
B) reliability
C) discriminant validity
D) content validity

Question 42

Shortening a test would be most likely to

A) harm its internal consistency
B) harm its content validity
C) harm its discriminant validity
D) boost its reliability

Question 43

If participants' scores on a social intelligence test are consistent from one time of testing to the next, the test is ________. If participants' scores on a social intelligence test are accurate indicators of their social intelligence, the test is ______

A) reliable, valid
B) valid, reliable
C) reliable, reliable
D) valid, valid

Question 44

If participants' scores on a new social intelligence test are correlate with their scores on an established test of social intelligence, the new test has ________ validity. If participants' scores on the new social intelligence test do not correlate with their scores on an extroversion test, the test has ______ validity.

A) construct, internal
B) convergent, discriminant
C) discriminant, convergent
D) internal, construct

Question 45

Hypothetical characteristics of people, inferred from observations, are called

A) constructs
B) operational definitions
C) models
D) relationships

Question 46

Reliability has no effect on which of the following?

A) validity
B) bias

Question 47

Which of the following is a weakness of self-report measures?

A) They are vulnerable to observer bias.
B) They lack content validity.
C) They are unreliable.
D) They are influenced by subject biases.

Question 48

Using the blind technique would probably reduce all of the following EXCEPT

A) experimenter bias
B) subject bias
C) Hawthorne effect
D) random error

Question 49

Standardization would probably reduce

A) experimenter bias
B) random error
C) Hawthorne effect
D) both a and b

Question 50

Which of the following would be the most effective strategy for reducing the Hawthorne effect:

A) training observers to be unbiased
B) the use of placebos
C) standardization
D) the use of instruments

Question 51

Psychologists believe that manipulation checks

A) should be used even when it seems obvious that the manipulation is manipulating what it should be manipulating
B) increase the reliability of a manipulation
C) are a good substitute for placebo treatments
D) do not need to be used if the manipulation is consistent with theory

Question 52

____ manipulations are most easily standardized whereas ___ manipulations are hardest to standardize.

A) instructional; environmental
B) environmental; instructional
C) instructional; stooges
D) stooges; instructional

Question 53

_____ manipulations are most vulnerable to subject biases, whereas _____ manipulations are most vulnerable to experimenter biases.

A) instructional; environmental
B) environmental; instructional
C) instructional; stooges
D) stooges; instructional

Question 54

If experimenter bias is your biggest concern, you would use

A) an instructional manipulation
B) an environmental manipulation
C) a stooge manipulation
D) a manipulation check

Question 55

If subject bias is your biggest concern, you would use:

A) an instructional manipulation
B) an environmental manipulation
C) a stooge manipulation
D) a manipulation check

Question 56

Measures can be reliable and still have a large amount of random error.

Question 57

Interobserver reliability coefficients are a good way to assess how much random error is affecting scores assigned by observers.

Question 58

Random error and bias are the same thing.

Question 59

Random error is a more serious threat to validity than bias.

Question 60

A valid measure is also a reliable measure.

Question 61

An unreliable measure cannot be a valid measure.

Question 62

Due to the nature of psychological constructs, researchers do not need to be concerned with the effects of social desirability on the results of their studies.

Question 63

The use of unobtrusive measures is a could way to combat the effects of subject biases.

Question 64

The use of blind procedures is a good way of reducing experimenter bias.