Deck 8: Reliability and Validity

Question 1

When we measure the same thing on more than one occasion to see if we get the same score, which type of reliability are we testing?

A)test-retest reliability
B)split-half reliability
C)Hoyt analysis of variance reliability
D)scalar reliability

Answer 1

A

Question 2

Kris has created a new scale to measure provider satisfaction in nursing homes. When the measure is completed, Kris asks her friend Dave to review it and subjectively assess if the content appears to be related to what she has intended to measure (provider satisfaction). What type of validity is Kris asking Dave to assess?

A)convergent
B)criterion
C)concurrent
D)content

Answer 2

D

Question 3

Which threat to validity asks the question, "Are the results yours, or have multiple treatments by yourself and others combined to give the effect?"

A)mono-operation bias and mono-method bias
B)inadequate preoperational explication of concepts
C)restricted generalizability across constructs
D)interaction of different treatments

Answer 3

D

Question 4

Which type of reliability testing is concerned with whether or not a person's score on a new measure can be used to predict future scores on a different measure?

A)convergent
B)predictive
C)concurrent
D)construct

Question 5

John has created a scale that measures "extraterrestrial communication competence." John is torn between which type of validity testing he should employ. Which approach to validity should John utilize?

A)retrospective and face validity
B)face and predictive validity
C)concurrent and construct validity
D)John should employ all approaches to validity that can be employed

Question 6

John then takes his Extraterrestrial Communication Competence Scale and has participants fill it out along with Weiner's Communication Competence Scale because he theorizes the two should be related. What type of validity is this?

A)retrospective
B)face
C)predictive
D)concurrent

Question 7

John also conducts a series of interviews with abductees and determines their level of communication competence based on their interviews. He then has them fill out the Extraterrestrial Communication Competence Scale. What type of validity is this?

A)retrospective
B)face
C)predictive
D)concurrent

Question 8

After initially validating his Extraterrestrial Communication Competence Scale, John realizes that he has not really determined what the scale was supposed to measure in the first place. Which validity threat is John facing?

A)interaction of testing and treatment
B)mono-operation bias
C)confounding constructs and levels of constructs
D)inadequate preoperational explication of concepts

Question 9

Which type of single administration reliability test is most commonly used by social scientists?

A)test-retest reliability
B)alternate forms reliability
C)split-half reliability
D)Cronbach's alpha (α)

Question 10

Zacary has just finished running Cronbach's alpha for his newly developed touch avoidance scale. During his reliability analysis, Zacary finds out that his scale's alpha reliability is .72. How would we characterize this reliability?

A)excellent
B)good
C)respectable
D)minimally acceptable

Question 11

While filling out a scale on drug use, Charlie states that he has never done drugs even though he gets high three or four times daily. Which validity problem does this represent?

A)acquiescence
B)socially desirable bias responding
C)hypothesis guessing
D)experimenter expectations

Question 12

Which validity problem describes a research participant who attempts to give the research responses indicative of the exact opposite of what he or she wants or to give completed random responses?

A)acquiescence
B)socially desirable responding
C)hypothesis guessing
D)screw-you-effect

Question 13

Johan is testing a new measure of public speaking anxiety. The scale consists of 10 Likert-type items. To test for reliability, Johan compares the even and odd numbered questions. What type of reliability test is Johan completing?

A)alternative forms
B)Chronbach's alpha
C)split-half
D)test-retest

Question 14

Jenny is running a study examining people's perceptions of "fake news." Her newly created measure for "fake news susceptibility" had a Cronbach's alpha of .86. How would you characterize the alpha reliability of Jenny's measure?

A)excellent
B)good
C)respectable
D)undesirable

Question 15

In his study, Joshua reports the following in his instrumentation section:
"The alpha reliability found for the Perceptions of Religious Communication in the current study was 0.75 (M = 32.53, SD = 7.9)."
Which of the following statements is TRUE?

A)The Cronbach alpha in this study is considered undesirable.
B)The mean for the Perceptions of Religious Communication measure was 7.92.
C)The split-half reliability for this study was .75.
D)Alpha reliability for this measure is considered respectable.

Question 16

Which of the following is NOT a way to improve the alpha reliability of a measure?

A)adding more items
B)avoiding ambiguous items
C)writing clear instructions
D)ensuring the concurrent validity

Question 17

The Personal Report of Communication Apprehension-24 (PRCA-24) is not a _________ measure of ethnocentrism.

A)reliable
B)explanatory
C)valid
D)intervening

Question 18

Amy created a new measure of media bias. She first shows one of her colleagues the new measure to see if the measure appears valid. What type of validity is Amy ensuring?

A)criterion
B)face or content
C)predictive or prospective
D)retrospective

Question 19

___________ validity occurs when a researcher has previously measured the criterion and then attempts to relate it to a newly developed measure at a later point.

A)criterion
B)face or content
C)predictive or prospective
D)retrospective

Question 20

A scale can be reliable but not valid.

Question 21

Even if faking is present in a research study, validity can still be sound.

Question 22

Predictive (prospective) validity is the form of validity concerned with whether scores taken at the same time are related to each other.

Question 23

Scalar reliability is the most common form of reliability assessed in quantitative research in communication.

Question 24

Factorial validity is based on a statistical technique known as factor analysis.

Question 25

Validity is directly associated with the number of items included in the measuring instrument.

Question 26

Face or content validity does not make use of correlation coefficients.

Question 27

Explain four different types of reliability.

Question 28

Explain the importance of validity in communication research studies.

Question 29

Explain three methods to test a measure's construct or factorial validity.

Question 30

List and explain 3 problems with validity.

Question 31

Describe an unreliable pattern of participant response to a questionnaire.

Question 32

Match each of the following terms with the correct statement.
-Test-retest Reliability

A)Measuring a concept on more than one occasion and comparing scores.
B)Concerned with whether or not a person's score on a new measure can be used to estimate future scores.
C)Singular administration of a measure.
D)The accuracy that a measure has in producing stable, consistent measurements.
E)Using two instruments to measure the same concept and checking the reliability between the two instruments.

Question 33

Match each of the following terms with the correct statement.
-Predictive Validity

A)Measuring a concept on more than one occasion and comparing scores.
B)Concerned with whether or not a person's score on a new measure can be used to estimate future scores.
C)Singular administration of a measure.
D)The accuracy that a measure has in producing stable, consistent measurements.
E)Using two instruments to measure the same concept and checking the reliability between the two instruments.

Question 34

Match each of the following terms with the correct statement.
-Mono-Operational Bias

A)Measuring a concept on more than one occasion and comparing scores.
B)Concerned with whether or not a person's score on a new measure can be used to estimate future scores.
C)Singular administration of a measure.
D)The accuracy that a measure has in producing stable, consistent measurements.
E)Using two instruments to measure the same concept and checking the reliability between the two instruments.

Question 35

Match each of the following terms with the correct statement.
-Reliability

A)Measuring a concept on more than one occasion and comparing scores.
B)Concerned with whether or not a person's score on a new measure can be used to estimate future scores.
C)Singular administration of a measure.
D)The accuracy that a measure has in producing stable, consistent measurements.
E)Using two instruments to measure the same concept and checking the reliability between the two instruments.

Question 36

Match each of the following terms with the correct statement.
-Alternate Forms Reliability

A)Measuring a concept on more than one occasion and comparing scores.
B)Concerned with whether or not a person's score on a new measure can be used to estimate future scores.
C)Singular administration of a measure.
D)The accuracy that a measure has in producing stable, consistent measurements.
E)Using two instruments to measure the same concept and checking the reliability between the two instruments.