Deck 4: Validity and Reliability: Threats to Research Integrity

Question 1

Validity is closely related to the principle of operationalization, which addresses how variables are measured.

Answer 1

True

Question 2

Oftentimes a single measure may not provide a true measure of a variable. In this case, researchers can combine several different measures together.

Answer 2

True

Question 3

Higher values of Cronbach's alpha represent higher reliability of a certain scale or index.

Answer 3

True

Question 4

Many researchers make a useful analogy to help understand validity and reliability when developing a case by comparing them to a target and marksmanship. The tightness of the bullet holes reflects validity.

Question 5

Reliability can be tested in several methods, including test-retest reliability, intraobserver reliability, interitem reliability, and alternate-forms reliability.

Question 6

Natural disaster or climate change is the only example of history threatening internal validity.

Question 7

History is the most difficult threat to internal validity because there is not much to mitigate its effect.

Question 8

Testing effect is a problem when change in the measuring instrument is made between pretest and posttest.

Question 9

Because of statistical regression, it is important to analyze the data for outliers and consider their effects.

Question 10

The best way to overcome selection bias is to use randomization when assigning research participants to different groups.

Question 11

Diffusion of treatment in regards to validity and reliability means that the stimulus in an experiment was administered on time.

Question 12

More than one threat to internal validity can occur simultaneously, which can be named confounding interactions.

Question 13

The Hawthorne effect and placebo effect are examples of reactivity.

Question 14

Criterion validity determines how well a measure covers the meanings included in the concept.

Question 15

To reify something means to treat something that is not real as if it were real.

Question 16

Which of the following is closely related to the term validity?

A) Approximation to the truth
B) Reflection of a characteristics or property in the real world
C) Accuracy
D) All of the above

Question 17

Which of the following is closely related to the term reliability?

A) Consistency
B) Repeatability
C) Precision
D) All of the above

Question 18

To improve the reliability of an instrument, researchers may use the same exam or questionnaire more than one time, which is known as __________.

A) test-retest reliability
B) intraobserver reliability
C) interim reliability
D) alternate-forms reliability

Question 19

Qualitative researchers make observations more than one time, and the same result is obtained, which is called ___________ reliability.

A) test-retest
B) intraobserver
C) interim
D) alternate-forms

Question 20

To determine reliability, researchers can take multiple questions when measuring the factor of interest, which is known as __________ reliability.

A) test-retest
B) intraobserver
C) interim
D) alternate-forms

Question 21

Which of the following is false regarding internal validity?

A) Internal validity exists when a relationship between two variables is causal.
B) In other words, each of the variables must vary together.
C) However, the direction of the change between variables does matter.
D) The covariance between variables may result from some unknown variable(s) called exogenous variables.

Question 22

__________ validity is concerned with how well the study findings are generalizable to other settings.

A) Internal
B) External
C) Statistical conclusion
D) Content

Question 23

Among various threats to internal validity, __________ refers to any events that happen during the course of a study that may affect the study outcomes.

A) history
B) maturation
C) testing effect
D) instrumentation

Question 24

Which of the following can be an example of history endangering internal validity?

A) Natural disaster such as Hurricane Katrina
B) Terrorist attack
C) Economic crisis
D) All of the above

Question 25

Among threats to internal validity, __________ refers to any physical or psychological changes taking place within participants that occur with the passing of time regardless of the experimental manipulation.

A) history
B) maturation
C) testing effect
D) instrumentation

Question 26

Which of the following is false regarding maturation?

A) Maturation threatens internal validity mainly because research participants get older and experienced.
B) It is true that research participants' emotional make-up can influence a study's outcome.
C) There is no solution to maturation effects to interval validity.
D) None of the above

Question 27

__________ occurs when research participants become so familiar with the survey instruments that their answers are influenced.

A) History
B) Maturation
C) Testing effect
D) Instrumentation

Question 28

Which of the following can be a solution to testing effect?

A) Random assignment
B) Altering the original questions with wording
C) Providing post-test
D) All of the above

Question 29

__________lowers the level of internal validity when change in the measuring instrument is made between the applications.

A) History
B) Maturation
C) Testing effect
D) Instrumentation

Question 30

Which of the following is false regarding intercoder reliability?

A) Intercoder reliability occurs when a multiple number of researches interview the same participant, but get different responses.
B) It is possible that researchers interpret the same participant's responses differently.
C) Intercoder reliability threat can occur only in qualitative research.
D) Intercoder reliability threat can be reduced with proper training and by utilizing research assistants who are motivated and conscience.

Question 31

__________ occurs when outliers move closer to the mean on subsequent testing.

A) Normal distribution
B) Statistical regression
C) Validity
D) Selection bias

Question 32

What is the simplest solution to statistical regression?

A) To remove outliers from the study or convert outliers to the mean value.
B) To allow research participants to retake the exam until they are satisfied with their result
C) To retrieve another sample
D) There is no solution to statistical regression.

Question 33

__________ occurs when the research participants in the experimental group are not similar to those in the control group.

A) Statistical regression
B) Selection bias
C) Mortality
D) Diffusion of treatment

Question 34

__________ occurs when research participants drop out of the study before it concludes.

A) Mortality
B) Diffusion of treatment
C) Compensatory rivalry
D) Experimenter expectancy

Question 35

The research participants in the control group may get the information on the treatment that the participants in the experimental group receive, which can contaminate the study outcome. This threat to internal validity refers to

A) mortality.
B) diffusion of treatment.
C) compensatory rivalry.
D) experimenter expectancy.

Question 36

__________ occurs when the research participants in the control group learn of special treatment given to the experimental group and alter their behavior.

A) Mortality
B) Diffusion of treatment
C) Compensatory rivalry
D) Experimenter expectancy

Question 37

Researchers can subconsciously and unintentionally make their research participants aware of the research hypotheses, which ultimately influences their behavior. This threat to internal validity refers to

A) compensatory rivalry.
B) experimenter expectancy.
C) halo effect.
D) Hawthorne effect.

Question 38

Which of the following is true regarding external validity?

A) External validity means the ability to generalize the current findings across types of persons, settings, and times.
B) Reactivity is a threat to external validity known as Hawthorne effect.
C) Another form of reactivity is the placebo effect.
D) All of the above

Question 39

The Graduate Record Examination (GRE) is argued to help predict students' potential for success in graduate programs. Similarly, the LSAT is used to predict law school applicants' potential for success in legal studies. GRE and LSAT are thought to have strong __________ validity.

A) internal
B) external
C) criterion
D) construct

Question 40

One's hair color is not a predictor of future athletic success because it lacks __________ validity.

A) external
B) criterion
C) construct
D) content

Question 41

__________ is the ability to measure what one is intending to measure.

Question 42

__________ is the ability to consistently produce the same results over time.

Question 43

__________ is a collection of questions to measure a research concept.

Question 44

__________ is a statistical procedure to measure scale reliability. It produces a numerical value indicating how good the scale is actually measuring what it intended.

Question 45

__________ occurs when two identical measures of the topic are taken on two separate occasions and the results are the same or when two separate researchers administer a test and obtain the same results.

Question 46

__________ is measured when research participants are given slightly different questions of the same concept.

Question 47

__________ addresses if a relationship between two variables is causal.

Question 48

__________ is the attempt to statistically account for all of the variables that could influence the variable of interest.

Question 49

__________ is the ability of a specific study outcome to be able to generalize to a larger population or setting.

Question 50

__________ in a research study is the external events that occur with potential to influence the findings.

Question 51

__________ is any changes in the respondent due to maturity, socialization, age, or education, which can influence the findings.

Question 52

__________ occurs when respondents become so familiar with the survey instrument that it influences the findings.

Question 53

__________ is a threat to the quality of a study when changes are made to the testing instrument.

Question 54

__________ is a threat to research when a respondent's scores move closer to mean values overtime.

Question 55

__________ are the extreme values that jeopardize a normal distribution of data.

Question 56

__________ is a threat to internal validity that occurs in sampling process when random assignment to research groups is not properly conducted.

Question 57

__________ is a threat to internal validity when original participants in a study drop out, die, or are otherwise excluded from the research project before the study concludes.

Question 58

__________ is a threat to internal validity when research participants learn too much information about the study.

Question 59

__________ is a threat to internal validity when control group participants learn of potential benefits of the experimental group's treatment and then alter their own behavior based o the information.

Question 60

__________ is a threat when a researcher's judgment, opinions, or own personal bias about participants influence the study outcome.

Question 61

__________ is a label given when respondents acknowledge that they are being studied and somehow alter their normal behavior or opinion.

Question 62

__________ is when a respondent is given a "make believe" treatment but believes the effects are real.

Question 63

__________ exclude participants and researchers from knowing which groups of respondents belong to the experimental group.

Question 64

__________ utilizes a well-established measure to compare to an alternative measure that together provides more predictability.

Question 65

__________ is when multiple measures of the constructs of a theory are related to one another in the expected direction.