Deck 7: From Manipulative to Analytic Control: Quasi-Experimentation

Question 1

Which of the following people played a significant role in encouraging researchers to reconsider viewing methods in a hierarchical manner?

A) Donald T. Campbell
B) John Stuart Mill
C) Ronald F. Campbell
D) Abraham Ribicoff

Answer 1

A

Question 2

Which of the following was one of the first examples demonstrating the logic of quasi-experimentation.

A) evaluation of the 1990's opioid crackdown in Indiana
B) evaluation of the 1970's crack epidemic in New York
C) evaluation of the 1950's speeding crackdown in Connecticut
D) evaluation of the 1920's bootlegging crackdown in Louisiana

Answer 2

C

Question 3

______ is a time series of a particular outcome of interest that is used to establish an underlying trend, which is "interrupted" by an intervention at a known point in time.

A) Multiple time-series design
B) Time-series design
C) Temporal precedence
D) Interrupted time-series design

Answer 3

D

Question 4

______ is a quasi-experimental design where measurements are taken from two groups of test units. The experimental group is exposed to a treatment/independent variable, and then another series of measurements is taken from both groups.

A) multiple time-series design
B) time-series design
C) temporal precedence
D) interrupted time-series design

Question 5

______ control involves the ability to create situations and randomly assign people to groups; ______ control involves using our heads to anticipate and address all rival plausible explanations.

A) Manipulative control; experimental control
B) Analytical control; experimental control
C) Manipulative control; analytical control
D) Manipulative control; quasi-experimental control

Question 6

Julie has designed an experiment that allows her to randomly assign people to one of two groups. What type of control does Julie have?

A) manipulative control
B) analytical control
C) experimental control
D) quasi-experimental control

Question 7

Nia is conducting a study with intact groups; thus, she needs to anticipate and address all rival plausible explanations. What type of control does Nia need?

A) manipulative control
B) analytical control
C) experimental control
D) quasi-experimental control

Question 8

Jessie would like to evaluate the academic performance of students in two different high schools. Jessie cannot randomly assign students to high schools, so how can Jessie minimize rival plausible explanations?

A) manipulative control
B) analytical control
C) experimental control
D) quasi-experimental control

Question 9

A situation in which you cannot randomly assign people, you should look for ______ to make reasonable guesses about plausibility.

A) waitlist control groups
B) analytical control groups
C) quasi-experimental control groups
D) "nonequivalent" control groups

Question 10

______ involve potential participants who are on a waitlist for a service or treatment can be used as a control group to evaluate the effectiveness of that service or treatment.

A) Waitlist control groups
B) Analytical control groups
C) Quasi-experimental control groups
D) "Nonequivalent" control groups

Question 11

A dentist wants to try a new treatment plan for gingivitis but can only offer it to a certain number of people. He wants to use the remaining eligible people as a control until he can give them the treatment later. What is this control group called?

A) waitlist control groups
B) analytical control groups
C) quasi-experimental control groups
D) "nonequivalent" control groups

Question 12

Lily is attempting to compare the impact on new drug laws on crime rates in Colorado by comparing it to states that have not adapted similar laws. However, she is finding it hard to find a comparison state. Which internal validity concern is likely the cause?

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 13

The ______ acts to minimize the groups' separation and heighten their similarity.

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 14

The ______ occurs when the treatment or program being evaluated involves goods and/or services considered desirable and where there is a large disparity between groups.

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 15

If members in the less desirable group know that they are in the disadvantaged group what internal validity threat may occur?

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 16

Celia is conducting a study comparing the athleticism of high school athletes to members of the band through a game of tug-a-war. The band is aware that they are the disadvantaged group, and this causes them to win the game. What threat to validity may have caused this win?

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 17

The ______ is named after an instance where a person outperformed a steam drill when they learned that their performance was to be compared to it.

A) John Stuart Mill Effect
B) Compensatory Rivalry Effect
C) John Henry Effect
D) John Henry Compensatory Rivalry Effect

Question 18

______ may occur if those in the control group perceive the result as adverse and inevitable.

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) diffusion of treatment

Question 19

A local group of dancers know that their ability to learn and replicate dance moves are being compared to the nationally renowned Jabbawockeez. The local group does not put forth their usual effort and does very poorly. What threat to internal validity likely caused this?

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) mortality

Question 20

______ occurs when some individuals drop out of the research before it's completed.

A) Compensatory equalization
B) Compensatory rivalry
C) Resentful demoralization
D) Mortality

Question 21

Single-session, short-term laboratory experimentation virtually precludes ______ as a problem. But in the field, where time-series data and a succession of follow-ups are more likely, this problem increases in relevance.

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) Mortality

Question 22

Alicia is conducting a study on marathon runners by interviewing them at multiple points during the marathon, however, some individuals she interviewed leave the marathon without completing. What threat to internal validity is this?

A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) mortality

Question 23

According to Weiss, which of the following is NOT one of the three major areas of political consideration should evaluation researchers be aware of in their research?

A) Programs are political creatures.
B) the personal stance of the evaluator
C) politics of the decision-making process
D) the political stance of the evaluation itself

Question 24

______ is the view that a better world can only be created by empirically driven evaluators.

A) Empirical morality
B) Resentful demoralization
C) Evaluation imperialism
D) Empirical imperialism

Question 25

Weiss offers three pieces of advice for dealing with politics in evaluation research. Which of the following is NOT included in her advice?

A) put program goals "in sensible perspective"
B) minimize the involvement of politicians
C) evaluate a particularly strong version of the program before, or along with, the evaluation of the ordinary levels at which it functions"
D) do something other than evaluation research

Question 26

The three criteria of causality should not be considered when assessing quasi-experimentation.

Question 27

Rival plausible explanations are not a concern in traditional laboratory experimentation.

Question 28

When we cannot randomly assign people, we should start looking for "nonequivalent" control groups to allow us to make "reasonable guesses."

Question 29

Time-series designs should not be used in quasi-experimentation.

Question 30

In traditional laboratory research, it is easy to isolate experimental and control groups from each other.

Question 31

Quasi-experimentation does not threaten internal validity.

Question 32

Mortality refers to a situation in which some individuals drop out of the research before it's completed

Question 33

Evaluation researchers should be aware of political considerations.

Question 34

Compare and contrast how rival plausible explanations are eliminated in traditional laboratory experimentation compared to in quasi-experimentation.

Question 35

Summarize and distinguish between the five threats to internal validity relevant to quasi-experimentation.

Question 36

Identify the key differences between manipulative control and analytic control and discuss when each approach should be used.

Question 37

Summarize what it means to have a John Henry effect in your research and discuss how it may threaten internal validity.

Question 38

Consider the logic of quasi-experimentation and discuss the advantages and disadvantages to the use of quasi-experimentation. Provide an example of a study in which quasi-experimentation should be used.

Question 39

Consider the politics of evaluation research and discuss how and why politics should be considered in evaluation research. Now imagine that your local government wants to implement a basic income of $1,000 per month for citizens earning less than $200,000 per year and they want you to evaluate the programs efficacy. Discuss the following:
a. name the independent variable in this study.
b. propose a possible dependent variable for this study.
c. What are two rival plausible explanations?
d. evaluate the study's internal validity, identify what threat(s) to internal validity you believe might be present, and state how you might safeguard the study to control for those threats.
e. discuss the potential use of time-series data and why it may (or may not) be helpful to this study.
f. discuss the role of politics in this process