Deck 10: Quasi-Experimental Designs

Question 1

Selection is a threat to the internal validity of which of the following designs?

A) Nonequivalent comparison group design
B) Interrupted time-series design
C) Regression-discontinuity design
D)All of the above

Answer 1

A

Question 2

Dr. Johannsen wants to test whether recovering alcoholics perform worse on tests of motor coordination than a group of control participants. Unbeknownst to him, the control group meets at an amusement park three hours before testing and goes on as many rides as possible. They perform worse than the alcoholic group. This is an example of an)

A) regression effect.
B) selection-maturation effect.
C) regression artifact.
D) selection-history effect.

Answer 2

D

Question 3

The nonequivalent comparison group design can yield several possible outcomes. In one scenario, the experimental group scores higher than the control group at the start of the experiment, and only the experimental group's scores increase from pretesting to posttesting. This could reflect an influence of the independent variable. However, it could also reflect

A) a local instrumentation effect.
B) a attrition effect both groups.
C) simply that the experimental group was studied for a longer time period than the control group.
D) selection-maturation effect.

Answer 3

D

Question 4

When one does not have the ability to , one may consider using quasi-experimental designs.

A) measure the dependent variable quantitatively
B) randomly assign participants to groups
C) manipulate the independent variable
D) make multiple measures of the dependent variable

Question 5

When using a quasi-experimental design, rival hypotheses can sometimes be ruled out by

A) eliminating the pre-test
C) eliminating the IV
C) eliminating the control group
D) making a complex prediction

Question 6

The nonequivalent comparison-group design does NOT have which of the following?

A) Manipulation of the independent variable
B) Random assignment of participants to the groups
C) A pretest
D) A posttest

Question 7

You have been asked to evaluate the effectiveness of a new anti-smoking campaign in a local high school. You are able to collect pretest information on smoking rates before the campaign begins. You know that the entire school will be required to participate in the campaign and so random assignment is impossible.You are also aware that a reduction in smoking after the campaign could be caused by any number of other confounding variables.What can you do in this situation to enhance your ability to determine if the campaign was effective in reducing smoking?

A) test the students at the end of the campaign and then again a year later - if reductions in smoking remain at that time then you can be confident that the campaign was effective
B) conduct extensive postexperimental interviews to determine the cause of any change in smoking rates
C) identify a similar school that is not using the campaign and include them in a nonequivalent comparison group design
D) interview parents of the students

Question 8

The non-equivalent comparison group design is a quasi-experimental design in which, for reasons of practicality, we cannot insure that the control and experimental groups are equivalent to each other when the experiment begins. The major interpretational difficulty imposed by this design is

A) knowing whether the two groups are different from each other on the dependent measure once the experiment is complete.
B) deciding how much each group has to gain on the posttest compared to the pretest to be sure that the differences are reliable.
C) determining when we have collected enough data points to make a statement about the experiment's outcome.
D) being sure that any differences between groups at the end of the experiment result from the independent variable's influence alone, and are not related to preexisting differences.

Question 9

Which of the following designs is the "best" in terms of eliminating potential confounds?

A) nonequivalent comparison group
B) posttest-only design with nonequivalent groups
C) pretest-posttest control-group design
D) interrupted time series design

Question 10

What is the main distinction between the nonequivalent comparison group design and the pretest-posttest control group design?

A) the use of pretesting in the latter
B) the use of statistical data analysis in the latter
C) the random assignment of participants to groups in the latter
D) more than one independent variable in the latter

Question 11

The internal validity threat known as "history" is a problem for which of the following designs?

A) Nonequivalent comparison group design
B) Interrupted time-series design
C) Regression-discontinuity design
D) None of the above

Question 12

One strategy for dealing with preexisting differences between experimental and control groups in the nonequivalent comparison group design is to match the participants.

A) this procedure is as effective as randomly assigning participants from the start
B) this procedure eliminates local history as a confounding factor
C) this procedure, however, may introduce a regression-artifact phenomenon whenextreme scores are frequently needed to match
D) this procedure, however, confounds the research even more than the preexisting differences

Question 13

The selection-history effect is particularly a problem in which quasi-experimental design?

A) non-equivalent comparison group
B) time series
C) interrupted time series
D) regression discontinuity

Question 14

One essential characteristic of quasi-experimental designs that distinguishes them from strongexperimental designs is a lack of

A) volunteer participants.
B) control of extraneous variables.
C) experimenters who can conduct them.
D) an interest in laboratory experimentation.

Question 15

In which of the following circumstances would a researcher be most likely to choose a quasi- experimental research design?

A) when there are more than three important confounding variables to control
B) when doing research in an applied or natural setting where control is difficult to impose
C) when controlling experimenter and participant expectation effects would involve an unacceptable level of deception
D) when it is difficult to recruit enough participants to make standard statistical testing valid

Question 16

Quasi-experimental designs lack the controls of strong experimental designs.

A) however, causal inferences can be made to the extent that rival hypotheses can be ruled out
B) as such, causal inferences should never be made
C) however, in most cases such deficiencies can be safely ignored
D) thus, such studies should not be performed by ethical researchers

Question 17

The most common reason for the use of quasi-experimental research designs is that

A) the participants are maturing too rapidly.
B) the dependent variable cannot be measured reliably.
C) it is unethical to manipulate the independent variable .
D) participants cannot be randomly assigned to groups.

Question 18

In an attempt to reduce the use of gasoline, a city imposed a $2.50 per gallon tax on gasoline in 2005. A researcher working for the city, located data on annual gallons used for the 5 years leading up to the gas tax to provide a baseline) and located data for the 5 years after the start of the gasoline tax to show the consumption pattern after the tax). Which of the following best describes this study design?

A) A-B-A-B design
B) Changing-criterion design
C) Interrupted time-series design
D) Regression-discontinuity design

Question 19

One "additive/interaction threat" to the nonequivalent comparison-group design might occur is one group of participants becomes more experienced, tired, or bored than those in the other group. What is this threat?

A) Selection-attrition
B) Selection-maturation
C) Selection-instrumentation
D) Selection-regression

Question 20

A researcher is carrying out a quasi-experimental study. She has the experimental groups in one lab and a control group in another lab. During the intervention phase of her study, the control school has a small fire that forces participants to leave the lab for one hour. She finds huge differences between the experimental and control groups after the study is completed. While there could be an effect of her intervention, a possible rival hypothesis might be:

A) There was a maturation effect
B) There was an instrumentation effect
C) There was a differential history or selection-history effect
D) There was a selection-maturation effect

Question 21

Dr. Toris wants to test an incentive program to increase attendance in her classes. For the first 8 weeks of the semester she does not use the program, and records attendance rates each week. Then she institutes the incentive program for the last 8 weeks and continues to record attendance rate each week. She detects an increase in attendance over the last 8 weeks. This is an example of which experimental design?

A) one group pretest-posttest
B) interrupted time series
C) non-equivalent control group
D) multiple baseline

Question 22

Dr. Hunter is concerned about pre-test differences in her nonequivalent comparison group design. What could she do to minimize pre-test differences without changing to a strong experimental design?

A) let students self select into groups
B) match participants in variables closely related to the IV
C) eliminate the pre-test
D) there is nothing that can be done to control for pre-test difference

Question 23

Visual inspection of the pre- and post-treatment pattern of dependent measure scores is typical of which quasi-experimental design?

A) interrupted time series
B) non-equivalent control group
C) before-after
D) factorial

Question 24

Of the major extraneous variables discussed in this chapter, which is most likely a source of confounding in an interrupted time-series design?

A) participant expectations
B) history
C) attrition
D) regression

Question 25

Making a causal inference from quasi-experimental evidence requires all but which of the following:

A) the suspected cause must covary with the effect
B) the suspected cause must precede the effect
C) rival hypotheses must be ruled out or be highly implausible.
D) Bayesian moving average statistical analyses.

Question 26

Which of the following threats to internal validity is most likely to be a problem in the nonequivalent comparison group design?

A) history
B) maturation
C) selection bias
D) all of the above are potential problems

Question 27

When using the nonequivalent comparison group design, you can increase internal validity by

A) allowing students to self select into groups
B) allowing parents to decide which group their children will be in
C) not allowing students to self select into groups
D) not pre-testing participants

Question 28

Sometimes it is difficult if not impossible to find a control group that is sufficiently comparable to an experimental group to allow for a traditional two-group experimental design. In these cases, one may want to use a time-series design

A) which is a quasi-experimental design involving multiple measures of the dependent variable on a single group, both before and after treatment.
B) which is a strong experimental design involving multiple measures of the dependent variable on a single group, both before and after treatment.
C) which is an experimental design using multiple dependent measures on the experimental group, then taking measures on control participants over time, as he or she can find them.
D) staggers the time of presentation of the independent variable across participants.

Question 29

In the physical education department, Dr. Shanz asks two of his strength training classes to participate in an experiment. One class will follow a traditional program of index finger strength enhancement, while the other class will try a new, experimental procedure. At the beginning of the semester, the class that will use the experimental method starts out with, on average, weaker index fingers than the other class. At the end of the semester, the class using the traditional method shows no significant increase in finger strength, while the other experimental class now has fingers that are, on average, significantly stronger than those of the control class. This is an example of an) effect.

A) interaction
B) crossover
C) intragroup regression
D) selection-maturation

Question 30

Differential drop out rates in your experimental and control groups selection-attrition) is most likely to be a problem in which of the following designs:

A) time series
B) regression discontinuity
C) pretest-posttest control group
D) nonequivalent comparison group

Question 31

In a nonequivalent comparison group study,

A) the experimental group gets a pretest, then the experimental treatment, then a posttest.
B) there is no control group.
C) to interpret the results, the pretest scores have to be identical for the two groups.
D) random assignment to groups is common.

Question 32

The most common of the quasi experimental designs is the nonequivalent comparison group design. What is the dependent variable most commonly used in this design?

A) latency measure
B) difference or change scores posttest - pretest)
C) number or percent correct
D) total correct

Question 33

The presence of a crossover effect in one's data can usually rule out the influence of which rival hypothesis?

A) history
B) regression-artifact
C) local history
D) attrition

Question 34

Which of the following is NOT a bias that commonly exits in nonequivalent comparison group designs?

A) selection bias
B) selection-attrition bias
C) selection-regression bias
D) testing bias

Question 35

In a study comparing two nonequivalent groups, a selection-history confound occurs when

A) some event intervenes between pretest and posttest and affects just one of the groups.
B) some event intervenes between pretest and posttest and affects both selected groups equally.
C) selection influences one of the groups and history influences the other.
D) some historical event causes a failure of the random assignment procedure.

Question 36

The nonequivalent comparison group design can yield several possible outcomes. A possible outcome of the non-equivalent comparison group design is that the control group scores higher than the experimental group on the pretest and posttest but the experimental group improves more on posttesting. This could reflect the fact that the independent variable had an effect on the experimental group. It could also reflect

A) an unreliable dependent measure.
B) the fact that the control group experienced less of the independent variable than the experimental group did.
C) regression-artifact effect if the experimental group had been selected for their low initial scores.
D) a selection-maturation effect if the experimental group had been selected for their older age.

Question 37

The nonequivalent comparison group design can yield several possible outcomes. A possible outcome of the non-equivalent comparison group design is that the groups differ on the pretest and both groups improve on the posttest, but to different degrees. This could reflect the fact that the independent variable had an effect on the experimental group. It could also reflect

A) an unreliable dependent measure.
B) the fact that the control group experienced less of the independent variable than the experimental group did.
C) a participant-expectation effect.
D) a selection-maturation effect.

Question 38

Dr. Kaiser, a sports psychologist, wants to know if he can improve the cardiovascular endurance in elementary school children. He is allowed to study two physical education classes at a local school, one boys' class and one girls' class. On the pretest, the girls on average have slightly better cardiovascular endurance as measured on the step test. He administers the year- long aerobic program to the girls' class. The posttests revealed that the girls' cardiovascular endurance improved much more than the boys.' Dr. Kaiser must now decide if the program really worked or

A) if there was a regression artifact.
B) if there was a selection-maturation effect.
C) if girls always do better in aerobic exercises.
D) if there was an attrition effect.

Question 39

Imagine a hypothetical study using a nonequivalent comparison group designed to assess the effectiveness of an anti-drug, "Just Say No," campaign. In this study students at school X receive a semester-long series of "Just Say No" lectures. Students at school Y across town do not receive the intervention. Unfortunately, during the semester a student at school Y died after an accidental drug overdose. This incident may have influenced drug use at school Y but not school X) and consequently make it impossible to interpret our research findings. Which of the following threats to internal validity is present in this study?

A) selection-instrumentation
B) selection-maturation
C) selection-history
D) selection-testing

Question 40

The is to as nonequivalent comparison group design is to posttest-only design with nonequivalent groups.

A) one group pretest-posttest design; interrupted time series design
B) interrupted time series design; one group pretest-posttest design
C) pretest-posttest control group design; interrupted time series design
D) interrupted time series design; regression discontinuity design

Question 41

Which of the following is NOT one of the requirements of the regression-discontinuity design.

A) group assignment must be based on a cutoff score
B) the cutoff score should be located at one of the extreme ends of the distribution
C) the experimenter must control the assignment of participants to groups
D) all participants must be from the same population

Question 42

A small town decides to ban cell phone texting while driving. Before putting the law into effect, officials measure the amount of cell phone use among drivers by placing observers at a busy intersection and recording the percentage of drivers using phones. They do this over 6 consecutive one-week intervals, each week being the middle week of a month i.e., six months of pretesting). Right after the law is passed, officials repeat the observational study for another six months. Which of the following is true about the design of this study?

A) it is an interrupted time series design
B) it is an example of a nonequivalent control group design
C) without a control group, there is no way to evaluate trends with this design
D) because there are multiple observations this would qualify as a regression discontinuity design

Question 43

In a study designed to examine the effects of an intensive reading program, Vaughn et. al 2009) screened 2nd grade students and separated good readers from poor readers. The poor readers were given the intervention treatment. At the end of 26 weeks of intervention significant improvement in the poor readers was noted. What type of experimental design was employed in this study?

A) regression discontinuity
B) nonequivalent comparison group
C) interrupted time series
D) median split design

Question 44

What is the "interruption" in an interrupted time series design?

A) introduction of the independent variable
B) instrumentation failure
C) a period of at least two weeks
D) a brief period when measurements are taken.

Question 45

If you have conducted a study using the regression-discontinuity design, how would you tell if your treatment had an effect?

A) by looking at the data to see if the regression line of the posttest scores differs from the regression line of pretest scores.
B) by statistically analyzing the data using Analysis of Variance.
C) by comparing the pretest scores with the posttest scores to see if there was a significant change.
D) by determining if there is a discontinuity in the regression line for the people above and below the cutoff score.

Question 46

In using an interrupted time-series design, how does one determine whether the independent variable had an effect?

A) by looking for changes in the response pattern from pre- to post-treatment
B) by using the chi-square statistic
C) by comparing pre-treatment mean performance to post-treatment mean performance
D) by comparing the treatment group's data to a single observation of a control group

Question 47

In an) design, the experimenter takes multiple measures on the dependent variable both before and after treatment.

A) non-equivalent control group
B) interrupted time series
C) multivariate
D) before-after

Question 48

What is the primary threat to internal validity that could confound the results of a regression discontinuity study?

A) maturation
B) selection-history
C) attrition
D) testing

Question 49

The National Science Foundation has a policy of, each year, selecting the best young scientists and giving them a 5 year grant to allow them to work on their program of research. These scientists are selected based on the number of publications they have produced in the prior year. All scientists who have published 4 or more articles in the prior years receive these grants. The National Science Foundation wants to find out if this program is effective. The research design that could be used to best test the effect of this program is

A) posttest-only design.
B) regression-discontinuity design.
C) one group pretest-posttest design.
D) pretest-posttest design.