Deck 5: Critically Appraising Experiments

Question 1

Which of the following experimental designs is most vulnerable to the impact of testing effects?

A) Classic pretest-posttest control group design.
B) Posttest-only control group design.
C) Solomon four-group design.
D) All of the above.

Answer 1

A

Question 2

If the experimental and control groups both improve significantly, then we should conclude that:

A) the intervention and control conditions both were causes of the improvement.
B) the results do not support the notion that the intervention is effective.
C) some alternative explanation like history or the passage of time plausibly could have caused the improvement in both groups.
D) Both b and c are correct.

Answer 2

D

Question 3

Controlling for history in an experiment means that:

A) the possibility that contemporaneous events could affect things is eliminated.
B) all clients have had the same clinical histories.
C) we can logically ascertain whether contemporaneous events represent a plausible explanation for the differences in outcome between treatment and control groups.
D) None of the above.

Answer 3

C

Question 4

Which of the following statements is correct about random assignment in a posttest-only experimental design:

A) It never provides an adequate basis for supposing that both groups would have had very similar pretest scores.
B) It always provides an adequate basis for supposing that both groups would have had very similar pretest scores.
C) It provides an adequate basis for supposing that both groups would have had very similar pretest scores only when the sample size is large enough to permit such a supposition.
D) None of the above.

Question 5

Which statement is the most appropriate interpretation of the following results of a Solomon four-group design:

A) The treatment was effective, but there were also testing effects.
B) The treatment was effective, and there is no indication of testing effects.
C) The improvement appears to be attributable to testing effects.
D) None of the above.

Question 6

If both groups both improve significantly in the following alternative treatment groups design, then we should conclude that:

A) Both treatments were effective.
B) Neither treatment was effective.
C) Both treatments might be effective or ineffective because we cannot rule out some alternative explanation like history or the passage of time as a plausible cause of the improvement in both groups.
D) None of the above.

Question 7

What should we conclude regarding the effectiveness of treatments A and B on self-esteem if both improve significantly in the following alternative treatment groups design results?

A) Both treatments appear to be equally effective.
B) Neither treatment was effective.
C) Both treatments might be effective or ineffective because we cannot rule out some alternative explanation like history or the passage of time as a plausible cause of the improvement in both groups.
D) None of the above.

Question 8

What should we conclude regarding the effectiveness of EMDR in lowering trauma symptoms in light of the following dismantling design results?

A) EMDR appears to be effective, and the eye movement component appears to be a necessary component.
B) The eye movement component contributes nothing to the effectiveness of EMDR.
C) EMDR is not effective.
D) None of the above.

Question 9

Random assignment with a very large sample of clients to two different groups of practitioners, one of which provides an innovative intervention and the other of which provides treatment as usual, is sufficient for ruling out which of the following explanations for results showing that the experimental group's outcome was significantly better than the control group's outcome?

A) Lack of practitioner equivalence.
B) Differential attrition rates.
C) Obtrusive observation.
D) All of the above.
E) None of the above.

Question 10

Random assignment always removes the need for a pretest.

Question 11

The posttest-only control group design is always feasible regardless of the nature of the outcome being assessed.

Question 12

Random assignment to two alternative treatments removes the possible value of a control group regardless of the results.

Question 13

The classic pretest-posttest control group design controls for the possible impact of research reactivity.

Question 14

Extremely obtrusive observation can be a fatal flaw in an otherwise well-designed experiment.

Question 15

Some forms of research reactivity can make a very effective intervention appear to be less effective.

Question 16

Treatment diffusion can make a very effective intervention appear to be less effective.

Question 17

A lack of treatment fidelity can make a very effective intervention appear to be less effective.

Question 18

The classic pretest-posttest control group design controls for the possible impact of differential attrition.

Question 19

Diagram and explain the logic of six different experimental designs.

Question 20

Explain the implications of sample size regarding need for pretests.

Question 21

Provide examples of three different forms of research reactivity and explain why random assignment is not sufficient to control for them.

Question 22

Explain the difference between obtrusive and unobtrusive observation, and provide examples illustrating how each can be a matter of degree.

Question 23

Explain how differential attrition can impact the interpretation of findings in experiments.

Question 24

Explain the need for practitioner equivalence between alternative treatment groups in experiments.