Exam 4: Criteria for Inferring Effectiveness: How Do We Know What Works

A special unit in a child welfare agency provides an innovative treatment to those cases with the worst assessment scores. To evaluate the unit's effectiveness, the degree of improvement among its clients in assessment scores is compared to the degree of improvement among cases in other units. The evaluation finds that the cases in the special unit had a significantly higher average degree of improvement than did the cases in the other units. From this information we can conclude that:

History refers to the possibility that other events may have coincided with the provision of the intervention and may be the real cause of the observed outcome.

If a study has excellent internal validity and finds that an intervention is very effective, we can conclude that:

Statistical chance refers to flukes in random assignment that can be the reason why one group's outcome is better than another's.

The best way to avoid a selectivity bias is to:

As a threat to internal validity, history refers to:

When reading a report of a study that failed to get statistically significant results, you should take into account its sample size.

Immediately after experiencing a trauma, a client begins receiving trauma-focused therapy and continues receiving the intervention for 3 months. At the conclusion of treatment the client's trauma symptoms are much less severe than at the start of treatment. From this information we can conclude that:

A serious measurement bias can destroy the credibility of an otherwise rigorously designed randomized experiment.

A free, voluntary continuing education workshop in EBP is offered to practitioners in a large agency. Several months after the workshop is provided, a program evaluator finds that workshop attendees are much more likely to be engaging in EBP than those who did not attend. From this information we can conclude that:

Random assignment of clients to treatment conditions means the same as haphazard assignment to treatment conditions.

The external validity of a study pertains to whether the study participants, intervention procedures, and results seem applicable to the evidence-based practitioner's practice context.

Internal validity refers to:

If a study is internally valid we can generalize it to our practice context.

To conclude logically that an intervention really is the cause of a particular outcome:

Define external validity and describe its role in making EBP decisions.

Define internal validity, list the main threats to it, and explain each threat.

Explain the need to rule out statistical chance as a plausible explanation for study outcomes.

Which of the following statements is/are true about statistical significance?

Which of the following questions is/are important when critically appraising a study that purports to test the effectiveness of an intervention, program, or policy?