Deck 14: Program Evaluation

Question 1

Agency administrators and practitioners are likely to encourage maximum scientific rigor in evaluations of program outcome,irrespective of the funding implications.

Answer 1

False

Question 2

In-house evaluators,as compared to external evaluators,are likely to have

A) greater access to program information and personnel.
B) more knowledge about program processes.
C) less independence from agency administrators.
D) all of these.

Answer 2

D

Question 3

Which of the following designs can be used in program evaluation?

A) Pre-experimental designs.
B) Group experiments.
C) Surveys.
D) Qualitative designs
E) All of these.

Answer 3

E

Question 4

In order to foster the eventual utilization of an evaluation,one of the steps evaluators should take is

A) make sure stakeholders are not involved in planning the evaluation.
B) ensure that program personnel do not influence the evaluation design.
C) involve stakeholders throughout all phases of the evaluation.
D) withhold the sharing of the evaluation report with stakeholders until it has been perfected and finalized.

Question 5

Agency administrators who ask in-house program evaluators to assess their program's effectiveness are likely to

A) promote an atmosphere in which the paramount priority is on free, scientific inquiry.
B) give the evaluator complete autonomy in designing the research and interpreting its findings.
C) desire a study that is methodologically as rigorous as possible.
D) none of these.

Question 6

Which of the following statements is NOT true about program evaluation?

A) It differs dramatically from most of the research done by social workers.
B) It refers to the purpose of research more than to any particular research methods.
C) It can apply to many diverse research designs and methods.
D) It can be used not only to assess the impact of human service programs but also to improve their implementation.

Question 7

Which of the following statements is NOT true about the utilization of program evaluation results?

A) They are less likely to be used if they contradict deeply held beliefs.
B) If the results are clear and have obvious policy implications, they will always be used.
C) They are less likely to be used when they threaten vested interests.
D) None of these; they are all true.

Question 8

Recommendations to maximize compliance with the case assignment protocol in experiments or quasi-experiments include

A) having agency staff implement the case assignment decisions.
B) NOT explaining to agency staff the purpose and nature of the case assignment protocol.
C) pilot testing the randomization procedure in the agency.
D) assigning control of the case assignment protocol to multiple agency staff members.

Question 9

Which of the following statements is/are true about logic models in program evaluation?

A) They graphically depict the essential components of s program.
B) They might be required by funding sources.
C) They can be constructed in various ways.
D) All of these

Question 10

Program evaluators should involve stakeholders in planning an evaluation.

Question 11

Videotaping treatment sessions should be used only as a last resort when assessing intervention fidelity.

Question 12

Which of the following statements is true about the history of program evaluation?

A) It has been widespread throughout the history of social welfare.
B) It became widespread during the late 1950s.
C) Interest in it waned during the 1960s.
D) It has been opposed by political conservatives throughout its history.

Question 13

Which of the following statements is true about pilot studies in program evaluation?

A) They can detect problems in carrying out the main study as intended.
B) They are useless until after a grant application is funded.
C) They increase the likelihood of imitation of treatment occurring in the main study.
D) They increase the chances of staff resistance to the main study.

Question 14

Evaluators who report negative findings about a program will usually be respected by stakeholders as competent and honest,provided that their methodology is reasonable.

Question 15

To avoid or alleviate practical pitfalls when carrying out experiments in social work agencies,researchers should:

A) Exclude agency staff members from the research planning phase.
B) Keep research staff members away from program staff during the study.
C) Avoid developing or using treatment manuals for the experimental or control conditions.
D) Refrain from recruiting clients on an ongoing basis throughout the study.
E) none of these.

Question 16

It is safe to assume that agency staff members who initially approve of an experiment will remain in close compliance with its research protocol throughout the study.

Question 17

External evaluators are always objective and independent of stakeholder influences in the way they design,carry out,and report their evaluations.

Question 18

Which of the following can be a purpose of program evaluation?

A) To assess program success.
B) To assess problems in program implementation.
C) To obtain information needed in program planning.
D) All of these.

Question 19

Which of the following statements is/are true about problems in client recruitment and retention in experimental and quasi-experimental evaluations in social work agencies?

A) Referring agencies might dump clients that they don't want to serve.
B) Referring agencies might be reluctant to refer any clients.
C) Referring agencies might resent having their referrals assigned to a control condition.
D) Clients may resent the use of randomized procedures to determine which service they receive.
E) Agency estimates of the number of likely referrals may be overly optimistic. f. All of these.

Question 20

Which of the following statements is/are true about carrying out experiments and quasi-experiments in social agencies

A) The intervention being tested might not be implemented in the intended fashion.
B) The control condition might be contaminated by interaction between clients in the experimental and control groups.
C) Practitioners might violate the research protocol for case assignment to make sure that clients receive the intervention being evaluated.
D) Reluctance by outside agencies to refer clients might hinder obtaining a sufficient sample size.
E) All of these.

Question 21

Discuss the value of pilot studies in program evaluation.

Question 22

If a rigorous experimental assessment of program outcome finds that a program is not effective in attaining its goals,then there is no need to evaluate the program's implementation.

Question 23

Quantitative and qualitative methods can be combined in program evaluations.

Question 24

The best way to avoid contaminating the control condition is to locate the experimental and control conditions in the same building

Question 25

Qualitative methods offer a number of techniques that on-site research staff members can use in attempting to observe research implementation pitfalls.

Question 26

Identify and explain four practical pitfalls commonly encountered when carrying out experiments and quasi-experiments in social agencies.

Question 27

Managed care companies tend to be disinterested in the outcomes of program evaluations.

Question 28

Describe the components of a logic model that might fit a program with which you are familiar.

Question 29

What steps should program evaluators take in order to foster the utilization of their studies?

Question 30

Describe the impact managed care has had on program evaluation.

Question 31

Discuss how qualitative methods can be used to try to avoid or alleviate the pitfalls identified above in essay number 5.

Question 32

What are the special problems that program evaluators encounter that other types of researchers are less likely to encounter?