Deck 14: Program Evaluation

Question 1

Evaluators who report negative findings about a program will usually be respected by stakeholders as competent and honest,provided that their methodology is reasonable.

Answer 1

False

Question 2

In order to foster the eventual utilization of an evaluation,one of the steps evaluators should take is to

A) make sure stakeholders are not involved in planning the evaluation.
B) ensure that program personnel do not influence the evaluation design.
C)involve stakeholders throughout all phases of the evaluation.
D) withhold the sharing of the evaluation report with stakeholders until it has been perfected and finalized.

Answer 2

C

Question 3

Agency administrators who ask in-house program evaluators to assess their program's effectiveness are likely to

A) promote an atmosphere in which the paramount priority is on free, scientific inquiry.
B) give the evaluator complete autonomy in designing the research and interpreting its findings.
C) desire a study that is methodologically as rigorous as possible.
D) imply that they expect the results of the evaluation to be positive.

Answer 3

D

Question 4

If a rigorous experimental assessment of program outcome finds that a program is not effective in attaining its goals,then there is no need to evaluate the program's implementation.

Question 5

In order to maximize scientific objectivity,program evaluators should avoid involving stakeholders in planning an evaluation.

Question 6

A major disadvantage of the community forum approach to needs assessment is the representativeness of the opinions expressed.

Question 7

Agency administrators and practitioners are likely to encourage maximum scientific rigor in evaluations of program outcome,irrespective of the funding implications.

Question 8

Which of the following is true about the impact of managed care on program evaluation?

A) It has reduced the emphasis on program evaluation.
B) It has reduced the pressure on programs to measure their outcomes.
C) It has reduced the extent to which vested interests are involved in program evaluations.
D) It has increased the extent to which vested interests are involved in program evaluations.

Question 9

One criticism of goal attainment evaluations deals with the often fuzzy link between program mission statements and operational indicators of program outcome.

Question 10

External evaluators may not be objective and independent of stakeholder influences in the way they design,carry out,and report their evaluations.

Question 11

An importanactor to consider in planning for the eventual feasibility and utilization of an evaluation is whether commitment to all aspects of the evaluation is obtained in advance from intended information users and other stakeholders.

Question 12

Because the rates under treatment approach to needs assessment only looks at those currently utilizing services,it is vulnerable to underestimating the true extent of need.

Question 13

Which of the following statements is NOT true about the utilization of program evaluation findings?

A) They are less likely to be used if they contradict deeply held beliefs.
B) If the results are clear and have obvious policy implications, they will always be used.
C) They are less likely to be used when they threaten vested interests.
D) How they are written up rarely has any bearing on their potential utilization.

Question 14

Agency staff members who initially approve of a program evaluation design may nevertheless fail to comply closely with its research protocol throughout the study.

Question 15

External evaluators,as compared to In-house evaluators,are likely to:

A) have less access to program information and personnel.
B) have more knowledge about program processes.
C) have less independence from agency administrators.
D)experience no pressure whatsoever from stakeholders regarding desired results.

Question 16

Which of the following statements is true about the goal attainment model of program evaluation?

A) It emphasizes qualitative methods and deemphasizes experimental and quasi-experimental designs.
B) One problem is thaormal goals are often stated in grandiose and vague terms.
C) No more than one operational indicator of success should be measured.
D) Cost-effectiveness analysis and cost-benefit analysis both attempt to monetize program outcomes.

Question 17

Which of the following statements is true about evaluation for program planning?

A) A prime advantage of the key informants approach to needs assessment is that the information comes directly from the target population.
B) It makes a difference whether we define needs in normative terms or in terms of demand.
C) Feasibility problems are the prime disadvantage of the community forum approach to needs assessment.
D) A prime disadvantage of the social indicators approach to needs assessment is its obtrusiveness.

Question 18

Which of the following statements is/are true about program evaluation?

A) It can be used only to assess a program's effectiveness.
B) It refers to only one particular research method.
C) It can apply to many diverse research designs and methods.
D) It can be used only to assess a program's implementation.

Question 19

Which of the following statements is true about monitoring program implementation?

A) If an outcome evaluation is well designed, there is no need to monitor program implementation.
B) Some programs have unsuccessful outcomes simply because they are not implemented properly.
C) If we find that a program has not been implemented properly, all the more reason to evaluate its outcome.
D) Whereas process evaluations are usually applied when evaluating outcome, experiments are most typically associated with monitoring program implementation.

Question 20

Which of the following statements is true about logic models?

A) They portray program components graphically.
B) They are never theory-based.
C) Funding sources almost never require them as part of grant applications.
D) They exclude attention to program outcomes.

Question 21

Discuss and provide an example of the value of the value of qualitative approaches to program evaluation.

Question 22

Focus groups are an example of a qualitative approach to needs assessment.

Question 23

Describe the impact managed care has had on program evaluation.

Question 24

What are the special problems that program evaluators encounter that other types of researchers are less likely to encounter?

Question 25

Cost-benefit techniques enable evaluators to put a true dollar value on all possible program costs and benefits,without respect to vested interests and value judgments.

Question 26

The extent of need for a program or service always is accurately gauged by the amount of expressed demand for that program or service.

Question 27

What are the advantages and disadvantages of evaluating program outcome and efficiency,as compared to monitoring program implementation?

Question 28

Why might the increasing importance thaunding sources attach to program evaluation be a mixed blessing in terms of the way politics can affect program evaluations?

Question 29

What steps should program evaluators take in order to foster the utilization of their studies?

Question 30

Describe five different approaches to needs assessment and their advantages and disadvantages.