Question 1

If unsuccessful program innovations are not identified,

Accepted Answer

A)  others are more likely to implement ineffective programs. 
B)  no one will forget them. 
C)  a Type I error has occurred. 
D)  implementation problems may have caused the failure. 
A)  others are more likely to implement ineffective programs. 
B)  no one will forget them. 
C)  a Type I error has occurred. 
D)  implementation problems may have caused the failure. A

Question 2

Although some anxiety about evaluation is normal, a high level of evaluation apprehension can lead to ineffective reactions to evaluations that are not totally favorable. Describe some of the fears and show how they lead to problems between evaluators and staff members.

Accepted Answer

Evaluation apprehension can lead to a variety of fears for staff members. Some common fears include the fear of being judged or criticized, the fear of not meeting expectations, the fear of negative consequences such as a poor performance review or even job loss, and the fear of being compared unfavorably to colleagues. These fears can lead to problems between evaluators and staff members in a few different ways.

First, staff members who are highly apprehensive about evaluation may become defensive or closed off during the evaluation process. They may be less likely to engage in open and honest communication with their evaluators, which can hinder the effectiveness of the evaluation and make it difficult for the evaluator to accurately assess the staff member's performance.

Second, high levels of evaluation apprehension can lead to increased stress and anxiety for staff members, which can negatively impact their performance. This can create a self-fulfilling prophecy, where the staff member's fear of a negative evaluation actually leads to a poorer performance, reinforcing their fears and creating a cycle of anxiety and underperformance.

Finally, evaluation apprehension can also lead to strained relationships between evaluators and staff members. Staff members who feel highly anxious about evaluation may perceive their evaluators as overly critical or unfair, leading to feelings of resentment and mistrust. This can make it difficult for evaluators to provide constructive feedback and support for staff members, and can ultimately damage the working relationship between the two parties.

Overall, high levels of evaluation apprehension can lead to ineffective reactions to evaluations and create barriers to open communication, performance improvement, and positive working relationships between evaluators and staff members. It is important for both parties to be aware of these fears and work together to create a supportive and constructive evaluation process.

Question 3

What are the steps that evaluators can take to encourage others to take to reduce the fear of evaluation?

Accepted Answer

To reduce the fear of evaluation among those being evaluated, evaluators can take several steps to create a more positive and constructive experience. Here are some strategies that can be implemented:

1. **Clarify the Purpose**: Clearly communicate the purpose of the evaluation. Ensure that individuals understand that the evaluation is meant to be a tool for growth and improvement, not just a judgment of their performance.

2. **Involve Stakeholders in the Process**: Engage those being evaluated in the design and implementation of the evaluation process. This can help them feel more in control and less threatened by the evaluation.

3. **Build Trust**: Establish a relationship of trust with those being evaluated. Be transparent about the evaluation criteria and process. Trust can reduce fear and encourage openness to feedback.

4. **Focus on Strengths**: While it's important to identify areas for improvement, also highlight the strengths and achievements of those being evaluated. This can help build confidence and reduce anxiety.

5. **Provide Support and Resources**: Offer resources, training, or coaching to help individuals improve in areas identified during the evaluation. Knowing that support is available can alleviate fears of being left to deal with weaknesses alone.

6. **Use a Collaborative Approach**: Encourage a dialogue rather than a one-way communication. Allow those being evaluated to ask questions, provide context, and discuss their perspectives on the evaluation findings.

7. **Ensure Fairness and Consistency**: Apply the same standards and procedures to all individuals being evaluated. Inconsistencies can lead to mistrust and increased fear.

8. **Offer Constructive Feedback**: Provide feedback that is specific, actionable, and focused on behavior or outcomes rather than personal attributes. This helps individuals understand what they can change or improve.

9. **Promote a Growth Mindset**: Encourage a culture that values learning and development. Emphasize that everyone has the potential to grow and that setbacks are part of the learning process.

10. **Follow Up**: After the evaluation, check in with those evaluated to see how they are progressing. Offer additional support if needed and acknowledge improvements and efforts to address feedback.

11. **Celebrate Successes**: Recognize and celebrate improvements and achievements that result from the evaluation process. This can reinforce the value of the evaluation and reduce fear over time.

12. **Confidentiality**: Ensure that evaluation results are handled with confidentiality. This can reduce the fear of being negatively judged by peers or superiors.

By taking these steps, evaluators can help create a more positive evaluation experience that encourages personal and professional development rather than fear.

Question 4

When an evaluator fails to document successful achievement of outcome goals, but the innovation seems to makes sense anyway, evaluators

Accepted Answer

A)  can speak of the innovation having "common sense plausibility" and urge that further evaluations be conducted. 
B)  can question the integrity of those who firmly supported it. 
C)  should immediately raise the possibility of a Type I error. 
D)  should actively try to persuade program supporters to change their attitudes. 
A)  can speak of the innovation having "common sense plausibility" and urge that further evaluations be conducted. 
B)  can question the integrity of those who firmly supported it. 
C)  should immediately raise the possibility of a Type I error. 
D)  should actively try to persuade program supporters to change their attitudes.

Question 5

Given the limited development of the science of program evaluation, weaknesses in every study could lead to all results being questioned. It is indefensible to use this fact to

Accepted Answer

A)  explain away all program weaknesses identified by an evaluation. 
B)  call for further evaluation. 
C)  work closely with stakeholders to interpret the results. 
D)  suggest improvement in further evaluations. 
A)  explain away all program weaknesses identified by an evaluation. 
B)  call for further evaluation. 
C)  work closely with stakeholders to interpret the results. 
D)  suggest improvement in further evaluations.

Question 6

The conclusions of evaluation reports should seldom

Accepted Answer

A)  deal with failure. 
B)  give support for a program. 
C)  give blanket approval or disapproval of a program. 
D)  be based on specific data. 
A)  deal with failure. 
B)  give support for a program. 
C)  give blanket approval or disapproval of a program. 
D)  be based on specific data.

Question 7

Program managers often look for a simple, definitive conclusion from an evaluation (e.g. good/bad), but many evaluators prefer to deal with the findings as working hypotheses on which to base

Accepted Answer

A)  funding decisions. 
B)  research articles. 
C)  staff salary increases. 
D)  program improvement efforts. 
A)  funding decisions. 
B)  research articles. 
C)  staff salary increases. 
D)  program improvement efforts.

Question 8

Strongly polarized attitudes toward a policy, such as limitations on handgun ownership, will result in people who

Accepted Answer

A)  are able to find common ground with each other to draw reasoned conclusions from an evaluation of the policy. 
B)  are eager for the evaluation findings. 
C)  trust the evaluator seeking to remain neutral on the issue. 
D)  are likely to select those findings of the evaluation that support their views and reject opposing ideas. 
A)  are able to find common ground with each other to draw reasoned conclusions from an evaluation of the policy. 
B)  are eager for the evaluation findings. 
C)  trust the evaluator seeking to remain neutral on the issue. 
D)  are likely to select those findings of the evaluation that support their views and reject opposing ideas.

Question 9

Sometimes an innovative program will fail to do better than currently used approaches,

Accepted Answer

A)  but the new approach may still have certain positive features (e.g. lower cost) that suggest it be adopted anyway. 
B)  but the possibilities of this are extremely small. 
C)  and the evaluator can expect to see recommendations carried out with little hesitation. 
D)  but for the sake of the morale of the staff, this failure should be ignored or explained away. 
A)  but the new approach may still have certain positive features (e.g. lower cost) that suggest it be adopted anyway. 
B)  but the possibilities of this are extremely small. 
C)  and the evaluator can expect to see recommendations carried out with little hesitation. 
D)  but for the sake of the morale of the staff, this failure should be ignored or explained away.

Question 10

If program staff members are worried about the findings of an evaluation,

Accepted Answer

A)  they will resist adopting ideas for change contained in an evaluation report. 
B)  they are likely to grasp at any suggestion. 
C)  they will be eager to innovate. 
D)  one can assume that the staff has pride in its work. 
A)  they will resist adopting ideas for change contained in an evaluation report. 
B)  they are likely to grasp at any suggestion. 
C)  they will be eager to innovate. 
D)  one can assume that the staff has pride in its work.

Question 11

Internal evaluators have an advantage over external evaluators in that internal evaluators

Accepted Answer

A)  are often friends of the program staff or administrators. 
B)  do not have a long-term commitment to the agency. 
C)  can repeatedly remind program managers of the evaluation and its findings. 
D)  have a perspective gained by working in many different agencies. 
A)  are often friends of the program staff or administrators. 
B)  do not have a long-term commitment to the agency. 
C)  can repeatedly remind program managers of the evaluation and its findings. 
D)  have a perspective gained by working in many different agencies.

Question 12

Evaluators are more likely to see evaluations used if

Accepted Answer

A)  reports are simple, clearly labeling failures as failures. 
B)  suggestions are only tangentially related to the program. 
C)  staff feel respected. 
D)  staff are concerned about program termination if goals are not met. 
A)  reports are simple, clearly labeling failures as failures. 
B)  suggestions are only tangentially related to the program. 
C)  staff feel respected. 
D)  staff are concerned about program termination if goals are not met.

Question 13

Acceptance of valid, unfavorable feedback is enhanced when managers can view such feedback

Accepted Answer

A)  like a baseball score-its only a game. 
B)  like medical tests-if the results show that the patient is ill, the doctor does not reject the test. 
C)  with regard to the source of the feedback-students or employees do not have the big picture. 
D)  like a horse race-there can be only one winner, but the losers may well still be fast horses. 
A)  like a baseball score-its only a game. 
B)  like medical tests-if the results show that the patient is ill, the doctor does not reject the test. 
C)  with regard to the source of the feedback-students or employees do not have the big picture. 
D)  like a horse race-there can be only one winner, but the losers may well still be fast horses.

Question 14

If only the more costly elements of a program are eliminated because of negative findings,

Accepted Answer

A)  one should question the value of the evaluation. 
B)  the staff probably has excessive power. 
C)  the evaluator did not do a thorough job of communicating the negative nature of the findings. 
D)  the evaluation served a useful purpose, especially if additional study is to be done. 
A)  one should question the value of the evaluation. 
B)  the staff probably has excessive power. 
C)  the evaluator did not do a thorough job of communicating the negative nature of the findings. 
D)  the evaluation served a useful purpose, especially if additional study is to be done.

Question 15

It is valid to label a program as ineffective when

Accepted Answer

A)  a statistical comparison between program and comparison groups is non-significant. 
B)  statistical comparisons between randomly-assigned program participants and a control group are non-significant and statistical power is 95%. 
C)  the program lacks common sense plausibility but the statistical test is significant (i.e. p < .05). 
D)  statistical comparisons between randomly formed program and control groups are non- significant and statistical power is less than 5%. 
A)  a statistical comparison between program and comparison groups is non-significant. 
B)  statistical comparisons between randomly-assigned program participants and a control group are non-significant and statistical power is 95%. 
C)  the program lacks common sense plausibility but the statistical test is significant (i.e. p < .05). 
D)  statistical comparisons between randomly formed program and control groups are non- significant and statistical power is less than 5%.

Question 16

Implementing the results of evaluation studies is encouraged when

Accepted Answer

A)  the staff holds the evaluator in fearful awe. 
B)  the written results are permitted to "speak for themselves." 
C)  the evaluation feeds back into program planning. 
D)  evaluators keep a proper "hands off" attitude toward the program. 
A)  the staff holds the evaluator in fearful awe. 
B)  the written results are permitted to "speak for themselves." 
C)  the evaluation feeds back into program planning. 
D)  evaluators keep a proper "hands off" attitude toward the program.

Question 17

The most important contribution that an evaluator can make to a program is to

Accepted Answer

A)  encourage the development of a learning culture among the program staff and organization management. 
B)  get staff to care about the participants of the program. 
C)  perform an extremely rigorous evaluation that would draw attention to the organization sponsoring the program. 
D)  be sure that the staff respects the credentials of the evaluator. 
A)  encourage the development of a learning culture among the program staff and organization management. 
B)  get staff to care about the participants of the program. 
C)  perform an extremely rigorous evaluation that would draw attention to the organization sponsoring the program. 
D)  be sure that the staff respects the credentials of the evaluator.

Question 18

Describe how "the evaluator's perspective" differs from that of the staff and program managers.

Accepted Answer

The evaluator's perspective differs from

Question 19

What are the steps that evaluators can take on their own to reduce the fear of evaluation?

Accepted Answer

There are several steps that evaluators

Question 20

Evaluation conclusions are not likely to be applied and indeed should not be applied when

Accepted Answer

A)  a fixation on outcome evaluation led to the use of methods that could not have produced interpretable findings. 
B)  evaluations focus on the extent of program implementation. 
C)  the costs of implementing the findings are carefully described in the evaluation report. 
D)  stakeholders have had input into the recommendations. 
A)  a fixation on outcome evaluation led to the use of methods that could not have produced interpretable findings. 
B)  evaluations focus on the extent of program implementation. 
C)  the costs of implementing the findings are carefully described in the evaluation report. 
D)  stakeholders have had input into the recommendations.

If unsuccessful program innovations are not identified,

Although some anxiety about evaluation is normal, a high level of evaluation apprehension can lead to ineffective reactions to evaluations that are not totally favorable. Describe some of the fears and show how they lead to problems between evaluators and staff members.

What are the steps that evaluators can take to encourage others to take to reduce the fear of evaluation?

When an evaluator fails to document successful achievement of outcome goals, but the innovation seems to makes sense anyway, evaluators

Given the limited development of the science of program evaluation, weaknesses in every study could lead to all results being questioned. It is indefensible to use this fact to

The conclusions of evaluation reports should seldom

Program managers often look for a simple, definitive conclusion from an evaluation (e.g. good/bad), but many evaluators prefer to deal with the findings as working hypotheses on which to base

Strongly polarized attitudes toward a policy, such as limitations on handgun ownership, will result in people who

Sometimes an innovative program will fail to do better than currently used approaches,

If program staff members are worried about the findings of an evaluation,

Internal evaluators have an advantage over external evaluators in that internal evaluators

Evaluators are more likely to see evaluations used if

Acceptance of valid, unfavorable feedback is enhanced when managers can view such feedback

If only the more costly elements of a program are eliminated because of negative findings,

It is valid to label a program as ineffective when

Implementing the results of evaluation studies is encouraged when

The most important contribution that an evaluator can make to a program is to

Describe how "the evaluator's perspective" differs from that of the staff and program managers.

What are the steps that evaluators can take on their own to reduce the fear of evaluation?

Evaluation conclusions are not likely to be applied and indeed should not be applied when

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Developing Measures of Implementation and Outcomes

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Outcome Evaluations With One Group

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

Filters

Exam 14: How to Encourage Utilization

If unsuccessful program innovations are not identified,

Although some anxiety about evaluation is normal, a high level of evaluation apprehension can lead to ineffective reactions to evaluations that are not totally favorable. Describe some of the fears and show how they lead to problems between evaluators and staff members.

What are the steps that evaluators can take to encourage others to take to reduce the fear of evaluation?

When an evaluator fails to document successful achievement of outcome goals, but the innovation seems to makes sense anyway, evaluators

Given the limited development of the science of program evaluation, weaknesses in every study could lead to all results being questioned. It is indefensible to use this fact to

The conclusions of evaluation reports should seldom

Program managers often look for a simple, definitive conclusion from an evaluation (e.g. good/bad), but many evaluators prefer to deal with the findings as working hypotheses on which to base

Strongly polarized attitudes toward a policy, such as limitations on handgun ownership, will result in people who

Sometimes an innovative program will fail to do better than currently used approaches,

If program staff members are worried about the findings of an evaluation,

Internal evaluators have an advantage over external evaluators in that internal evaluators

Evaluators are more likely to see evaluations used if

Acceptance of valid, unfavorable feedback is enhanced when managers can view such feedback

If only the more costly elements of a program are eliminated because of negative findings,

It is valid to label a program as ineffective when

Implementing the results of evaluation studies is encouraged when

The most important contribution that an evaluator can make to a program is to

Describe how "the evaluator's perspective" differs from that of the staff and program managers.

What are the steps that evaluators can take on their own to reduce the fear of evaluation?

Evaluation conclusions are not likely to be applied and indeed should not be applied when

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Developing Measures of Implementation and Outcomes

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Outcome Evaluations With One Group

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

Filters