Question 1

A posttest-only evaluation design is often useful when

Accepted Answer

A)  the stakeholders do not have advanced training in research methods. 
B)  there is a definite, objective criterion of successful outcome. 
C)  there is little time or resources for an evaluation. 
D)  program success is hard to define. 
A)  the stakeholders do not have advanced training in research methods. 
B)  there is a definite, objective criterion of successful outcome. 
C)  there is little time or resources for an evaluation. 
D)  program success is hard to define. B

Question 2

The use of small samples can lead unsophisticated evaluators to conclude falsely that

Accepted Answer

A)  Type I errors were held to only 5%. 
B)  findings could be applied to additional populations. 
C)  a program had no impact. 
D)  statistically significant results were observed. 
A)  Type I errors were held to only 5%. 
B)  findings could be applied to additional populations. 
C)  a program had no impact. 
D)  statistically significant results were observed. C

Question 3

Another name for the threat to internal validity called "attrition" is

Accepted Answer

A)  history. 
B)  participant loss. 
C)  instrumentation. 
D)  regression to the mean. 
A)  history. 
B)  participant loss. 
C)  instrumentation. 
D)  regression to the mean. B

Question 4

When initially extreme scores are followed by less extreme values (even when hypothesized), one wonders whether _____________ can account for the change.

Accepted Answer

A)  maturation 
B)  selection 
C)  regression to the mean 
D)  testing 
A)  maturation 
B)  selection 
C)  regression to the mean 
D)  testing

Question 5

Develop two hypothetical situations in which threats to internal validity would serve to create
 an apparent positive program effect and two in which an apparent negative effect would be
 created.

Accepted Answer

Hypothetical situations in which threats

Question 6

The question of how much improvement in a treatment group is enough

Accepted Answer

A)  cannot be settled empirically. 
B)  should be determined after examining the results. 
C)  is based on internal validity considerations. 
D)  is an irrelevant issue. 
A)  cannot be settled empirically. 
B)  should be determined after examining the results. 
C)  is based on internal validity considerations. 
D)  is an irrelevant issue.

Question 7

The threat to internal validity called "history" refers to

Accepted Answer

A)  predictable changes in people that can be expected solely due to the passage of time. 
B)  events happening in the community that will change the behavior of the program participants. 
C)  the improved skill of the staff as they come to understand the history of the program. 
D)  improvement in the skills of the evaluator in measuring outcome variables. 
A)  predictable changes in people that can be expected solely due to the passage of time. 
B)  events happening in the community that will change the behavior of the program participants. 
C)  the improved skill of the staff as they come to understand the history of the program. 
D)  improvement in the skills of the evaluator in measuring outcome variables.

Question 8

The threat to internal validity called "testing"

Accepted Answer

A)  refers to changes in behavior due to the observation techniques. 
B)  refers to the use of measurement procedures themselves. 
C)  refers to school-based program evaluations. 
D)  cannot be a threat to pretest/posttest designs. 
A)  refers to changes in behavior due to the observation techniques. 
B)  refers to the use of measurement procedures themselves. 
C)  refers to school-based program evaluations. 
D)  cannot be a threat to pretest/posttest designs.

Question 9

The threat to internal validity called "instrumentation" refers to

Accepted Answer

A)  failures in the electronic equipment sometimes used to collect data. 
B)  problems that are beyond the evaluator's control. 
C)  changes in the meaning of the scores of the measures used. 
D)  that which cannot be a threat to pretest/posttest designs. 
A)  failures in the electronic equipment sometimes used to collect data. 
B)  problems that are beyond the evaluator's control. 
C)  changes in the meaning of the scores of the measures used. 
D)  that which cannot be a threat to pretest/posttest designs.

Question 10

Explain which threats to internal validity need to be considered in a comparison of the mean scores on a test administered on the first and last days of this course.

Accepted Answer

There are several threats to internal va

Question 11

Repeatedly dividing the sample on the basis of demographic descriptors to look for differences in the means of outcome variables for different subgroups of the sample

Accepted Answer

A)  raises the chances of a Type I error. 
B)  raises the chances of hindsight error. 
C)  lowers the internal validity of a study. 
D)  increases the level of external validity. 
A)  raises the chances of a Type I error. 
B)  raises the chances of hindsight error. 
C)  lowers the internal validity of a study. 
D)  increases the level of external validity.

Question 12

Pretest-posttest designs permit the evaluator to learn whether

Accepted Answer

A)  the program caused a change in the outcome variable. 
B)  the participant improved during the program. 
C)  improvement was great enough to reject threats to internal validity. 
D)  Type I errors could account for a change. 
A)  the program caused a change in the outcome variable. 
B)  the participant improved during the program. 
C)  improvement was great enough to reject threats to internal validity. 
D)  Type I errors could account for a change.

Question 13

The fact that people who choose to, or who are chosen to be, in a program are different from those who are not in a program creates the threat to internal validity called

Accepted Answer

A)  history. 
B)  regression to the mean. 
C)  selection. 
D)  testing. 
A)  history. 
B)  regression to the mean. 
C)  selection. 
D)  testing.

Question 14

Internal validity refers to whether

Accepted Answer

A)  a statistically significant change occurred. 
B)  Type II errors occurred. 
C)  a research design provides sufficient control that one can learn whether the program caused a change in an outcome. 
D)  the findings can be generalized to other similar groups. 
A)  a statistically significant change occurred. 
B)  Type II errors occurred. 
C)  a research design provides sufficient control that one can learn whether the program caused a change in an outcome. 
D)  the findings can be generalized to other similar groups.

Question 15

Consider a program to assist new parents to develop parenting skills. Why might their views of their own preprogram skill levels change as a result of their own development?

Accepted Answer

As new parents engage in the program to

Question 16

Retrospective pretests were developed to be sensitive to

Accepted Answer

A)  hostile participants in the evaluation. 
B)  expert opinion about what went wrong in the implementation of the program. 
C)  how people believe that their parents raised them. 
D)  changes created by the program that may have changed how participants understand themselves. 
A)  hostile participants in the evaluation. 
B)  expert opinion about what went wrong in the implementation of the program. 
C)  how people believe that their parents raised them. 
D)  changes created by the program that may have changed how participants understand themselves.

Question 17

The pretest/posttest design using only the program's clients can help introduce the staff to the merits of program evaluation because

Accepted Answer

A)  this is a very sophisticated research design. 
B)  the threats to internal validity are all controlled except for instrumentation and the interaction of selection by maturation. 
C)  a plan to compare the program group with people in other programs might be rejected because the staff feels too threatened. 
D)  it is easier to take the needs of all stakeholders into account using this design. 
A)  this is a very sophisticated research design. 
B)  the threats to internal validity are all controlled except for instrumentation and the interaction of selection by maturation. 
C)  a plan to compare the program group with people in other programs might be rejected because the staff feels too threatened. 
D)  it is easier to take the needs of all stakeholders into account using this design.

Question 18

Evaluation designs should be selected so that

Accepted Answer

A)  the stakeholders' questions can be answered. 
B)  basic knowledge in the field can be gained. 
C)  the evaluation will merit publication. 
D)  the expectations of the major stakeholders will be supported. 
A)  the stakeholders' questions can be answered. 
B)  basic knowledge in the field can be gained. 
C)  the evaluation will merit publication. 
D)  the expectations of the major stakeholders will be supported.

Question 19

Contrast the effects of attrition depending on whether the most successful participants leave prematurely (because they feel they no longer need the program) or the less successful participants leave prematurely (because they feel that they are not making progress).

Accepted Answer

Attrition can have different effects dep

Question 20

Illustrate how advances in medical technology can make it appear as though the incidence of a disease has changed over time.

Accepted Answer

Advances in medical technology can make

A posttest-only evaluation design is often useful when

The use of small samples can lead unsophisticated evaluators to conclude falsely that

Another name for the threat to internal validity called "attrition" is

When initially extreme scores are followed by less extreme values (even when hypothesized), one wonders whether _____________ can account for the change.

Develop two hypothetical situations in which threats to internal validity would serve to create an apparent positive program effect and two in which an apparent negative effect would be created.

The question of how much improvement in a treatment group is enough

The threat to internal validity called "history" refers to

The threat to internal validity called "testing"

The threat to internal validity called "instrumentation" refers to

Explain which threats to internal validity need to be considered in a comparison of the mean scores on a test administered on the first and last days of this course.

Repeatedly dividing the sample on the basis of demographic descriptors to look for differences in the means of outcome variables for different subgroups of the sample

Pretest-posttest designs permit the evaluator to learn whether

The fact that people who choose to, or who are chosen to be, in a program are different from those who are not in a program creates the threat to internal validity called

Internal validity refers to whether

Consider a program to assist new parents to develop parenting skills. Why might their views of their own preprogram skill levels change as a result of their own development?

Retrospective pretests were developed to be sensitive to

The pretest/posttest design using only the program's clients can help introduce the staff to the merits of program evaluation because

Evaluation designs should be selected so that

Contrast the effects of attrition depending on whether the most successful participants leave prematurely (because they feel they no longer need the program) or the less successful participants leave prematurely (because they feel that they are not making progress).

Illustrate how advances in medical technology can make it appear as though the incidence of a disease has changed over time.

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Developing Measures of Implementation and Outcomes

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

How to Encourage Utilization

Filters

Exam 9: Outcome Evaluations With One Group

A posttest-only evaluation design is often useful when

The use of small samples can lead unsophisticated evaluators to conclude falsely that

Another name for the threat to internal validity called "attrition" is

When initially extreme scores are followed by less extreme values (even when hypothesized), one wonders whether _____________ can account for the change.

Develop two hypothetical situations in which threats to internal validity would serve to create an apparent positive program effect and two in which an apparent negative effect would be created.

The question of how much improvement in a treatment group is enough

The threat to internal validity called "history" refers to

The threat to internal validity called "testing"

The threat to internal validity called "instrumentation" refers to

Explain which threats to internal validity need to be considered in a comparison of the mean scores on a test administered on the first and last days of this course.

Repeatedly dividing the sample on the basis of demographic descriptors to look for differences in the means of outcome variables for different subgroups of the sample

Pretest-posttest designs permit the evaluator to learn whether

The fact that people who choose to, or who are chosen to be, in a program are different from those who are not in a program creates the threat to internal validity called

Internal validity refers to whether

Consider a program to assist new parents to develop parenting skills. Why might their views of their own preprogram skill levels change as a result of their own development?

Retrospective pretests were developed to be sensitive to

The pretest/posttest design using only the program's clients can help introduce the staff to the merits of program evaluation because

Evaluation designs should be selected so that

Contrast the effects of attrition depending on whether the most successful participants leave prematurely (because they feel they no longer need the program) or the less successful participants leave prematurely (because they feel that they are not making progress).

Illustrate how advances in medical technology can make it appear as though the incidence of a disease has changed over time.

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Developing Measures of Implementation and Outcomes

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

How to Encourage Utilization

Filters