Question 1

Selecting one particular variable to be the criterion of program effectiveness will

Accepted Answer

A)  probably corrupt it. 
B)  complicate the data analysis. 
C)  reflect the complexity of the program. 
D)  be desired by most staff members. 
A)  probably corrupt it. 
B)  complicate the data analysis. 
C)  reflect the complexity of the program. 
D)  be desired by most staff members. A

Question 2

The choice of the criteria of effectiveness is very sensitive in program evaluation because

Accepted Answer

A)  no one is ever satisfied with the criteria selected. 
B)  program staff are always trying to select criteria that are too easy to achieve. 
C)  the choice of criteria can determine the outcome of the evaluation. 
D)  most evaluations show that human services are highly effective. 
A)  no one is ever satisfied with the criteria selected. 
B)  program staff are always trying to select criteria that are too easy to achieve. 
C)  the choice of criteria can determine the outcome of the evaluation. 
D)  most evaluations show that human services are highly effective. C

Question 3

Negative side-effects of programs are important for evaluators to consider because

Accepted Answer

A)  staff members frequently do not consider them. 
B)  participants usually experience some strong negative side effects. 
C)  some participants will be harmed by any program. 
D)  program plans usually mention possible negative side effects. 
A)  staff members frequently do not consider them. 
B)  participants usually experience some strong negative side effects. 
C)  some participants will be harmed by any program. 
D)  program plans usually mention possible negative side effects. A

Question 4

Among the advantages of using program participants as a source of evaluation data is:

Accepted Answer

A)  the information is relatively inexpensive. 
B)  measurement is not reactive. 
C)  their views about program quality are likely to be very valid. 
D)  participants provide particularly reliable reactions. 
A)  the information is relatively inexpensive. 
B)  measurement is not reactive. 
C)  their views about program quality are likely to be very valid. 
D)  participants provide particularly reliable reactions.

Question 5

Including observations of actual behavior among the measures of program success

Accepted Answer

A)  makes the evaluation too expensive. 
B)  creates incredible anxiety among the program staff members. 
C)  adds markedly to the credibility of the evaluation. 
D)  is usually felt to be unnecessary or unethical. 
A)  makes the evaluation too expensive. 
B)  creates incredible anxiety among the program staff members. 
C)  adds markedly to the credibility of the evaluation. 
D)  is usually felt to be unnecessary or unethical.

Question 6

The criteria selected for measuring program effectiveness should be sensitive to changes caused by the program. A good way to determine if a variable might be sensitive to change is to

Accepted Answer

A)  ask program participants for their opinions. 
B)  depend on program staff for ideas. 
C)  see if the variable distinguishes between groups of people who differ in degree to which they achieved planned program outcomes. 
D)  use well-constructed, standard personality or intelligence tests. 
A)  ask program participants for their opinions. 
B)  depend on program staff for ideas. 
C)  see if the variable distinguishes between groups of people who differ in degree to which they achieved planned program outcomes. 
D)  use well-constructed, standard personality or intelligence tests.

Question 7

An important characteristic of self-report surveys is that

Accepted Answer

A)  they can be constructed in a short time. 
B)  they can be used in any program setting. 
C)  considerable thought needs to be devoted to designing them. 
D)  any criterion of program effectiveness can be measured in a survey. 
A)  they can be constructed in a short time. 
B)  they can be used in any program setting. 
C)  considerable thought needs to be devoted to designing them. 
D)  any criterion of program effectiveness can be measured in a survey.

Question 8

Distinguish between the characteristics of tests used in classrooms versus tests to measure stable psychological characteristics.

Accepted Answer

Tests used in classrooms are typically d

Question 9

When obtaining information from program participants, one should

Accepted Answer

A)  never use objective variables. 
B)  depend on the program staff to choose the variables. 
C)  question whether participants are truly stakeholders. 
D)  ask only about issues familiar to the participants. 
A)  never use objective variables. 
B)  depend on the program staff to choose the variables. 
C)  question whether participants are truly stakeholders. 
D)  ask only about issues familiar to the participants.

Question 10

The relative costs of using various methods of data collection in program evaluations

Accepted Answer

A)  cannot be considered because truth is more important than cost. 
B)  are always a trivial factor in the whole study. 
C)  seldom exceed the cost to duplicate surveys or forms. 
D)  should be weighed against the validity of the data sought. 
A)  cannot be considered because truth is more important than cost. 
B)  are always a trivial factor in the whole study. 
C)  seldom exceed the cost to duplicate surveys or forms. 
D)  should be weighed against the validity of the data sought.

Question 11

The criteria of effectiveness used in a program evaluation should

Accepted Answer

A)  be limited to the objectives of the program. 
B)  be determined by the program staff. 
C)  not focus on any negative outcomes of the services. 
D)  be related to the information needs of program stakeholders. 
A)  be limited to the objectives of the program. 
B)  be determined by the program staff. 
C)  not focus on any negative outcomes of the services. 
D)  be related to the information needs of program stakeholders.

Question 12

Program participants are likely to be defensive about answering an evaluator's questions whenever

Accepted Answer

A)  they feel that they have something to lose by providing their comments. 
B)  they feel that the program might be improved by their answers. 
C)  the evaluator contacts them in person. 
D)  they are particularly pleased with the program. 
A)  they feel that they have something to lose by providing their comments. 
B)  they feel that the program might be improved by their answers. 
C)  the evaluator contacts them in person. 
D)  they are particularly pleased with the program.

Question 13

Contrast the strengths and weaknesses of self-report surveys with observations of behavior.

Accepted Answer

Self-report surveys and observations of

Question 14

Asking individuals familiar with the kinds of services provided to make direct observations of the program

Accepted Answer

A)  is seldom disruptive of program services. 
B)  could produce very unbiased data if the experts are involved in neither the program nor its competitors. 
C)  is among the least expensive evaluation data available. 
D)  should never be used because it could threaten the staff. 
A)  is seldom disruptive of program services. 
B)  could produce very unbiased data if the experts are involved in neither the program nor its competitors. 
C)  is among the least expensive evaluation data available. 
D)  should never be used because it could threaten the staff.

Question 15

What should you say if the following item were suggested for use in an evaluation of teaching: "My teacher was well-informed and well-meaning"?

Accepted Answer

A)  The item mentions two aspects of a teacher. 
B)  It is best to avoid the use of a personal pronoun. 
C)  Teaching cannot be evaluated. 
D)  This is a good item; let's use it. 
A)  The item mentions two aspects of a teacher. 
B)  It is best to avoid the use of a personal pronoun. 
C)  Teaching cannot be evaluated. 
D)  This is a good item; let's use it.

Question 16

Desirable aspects of gathering data for program evaluation from records include the

Accepted Answer

A)  freedom to choose the most relevant outcome variables. 
B)  uniformly high validity of archival data. 
C)  thoroughness with which program records are routinely kept in many settings. 
D)  non-reactivity of the data selected from program records. 
A)  freedom to choose the most relevant outcome variables. 
B)  uniformly high validity of archival data. 
C)  thoroughness with which program records are routinely kept in many settings. 
D)  non-reactivity of the data selected from program records.

Question 17

When using official records gathered over a period of years, evaluators must be wary of

Accepted Answer

A)  the inter-rater reliability of records. 
B)  program participants who refuse to cooperate with the evaluation. 
C)  threat to insecure program managers. 
D)  changes in the definitions of the variables. 
A)  the inter-rater reliability of records. 
B)  program participants who refuse to cooperate with the evaluation. 
C)  threat to insecure program managers. 
D)  changes in the definitions of the variables.

Question 18

Using multiple measures of program achievement may not add to the validity of an evaluation if

Accepted Answer

A)  the measures share the same biases. 
B)  all measures are based on interviews. 
C)  few measures have reliabilities exceeding .80. 
D)  objective data sources such as expert observers or artifacts were not used. 
A)  the measures share the same biases. 
B)  all measures are based on interviews. 
C)  few measures have reliabilities exceeding .80. 
D)  objective data sources such as expert observers or artifacts were not used.

Question 19

In selecting indexes or variables that are to reflect program success, it is useful to choose variables that are

Accepted Answer

A)  historical, instrumental, and well-known. 
B)  important, sensitive to change, and cost-effective. 
C)  summative, formative, and sensitive. 
D)  hidden, covert, and implicit. 
A)  historical, instrumental, and well-known. 
B)  important, sensitive to change, and cost-effective. 
C)  summative, formative, and sensitive. 
D)  hidden, covert, and implicit.

Question 20

In evaluating the reliability of a measure to be used in program evaluation, evaluators

Accepted Answer

A)  recognize that means of groups are more stable than the scores of individual assessments. 
B)  realize that reliability is not very important in the area of program evaluation. 
C)  must choose between having a valid measure or having a reliable measure; they can't have both. 
D)  know that surveys are probably more reliable than program effectiveness indexes based on program records. 
A)  recognize that means of groups are more stable than the scores of individual assessments. 
B)  realize that reliability is not very important in the area of program evaluation. 
C)  must choose between having a valid measure or having a reliable measure; they can't have both. 
D)  know that surveys are probably more reliable than program effectiveness indexes based on program records.

Selecting one particular variable to be the criterion of program effectiveness will

The choice of the criteria of effectiveness is very sensitive in program evaluation because

Negative side-effects of programs are important for evaluators to consider because

Among the advantages of using program participants as a source of evaluation data is:

Including observations of actual behavior among the measures of program success

The criteria selected for measuring program effectiveness should be sensitive to changes caused by the program. A good way to determine if a variable might be sensitive to change is to

An important characteristic of self-report surveys is that

Distinguish between the characteristics of tests used in classrooms versus tests to measure stable psychological characteristics.

When obtaining information from program participants, one should

The relative costs of using various methods of data collection in program evaluations

The criteria of effectiveness used in a program evaluation should

Program participants are likely to be defensive about answering an evaluator's questions whenever

Contrast the strengths and weaknesses of self-report surveys with observations of behavior.

Asking individuals familiar with the kinds of services provided to make direct observations of the program

What should you say if the following item were suggested for use in an evaluation of teaching: "My teacher was well-informed and well-meaning"?

Desirable aspects of gathering data for program evaluation from records include the

When using official records gathered over a period of years, evaluators must be wary of

Using multiple measures of program achievement may not add to the validity of an evaluation if

In selecting indexes or variables that are to reflect program success, it is useful to choose variables that are

In evaluating the reliability of a measure to be used in program evaluation, evaluators

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Outcome Evaluations With One Group

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

How to Encourage Utilization

Filters

Exam 4: Developing Measures of Implementation and Outcomes

Selecting one particular variable to be the criterion of program effectiveness will

The choice of the criteria of effectiveness is very sensitive in program evaluation because

Negative side-effects of programs are important for evaluators to consider because

Among the advantages of using program participants as a source of evaluation data is:

Including observations of actual behavior among the measures of program success

The criteria selected for measuring program effectiveness should be sensitive to changes caused by the program. A good way to determine if a variable might be sensitive to change is to

An important characteristic of self-report surveys is that

Distinguish between the characteristics of tests used in classrooms versus tests to measure stable psychological characteristics.

When obtaining information from program participants, one should

The relative costs of using various methods of data collection in program evaluations

The criteria of effectiveness used in a program evaluation should

Program participants are likely to be defensive about answering an evaluator's questions whenever

Contrast the strengths and weaknesses of self-report surveys with observations of behavior.

Asking individuals familiar with the kinds of services provided to make direct observations of the program

What should you say if the following item were suggested for use in an evaluation of teaching: "My teacher was well-informed and well-meaning"?

Desirable aspects of gathering data for program evaluation from records include the

When using official records gathered over a period of years, evaluators must be wary of

Using multiple measures of program achievement may not add to the validity of an evaluation if

In selecting indexes or variables that are to reflect program success, it is useful to choose variables that are

In evaluating the reliability of a measure to be used in program evaluation, evaluators

Program Evaluation : an Overview

Planning an Evaluation

Developing and Using a Theory of the Program

Ethics in Program Evaluation

The Assessment of Need

Monitoring the Implementation and the Operation of Programs

Qualitative Evaluation Methods

Outcome Evaluations With One Group

Quasi-Experimental Approaches to Outcome Evaluation

Using Experiments to Evaluate Programs

Analyses of Costs and Outcomes

Evaluation Reports: Interpreting and Communicating Findings

How to Encourage Utilization

Filters