Deck 13: Evaluation Research

Question 1

A systems model of evaluation includes all of the following but:

A) Inputs.
B) Program theory.
C) Feedback.
D) Stakeholders.

Answer 1

B

Question 2

A program may be unsuccessful because:

A) The practice theory is incorrect.
B) Inputs are sufficient.
C) All the activites are present.
D) Stakeholders agree about outcomes.

Answer 2

A, d

Question 3

A logic model may:

A) Describe theory and its link to change.
B) Connect resources and activities to outcomes.
C) Link activities to outputs.
D) All of the above.

Answer 3

D

Question 4

Inputs may include all but which of the following?

A) Money
B) Social workers
C) Social work interns
D) Case management

Question 5

_______ describes activities undertaken by the program staff.

A) Units of service
B) Initial outcomes
C) Service completions
D) Assumptions

Question 6

Units of service can be described by:

A) Behaviors.
B) Attitudes.
C) Duration of time.
D) Feelings.

Question 7

A distinctive feature of evaluation research is that variation in both outputs and outcomes influences the inputs to the program through a ________ process.

A) Reaction
B) Evaluative
C) Stakeholder
D) Feedback

Question 8

Which method may be used for need assessment:

A) Experimental group designs.
B) Single subject designs.
C) Social indicators.
D) Process assessment.

Question 9

Eastern City proposes developing a community policing program to reduce crime in one neighborhood. The police department, in conjunction with the mayor's office, held several meetings in which they discussed this proposal with neighborhood residents, business owners, and social organizations. In this example, the police department, the mayor's office, and neighborhood residents are all:

A) Program participants.
B) Key informants.
C) Research subjects.
D) Stakeholders.

Question 10

"Is the program serving its target question," is a question answered by:

A) Need assessment.
B) Process evaluation.
C) Outcome evaluation.
D) Efficiency evaluation.

Question 11

Program coverage can be monitored using:

A) Need assessment.
B) Flowcharts.
C) Program records.
D) Qualitative methods.

Question 12

______________ emphasize(s) the importance of researcher expertise and maintenance of some autonomy to develop the most trustworthy, unbiased program evaluation.

A) Process evaluation
B) Social science approaches
C) Efficiency analysis
D) Stakeholders' perceptions

Question 13

To answer the question, "Did the program work? " researchers rely on which form of evaluation research?

A) Needs assessment
B) Logic modeling
C) Process evaluation
D) Outcome evaluation

Question 14

Outcome evaluation generally relies on what sort of research design?

A) Experimental and quasi-experimental
B) Open-ended surveys
C) One-shot case studies
D) Key informant interviewing

Question 15

In a social program, _________ go into a program, while _________ is the direct products of the program's service delivery process.

A) Outcome; output
B) Output; feedback
C) Feedback; output
D) Input; output

Question 16

_________ evaluation compares what happened after a program with what would have happened had there been no program.

A) Outcome
B) Needs
C) Efficiency
D) Participatory

Question 17

To evaluate a program designed to improve self-esteem for elementary school girls, girls were divided into treatment and control groups. Girls in the treatment group were required to attend an after-school program, while girls in the control group went home on time. Scores on self-esteem measures were then compared between the two groups of girls. This type of program design is associated with:

A) Needs assessment.
B) Efficiency evaluation.
C) Process evaluation.
D) Outcome evaluation.

Question 18

When Border City was considering whether they should investigate the citizenship status of students attending their public schools, an evaluation of the program suggested that it would be more expensive to investigate the students than to educate the few non-citizens that were attending. This evaluation is a type of:

A) Needs assessment.
B) Efficiency evaluaton.
C) Process evaluation.
D) Outcome evaluaton.

Question 19

Evaluation research has shown that the DARE program does not lessen the use of illicit drugs among DARE students, while they are enrolled in the program or afterwards.

Question 20

Evaluation research is its own unique method like other research designs.

Question 21

Progam process is the set of acitivites that constitute the intervention.

Question 22

Outputs describe the impact of the program.

Question 23

A logic model is a representation of the components of of a social work program.

Question 24

In a logic model, service completions are a type of outcome.

Question 25

The program assumptions identify the expected outcomes.

Question 26

Evaluation research is its own unique method like other research designs.

Question 27

Most evaluation researchers attempt to measure multiple outcomes

Question 28

Stakeholders are individuals or groups that evaluate social programs.

Question 29

The type of evaluation research that determines if a new social program is needed or an old social program is still needed is called an outcome evaluation.

Question 30

Social indicators such as poverty rate or school drop-out rate may be used in a needs assessment.

Question 31

Stakeholders are individuals and groups that have some interest in the program

Question 32

The investigation of how a social program works is operating a process evaluation.

Question 33

An outcome evalation may answer the question, "Is the program serving its target population?
"

Question 34

Qualitative methods may be used in a process evaluation.

Question 35

Flowcharting is a method to answer the question, "Are the mssing activities in the design of the program?
"

Question 36

Evaluation research that determines whether a program has had an effect is called an outcome evaluation.

Question 37

The term formative evaluation may be used instead of process evaluation when the evaluation findings are used to help shape and refine the program

Question 38

There are two kinds of efficiency analyses: cost-benefit and cost-effective analyses.

Question 39

Cost-benefit analysis reports the actual outcomes in lieu of assigning a cash value to the outcomes.

Question 40

A benefit to one group may be a cost to another group.

Question 41

The alternative to black boxes in evaluation research is a program theory.

Question 42

Outcome evaluations that do not use an experimental design do not provide useful information and may not be ethically permissible.

Question 43

Cost-effectiveness analysis is a common alternative to cost-benefit analysis.

Question 44

Stakeholder approaches encourage researchers to be responsive to program stakeholders.

Question 45

Relevance validity regards the usefulness of evaluation data to individuals or communities.

Question 46

Match the parts of a program with their descriptions.
-Input

A) Information about the service delivery system, results, and operations that is available to the program
B) Individuals and groups who have some basis of concern with the program
C) The impact of the program process on the cases processed
D) Resources, materials, clients and staff that go into a program
E) Services delivered or new products produced by the program process

Question 47

Match the parts of a program with their descriptions.
-Output

A) Information about the service delivery system, results, and operations that is available to the program
B) Individuals and groups who have some basis of concern with the program
C) The impact of the program process on the cases processed
D) Resources, materials, clients and staff that go into a program
E) Services delivered or new products produced by the program process

Question 48

Match the parts of a program with their descriptions.
-Outcome

A) Information about the service delivery system, results, and operations that is available to the program
B) Individuals and groups who have some basis of concern with the program
C) The impact of the program process on the cases processed
D) Resources, materials, clients and staff that go into a program
E) Services delivered or new products produced by the program process

Question 49

Match the parts of a program with their descriptions.
-Feedback

A) Information about the service delivery system, results, and operations that is available to the program
B) Individuals and groups who have some basis of concern with the program
C) The impact of the program process on the cases processed
D) Resources, materials, clients and staff that go into a program
E) Services delivered or new products produced by the program process

Question 50

Match the parts of a program with their descriptions.
-Stakeholder

A) Information about the service delivery system, results, and operations that is available to the program
B) Individuals and groups who have some basis of concern with the program
C) The impact of the program process on the cases processed
D) Resources, materials, clients and staff that go into a program
E) Services delivered or new products produced by the program process

Question 51

Match the researcher and stakeholder orientations with their descriptions
-Social science approaches

A) The evaluator forms a task force of program stakeholders who help shape the evaluation project so that they are most likely to use its results.
B) Emphasize the importance of researcher expertise and maintenance of some autonomy in order to develop the most trustworthy, unbiased program evaluation.
C) Program participants are engaged with the researchers as co-researchers and help to design, conduct, and report the research.
D) Researchers do not know the intended outcomes of the program, but assess and compare needs of participants to a wide array of program outcomes, mainly to identify unanticipated outcomes.

Question 52

Match the researcher and stakeholder orientations with their descriptions
-Participatory research

A) The evaluator forms a task force of program stakeholders who help shape the evaluation project so that they are most likely to use its results.
B) Emphasize the importance of researcher expertise and maintenance of some autonomy in order to develop the most trustworthy, unbiased program evaluation.
C) Program participants are engaged with the researchers as co-researchers and help to design, conduct, and report the research.
D) Researchers do not know the intended outcomes of the program, but assess and compare needs of participants to a wide array of program outcomes, mainly to identify unanticipated outcomes.

Question 53

Match the researcher and stakeholder orientations with their descriptions
-Goal-free evaluation

A) The evaluator forms a task force of program stakeholders who help shape the evaluation project so that they are most likely to use its results.
B) Emphasize the importance of researcher expertise and maintenance of some autonomy in order to develop the most trustworthy, unbiased program evaluation.
C) Program participants are engaged with the researchers as co-researchers and help to design, conduct, and report the research.
D) Researchers do not know the intended outcomes of the program, but assess and compare needs of participants to a wide array of program outcomes, mainly to identify unanticipated outcomes.

Question 54

Match the researcher and stakeholder orientations with their descriptions
-Utilization-focused evaluation

A) The evaluator forms a task force of program stakeholders who help shape the evaluation project so that they are most likely to use its results.
B) Emphasize the importance of researcher expertise and maintenance of some autonomy in order to develop the most trustworthy, unbiased program evaluation.
C) Program participants are engaged with the researchers as co-researchers and help to design, conduct, and report the research.
D) Researchers do not know the intended outcomes of the program, but assess and compare needs of participants to a wide array of program outcomes, mainly to identify unanticipated outcomes.

Question 55

Match the type of evaluation research with its description.
-Needs assessment

A) Evaluates how services are delivered
B) How much it costs to achieve a given effect
C) Evaluates costs versus benefits and cost-effectiveness of program
D) Evaluates whether a new program is needed or an existing program should continue
E) Evaluates outcomes in terms of what would have happened if there had been no program

Question 56

Match the type of evaluation research with its description.
-Cost effectiveness analysis

A) Evaluates how services are delivered
B) How much it costs to achieve a given effect
C) Evaluates costs versus benefits and cost-effectiveness of program
D) Evaluates whether a new program is needed or an existing program should continue
E) Evaluates outcomes in terms of what would have happened if there had been no program

Question 57

Match the type of evaluation research with its description.
-Process evaluation

A) Evaluates how services are delivered
B) How much it costs to achieve a given effect
C) Evaluates costs versus benefits and cost-effectiveness of program
D) Evaluates whether a new program is needed or an existing program should continue
E) Evaluates outcomes in terms of what would have happened if there had been no program

Question 58

Match the type of evaluation research with its description.
-Outcome evaluation

A) Evaluates how services are delivered
B) How much it costs to achieve a given effect
C) Evaluates costs versus benefits and cost-effectiveness of program
D) Evaluates whether a new program is needed or an existing program should continue
E) Evaluates outcomes in terms of what would have happened if there had been no program

Question 59

Match the type of evaluation research with its description.
-Efficiency analysis

A) Evaluates how services are delivered
B) How much it costs to achieve a given effect
C) Evaluates costs versus benefits and cost-effectiveness of program
D) Evaluates whether a new program is needed or an existing program should continue
E) Evaluates outcomes in terms of what would have happened if there had been no program

Question 60

How does evaluation research differ from other types of social work research?

Question 61

List the basic components of a logic model.

Question 62

In what circumstances should qualitative methods be used in evaluation research?
In what circumstances should quantitative methods be used?
Use social work examples in your answer.

Question 63

What is the difference between an evaluation research project that assumes a black box and one that seeks to develop a program theory?

Question 64

Outline a process evaluation of a program designed to reduce the spread of HIV by providing clean needles, counseling, and drug substitutes to intravenous drug users.

Question 65

Describe how quantitative and qualitative research designs can be used in evaluation research. When would it be ideal to use these designs?

Question 66

Explain what is meant by relevance validity?