Deck 10: Evaluation and Policy Analysis

Question 1

Today, professional evaluation researchers have realized that 10-2)

A)It is more than enough to simply perform experiments to see if a program works
B)It is not enough to perform rigorous experiments to determine program efficacy
C)They are responsible for making sure that their results can be understood and utilized by practitioners so that they can make decisions about keeping or ending programs
D)Both B and C are true

Answer 1

D

Question 2

Which type of research is the preferred method for maximizing internal validity? 10-9)

A)Surveys
B)Experiments
C)Participant observation
D)Evaluation

Answer 2

B

Question 3

The U.S.Department of Justice DOJ)Project Safe Neighborhoods, provided funding to 12 relatively large cities in order to 10-7)

A)Develop and implement a comprehensive anti-gang initiative CAGI)
B)Develop and implement a participant observation study of neighborhoods
C)Continue a comprehensive research project of inner city neighborhoods
D)All of the above

Answer 3

A

Question 4

The focus of evaluation projects include 10-5)

A)Discovering the program's impact
B)Understanding how efficient the program is
C)Both A and B
D)Neither A nor B

Question 5

Once a program has been started, it is important to determine whether the program is reaching the target individuals or groups and to discover if the program is actually operating as expected.This is called 10-6)

A)An assessment evaluation
B)A needs assessment
C)A process evaluation
D)An evaluability assessment

Question 6

Program process refers to the 10-3)

A)Inputs that go into a program
B)Direct products of the program's service delivery
C)The complete treatment or service delivered by the program
D)All of the above

Question 7

Evaluation research and traditional social science are different because 10-4)

A)Evaluation is not designed to test the implications of a social theory
B)The goal of evaluation research is not to create a broad theoretical explanation for what is found
C)Researchers cannot design evaluation studies simply in accord with the highest scientific standards
D)All of the above

Question 8

Outputs are the 10-3)

A)Direct products of the program's service delivery
B)Complete treatment or service delivered by the program
C)Resources and raw materials that go into a program
D)Services delivered or new products produced by the program process

Question 9

Evaluation research is conducted for the purpose of 10-2)

A)explaining why variables affect one another
B)describing a population
C)investigating social programs
D)systematically assessing the usefulness of exploratory research

Question 10

The bulk of the published evaluation studies in criminal justice are devoted to some type of 10-8)

A)Participant observation research
B)Impact assessment
C)Outcome assessment
D)Quantitative research

Question 11

Resources, raw materials, clients, and staff that go into a program as known as 10-3)

A)Incomes
B)Inputs
C)Outcomes
D)Program process

Question 12

The analysis of the extent to which a treatment or other service as the intended effect is an)10-8)

A)Program evaluation
B)Resource evaluation
C)Outcome evaluation
D)Impact evaluation

Question 13

If current research participants who are already in a program are compared with nonparticipants 10-9)

A)It is likely that the treatment group will be comparable to the control group
B)It is probable that the control group will not be comparable to the treatment group
C)It is unlikely that the treatment group will be comparable to the control group
D)These two groups cannot be compared in research

Question 14

Research that has an impact on policy is known as 10-1)

A)Exploratory research
B)Descriptive research
C)Explanatory research
D)Applied research

Question 15

The impact of the program process on the cases processed is known as 10-3)

A)Incomes
B)Inputs
C)Outcomes
D)Program process

Question 16

The goal of evaluation research is 10-5)

A)Different than the goal of other types of social science research
B)Primarily the same as the goal for all social science research
C)To cut as many programs as possible in order to save the government millions of dollars
D)None of the above

Question 17

Information about service delivery system outputs, outcomes or operations is 10-4)

A)Looped information
B)Feedback
C)Stakeholding information
D)confidential

Question 18

A needs assessment attempts to discover 10-5)

A)The demographics of a population in order to decide if a program is necessary
B)Whether a population actually needs to have a particular program
C)Both A and B
D)Neither A nor B

Question 19

A type of evaluation research that is conducted to determine whether it is feasible to evaluate a program's effects with available time and resources is known as 10-6)

A)An assessment evaluation
B)An evaluability assessment
C)A needs assessment
D)An operational assessment

Question 20

Stakeholders are 10-4)

A)Individuals and groups who have some basis of concern with the program
B)People who have no connection to a program
C)Clients, staff, managers or funders of the program
D)Only A and C

Question 21

The most powerful alternatives to true randomized experimental designs are

A)Participant observation design 10-16)
B)Nonequivalent control group design
C)Time series design
D)Both A and B

Question 22

When a researcher attempts to discover if a program worked as intended, he is employing an impact evaluation.10-8)

Question 23

Evaluation research may be undertaken to identify ways to improve delivery of services.10-5)

Question 24

When at least one or more elements of a true experimental design is absent, it is known as a 10-16)

A)Nonprobability study
B)Quasi-experimental design
C)Randomly assigned control
D)None of the above

Question 25

Policy research is a 10-19)

A)Method rather than a process
B)Information method to understand a process
C)Process rather than a method
D)Summary method that summarizes the process

Question 26

The impact of program process on the cases processed is known as outcomes.10-3)

Question 27

The time series research conducted by Bye in 2008 10-17)

A)Examined drug abuse and juvenile delinquency between 1950-2000
B)Examined per capita alcohol consumption and homicide rate data general between the 1950s and 2002
C)Examined alcohol use and violence in western European countries
D)None of the above

Question 28

An approach that encourages researchers to be responsive to program stakeholders is 10-12)

A)Utilization-focused evaluation
B)Politically based evaluation
C)Supportive relation evaluation
D)Action-focused evaluation

Question 29

The direct products of a program's service delivery process are its outputs.10-3)

Question 30

Policy research is a method of data collection.10-19)

Question 31

This type of evaluation occurs when an evaluation of program outcomes ignores the process by which the program produced the effect 10-11)

A)Effective box evaluation
B)Black box evaluation
C)Hidden box evaluation
D)Midnight blue box evaluation

Question 32

The direct impact on research participants 10-21)

A)Lessens the attention that evaluation researchers have to give to human subject concerns
B)Heightens the attention that evaluation researchers have to give to human subjects concerns
C)Should not affect human subject concerns
D)None of the above

Question 33

A needs assessment can attempt to discover the need for a new program in a community.10-5)

Question 34

The design used by D'Amico and Fromme was a true experimental design.10-9)

Question 35

When a researcher is not able to randomly assign participants to experimental and control conditions, they may use an)10-17)

A)Nonequivalent control group design
B)Participant observation design
C)Time series design
D)Counterfactual design

Question 36

How can qualitative methods add to quantitative evaluation research studies? 10-18)

A)By adding more depth, detail and nuance
B)By helping to add understanding of how social programs actually operate
C)The more complex the social program the more value qualitative methods add to the evaluation process
D)All of the above

Question 37

One of the main initiatives that spawned the growth of evaluation research in the U.S.was the "War on Poverty" that was part of the Great Society legislation of the 1960s.10-2)

Question 38

Evaluation research is conducted for a distinctive purpose: to investigate social programs.10-2)

Question 39

A type of evaluation research that compares program costs to program effects is 10-10)

A)A cost-benefit analysis
B)A cost-effectiveness analysis
C)Both A and B
D)Neither A nor B

Question 40

An efficiency analysis compares a program's effects with its costs.10-10)

Question 41

Policy research is a process rather than a method.10-19)

Question 42

What is a needs assessment? When is it used? 10-5)

Question 43

Stakeholder approaches encourage researchers not to be responsive to participants.10-12)

Question 44

Why is a true experimental design the strongest method for conducting impact evaluations?

Question 45

Why is a true experimental design the strongest method for conducting impact evaluations? 10-14)

Question 46

How is an evaluability assessment different from a needs assessment? 10-6)

Question 47

Data obtained from a true experimental design provide the best way to determine that the criteria have been met.10-15)

Question 48

In general, quasi-experimental designs are much less powerful alternatives to true randomized experimental designs.10-16)

Question 49

Qualitative methods do not mix well with quantitative methods.10-18)

Question 50

What is the history of evaluation research? What is its current status?

Question 51

A nonequivalent control group design is a quasi-experimental design is one in which there are experimental and comparison groups that are designed before the treatment occurs but are not created by random assignment.10-17)

Question 52

Define and explain each of the types of evaluation research.When is each appropriate?

Question 53

What is the difference between applied and basic research? 10-1)

Question 54

Explain the concept of "black box" evaluation.What is the value of opening the black box? 10-11)

Question 55

It is always possible to use randomized experiments to evaluate the impacts of a program.10-16)

Question 56

A cost-benefit analysis is a type of evaluation research that compares program costs to the economic value of program benefits.10-10)

Question 57

A black box evaluation is guided by a theory that specifies the process of a program and its effectiveness.10-11)

Question 58

Problem-oriented policing strategies are increasingly used by urban jurisdictions to reduce crime in high-activity crime places.10-13)

Question 59

Diagram the evaluation research process as a feedback system.10-3)

Question 60

What is the role of program theory and its value in evaluation research? 10-11)

Question 61

How are stakeholder approaches different from social science approaches?

Question 62

What is an integrated approach? What is the goal?

Question 63

In what ways can evaluation research make a difference in peoples' lives? 10-21)

Question 64

What are process evaluations? Why are process evaluations important?

Question 65

What is policy research? What is its goal? 10-19)