Deck 10: Evaluation and Policy Analysis

Question 1

Did the treatment or program (independent variable) effect change in the dependent variable?" This part of the research is variously called ______.

A) impact evaluation or impact analysis
B) modalities
C) outcome evaluation
D) independent/dependent analysis

Answer 1

A

Question 2

The process evaluation component of the research into Project Safe Neighborhoods by McGarrell, et al., included surveys given to CAGI project coordinators to ______.

A) determine the proper population for the research
B) evaluate whether the population was correct, as chosen
C) evaluate whether the initiative was implemented as planned
D) determine the influence of CAGI as implemented

Answer 2

C

Question 3

A type of evaluation research that attempts to determine the needs of some population that might be met with a social program is a(n) ______.

A) needs assessment
B) evaluability assessment
C) process evaluation (monitoring)
D) program montage

Answer 3

A

Question 4

Piza et al. (2016) conducted a ______ of a program in Newark, NJ pairing proactive CCTV monitoring with directed police patrol.

A) efficiency analysis
B) cost-effectiveness analysis
C) cost-benefit analysis
D) black box evaluation

Question 5

After a program has begun, evaluators are often called on to determine ______.

A) if the theoretical aspects are sound.
B) what resources are being expended in the conduct of the program.
C) if there is congruence between expenditures and theoretical underpinnings.
D) a similar population might benefit from a program.

Question 6

In black box evaluation, the focus of the researcher is on whether cases seem to have changed as a result of their ______.

A) entrance into some social science program
B) exit out of a social science program
C) entrance into a program to measure their outputs
D) exposure to the program between the time they entered the program as inputs and when they exited the program as outputs

Question 7

The evaluation process as a whole and the feedback process in particular can only be understood in relation to the interests and perspectives of ______.

A) stakeholders
B) the government
C) exploratory research
D) highest scientific standards

Question 8

A process evaluation can employ a wide range of ______.

A) variables
B) indicators
C) formats
D) modalities

Question 9

It was not until the end of the ______ that social research became immersed in the workings of government with the common goal of improving society.

A) 1950s
B) 1960s
C) 1970s
D) 1980s

Question 10

Research with an impact on policy choices in the immediate future is known as ______ research.

A) statistical
B) applied
C) sampling
D) singular approach

Question 11

Evaluation research that investigates the process of service delivery is a(n) ______.

A) needs assessment
B) evaluability assessment
C) process evaluation (monitoring)
D) program montage

Question 12

Regardless of the treatment or program being examined evaluations are systematic because they employ social research approaches to ______.

A) gathering valid and reliable data
B) finding reliable data
C) understanding why people act the way they act
D) looking at programs in a singular way

Question 13

A necessary component of an evaluation research project that compares program effects with costs is known as a(n) ______.

A) efficiency analysis
B) cost-benefit analysis
C) impact/efficiency plan
D) experimental design

Question 14

The process of service delivery is also known as the program ______.

A) evaluation
B) outputs
C) process
D) delivery

Question 15

A type of evaluation research conducted to determine whether it is feasible to evaluate a program's effects within the available time and resources is a(n) ______.

A) needs assessment
B) evaluability assessment
C) process evaluation (monitoring)
D) program montage

Question 16

Variation in both outputs and outcomes in turn influences the inputs to the program through ______.

A) the recruitment process
B) an output system
C) a never-ending loop
D) a feedback process

Question 17

D'Amico and Fromme (2002) studied the impact of the Risk Skills Training Program (RSTP) and how it compared to the Drug Abuse Resistance Education-Abbreviated (DARE-A) program, using which type of design?

A) computer assisted surveys
B) true experimental design
C) quasi-experimental design
D) snowball design

Question 18

The difference between evaluation research and traditional social science research is that evaluation research ______.

A) is designed to test the implications of a social theory
B) is not designed to test the implications of a social theory
C) uses only quantitative research methods
D) creates a broad theoretical explanation for what is discovered

Question 19

Stakeholders are those individuals and groups who ______.

A) are peripherally related to the program
B) are uninterested parties with relation to the program
C) have some basis of concern with the program
D) are, in reality, the researchers themselves

Question 20

A type of evaluation research that compares program costs to program effects is ______.

A) cost-benefit analysis
B) efficiency analysis
C) cost-effectiveness analysis
D) an RSTP program

Question 21

It may not be ethical simply to bar some potential participants from the programs when ______.

A) resources are scarce
B) an ongoing entitlement program is being evaluated
C) there are less effective benefits available
D) successful programs are not quantifiable

Question 22

______ designs are the most powerful alternatives to true randomized experimental designs.

A) Integrated
B) Quasi-experimental
C) Stakeholder
D) Program theory

Question 23

Human subjects' considerations multiply when the focus is ______.

A) needs assessment
B) cost-benefit analysis
C) cost-effectiveness analysis
D) program impact

Question 24

The process of evaluation research is a simple systems model.

Question 25

The primary reason why randomization is likely not possible in evaluation research is that ______.

A) it can be difficult to obtain the cooperation of the research board
B) the program is usually outside the control of the evaluator
C) although there may be cooperation of the stakeholders there likely will not be from the other researchers
D) experimental designs are not very powerful

Question 26

Evaluation research is a unique component of research designs.

Question 27

Data obtained from a true experimental design provide the best way to determine that ______.

A) the assessment of change in the independent variable is present
B) showing the time ordering was not necessary
C) all three criteria necessary for establishing a causal relationship has been shown
D) there is no need for random assignment because the population was not random

Question 28

If an investigation of program process is constructed, a(n) ______.

A) theory-driven outline may help define the researcher orientation
B) program theory may be developed
C) evaluation of results in the professional social science community is appropriate
D) methodology may have to be created

Question 29

The direct products of the program's service delivery process are its outputs.

Question 30

An orientation to evaluation research that expects researchers to respond to concerns of people involved with stakeholders as well as to the standards and goals of the social scientific community are known as ______ approaches.

A) social science
B) integrated
C) stakeholder
D) entrepreneurial

Question 31

The clustering of violent crime at particular locations suggests that there are important features or dynamics at these locations that ______.

A) give rise to violent situations.
B) lead them to have lower criminogenic clusters
C) are increasingly identified as urbanogenic clusters
D) lead to community uprisings of placid behavior

Question 32

The difference between evaluation research and traditional social science research is that evaluation research is designed to test the implications of a social theory.

Question 33

______ approaches emphasize the importance of researcher expertise

A) Social science
B) Integrative
C) Stakeholder
D) Entrepreneurial

Question 34

A quasi-experimental design is the preferred method for maximizing internal validity.

Question 35

Educational and vocational training opportunities in prison, the availability of legal counsel, and treatment for substance abuse are all potentially important benefits, and an evaluation research project ______.

A) may impact the cost-benefit analysis of them
B) could increase the types of availability
C) can change both their type and their availability
D) may impact effectiveness

Question 36

______ approaches encourage researchers to be responsive to people who are involved with the program.

A) Social science
B) Integrative
C) Stakeholder
D) Entrepreneurial

Question 37

By the mid-1970s, evaluators were called on not only to assess the overall effectiveness of programs but also to determine ______.

A) whether programs were being implemented as intended
B) if there were enough bureaucratic staff to follow rules & regulations
C) what other types of semi-rigorous experimentations needed to be done
D) if practitioners had made appropriate decisions about programs

Question 38

The goal of policy research is to inform those who make policy about ______.

A) (only) the positive effects of the research
B) (only) the negative effects of the research
C) the possible alternative courses of action in response to some identified problem
D) taking a less rigorously design to preserve noneffectiveness

Question 39

Because Taylor and Woods (2010) could not randomly assign LEAs to experimental and control conditions, they used a(n) ______ design.

A) equivalent group
B) quasi-experimental
C) nonequivalent control group
D) true experimental

Question 40

One private organization that has become a leader in publicizing reviews related to criminal justice policy research is the ______.

A) Carter Collaboration
B) Smithson Collaborative
C) Rand Institute
D) Campbell Collaboration

Question 41

The first evaluator of the evaluation research is the funding agency, not the professional social science community.

Question 42

In quasi-experimental designs, there is a comparison group that is comparable to the experimental group in critical ways but the subjects are not randomly assigned to the comparison and experimental groups.

Question 43

When an ongoing entitlement program is being evaluated and experimental subjects would normally be eligible for program participation, it may be ethical simply to bar some potential participants from the programs.

Question 44

A program theory describes what has been learned about how the program has its effect.

Question 45

Program impact may be evaluated with quasi-experimental designs or survey or field research methods without a randomized experimental design.

Question 46

The program theory is developed by the researcher to clarify and develop the researchers' theory of the program.

Question 47

A quasi-experimental design in which there are experimental and comparison groups that are designated before the treatment occurs but are not created by random assignment is a nonequivalent control group design.

Question 48

Data from a quasi-experimental design provide the best way to determine that all criteria have been met in a study.

Question 49

Evaluability assessments often rely on quantitative methods.

Question 50

As part of its Project Safe Neighborhoods, the U.S. Department of Justice (DOJ) provided funding to 12 relatively large cities to develop and implement a comprehensive anti-gang initiative (CAGI) intended to prevent and reduce gang violence.

Question 51

A private organization that has become a leader in publicizing reviews related to criminal justice policy research is the Carter Collaboration.

Question 52

The analysis of the extent to which a treatment or other service has the intended effect is an outcome evaluation.

Question 53

One of the only true experiments to examine the efficacy of problem-oriented policing was conducted by Anthony Braga and his colleagues in 1999, and it remains a model evaluation study today.

Question 54

Evaluators are often called on to document the extent to which implementation has taken place, whether the program is reaching the target individuals or groups, whether the program is actually operating as expected, and what resources are being expended in the conduct of the program.

Question 55

Typically, evaluation research that attempts to identify the effects of a treatment, law, or program typically is quantitative.

Question 56

Because policy research often draws on the findings of evaluation research projects and involves working for a client as is the case in evaluation research, policy researchers confront many of the same challenges as do evaluation researchers.

Question 57

Evaluation research questions are amenable to a true experimental design.

Question 58

What is evaluation research? Is it a method of data collection or unique research design?

Question 59

Assessing needs, determining evaluability, and examining the process of treatment delivery have many special ethical dimensions.

Question 60

A needs assessment attempts to answer whether a new program is needed.

Question 61

What is an Evaluability Assessment? How is it accomplished?

Question 62

What is the social science approach? Stakeholder approach? How are they different?

Question 63

Most social science research assumes that the researcher specifies the research questions, the applicable theory or theories, and the outcomes to be investigated. In program evaluation, however, the research question is often set by the program sponsors or the government agency that is responsible for reviewing the program. How does that affect the research?

Question 64

Describe the process of evaluation research.

Question 65

How has evaluation research changed over the years, particularly from the 1500s forward?

Question 66

Describe the research done on Problem-Oriented Policing in Violent Crime Areas. What kind of design was used? What did they discover?

Question 67

Give an examples of the types of questions that evaluation projects may focus on that are related to the operation and impact of a program.

Question 68

What is Process Evaluation (Program Monitoring)?

Question 69

Did the research by Braga et al. (1999) meet all of the criteria for a true experimental design? Describe the research and what they found.

Question 70

What does it mean to say that evaluation process as a whole can be understood in relation to stakeholders? As part of your answer, explain what stakeholders are and how this type of research is different than traditional social science research.

Question 71

What is a Needs Assessment? How is it accomplished?

Question 72

Once we have decided on or identified the goal or focus for a program evaluation, there are still important decisions to be made about how to design the specific evaluation project. What are the most important decisions to be made?

Question 73

Describe the research done by the US Department of Justice as part of its Project Safe Neighborhoods. Was it deemed to be effective? Why?

Question 74

What is a Black Box evaluation?