Deck 13: Evaluation and Policy Research

Question 1

In Project New Hope,677 low-income adults were offered jobs,childcare,and healthcare,while others were offered none of these things.In the end,only 27% of those who were offered jobs stuck with it long enough to lift themselves out of poverty.Levels of depression did not decrease,nor did self-esteem increase.However,the classroom performance of the participants' male children increased.This example demonstrates the importance of ______.

A)theory-driven evaluation
B)process evaluation
C)measuring multiple outcomes
D)efficiency analysis

Answer 1

C

Question 2

Evaluation research is unlike traditional social science research because ______.

A)it is designed to test the implications of a social theory
B)program stakeholders have influence in how the study is designed
C)evaluation research cannot ethically use randomization
D)evaluation research rarely uses quantitative methods

Answer 2

B

Question 3

The economic value of a social program when compared to the costs of that program is established in ______.

A)a cost-benefit analysis
B)a cost-effectiveness analysis
C)a process analysis
D)a summative assessment

Answer 3

A

Question 4

The Government Performance and Results Act of 1993 requires what of government programs?

A)IRB oversight
B)cost-benefit of analysis
C)needs assessment
D)some sort of evaluation

Question 5

Evaluability assessments tend to rely on ______.

A)experimental designs
B)qualitative methods
C)surveys
D)all of these

Question 6

Which of the following is not a typical input in social programs?

A)clients
B)customers
C)feedback
D)participants

Question 7

To find out if a new program is needed in a community,conduct a/an ______.

A)process evaluation
B)evaluability assessment
C)impact analysis
D)needs assessment

Question 8

When evaluating their program,in which group exercise classes tended to increase several measures of participant health,researchers concluded that it didn't really matter how the group classes worked,just so long as it worked.This is an example of ______.

A)black box theory
B)program theory
C)utilization theory
D)theory-driven evaluation

Question 9

A study specifically designed to investigate whether a program can be evaluated is known as a(n)______.

A)needs assessment
B)evaluability assessment
C)process evaluation
D)impact analysis

Question 10

A community hires a researcher to determine whether a community center would be a benefit,in terms of reducing delinquency,promoting community service,and developing networks.To provide an answer,the researcher would conduct what sort of evaluation on community services?

A)needs assessment
B)evaluability assessment
C)process evaluation
D)impact analysis

Question 11

If a program evaluation is written without regard to whether the program had the intended effect,but with very careful attention to how it will be received by other social science peers,it has what kind of orientation or approach?

A)process
B)stakeholder
C)social science
D)theoretical

Question 12

Policy researchers need to include policy makers and regulators in their planning and should consider who will be affected by a policy decision.What caution to policy research does this best represent?

A)credible
B)meaningful and engaging
C)responsible
D)creative

Question 13

Number of meals served or number of people sheltered over a year are examples of ______.

A)inputs
B)outputs
C)outcomes
D)program progress

Question 14

Theory-driven evaluation does which of the following?

A)assesses the relative costs of a social program
B)determines the generalizability of a program
C)guides the investigation of a program process
D)suggests which outcomes should be measured

Question 15

The services delivered or new products produced by a social program are ______.

A)program process
B)program outputs
C)program theory
D)program participation

Question 16

Evaluation research began and developed in which time period?

A)1910-1920s
B)1930-1940s
C)1950-1960s
D)1970-1980s

Question 17

One important difference between evaluation research and other social scientific research is ______.

A)evaluation research is often guided by politics
B)evaluation research rarely uses qualitative research
C)evaluation research rarely uses quantitative research
D)evaluation research is conducted by professional researchers

Question 18

Information about service delivery system outputs,outcomes,or operations that can guide program input is called ______.

A)outputs
B)outcomes
C)feedback
D)program process

Question 19

Case managers and volunteers are examples of ______.

A)inputs
B)outputs
C)outcomes
D)program progress

Question 20

What is one of the greatest challenges for designing quality policy research.

A)making sure all negative consequences are considered
B)coming up with new solutions
C)credibility of evidence
D)timeframe difference between the research process and policy decision-making

Question 21

The historical growth of evaluation research by social scientists is linked to which of the following?

A)growth in private-sector social programs
B)growth in government-funded social programs
C)growth in faith-based social programs
D)decline in government-funded social programs

Question 22

If a researcher takes a social science approach to conducting a program evaluation,the people who will assess the quality of that researcher's report will be ______.

A)other social scientists
B)the funding agency
C)the participants
D)the stakeholders

Question 23

When the goal of evaluation is to compare alternative programs,it may be preferable to ______.

A)randomize whole groups,not individuals,to different groups
B)randomize individuals,not whole groups,to different groups
C)randomly sample small groups from a small population
D)make sure to have a small sample size

Question 24

Impact analysis generally relies on what sort of research design?

A)experimental and quasi-experimental
B)open-ended surveys
C)closed-ended surveys
D)one-shot case studies

Question 25

Who is eliminated from the research process in appreciative inquiry?

A)stakeholders
B)sponsors
C)professional researchers
D)staff

Question 26

Cost-benefit analyses and cost-effectiveness analyses are both what kind of evaluation research?

A)needs assessment
B)evaluability assessment
C)efficiency analysis
D)impact analysis

Question 27

Evaluation research that investigates the process of service delivery is known as ______.

A)needs assessment
B)evaluability assessment
C)process evaluation
D)impact analysis

Question 28

Evaluation research is conducted to investigate social programs.

Question 29

In the DARE program,which of the following was NOT a stakeholder?

A)Parents worried about drug abuse.
B)Police officers trained as DARE officers.
C)Law enforcement in general.
D)All of these were stakeholders.

Question 30

To evaluate a program designed to improve math scores for elementary school girls,girls were divided into treatment and control groups.Girls in the treatment group were required to attend an after-school program,while girls in the control group went home on time.Math exam scores were then compared between the two groups of girls.This type of program design is associated with ______.

A)needs assessment
B)evaluability assessment
C)process evaluation
D)impact analysis

Question 31

To lessen the potential detrimental impact of social programs,researchers should do all of the following,EXCEPT ______.

A)vary treatments among settings rather than among individuals within a setting
B)use the minimum sample size required to adequately test the results
C)minimize the number in the untreated control group
D)compare the presence and absence of treatments,rather than those that vary in intensity

Question 32

The type of evaluation research that determines if a new social program is needed or an old social program is still needed is called an impact assessment.

Question 33

Appreciative inquiry does NOT do which of the following?

A)consider participants' ideas about a program
B)evaluate a social program
C)have a professional social researcher
D)create a structured dialogue about needed changes

Question 34

According to U.S.law,some sort of evaluation is required of all government programs.

Question 35

Which of the following is a typical reason why a program cannot be evaluated?

A)Management only wants its superior performance confirmed.
B)Staff doesn't trust management to check on their performance.
C)Personnel have no clear sense of what the program is trying to achieve.
D)All of these

Question 36

If a researcher wants to find out how a program actually produces its outcomes,she or he could develop a/an ______.

A)impact evaluation
B)process evaluation
C)program theory
D)efficiency analysis

Question 37

If an evaluation takes a black box approach,which is least important?

A)evaluability
B)social science orientation
C)peer evaluation
D)causal mechanism

Question 38

Evaluation research developed in tandem with government expansion during the Great Depression and WWII.

Question 39

The direct product that a program delivers is the outcome.

Question 40

The preferred method for maximizing internal validity in impact analysis is ______.

A)process theory
B)stakeholder orientation
C)black box orientation
D)experimental design

Question 41

In what circumstances should qualitative methods be used in evaluation research? In what circumstances should quantitative methods be used? Use examples to demonstrate your answer.

Question 42

Impact analyses typically use quantitative data.

Question 43

Describe the special ethical considerations associated with evaluation research.How might a researcher lessen the detrimental impacts to subjects?

Question 44

Stakeholders objectively define needs in a needs assessment.

Question 45

Describe the potential problems for stakeholder orientation in evaluation research,in general and using the example of reducing welfare payments in favor of a workfare program.

Question 46

With a black box evaluation study,understanding how a program works is not important.

Question 47

If evaluation findings will be used to help shape and refine a social program,it is known as a formative evaluation.

Question 48

In impact analysis,an experimental design is preferred to maximize internal validity.

Question 49

How and why did evaluation research develop? How does it differ from other types of social science research?

Question 50

Evaluability assessments generally rely on quantitative methods.

Question 51

The investigation of how a social program works is called a mechanism evaluation.

Question 52

Process evaluation investigates how a service is delivered.

Question 53

Why is evaluation research more political than other forms of social research?

Question 54

Quasi-experimental designs are the preferred method for proving that a program created (or caused)a desired effect.

Question 55

Based on the information in Chapter 13,how would you evaluate the success of the DARE program? Explain your answer fully.

Question 56

What are the limitations of evaluation research when compared to other forms of social scientific research?

Question 57

What is the difference between an evaluation research project that assumes a black box and one that seeks to develop a program theory? What are the advantages and disadvantages to each?

Question 58

Researchers should always emphasize a strict social science orientation when conducting evaluation research.

Question 59

Outline the basics of evaluation research.How would a researcher use these basics in order to evaluate the DARE program detailed in the text?

Question 60

What steps can be taken to lessen the potential detrimental impact of social programs on participants? What other ethical issues should evaluation researchers consider both in terms of social science ethics and federally mandated criteria?