Deck 12: Evaluation Research

Question 1

In a study that examined whether the use of mediation in divorces resulted in "fairer" settlements, the intervention is:

A)the use of mediation.
B)the "fairer" settlement.
C)the actual divorce.
D)the judge's decision.
E)the courtroom.

Answer 1

A

Question 2

Which of the following topics would NOT be appropriate for evaluation research?

A)Examining whether fee reduction at the local swimming pool increases pool use.
B)Examining whether women who get mammograms detect breast cancer earlier than women who do not.
C)Examining whether changes in speed limits influence traffic fatalities.
D)Examining whether the use of mediation in divorces results in "fairer" settlements.
E)Examining whether southern Americans are more likely to be Republicans or Democrats.

Answer 2

E

Question 3

Professor Henley is interested in comparing the use of the legal system in different countries. Henley examines the number of arrests per 1,000 population, the number of jury trials per 1,000 court cases, the number of lawyers per capita, and the number of people incarcerated per 1,000 population. Henley is doing

A)social indicator research.
B)cost/benefit analysis.
C)time-series designs.
D)computer simulations.
E)intervention analysis.

Answer 3

A

Question 4

You are conducting a study of how effective a midnight basketball program is on reducing inner city juvenile crime. You measure the number of crimes reported to the police before and after the program is instituted, and you compare your findings to the crime rates in another city of similar size that does not have such a program. What sort of evaluation design are you using?

A)The time series
B)The multiple time series
C)The nonequivalent control group design
D)The posttest-only control group design
E)The one-shot case study

Question 5

Which of the following is NOT one of the types of measurements that evaluation researchers must know how to deal with?

A)Specifying populations
B)Specifying outcomes
C)Operationalizing success/failure
D)Specifying interventions
E)Specifying ethical concerns

Question 6

Rebecca was hired to determine what types of recreational programs teenagers wanted and would use in the town of Methods, U.S.A. Rebecca was hired to conduct a:

A)cost-benefit study.
B)monitoring study.
C)needs assessment study.
D)time-series study.
E)content analysis study.

Question 7

The national speed limit in the United States was reduced to 55 mph in 1974. Proponents and critics of this legislation have looked at the number of traffic fatalities on major U.S. highways (e.g., I-90, I-65, and I-75)for the years preceding and following this legislation. Sonny examined the traffic fatalities on each of the major U.S. highways for every year from 1968 to 2005. Sonny used a:

A)pretest-posttest one-group design.
B)time-series design.
C)multiple time-series design.
D)nonequivalent control group design.
E)one-shot case study.

Question 8

For the last 15 years Jenny tracked the divorce and marriage rates in the United States. Jenny is doing a:

A)monitoring study.
B)needs assessment study.
C)cost-benefit analysis.
D)logistical study.
E)time-series study.

Question 9

Which of the following would NOT be appropriate for measurement in evaluation research?

A)The dependent variable(s)
B)The experimental context
C)The outcome(s)of the experiment
D)The independent variable(s)
E)The researcher's context

Question 10

When performing evaluation research, creating one's own measure of a variable has all but which of the following benefits?

A)Greater relevance to project
B)Increased validity
C)Saves time and resources
D)Allows unresearched variables to be measured
E)Can be perfectly tailored to your project

Question 11

Computer simulation has mostly removed the need to conduct actual evaluation research.

Question 12

Which of the following statement about the Sabido methodology is FALSE?

A)The methodology has used television shows to promote literacy and family planning.
B)The methodology pays attention to the kinds of characters portrayed in TV shows.
C)The methodology can be used to evaluate social programs.
D)The methodology is not used to construct or develop social programs.
E)The methodology has been used to enhance the status of women.

Question 13

Social indicators research combines:

A)evaluation research with interviewing.
B)evaluation research with the analysis of existing data.
C)the analysis of existing data with questionnaires.
D)the analysis of existing data with the comparative/historical approach.
E)evaluation research with the comparative/historical approach.

Question 14

In reality, only quasi-experimental designs can be used in evaluation research.

Question 15

Which of the following is FALSE regarding time-series designs?

A)They enable detection of whether a process of maturation could be at work.
B)Regular fluctuations that occur apart from the experimental stimulus can be identified.
C)They protect against the possible influence of some extraneous factor.
D)They involve multiple observations at different points in time.
E)They are considered quasi-experimental designs.

Question 16

Producers of TV shows that depict violence often ignore the studies that indicate that children who watch violence are more likely to be aggressive, have fewer friends, and have poorer relationships with their parents than children who do not watch violent shows. The producers of the violent TV shows probably ignore the research because:

A)they did not understand the researcher.
B)the findings contradict deeply held beliefs.
C)the findings conflict with the producers' vested interests.
D)of logistical problems.
E)of administrative control interference.

Question 17

You are planning a study on whether a course on study skills will improve the test scores of high school students. You selected two high schools where the children appear similar in terms of IQ scores, family income, family religion, and the like. You match two classes from each grade (9 to 12)from each high school. In one high school you plan to offer a course on study skills in the selected classes and in the other high school you plan to have students follow their "normal" curriculum. What kind of study are you conducting?

A)needs assessment study
B)monitoring study
C)nonequivalent control group study
D)outcomes assessment study
E)time-series study

Question 18

You have been developing an antismoking campaign for high school students that has showed promising results in small tests. The campaign is being funded by the FDA, which is ready to hire you and make your campaign national if it shows success in broader experiments. You have identified two high schools in New York City that have similar student profiles and have received permission to conduct your first full-scale test of your antismoking campaign. You intend to present your campaign to classes in one school and have no intervention in the other school. ​ Which of the following is NOT an ethical concern you should take into account as you begin your research?

A)You may feel external pressure from the FDA to find particular results.
B)You may feel internal pressure for the campaign to succeed so you can take it national.
C)When parents hear that their students are not being involved in the campaign, they may be outraged.
D)School faculty and administrators may have an effect on the outcome of the research depending on their support for your work.
E)You may harm your subjects by collecting demographic data about them.

Question 19

Evaluation research as a scientific undertaking is a matter of finding out whether something happened or didn't happen.

Question 20

You are conducting a study of how effective a midnight basketball program is on reducing inner city juvenile crime. You measure the number of crimes reported to the police before and after the program is instituted, and you compare your findings to the crime rates in another city of similar size that does not have such a program. What is the experimental stimulus of your research?

A)The crime rate in the city without the basketball program
B)The crime rate in the city with the midnight basketball program
C)The number of kids who take part in the program
D)The midnight basketball program
E)The crime rate after the program was instituted

Question 21

Although some research indicates that the depiction of violence on TV has no effect on children, many parents still screen their children's TV selections to prevent the selection of violent programs. From a methodologist's perspective, parents probably ignore the implication of this research because:

A)they may not be interested in the research.
B)the research may be unreliable.
C)logistical problems enter.
D)administrative control interferes.
E)they may not understand the research and/or the research may contradict deeply held beliefs.

Question 22

When a control group is not feasible for practical or ethical purposes, the use of a nonequivalent control group can be beneficial.

Question 23

Quantitative evaluations can yield only quantitative results.

Question 24

Applied research is a form of evaluation research.

Question 25

What kinds of topics are appropriate for evaluation research? Please provide at least one example of a topic that is appropriate and one of a topic that isn't appropriate and describe why.

Question 26

What are the potential benefits of implementing computer simulation in social indicators research? What are the potential problems with using it?

Question 27

A potential shortcoming of using computer simulation in the field of social indicators research is:

A)it will eliminate the need for external researchers, eliminating important jobs.
B)people are unpredictable, and thus the results of simulations are not necessarily guaranteed.
C)the cost of a computer capable of running simulations is extremely high.
D)it can only be used for physical, not social variables.
E)it can save organizations significant costs.

Question 28

You never want to use existing measurements when conducting evaluation research.

Question 29

Compare and contrast at least three evaluation designs. What are the strengths and weaknesses of each?

Question 30

Evaluation research refers to a research purpose rather than a specific research method.

Question 31

Creating new measures for variables in evaluation research has the advantage of:

A)greater possible relevance and validity to the study.
B)known degrees of reliability and validity.
C)greater possible reliability to the study.
D)being less time-consuming than using an already devised measure.
E)more precision.

Question 32

Social indicators research often uses existing statistics.

Question 33

What is meant by social indicators research? Give an outline of an original study showing how this type of research would be used.

Question 34

Evaluation research, because it occurs within real life, has special ethical problems. What are some of these problems?

Question 35

A new law has been enacted increasing the penalty for drunk driving in order to reduce the number of alcohol-related traffic fatalities. The number of fatalities one year before and after the law was enacted is measured, as well as a count of the number of high-profile accidents. The experimental stimulus in this project would be:

A)the new law.
B)the rate of traffic fatalities before the law began.
C)the number of traffic fatalities after the law was enacted.
D)the number of arrests for driving under the influence after the law began.
E)the number of high-profile traffic accidents.

Question 36

A friend of yours was asked to evaluate the effect of an advertising campaign designed to get people to have their blood pressure checked. After examining the records kept in the target area, your friend showed you the following graph where X indicates an observation. ​ ​

A)The ad campaign is producing the intended effect.
B)The independent variable, the campaign, is affecting the dependent variable, the percent having their blood pressure checked.
C)More observations are needed to determine whether the independent variable is having the intended effect.
D)The ad campaign has no effect.
E)The independent variable, the campaign, is not affecting the dependent variable, the percent having their blood pressure checked.

Question 37

The denominator in a per capita rate is:

A)1,000.
B)the size of the population.
C)the size of the sample.
D)the GNP.
E)the number of households in the population.

Question 38

As programs are refined, eventually computer simulation will reduce cost because significant problems can be modeled before a program is put in place.

Question 39

Pregnant women planning to deliver their baby in hospital A were given a poison control test. After the delivery of their baby, the new moms attended a two-hour training session on poison control. One week later, at the baby's one-week check-up exam, the new moms retook the poison control test. Located in the same city, pregnant women planning to deliver their baby in hospital B were given a poison control test. One week later, at the baby's one-week check-up exam, these new moms retook the poison control test. What design is being used?

A)Pretest-posttest control group
B)Time series
C)Nonequivalent control group
D)Cross-sectional design
E)Multiple time series