Question 1

The stakeholders in a program being evaluated include

Accepted Answer

A)  opinion leaders in the community where the program is implemented. 
B)  the agencies that fund the program. 
C)  program clients. 
D)  all of the above. 
A)  opinion leaders in the community where the program is implemented. 
B)  the agencies that fund the program. 
C)  program clients. 
D)  all of the above.

Question 2

A local history confound occurs in an evaluation study when

Accepted Answer

A)  participants in the control group learn about the treatment being given to the experimental group. 
B)  an event outside the research affects only the control group or only the experimental group. 
C)  members of the control group feel demoralized because they know they are not receiving a treatment. 
D)  a program is a poor fit with the clients' culture. 
A)  participants in the control group learn about the treatment being given to the experimental group. 
B)  an event outside the research affects only the control group or only the experimental group. 
C)  members of the control group feel demoralized because they know they are not receiving a treatment. 
D)  a program is a poor fit with the clients' culture.

Question 3

Explain how one can assess whether an evaluation research program was properly implemented.

Accepted Answer

Assessing whether an evaluation research

Question 4

As a tool for evaluation research, meta-analysis can

Accepted Answer

A)  assess the average effect of an intervention across multiple evaluations of the intervention. 
B)  provide information about the generalizability of the intervention. 
C)  estimate the effects of possible moderator variables on the intervention. 
D)  do all of the above 
A)  assess the average effect of an intervention across multiple evaluations of the intervention. 
B)  provide information about the generalizability of the intervention. 
C)  estimate the effects of possible moderator variables on the intervention. 
D)  do all of the above

Question 5

What is a target population in evaluation research? How can a researcher ensure that the selected population is unbiased? What are the consequences of any potential bias?

Accepted Answer

A target population in evaluation resear

Question 6

Instrumental utilization of the results of evaluation research occurs when the results

Accepted Answer

A)  are used to improve future research designs. 
B)  are directly used for making decisions or solving problems. 
C)  influence a policy maker's thinking on an issue. 
D)  are used to convince people to support a political position. 
A)  are used to improve future research designs. 
B)  are directly used for making decisions or solving problems. 
C)  influence a policy maker's thinking on an issue. 
D)  are used to convince people to support a political position.

Question 7

What is the Reliable Change Index? Give one example of how it can be used in evaluation research.

Accepted Answer

The Reliable Change Index (RCI) is a sta

Question 8

Dr. Kuh has designed a program to help first generation college students succeed in their first year. A proximal outcome would be

Accepted Answer

A)  midterm grades for the students' first fall semester. 
B)  four-year graduation rates. 
C)  courses chosen for the sophomore year. 
D)  scores on the Graduate Record Exam. 
A)  midterm grades for the students' first fall semester. 
B)  four-year graduation rates. 
C)  courses chosen for the sophomore year. 
D)  scores on the Graduate Record Exam.

Question 9

Describe how unintended effects can operate at the program implementation stage. Are those effects generally positive or negative?

Accepted Answer

Unintended effects can operate at the pr

Question 10

Evaluation research is perceived by decision makers to be useful when it

Accepted Answer

A)  addresses the needs of all stakeholder groups. 
B)  provides information that can aid in program development. 
C)  focuses on dependent variables that policy-makers see as important. 
D)  both b and c. 
A)  addresses the needs of all stakeholder groups. 
B)  provides information that can aid in program development. 
C)  focuses on dependent variables that policy-makers see as important. 
D)  both b and c.

Question 11

________ outcomes occur in clients' broader social environment and are often reflected in social indicators.

Accepted Answer

A)  Social impact 
B)  Proximal 
C)  Policy-focused 
D)  Client-centered 
A)  Social impact 
B)  Proximal 
C)  Policy-focused 
D)  Client-centered

Question 12

A ________ evaluation addresses the overall effectiveness of a program relative to its goals.

Accepted Answer

A)  summative 
B)  cumulative 
C)  formative 
D)  process 
A)  summative 
B)  cumulative 
C)  formative 
D)  process

Question 13

Define control group contamination. Describe three possible sources of this type of contamination.

Accepted Answer

Control group contamination refers to th

Question 14

You are evaluating a program to reduce employee turnover in XYZ Company. You randomly assigned the employees in the Eastville plant to be the experimental group and the employees in the Centerville plant, located 50 miles from Eastville, to be the control group. During the course of your research, another company in Centerville expands its workforce and offers a higher pay rate than XYZ. This event has created the possibility of a(n) ________ confound for your study.

Accepted Answer

A)  local history 
B)  treatment diffusion 
C)  competition 
D)  instrumentation change 
A)  local history 
B)  treatment diffusion 
C)  competition 
D)  instrumentation change

Question 15

Dr. Nezu believes removing a component from a program is unlikely to change its effectiveness. Which design would best test his hypothesis?

Accepted Answer

A)  dismantling design 
B)  comparative outcome design 
C)  program package design 
D)  client variation design 
A)  dismantling design 
B)  comparative outcome design 
C)  program package design 
D)  client variation design

Question 16

The client population of a program consists of

Accepted Answer

A)  everyone living in the geographic area served by the program. 
B)  the intended recipients of the program's services. 
C)  the people who make use of the program's services. 
D)  decision-making and policy-setting officials at the agencies that fund the program. 
A)  everyone living in the geographic area served by the program. 
B)  the intended recipients of the program's services. 
C)  the people who make use of the program's services. 
D)  decision-making and policy-setting officials at the agencies that fund the program.

Question 17

The reliable change index

Accepted Answer

A)  is a means of operationally defining clinically significant change. 
B)  provides a standard way to determine whether non-chance change has occurred. 
C)  provides information about change that is theoretically, but not clinically, useful. 
D)  has not been tested in field research. 
A)  is a means of operationally defining clinically significant change. 
B)  provides a standard way to determine whether non-chance change has occurred. 
C)  provides information about change that is theoretically, but not clinically, useful. 
D)  has not been tested in field research.

Question 18

Under what circumstance might null results in an evaluation research study have practical significance?

Accepted Answer

A)  The researchers failed to complete a power analysis. 
B)  A local history confound was identified. 
C)  Two versions of a program produced the same outcome, but one cost less. 
D)  Members of an important stakeholder group recommended dropping the program. 
A)  The researchers failed to complete a power analysis. 
B)  A local history confound was identified. 
C)  Two versions of a program produced the same outcome, but one cost less. 
D)  Members of an important stakeholder group recommended dropping the program.

Question 19

How are difference scores computed? Why are these scores considered unreliable by some statisticians? What factors can increase their reliability?

Accepted Answer

Difference scores are computed by subtra

Question 20

Define impact assessment in evaluation research. Name three criteria for evaluating impact.

Accepted Answer

Impact assessment in evaluation research

The stakeholders in a program being evaluated include

A local history confound occurs in an evaluation study when

Explain how one can assess whether an evaluation research program was properly implemented.

As a tool for evaluation research, meta-analysis can

What is a target population in evaluation research? How can a researcher ensure that the selected population is unbiased? What are the consequences of any potential bias?

Instrumental utilization of the results of evaluation research occurs when the results

What is the Reliable Change Index? Give one example of how it can be used in evaluation research.

Dr. Kuh has designed a program to help first generation college students succeed in their first year. A proximal outcome would be

Describe how unintended effects can operate at the program implementation stage. Are those effects generally positive or negative?

Evaluation research is perceived by decision makers to be useful when it

________ outcomes occur in clients' broader social environment and are often reflected in social indicators.

A ________ evaluation addresses the overall effectiveness of a program relative to its goals.

Define control group contamination. Describe three possible sources of this type of contamination.

Dr. Nezu believes removing a component from a program is unlikely to change its effectiveness. Which design would best test his hypothesis?

The client population of a program consists of

The reliable change index

Under what circumstance might null results in an evaluation research study have practical significance?

How are difference scores computed? Why are these scores considered unreliable by some statisticians? What factors can increase their reliability?

Define impact assessment in evaluation research. Name three criteria for evaluating impact.

Behavioral Science

Research Strategies: an Overview

The Ethical Treatment of Research Participants

The Professional and Social Responsibilities of Scientists

Formulating a Research Question

Developing a Measurement Strategy

The Internal Validity of Research

The External Validity of Research

True Experiments

Field Research

Correlational Designs

Factor Analysis, Path Analysis, and Structural Equation Modeling

The Single-Case Research Strategy

Qualitative Research and Interviewing, Rachel Kraus, Ball State University

Survey Research

Data Collection

Interpreting Research Results

Literature Reviewing

Filters

Exam 16: Evaluation Research

The stakeholders in a program being evaluated include

A local history confound occurs in an evaluation study when

Explain how one can assess whether an evaluation research program was properly implemented.

As a tool for evaluation research, meta-analysis can

What is a target population in evaluation research? How can a researcher ensure that the selected population is unbiased? What are the consequences of any potential bias?

Instrumental utilization of the results of evaluation research occurs when the results

What is the Reliable Change Index? Give one example of how it can be used in evaluation research.

Dr. Kuh has designed a program to help first generation college students succeed in their first year. A proximal outcome would be

Describe how unintended effects can operate at the program implementation stage. Are those effects generally positive or negative?

Evaluation research is perceived by decision makers to be useful when it

________ outcomes occur in clients' broader social environment and are often reflected in social indicators.

A ________ evaluation addresses the overall effectiveness of a program relative to its goals.

Define control group contamination. Describe three possible sources of this type of contamination.

Dr. Nezu believes removing a component from a program is unlikely to change its effectiveness. Which design would best test his hypothesis?

The client population of a program consists of

The reliable change index

Under what circumstance might null results in an evaluation research study have practical significance?

How are difference scores computed? Why are these scores considered unreliable by some statisticians? What factors can increase their reliability?

Define impact assessment in evaluation research. Name three criteria for evaluating impact.

Behavioral Science

Research Strategies: an Overview

The Ethical Treatment of Research Participants

The Professional and Social Responsibilities of Scientists

Formulating a Research Question

Developing a Measurement Strategy

The Internal Validity of Research

The External Validity of Research

True Experiments

Field Research

Correlational Designs

Factor Analysis, Path Analysis, and Structural Equation Modeling

The Single-Case Research Strategy

Qualitative Research and Interviewing, Rachel Kraus, Ball State University

Survey Research

Data Collection

Interpreting Research Results

Literature Reviewing

Filters