Deck 8: Conducting Research That Evaluates Services

Question 1

Evaluative studies are different from others because evaluative studies will always have a(n) ______.

A) study sample from a study population to which you wish to generalize your findings
B) research purpose to guide the study
C) study plan to follow
D) intervention or program to be evaluated

Answer 1

intervention or program to be evaluated

Question 2

In human service evaluation, the concept of input refers to the ______.

A) quality of the services being offered
B) nature of the need being addressed
C) client outcomes being sought
D) efficiency of the services being offered

Answer 2

nature of the need being addressed

Question 3

Which of the following would be an example of a client target behavior for an evaluation study?

A) the clients of the Adoption Service
B) the sample of people to be evaluated
C) the depression level of the clients begin treated
D) the statistical test to be used to test the study hypothesis

Answer 3

the depression level of the clients begin treated

Question 4

A good definition of the target behavior in evaluative research is most helpful for the researcher in selecting the ______.

A) tool for measuring client progress
B) intervention to treat the target behavior
C) statistic to test the study hypothesis
D) words to use in the first sentence of the study conclusions

Question 5

Analyzing the target behavior is most helpful for the research when selecting the ______

A) intervention to be employed
B) tool for measuring client progress
C) words to use to introduce the study in the study report
D) statistic to test the study hypothesis

Question 6

Which of the following best describes evidence in evaluative research?

A) the wisdom that comes from practice experience
B) any information asserted on behalf of a claim
C) credible information collected from studies of a scientific nature
D) information that has been asserted as true by a majority of people

Question 7

When seeking evidence about a given target behavior, it is advisable for the social work researcher to start the search with attempts to find a ______ in regard to the selected target behavior.

A) single research report of evidence
B) systematic review of evidence
C) a meta-analysis of evidence
D) a traditional literature review of evidence

Question 8

When evaluating a human service system or general program of service, you might ask the following questions: (1) Are the services of the program addressing client need? (2) Are we delivering good quality services? (3) Are we offering out services efficiently? (4) Are our client satisfied with the services they have received? What is the most important question that has not been asked yet?

A) Has the cost per unit of service increased in recent years?
B) To what extent do our personnel meet the necessary qualifications for their jobs?
C) Have our services improved client outcome?
D) Have we redefined our mission in recent years?

Question 9

Which of the following questions addresses the issue of service efficiency?

A) Is our cost per therapy session higher than that for other agencies that provide comparable therapy?
B) Is the total cost of our program higher than that of a comparable agency?
C) To what extent are our clients satisfied with our client charges for service?
D) To what extent do our clients believe they are receiving services of good quality?

Question 10

The mechanism for determining if chance is a reasonable explanation of the data you have analyzed in regard to pretest and posttest scores for depression is ______.

A) the descriptive statistics that describe the characteristics of the clients in your study
B) the inferential statistics that examine data about your clients in regard to an explanatory research question
C) whether you can generalize your findings on a scientific basis
D) the results of your statistical analysis of your data for pretest and posttest scores

Question 11

In an evaluative study, you can conclude that your service was effective if your data on client outcome ______.

A) went in the hypothesized direction
B) went in the hypothesized direction and was statistically significant
C) were found to be statistically significant
D) provided support for your services among your funding sources

Question 12

Suppose you are evaluating the effectiveness of your strengths-based school social work tutoring services in the improvement of the likelihood that your at-risk middle school students will graduate from high school. You will provide your services in hourly sessions that are given three times per week for 8 weeks. These tutoring sessions will be offered by school social workers with the BSW degree. The students' grades in school will be measured as the outcome of the service. What of the following would be the best way to characterize the structure of this service?

A) to improve the likelihood that the clients will graduate from high school because improved grades will make this more likely
B) to improve the grades of the clients
C) to offer strengths-based tutoring services that follow the tenets of strengths-based social work practice
D) to provide strengths-based school social work tutoring services in hourly sessions that are given three times per week for 8 weeks

Question 13

Suppose you are evaluating the effectiveness of your strengths-based school social work tutoring services in the improvement of the likelihood that your at-risk middle school students will graduate from high school. You will provide your services in hourly sessions that are given three times per week for 8 weeks. These tutoring sessions will be offered by school social workers with the BSW degree. The students' grades in school will be measured as the outcome of the service. What of the following would be the best way to characterize the goal of this service?

A) to improve the likelihood that the clients will graduate from high school
B) to improve the grades of the clients
C) to offer strengths-based tutoring services that follow the tenets of strengths-based social work practice
D) to provide strengths-based school social work tutoring services in hourly sessions that are given three times per week for 8 weeks

Question 14

Suppose you are evaluating the effectiveness of your strengths-based school social work tutoring services in the improvement of the likelihood that your at-risk middle school students will graduate from high school. You will provide your services in hourly sessions that are given three times per week for 8 weeks. These tutoring sessions will be offered by school social workers with the BSW degree. The students' grades in school will be measured as the outcome of the service. What of the following would be the best way to characterize the objective of this service?

A) to improve the likelihood that the clients will graduate from high school
B) to improve the grades of the clients
C) to offer good quality services
D) to provide strengths-based school social work tutoring services in hourly sessions that are given three times per week for 8 weeks

Question 15

Which of the following is the basis for declaring that you can logically generalize your findings to your study population?

A) when you have a random sample
B) when you are employing a quasi-experimental research design
C) when you have data that show similarity between your sample and the target population on variables that are relevant to your study
D) when you have proof that your study tools are reliable

Question 16

A major advantage of composing a measurement scale yourself, rather than using a published one, is that it will ______.

A) likely be more relevant to the specific outcome behaviors of the client you are serving
B) likely be proven to be more reliable and valid than a published scale
C) be better for use with the quasi-experimental research design, the design most used by social work practitioners
D) lead to better generalization of study results

Question 17

When you search for a published tool for measuring client outcome, you should pay careful attention to the ______.

A) nature of the intervention you are employing
B) research design that you are employing
C) method you employ to secure your study sample from your study population
D) definition of the variable you want to measure

Question 18

The comparison group evaluative research design is better than the one-group pretest-posttest design because you are in a better position to ______.

A) generalize your study findings to the study population and assert that generalization is a major strength of your study methods
B) assert that it was your intervention that caused the measured client growth rather than something else, like normal growth over time
C) assert that you have achieved reliability and validity in regard to the measurement of your study variables
D) assert that your data were found to be statistically significant

Question 19

Which of the following would be considered a single-subject research design?

A) the comparison group design whereby outcomes are compared between a group of clients who received the treatment and a group of people who did not get the treatment
B) the AB design where a single client is measured on outcome several times before treatment begins and several times during the treatment period
C) the one-group pretest-posttest design whereby a group of clients are measured on outcome one time before treatment begins and one time at the end of treatment
D) the basic experimental design whereby a group of clients are randomly assigned to either a treatment group or a control group with comparisons on outcome between the two groups

Question 20

What is meant by the concept of threats to internal validity?

A) things other than the treatment that might be the cause of measured client growth
B) things that prove that the research results cannot be generalized to the study population
C) things that mean the data can be explained by chance
D) things that refer to the client's target behavior in regard to severity and relevance

Question 21

The study hypothesis is an educated guess about the results you expect to find from your data, with reference to the variables being measured.

Question 22

The study hypothesis should assert causation, such as "the posttest scores for depression will be lower than the pretest scores because the treatment addresses the cause of depression."

Question 23

Practical significance is an informed opinion about whether the data results were noteworthy.

Question 24

The first task in the conclusions section of the research report is to summarize the results of your study in regard to the research question.

Question 25

The one-group pretest-posttest research design addresses maturation as a threat to internal validity.

Question 26

Face validity is the extent that key informants express the belief that the measurement tool is accurate in measuring the designated variable.

Question 27

The model of the intervention is the conceptual framework that serves as the logical rationale for offering the intervention in light of what is known about the dynamics of the target behavior being addressed.

Question 28

The objective of the intervention is the statement of the long-term benefits of the intervention for the client, so you might call it a measured amount of progress toward the achievement of the objective.

Question 29

If your target behavior is depression, you would focus the statement of the objective on depression, and you would measure depression.

Question 30

In evaluative research, there is only one criterion for determining if your data supported your hypothesis: you can report that your data supported your study hypothesis if the data went in the predicted direction.

Question 31

What is practical significance? What is statistical significance? Can you have practical significance from the results of your evaluative study if you do not have statistical significance? Explain.

Question 32

What is the study sample you could employ to evaluate practice in a familiar agency? What is the study population for this study? Can you generalize your study findings to the study population on a scientific basis? Explain why. What information would you need in order to generalize your findings on a logical basis?

Question 33

Describe an evaluation of practice you might undertake in a given agency. What research design might you employ in this study? Does this design address maturation as a threat to internal validity? Explain why you believe that maturation should or should not be of special concern in this situation.

Question 34

What should be the focus of the first sentence of the conclusions section of a research report on an evaluative study? Present an imaginary first sentence of a conclusions section that fulfills this expectation.
Ans. The focus of the first sentence of the conclusions section of an evaluative research report would be the results of the study. While the conclusions section will typically have information on the strengths and limitations of the study as well as the implications of the results for social work practice, the initial focus of this section should be the results. For example, the first sentence might say "The data from this study supports the conclusion that cognitive behavioral therapy is effective in the reduction of depression for women suffering from postpartum depression."

Question 35

What are two of the guidelines for stating the study hypothesis for an evaluative study? Prepare a study hypothesis that fulfils this requirement.