Deck 14: Selecting a Research Design for a Group Evaluation Study

Question 1

The main reason we need to know the research design employed in an evaluative study is that we will be in a better position to determine whether ______.

A) we can scientifically generalize our study findings to our study population
B) it was the intervention that caused the measured client growth rather than something else
C) our data can be explained by chance
D) we can logically generalize our study findings to our study population

Answer 1

it was the intervention that caused the measured client growth rather than something else

Question 2

What is one of the three conditions for determining causation?

A) There must be a relationship between the cause and the effect.
B) There must be other causes of the effect that are just as important as the variable that is the potential cause.
C) A change in the effect must precede a change in the cause in time order.
D) There must be a random sample that is used to analyze causation.

Answer 2

There must be a relationship between the cause and the effect.

Question 3

A threat to internal validity refers to ______.

A) various aspects of the intervention
B) a danger in the intervention identified by research that has been done on its effect for the client
C) the aspects of the intervention that have proven to be least effective
D) things other than the intervention that might be the cause of the measured client growth

Answer 3

things other than the intervention that might be the cause of the measured client growth

Question 4

Maturation, as a threat to internal validity, refers to ______.

A) normal growth over time
B) changes in the client's environment during the treatment period
C) changes in the client's environment before the treatment period
D) the effect of being tested while being treated

Question 5

History, as a threat to internal validity, refers to ______.

A) normal growth over time
B) changes in the client's environment during the treatment period
C) changes in the client's environment before the treatment period
D) the effect of being tested while being treated

Question 6

The ______ design is an example of a research design for the evaluation of service given to a group of clients.

A) AB
B) random sample
C) one-group pretest-posttest
D) limited AB

Question 7

Which research design controls for maturation as a threat to internal validity?

A) the comparison group design
B) the one-group pretest-posttest design
C) both the comparison group design and the one-group pretest-posttest design
D) the B design

Question 8

Which of the following statements about internal validity is true?

A) One of the most important threats to internal validity is being unable to generalize findings on a scientific basis.
B) The sophistication of the statistical test we employ is a major indicator of the extent that threats to internal validity exist in our situation.
C) The better our sampling method, the more we need to be concerned with threats to internal validity.
D) When we speak of threats to internal validity, we are speaking of probabilities, not possibilities.

Question 9

What are the general types of research designs for the evaluation of human services?

A) group research designs and hyperbolic research designs
B) group research designs and single-subject research designs
C) single-subject research designs and single-system research designs
D) single-subject research designs, group research designs, and hyperbolic research designs

Question 10

When you employ the one-group pretest-posttest research design, you engage in which steps?

A) After you measure each client several times during a baseline period and several times during a treatment period, you compare the differences to see if there was improvement and if the data were statistically significant.
B) You give two groups of people a pretest and a posttest and examine the data to see if there was improvement and if the data were statistically significant.
C) You measure one group of clients one time before treatment begins and one time at the end of treatment and you compare the difference to see if there was improvement, and whether the data were statistically significant.
D) You measure a single client one time before treatment begins and several times during the treatment period and examine the data to see if there was improvement and if the data were statistically significant.

Question 11

Which of the following does the best job of controlling for threats to internal validity?

A) the basic experimental design
B) the one-group pretest-posttest design
C) the posttest-only one-group design
D) the limited AB single-subject design

Question 12

When you employ an experimental research design, you ______.

A) measure a single client several times throughout the treatment period
B) employ a random sample of people for your study
C) select study subjects on a purposive basis
D) assign study subjects to their group status on a random basis

Question 13

In which of the following situations is it most likely that maturation is a probable threat to internal validity and should be a special concern when selecting a research design?

A) You will provide a 6-week treatment for a group of depressed clients who have been depressed for a minimum of a year.
B) You will provide a 1-year treatment program for a group of people who have recently (within the past month) lost a spouse to death. You will measure depression as the vehicle of success.
C) You will provide tutoring to a group of at-risk middle school students for 8 weeks and you will compare their grades during the current 8-week grading period to the previous 8-week grading period. These clients have been experiencing poor grades for at least a year.
D) You will measure knowledge of job hunting skills once at the beginning of a 3-hr training session for unemployed clients and again at the end of this training session.

Question 14

You find that maturation is a threat to the internal validity and should be a special concern in your proposed study. However, the only feasible design in this situation doesn't control the threat of maturation. What should you do?

A) You should not conduct an evaluation study in this situation.
B) You should conduct the study with the feasible design and you do not need to worry about this problem as a major limitation of your study methods.
C) You should conduct the study with the best design that is feasible and report its failure to address maturation as a major limitation of your study methods.
D) You should encourage others to conduct this study for you and let them worry about threats to internal validity.

Question 15

The research design that you employ in an evaluative study will reveal the ______.

A) structure whereby data will be collected and services will be offered
B) steps to employ in the selection of the subjects for your study
C) number of clients who will be included in your study
D) model of the intervention that you will employ

Question 16

If job satisfaction is the cause of job performance, you would find research studies that revealed that ______.

A) marital harmony was found to be correlated with job satisfaction
B) changes in job performance were found to follow changes in job satisfaction
C) changes in job satisfaction were found to follow changes in job performance
D) job satisfaction was found to be correlated with work satisfaction

Question 17

What are some of the most important things that indicate whether maturation should be a threat to internal validity of special concern in a given situation?

A) the duration of the treatment and the sampling method employed
B) the reliability and the validity of the tool used to measure client progress
C) the sampling method employed and the time between measurements
D) the length of time the target behavior has been experienced and the length of time of the intervention

Question 18

______ designs do the best job of controlling for threats to internal validity.

A) Experimental
B) Preexperimental
C) Nonexperimental
D) Quasi-experimental

Question 19

Which of these evaluative research designs does the best job of supporting the argument that it was the treatment that caused the measured client growth in the research study?

A) the one-group pretest-posttest design
B) the alternative treatment design
C) the basic experimental design
D) the B design

Question 20

What tasks should you complete before you choose the research design for your evaluation study?

A) select the statistic to test your study hypothesis and analyze your data
B) determine the target behavior, the study population, and the tool for measuring client progress
C) connect your knowledge base to the study conclusions you will draw
D) collect and analyze your data

Question 21

If a client has experienced the target behavior for more than a year, history should be considered a major threat to internal validity.

Question 22

The length of time the client has experienced the target behavior is an important determinant of whether we should consider maturation a threat to internal validity that should be of special concern.

Question 23

A group research design is only used when your clients are being given a group treatment (e.g., a support group); it is not appropriate for a group of clients being treated individually for a given target behavior using a common measurement tool.

Question 24

The posttest-only control group research design controls for maturation and history as threats to internal validity.

Question 25

According to the book, we should focus our attention on identifying those threats to internal validity that should be of special concern in our specific situation, rather than all threats to internal validity.

Question 26

The better your research design, the better you are able to claim that your data cannot be explained by chance.

Question 27

The comparison group design is better than the one-group pretest-posttest design at controlling for maturation as a threat to internal validity because the gain on the measured behavior for the comparison group is an indicator of the effect of normal growth over time.

Question 28

There are three conditions for determining causation in research: (1) there must be a relationship between the cause and the effect; (2) a change in the cause must precede a change in the effect in time order; and (3) you must rule out other causes of the effect.

Question 29

Statistical regression as threat to internal validity refers to the fact that extreme scores for a scale (i.e., very high or very low scores) tend naturally to move toward the mean with repeated measurements; thus, it would not make sense to take only the clients with the extremely low pretest scores as your clients for an evaluation study and employ the one-group pretest-posttest design.

Question 30

When you fail to view your evaluation of practice as having any of the common threats to internal validity, you do not need to discuss this in your report of your study.

Question 31

Answer the following four questions:
(1) What is maturation as a threat to internal validity?
(2) What is history as a threat?
(3) Does the one-group pretest-posttest research design control for maturation as a threat to internal validity?
(4) Explain why or how your answer to Question 3 is true or false.

Question 32

Suppose you need to evaluate your practice where you are treating a group of 12 victims of violent crimes. These clients were victimized at some time in the past 3 months and they are experiencing anxiety as a result, and you will measure anxiety as one means of evaluating the success of your service. You are providing a group treatment intervention where you will meet with your 12 clients once per week for a period of 6 months. You have four questions to answer:
(1) What are the two criteria for selecting a research design that you should answer?
(2) How do you answer these two questions in this situation?
(3) What research design do you think would be feasible in the typical practice situation where you faced this situation?
(4) Does this design control for the threats that should be of special concern?

Question 33

The measurement structure of the group research design that is labeled the posttest-only research design is that a single group of clients are measured on the target behavior only at the end of the treatment period. Describe the measurement structure of each of these research designs: (1) the one-group pretest-posttest design, (2) the comparison group design, and (3) the alternative treatment design.

Question 34

Illustrate how you might evaluate your own practice using a group research design by answering each of the following questions:
(1) What is the target behavior that you will measure?
(2) What research design will you employ?
(3) What threats to internal validity do you believe should be of special concern in this situation, and why do you believe this threat should be of special concern?
(4) Does your chosen research design control for the threats of special concern (If there are no threats, just say so as your answer to this question)?

Question 35

One way of classifying various group research designs is to place them into the categories of (1) preexperimental designs, (2) quasi-experimental designs, and (3) experimental designs. Describe the distinction between these categories of research designs and give one example of each type.