Deck 6: Research Validity

Question 1

A researcher hypothesizes that Hispanic children will do worse on a verbal aptitude test than will Asian children. As she administers the tests to each child, she unintentionally gives the Asian children lots of hints and extra time to finish but she has no interaction with the Hispanic children and takes their papers away as soon as the allotted time has passed. This is an example of

A) participant reactivity.
B) the experimenter-expectancy effect.
C) the negative presentation effect.
D) intertreatment interaction.

Answer 1

B

Question 2

Dr. Ferenz, an older male psychologist, conducts extensive interviews with rape victims. From these interviews he finds that the rape victims display little about the traumatic effects of this crime. However, Dr. Martinez, a middle-aged female psychologist, replicates his study and finds a rich portrait of the after-effects of rape. Why would the results of these two studies differ so widely?

A) the two probably asked very different kinds of questions, due to experimenter expectancy
B) the different results are probably due to an experimenter-attribute effect because female rape victims are probably more comfortable disclosing information to a woman than to a man
C) the participants probably figured out what each experimenter wanted to know, and gave them the answers they wanted, a demand characteristic
D) Dr. Martinez was probably "mining" her data for facts that weren't really there, an experimenter-bias effect

Answer 2

B

Question 3

Which of the following is NOT an example of psychosocial experimenter attributes?

A) hostility toward the participant
B) the experimenter's gender
C) the need for acceptance
D) the experimenter's frustration level

Answer 3

B

Question 4

For her senior thesis, Nancy surveys fellow students about cheating. Her findings suggest that cheating is virtually non-existent on her campus. This is not consistent with the fact that, in the past semester, two dozen students have been found guilty by the Honor Board of cheating. Which of the following explanations would fall under the heading of "reactivity?"

A) Nancy's data were a "one shot" picture, and therefore could not be too accurate
B) the questions on Nancy's survey probably were not phrased well enough to collect accurate data
C) it is known that participants in research like to present themselves as positively as possible, so students that cheated probably lied and did not report their cheating on the survey
D) Nancy's survey was confounded by being too demanding

Question 5

An experiment was designed to test the effects of rehearsal on memory. In order to hold the extraneous variable "time of day" constant, the experimenter decides to run two groups simultaneously. The rehearsal group is given the instructions live and the other non-rehearsal group sees the same instructions in the next room via closed-circuit television. Live-versus- television is

A) an extraneous variable, but not a confounding variable.
B) a confounding variable, but not an extraneous variable.
C) neither an extraneous nor a confounding variable.
D) both an extraneous and a confounding variable.

Question 6

Which of the following could be used to eliminate the confounding influence of extraneous variables?

A) holding the variable constant
B) rejecting the participant that is above or below a certain criteria
C) using random assignment
D) a and c

Question 7

Whether experimental research is conducted in the laboratory, a doctor's office, or at the mall is considered the

A) research participants.
B) independent variable.
C) experimental setting.
D) dependent variable.

Question 8

The quality of an experiment is judged by its internal validity. When we say that an experiment is internally valid, it means that

A) our operational definitions of the dependent and independent variables accurately capture the concepts those variables are meant to represent.
B) within the context of the experiment, each time we take a measure of the dependent variable from a participant, we get roughly the same result.
C) our operational definitions of variables are valid, but only within the context of that one experiment.
D) we can confidently state that any observed changes in the dependent variable result from manipulations of the independent variables) and nothing else.

Question 9

A variable is one that is not controlled by the experimenter; if this variable has a differential impact on the different groups in the study then it is called a variable.

A) confounding; extraneous
B) extraneous; confounding
C) independent; extraneous
D) independent; confounding

Question 10

Which of the following is an example of the construct of research participants?

A) individuals with ADHD
B) a laboratory observation room
C) reaction time of the participants
D) the experimenter

Question 11

Which of the following is an example of the phenomenon of "demand characteristics" acting in an experiment?

A) when participants complete the experiment, they are instructed not to tell anyone else about what they have experienced
B) participants in one study report only once, for half an hour, to the research lab, but participants in another study have to report three times, for an hour each time, and resent having to spend so much time
C) participants in a study are drawn from the large Introductory Psychology classes, and students who have already participated pass rumors to others about what the experiment is trying to "prove" and try to determine what the experiment is about from the equipment in the laboratory
D) in one psychology experiment, the researcher does not allow participants to withdraw from the study, even if they do not like the kinds of tasks they are asked to perform

Question 12

A sports psychologist develops a new technique that might improve accuracy in shooting free throws in school children. To test her technique she arranges to give the new program to a girls' physical education class and the standard technique to the boys' physical education class. She finds that indeed the girls' accuracy in shooting improved more than the boys' accuracy. A colleague correctly points out

A) that as this was an experiment the new technique caused the observed differences.
B) that gender was an extraneous variable that was held constant.
C) that the experiment is confounded by gender and is therefore inconclusive.
D) that the results were actually caused by gender and not the technique.

Question 13

Using operational definitions to define research participants, the experimental setting, the independent or dependent variable is important for which type of validity?

A) statistical conclusion validity
B) construct validity
C) discriminant validity
D) projective validity

Question 14

Which of the following are types of experimenter attributes that may bias a research study?

A) psychosocial attributes
B) situational factors
C) biosocial attributes
D) all of the above

Question 15

Which of the following may compromise the statistical conclusion validity of an experiment?

A) if the researcher does not have a Ph.D. in statistics
B) if the experiment has too few participants, thus reducing power
C) if the experimental findings are limited to a particular group of participants
D) if the research participants have a positive self-presentation motive

Question 16

Dr. Gomez completed her study investigating the effect of stress on consumption of carbohydrates and, based on her analysis of the data, concluded that increasing a person's stress level leads to an increase consumption of carbohydrates. If this inference is correct, the study has achieved

A) statistical significance.
B) demand characteristics.
C) treatment diffusion.
D) statistical conclusion validity.

Question 17

Which of the following is an example of an "experimenter-expectancy effect?"

A) participants in a study are nicebehave differently for the attractive experimenter than for the less attractive experimenter
B) a researcher finds that his participants did not behave as he expected them to, so he changes his statement of his hypothesis when he writes up his results
C) participants come to experiments with preconceived notions of what they are supposed to do, and perform based on those notions rather than on what they are asked to do.
D) the experimenter "expects" females to experience more difficulty on a math task so the experimenter unintentionally behaves in ways that undermine the performance of females on the math task

Question 18

Without internal validity, an experimenter cannot

A) control extraneous variables.
B) manipulate the independent variables).
C) measure the dependent variable.
D) make statements about cause and effect.

Question 19

We like to believe that the only major influence on participant performance in an experiment is the manipulation of the independent variable. In fact, some research has shown that

A) performance can be influenced by the participants' perceptions of what the experimenter "wants" them to do.
B) human behavior is relatively uninfluenced by manipulation of environmental conditions.
C) human participants are very good at figuring out the purpose of an experiment, and enjoy subverting the research by giving data that disconfirm the hypothesis.
D) manipulations of environmental conditions are to powerful to test in research.

Question 20

When experimenters have a vested interest in confirming their hypotheses, we sometimes see that

A) researchers can subtly, unintentionally influence participants' performances.
B) the evidence doesn't actually support their hypotheses.
C) data analysis changes the way the hypothesis is framed.
D) subjects try harder to mess up the data.

Question 21

A school district decided to try a new computerized testing system. Two schools were selected for the trial based on the students being similar at the beginning of the school year. Students at one school were given the paper-and-pencil version of the test while the students at another school received the computerized test version. The results indicated that the students performed better on the computerized version. However, someone pointed out that on the day of the test there was a fire drill at the school using the paper-and-pencil test. This fire drill confound would be described as

A) a selection-history bias interaction.
B) a selection bias.
C) a testing effect.
D) an instrumentation effect.

Question 22

In a study designed to assess the effectiveness of a new therapeutic technique to help patients stop smoking, research participants are asked to record in a diary each time they smoke a cigarette. At the beginning of the two week experiment participants are very careful about noting every time they smoke a cigarette. However by the end of the two weeks they often forget to record each cigarette they smoke making it appear that they smoked less at the end of the experimental treatment. The results in this study are confounded by

A) instrumentation.
B) attrition.
C) history.
D) maturation.

Question 23

Dr. Prentice, principal of an elementary school, instructs her teachers to employ a new aggression-reduction program with the second graders, and to keep using it on that group of students for the next three years. She finds that relative to the start of the program, aggression in that group of students is down 35%. This example illustrates the possibility of as a plausible rival hypothesis.

A) history
B) maturation
C) attrition
D) selection

Question 24

"External validity" of an experiment refers to the extent to which

A) the operational definitions of variables really do represent the concepts they are meant to capture.
B) we can be confident that the experiment is testing the hypothesis we think it is testing.
C) we can make causative statements about the relationship between the dependent and independent variables.
D) the results of an experiment can be generalized across people, settings, times, treatments, and outcomes.

Question 25

The confounding effects referred to collectively as "maturation" include all of the following plausible rival hypotheses EXCEPT

A) aging.
B) instruction.
C) fatigue.
D) boredom.

Question 26

Which of the following is NOT true regarding a testing confounding variable?

A) Involves changes in the measurement of the DV from pre to post-test.
B) Typically can be controlled for with a control group.
C) Only comes into play when participants are tested more than once within a study.
D) Changes in participants' scores on the second administration of the test is affected by the first administration.

Question 27

Dr. Jones is conducting some research on two college campuses involving food allergies. At one campus educational information on the diagnosis of food allergies is disseminated Campus A) while it is not distributed at the other campus Campus B). Dr. Jones measures through random email surveys, students' knowledge of food allergies. During the week she is disseminating the information at Campus A the student newspaper also runs a story about students with food allergies which is not part of Dr. Jones' program. If the student newspaper article influences students' knowledge of food allergies, what potential confounding variable is at work in this example?

A) history
B) maturation
C) differential history
D) differential attrition

Question 28

One reason it is difficult to conduct long-term studies of the same group of participants is that physical and psychological changes in the participants over time can become confounded with the independent variables) of the experiment. Researchers refer to this as the problem of

A) maturation.
B) education.
C) regression artifact.
D) statistical progression.

Question 29

Dr. Kenji wants to determine whether his new phonics program helps children's reading comprehension. He gives a fifth grade class the program and also studies a second grade class that does not go through the program. He finds that the fifth graders read with better comprehension. Identify the confounding variable in this research design.

A) the grade the students are in
B) the phonics program
C) reading comprehension scores
D) not giving one group the program

Question 30

A health psychology student wanted to know if giving an electric shock to participants each time they inhale from a cigarette will reduce how deeply they inhale. She arranges for smokers to be randomly assigned to two groups: the first group receives no shock and the second group a painful shock for each puff. Some participants quit the experiment, mostly from the painful- shock group. At the end, there was no difference between the two groups on depth of inhaling. Which of the following is a correct conclusion?

A) electric shock has no effect on depth of inhalation.
B) the study is confounded by differential attrition.
C) the study is confounded by selection bias.
D) the study was confounded by regression artifact.

Question 31

In the context of extraneous variables, "history" refers to

A) events that can influence the dependent variable, that occur between the first and second measurement of that variable.
B) events that can influence how accurately one can control the independent variable.
C) differences in participants' knowledge of the subject matter of the experiment in which they are participating.
D) biases brought to the study by one pre-selected group and not another.

Question 32

"Regression artifact" refers to the mathematical fact that

A) people who score above average on pretests tend to have lower scores on posttests, and those who score below average on pretests tend to increase their scores on posttests.
B) any participant's score on a pretest can be expected to be lower on a posttest, because of the "law of averages."
C) participants who have extreme scores, high or low, on a pretest tend to have posttest scores that are even more extreme.
D) posttests are often unnecessary, since they tend to give, on average, the same results as those obtained on pretests.

Question 33

Dr. Dunn is conducting some research involving a new math learning program. To test the effectiveness of the new math program, he selects two sections of his high school math classes to participant in his study. His 1st period 9th grade math class will be the experimental group and his 4th period 11th grade math class will be the control group. Which of the following confounding variable threats is possible in this example?

A) selection-maturation
B) selection-instrumentation
C) selection-testing
D) selection-regression

Question 34

Janice is taking the SAT to meet an entrance requirement for a college. On her first attempt she scores way below average. She decides to review and then take the test again to see if she can improve her score. She then scored higher the second time she took the test. This example illustrates the possibility of as a plausible rival hypothesis.

A) history
B) maturation
C) regression artifact
D) selection

Question 35

Selection bias can be a potential threat to internal validity when subjects are assigned to groups

A) in a random fashion
B) on the basis of different criteria for the different groups
C) in a way that insures that similar people are in each group
D) none of the above

Question 36

In an experiment on the effects of massed versus distributed practice on learning, introductory psychology students who volunteered for extra credit were randomly assigned to one of two groups. Both groups were to practice a perceptual-motor task five times for 10 minutes each. Group 1 did all five 10-minute practice sessions in one evening while group 2 was asked to do one 10-minute practice session a night for five consecutive nights. All of the participants in Group 1 completed the experiment, but only 30% of the participants in Group 2 showed up for all five nights. This study is an example of confounding by

A) differential attrition.
B) selection bias.
C) history.
D) demand characteristics.

Question 37

An urban community clinic wants to try a new method of getting drug addicts to kick their addiction. They advertise for participants in their lobby, and allow volunteers to choose to participate either in the treatment or control group. Three months later, they find that 65% of the volunteers are off drugs. This example illustrates the possibility of as a plausible rival hypothesis.

A) history
B) maturation
C) regression artifact
D) selection

Question 38

The confounding variable where participants are lost from one research group and not others is called

A) attrition
B) selection
C) differential history
D) differential attrition

Question 39

Sanjay is studying whether his racism awareness program increases people's awareness of issues of race and ethnicity in his city. On Monday, he gives an attitude survey to his participants to gauge their awareness. On Tuesday through Thursday, he puts them through his program. On Wednesday, a local predominantly black church is firebombed and the KKK claims responsibility. On Friday, Sanjay gives his awareness survey again and finds awareness of racial issues has increased dramatically. This example illustrates the possibility of as a plausible rival hypothesis.

A) history
B) maturation
C) regression artifact
D) selection

Question 40

Some experiments require observers to take many measures of the dependent variable over time and across participants. If they become better observers of the dependent variable during the course of the study, it produces a confound referred to as

A) instability.
B) unreliability.
C) instrumentation.
D) insensitivity.

Question 41

Which of the following is a way to increase the external validity of a study?

A) increase internal validity
B) decrease the use of random selection
C) replication of the study
D) establish cause and effect

Question 42

Dr. John, in his study investigating the effect of stress on consumption of carbohydrates, found that stress increased the consumption of carbohydrates. However, he also found that stress increased the desire to consume carbohydrates, the amount of time a person thought about carbohydrates, and the amount of time a person thought about eating. Because stress had the same affect on these other measures that were related to consumption of carbohydrates, the study had

A) outcome validity.
B) treatment variation validity.
C) dependent variable validity.
D) ecological validity.

Question 43

When the results of a study can be generalized across variations in the treatment we have

A) treatment variation validity.
B) outcome validity.
C) temporal validity.
D) ecological validity.

Question 44

Issues regarding external validity can be partitioned into several categories. Which of the following is NOT one of those categories?

A) population
B) ecological
C) spatial
D) temporal

Question 45

If an experiment is ecologically valid, then the effect of the treatment used in that experiment should be

A) stronger in the real world than in the laboratory.
B) evident in other settings.
C) stronger in the laboratory but still detectable in the real world.
D) more easily measured in conditions in which control is possible.

Question 46

When the results of a study can be generalized across different but related dependent variables, that is called

A) treatment variation validity.
B) outcome validity.
C) temporal validity.
D) ecological validity.

Question 47

Why is it important for experiments to be externally valid?

A) so that we know that the results of our studies have significance beyond the situation in which they were conducted.
B) so that we know which variables influence which other variables.
C) so that others do not have to duplicate our work later.
D) so that we can control extraneous variables

Question 48

In most cases there is a relationship between internal and external validity.

A) covarying
B) direct
C) indirect
D) inverse

Question 49

In general, the accessible population is

A) easily made to represent the target population.
B) the same as the target population.
C) often not representative of the target population.
D) larger than the target population.

Question 50

Failure to identify interactive effects of independent variables can result in

A) confounding variables
B) a failure to generalize results
C) external validity
D) none of the above

Question 51

While studying the effects of overcrowding on aggressive behavior in elephants, Dr. Webster made sure to have his research assistants gather data before, during and after the breeding season. Her concern was probably due to

A) a limited sample population.
B) a possible seasonal variation effect.
C) mediation of expectancy.
D) a possible attrition effect.

Question 52

Which of the following might contain a serious threat to the external validity of a study?

A) a survey with a return rate of 30%
B) statistically significant findings in both the clinical setting and at home
C) evidence of a consistent effect in the winter and the spring
D) evidence that shows SAT scores and GPA change with improved study habits

Question 53

Dr. Tubs developed a 7-step treatment plan for weight loss. Because of the success of this treatment plan it became a standard part of the education of all graduate clinical psychology students. However, many of these students, after completing their education and setting up their own private practice, modified this treatment plan as they continued to treat obese individuals. Although the 7-step treatment plan was slightly modified in many ways by many different clinical psychologists, it was still very effective for weight loss. This treatment plan

A) was probably not very effective because the success of the plan was dependent on the skill of the therapist to motivate the client.
B) had dependent variable validity.
C) had treatment outcome validity because it worked regardless of how it was changed.
D) had internal validity because it worked regardless of how it was changed.

Question 54

Most psychological researchers want to know about behavior of people in general, but are often restricted to studying the college students immediately available to them on their campus. "All people" is the population while "college students" is the population.

A) valid; accessible
B) target; sample
C) valid; sample
D) target; accessible

Question 55

Paul wanted to insure that the results of his study could be generalized so he manipulated the independent variable and then identified and measured all the dependent variables that he thought might be affected by the independent variable. What type of validity was Paul focusing on?

A) internal validity
B) population validity
C) ecological validity
D) outcome validity

Question 56

Jacques surveys fellow students to rate their preferences for various courses on campus. He finds that the sciences are least popular whereas the humanities are most popular. However, he neglects the fact that he performed his survey right after midterm grades were announced. His friend Ghislaine follows up by doing the same survey at the beginning of the following semester and gets very different results. This may be an issue of

A) ecological validity.
B) temporal validity.
C) internal validity.
D) global validity.

Question 57

Although most research is done in the laboratory setting, eventually we would want to know that laboratory results are applicable in a real world setting. This is an issue of

A) global validity.
B) ecological validity.
C) temporal validity.
D) internal validity.

Question 58

When the results of a study can be generalized across settings, situations, and environmental conditions, the study has

A) treatment variation validity.
B) outcome validity.
C) temporal validity.
D) ecological validity.

Question 59

What is an important way to help insure that one's experimental sample of participants is adequately representative of the population from which it is drawn?

A) pretest all participants
B) survey participants about demographic characteristics
C) randomly select participants from the population
D) use a small sample

Question 60

External validity is the extent to which the results of a particular experiment can be generalized to the

A) population.
B) other settings.
C) other times.
D) all of the above.

Question 61

Although external validity is a desirable quality in an experiment, it is often attained at the expense of

A) generalizability.
B) random selection.
C) applicability.
D) internal validity.