Deck 8: The External Validity of Research

Question 1

Explain the concept of "generalizing across." Is this concept generally more important to basic or to applied researchers? Explain your reasoning.

Answer 1

No Answer

Question 2

Explain the concept of "generalizing to." Is this concept generally more important to basic or to applied researchers? Explain your reasoning.

Answer 2

No Answer

Question 3

Distinguish between generalizability and ecological validity.

Answer 3

No Answer

Question 4

Distinguish among the structural, functional, and conceptual components of external validity.

Question 5

What are boundary conditions of a theory? How do researchers establish those conditions?

Question 6

Define the structural component of external validity. What is the focus of this component and what research issues does it address?

Question 7

Define the conceptual component of external validity. What is the focus of this component and what major research issue does it address?

Question 8

Define the functional component of external validity. What is the focus of this component and what major research issue does it address?

Question 9

Name and define four of the major dimensions researchers want their findings to generalize across.

Question 10

Explain how aspects of a physical setting can affect the generalizability of research results.

Question 11

Imagine you are planning a study in which you observe people's behavior. List three things you can do to reduce the possibility that your participants will know they are under observation and so change their behavior because of it.

Question 12

List three characteristics of researchers that can affect participants' responses. How can you reduce or eliminate these effects?

Question 13

Explain how selection of a research sample can affect the generalizability of research.

Question 14

Describe the characteristics of the most typical sample used in psychological research. What are the limitations of using this type of sample?

Question 15

Are the results of research collected from a crowdsourcing site more generalizable than results of a similar study collected using college students? Explain your reasoning.

Question 16

Research has demonstrated that college student samples are WEIRD. Name the characteristics this acronym refers to.

Question 17

List four characteristics on which research volunteers differ from non-volunteers.

Question 18

Non-volunteers, by definition, do not participate in research. Describe two ways that researchers have identified their characteristics.

Question 19

List two research procedure factors that affect the external validity of a study. Explain how each can be addressed at the research design stage.

Question 20

Imagine a researcher found that the effects of a manipulated independent variable interacted with a characteristic of the participants who completed her study. How might this affect the generalizability of the results?

Question 21

Explain how the issue of generalizability applies to operational definitions of a construct. Describe a strategy you would use to address this issue in your own research.

Question 22

Describe three characteristics of dependent variables used in laboratory research that limit their generalizability.

Question 23

Describe an experimental task from a study you are familiar with that is low in ecological validity. How well do these results apply to the "real world?" Explain your reasoning.

Question 24

What is culture? What is it NOT and why?

Question 25

How might culture affect the generalizability of research results? Give an example based on your own research interests.

Question 26

Explain the relationship between time and external validity.

Question 27

Imagine you are conducting an experiment in a natural setting and you randomly assign the people in that setting to experimental and control conditions. Explain why this procedure does not ensure that your results are generalizable.

Question 28

Explain how the conceptual and functional components of external validity are interrelated.

Question 29

Which type of external validity is more of a concern for policy makers? Explain your reasoning.

Question 30

Distinguish between chronic and acute manipulations of independent variables. Give an example of each.

Question 31

Describe two ways in which generalizability can be assessed.

Question 32

Explain why using multiple tests of a hypothesis improves the generalizability of research results.

Question 33

Explain why using multiple operational definitions of a dependent variable improves the generalizability of research results.

Question 34

Consider the question of whether laboratory research has value in applied settings. What are your views on this issue? What is the basis for those views?

Question 35

Defenders of laboratory research believe that some research goals are best addressed in a laboratory environment. List three of those goals and explain the argument for using laboratory research to best achieve them.

Question 36

What is analog research and how can it be used to address the issue of the generalizability of laboratory research to applied settings?

Question 37

Explain why the debate about whether the results of laboratory research generalize to applied settings is unlikely to be resolved.

Question 38

Do you believe it is more important to focus on internal or external validity when designing an experiment? Explain your reasoning.

Question 39

The term external validity refers to the extent to which

A) the characteristics of the research situation match the characteristics of the situation in which the results of the research will be applied.
B) the findings of a study represent general principles of behavior.
C) the conclusions drawn from the data analysis are correct.
D) confounds are absent from the research design.

Question 40

The term _______ refers to the degree to which the characteristics of the research situation match the characteristics of the situation in which the results of the research will be applied.

A) ecological validity
B) generalizability
C) boundary conditions
D) theoretical validity

Question 41

The term _______ refers to the extent to which the results of a study can be replicated across variations in operational definitions, research settings, or research populations.

A) ecological validity
B) theoretical validity
C) boundary conditions
D) generalizability

Question 42

Dr. López is the director of a long-term care facility. She reads about a laboratory study showing that a new treatment improved older adults' memory. Dr. López wonders whether the treatment would be effective for the residents in her facility. Which aspect of external validity is likely MOST important to her?

A) generalizing to
B) generalizing across
C) conceptual external validity
D) boundary conditions of the tested theory

Question 43

Dr. Abboud reads a study showing that young children with an attention disorder focus better if they can hold a stress ball during class. He wants to know whether this method also would be effective for adolescents and college students. Which aspect of external validity is MOST relevant to his question?

A) generalizing to
B) generalizing across
C) the conceptual component
D) the functional component

Question 44

__________ addresses whether the results of a study can be replicated under different conditions; _________ addresses whether the results can be applied to a particular setting.

A) Ecological validity; generalizability
B) Functional validity; ecological validity
C) Ecological validity; functional validity
D) Generalizability; ecological validity

Question 45

The _______ component of external validity is concerned with the method by which a study is carried out.

A) structural
B) functional
C) conceptual
D) ecological

Question 46

The _______ component of external validity is concerned with whether the psychological processes that operate in a study are similar to the psychological processes at work in a particular natural setting.

A) structural
B) functional
C) conceptual
D) methodological

Question 47

The _______ component of external validity is concerned with the degree to which the problems studied in research correspond to problems considered important in a natural setting.

A) structural
B) functional
C) conceptual
D) boundary setting

Question 48

A plant manager wants to know whether an intervention found to be effective in a laboratory study would improve the morale of the employees in her workplace. She is likely MOST interested in

A) setting boundary conditions for the intervention.
B) whether the study had good internal validity.
C) the ecological validity of the research results.
D) whether employee morale is a problem in other work settings.

Question 49

Which of the following factors is NOT related to the structural component of external validity?

A) research participants
B) culture
C) the research setting
D) psychological processes

Question 50

When thinking about the research setting as a factor that could influence the generalizability of research findings, one needs to think about whether

A) attributes of the person(s) collecting the data affect the participants.
B) the setting elicits reactivity.
C) the characteristics of some study participants will influence the responses of other individual participants.
D) all of the above.

Question 51

The functional component of external validity pertains primarily to the issue of

A) setting boundary conditions.
B) ecological validity.
C) theoretical validity.
D) the use of acute manipulations.

Question 52

Researcher Carey wants to establish the boundary conditions of a theory. She is interested in the _______component(s) of external validity.

A) structural
B) functional
C) conceptual
D) mundane

Question 53

The term ________ refers to creating a research setting that is similar to the natural setting in which the behavior being studied occurs.

A) experimental realism
B) mundane realism
C) ecological realism
D) functional realism

Question 54

Dr. Marchel's laboratory has a one-way mirror. He is not using the mirror to observe participants, but he still thinks that its presence will affect participants' responses. Dr. Marchel is concerned about the ________ component(s) of external validity.

A) structural
B) functional
C) conceptual
D) mundane

Question 55

Dr. Balewa conducts laboratory research with animals and wonders how well the conditions in her studies mimic those of the animals' natural habitats. Dr. Balewa is concerned with

A) experimental realism.
B) mundane realism.
C) ecological realism.
D) functional realism.

Question 56

Dr. Wern is conducting a simulation study in which college student participants make stock purchases based on characteristics of the companies. The participants appear to be reading the information carefully and spending time on their decisions. Dr. Wern's study is high on

A) experimental realism.
B) mundane realism.
C) ecological realism.
D) structural realism.

Question 57

Dr. Aronson has found a way to manipulate an independent variable so that the research participants are psychologically involved in the situation and give realistic responses. His study is high on

A) experimental realism.
B) mundane realism.
C) ecological realism.
D) structural realism.

Question 58

Which of the following statements about participant samples used in psychological research is TRUE?

A) Most psychological researchers use participant samples that are representative of their population of interest.
B) In general, people who volunteer to participate in psychological research are similar in their personal characteristics to non-volunteers.
C) Psychological research relies primarily on participants from North America and Western Europe.
D) If a study used college students as participants, the internal validity of that research is questionable.

Question 59

Which of the following statements about participant samples used in psychological research is FALSE?

A) One way to overcome the limitations of using college students as participants is to select those who have characteristics that are similar to people in the natural setting of interest.
B) In general, sampling procedures have little impact on the external validity of a study's findings.
C) Researchers should consider whether the participants' characteristics might interact with the independent variable to affect the results of a study.
D) The participant samples used in psychological research tend to be restricted in terms of age, race/ethnicity, and nationality.

Question 60

Which of the following statements about the role of research procedures in external validity is TRUE?

A) Having two few levels of an independent variable can sometimes jeopardize the external validity of the research.
B) The artificiality of laboratory research affects ecological validity but not generalizability.
C) It is possible to have multiple operational definitions of dependent variables, but not of independent variables.
D) One advantage of non-laboratory research is that such studies only rarely have any artificial aspects to them.

Question 61

Which of the following statements about the relationship between research procedures and ecological validity is/are TRUE?

A) The use of artificial research procedures might reduce an experiment's external validity, but can enhance internal validity through increased control.
B) Using college students as research participants has led researchers to use artificial, academic tasks that differ from those that most people perform in natural settings.
C) Dependent measures in laboratory research are often assessed in ways that restrict participants' behavior compared to the range of behaviors available in natural settings.
D) All of the above are true.

Question 62

Which of the following statements about the role of research procedures in external validity is/are TRUE?

A) Responses to similar stimuli can vary greatly from culture to culture.
B) Research findings can become obsolete over time, so researchers should conduct replication studies to assess whether the results are vulnerable to change.
C) Research designs should take into account the amount of time required for an independent variable to have an effect.
D) All of the above are true.

Question 63

Research has found that, compared to samples of participants based on college students, samples drawn from crowdsourcing sites such as Amazon's Mechanical Turk

A) overrepresent older participants.
B) underrepresent White people.
C) are more diverse.
D) all of the above

Question 64

Compared to non-volunteers, volunteers for research studies are more likely to be

A) conventional.
B) authoritarian.
C) from urban areas.
D) low in socioeconomic status.

Question 65

This approach to identifying the characteristics of nonvolunteers compares the responses of people who complete the study after the deadline to those who provide responses on time.

A) follow-up approach
B) wave approach
C) intentions approach
D) archival method

Question 66

This approach to identifying the characteristics of nonvolunteers surveys a small group of non-participants using a method that is more likely to generate a response.

A) follow-up approach
B) wave approach
C) intentions approach
D) archival method

Question 67

Which of the following is NOT a way to improve the external validity of research based on a college sample?

A) select college students who have experience in the natural setting of interest.
B) explain the experimental task so that participants perceive it similarly to people from the natural setting of interest
C) ensure that the participants have no prior training on the variable of interest
D) administer an interest profile and select participants who are similar to people in the natural setting

Question 68

Dr. Buss has a confederate insult or not insult a participant to induce aggression. He then measures how much the participant likes the confederate on a 7-point scale. Which criticism is MOST relevant to the external validity of this procedure?

A) The confederate is not representative of the population of people who insult others.
B) The liking measure does not represent how aggression is usually expressed outside of the laboratory.
C) The interaction took place over a short period of time.
D) The single-item liking measure has low reliability

Question 69

Which of the following is/are TRUE about the external validity of operational definitions?

A) Using multiple operational definitions of a construct lowers the external validity of one's findings.
B) Statistical procedures cannot be used to test the external validity of one's operational definitions.
C) Any one operational definition should be considered a sample drawn from a population of possible operational definitions.
D) All of the above are true

Question 70

Increasing confidence can be had in the generalizability of a research finding

A) as the number of successful replications increases.
B) when the same results are obtained using different operational definitions of variables.
C) when the same results are obtained in different research settings.
D) all of the above.

Question 71

An interaction effect in a factorial design indicates

A) that there is a problem with the structural component of external validity of the study.
B) the boundary conditions of the theory have been established.
C) a limitation on the generalizability of the study's findings.
D) a problem with the research population used in the study.

Question 72

Culture does not refer simply to race or nationality because

A) the people within these groups may not share the same values and beliefs.
B) responses to stimuli do not vary between members of these groups.
C) these variables are generally not assessed anonymously.
D) all of the above

Question 73

People from _________ cultures focus on _______.

A) individualistic; personal needs and goals.
B) collectivist; the self
C) individualistic; the needs of the group
D) collectivist; self-serving biases

Question 74

.Gergen (1973) classified the study of the sociocultural aspects of behavior as the study of history. His reasoning was that

A) there are cross-cultural differences in most behaviors.
B) cultural changes can render research findings obsolete.
C) conceptual replications fail more often than do exact replications.
D) most laws of behavior are immutable.

Question 75

Practitioners might take the position that research results do not apply because the psychological processes operating in a therapy setting differ from those that were operating in the lab. These practitioners are challenging the ________ aspect of the external validity of the research.

A) structural
B) functional
C) conceptual
D) natural

Question 76

When researchers want to apply the results of a study to a natural setting but people in the setting see the research question having little relevance to their situation, the ________aspect of the external validity of the research is being challenged.

A) structural
B) functional
C) conceptual
D) natural

Question 77

Which of the following statements about external validity is/are TRUE?

A) For research to be high on the conceptual component of external validity, it must be high on both the number of policy variables investigated and on the utility of the dependent variables.
B) Structural external validity guarantees functional and conceptual external validity.
C) If the results of a field study are the same as the results of a lab study, one can conclude that the same psychological processes were operating in both settings.
D) All of the above are true.

Question 78

Which of the following statements about external validity is TRUE?

A) External validity is best thought of as a characteristic of a set of findings or of a theory, not of an individual study.
B) The results of research conducted in field settings are, by their nature, more generalizable than the results of laboratory research.
C) It is not possible to empirically test the generalizability of a set of research findings.
D) The three aspects of external validity (structural, functional, and conceptual) are unrelated to one another.

Question 79

The external validity of research results can be enhanced by

A) including generalization factors, such as participant gender or race, in the study's design.
B) designing research projects to systematically replicate findings.
C) interpreting the research results in light of the limits its design places on generalizability.
D) doing all of the above.

Question 80

The concept of policy variable is most relevant to the ________ component of ecological validity.

A) structural
B) functional
C) conceptual
D) external