Deck 8: Conducting Experiments Outside the Laboratory

Question 1

Ultimately, field settings and laboratories are conceptually distinguished in terms of which of the following? (circle all that apply)

A) researchers' ability to infer causal relationships
B) the location in which an experiment is conducted
C) participants' awareness of their involvement in a study
D) the extent to which a study is basic or applied research

Answer 1

C

Question 2

Researcher A studied the effect of mood on volunteering by using college students and a paper-based mood-induction where participants write a 10 minute essay on a personal event where that made them extremely happy (sad). At a different university, Researcher B replicated this study with college students by having participants watch a pilot-tested happy (sad) movie. Researcher B's study would be considered a(n) __________ of Researcher A's study.

A) conceptual replication
B) exact replication
C) robust replication
D) methodological replication

Answer 2

A

Question 3

Researcher A studied the effect of mood on volunteering by using college students and a paper-based mood-induction where participants write a 10 minute essay on a personal event where that made them extremely happy (sad). In a subsequent study, Researcher B used the same procedures as Researcher A, but recruited middle-aged adults as participants. Researcher B's study would be considered a(n) __________ of Researcher A's study.

A) conceptual replication
B) exact replication
C) robust replication
D) methodological replication

Answer 3

B

Question 4

_____(a)_____ use different experimental contexts and different operational definitions of the same variables; _____(b)_____ use different experimental contexts and similar operational definitions.

A) field replications; laboratory replications
B) exact replications; conceptual replications
C) conceptual replications; exact replications
D) laboratory replications; field replications

Question 5

Which of the following statements are true of field studies, in relation to laboratory studies? (circle all that apply)

A) It is never possible to determine causal relationships in field studies because researchers cannot randomly assign participants to conditions.
B) The findings from field studies are more representative of the larger population than studies primarily using college students.
C) Field studies could be conducted in a laboratory environment.
D) Replication studies in the field function best when the operationalizations of IVs and DVs are adapted to the fit the idiosyncratic context of the field in which the study takes place.

Question 6

In comparison to laboratory experiments, field studies are often _____(a)_____ vulnerable to demand characteristics and _____(b)_____ vulnerable to social desirability biases.

A) more; less
B) less; more
C) more; more
D) less; less

Question 7

Which of the following examples would NOT be considered an unobtrusive measure? (circle all that apply)

A) measuring a person's galvanic skin response and pulse after seeing a scary movie (rather than have them fill out a survey)
B) using a pressure sensor underneath a clipboard to assess the amount of pressure respondents apply when completing a survey (as a measure of stress)
C) weighing the amount of trash in people's trash cans
D) counting the steps a person takes between two specified points

Question 8

In general, the following statements are true regarding online research studies: (circle all that apply)

A) the findings of online research studies mirror the findings of laboratory experiments of the same variables across many psychological constructs
B) online studies have less external validity than laboratory experiments
C) in online research, it is fairly easy to adapt almost any experimental manipulation designed for the laboratory into an online version of the same manipulation
D) the samples of online research studies are often more diverse than the traditional college student sample.

Question 9

In terms of the myths of online research and Internet samples, which of the following statements are more myth than reality? In other words, which of the following statements are untrue (or at least do not currently have much empirical support)?

A) all things being equal, online participants are less motivated than in-person participants
B) online samples are composed of less well-adjusted, introverted, shut-in types that reflect only a small portion of the greater population
C) online replications of laboratory experiments often have low correspondence with the original laboratory version of the study.
D) A and C
E) B and C
F) all of the above

Question 10

Replication is essential to establishing validity. To establish construct validity:

A) researchers must reproduce the operationalizations of a prior study
B) a different participant population must be used
C) researchers must demonstrate that the independent variable has a similar effect on the dependent variable in a different research setting
D) operationalizations of variables must be different from those used in previous studies
E) an exact replication is needed

Question 11

In a field experiment where a researcher is interested in observing individuals' giving behavior as they encounter a confederate on the sidewalk:

A) randomization would not be possible
B) participants who do not experience the treatment should not be measured on the dependent variable
C) the presence of the confederate is an independent variable
D) debriefing is not necessary
E) recording the amount of money donated would be an unobtrusive measure

Question 12

Indirect or unobtrusive measures:

A) are likely to be less reliable than more direct measures
B) are likely to be less valid than more direct measures
C) reduce the effect of social desirability on participant responses
D) cannot be combined with direct measures
E) all of the above

Question 13

Consider an experiment conducted over the Internet with a large, diverse sample, with results indicating that when people feel excluded or ostracized they tend to report lower self-esteem. The ostracism - self-esteem effect is probably:

A) an artifact
B) generalizable across participants and settings
C) a small effect
D) difficult to replicate in the real world

Question 14

Internet-based research:

A) yields samples that are comprised largely of educated, White males
B) results do not correspond with results obtained from standard research methods
C) must confront the issue of having unmotivated participants
D) is particularly useful for recruiting specific populations

Question 15

Using the same variable, recycling behavior, provide an example for an indirect measurement of recycling and an unobtrusive (but not indirect) measurement of recycling. In your response, be sure to explain how and why your examples are unobtrusive and indirect measures.
Indirect measure of recycling: two potential trays of paper could be made available to participants (e.g., for scratch paper). One tray is full of paper with obvious marks on the top of each sheet, but much of the paper is still usable (e.g., labeled something like "scratch paper"), and the paper is slightly less bright; while the other tray is filled with obviously fresh, untouched paper of a brighter and whiter hue. Choice of scratch paper to use would be an indirect measure of people's commitment to reducing their consumption and recycling material when possible.
Unobtrusive measure of recycling: offering people a free water bottle, and recording whether they put that bottle in the recycling or in the trash.

Question 16

Describe the distinction between indirect measurement and unobtrusive measurements. Are all indirect measurements unobtrusive? Are all unobtrusive measurements indirect? Why or why not?

Question 17

Your research advisor conducted a laboratory experiment regarding the effects of mood (happy, sad) on people's preference for abstract or representational paintings. Specifically, mood was manipulated by having college student participants watch a pilot-tested happy or sad scene from the same movie, and then asking participants to view images of relatively unknown abstract and representational paintings (counterbalanced) and rate which one they liked more. You want to replicate the findings of this study, but do not have access to a college student sample - so you are going to conduct the study online. Are the experimental procedures of the laboratory experiment easily adaptable to an online format - why or why not? What could you do differently with the mood manipulation to make it more conducive to an online format? You are worried about your online participants being unmotivated and losing interest halfway through the experimental treatment. How might you decrease the likelihood that participants will leave the experiment early? In your response: (a) describe some of the considerations that go into translating a laboratory experiment into an online format, and discuss whether the procedures in the example above are conducive to an online format; (b) how could you manipulate mood (happy, sad) in a more online-friendly way; and (c) describe at least two ways that you would be able to increase the likelihood that participants stay focused and complete the study.

Question 18

Your research advisor is interested in conducting a field experiment at local parks investigating the bystander effect and the distance between recyclable materials on the ground and the nearest recycling receptacle, on people's likelihood of recycling. However, she does not have any time to design the study. Instead, you are tasked with designing the study. Specifically, how would you operationalize the IVs and DV? How would you achieve random assignment in the context of this study? In your response, be sure to (a) identify and describe the operations for two IVs and one DV; (b) explain how participants would be randomly assigned to experimental conditions; and (c) explain how your design would be able to demonstrate the causal effects of the IVs on recycling, rather than have people's recycling behavior (i.e., the DV) vary based on their preexisting commitment to recycling.

Question 19

Discuss some of the advantages and disadvantages of field experiments. What are some advantages of conducting studies online?

Question 20

What are some considerations and practical issues to address when conducting research on Discussion questions for CBL Chapter 8?

Question 21

You are interested in studying the effects that gambling has on temporal discounting (i.e., devaluing rewards a person might get in the future relative to smaller rewards "right now"). Accordingly, you decide that there is no better way to research this relationship than with a field experiment in Las Vegas. However, you are still unsure how to operationalize your variables, and how to achieve some sort of random assignment to conditions. Specifically, how could you control as much of the experimental context as possible, given that Las Vegas is such a crowded and lively place? What could you use to your advantage? Is there a way you could assess temporal discounting using unobtrusive measures - if so, how; if not, why not? How could you demonstrate that the act of gambling caused a change in people's temporal discounting "ratio" (i.e., their weighting of future rewards relative to immediate rewards changes after gambling)? Should you just use laboratory-validated gambling simulations and temporal discounting measures in your field study - why or why not?

Question 22

Sometimes, the psychological phenomena we are trying to measure are already abstract constructs (e.g., cognitive dissonance) that we are trying to approximate with written responses or observed behaviors. Developing unobtrusive or indirect measures of already abstract constructs can therefore be even more difficult (although nonetheless important). In a laboratory setting, how could you use unobtrusive measures to assess the extent to which a person is currently experiencing cognitive dissonance? Would this laboratory-based, unobtrusive measure of dissonance be equally valid in field settings - why or why not? In representative field settings (i.e., contexts with high ecological validity), is there a way you might be able to indirectly or unobtrusively determine whether people were in a dissonant state?

Question 23

What are your thoughts on the ecological validity of Internet research? Specifically, given the de-individuation and self-presentation bias (e.g., via one's avatar) that can occur on the Internet, would you expect a sample of Internet participants' responses on attitude and behavioral intent measures to reflect the attitudes and intentions of a non-Internet sample? Would you expect Internet responses to differ based on where you recruited participants? For example, would you expect ecological validity to be greatest for a sample recruited from Second Life, Amazon's crowdsourcing website Mechanical Turk, or Facebook? Are all Internet sources created equal when it comes to recruiting participants - why or why not? If we consider the Internet a type of "field setting," is it possible to collect indirect or unobtrusive measures in an Internet sample - what would be an example?