Deck 11: More on Experiments: Confounding and Obscuring Variables

Question 1

Which of the following is NOT a method researchers used to identify or correct for attrition?

A) Drop the original/pretest scores of participants who left the study
B) Not allow participants to leave a study once it has started
C) Determine whether those who dropped out of the study had a different pattern of scores than those who stayed in the study
D) Use a control group

Answer 1

Not allow participants to leave a study once it has started

Question 2

Dr.Hoff is curious as to whether children in a kindergarten classroom will be friendlier after talking about friendship.Immediately before and after the discussion, she has several undergraduate research assistants code the "friendliness" of a group of kindergarteners.Which of the following threats should Dr.Hoff be most concerned about?

A) History
B) Maturation
C) Instrumentation
D) Testing

Answer 2

Instrumentation

Question 3

Dr.LaGuardia is curious as to whether children in a daycare center will share more after watching an educational video on sharing.Immediately before and after the video, he has several undergraduate research assistants code the sharing behavior of a group of 33 four-year-olds.Which of the following would you recommend to him to decrease the threat of instrumentation?

A) Asking the same research assistants to code the same children at pretest and posttest
B) Establishing different coding manuals for pretest and posttest
C) Using clear coding manuals
D) Using only one research assistant to code all the videos

Answer 3

Using clear coding manuals

Question 4

Which of the following can help prevent testing effects?

A) Using a comparison group
B) Establishing reliability of the measure
C) Using a clear coding manual
D) Employing a pretest-only design

Question 5

Observer bias can threaten which of the following big validities?

A) Internal validity and construct validity
B) External validity and construct validity
C) Internal validity and external validity
D) Internal validity only

Question 6

RESEARCH STUDY 11.1: In previous studies, Dr.Schulenberg has established that finding meaning in one's everyday work activities can lead to greater success in the workplace (e.g., productivity, creativity).He is curious as to whether this can happen in the college classroom.Specifically, he is curious whether finding meaning in one's classroom experience can lead to greater academic performance.In the spring semester, he has his teaching assistant randomly assign half the class to write a paragraph each class period about how the material has meaning for their lives (meaning group).The other half writes a paragraph about what they did to prepare for class (preparation group).He does not know which of his students are writing which paragraph, and the students are not aware they are responding to different writing assignments.To measure academic performance, he gives the students a midterm essay exam and a final exam. Dr.Schulenberg likely designed his study so that neither he nor his students knew which group they were in to address which of the following?

A) Placebo effects
B) Attrition
C) Demand characteristics
D) Maturation

Question 7

RESEARCH STUDY 11.1: In previous studies, Dr.Schulenberg has established that finding meaning in one's everyday work activities can lead to greater success in the workplace (e.g., productivity, creativity).He is curious as to whether this can happen in the college classroom.Specifically, he is curious whether finding meaning in one's classroom experience can lead to greater academic performance.In the spring semester, he has his teaching assistant randomly assign half the class to write a paragraph each class period about how the material has meaning for their lives (meaning group).The other half writes a paragraph about what they did to prepare for class (preparation group).He does not know which of his students are writing which paragraph, and the students are not aware they are responding to different writing assignments.To measure academic performance, he gives the students a midterm essay exam and a final exam. Which of the following aspects of Dr.Schulenberg's study allows him to prevent observer bias?

A) Keeping his students unaware of which type of essay they are writing
B) Having his teaching assistant assign students randomly to the two groups
C) Grading the exams himself (a Ph.D.) instead of having his teaching assistant (a college senior) do it
D) Ensuring the person grading the exams is unaware of each student's writing group

Question 8

RESEARCH STUDY 11.1: In previous studies, Dr.Schulenberg has established that finding meaning in one's everyday work activities can lead to greater success in the workplace (e.g., productivity, creativity).He is curious as to whether this can happen in the college classroom.Specifically, he is curious whether finding meaning in one's classroom experience can lead to greater academic performance.In the spring semester, he has his teaching assistant randomly assign half the class to write a paragraph each class period about how the material has meaning for their lives (meaning group).The other half writes a paragraph about what they did to prepare for class (preparation group).He does not know which of his students are writing which paragraph, and the students are not aware they are responding to different writing assignments.To measure academic performance, he gives the students a midterm essay exam and a final exam. The study described above is an example of which of the following?

A) A one-group, pretest/posttest design
B) A null effect
C) A double-blind study
D) Observer bias

Question 9

The addition of a comparison group can address which of the following threats to internal validity?

A) Instrumentation
B) Maturation
C) Selection-history
D) Attrition

Question 10

When interrogating experiments, on which of the big validities should a person focus?

A) Internal validity
B) External validity
C) Construct validity
D) Statistical validity

Question 11

Which of the following cannot be found in a one-group, pretest/posttest design?

A) A random sample
B) A valid measure
C) A comparison group
D) A manipulation

Question 12

Which of the following is true of instrumentation threats?

A) They are the same as testing threats.
B) They occur only when using mechanical instruments (e.g., blood pressure cuffs, scales).
C) They can be avoided with counterbalancing.
D) They are only problematic in observational research.

Question 13

Regression is especially problematic in which of the following situations?

A) When the experimental and comparison groups are equal at posttest
B) When one group has an extremely high score at pretest
C) When the experimental and comparison groups are equal at pretest
D) When one group has an extremely low score at posttest

Question 14

Spontaneous remission in clinical studies is an example of which of the following threats to internal validity?

A) Regression
B) Attrition
C) Maturation
D) Placebo effects

Question 15

Dr.Whetstone is curious about how self-esteem changes as a result of a new counseling program.She is concerned about testing threats in her study.Which of the following would you recommend to her as a way to address this type of threat?

A) Refusing to let participants drop out of the study
B) Conducting a pretest-only study
C) Using a different measure of self-esteem at pretest than at posttest
D) Omitting a comparison group

Question 16

RESEARCH STUDY 11.1: In previous studies, Dr.Schulenberg has established that finding meaning in one's everyday work activities can lead to greater success in the workplace (e.g., productivity, creativity).He is curious as to whether this can happen in the college classroom.Specifically, he is curious whether finding meaning in one's classroom experience can lead to greater academic performance.In the spring semester, he has his teaching assistant randomly assign half the class to write a paragraph each class period about how the material has meaning for their lives (meaning group).The other half writes a paragraph about what they did to prepare for class (preparation group).He does not know which of his students are writing which paragraph, and the students are not aware they are responding to different writing assignments.To measure academic performance, he gives the students a midterm essay exam and a final exam. Imagine that in Dr.Schulenberg's study, he notes that all of the students do extremely well on the midterm exam.When he looks at the results of the final exam, he notices that all the students' exam scores went down.Which of the following pattern of results would suggest that there is a threat to internal validity?

A) The final exam scores were lower in the preparation group than the meaning group.
B) The final exam scores were equally low in both groups.
C) The final exam scores were lower than the national average.
D) The final exam scores were lower than the final exam scores of students of his colleague, Dr. Fao.

Question 17

Which of the following threats to internal validity can apply even when a control group is used?

A) Attrition
B) History threats
C) Instrumentation threats
D) Demand characteristics

Question 18

RESEARCH STUDY 11.1: In previous studies, Dr.Schulenberg has established that finding meaning in one's everyday work activities can lead to greater success in the workplace (e.g., productivity, creativity).He is curious as to whether this can happen in the college classroom.Specifically, he is curious whether finding meaning in one's classroom experience can lead to greater academic performance.In the spring semester, he has his teaching assistant randomly assign half the class to write a paragraph each class period about how the material has meaning for their lives (meaning group).The other half writes a paragraph about what they did to prepare for class (preparation group).He does not know which of his students are writing which paragraph, and the students are not aware they are responding to different writing assignments.To measure academic performance, he gives the students a midterm essay exam and a final exam. Imagine that in Dr.Schulenberg's study, he notes that all of the students do extremely well on the midterm exam.When he looks at the results of the final exam, he notices that all the students' exam scores went down.Given this information, which of the following threats might be present in his study?

A) Regression
B) Attrition
C) Maturation
D) Observer bias

Question 19

To be a history threat, the external event must occur:

A) constantly during the experiment.
B) at the beginning of the experiment.
C) systematically, affecting most members of the group.
D) intentionally, induced by the experimenters.

Question 20

Which of the following studies would NOT have a possible threat of observer bias?

A) A study looking at the effect of eating fruits and vegetables and general physical fitness
B) A study looking at the effect of tutoring on SAT scores
C) A study looking at whether increasing the number of hours listening to music increases singing ability
D) A study looking at the effect of increasing the rate of text messaging in a day on writing/English composition skill

Question 21

In what way does high within-groups variance obscure between-groups variance?

A) It leads to null results.
B) It leads to larger effect sizes.
C) It causes more overlap between experimental/comparison groups.
D) It limits the type of statistical analyses that can be conducted.

Question 22

Which of the following things can be done to reduce measurement error?

A) Using less precise measurements
B) Using more reliable measurements
C) Using a pretest/posttest design
D) Increasing unsystematic variance

Question 23

RESEARCH STUDY 11.2: Dr.Bloedorn is a health psychologist who researches nutrition.She is curious as to whether a new drink additive will help people consume fewer calories during a meal.The drink additive is a white, odorless, tasteless powder that a person can add to any drink.She collects a random sample of 63 overweight students on campus and measures the calories they eat during lunch using a bomb calorimeter.She then gives this additive to the same 63 participants to use at dinner and measures how many calories they eat (again using the bomb calorimeter). The addition of a control group that does not use the drink additive would help Dr.Bloedorn address which of the following threats to internal validity?

A) History
B) Demand characteristics
C) Instrumentation
D) Placebo effects

Question 24

RESEARCH STUDY 11.2: Dr.Bloedorn is a health psychologist who researches nutrition.She is curious as to whether a new drink additive will help people consume fewer calories during a meal.The drink additive is a white, odorless, tasteless powder that a person can add to any drink.She collects a random sample of 63 overweight students on campus and measures the calories they eat during lunch using a bomb calorimeter.She then gives this additive to the same 63 participants to use at dinner and measures how many calories they eat (again using the bomb calorimeter). The addition of a group that does not use the drink additive but adds a similar-looking substance that they think is the additive would help Dr.Bloedorn address which of the following threats to internal validity?

A) History
B) Observer bias
C) Instrumentation
D) Placebo effects

Question 25

RESEARCH STUDY 11.2: Dr.Bloedorn is a health psychologist who researches nutrition.She is curious as to whether a new drink additive will help people consume fewer calories during a meal.The drink additive is a white, odorless, tasteless powder that a person can add to any drink.She collects a random sample of 63 overweight students on campus and measures the calories they eat during lunch using a bomb calorimeter.She then gives this additive to the same 63 participants to use at dinner and measures how many calories they eat (again using the bomb calorimeter). Imagine that Dr.Bloedorn finds no difference between the calories consumed with the drink additive and without.This is known as:

A) observer bias.
B) a maturation effect.
C) a null effect.
D) the placebo effect.

Question 26

A participant's score on a dependent variable is a combination of which of the following?

A) Measurement error and their true score
B) Systematic variance and error variance
C) Manipulation effect and observer bias
D) Their mean score and their standard deviation

Question 27

A researcher's attempt to control sounds, smells, and even temperature in a testing environment is meant to reduce which of the following?

A) Between-group variance
B) Situation noise
C) Weak manipulations
D) Demand characteristics

Question 28

Which of the following is true of ceiling and floor effects?

A) They cannot be detected by manipulation checks.
B) They are only problematic in pretest/posttest designs.
C) They can be caused by poorly designed dependent variables.
D) They don't affect the results of a study.

Question 29

Which of the following is NOT a reason that a study might yield a null result?

A) A lack of between-group difference
B) Too much within-group variance
C) A "true" null result
D) Use of a within-subjects design

Question 30

Why is there a publication bias against null effects?

A) Null effects are not real.
B) Null effects are the result of badly designed studies.
C) People generally want to read about independent variables that matter.
D) Null results happen less often than significant results.

Question 31

RESEARCH STUDY 11.2: Dr.Bloedorn is a health psychologist who researches nutrition.She is curious as to whether a new drink additive will help people consume fewer calories during a meal.The drink additive is a white, odorless, tasteless powder that a person can add to any drink.She collects a random sample of 63 overweight students on campus and measures the calories they eat during lunch using a bomb calorimeter.She then gives this additive to the same 63 participants to use at dinner and measures how many calories they eat (again using the bomb calorimeter). Which of the following threats to internal validity will Dr.Bloedorn NOT be worried about?

A) Testing effect
B) Attrition
C) Demand characteristics
D) Placebo effects

Question 32

Testing threats involve , whereas instrumentation threats involve .

A) researchers; participants
B) participants; measurements
C) independent variables; dependent variables
D) within-groups; between groups

Question 33

Which of the following things can be done to reduce the effect of individual differences?

A) Using a matched-groups design
B) Decrease power
C) Collecting measurements from diverse groups of people
D) Using a between-groups design

Question 34

When a double-blind study is not possible, an acceptable alternative may be .

A) a matched groups design
B) a within-groups design
C) the use of a control group
D) a masked design

Question 35

Dr.Deveraux has conducted a study that has resulted in a null effect.Nonetheless, she suspects that there truly is a causal relationship between her independent and dependent variables.Which of the following is UNLIKELY to be to blame?

A) A weak manipulation
B) An insensitive measure
C) Too many participants
D) A reverse confound

Question 36

A confound that keeps a researcher from finding a relationship between two variables is known as a(n) confound.

A) weak
B) insensitive
C) null
D) reverse

Question 37

Dr.Morimoto is curious as to whether exposing people to violent video games causes them to be more aggressive.He assigns half his participants to play a video game for 5 minutes and the other half to play for 7 minutes.He finds that there is no relationship between playing the game longer and being more aggressive.What might be to blame for this null effect?

A) A weak manipulation
B) An insensitive measure
C) A reverse confound
D) Ceiling effect

Question 38

Which of the following is a reason why a researcher might choose to conduct a double-blind placebo control group study?

A) To eliminate testing effects
B) To control for observer bias
C) To prevent attrition
D) To control for regression to the mean

Question 39

Unsystematic variability in a study is also known as:

A) error variance.
B) null effect.
C) group inconsistency.
D) situational variability.

Question 40

Dr.Sanderson is curious as to whether exposing people to violent video games causes them to be more aggressive.She assigns half her participants to play a violent video game for 5 minutes and the other half to play the same game for 25 minutes.Afterward, she has them play a board game and has a well-trained coder determine whether they are very aggressive in their playing style, barely aggressive, or not at all aggressive.She finds that a vast majority of her participants, regardless of group assignment, are rated as very aggressive.This outcome would be known as a(n):

A) weak manipulation.
B) ceiling effect.
C) floor effect.
D) outlier.

Question 41

Explain what a manipulation check is and how it can be used to address issues of weak manipulations and insensitive measures.

Question 42

RESEARCH STUDY 11.3: Dr.Paddock is a counseling psychologist who is interested in decreasing adjustment issues in first-year college students.She is curious if having students create collages of their first few weeks of school and then mail them home will help students feel they have integrated their new life with their old and, as a result, will help them feel less homesick.She samples a group of 100 incoming college freshmen at her university and measures how homesick they are during the first week of school.During week 4 of school, she has them make the collage and send it home.During week 7 of school, she measures their homesickness again.She notices a significant reduction in the amount of homesickness from the pretest to the posttest and concludes that her treatment is effective.
Name two threats to internal validity that are likely to be present in Dr.Paddock's study, given her particular design.What other explanation do these threats provide for the results found by Dr.Paddock?

Question 43

Ceiling effects can affect:

A) certain groups more than others.
B) independent variables only.
C) dependent variables only.
D) both independent and dependent variables.

Question 44

Dr.Anderson conducted a study investigating whether exercising immediately before an exam boosted cognitive abilities.She randomly assigned participants to either do jumping jacks or count to 60 before giving them a geometry test.She did not find any differences between groups but believes there may have been a ceiling effect.What might be a clue that this was the case?

A) All participants' scores are centered around 100% correct.
B) Participants in the counting group showed more variability in scores.
C) Participants in the exercise group took longer to complete the test.
D) Some participants verbally noted that the test was too easy.

Question 45

What can researchers do to reduce the risk of measurement error?

A) Select measures that have high reliability and validity
B) Make conditions comparable in each experimental group
C) Use large sample sizes
D) Use a strong manipulation

Question 46

After finding a null result in her study, a researcher decides to conduct a follow-up study where she limits who the findings will generalize to.Which cause of within-group variance is she trying to reduce?

A) Measurement error
B) Individual differences
C) Situation noise
D) Lack of power

Question 47

To avoid insensitive measures, it is important to use measures that include:

A) large differences between the levels of the variable.
B) measurement error.
C) ceiling effects.
D) multiple increments of the variable.

Question 48

Name the three threats to internal validity that can occur in studies regardless of the use of a comparison group.Why doesn't a comparison group help in these cases?

Question 49

Ceiling effects can lead to:

A) large variance within groups.
B) large variance between groups.
C) small variance between groups.
D) all participants performing poorly.

Question 50

Dr.Gong conducts a study where she randomly assigns participants to different experimental conditions.The testing for each condition occurs in a different room of the psychology building.After collecting her data, she learns that the air conditioning in one of the rooms had been turned off during data collection.Which of the following threats to internal validity should Dr.Gong be concerned with?

A) Design confound
B) Selection effect
C) Order effect
D) Maturation effect

Question 51

Using the same setting and consistent protocols for each participant in a study will reduce the effect of:

A) measurement error.
B) individual differences.
C) situation noise.
D) floor effects.

Question 52

RESEARCH STUDY 11.3: Dr.Paddock is a counseling psychologist who is interested in decreasing adjustment issues in first-year college students.She is curious if having students create collages of their first few weeks of school and then mail them home will help students feel they have integrated their new life with their old and, as a result, will help them feel less homesick.She samples a group of 100 incoming college freshmen at her university and measures how homesick they are during the first week of school.During week 4 of school, she has them make the collage and send it home.During week 7 of school, she measures their homesickness again.She notices a significant reduction in the amount of homesickness from the pretest to the posttest and concludes that her treatment is effective.
What is a comparison group? What kind of comparison group could Dr.Paddock add to her study to address several threats to internal validity? (In other words, what would this comparison group do/not do?) Explain why this addition would help address threats to internal validity.

Question 53

Dr.Merrick designs a study comparing the effects of mindfulness training to cognitive training on flexible thinking.In designing this study, what would be an appropriate manipulation check?

A) Testing that the flexible thinking task is hard enough to show group differences
B) Measuring heart rate to ensure that mindfulness has a calming effect
C) Ensuring that no participants had prior experience with mindfulness training
D) Adding a third group that receives flexibility training

Question 54

Which of the following threats to internal validity would result in group differences prior to the start of the study?

A) Design confound
B) Selection effect
C) Order effect
D) Maturation effect

Question 55

RESEARCH STUDY 11.3: Dr.Paddock is a counseling psychologist who is interested in decreasing adjustment issues in first-year college students.She is curious if having students create collages of their first few weeks of school and then mail them home will help students feel they have integrated their new life with their old and, as a result, will help them feel less homesick.She samples a group of 100 incoming college freshmen at her university and measures how homesick they are during the first week of school.During week 4 of school, she has them make the collage and send it home.During week 7 of school, she measures their homesickness again.She notices a significant reduction in the amount of homesickness from the pretest to the posttest and concludes that her treatment is effective.
Imagine in Dr.Paddock's study that the pretest scores were incredibly high, indicating a large amount of homesickness in her sample.What kind of threat to internal validity does this pose? How does this affect her conclusion that her treatment for homesickness worked?

Question 56

RESEARCH STUDY 11.3: Dr.Paddock is a counseling psychologist who is interested in decreasing adjustment issues in first-year college students.She is curious if having students create collages of their first few weeks of school and then mail them home will help students feel they have integrated their new life with their old and, as a result, will help them feel less homesick.She samples a group of 100 incoming college freshmen at her university and measures how homesick they are during the first week of school.During week 4 of school, she has them make the collage and send it home.During week 7 of school, she measures their homesickness again.She notices a significant reduction in the amount of homesickness from the pretest to the posttest and concludes that her treatment is effective.
Imagine in Dr.Paddock's study that only 90 of the original participants completed the measure of homesickness during week 7 (10 participants had left the university and were unavailable).Name two things that Dr.Paddock can do to address this possible threat to internal validity and why these should be done.

Question 57

RESEARCH STUDY 11.4: Armand conducts a study for his research method class.He is curious as to whether watching romantic movies makes people more committed to their romantic relationship.He collects a sample of men in dating relationships and divides them into two groups.One group watches a 5-minute clip of a movie in which the main characters are having a romantic first date.The second group watches a 5-minute clip from the same movie in which the main characters break up.After the participants watch the movie clip, they are then asked to write a sentence about their relationship.Armand counts the number of uses of the words we and us as a measure of commitment.After conducting the study, he finds that there is not a statistically significant difference between his two groups.
Provide three reasons why Armand's study may have resulted in null effects.

Question 58

RESEARCH STUDY 11.4: Armand conducts a study for his research method class.He is curious as to whether watching romantic movies makes people more committed to their romantic relationship.He collects a sample of men in dating relationships and divides them into two groups.One group watches a 5-minute clip of a movie in which the main characters are having a romantic first date.The second group watches a 5-minute clip from the same movie in which the main characters break up.After the participants watch the movie clip, they are then asked to write a sentence about their relationship.Armand counts the number of uses of the words we and us as a measure of commitment.After conducting the study, he finds that there is not a statistically significant difference between his two groups.
Armand's professor suggests that the null effect may be due to insufficient between-group variance.Describe two ways that Armand's study may have had poor between-group variance.

Question 59

RESEARCH STUDY 11.3: Dr.Paddock is a counseling psychologist who is interested in decreasing adjustment issues in first-year college students.She is curious if having students create collages of their first few weeks of school and then mail them home will help students feel they have integrated their new life with their old and, as a result, will help them feel less homesick.She samples a group of 100 incoming college freshmen at her university and measures how homesick they are during the first week of school.During week 4 of school, she has them make the collage and send it home.During week 7 of school, she measures their homesickness again.She notices a significant reduction in the amount of homesickness from the pretest to the posttest and concludes that her treatment is effective.
Imagine in Dr.Paddock's study that only 90 of the original participants completed the measure of homesickness during week 7 (10 participants had left the university and were unavailable).What kind of threat to internal validity does this pose? How does this affect her conclusion that her treatment for homesickness worked?

Question 60

RESEARCH STUDY 11.4: Armand conducts a study for his research method class.He is curious as to whether watching romantic movies makes people more committed to their romantic relationship.He collects a sample of men in dating relationships and divides them into two groups.One group watches a 5-minute clip of a movie in which the main characters are having a romantic first date.The second group watches a 5-minute clip from the same movie in which the main characters break up.After the participants watch the movie clip, they are then asked to write a sentence about their relationship.Armand counts the number of uses of the words we and us as a measure of commitment.After conducting the study, he finds that there is not a statistically significant difference between his two groups.
Armand's professor suggests that the null effect may be due to a large amount of within-group variance.State three causes of within-group variance in Armand's study.

Question 61

RESEARCH STUDY 11.4: Armand conducts a study for his research method class.He is curious as to whether watching romantic movies makes people more committed to their romantic relationship.He collects a sample of men in dating relationships and divides them into two groups.One group watches a 5-minute clip of a movie in which the main characters are having a romantic first date.The second group watches a 5-minute clip from the same movie in which the main characters break up.After the participants watch the movie clip, they are then asked to write a sentence about their relationship.Armand counts the number of uses of the words we and us as a measure of commitment.After conducting the study, he finds that there is not a statistically significant difference between his two groups.
Armand's professor proposes that he replicate the study and double the number of participants he recruits.State which cause(s) of within-group variability will be helped by adding more participants and why this will help.

Question 62

Explain why null effects are important in science but are hard to find in popular media.

Question 63

Explain how within-group variance can obscure between-group differences.

Question 64

Name two study designs that would address issues of individual differences contributing to within-group variance.If these two designs are not possible, what else could a researcher do?