Deck 17: Data Collection

Question 1

Distinguish between a target population, a study population, and a research sample. Give an example of each.

Answer 1

No Answer

Question 2

Distinguish between probability sampling and nonprobability sampling. List one advantage and one disadvantage of each.

Answer 2

No Answer

Question 3

Briefly describe the process of selecting a research sample using probability sampling.

Answer 3

No Answer

Question 4

What is a quota matrix? Describe how it can be used in a stratified random sampling procedure. Then describe how it can be used in a nonprobability sampling procedure.

Question 5

Explain why many psychological research studies are based on convenience samples.

Question 6

Define these terms: cluster sampling, purposive sampling, snowball sampling.

Question 7

What is statistical power and why is it important to estimate power prior to conducting a research study?

Question 8

Explain the relationship between sample size and statistical power.

Question 9

Explain the role effect size plays in statistical power analysis. Describe three approaches a researcher can use to estimate the effect size for a power analysis.

Question 10

When determining sample size, explain why there is a tradeoff between ease of data collection and protection against making a Type II error.

Question 11

Describe three ways that inadequate statistical power can affect the validity of a research study.

Question 12

List three characteristics of an effective research setting.

Question 13

Distinguish between experimental and mundane realism. Give an example of each.

Question 14

List three characteristics of effective instructions for research participants.

Question 15

Why is it important to debug an experimental procedure prior to beginning a research study? Describe the steps of an effective debugging.

Question 16

What is the purpose of conducting a pilot study prior to beginning data collection?

Question 17

What is a post-experimental interview? Describe three types of information a researcher can gain from conducting such an interview.

Question 18

How is a post-experimental interview similar to and different from a debriefing?

Question 19

What is a manipulation check and what can a researcher learn from conducting such a check?

Question 20

Describe two situations in which a research assistant might be harmed while collecting data for an experiment. How can the potential for harm be reduced in these situations?

Question 21

List four guidelines for effectively supervising research assistants.

Question 22

Distinguish between the translational approach and the novel approach to data collection via the Internet.

Question 23

Compare the validity of online research with the validity of traditional laboratory research.

Question 24

List four advantages of online research compared to traditional "offline" research.

Question 25

Name four disadvantages of online research. How important or unimportant do you believe those disadvantages are? Explain your reasoning.

Question 26

What is crowdsourcing? What are the limitations of using crowdsourcing for data collection?

Question 27

Describe the factors that can increase attrition rates in online data collection.

Question 28

Describe a procedure you could use to solicit participation in a research study offered online. What are the strengths and weaknesses of your procedure?

Question 29

Distinguish between passive and active recruitment strategies for online research. Give an example of each.

Question 30

Describe three reasons why internal validity can be higher in online research compared to a traditional laboratory study.

Question 31

What types of incentives are most effective and which are least effective, according to research on this topic?

Question 32

Under what conditions would an online research study violate the ethical principle of voluntary participation?

Question 33

Under what conditions might an individual's privacy be violated during an online research study? What can you do to ensure that this does not happen?

Question 34

Describe a procedure for obtaining informed consent for an Internet research study.

Question 35

What steps can you take to monitor participants in an Internet research study for potential harm?

Question 36

Describe a procedure for conducting an effective debriefing for an online research study.

Question 37

Name and define three types of archival data. Give an example of each.

Question 38

Give an example of a source for archived survey data. What type of data are included? What would be an advantage and a disadvantage of using data from that source?

Question 39

Describe two advantages and two disadvantages of archival research.

Question 40

Describe the criteria for evaluating the quality of archival survey data.

Question 41

Distinguish between selective deposit and selective survival of archival data. How would these concepts affect the validity of archival data?

Question 42

Define the ecological fallacy. How might it lead a researcher to draw erroneous conclusions from archival data?

Question 43

_______ refers to the people to whom we want the results of our research to apply.

A) Target population
B) Study population
C) Research sample
D) Quota sample

Question 44

This term refers to the people whose characteristics meet our operational definition of the hypothetical construct of the people in the identified group of interest.

A) target population
B) study population
C) research sample
D) quota sample

Question 45

This term refers to the people who actually participate in the research and from whom we collect data.

A) target population
B) study population
C) research sample
D) quota sample

Question 46

The term random sampling refers to using a random process to

A) assign research participants to conditions of an independent variable.
B) determine the order of presentation of conditions of a within-subjects experiment.
C) select research participants from a population.
D) determine the appropriate number of participants to include in a study.

Question 47

In ________ sampling, every member of the study population has a known chance of being selected for the research sample.

A) convenience
B) probability
C) quota
D) purposive

Question 48

Researchers use ________ sampling to ensure that a random sample of a population reflects the proportion of people within the population who have key characteristics (e.g., gender, age).

A) stratified random
B) systematic
C) convenience
D) purposive

Question 49

In ________ sampling, researchers identify groups of people who meet the definition of the study population and then draw a random sample of those groups, using all members of each selected group as research participants.

A) stratified random
B) convenience
C) cluster
D) purposive

Question 50

In ________ sampling, researchers start with a list of all the people in a study population and select every nth name for the sample, where n equals the proportion of people on the list to be sampled.

A) systematic
B) convenience
C) stratified random
D) purposive

Question 51

In ________ sampling, researchers use their judgment to select members of their sample based on the goals of the research.

A) systematic
B) convenience
C) stratified random
D) purposive

Question 52

In ________ sampling, people who are initially selected for a sample nominate others whom they think might be willing to participate in the research.

A) systematic
B) convenience
C) purposive
D) snowball

Question 53

_______sampling offers the ability to proportionately represent small subgroups of the study population.

A) Stratified random
B) Convenience
C) Simple random
D) Snowball

Question 54

Researchers administer a reliable and valid self-esteem inventory to college students. They use a pre-set cut-off score to operationally define low self-esteem and recruit students who score below the cut-off. The researchers are using ________ sampling.

A) simple random
B) convenience
C) systematic
D) purposive

Question 55

A researcher using convenience sampling structures the sample so that it consists of 50% women and 50% men. The research is using ________ sampling.

A) stratified random
B) quota
C) purposive
D) power

Question 56

Which of the following is NOT a form of probability sampling?

A) stratified random sampling
B) cluster sampling
C) systematic sampling
D) quota sampling

Question 57

When determining the appropriate sample size for a study, one should consider, among other factors, the

A) size of the effect one is trying to detect in the research.
B) way the data are collected (survey, interview, etc.)
C) research population (college students, older adults, etc.)
D) all of the above.

Question 58

Statistical power is the probability of

A) avoiding a Type I error.
B) making a Type II error
C) avoiding a Type II error
D) making a Type I error.

Question 59

If all other factors remain the same, as sample size increases the statistical power of a study

A) decreases.
B) increases.
C) stays the same.
D) depends on whether an ANOVA or a t-test is conducted.

Question 60

To determine the appropriate sample size for a study, one needs to know

A) the minimum effect size one is interested in detecting.
B) the alpha level to be used for the statistical tests.
C) whether one will be using a one-tailed or two-tailed statistical test.
D) all of the above.

Question 61

Which of the following statements about sample size is TRUE?

A) All else being equal, larger sample sizes provide more statistical power.
B) The smaller the sample size, the more likely one is to be able to detect a small effect of the independent variable.
C) Two-tailed tests require smaller sample sizes than one-tailed tests.
D) Studies that set lower alpha levels require smaller sample sizes than studies using higher alpha levels.

Question 62

A large sample offers greater protection against making a Type II error at the cost of

A) being limited to using a two-tailed test.
B) time or money spent on data collection.
C) a greater chance of overlooking subtle effects.
D) overestimating the critical effect size.

Question 63

Which of the following can result from conducting research with inadequate statistical power?

A) small effects can be overlooked
B) the true magnitude of an effect can be overestimated
C) others might not be able to replicate the findings
D) all of the above

Question 64

When participants experience a smooth flow of events while data are being collected, the research setting is referred to as being

A) simple.
B) psychologically involving.
C) coherent.
D) consistent.

Question 65

A research setting which avoids unnecessary complexities is referred to as being

A) simple.
B) psychologically involving.
C) coherent.
D) consistent.

Question 66

A research setting that arouses and engages participants' interest is referred to as being

A) simple.
B) psychologically involving.
C) coherent.
D) consistent.

Question 67

A research setting has ________ realism if the events occurring in the research setting are likely to occur in some natural setting.

A) consistent
B) experimental
C) coherent
D) mundane

Question 68

In a _________research setting, all participants have the same experience except for the condition of the independent variable they are in.

A) simple
B) psychologically involving
C) coherent
D) consistent

Question 69

The concept of experimental realism is most closely related to the ________ component of external validity.

A) analogical
B) conceptual
C) functional
D) structural

Question 70

The research setting characteristic of ________ contributes to the reliability of the manipulation of the independent variable.

A) simplicity
B) consistency
C) being psychologically involving
D) coherence

Question 71

Although experimental realism is desirable, it has the disadvantage(s) of

A) increasing the complexity of the research procedures.
B) potentially distracting participants' attention from the manipulation of the independent variable.
C) leading to inconsistent experiences across research participants if experimenters or confederates make errors in the research procedure.
D) all of the above.

Question 72

Professor Diamond wants to be sure the participants understand the experimental instructions. She should write instructions that

A) are as simple as possible.
B) repeat key information
C) include examples that illustrate key points.
D) do all of the above.

Question 73

Research participants' compliance with experimental instructions can be enhanced by

A) using procedures high in mundane realism.
B) presenting oral rather than written instructions.
C) checking that participants understand the instructions.
D) doing any of the above.

Question 74

A ________ study is a preliminary piece of research conducted with a sample of research participants drawn from the study population.

A) prospective
B) pilot
C) test
D) trial

Question 75

Dr. Koontz is planning a rehearsal of his experimental procedures. He should ask his experimenters to

A) focus mainly on the difficult steps of the procedure.
B) bring in actual research participants.
C) memorize the script in advance.
D) conduct the rehearsal in the room that will be used for the actual study.

Question 76

Which of the following is NOT part of debugging a set of experimental procedures?

A) having colleagues critique the proposed research
B) analyzing collected data to identify problems
C) rehearsing the research procedures
D) conducting pilot studies

Question 77

________ is the process of looking for problems with research procedures in advance of actual data collection.

A) Trial ballooning
B) Debugging
C) Anticipatory review
D) Pre-experimental interviewing

Question 78

Which of the following is NOT a function of pilot studies?

A) modifying the experimental manipulation until statistically significant results are found
B) determining whether conducting a full-scale study is worth the time and effort
C) testing the validity of experimental manipulations
D) determining whether participants are interpreting the research situation as intended

Question 79

Which of the following is NOT one of the functions of the post-experimental interview?

A) meeting certain ethical obligations, such as explaining any deceptions that were used
B) helping participants understand the purposes of the research
C) obtaining participants' informed consent
D) conducting manipulation checks

Question 80

Information collected during the post-experimental interview can be used to

A) identify problems with the experimental procedure.
B) provide a new perspective on the hypotheses being tested.
C) assess the possible effects of demand characteristics on participants.
D) do all of the above.