Deck 11: Analyzing Experimental Data: Basic Concepts

Question 1

A difference between means that is statistically significant is

A)due to error variance.
B)likely to be a Type II error.
C)a null finding.
D)unlikely to be due to error variance.

Answer 1

D

Question 2

Power refers to the degree to which a study is likely to

A)discover large effects.
B)detect effects of the independent variable.
C)be replicated.
D)produce Type I errors.

Answer 2

B

Question 3

Type II errors are more likely when

A)a Type I error occurs.
B)the results are statistically significant.
C)power is low.
D)the effect size is large.

Answer 3

C

Question 4

Why is it possible for condition means to differ at the end of an experiment even if the independent variable had no effect?

A)A Type II error has occurred.
B)Error variance has created differences between conditions.
C)A null finding was obtained.
D)The power of the study was low.

Question 5

A power analysis is often used to determine

A)the number of participants needed for a study.
B)the likelihood of making a Type I error.
C)the strength of the independent variable.
D)whether to reject the null hypothesis.

Question 6

Failing to reject the null hypothesis when it is false is

A)confounding.
B)Type I error.
C)Type II error.
D)alpha.

Question 7

Researchers use null hypothesis significance testing to

A)understand why null findings were obtained.
B)test whether confounding has occurred.
C)test whether condition means differ more than they would if the difference were due to error variance.
D)separate the effects of the independent from the effects of participant variables.

Question 8

A Type I error can occur only when a researcher

A)accepts the null hypothesis.
B)rejects the null hypothesis.
C)fails to reject the null hypothesis.
D)rejects the experimental hypothesis.

Question 9

A Type I error occurs when a researcher

A)rejects a null hypothesis that is false.
B)fails to reject a null hypothesis that is false.
C)accepts a null hypothesis that is true.
D)rejects a null hypothesis that is true.

Question 10

The probability that a researcher is willing to make a Type I error is

A)the alpha-level.
B)beta.
C)power.
D)the confidence interval.

Question 11

An effect size of .40 indicates that

A)the size of the difference between two means is .40 of a standard deviation.
B)the odds of a response in one condition is .40 of the odds in another condition.
C)40% of the variability in the dependent variable is due to the independent variable.
D)the answer depends on which effect size indicator is being used.

Question 12

The null hypothesis states that

A)null findings will not be obtained.
B)there is no error variance in the data.
C)differences between conditions will be due only to the independent variable.
D)the independent variable did not have an effect.

Question 13

The proportion of the variance in one variable that can be accounted for by another variable is the

A)power.
B)statistical significance.
C)beta.
D)effect size.

Question 14

Why is p-hacking a questionable research practice?

A)It increases Type I errors.
B)It results in excessive null findings.
C)It requires discarding data.
D)It reduces the power of a study.

Question 15

Null hypothesis significance testing has been criticized because

A)no hypothesis is ever truly false.
B)the conclusion that a result is significant is based on an arbitrary criterion.
C)even when an effect is significant, Type I error is always possible.
D)the lack of significant results causes studies to fail.

Question 16

The power of an experiment was .87. This indicates that

A)there is an 87% chance of making a Type I error.
B)87% of the participants in one condition will score higher than the average participant in the other condition.
C)beta is .87.
D)the experiment has an 87% chance of detecting any effect of the independent variable that might occur.

Question 17

Researchers reject the null hypothesis when

A)statistical analyses showed that the p-value is large.
B)the difference between condition means is larger than would be expected on the basis of error variance alone.
C)the effect size is smaller than .05.
D)they want to avoid a Type I error.

Question 18

If statistical analyses show that the difference between condition means is larger than would be expected on the basis of error variance alone, a researcher will

A)accept the null hypothesis.
B)reject the null hypothesis.
C)fail to reject the null hypothesis.
D)reject the experimental hypothesis.

Question 19

Cohen's d expresses the size of an effect in terms of

A)correlations.
B)standard deviations.
C)amount of variance accounted for.
D)the odds of a response.

Question 20

The alpha-level that researchers generally use is designed to ensure Type I errors occur on less than _____ of statistical decisions.

A))5%.
B)1%.
C)5%.
D)50%.

Question 21

What information does a 95% confidence interval provide?

Question 22

Why is it insufficient for a researcher to simply inspect the condition means to determine whether the independent variable affected scores on the dependent variable?

Question 23

In deciding whether to reject the null hypothesis, researchers may make one of two errors. Explain.

Question 24

Based on statistical analyses, a researcher will make one of two decisions regarding the null hypothesis. What are they?

Question 25

Which of the following 95% confidence intervals for the difference between two means most likely indicates that the two means are truly different from one another in the population?

A)95% CI [-.05, .05]
B)95% CI [-1.2, 17.6]
C)95% CI [-7.2, -3.4]
D)95% CI [.00, 5.6]

Question 26

From looking at a 95% confidence interval for the difference between two means, how can we tell whether the two means are significantly different?

Question 27

The confidence interval for a mean tells us

A)how likely it is that a result can be replicated.
B)how confident we can be that a significant effect is not a Type I error.
C)the range of likely values for the population mean.
D)how many participants we need in order to be confident of the value of the mean we obtained.

Question 28

Why do we say that we "fail to reject" the null hypothesis rather than "accept" it?

Question 29

Distinguish among the three kinds of effect size indicators.

Question 30

Discuss three problems that are associated with null hypothesis significance testing.

Question 31

A 95% confidence interval (CI)

A)includes 95% of the sample means.
B)is more narrow than a 99% CI.
C)is the same as the 95% margin of error.
D)is symmetrical around the sample mean.

Question 32

What is a null hypothesis?

Question 33

What is power? Do researchers want power to be low or high, and why?