Deck 12: Understanding P Values and Effect Size Indicators

Question 1

Based on her results, Kathy is unable to reject the null hypothesis, but she does feel that the effect size looks promising. What should Kathy do next?

A) recalculate the effect size
B) use a less stringent alpha level
C) do a statistical power analysis
D) reevaluate her hypothesis

Answer 1

C

Question 2

Mistakenly rejecting the null hypothesis when it is true and should not have been rejected is technically called a(n)

A) Type I error.
B) Type II error.
C) alpha error.
D) beta error.

Answer 2

A

Question 3

The p value is not sufficient for indicating the importance of the outcome of a study. The reason for this is because

A) p values are too small.
B) the size of the study affects the p value.
C) the magnitude of the test statistic must also be known.
D) effect sizes are related to p values.

Answer 3

B

Question 4

Accepting the null hypothesis when it should have been rejected is an example of

A) a Type I error.
B) alpha.
C) statistical power.
D) a Type II error.

Question 5

The strength of the relationship between X and Y may be operationally defined as

A) the p value.
B) power of the study.
C) the effect size.
D) alpha.

Question 6

Which of the following examples for reporting p values reflects the recommendations of many statisticians?

A) p ≤ .05
B) p < .05
C) .01 < p < .05
D) p = .032

Question 7

Which of the following is NOT a name for the probability of making a Type I error?

A) alpha
B) significance level
C) beta
D) p value

Question 8

The counternull statistic is useful for minimizing which of the following common errors (in thinking about effect sizes)?

A) the ecological fallacy
B) inferring that a failure to reject the null hypothesis also implies an effect size of zero
C) error of equating the rejection of the null hypothesis with having demonstrated a scientifically important effect
D) b and c

Question 9

Power is equal to

A) alpha.
B) beta.
C) 1 minus alpha.
D) 1 minus beta.

Question 10

One problem with NHST is that it can be misconstrued as an indicator of

A) statistical significance.
B) practical importance.
C) theoretical significance.
D) statistical differences.

Question 11

The "error of blindness" describes which hypothesis decision outcome?

A) Rejecting the null hypothesis when the null hypothesis should not be rejected.
B) Failing to reject the null hypothesis when the null hypothesis should not be rejected.
C) Rejecting the null hypothesis when the null hypothesis should be rejected.
D) Failing to reject the null hypothesis when the null hypothesis should be rejected.

Question 12

Which of the following refers to the probability of making a Type II error?

A) significance level
B) beta
C) alpha
D) p-value

Question 13

Type I error is to __________ as Type II error is to __________.

A) blindness; gullibility
B) significance; practicality
C) statistical power; effect size
D) gullibility; blindness

Question 14

When the alternative hypothesis makes a specific prediction, one should use

A) the null p value.
B) a two-tailed p value.
C) a one-tailed p value.
D) an alternative p value.

Question 15

Statistically speaking, the experimental hypothesis is generally referred to as the __________ hypothesis.

A) effective
B) null
C) working
D) alternative

Question 16

At the beginning of a study, the researcher hypothesizes that there is no difference between the experimental and control groups. This is an example of a(n)

A) Type I error.
B) null hypothesis.
C) alternative hypothesis.
D) significant hypothesis.

Question 17

Power is the probability of not making a(n)

A) Type I error.
B) Type II error.
C) statistical error.
D) experimental error.

Question 18

Stacy purchases a new product that claims it will help her sleep better at night. However, after using it, Stacy discovers that it has no effect on her sleeping habits. Which of the following statements would best characterize Stacy's decision to buy the product?

A) She made a Type I error.
B) She made a Type II error.
C) She made a statistical significance error.
D) She made the correct decision.

Question 19

The initials "NHST" stand for

A) Normal HypothesiS Testing.
B) Null HypothesiS Testing.
C) Normal Hypothesis Significance Testing.
D) Null Hypothesis Significance Testing.

Question 20

____________ tells us the nonnull magnitude of the effect size that is supported by exactly the same amount of evidence as is the null value of the effect size.

A) The chi-square statistic
B) The Pearson's r
C) The counternull statistic
D) The coefficient of determination

Question 21

Through statistical power analysis, one can estimate the __________ needed to adequately test a hypothesis.

A) number of participants
B) effect size
C) significance test
D) p value

Question 22

When would one use a one-tailed p value rather than a two-tailed p value? Why would one prefer to use a one-tailed p value in these situations?

Question 23

What are the two common errors in thinking about effect sizes that the counternull statistic helps the researcher minimize? Is this the same as Type I and Type II errors?

Question 24

Smaller sample sizes tend to __________ confidence intervals, whereas larger sample sizes tend to __________ confidence intervals.

A) have no effect on; shrink
B) widen; have no effect on
C) shrink; widen
D) widen; shrink

Question 25

What is the difference between Type I and Type II errors? Which error do scientists seem to be more concerned with? Why?

Question 26

How is null hypothesis significance testing (NHST) sometimes misunderstood or misused?

Question 27

What role does probability have in significance testing?

Question 28

Why are Type I and Type II errors characterized respectively as being errors of gullibility and blindness to a relationship?

Question 29

Tamara wants to know how many subjects she should use in her study. To answer this question, Tamara should

A) calculate the study's effect size.
B) use null hypothesis significance testing.
C) conduct a power analysis.
D) construct a BESD.

Question 30

How can a statistical power analysis be useful when planning an experimental study?

Question 31

The power of a statistical test is determined by all of the following factors EXCEPT

A) the p value associated with the statistical test.
B) the internal validity of the study.
C) the sample size used in the study.
D) the effect size associated with the statistical test.

Question 32

Why does "nonsignificance" not mean the same thing as "no effect" in null hypothesis significance testing?

Question 33

Killeen's p_rep indicates the probability of replicating the same ___________ of effect of the original study, given the same N and procedures used.

A) size
B) outcome
C) direction
D) power

Question 34

What is meant by the power of a statistical significance test?

Question 35

Why would one want to calculate the confidence interval associated with an observed effect size?