Deck 6: Making Sense of Statistical Significance:decision Errors, Effect Size, and Statistical Power

Question 1

Using the .01 level of significance means that,in the long run,

A)a Type I error occurs 1 time in 20.
B)a Type I error occurs 1 time in 100.
C)a Type II error occurs 1 time in 20.
D)a Type II error occurs 1 time in 100.

Answer 1

a Type I error occurs 1 time in 100.

Question 2

The probability of rejecting the null hypothesis when in fact it is true is

A)a Type II error.
B)the power of rejection.
C)alpha.
D)beta.

Answer 2

alpha.

Question 3

Effect size is

A)the amount of statistical importance a particular finding has in psychological practice.
B)the power of a study divided by the sample size.
C)the sample size multiplied by 1 - beta.
D)the degree to which an experimental manipulation separates two populations.

Answer 3

the degree to which an experimental manipulation separates two populations.

Question 4

The conventional levels of significance of 5% and 1%

A)provide maximum protection against Type I errors.
B)provide maximum protection against Type II errors.
C)are a compromise between the risks of making Type I and Type II errors.
D)are more related to power and sample-size considerations than decision errors.

Question 5

Failing to reject the null hypothesis when the research hypothesis is true is

A)effect size.
B)power.
C)a Type I error.
D)a Type II error.

Question 6

Type II errors concern psychologists because

A)they could mean that good theories or useful practical procedures are not used.
B)they could mean that experiments must be repeated to confirm positive results.
C)rejecting the null hypothesis should only occur when the research hypothesis is true.
D)future research might be based on results mistakenly declared significant.

Question 7

A cognitive psychologist tests whether memory for abstract ideas improves when the information is presented both verbally and visually.Which of the following scenarios represents a Type I error?

A)The psychologist concludes that memory does not improve when the true situation is that it does not improve.
B)The psychologist concludes that memory does not improve when the true situation is that it does improve.
C)The psychologist concludes that memory does improve when the true situation is that it does improve.
D)The psychologist concludes that memory does improve when the true situation is that it does not improve.

Question 8

A child psychologist conducts a study to test whether infants born prematurely begin to walk earlier than do infants in general.Which of the following scenarios represents a Type I error?

A)The psychologist concludes that premature infants do walk earlier than infants in general when the true situation is that they actually do not.
B)The psychologist concludes that premature infants do walk earlier than infants in general when the true situation is that they actually do.
C)The psychologist concludes that premature infants do not walk earlier than infants in general when the true situation is that they actually do.
D)The psychologist concludes that premature infants do not walk earlier than infants in general when the true situation is that they actually do not.

Question 9

Rejecting the null hypothesis when the research hypothesis is false is

A)effect size.
B)power.
C)a Type I error.
D)a Type II error.

Question 10

A child psychologist conducts a study to test whether infants born prematurely begin to walk earlier than do infants in general.Which of the following scenarios represents a Type II error?

A)The psychologist concludes that premature infants do walk earlier than infants in general when the true situation is that they actually do not.
B)The psychologist concludes that premature infants do walk earlier than infants in general when the true situation is that they actually do.
C)The psychologist concludes that premature infants do not walk earlier than infants in general when the true situation is that they actually do.
D)The psychologist concludes that premature infants do not walk earlier than infants in general when the true situation is that they actually do not.

Question 11

A cognitive psychologist tests whether memory for abstract ideas improves when the information is presented both verbally and visually.Which of the following scenarios represents a Type II error?

A)The psychologist concludes that memory does not improve when the true situation is that it does not improve.
B)The psychologist concludes that memory does not improve when the true situation is that it does improve.
C)The psychologist concludes that memory does improve when the true situation is that it does improve.
D)The psychologist concludes that memory does improve when the true situation is that it does not improve.

Question 12

The alpha level is

A)usually determined while analyzing experimental data.
B)the chance of obtaining an inconclusive result when the research hypothesis is true.
C)the probability of a Type I error.
D)always set at 1%.

Question 13

One important advantage of using effect sizes is that

A)they indicate all the relevant parameters of a group of scores.
B)they directly indicate the statistical significance of the research hypothesis.
C)they provide the same information as statistical significance,but they permit meta-studies.
D)they are standardized scores that make comparisons of different studies easier.

Question 14

The degree to which the experimental manipulation separates the two populations of individual scores is

A)experimental effectiveness.
B)power.
C)the effect size.
D)the significance level.

Question 15

Beta is the probability that

A)if the research hypothesis is true,the experiment would still fail to support it.
B)the research will be able to correctly reject the null hypothesis.
C)there will be neither a Type I nor a Type II error.
D)an alpha error will not occur.

Question 16

Setting the significance level at .20 increases the chance of

A)obtaining a nonsignificant result.
B)failing to reject the null hypothesis.
C)a Type I error.
D)a Type II error.

Question 17

Cohen proposed effect-size conventions based on the effects observed in psychology research in general because

A)experimenters need to assess whether the obtained effect size will allow them to reject the null hypothesis.
B)determining how large an effect can be expected in a given experiment is difficult.
C)they follow from the original scales he developed.
D)they are more accurate than calculating a minimum meaningful difference.

Question 18

A Type I error is the result of

A)improper measurement techniques on the part of the researcher.
B)failing to reject the null hypothesis when,in fact,it is true.
C)incorrectly rejecting the null hypothesis.
D)incorrectly failing to reject the null hypothesis.

Question 19

Setting the significance level cutoff at .10 instead of the more usual .05 increases the likelihood of

A)a Type I error.
B)a Type II error.
C)failing to reject the null hypothesis.
D)accepting the null hypothesis when,in fact,it is false.

Question 20

If beta equals 70%,then alpha

A)equals 30%.
B)equals 50%.
C)equals 70%.
D)cannot be determined from information provided.

Question 21

The effect size and the number of participants are two important determinants of

A)the minimum significant result.
B)experimental significance.
C)power.
D)alpha.

Question 22

Power is the probability that

A)if the research hypothesis is false,the experiment will support the null hypothesis.
B)if the research hypothesis is true,the experiment will support it.
C)a Type I error will not be made.
D)the research hypothesis will not be accepted incorrectly.

Question 23

In actual practice,the usual reason for determining power before conducting a study is

A)to eliminate the possibility that a mistake may occur.
B)to ensure that regardless of whether the research hypothesis is true,the experiment will yield a significant result.
C)to determine the number of participants needed to have a reasonable level of power.
D)to anticipate the likelihood that the experiment will need to be repeated.

Question 24

If the effect size and the desired level of power are known,the power formula can be used to determine the

A)minimum meaningful difference.
B)number of participants needed.
C)population distribution.
D)alpha level of the experiment.

Question 25

When conducting an experiment with 95% power,failing to reject the null hypothesis means that

A)there is no difference between the experimental and control populations.
B)the experiment was poorly conducted.
C)the null hypothesis is true.
D)the chance that the data came from populations with a difference larger than that predicted when calculating power is less than 5%.

Question 26

If an experimenter CANNOT manipulate the effect size of an experiment to increase power,the aspect of a study that can usually be changed easily to increase power is

A)the sample size.
B)the beta level.
C)the population parameters.
D)the sample mean.

Question 27

Cohen's (1962)review of power in psychological studies revealed that

A)power levels were reasonable for most studies.
B)most researchers used large sample sizes to compensate for low power.
C)no study had power greater than 50%.
D)most studies were not worth repeating.

Question 28

The statistical method currently used to combine the results of multiple studies is

A)meta-analysis.
B)power analysis.
C)regression analysis.
D)retrospective analysis.

Question 29

The probability of NOT making a Type II error is

A)the experimental safety region.
B)power.
C)alpha.
D)beta.

Question 30

If a study conducted at the .05 significance level has 80% power

A)alpha = 5%,beta = 80%
B)alpha = 5%,beta = 20%
C)alpha = 95%,beta = 80%
D)alpha = 95%,beta = 20%

Question 31

A psychologist conducts a study and finds that d = -.63.This effect size would most likely be described as

A)small
B)medium
C)large
D)an error because d cannot be negative

Question 32

Some psychologists have argued that simple significance tests are misleading,and that a better statistic to evaluate the outcome of an experiment would be the

A)effect size.
B)level of power.
C)probability of a Type II error.
D)number of participants divided by the standard deviation.

Question 33

Effect size is an important tool for making sense of research results because it

A)indicates the degree to which a result is statistically significant.
B)indicates the size of the statistical effect.
C)limits variance and therefore increases power.
D)is always smaller than the variance.

Question 34

If the research hypothesis is true but the study has a low level of power,then

A)the probability that the study will have a significant result is high.
B)the probability that the study will have a significant result is low.
C)the null hypothesis will almost certainly be rejected.
D)beta is necessarily low.

Question 35

When the standard deviation of the original population is small,the experiment tends to

A)have a higher level of power.
B)have a lower level of power.
C)show a less significant difference between the two populations.
D)have a high level of Type II error.

Question 36

While no research hypothesis is ever definitely false,failing to reject the null hypothesis in a study that has a high level of power allows one to

A)suspect that the research hypothesis may still be true.
B)conclude that the research hypothesis is most likely false.
C)make no statements about the research hypothesis.
D)reject the notion that the effect size has anything to do with statistical significance.

Question 37

Effect size is one of the two major factors that contribute to power; the other factor is

A)the sample's standard deviation.
B)the minimum meaningful difference.
C)the sample size.
D)the mean of the known population.

Question 38

Standard power tables are useful for

A)directly determining the power of an experiment.
B)determining the predicted score for the group exposed to the experimental manipulation.
C)determining the predicted effect size of a proposed experiment.
D)determining the probability of falsely accepting the research hypothesis.

Question 39

James asks his friend Rob for advice on how to increase the power of his experiment.Which of Rob's suggestions is incorrect?

A)use a larger sample size
B)use a less diverse population
C)use a two-tailed instead of a one-tailed test
D)use a more intense experimental manipulation to increase effect size

Question 40

A psychologist conducts a study and finds that d = 1.41.This effect size would most likely be described as

A)small
B)medium
C)large
D)an error because d cannot be greater than 1

Question 41

The procedure for combining the results of several different studies is __________.

Question 42

Increasing sample size __________ power.

Question 43

d = (μ1 - μ2)/ __________.

Question 44

According to Cohen,a good general rule for when it is worthwhile to conduct a study is when power is at least __________%.

Question 45

The degree of separation between populations due to the independent variable is one way of understanding __________.

Question 46

The probability of making a Type I error is __________.

Question 47

A way to increase effect size is to change the procedures of a study so that the __________ of the population is decreased.

Question 48

If power is 45%,then beta equals __________.

Question 49

Failing to reject the null hypothesis when in fact it is false is a(n)__________.

Question 50

To determine how likely it is that the result of a planned experiment will be significant if,in fact,the research hypothesis is true,an experimenter can figure __________.

Question 51

When the known and comparison populations overlap by 85%,the effect size is __________.

Question 52

A statistic that combines two of the main influences on power,the expected difference between populations and the population standard deviation,is __________.

Question 53

The effect size of a study is a good indicator of its practical importance because,unlike statistical significance,it is not affected by __________.

Question 54

Type I errors occur when a statistically significant result is obtained when in fact __________.

Question 55

Rejecting the null hypothesis when in fact it is true is a(n)__________.

Question 56

Using the .01 level of significance instead of the .05 level increases the chance of making a(n)__________.

Question 57

The probability of making a Type II error is __________.

Question 58

Most of the time,experimenters determine the power of a planned experiment by using a(n)__________.

Question 59

p < .05 means that there is less than a 5% chance of making a(n)__________.

Question 60

The probability of not making a Type II error is __________.

Question 61

A new company has made the claim that its test preparation program will improve SAT scores by 25 points.A skeptical educational psychologist has decided to test this theory and has enlisted 25 students who are willing to participate in the program.Assume the standard SAT mean is 550 with a standard deviation of 100.
a.What is the predicted effect size for an increase of 25 points?
b.Explain the logic of what has been done.
c.Advanced topic: Sketch the distributions involved.
d.Advanced topic: If the data are analyzed at the .05 significance level,what will be the power of the study?
e.Advanced topic: Explain your answer to a person who understands hypothesis testing with means of samples,but who is not familiar with power.

Question 62

Interpreting the results of theoretical studies is more likely to emphasize significance level,while interpreting the results of applied studies is more likely to emphasize __________.

Question 63

Advanced topic: A social psychologist is designing a study to see whether people who see a particular film are more likely to use violence as a solution to a problem.The psychologist plans to use a scale that assesses this construct,which has a population mean of 78 and a standard deviation of 14.The psychologist expects an increase of eight points above the population mean.Fifteen participants will be enrolled in the study,and the data will be analyzed at the .05 level.
a.What is the power of the study?
b.Sketch the distributions involved.
c.Explain your answer to a person who understands hypothesis testing with means of samples,but who is not familiar with effect size or power.
d.What is the predicted effect size?
e.What are three things that might be done to increase the power of the study?

Question 64

The smaller the population standard deviation,the greater the power,because the distributions of means based on the known and comparison populations are both __________.

Question 65

A researcher obtains a statistically significant result.However,if he used a large sample,then the result may not __________.

Question 66

An organizational psychologist conducted a study to see whether upgrading a company's older computer system to newly released,faster machines would cause an increase in productivity from the current average of 120 units with a standard deviation of 20.The new system will be tested in a single department with 45 employees.The company has decided that an increase of less than 10 units will not justify purchasing the new system.
a.What is the predicted effect size for an increase of 10 points?
b.Explain the logic of what has been done.
c.Advanced topic: Sketch the distributions involved.
d.Advanced topic: If the data are analyzed at the .01 significance level,what will be the power of the study?
e.Advanced topic: Explain your answer to a person who understands hypothesis testing with means of samples,but who is not familiar with power.

Question 67

Advanced topic: A psychologist investigating hearing thresholds knows that an average participant gets a score of 42 with a standard deviation of 15 on a standard hearing test.The psychologist plans to add a distraction task to the standard test for 10 participants and evaluate the effect of the distraction at the .01 level.The psychologist believes that the distraction should result in a 12-point reduction in scores .During the planning process,the psychologist realizes that 10 participants may not be enough for adequate power and has asked a consultant to calculate the power for this experiment.
a.What is the current power of the planned study?
b.Sketch the distributions involved.
c.What are three ways to increase the power of this study?
d.What is the predicted effect size?
e.Explain these procedures to a person who understands hypothesis testing with means of samples,but who is not familiar with effect size or power.

Question 68

A study in which the results are not sufficient to reject the null hypothesis is truly inconclusive if the __________ is low.

Question 69

In studies using a very large number of participants,it is common to get statistically significant results that have a very small __________.