Deck 12: Data Analysis and Interpretation: Part II Tests of Statisticalsignifi Cance and the Analysis Story

Question 1

Figure:
Read the following research report and answer the questions that follow.
A researcher compares men's and women's attitudes toward dating their best friend's former girl/boy friend.Specifically,college students are asked to read a story describing a situation in which their best friend breaks up with a partner and they later have an opportunity to date their best friend's former partner.Men and women rate the likelihood they would choose to date their best friend's former partner using a 10-point scale (0 = not at all likely and 9 = completely likely).In addition,participants rate the extent to which a similar situation has occurred to them using a 10-point scale (0 = not at all and 9 = completely).The results of this study are presented below:
How many men and women were there in this study (assuming equal numbers of men and women)?

Answer 1

We know there were 30 participants total because the df are 28;thus,there were 15 men and 15 women.

Question 2

Figure:
Use the ANOVA Summary Table to answer the questions that follow:
What is the total number of subjects,and assuming equal group size,how many subjects are there in each group?

Answer 2

The total number of subjects in 42;with six conditions in this complex design there are 7 subjects per group.

Question 3

Figure:
Use the ANOVA Summary Table to answer the questions that follow:
What values form the numerator and denominator for the F-ratio (F-test)for each statistical effect?

Answer 3

The denominator for each F-test is the same,the Mean Square Error (10.25).The numerator for each test is the Mean Square associated with each effect:Factor A,2.38;Factor B,36.45;A X B,102.45.

Question 4

(a)What is a Type I error and what is a Type II error when using NHST? (b)Which is more common in psychological research?

Question 5

The level of significance,or alpha,in psychological research is generally set at

A).01.
B).05.
C).10.
D).50.

Question 6

Figure:
Use the ANOVA Summary Table to answer the questions that follow:
Which results are statistically significant? Explain how you arrived at this decision.

Question 7

Null hypothesis significance testing is used to compare two means in an independent groups design.(a)What is the null hypothesis? (b)What is required to "reject" the null hypothesis?

Question 8

Explain the difference between experimental sensitivity and statistical power and identify factors that influence each.

Question 9

What does NHST tell us when our results are "statistically significant"?

Question 10

When a result is "statistically significant," why should we not immediately claim that our results are "important" (either scientifically or practically)?

Question 11

The probability we use to define a statistically significant outcome is called the

A)level of significance.
B)null hypothesis.
C)effect size
D)level of rejection.

Question 12

The null hypothesis is the assumption that the independent variable

A)had an effect.
B)did not have an effect.
C)interacts with another variable.
D)is statistically significant.

Question 13

Null hypothesis significance testing uses the laws of probability to estimate the likelihood of an outcome assuming that

A)the null hypothesis is false.
B)an effect is present.
C)only chance factors caused the outcome.
D)the independent variable workeD.

Question 14

Figure:
Read the following research report and answer the questions that follow.
A researcher compares men's and women's attitudes toward dating their best friend's former girl/boy friend.Specifically,college students are asked to read a story describing a situation in which their best friend breaks up with a partner and they later have an opportunity to date their best friend's former partner.Men and women rate the likelihood they would choose to date their best friend's former partner using a 10-point scale (0 = not at all likely and 9 = completely likely).In addition,participants rate the extent to which a similar situation has occurred to them using a 10-point scale (0 = not at all and 9 = completely).The results of this study are presented below:
Were the results statistically significant? Explain your decisions.

Question 15

Figure:
Use the ANOVA Summary Table to answer the questions that follow:
Describe the analyses that should be conducted next.

Question 16

Briefly describe the logic of the analysis of variance or F-test for a single-factor random groups design.

Question 17

Figure:
Use the ANOVA Summary Table to answer the questions that follow:
How many levels are there for Factor A and Factor B?

Question 18

Figure:
Read the following research report and answer the questions that follow.
A researcher compares men's and women's attitudes toward dating their best friend's former girl/boy friend.Specifically,college students are asked to read a story describing a situation in which their best friend breaks up with a partner and they later have an opportunity to date their best friend's former partner.Men and women rate the likelihood they would choose to date their best friend's former partner using a 10-point scale (0 = not at all likely and 9 = completely likely).In addition,participants rate the extent to which a similar situation has occurred to them using a 10-point scale (0 = not at all and 9 = completely).The results of this study are presented below:

-)The square root of 28 (the df)is approximately 5.3.What is Cohen's d for the difference between ratings for men and women for the question regarding the likelihood of dating? According to Cohen's criteria,how would you describe this effect?

Question 19

Figure:
Read the following research report and answer the questions that follow.
A researcher compares men's and women's attitudes toward dating their best friend's former girl/boy friend.Specifically,college students are asked to read a story describing a situation in which their best friend breaks up with a partner and they later have an opportunity to date their best friend's former partner.Men and women rate the likelihood they would choose to date their best friend's former partner using a 10-point scale (0 = not at all likely and 9 = completely likely).In addition,participants rate the extent to which a similar situation has occurred to them using a 10-point scale (0 = not at all and 9 = completely).The results of this study are presented below:
Describe whether a Type I error and whether a Type II error are possible in this study.

Question 20

Figure:
Read the following research report and answer the questions that follow.
A researcher compares men's and women's attitudes toward dating their best friend's former girl/boy friend.Specifically,college students are asked to read a story describing a situation in which their best friend breaks up with a partner and they later have an opportunity to date their best friend's former partner.Men and women rate the likelihood they would choose to date their best friend's former partner using a 10-point scale (0 = not at all likely and 9 = completely likely).In addition,participants rate the extent to which a similar situation has occurred to them using a 10-point scale (0 = not at all and 9 = completely).The results of this study are presented below:
What do the p values associated with each t-test tell you besides whether the result is statistically significant?

Question 21

If a researcher were to use a .01 level of significance rather than the .05 level,the probability of a Type I error would

A)not change.
B)increase.
C)decrease.
D)not able to tell without more information

Question 22

Results that are "statistically significant" may not be of interest to the scientific community because

A)the study's methodology was poor.
B)the results have little external validity.
C)the treatment effect is too small to be of practical value.
D)any of these reasons

Question 23

The initial overall analysis of an experiment based on the analysis of variance is called

A)the interaction F-test.
B)the omnibus null test.
C)an omnibus F-test.
D)a multiple-group analysis.

Question 24

Which of the following factors is not likely to be related to the power of a statistical test comparing two means?

A)sample size
B)whether t or F is used
C)level of significance
D)size of the treatment effect

Question 25

When researchers have a good estimate of the expected effect size for an independent variable prior to conducting a study,they should

A)compute an inferential statistics test.
B)perform a power analysis.
C)draw confidence intervals for their means.
D)all of these

Question 26

When deciding how to make a claim about the effect of a variable,researchers should

A)be aware of limitations association with null hypothesis significance testing.
B)report the results of significance testing instead of measures of effect size.
C)use the most complicated analysis to tease out the fullest effect.
D)use null hypothesis significance testing instead of confidence intervals.

Question 27

A researcher plans to conduct a study to compare two groups.Based on previous research,she anticipates a medium effect size.She does a power analysis with alpha level of .05.This power analysis tells her

A)the t-test outcome and degrees of freedom she should expect.
B)the sample size she will need to observe a statistically significant finding.
C)Cohen's d for her study.
D)the practical significance of her research.

Question 28

The most frequently used inferential statistics test when comparing more than two means is

A)analysis of variance (ANOVA).
B)repeated measures t.
C)t-test for independent groups.
D)correlation coefficient,r.

Question 29

The degrees of freedom for a t-test for independent groups are

A)(n₁ + n₂)- 1.
B)(n₁ + n₂)- 2.
C)N - 1.
D)2(N)- 1.

Question 30

The F-test provides a statistic that represents the ratio of between-group variation in the data to

A)sample size.
B)total variation in the data.
C)degrees of freedom.
D)within-group variation.

Question 31

The most common error associated with null hypothesis testing in psychological research is

A)random sampling.
B)too much power.
C)Type I error.
D)Type II error.

Question 32

The probability we use to define a statistically significant outcome is equivalent to

A)Type II error.
B)the null hypothesis.
C)level of rejection.
D)alpha.

Question 33

The primary factor that researcher use to control power of a statistical test is

A)size of the treatment effect.
B)level of significance.
C)population size.
D)sample size.

Question 34

A result that is not "statistically significant" means that

A)the null hypothesis is definitely false.
B)the null hypothesis is definitely true.
C)we should conclude that the independent variable had no effect whatsoever.
D)without more information,we should be cautious about concluding that the independent variable had more than a trivial effect.

Question 35

The expected value of the F-statistic when the null hypothesis is true is

A)1.00
B).50.
C).05
D)0.0.

Question 36

The appropriate inferential test when each subject participates in both conditions of an experiment is

A)a between-subjects t-test.
B)a between-conditions t-test.
C)a repeated measures t-test.
D)a t-test for independent groups.

Question 37

The t-test for independent groups is the appropriate inferential test when

A)comparing two means.
B)a single random sample is taken from a population.
C)comparing more than two means.
D)calculating a correlation between groups.

Question 38

A psychologist conducts a study to test a new treatment.The effect of treatment is statistically significant;the effect size is small according to accepted criteria.When considering the practical or clinical significance of the finding,the psychologist is likely to be concerned about

A)the financial cost of implementing the treatment.
B)the external validity of the finding.
C)how the study was conducted.
D)all of these

Question 39

A "statistically significant" finding is generally reported as

A)proving the researcher's hypothesis.
B)proving that the null hypothesis is false.
C)supporting the researcher's hypothesis.
D)proving that the null hypothesis is true.

Question 40

If a researcher were to use a .01 level of significance rather than the .05 level,the probability of a Type II error would

A)increase.
B)decrease.
C)not change.
D)not able to tell without more information

Question 41

Suppose the omnibus analysis of variance for a 2 × 2 complex design reveals only a statistically significant main effect of one independent variable.To understand the effect of this variable the researcher should

A)compute simple main effects analyses.
B)use t-tests to compare two means at a time.
C)examine the means for the independent variable collapsed across the other independent variable.
D)all of these

Question 42

The primary way that analysis of variance for repeated measures differs from that for an independent groups design is in the estimation of

A)between-group variation.
B)variability between means.
C)error variation.
D)systematic variation.

Question 43

Confidence intervals drawn around group means in a complex design provide information regarding

A)the null hypothesis.
B)the probable pattern of population means.
C)interaction effects.
D)the statistical significance of any differences among population means.

Question 44

The ANOVA Summary Table for a two-factor,mixed design is divided into two parts;the between subjects section and the __________ section.

A)within-subjects
B)interaction
C)eta-squared
D)repeated-subjects

Question 45

Two measures of effect size for an independent groups design with more than two means are

A)eta-squared and Cohen's f.
B)Cohen's d and the F-test.
C)the t-test and the F-test.
D)F squared and rho.

Question 46

A common measure of effect size for a complex design using ANOVA is

A)eta squared.
B)N - 1.
C)Cohen's d.
D)Mean Square Error.

Question 47

The results of an omnibus F-test for a complex design experiment allow the researcher to

A)know exactly which means significantly differ from one another.
B)interpret the outcome of the experiment without knowing the descriptive statistics for the cells of the design.
C)know the strength of the relationships between the independent and dependent variables.
D)know whether the interaction effects and main effects were statistically significant.

Question 48

If the omnibus analysis of variance for a complex design reveals a statistically significant interaction effect,the source of the interaction effect may be identified using

A)simple main effects analyses.
B)null comparisons.
C)overall interaction analyses.
D)complex comparisons.

Question 49

= 7.30,p = .01.On the basis of this information we may conclude that there were _______ independent groups in this experiment.

A)two
B)three
C)four
D)five

Question 50

In general,a repeated measures design is likely to be ________ than a random groups design.

A)less sensitive
B)more sensitive
C)more accurate
D)less powerful

Question 51

Before examining the "significance" of an inferential test,a researcher should carefully examine the

A)descriptive (summary)statistics.
B)statement of the null hypothesis.
C)results of previous experiments.
D)probability values.

Question 52

If the omnibus analysis of variance for a complex design reveals a statistically significant interaction effect,the source of the interaction effect may be identified using simple main effects analyses and,when there are more than two levels of an independent variable,also

A)complex comparisons.
B)null comparisons.
C)simple correlations between variables.
D)comparisons between two means.

Question 53

When our results are "statistically significant" we know the

A)nature of the effect of the independent variable.
B)means and standard deviations for the effect.
C)effect size between the independent and dependent variables is large.
D)independent variable produced an effect on the dependent variable.

Question 54

A researcher manipulated one independent variable in a complex design experiment using a random groups design and manipulated the second independent variable using a repeated measures design.The researcher's plan for data analysis should include

A)a correlation between the first and second independent variables.
B)a single-factor ANOVA for the first independent variable and a repeated measures t-test for the second independent variable.
C)a two-factor ANOVA for a mixed design.
D)a confidence interval for the overall mean in the experiment.

Question 55

When reporting the results of a complex design experiment,which of the following should not be included?

A)summary statistics for cells in the design in the text,a table,or a figure.
B)results for omnibus F-test,with exact probabilities.
C)ideas for future research involving the variables
D)verbal description of any statistically significant interaction effects

Question 56

= 7.30,p = .01.On the basis of this information we may conclude that

A)there were two independent groups in the experiment.
B)there were 42 total subjects in the experiment.
C)the results are statistically significant.
D)all of these

Question 57

Following an omnibus F-test a researcher may learn about specific sources of systematic variation in a single-factor experiment by performing

A)another F-test.
B)comparisons of two means.
C)correlation coefficients.
D)correlational analyses.