Deck 7: Statistical Significance, Effect Size, and Confidence Intervals

Question 1

What, exactly, does a p value tell you?

Answer 1

The probability that a result (i.e., statistic) was obtained by chance, or random sampling error.

Question 2

In the population of students who take the SAT, the average combined score is 1600. I wonder whether students in Ohio differ from the population of students who take the SAT. Suppose I select a random sample of 100 students from Ohio and find that their average SAT score was 1630 with a standard deviation of 200.
a. Write the null and alternative hypotheses for this research question.
b. Is the difference between the sample mean and the population mean is statistically significant using an alpha level of .05?
c. Then calculate the 99% confidence interval for the mean and wrap words around it.

Answer 2

a. Null hypothesis: The sample mean for Ohio students equals the population mean on the SAT.
Alternative hypothesis: The sample mean for Ohio students does not equal the population mean.
b.First, we must first calculate a standard error of the mean:
$S_{\bar{x}}$ = $\frac{200}{\sqrt{100}}$
= 200/10 = 20
Next, calculate the t value for this problem and determine whether it is statistically significant.
First of all, we do not know the population standard deviation, so we have to calculate a t value, not a z score. To do this, we must first calculate a standard error of the mean:
Here is how I can calculate a t value: t = $\frac{1630-1600}{20}$ = 1.5
The degrees of freedom for this problem is n - 1, so 100 - 1 = 99. Now look in Appendix B. Notice that 99 is between 120 and 60. When we have a situation like this, go with the smaller df in Appendix B, unless your df is much closer to one of the values in Appendix B than the other. So let's take a look at the row with 60 degrees of freedom in Appendix B. We are looking for the value closest to our t value of 1.5. We can see that our observed t value of 1.5 is between 1.296 and 1.671 for df = 60. Go up to the top of these two columns to where it says ..20 and .10. That is the probability of getting a t value of 1.5 by chance with 60 degrees of freedom. Out t value is 1.5, so the probability of getting this value by chance is between .10 and .20. This is LARGER THAN our alpha level of .05, so we will conclude the difference between our sample and population means was NOT statistically significant. Ohio student did not do better, on average, than the population of SAT takers.
c. CI = sample mean +/- (t)( $S_{\bar{x}}$ )
We already know the sample mean (1630) and the standard error of the mean (20). So all we need to do is look up the t value in Appendix B. We are still using 60 degrees of freedom and we are looking for the column that says .01 for 2-tailed test. We are using .01 here because our confidence interval is a 99% CI, and 1-.99 = .01. So using this information we find a t value of 2.660. Now we do the math.
CI₉₉ = 1630 +/- (2.660) (20) $\rightarrow$ 1630 +/- 53.20. So our confidence interval is 1576.80 to 1683.20. We are 99% confident that the population mean is between 1576.80 and 1683.20.

Question 3

Suppose that I have a sample of 25 women and they spend an average of $100 a week dining out, with a standard deviation of $20. Create a 95% confidence interval for the mean and wrap words around your results.

Answer 3

CI₉₅ = 100 +/- (4)(2.064) $\rightarrow$ 100 +/- 8.26 $\rightarrow$ 91.74, 108.26. We are 95% confident that the population mean is in this interval.

Question 4

When we say that a result is statistically significant what exactly does that mean? (Your answer should include a discussion of the role of the alpha level and the p value).

Question 5

When you say that a result is NOT statistically significant, what exactly does that mean?

Question 6

My wife used to work for the Gap. Ever since she started working there, I've become very sensitive to the way people dress, and it seems to me like every adult in America between the ages of 18-40 owns about six items of gap clothing. Upon hearing this, you think "Dr. Urdan's crazy! That's way too high! I bet the average adult doesn't own nearly that many articles of Gap clothing." Suppose you wanted to test this hypothesis. You select a random sample of 20 American adults between the ages of 18-40 and find that, on average, each owns five items of Gap clothing, with a standard deviation of 1. Using an alpha level of .01, tell me what you conclude about our competing hypotheses. Wrap words around your final result.

Question 7

My mother is one of those crazy cat people who has 20 cats. She thinks because she loves them so much they are healthier than most cats. I want to know whether this is true. Suppose that in the population of cats, the average "health index" score is 10. The average "health index" score for my mom's cats is 11, with a standard deviation of 2. (For the purposes of this question, pretend that my mom's cats represent a random sample taken from the larger population.)
a. State the null and alternative hypotheses
b. Chose an alpha level.
c. Decide whether you are doing a 1-tailed or 2-tailed test (explain why)
d. State your degrees of freedom.
e. Find and report your critical value for t.
f. Compute your observed t value.
g. Decide whether to reject or fail to reject H_o
h. What does that mean, exactly, to say that this result is statistically significant?
i. Calculate and report a 95% confidence interval for the mean and wrap words around it.

Question 8

Suppose I have a randomly selected sample of 25 first-grade girls. I find that this sample has an average 5 dolls with a standard deviation of 1.5. Calculate a 99% confidence interval for the population mean and wrap words around it