Deck 15: Random- and Mixed-Effects Analysis of Variance Models

Question 1

Complete the following ANOVA summary table for a two-factor model, where there are two levels of factor A (fixed program effect) and five levels of factor B (random school effect). Each cell of the design includes seven students ( $\alpha$ = .05).

$\begin{array}{ccccccc}\hline \text { Source } & S S & d f & M S & F & \text { Critical Value } & \text { Decision } \\\hline \mathrm{A} & 100 & - & - & - & - & - \\\mathrm{B} & 210 & - & - & - & - & - \\\mathrm{AB} & - & - & 20 & - & - & - \\\text { Within } & - & - & - & & & \\\text { Total } & 690 & - & & & & \\\hline\end{array}$

Answer 1

There are 2 levels of the fixed factor A, so J = 2.
There are 5 levels of the random factor B, so K = 5.
Each cell has 7 students, so n = 7. N = 2*5*7 = 70.

df_A = J- 1 = 2 - 1 = 1, df_B = K - 1 = 5 - 1 = 4, df_AB = (J -1)(K- 1) = 1*4 = 4, df_with = N - JK = 70 - 2*5 = 60, df_total = N - 1 = 70 - 1 = 69.

SS_AB = MS_AB*df_AB = 20*4 = 80;
SS_with = SS_total - SS_A - SS_B - SS_AB = 690 100 210 80 = 300.

MS_A = SS_A/df_A = 100/1 = 100; MS_B = SS_B/df_B = 210/4 = 52.5;
MS_with = SS_with/df_with = 300/60 = 5.

F_A = MS_A/MS_AB = 100/20 = 5; critical value for A = _.05F_1,4 = 7.71 > F_A, fail to reject H₀.

F_B = MS_B/MS_with = 52.5/5 = 10.5; critical value for B = _.05F_4,60 = 2.53 < F_B, reject H₀.

F_AB = MS_AB/MS_with = 20/5 = 4; critical value for AB = _.05F_4,60 = 2.53 < F_AB, reject H₀.

$\begin{array}{ccccccc}\hline \text { Source } & S S & d f & M S & F & \text { Critical Value } & \text { Decision } \\\hline \text { A } & 100 & 1 & 100 & 5 & 05 F_{1,4}=7.71 & \text { fail to reject } H_{0} \\\mathrm{~B} & 210 & 4 & 52.5 & 10.5 & 05 F_{4,00}=2.53 & \text { reject } H_{0} \\\mathrm{AB} & 80 & 4 & 20 & 4 & 05 F_{4, \infty 0}=2.53 & \text { reject } H_{0} \\\text { Within } & 300 & 60 & 5 & & & \\\text { Total } & 690 & 69 & & & & \\\hline\end{array}$

Question 2

Complete the following ANOVA summary table for a two-factor model, where there are three levels of factor A (random dosage effect) and four levels of factor B (random physician effect). Each cell of the design includes six patients ( $\alpha$ = .05).

$\begin{array}{ccccccc}\hline \text { Source } & S S & d f & M S & F & \text { Critical Value } & \text { Decision } \\\hline \mathrm{A} & 25 & - & - & - & - & - \\\mathrm{B} & - & - & 15 & - & - & - \\\mathrm{AB} & - & - & - & - & - & - \\\text { Within } & - & - & 1 & & & \\\text { Total } & 160 & - & & & & \\\hline\end{array}$

Answer 2

There are 3 levels of the fixed factor A, so J = 3.
There are 4 levels of the random factor B, so K = 4.
Each cell has 6 students, so n = 6. N = 3*4*6 = 72.
df_A = J- 1 = 3 - 1 = 2, df_B = K - 1 = 4 - 1 = 3, df_AB = (J - 1)(K - 1) = 2*3 = 6, df_with = N - JK = 72 - 3*4 = 60, df_total = N - 1 = 72 - 1 = 71.

SS_B = MS_B*df_B = 15*3 = 45; SS_with = MS_with*df_with = 1*60 = 60;
SS_AB = SS_total - SS_A - SS_B - SS_with = 160 - 25 - 45 - 60 = 30.
MS_A = SS_A/df_A = 25/2 = 12.5; MS_AB = SS_AB/df_AB = 30/6 = 5.

F_A = MS_A/MS_AB = 12.5/5 = 2.5; critical value for A = _.01F_2,6 = 10.93 > F_A, fail to reject H₀.

F_B = MS_B/MS_AB = 15/5 = 3; critical value for B = _.01F_3,6 = 9.78 < F_B, fail to reject H₀.

F_AB = MS_AB/MS_with = 5/1 = 5; critical value for AB= _.01F_6,60 = 3.12 < F_AB, reject H₀.

$\begin{array}{ccccccc}\hline \text { Source } & S S & D f & M S & F & \text { Critical Value } & \text { Decision } \\\hline \text { A } & 25 & 2 & 12.5 & 2.5 & .01 F_{2,6}=10.93 & \text { fail to reject } H_{0} \\\mathrm{~B} & 45 & 3 & 15 & 3 & .01 F_{3,6}=9.78 & \text { fail to reject } H_{0} \\\mathrm{AB} & 30 & 6 & 5 & 5 & .01 F_{6.00}=3.12 & \text { reject } H_{0} \\\text { Within } & 60 & 60 & 1 & & & \\\text { Total } & 160 & 71 & & & & \\\hline\end{array}$

Question 3

To examine development in the pedagogical content knowledge (PCK) of new teachers, 12 novice teachers are measured on their PCK at the beginning of their teaching career, at the end of the first semester, and at the end of the first school year. The scale that measures PCK ranges from 0 to 45, with higher scores reflecting greater levels of PCK. The data are shown below. Conduct a one-factor repeated measures ANOVA to determine the mean differences across time, using $\alpha$ = .05. Use the Bonferroni method to detect exactly where the differences are among the time points (if they are different).

$\begin{array}{|c|c|c|c|}\hline \text { Subject } & \text { Time 1 } & \text { Time 2 } & \text { Time } 3 \\\hline 1 & 16 & 17 & 28 \\\hline 2 & 17 & 23 & 32 \\\hline 3 & 11 & 21 & 30 \\\hline 4 & 15 & 23 & 35 \\\hline 5 & 21 & 32 & 40 \\\hline 6 & 14 & 22 & 31 \\\hline 7 & 12 & 19 & 31 \\\hline 8 & 17 & 21 & 25 \\\hline 9 & 11 & 16 & 24 \\\hline 10 & 8 & 14 & 20 \\\hline 11 & 14 & 24 & 28 \\\hline 12 & 12 & 19 & 25 \\\hline\end{array}$

Answer 3

Procedure:
Create a data set with three variables: Time 1, Time 2, and Time 3. The data set should have 12 cases, representing 12 teachers.

1. Go to Analyze $\rightarrow$ General Linear Model $\rightarrow$ Repeated Measures.
"2. For Within-Subject Factor Name, enter ""Time.""
For Number of Levels, enter 3 (i.e., 3 time points). Click Add.
For Measure Name, enter ""PCK."" Click Add."
3. Click Define. Move Time1 through Time3 in the left box to Within-Subjects Variables (Time).
4. Click Options. Move "Time" to Display Means for. Select Compare main effects. Select Bonferroni under Confidence interval adjustment. Select Descriptive statistics, Estimates of effect size, and Observed power. Click Continue.
5. To get profile plots, click Plots. Move "Time" to the box under Horizontal Axis. Click Add. Click Continue.
"6. Click OK.
Selected SPSS Output:


Pairwise Comparisons: Bonferroni Method
Measure: PCK

$\begin{array}{r}\hline95 \% \text { Confidence Interval for }\\\text { Difference } \quad\quad\quad\quad \\\begin{array}{ccccccc}\hline\text {(I) }&\text {(J) }&\text {Mean}&\text { Std}\\\text { Time}&\text { Time }&\text { Difference (I-J)}&\text { Error}&\text { Sig }&\text { Lower Bound }& \text { Upper Bound }\\\hline {1} & 2 & -6.917^{*} & .811 & .000 & -9.205 & -4.628 \\& 3 & -15.083^{*} & 1.076 & .000 & -18.119 & -12.048 \\\hline {2} & 1 & 6.917^{*} & .811 & .000 & 4.628 & 9.205 \\& 3 & -8.167^{*} & .796 & .000 & -10.412 & -5.922 \\\hline {3} & 1 & 15.083^{*} & 1.076 & .000 & 12.048 & 18.119 \\& 2 & 8.167^{*} & .796 & .000 & 5.922 & 10.412 \\\hline\end{array}\end{array}$
The results of the test of sphericity give no evidence against the assumption of sphericity (Mauchly's W = .825, $\chi$ ² = 1.925, df = 2, p = .382).
Based on the results of the one-factor repeated measures ANOVA, the effect of the within-subjects factor, Time, is significant at the .05 level (F_2,22 = 139.556, p < .001) with large effect size (partial $\eta$ ² = .927) and maximum observed power, implying that the novice teachers' average level of PCK has changed significantly across three time points.

Results of MCP, using the Bonferroni method, reveal significant differences among all pairs of time points. Specifically, the level of PCK increased significantly from Time 1 (i.e., the beginning of the school year) to Time 2 (i.e., the end of the first semester), from Time 2 to Time 3 (i.e., the end of the school year), and from Time 1 to Time 3. The results therefore suggest that the novice teachers' pedagogical content knowledge increases substantially in their first two semesters of teaching."

Question 4

Dr. Bellus wants to study how people's perception of attractiveness is affected by the gender of the subjects, and the facial expression of the object. Each participant views six headshots (in random order) of the same person, who demonstrates different facial expressions (joy, surprise, fear, anger, sadness, and disgust) in each picture. The participants are then asked to rate the attractiveness level of each picture. The rating is the dependent variable. Below is the selected output of a two-factor split plot ANOVA.

Test of Between-Subjects Effects
Test of Within-Subjects Effects

$\begin{array}{ccccc}\hline \text { Source } & \text { Type III SS } & d f & \text { MS } & F \\\hline \text { A (between-subjects factor) } & 226.81 & 1 & 226.81 & 3.04 \\\mathrm{~S} \text { (subject) } & 2084.61 & 28 & 74.45 & \\\hline\end{array}$

a. Identify the between-subjects factor and within-subjects factor.
$\begin{array}{ccccc}\hline \text { Source } & \text { Type III SS } & d f & \text { MS } & F \\\hline \text { B (within-subjects factor) } & 18802.53 & 5 & 3760.506 & 38.07 \\\text { A*B } & 324.22 & 5 & 64.84 & 0.66 \\\text { B*S }^{*} & 13829.70 & 140 & 98.78 & \\\hline\end{array}$

b. Are the effects significant ( $\alpha$ = .05)? What do these results tell us about people's perception of attractiveness?
c. Is MCP necessary? If so, on which effect(s) should we conduct MCP?

Question 5

A researcher interested in making generalizations about the entire population of categories of the independent variable, not just the levels that were sampled, should pursue what type of ANOVA model?

A) Covariate model
B) Fixed effect
C) Random effect
D) Repeated measure

Question 6

Which of the following is not a characteristic of a two-factor random effects ANOVA?

A) Two factors (or independent variables) each with two or more levels
B) The levels for one of the factors are randomly sampled from the population of levels (i.e., the random-effects factor) and all of the levels of interest for the second factor are included in the design (i.e., the fixed-effects factor)
C) Subjects are randomly selected and assigned to one category of one independent variable only
D) The dependent variable is measured at least at the interval level

Question 7

Which of the following are variations of a two-factor mixed-effects ANOVA?

A) One where factor A is fixed and factor B is fixed, and the other where factor A is random and factor B is random.
B) One where factor A is fixed and factor B is random, and the other where factor A is fixed and factor B is fixed.
C) One where factor A is fixed and factor B is random, and the other where factor A is random and factor B is fixed.
D) One where factor A is random and factor B is random, and the other where factor A is fixed and factor B is fixed.