Exam 6: Linear Regression With Multiple Regressors

Your textbook extends the simple regression analysis of Chapters 4 and 5 by adding an additional explanatory variable, the percent of English learners in school districts (PctEl). The results are as follows: $\widehat{\text { TestScore }}$ = 698.9 - 2.28 × STR and $\widehat{\text { TestScore} }$ = 698.0 - 1.10 × STR - 0.65 × PctEL Explain why you think the coefficient on the student-teacher ratio has changed so dramatically (been more than halved).

For this question, use the California Testscore Data Set and your regression package (a spreadsheet program if necessary). First perform a multiple regression of testscores on a constant, the student-teacher ratio, and the percent of English learners. Record the coefficients. Next, do the following three step procedure instead: first, regress the testscore on a constant and the percent of English learners. Calculate the residuals and store them under the name resYX2. Second, regress the student-teacher ratio on a constant and the percent of English learners. Calculate the residuals from this regression and store these under the name resX1X2. Finally regress resYX2 on resX1X2 (and a constant, if you wish). Explain intuitively why the simple regression coefficient in the last regression is identical to the regression coefficient on the student-teacher ratio in the multiple regression.

Give at least three examples from macroeconomics and three from microeconomics that involve specified equations in a multiple regression analysis framework. Indicate in each case what the expected signs of the coefficients would be and if theory gives you an indication about the likely size of the coefficients.

The following OLS assumption is most likely violated by omitted variables bias:

The adjusted R², or $\bar { R } ^ { 2 }$ , is given by

The dummy variable trap is an example of

You have collected a sub-sample from the Current Population Survey for the western region of the United States. Running a regression of average hourly earnings (ahe)on an intercept only, you get the following result: $\widehat{\text { ahe }}$ = $\widehat{\beta}$ ₀ = 18.58 a. Interpret the result. b. You decide to include a single explanatory variable without an intercept. The binary variable DFemme takes on a value of "1" for females but is "0" otherwise. The regression result changes as follows: $\widehat{\text { ahe} }$ = $\widehat{\beta}$ ₁×DFemme = 16.50×DFemme What is the interpretation now? c. You generate a new binary variable DMale by subtracting DFemme from 1, and run the new regression: $\widehat{\text { ahe }}$ = $\widehat{\beta}$ ₂×DMale = 20.09×DMale _{What is the interpretation of the coefficient now?} d. After thinking about the above results, you recognize that you could have generated the last two results either by running a regression on both binary variables, or on an intercept and one of the binary variables. What would the results have been?

One of your peers wants to analyze whether or not participating in varsity sports lowers or increases the GPA of students. She decides to collect data from 110 male and female students on their GPA and the number of hours they spend participating in varsity sports. The coefficient in the simple regression function turns out to be significantly negative, using the t-statistic and carrying out the appropriate hypothesis test. Upon reflection, she is concerned that she did not ask the students in her sample whether or not they were female or male. You point out to her that you are more concerned about the effect of omitted variables in her regression, such as the incoming SAT score of the students, and whether or not they are in a major from a high/low grading department. Elaborate on your argument.

Attendance at sports events depends on various factors. Teams typically do not change ticket prices from game to game to attract more spectators to less attractive games. However, there are other marketing tools used, such as fireworks, free hats, etc., for this purpose. You work as a consultant for a sports team, the Los Angeles Dodgers, to help them forecast attendance, so that they can potentially devise strategies for price discrimination. After collecting data over two years for every one of the 162 home games of the 2000 and 2001 season, you run the following regression: $\widehat {\text { Attend }}$ = 15,005 + 201 × Temperat + 465 × DodgNetWin + 82 × OppNetWin + 9647 × DFSaSu + 1328 × Drain + 1609 × D150m + 271 × DDiv - 978 × D2001; $R ^ { 2 }$ =0.416, SER = 6983 where Attend is announced stadium attendance, Temperat it the average temperature on game day, DodgNetWin are the net wins of the Dodgers before the game (wins-losses), OppNetWin is the opposing team's net wins at the end of the previous season, and DFSaSu, Drain, D150m, Ddiv, and D2001 are binary variables, taking a value of 1 if the game was played on a weekend, it rained during that day, the opposing team was within a 150 mile radius, the opposing team plays in the same division as the Dodgers, and the game was played during 2001, respectively. (a)Interpret the regression results. Do the coefficients have the expected signs? (b)Excluding the last four binary variables results in the following regression result: $\widehat {\text { Attend }}$ = 14,838 + 202 × Temperat + 435 × DodgNetWin + 90 × OppNetWin + 10,472 × DFSaSu, $R ^ { 2 }$ =0.410, SER = 6925 According to this regression, what is your forecast of the change in attendance if the temperature increases by 30 degrees? Is it likely that people attend more games if the temperature increases? Is it possible that Temperat picks up the effect of an omitted variable? (c)Assuming that ticket sales depend on prices, what would your policy advice be for the Dodgers to increase attendance? (d)Dodger stadium is large and is not often sold out. The Boston Red Sox play in a much smaller stadium, Fenway Park, which often reaches capacity. If you did the same analysis for the Red Sox, what problems would you foresee in your analysis?

In the multiple regression problem with k explanatory variable, it would be quite tedious to derive the formulas for the slope coefficients without knowledge of linear algebra. The formulas certainly do not resemble the formula for the slope coefficient in the simple linear regression model with a single explanatory variable. However, it can be shown that the following three step procedure results in the same formula for slope coefficient of the first explanatory variable, $X _ { 1 }$ : Step 1: regress Y on a constant and all other explanatory variables other than $X _ { 1 }$ , and calculate the residual (Res1). Step 2: regress $X _ { 1 }$ on a constant and all other explanatory variables, and calculate the residual (Res2). Step 3: regress Res1 on a constant and Res2. Can you give an intuitive explanation to this procedure?

(Requires Calculus)In the multiple regression model you estimate the effect on Y_i of a unit change in one of the X_i while holding all other regressors constant. This

The OLS formula for the slope coefficients in the multiple regression model become increasingly more complicated, using the "sums" expressions, as you add more regressors. For example, in the regression with a single explanatory variable, the formula is $\frac { \sum _ { i = 1 } ^ { n } \left( X _ { i } - \bar{ X } \right) \left( Y _ { i } - \bar { X } \right) } { \sum _ { i = 1 } ^ { n } \left( X _ { i } - \bar { X } \right) ^ { 2 } }$ whereas this formula for the slope of the first explanatory variable is $\hat { \beta } _ { 1 } = \frac { \sum _ { i = 1 } ^ { n } y _ { i } x _ { 1 i } \sum _ { i = 1 } ^ { n } x _ { 2 i } ^ { 2 } - \sum _ { i = 1 } ^ { n } y _ { i } x _ { 2 i } \sum _ { i = 1 } ^ { n } x _ { 1 i } x _ { 2 i } } { \sum _ { i = 1 } ^ { n } x _ { 1 i } ^ { 2 } \sum _ { i = 1 } ^ { n } x _ { 2 i } ^ { 2 } - \left( \sum _ { i = 1 } ^ { n } x _ { 1 i } x _ { 2 i } \right) ^ { 2 } }$ (small letters refer to deviations from means as in $z _ { i } = Z _ { i } - \bar { Z }$ ) in the case of two explanatory variables. Give an intuitive explanations as to why this is the case.

Consider the multiple regression model with two regressors X₁ and X₂, where both variables are determinants of the dependent variable. You first regress Y on X₁ only and find no relationship. However when regressing Y on X₁ and X₂, the slope coefficient $\widehat { \beta 1 }$ changes by a large amount. This suggests that your first regression suffers from

Females, on average, are shorter and weigh less than males. One of your friends, who is a pre-med student, tells you that in addition, females will weigh less for a given height. To test this hypothesis, you collect height and weight of 29 female and 81 male students at your university. A regression of the weight on a constant, height, and a binary variable, which takes a value of one for females and is zero otherwise, yields the following result: $\widehat { \text { Studentw } }$ = -229.21 - 6.36 × Female + 5.58 × Height, $R ^ { 2 }$ =0.50, SER = 20.99 where Studentw is weight measured in pounds and Height is measured in inches. (a)Interpret the results. Does it make sense to have a negative intercept? (b)You decide that in order to give an interpretation to the intercept you should rescale the height variable. One possibility is to subtract 5 ft. or 60 inches from your Height, because the minimum height in your data set is 62 inches. The resulting new intercept is now 105.58. Can you interpret this number now? Do you thing that the regression $R ^ { 2 }$ has changed? What about the standard error of the regression? (c)You have learned that correlation does not imply causation. Although this is true mathematically, does this always apply?

In the multiple regression model Y_i = ?₀ + ?₁X_1i+ ?₂ X_2i + ... + ?_kX_ki + u_i, i = 1,..., n, the OLS estimators are obtained by minimizing the sum of

You have obtained data on test scores and student-teacher ratios in region A and region B of your state. Region B, on average, has lower student-teacher ratios than region A. You decide to run the following regression $Y _ { i } = \mathcal { \beta } _ { 0 } + \mathcal { \beta } _ { 1 } X _ { 1 i } + \mathcal { \beta } _ { 1 } X _ { 2 i } + \beta _ { 3 } X _ { 3 i } + u _ { i }$ where $X _ { 1 }$ is the class size in region A, $X _ { 2 }$ is the difference in class size between region A and B, and $X _ { 3 }$ is the class size in region B. Your regression package shows a message indicating that it cannot estimate the above equation. What is the problem here and how can it be fixed?

It is not hard, but tedious, to derive the OLS formulae for the slope coefficient in the multiple regression case with two explanatory variables. The formula for the first regression slope is $\hat { \beta } _ { 1 } = \frac { \sum _ { i = 1 } ^ { n } y _ { i } x _ { 1 i } \sum _ { i = 1 } ^ { n } x _ { 2 i } ^ { 2 } - \sum _ { i = 1 } ^ { n } y _ { i } x _ { 2 i } \sum _ { i = 1 } ^ { n } x _ { 1 i } x _ { 2 i } } { \sum _ { i = 1 } ^ { n } x _ { 1 i } ^ { 2 } \sum _ { i = 1 } ^ { n } x _ { 2 i } ^ { 2 } - \left( \sum _ { i = 1 } ^ { n } x _ { 1 i } x _ { 2 i } \right) ^ { 2 } }$ (small letters refer to deviations from means as in $z _ { i } = Z _ { i } - \bar { Z }$ ). Show that this formula reduces to the slope coefficient for the linear regression model with one regressor if the sample correlation between the two explanatory variables is zero. Given this result, what can you say about the effect of omitting the second explanatory variable from the regression?

Consider the following multiple regression models (a)to (d)below. DFemme = 1 if the individual is a female, and is zero otherwise; DMale is a binary variable which takes on the value one if the individual is male, and is zero otherwise; DMarried is a binary variable which is unity for married individuals and is zero otherwise, and DSingle is (1-DMarried). Regressing weekly earnings (Earn)on a set of explanatory variables, you will experience perfect multicollinearity in the following cases unless:

In the process of collecting weight and height data from 29 female and 81 male students at your university, you also asked the students for the number of siblings they have. Although it was not quite clear to you initially what you would use that variable for, you construct a new theory that suggests that children who have more siblings come from poorer families and will have to share the food on the table. Although a friend tells you that this theory does not pass the "straight-face" test, you decide to hypothesize that peers with many siblings will weigh less, on average, for a given height. In addition, you believe that the muscle/fat tissue composition of male bodies suggests that females will weigh less, on average, for a given height. To test these theories, you perform the following regression: $\widehat { \text { Studentw } }$ = -229.92 - 6.52 × Female + 0.51 × Sibs+ 5.58 × Height, $R ^ { 2 }$ =0.50, SER = 21.08 where Studentw is in pounds, Height is in inches, Female takes a value of 1 for females and is 0 otherwise, Sibs is the number of siblings. Interpret the regression results.

(Requires Appendix material)Consider the following population regression function model with two explanatory variables: $Y _ { i } = \hat { \beta } _ { 0 } + \hat { \beta } _ { 1 } X _ { 1 i } + \hat { \beta } _ { 2 } X _ { 2 i }$ It is easy but tedious to show that SE( $\hat { \beta } _ { 2 }$ )is given by the following formula: $\sigma_{\hat{\beta_{1}}}^{2}=\frac{1}{n}\left[\frac{1}{1-\rho_{x_{1}, x_{2}}^{2}}\right] \frac{\sigma_{u}^{2}}{\sigma^{2} \mathrm{X}_{1}}$ Sketch how SE( $\hat { \beta } _ { 2 }$ )increases with the correlation between $X _ { 1 i }$ and $X _ { 2 i }$