Exam 6: Linear Regression With Multiple Regressors

Imagine you regressed earnings of individuals on a constant, a binary variable ("Male") which takes on the value 1 for males and is 0 otherwise, and another binary variable ("Female") which takes on the value 1 for females and is 0 otherwise. Because females typically earn less than males, you would expect

Females, on average, are shorter and weigh less than males.One of your friends, who is a pre-med student, tells you that in addition, females will weigh less for a given height.To test this hypothesis, you collect height and weight of 29 female and 81 male students at your university.A regression of the weight on a constant, height, and a binary variable, which takes a value of one for females and is zero otherwise, yields the following result: $\widehat { \text { Studentw } } = - 229.21 - 6.36 \times \text { Female } + 5.58 \times \text { Height } , R ^ { 2 } = 0.50 , \text { SER } = 20.99$ where Studentw is weight measured in pounds and Height is measured in inches. (a)Interpret the results.Does it make sense to have a negative intercept?

(Requires Calculus)For the case of the multiple regression problem with two explanatory variables, derive the OLS estimator for the intercept and the two slopes.

In the multiple regression model, the least squares estimator is derived by

In the multiple regression model with two explanatory variables $Y _ { i } = \beta _ { 0 } + \beta _ { 1 } X _ { 1 i } + \beta _ { 2 } X _ { 2 i } + u _ { i }$ the OLS estimators for the three parameters are as follows (small letters refer to deviations from means as in $z _ { i } = Z _ { i } - \bar { Z }$ ): =-- = = You have collected data for 104 countries of the world from the Penn World Tables and want to estimate the effect of the population growth rate $\left( X _ { 1 i } \right)$ and the saving rate $\left( X _ { 2 i } \right)$ (average investment share of GDP from 1980 to 1990) on GDP per worker (relative to the U.S.) in 1990. The various sums needed to calculate the OLS estimates are given below: =33.33;=2.025;=17.313 =8.3103;=.0122;=0.6422 $\sum _ { i = 1 } ^ { n } y _ { i } x _ { 1 i } = - 0.2304 ; \sum _ { i = 1 } ^ { n } y _ { i } x _ { 2 i } = 1.5676 ; \sum _ { i = 1 } ^ { n } x _ { 1 i } x _ { 2 i } = - 0.0520$ (a)What are your expected signs for the regression coefficient? Calculate the coefficients and see if their signs correspond to your intuition.

One of your peers wants to analyze whether or not participating in varsity sports lowers or increases the GPA of students.She decides to collect data from 110 male and female students on their GPA and the number of hours they spend participating in varsity sports. The coefficient in the simple regression function turns out to be significantly negative, using the t-statistic and carrying out the appropriate hypothesis test.Upon reflection, she is concerned that she did not ask the students in her sample whether or not they were female or male.You point out to her that you are more concerned about the effect of omitted variables in her regression, such as the incoming SAT score of the students, and whether or not they are in a major from a high/low grading department.Elaborate on your argument.

In the process of collecting weight and height data from 29 female and 81 male students at your university, you also asked the students for the number of siblings they have. Although it was not quite clear to you initially what you would use that variable for, you construct a new theory that suggests that children who have more siblings come from poorer families and will have to share the food on the table.Although a friend tells you that this theory does not pass the "straight-face" test, you decide to hypothesize that peers with many siblings will weigh less, on average, for a given height.In addition, you believe that the muscle/fat tissue composition of male bodies suggests that females will weigh less, on average, for a given height.To test these theories, you perform the following regression: $\widehat { \text { Studentw } } = - 229.92 - 6.52 \times$ Female $+ 0.51 \times$ Sibs $+ 5.58 \times$ Height, $R ^ { 2 } = 0.50 , S E R = 21.08$ where Studentw is in pounds, Height is in inches, Female takes a value of 1 for females and is 0 otherwise, Sibs is the number of siblings. Interpret the regression results.

The following OLS assumption is most likely violated by omitted variables bias: a. $E \left( u _ { i } \mid X _ { i } \right) = 0$ . b. $\left( X _ { i } , Y _ { i } \right) , i = 1 , \ldots , n$ are i.i.d draws from their joint distribution. c. there are no outliers for $X _ { i } , u _ { i }$ . d. there is heteroskedasticity.

The OLS residuals in the multiple regression model

(Requires Calculus)For the case of the multiple regression problem with two explanatory variables, show that minimizing the sum of squared residuals results in three conditions: $\sum _ { i = 1 } ^ { n } \widehat { u _ { i } } = 0 ; \sum _ { i = 1 } ^ { n } \widehat { u _ { i } } X _ { 1 i } = 0 ; \sum _ { i = 1 } ^ { n } \widehat { u _ { i } } X _ { 2 i } = 0$

$\text { In a multiple regression framework, the slope coefficient on the regressor } X _ { 2 i }$ a. takes into account the scale of the error term. b. is measured in the units of $Y _ { i }$ divided by units of $X _ { 2 i }$ . c. is usually positive. d. is larger than the coefficient on $X _ { 1 i }$ .

In the multiple regression model, the adjusted $R ^ { 2 } , \bar { R } ^ { 2 }$

You try to establish that there is a positive relationship between the use of a fertilizer and the growth of a certain plant.Set up the design of an experiment to establish the relationship, paying particular attention to relevant control variables.Discuss in this context the effect of omitted variable bias.

In the multiple regression with two explanatory variables, show that the TSS can still be decomposed into the ESS and the RSS.

When you have an omitted variable problem, the assumption that $E \left( u _ { i } \mid X _ { i } \right) = 0$ is violated. This implies that

In the multiple regression problem with $k$ explanatory variable, it would be quite tedious to derive the formulas for the slope coefficients without knowledge of linear algebra. The formulas certainly do not resemble the formula for the slope coefficient in the simple linear regression model with a single explanatory variable. However, it can be shown that the following three step procedure results in the same formula for slope coefficient of the first explanatory variable, $X _ { 1 }$ : Step 1: regress $Y$ on a constant and all other explanatory variables other than $X _ { 1 }$ , and calculate the residual (Res1). Step 2: regress $X _ { 1 }$ on a constant and all other explanatory variables, and calculate the residual (Res2). Step 3: regress Res1 on a constant and Res2. Can you give an intuitive explanation to this procedure?

The sample regression line estimated by OLS

When there are omitted variables in the regression, which are determinants of the dependent variable, then

You have obtained data on test scores and student-teacher ratios in region A and region B of your state.Region B, on average, has lower student-teacher ratios than region A.You decide to run the following regression $Y _ { i } = \beta _ { 0 } + \beta _ { 1 } X _ { 1 i } + \beta _ { 2 } X _ { 2 i } + \beta _ { 3 } X _ { 3 i } + u _ { i }$ where X1 is the class size in region A, X2 is the difference in class size between region A and B, and X3 is the class size in region B.Your regression package shows a message indicating that it cannot estimate the above equation.What is the problem here and how can it be fixed?

The probability limit of the OLS estimator in the case of omitted variables is given in your text by the following formula: $\hat { \beta } _ { 1 } \stackrel { p } { \rightarrow } \beta _ { 1 } + \rho _ { X u } \frac { \sigma _ { u } } { \sigma _ { X } }$ Give an intuitive explanation for two conditions under which the bias will be small.