Exam 18: The Theory of Multiple Regression

The extended least squares assumptions in the multiple regression model include four assumptions from Chapter 6 $\left( u _ { i } \text { has conditional mean zero; } \left( \boldsymbol { X } _ { i } , Y _ { j } \right) , i = 1 , \ldots , n \right.$ are i.i.d. draws from their joint distribution; $\boldsymbol { X } _ { i } \text {and } u _ { i }$ have nonzero finite fourth moments; there is no perfect multicollinearity). In addition, there are two further assumptions, one of which is

The assumption that X has full column rank implies that

Let there be q joint hypothesis to be tested. Then the dimension of r in the expression $R \beta = r$ is

Minimization of $\sum _ { i = 1 } ^ { n } \left( Y _ { i } - b _ { 0 } - b _ { 1 } X _ { 1 i } - \cdots - b _ { k } X _ { k i } \right) ^ { 2 }$ results in

Give several economic examples of how to test various joint linear hypotheses using matrix notation. Include specifications of $R \beta = r$ where you test for (i) all coefficients other than the constant being zero, (ii) a subset of coefficients being zero, and (iii) equality of coefficients. Talk about the possible distributions involved in finding critical values for your hypotheses.

For the OLS estimator $\hat { \boldsymbol { \beta } } = \left( \boldsymbol { X } ^ { \prime } \boldsymbol { X } \right) ^ { - 1 } \boldsymbol { X } ^ { \prime } \boldsymbol { Y }$ to exist, $X ^ { \prime } X ^ { \prime }$ must be invertible. This is the case when $X$ has full rank. What is the rank of a matrix? What is the rank of the product of two matrices? Is it possible that $X$ could have rank n ? What would be the rank of $X ^ { \prime } X$ in the case $n < ( k + 1 ) \text { ? }$ Explain intuitively why the OLS estimator does not exist in that situation.

The heteroskedasticity-robust estimator of $\sum _ { \sqrt { n } ( \hat { \beta } - \beta ) }$ is obtained

The multiple regression model in matrix form $\boldsymbol { Y } = \boldsymbol { X } \boldsymbol { \beta } + \boldsymbol { U }$ can also be written as

The GLS estimator

The leading example of sampling schemes in econometrics that do not result in independent observations is

The Gauss-Markov theorem for multiple regression proves that a. $\boldsymbol { M } _ { \boldsymbol { X } }$ is an idempotent matrix. b. the OLS estimator is BLUE. c. the OLS residuals and predicted values are orthogonal. d. the variance-covariance matrix of the OLS estimator is $\sigma _ { u } ^ { 2 } ( \boldsymbol { X } \boldsymbol { X } ) ^ { - 1 }$ .

Define the GLS estimator and discuss its properties when $\Omega$ is known. Why is this estimator sometimes called infeasible GLS? What happens when $\Omega$ is unknown? What would the $\Omega$ matrix look like for the case of independent sampling with heteroskedastic errors, where $\operatorname { var } \left( u _ { i } \mid X _ { i } \right) = \operatorname { ch } \left( X _ { i } \right) = \sigma ^ { 2 } X _ { 1 i } ^ { 2 } ?$ Since the inverse of the error variancecovariance matrix is needed to compute the GLS estimator, find $\Omega ^ { - 1 }$ . The textbook shows that the original model $\mathbf { Y } = \mathbf { X } \beta + \mathbf { U }$ will be transformed into $\tilde { \boldsymbol { Y } } = \tilde { \boldsymbol { X } } \boldsymbol { \beta } + \tilde { \boldsymbol { U } }$ where $\tilde { \boldsymbol { Y } } = \boldsymbol { F } \boldsymbol { Y } , \tilde { \boldsymbol { X } } = \boldsymbol { F } \boldsymbol { X } \text {, and } \tilde { \boldsymbol { U } } = \boldsymbol { F } \boldsymbol { U } \text {, and } \boldsymbol { \boldsymbol { F } ^ { \prime } \boldsymbol { F } } = \boldsymbol { \Omega } ^ { -1 }$ Find $F$ in the above case, and describe what effect the transformation has on the original data.

The linear multiple regression model can be represented in matrix notation as $\boldsymbol { Y } = \boldsymbol { X } \boldsymbol { \beta } + \boldsymbol { U }$ where X is of order n x(k+1) . k represents the number of

Prove that under the extended least squares assumptions the OLS estimator $\hat { \boldsymbol { \beta } }$ is unbiased and that its variance-covariance matrix is $\sigma _ { u } ^ { 2 } \left( \boldsymbol { X } ^ { \prime } \boldsymbol { X } \right) ^ { - 1 }$

The GLS assumptions include all of the following, with the exception of a. the $\boldsymbol { X } _ { i }$ are fixed in repeated samples. b. $X _ { i }$ and $u _ { i }$ have nonzero finite fourth moments. c. $E \left( \boldsymbol { U } \boldsymbol { U } ^ { \prime } \mid \boldsymbol { X } \right) = \boldsymbol { \Omega } ( \boldsymbol { X } )$ , where $\boldsymbol { \Omega } ( \boldsymbol { X } )$ is $n \times n$ matrix-valued that can depend on $\boldsymbol { X }$ . d. $E ( \boldsymbol { U } \mid \boldsymbol { X } ) = \mathbf { 0 } _ { n }$ .

The formulation $\boldsymbol { R } \boldsymbol { \beta } = \boldsymbol { r }$ to test a hypotheses

Your textbook derives the OLS estimator as $\hat { \boldsymbol { \beta } } = \left( \boldsymbol { X } ^ { \prime } \boldsymbol { X } \right) ^ { - 1 } \boldsymbol { X } ^ { \prime } \boldsymbol { Y }$ Show that the estimator does not exist if there are fewer observations than the number of explanatory variables, including the constant.What is the rank of X′X in this case?