Exam 17: The Theory of Linear Regression With One Regressor

What does the Gauss-Markov theorem prove? Without giving mathematical details, explain how the proof proceeds.What is its importance?

The following is not part of the extended least squares assumptions for regression with a single regressor: a. $\operatorname { var } \left( u _ { i } \mid X _ { i } \right) = \sigma _ { u } ^ { 2 }$ . b. $E \left( u _ { i } \mid X _ { i } \right) = 0$ . c. the conditional distribution of $u _ { i }$ given $X _ { i }$ is normal. d. $\quad \operatorname { var } \left( u _ { i } \mid X _ { i } \right) = \sigma _ { u , i } ^ { 2 }$ .

Consider the model $Y _ { i } = \beta _ { 1 } X _ { i } + u _ { i }$ , where the $X _ { i }$ and the $u _ { i }$ are mutually independent i.i.d. random variables with finite fourth moment and $E \left( u _ { i } \right) = 0$ . Let $\widehat { \beta } _ { 1 }$ denote the OLS estimator of $\beta _ { 1 }$ . Show that $\sqrt { n } \left( \widehat { \beta } _ { 1 } - \beta _ { 1 } \right) = \frac { \frac { \sum _ { i = 1 } ^ { n } X _ { i } u _ { i } } { \sqrt { n } } } { \frac { \sum _ { i = 1 } ^ { n } X _ { i } ^ { 2 } } { n } } .$

All of the following are good reasons for an applied econometrician to learn some econometric theory, with the exception of a. turning your statistical software from a "black box" into a flexible toolkit from which you are able to select the right tool for a given job. b. understanding econometric theory lets you appreciate why these tools work and what assumptions are required for each tool to work properly. c. learning how to invert a $4 \times 4$ matrix by hand. d. helping you recognize when a tool will not work well in an application and when it is time for you to look for a different econometric approach.

The extended least squares assumptions are of interest, because

Consider the model $Y _ { i } = \beta _ { 1 } X _ { i } + u _ { i }$ , where $u _ { i } = c X _ { i } ^ { 2 } e _ { i }$ and all of the $X$ 's and $e$ 's are i.i.d. and distributed $N ( 0,1 )$ . (a)Which of the Extended Least Squares Assumptions are satisfied here? Prove your assertions.

In practice, the most difficult aspect of feasible WLS estimation is

The Gauss-Markov Theorem proves that a. the OLS estimator is $t$ distributed. b. the OLS estimator has the smallest mean square error. c. the OLS estimator is unbiased. d. with homoskedastic errors, the OLS estimator has the smallest variance in the class of linear and unbiased estimators, conditional on $X _ { 1 } , \ldots , X _ { n }$ .

Finite-sample distributions of the OLS estimator and t-statistics are complicated, unless a. the regressors are all normally distributed. b. the regression errors are homoskedastic and normally distributed, conditional on $X _ { 1 } , \ldots , X _ { n }$ c. the Gauss-Markov Theorem applies. d. the regressor is also endogenous.

(Requires Appendix material) Your textbook considers various distributions such as the standard normal, $t, \chi^{2}$ , and $F$ distribution, and relationships between them. (a) $\text { Using statistical tables, give examples that the following relationship holds: } F _ { n _ { 1 } , \infty } = \frac { \chi _ { n _ { 1 } } ^ { 2 } } { n _ { 1 } } \text {. }$

An implication of $\sqrt { n } \left( \hat { \beta } _ { 1 } - \beta _ { 1 } \right) \stackrel { d } { \rightarrow } N \left( 0 , \frac { \operatorname { var } \left( v _ { i } \right) } { \left[ \operatorname { var } \left( X _ { i } \right) \right] ^ { 2 } } \right)$ is that

Consider the simple regression model $Y _ { i } = \beta _ { 0 } + \beta _ { 1 } X _ { i } + u _ { i }$ where $X _ { \mathrm { i } } > 0$ for all $i$ , and the conditional variance is $\operatorname { var } \left( u _ { i } \mid X _ { i } \right) = \theta X _ { i } ^ { 2 }$ where $\theta$ is a known constant with $\theta > 0$ . (a) Write the weighted regression as $\tilde { Y } _ { i } = \beta _ { 0 } \tilde { X } _ { 0 i } + \beta _ { 1 } \tilde { X } _ { 1 i } + \tilde { u } _ { i }$ . How would you construct $\tilde { Y } _ { i }$ , $\tilde { X } _ { 0 i }$ and $\tilde { X } _ { 1 i } ?$

Besides the Central Limit Theorem, the other cornerstone of asymptotic distribution theory is the

(Requires Appendix Material)This question requires you to work with Chebychev's Inequality. (a)State Chebychev's Inequality.

If the errors are heteroskedastic, then

One of the earlier textbooks in econometrics, first published in 1971, compared "estimation of a parameter to shooting at a target with a rifle.The bull's-eye can be taken to represent the true value of the parameter, the rifle the estimator, and each shot a particular estimate." Use this analogy to discuss small and large sample properties of estimators.How do you think the author approached the n → ∞ condition? (Dependent on your view of the world, feel free to substitute guns with bow and arrow, or missile.)

The following is not one of the Gauss-Markov conditions a. $\operatorname { var } \left( u _ { i } \mid X _ { 1 } , \ldots , X _ { n } \right) = \sigma _ { u } ^ { 2 } , 0 < \sigma _ { u } ^ { 2 } < \infty$ for $i = 1 , \ldots , n$ , b. the errors are normally distributed. c. $E \left( u _ { i } u _ { j } \mid X _ { 1 } , \ldots , X _ { n } \right) = 0 , i = 1 , \ldots , n , j = 1 , \ldots , n , i \neq j$ . d. $E \left( u _ { i } \mid X _ { 1 } , \ldots , X _ { n } \right) = 0$ .

(Requires Appendix material) If the Gauss-Markov conditions hold, then OLS is BLUE. In addition, assume here that $X$ is nonrandom. Your textbook proves the Gauss-Markov theorem by using the simple regression model $Y _ { i } = \beta _ { 0 } + \beta _ { 1 } X _ { i } + u _ { i }$ and assuming a linear estimator $\tilde { \beta } _ { 1 } = \sum _ { i = 1 } ^ { n } a _ { i } Y _ { i }$ . Substitution of the simple regression model into this expression then results in two conditions for the unbiasedness of the estimator: $\sum _ { i = 1 } ^ { n } a _ { i } = 0 \text { and } \sum _ { i = 1 } ^ { n } a _ { i } X _ { i } = 1 .$ The variance of the estimator is $\operatorname { var } \left( \tilde { \beta } _ { 1 } \mid X _ { 1 } , \ldots , X _ { n } \right) = \sigma _ { u } ^ { 2 } \sum _ { i = 1 } ^ { n } a _ { i } ^ { 2 }$ . Different from your textbook, use the Lagrangian method to minimize the variance subject to the two constraints. Show that the resulting weights correspond to the OLS weights. .

The link between the variance of $\bar { Y }$ and the probability that $\bar { Y }$ is within $\pm \delta \text { of } \mu _ { Y }$ is provided by