Ordinary Least Squares: Estimation, Assumptions, and Diagnostics

All flashcards

What is a regressor?

A predictor or explanatory variable included in a regression model to help explain or predict the response.

What is the error term?

The part of the response not captured by the model's included regressors; it represents unobserved influences.

What is the design matrix?

The matrix $X$ containing the regressor values for all observations. With $n$ observations and $p$ coefficients, it has dimensions $n\times p$.

How is an intercept represented in a design matrix?

By including a column of ones. The coefficient on that column shifts the fitted relationship vertically.

What does full column rank ensure?

It ensures that no column is an exact linear combination of the others, allowing the regression coefficients to be uniquely identified by standard OLS.

Quiz questions

1. What quantity does ordinary least squares minimize when estimating regression coefficients?Answer: The sum of squared differences between observed and fitted responses.
2. When $X$ has full column rank, which expression gives the OLS coefficient estimate?Answer: $(X^TX)^{-1}X^Ty$
3. Under the Gauss–Markov conditions, what does BLUE mean for the OLS estimator?Answer: It has the smallest variance among linear unbiased estimators.
4. In a regression with an intercept, what does $R^2=1-RSS/TSS$ summarize?Answer: The fraction of centered sample variation accounted for by the fitted model.
5. What can a fan-shaped spread in a residual plot suggest?Answer: The error variance may change across predictor values.