Linear Regression: Least Squares, Residuals, and Inference Assumptions

All flashcards

What is a linear regression model?

A model that uses a straight line, commonly written $\hat{y}=a+bx$, to predict a dependent variable $y$ from an independent variable $x$.

What do the intercept and slope represent?

The intercept $a$ is the predicted value of $y$ when $x=0$. The slope $b$ is the predicted change in $y$ for each one-unit increase in $x$.

How is a residual calculated?

Subtract the predicted value from the observed value: $e=y-\hat{y}$. A positive residual means the observation is above the prediction.

What is the method of least squares?

A procedure for finding the regression line that minimizes the sum of the squared residuals across the observed data points.

Why should predictions usually stay within the observed range of $x$?

Predictions outside that range are extrapolations. The relationship observed in the sample may not continue beyond the data, so such predictions can be unreliable.

Quiz questions

1. In a linear regression significance test, what does the normality assumption concern?Answer: For each value of $x$, population values of $y$ are normally distributed around the line.
2. What does the independence assumption require of regression errors?Answer: One observation’s error should not systematically predict another observation’s error.
3. Which data collection approach supports generalizing regression conclusions?Answer: A well-designed random sample or randomized experiment.
4. What does testing the significance of a correlation coefficient help assess?Answer: Whether sample data provide evidence of a linear relationship in the population.
5. What can a systematic pattern in residuals suggest?Answer: Errors may be dependent, or the model may have missed a feature of the data.