Linear Regression: Coefficients, Least Squares, and Extensions

All flashcards

What is linear regression?

A statistical model that estimates the relationship between a scalar response and one or more explanatory variables using a predictor function that is linear in its unknown parameters.

What is the difference between a response and a predictor?

The response is the outcome being modeled or predicted. A predictor is an explanatory variable used by the model to estimate that outcome.

How do simple and multiple linear regression differ?

Simple linear regression uses exactly one explanatory variable. Multiple linear regression uses at least two explanatory variables.

What does conditional mean mean in regression?

It is the expected value of the response for given predictor values, written $E(Y\mid X)$. Standard linear regression often models this quantity as an affine function of the predictors.

How can linear regression be used in supervised learning?

It learns a mapping from labeled training examples—predictors paired with known responses—and uses the fitted mapping to predict responses for new examples.

Quiz questions

1. What is the response variable in a regression model?Answer: The outcome whose value the model seeks to describe or predict.
2. What distinguishes simple linear regression from multiple linear regression?Answer: Simple regression has exactly one explanatory variable, while multiple regression has two or more.
3. What does a regression model most commonly represent as a function of predictors?Answer: The conditional mean of the response given the predictor values.
4. A model is linear regression primarily because it is linear in which quantities?Answer: Its unknown model parameters or coefficients.
5. A trained regression model is given predictor values for new cases but no responses. Which use is being made of it?Answer: Prediction, by estimating responses for new predictor values.