This lesson introduces the bias–variance tradeoff and explains why a model that fits its training data well may still perform poorly on new data.
Supervised learning uses a training dataset to estimate a relationship between input features and target values. A useful model should capture real patterns in the data and make accurate predictions for previously unseen examples. Performance on new examples is called generalization. A model that merely memorizes its training examples may have low training error but poor generalization.
All flashcards
What is bias?
Bias is systematic prediction error caused by assumptions or restrictions in a learning method. High bias can prevent a model from capturing important patterns.
What is variance?
Variance describes how much a fitted model changes across different training samples. High variance means the model is sensitive to small fluctuations in the data.
What is underfitting?
Underfitting occurs when a model is too restricted to capture important patterns, often producing high bias and poor performance even on training data.
What is overfitting?
Overfitting occurs when a model learns training-specific details, including noise, and consequently performs poorly on unseen data. It is commonly associated with high variance.
What is the bias–variance tradeoff?
It is the challenge of balancing systematic error and sensitivity to training data so that a model generalizes well to new examples.
Quiz questions
For squared-error prediction at a fixed input, what are the components of expected error under the stated noisy-data assumptions?Answer: Squared bias, prediction variance, and irreducible noise variance.
In k-nearest neighbors regression, what is the usual effect of increasing k?Answer: Predictions become smoother, usually with higher bias and lower variance.
What is cross-validation commonly used for when choosing a model?Answer: Estimating held-out performance to compare models or tune settings.
What is a common effect of bagging when it combines predictions from resampled training sets?Answer: It can reduce variability among predictions from individual fits.
Why might a Monte Carlo method discard early samples during burn-in?Answer: Discarding early samples can help control bias from the initial state.