Principal component analysis (PCA) is a linear method for simplifying data with many variables. It creates new directions that capture the greatest possible variation, helping us explore, visualize, and prepare data for further analysis.
A dataset may contain many measured variables, and some may carry overlapping information. For example, height measured in inches and height measured in centimeters are strongly related. Keeping both can add complexity without adding much new information.
All flashcards
What is PCA?
PCA is a linear transformation that creates orthogonal directions ordered by how much variance they capture, often allowing data to be represented with fewer dimensions.
What is a principal component?
A principal component is a unit direction formed from the original features. Its component score is the projection of an observation onto that direction.
How is the first principal component selected?
It is the unit direction that maximizes the variance of the projected observations.
What makes later components distinct from earlier ones?
Each later component is orthogonal to the preceding components and captures as much as possible of the variation remaining after those directions.
What does the ellipsoid analogy explain?
The ellipsoid's axes represent principal directions, and the spread along each axis represents variance. Long axes capture more variation than short axes.
Quiz questions
What is the effect of standardizing every feature to unit variance before PCA?Answer: It gives a correlation-based analysis rather than one based on raw covariances.
For the SVD B=UΣVT of centered data, which vectors give the principal directions?Answer: The columns of V, the right singular vectors.
What does the Eckart–Young result say about a truncated rank-L SVD?Answer: It gives the closest rank-L approximation in Frobenius norm.
How does factor analysis differ from PCA in its main aim?Answer: Factor analysis models shared variance using latent factors, while PCA summarizes total variance.
Which objective does PCA optimize when selecting its leading components?Answer: Capturing the greatest variance in the data.