202412121103
Status: #idea
Tags: Unsupervised Learning, Linear Algebra, Singular Value Decomposition
State: #nascient
Principal Components Analysis (PCA)
Data exists in a
But not all dimensions are made equal.
PCA seeks to find the dimensions across which variability is biggest.
The first component represents the highest variability (and the size of it represents how much variability), and the loading vector represents the direction of maximum variability.
The principal components are sorted in decreasing order such that that the last principal component
The principal components are orthogonal
Pros
- Useful for plotting high dimensional data.
- If our task align with those dimensions of high variability, a really nice technique
- Based on strong linear algebra foundations.
- Can be used for imputation as well
Cons
- Not really interpretable