Chapter 7 · Unsupervised Learning

ISLR Chapter 12 — structure without a target

Everything so far had an answer key. Here there is none. Unsupervised learning looks for structure in \(X\) alone: directions along which the data vary most, or groups of observations that resemble each other. The methods are simple; the difficulty is that there is no test set to tell you whether you found something real.

Principal component analysis rotates the predictor space so that the first few directions capture most of the variance, which makes it both a visualisation tool and a pre-processing step. Clustering — \(k\)-means for a flat partition, hierarchical for a nested one — assigns observations to groups. Both will always return an answer. Deciding whether that answer means anything is a judgement call, and this chapter is largely about learning to make it carefully.

What you should take away

  • Principal components as directions of maximum variance, and the loading vectors
  • The proportion of variance explained, scree plots, and why the elbow is a convention
  • Why scaling changes the components, and when not scaling is defensible
  • \(k\)-means: the algorithm, its dependence on starting values, and choosing \(k\)
  • Hierarchical clustering: linkage choices and how to read a dendrogram
  • The honest problem: no cross-validation, so no external check on the answer

Labs

PCA runs through prcomp(), clustering through kmeans() and hclust(). Work with the USArrests data from the textbook, then repeat the exercise on the World Happiness panel in this repository — it is more interesting precisely because you have priors about what the clusters should look like, and can check whether they hold.

sklearn.decomposition.PCA, sklearn.cluster.KMeans and scipy.cluster.hierarchy.dendrogram cover the same ground. Remember StandardScaler before PCA unless you have a reason not to.

Readings

Data

TipCluster, then argue

Run \(k\)-means on the World Happiness indicators with \(k = 3\), \(4\) and \(5\). For each, write one sentence naming what the clusters appear to represent. If you cannot name them, that is a finding too, and worth reporting rather than hiding.

Back to top