Chapter 1 · Statistical Learning

ISLR Chapter 2 — what we are estimating, and what we are paying for it

We begin with a distinction that quietly reorganises everything you learned in econometrics. Write the world as \(Y = f(X) + \varepsilon\). Econometrics usually wants \(f\) itself, or one slope inside it. Statistical learning often wants \(\hat{f}\) only because it produces good \(\hat{Y}\), and is willing to accept an \(\hat{f}\) nobody can interpret if the predictions are better.

That willingness has a price, and the price has a name: the bias–variance trade-off. Expected test error at a point decomposes into squared bias, variance, and irreducible noise. Bias falls as the model grows more flexible; variance rises. The minimum sits somewhere in between, and finding it is most of what the rest of the course is about.

What you should take away

• Why training error is a systematically optimistic estimate of test error
• The bias–variance decomposition, and why the U-shaped test error curve appears
• Parametric versus non-parametric approaches, and what each assumes
• Supervised versus unsupervised problems
• Why the Bayes error rate is a floor no classifier can pass

Slides

• Statistical Learning (PPTX)
• Assessing Model Accuracy (PPTX)

Labs

R R

The introductory lab walks through the language basics, then the Auto and Advertising data, then a bias–variance simulation you can run yourself.

• ch01.r — first session in R: vectors, matrices, plotting
• ch02.r — the Chapter 2 lab
• ch2.r — a shorter working version of the same lab
• bias-variance.r — simulating the trade-off directly
• ch02-10.qmd — applied exercise 10 (Boston housing), written up
• ch2-8.r · ch2-9.r · ch2-10.r — applied exercises 8, 9, 10
• ch2-stat-learning.rmd — R Markdown write-up
• stat-learn-lab.txt · statlearn-lab-01.txt — lab transcripts

Rendered output: Chapter 2 lab · Exercise 10 write-up · Chapter 1 notes

Python Python

• chapter-2.ipynb — the full Chapter 2 lab as a Jupyter notebook

Rendered output: Chapter 2 notebook

Readings

• Introduction (PDF)
• Statistical Learning (PDF)
• Statistical Learning — Chapter 2 handout (PDF)
• Chapter 2 slides handout (PDF)

Exercises

• Applied exercises, Chapter 2 (TXT)
• Worked answers (TXT)
• Chapter 2 multiple-choice bank (XLSX)

TipA question worth sitting with

If a more flexible method always fits the training data better, why does anyone ever choose a less flexible one? Answer it in your own words before the next lecture; the whole course is downstream of that answer.