Chapter 2 · Linear Regression
ISLR Chapter 3 — the familiar model, read for prediction
You have estimated hundreds of regressions. This chapter asks you to look at them differently. The same \(\hat{\beta}\) that econometrics interrogates for significance is here judged by one question only: how far off is \(\hat{y}\) on data the model has not seen? Standard errors do not disappear, but they stop being the point.
Linear regression also earns its keep as a baseline. Any elaborate method you build later must beat it on held-out data, and a surprising number do not. Keeping OLS in the comparison is intellectual honesty, cheaply bought.
What you should take away
- Simple and multiple regression, and why adding a variable can flip a sign
- Interpreting coefficients when regressors are correlated
- Interactions and polynomial terms as controlled non-linearity
- Diagnostics: leverage, outliers, non-constant variance, collinearity
- Qualitative predictors and the reference-category trap
- Why \(R^2\) on the training set tells you almost nothing about prediction
Slides
Labs
- ch03.qmd — the Chapter 3 lab, written up in Quarto
- ch3.r — the same lab as a plain script
- linear-regression.r — a compact walkthrough
- ch3-lab.txt — lab transcript
- gapminder-multiple-regression.r — multiple regression on Gapminder data
- map-function.r — fitting many regressions at once with
purrr::map()
Rendered output: Chapter 3 lab
The Python lab for this chapter uses statsmodels for the inference-flavoured output and scikit-learn for the predictive workflow. Start from the ISLP Chapter 3 notebook and work alongside the R lab above; the model specifications are identical.
Rendered output: PSX analysis
Readings
Data
- advertising.csv — the textbook’s running example
- auto.csv — fuel efficiency and vehicle characteristics
Exercises
Past midterm papers on the linear model, useful as practice under time pressure: