Chapter 8 · Deep Learning & Machine Learning in Economics

ISLR Chapter 10, and the argument about what prediction is for

Two things belong at the end of this course. The first is deep learning, which is what happens when you stack enough flexible layers that the model learns its own features rather than being handed them. The second is the harder question the whole semester has been circling: what is any of this for, in a discipline built on identifying causal effects?

Neural networks are not magic and not new. A single hidden layer with enough units can approximate any well-behaved function, which sounds impressive until you remember that so can a polynomial of high enough degree. What changed was data volume, hardware, and the optimisation tricks that make deep stacks trainable. For most tabular economic data, boosted trees still win. Where deep learning earns its keep is text, images, and sequences.

The second question has a cleaner answer than the noise around it suggests. Prediction and causal inference solve different problems. Some policy problems are genuinely prediction problems — who to screen, where to send an inspector, which claims to audit. Others need a counterfactual, and no amount of held-out accuracy provides one. Knowing which problem you have is the skill.

What you should take away

Single and multi-layer networks, activation functions, and what depth buys
Backpropagation and stochastic gradient descent, at working-knowledge level
Regularisation for networks: dropout, weight decay, early stopping
Convolutional and recurrent architectures, and the data types that call for them
When boosted trees beat deep learning on tabular data, which is most of the time
The prediction–causation distinction, and hybrid methods that use both

Slides

Rendered output: ML, Economics and AI lecture

Readings

Beyond the repository, two short pieces are worth the hour they take:

Varian, H. (2014), “Big Data: New Tricks for Econometrics”, Journal of Economic Perspectives
Mullainathan, S. and Spiess, J. (2017), “Machine Learning: An Applied Econometric Approach”, JEP

Where the course leaves you

You can now build a predictive model, tune it honestly, and say how well it will do on data it has not seen. That is a real skill and a scarce one. What it does not give you is permission to answer causal questions with predictive tools, and the most useful thing you can carry out of this room is the discipline to say which kind of question you are facing before you open the laptop.