Introduction to Machine Learning and AI#
This chapter sets the scene for the section on machine learning and AI.
Machine learning or AI? Like many, ahem, older practioners in this field, we think AI is a niche subset of machine learning and that most stories that are about “AI” are actually not about anything we would recognise as “intelligence” at all, but are really about pattern recognition and therefore are more about machines learning to see patterns and repeat them. So we’ll use machine learning throughout, but we know we’re fighting a losing battle on AI being the dominant term. On the flip side, you’ll get some snobby, usually older economists claiming that all machine learning is just logistic regression. They’re wrong too, and that line of criticism seems to have gone fascinatingly silent following the release of large language models like ChatGPT.
The truth is that machine learning, or AI, or whatever you want to call it is a tool like any other. A very powerful tool that can be enormously helpful in analysis and research, but a tool with pros and cons. It’s probably the most exciting methodological frontier in economics right now.
What we’ll cover#
First, you should know that machine learning and AI (artificial intelligence) are huge fields in themselves and we can’t possibly hope to cover everything here! In particular we won’t cover the theory behind the tools or go into detail on when they’re appropriate to use, or talk about loss functions. Nor will we get to the cutting-edge in machine learning, for example transformers and large language models.
What we will cover is examples of where to start in coding up machine learning models—the most standard algorithms via the most common packages. We’ll also see useful examples of machine learning code that covers the broad swathe of tasks it can perform. In particular, we’ll see examples of supervised and unsupervised machine learning; and how to prep data for machine learning. However, it is going to be about giving you a thread to pull on to find out more and get into the specifics of what you might need. There’s much more we won’t cover, such as “MLops”, interpretable machine learning, and theory.
We’ll mainly be using the scikit-learn package to do our machine learning because it has all of the standard algorithms and is a very high quality library with tons of support, and you can easily run it on a regular laptop. Power users of machine learning will likely want to branch out into tools like Pytorch, especially for deep learning.
We’ll also be pointing to lots of potentially useful further links where you can take your machine learning and AI to the next level.
When machine learning is useful in economics#
We said we’d try and avoid any theory, but there are some ideas that are hard to escape. We just want to spend a few moments thinking aloud about why and when machine learning is useful in economics. In the below, we assume that there is a function \(f\) that maps inputs \(\mathbf{x}\) into \(y\) that we estimate as
So, very broadly, these are the times when you might reach for machine learning in economics:
when you care more about prediction, ie \(\hat{y}\), than coefficients, \(\hat{\beta}\). In these cases, machine learning can outperform vanilla OLS in a wide range of empirical cases.
when you want a model for \(\hat{y}\) and there are non-linearities such that an OLS solution of a linear regression equation is highly likely to be misspecified. Some non-linearities, eg a quadratic dependence on age, may be obvious—but there are many that are not.
when \((\mathbf{x})\) is high-dimensional and it is not always possible to use OLS, or OLS ceases to become the most performant model.
for causal inference in certain situations, such as when there are a large number of ‘nuisance’ variables
when you wish to create data from an input source, eg digitising images of text, creating categorical groupings boiled down from a large number of other dimensions, web-scraping
We’ll see some of these, albeit briefly, in subsequent chapters!
Links to further courses#
If you want general courses on machine learning that go deeper, you might find the following links useful. Note that, for many applications in economics, you may not need deep learning and the other links are probably better places to start.