How well can we predict the future? In this podcast, Ariel speaks with Anthony Aguirre and Andrew Critch about the art of predicting the future, what constitutes a good prediction, and how we can better predict the advancement of artificial intelligence. They also touch on the difference between predicting a solar eclipse and predicting the weather, what it takes to make money on the stock market, and the bystander effect regarding existential risks.
Anthony is a professor of physics at the University of California at Santa Cruz. He’s one of the founders of the Future of Life Institute, of the Foundational Questions Institute, and most recently of metaculus.com, which is an online effort to crowdsource predictions about the future of science and technology. Andrew is on a two-year leave of absence from MIRI to work with UC Berkeley’s Center for Human Compatible AI. He cofounded the Center for Applied Rationality, and previously worked as an algorithmic stock trader at Jane Street Capital.
The following interview has been heavily edited for brevity, but you can listen to it in its entirety above or read the full transcript here.
Ariel: To start, what are predictions? What are the hallmarks of a good prediction? How does that differ from just guessing?
Anthony: I would say there are four aspects to a good prediction. One, it should be specific, well-defined and unambiguous. If you predict something’s going to happen, everyone should agree on whether that thing has happened or not. This can be surprisingly difficult to do.
Second, it should be probabilistic. A really good prediction is a probability for something happening.
Third, a prediction should be precise. If you give everything a 50% chance, you’ll never be terribly wrong, but you’ll also never be terribly right. Predictions are really interesting to the extent that they say something is either very likely or very unlikely. Precision is what we would aim for.
Fourth, you want to be well-calibrated. If there are 100 things that you predict with 90% confidence, around 90% of those things should come true.
The precision and the calibration kind of play off against each other, but it’s very difficult to be both about the future.
Andrew: Of the properties Anthony said, being specific, meaning it’s clear what the prediction is saying and when it will be settled — I think people really don’t appreciate how psychologically valuable that is.
People really undervalue the extent to which the specificity property of prediction is also part of your own training as a predictor. The last property that Anthony said, being calibration, is not just a property of a prediction. It’s a property of a predictor.
A good predictor is somebody who strives for calibration while also trying to be precise and get their probabilities as close to zero and one as they can.
Ariel: What is the difference between prediction versus just guessing or intuition? For example, knowing that the eclipse will happen in August versus not knowing what the weather will be like yet.
Andrew: The problem is that weather data is very unpredictable, and the locations of planets and moons and stars are predictable. I would say that it’s lack of a reliable model for making the prediction or a reliable method.
Anthony: There is an incredibly accurate prediction of the eclipse this coming August, but there is some tiny bit of uncertainty that you don’t see because we know so precisely where the planets are.
When you look at weather, there’s lots of uncertainty because we don’t have some measurement device at every position measuring every temperature and density of the atmosphere and the water at every point on earth. There’s uncertainty in the initial conditions, and then the physics amplifies those initial uncertainties into bigger uncertainties later on. That’s the hallmark of a chaotic physical system, which the atmosphere happens to be.
It’s an interesting thing that the different physical systems are so different in their predictability.
Andrew: That’s a really important thing for people to realize about predicting the future. They see the stock market, how unpredictable it is, and they know the stock market has something to do with the news and with what’s going on in the world. That must mean that the world itself is extremely hard to predict, but I think that’s an error. The reason the stock market is hard to predict is because it is a prediction.
If you’ve already made a prediction, predicting what is wrong about your prediction is really hard — if you knew that, you would have just made that part of your prediction to begin with. That’s something to meditate on. The world is not always as hard to predict as the stock market. I can predict that there’s going to be a traffic jam tomorrow on the commute from the East Bay to San Francisco, between the hours of 6:00 a.m. and 10:00 a.m.
I think some aspects of social systems are actually very easy to predict. An individual human driver, might be very hard to predict. But if you see 10,000 people driving down the highway, you get a strong sense of whether there’s going to be a traffic jam. Sometimes unpredictable phenomena can add up to predictable phenomena, and I think that’s a really important feature of making good long-term predictions with complicated systems.
Anthony: It’s often said that climate is more predictable than weather. Although the individual fluctuations day-to-day are difficult to predict, it’s very easy to predict that, in general, winter in the Northern Hemisphere is going to be colder than the summer. There are lots of statistical regularities that emerge, when you average over large numbers.
Ariel: As we’re trying to understand what the impact of artificial intelligence will be on humanity how do we consider what would be a complex prediction? What’s a simple prediction? What sort of information do we need to do this?
Anthony: Well, that’s a tricky one. One of the best methods of prediction for lots of things is just simple extrapolation. There are many physical systems that, once you can discern if they have a trend, you can fit a pretty simple function to.
When you’re talking about artificial intelligence, there are some hard aspects to predict, but also some relatively easy aspects to predict, like looking at the amount of funding that’s being given to artificial intelligence research or the computing power and computing speed and efficiency, following Moore’s Law and variants of it.
Andrew: People often think of mathematics as a source of certainty, but sometimes you can be certain that you are uncertain or you can be certain that you can’t be certain about something else.
A simple trend, like Moore’s Law, is a summary of what you see from a very complicated system, namely a bunch of companies and a bunch of people working to build smaller and faster and cheaper and more energy efficient hardware. That’s a very complicated system that somehow adds up to fairly simple behavior.
A hallmark of good prediction is, when you find a trend, the first question you should ask yourself is what is giving rise to this trend, and can I expect that to continue? That’s a bit of an art. It’s kind of more art than science, but it’s a critical art, because otherwise we end up blindly following trends that are bound to fail.
Ariel: I want to ask about who is making the prediction. With AI, for example, we see smart people in the field who predict AI will make life great and others are worried. With existential risks we see surveys and efforts in which experts in the field try to predict the odds of human extinction. How much can we rely on “experts in the field”?
Andrew: I can certainly tell you that thinking for 30 consecutive minutes about what could cause human extinction is much more productive than thinking for one consecutive minute. There are hard-to-notice mistakes about human extinction predictions that you probably can’t figure out from 30 seconds of reasoning.
Not everyone who’s an expert, say, in nuclear engineering or artificial intelligence is an expert in reasoning about human extinction. You have to be careful who you call an expert.
Anthony: I also feel that something similar is true about prediction. In general, making predictions is greatly aided if you have domain knowledge and expertise in the thing that you’re making a prediction about, but far from sufficient to make accurate predictions.
One of the experiences I’ve seen running Metaculus, is that there are people that know a tremendous amount about a subject and just are terrible at making predictions about it. Other people, who, even if their actual domain knowledge is lower, the fact that they are comfortable with statistics, that they’ve had practice making predictions are just much, much better at it.
Ariel: Anthony, with Metaculus, one of the things that you’re trying to do is get more people involved in predicting. What is the benefit of more people?
Anthony: There are a few benefits. One is that lots of people get the benefit of practice. Thinking about things that you tend to be more wrong on and what they might correlate with — that’s incredibly useful and makes you more effective.
In terms of actually creating accurate predictions, you’ll have more people who are really good at it. You can figure out who is good at predicting, and who is good at predicting a particular type of thing. One of the interesting things is that it isn’t just luck. There is a skill that people can develop and obtain, and then can be relied upon in the future.
Then, the third, and maybe this is the most important, is just statistics. Aggregating lots of people’s predictions tends to make a more accurate aggregate.
Andrew: I would also just like to say that I think the existence of systems like Metaculus are going to be really important for society improving its ability to understand the world.
Whose job is it to think for a solid hour about a human extinction risk? The answer is almost nobody. So we ought not to expect that just averaging the wisdom of the crowds is going to do super well on answering a question like that.
Ariel: Back to artificial intelligence and the question of timelines. How helpful is it for us to try to make predictions about when things will happen with AI? And who should make those predictions?
Andrew: I have made a career shift to coming up with trying to design control mechanisms for highly intelligent AI. I made that career shift, based on my own personal forecast of the future and what I think will be important, but I don’t reevaluate that forecast every day, just as I don’t reevaluate what neighborhood I should live in every day. You, at some point, need to commit to a path and follow that path for a little while to get anything done.
I think most AI researchers should, at some point, do the mental exercise of mapping out timelines and seeing what needs to happen, but they should do it deeply once every few years in collaboration with a few other people, and then stick to something that they think is going to help steer AI in a positive direction. I see a tendency to too frequently reevaluate timeline analyses of what’s going to happen in AI.
My answer to you is kind of everyone, but not everyone at once.
Anthony: I think there’s one other interesting question, which is the degree to which we want there to be accurate predictions and lots of people know what those accurate predictions are.
In general, I think more information is better, but it’s not necessarily the case that more information is better all the time. Suppose, that I became totally convinced, using Metaculus, that there was a high probability that artificial superintelligence was happening in the next 10 years. That would be a pretty big deal. I’d really want to think through what effect that information would have on various actors, national governments, companies, and so on. It could instigate a lot of issues. Those are things that I think we have to really carefully consider.
Andrew: Yeah, Anthony, I think that’s a great important issue. I don’t think there are enough scientific norms in circulation for what to do with a potentially dangerous discovery. Honestly, I feel like the discourse in most of science is a little bit head in the sand about the feasibility of creating existential risks from technology.
You might think it would be so silly and dumb to have some humans produce some technology that accidentally destroyed life, but just because it’s silly doesn’t mean it won’t happen. It’s the bystander effect. It’s very easy for us to fall into the trap of: “I don’t need to worry about developing dangerous technology, because if I was close to something dangerous, surely someone would have thought that through.”
You have to ask: whose job is it to be worried? If no one in the artificial intelligence community is point on noticing existential threats, maybe no one will notice the existential threats and that will be bad. The same goes for the technology that could be used by bad actors to produce dangerous synthetic viruses.
If you’ve got something that you think is 1% likely to pose an extinction threat, that seems like a small probability. Nonetheless, if 100 people have a 1% chance of causing human extinction, well someone probably has a good chance of doing it.
Ariel: Is there something hopeful that you want to add?
Anthony: Pretty much every decision that we make is implicitly built on a prediction. I think that if we can get better at predicting, individually, as a group, as a society, that should really help us choose a more wise path into the future, and hopefully that can happen.
Andrew: Hear, hear.
Visit metaculus.com to try your hand at the art of predicting.