What Is a Self-Fulfilling Prophecy?
Social media platforms, Carissa Véliz says, are built to keep you on them so they can predict what you do next. So one way to put it is that the easiest way to predict the future is to determine it.
In this episode · 17 chapters
- Starting at the end: what is a good life?
- Why the analog still matters
- Why we listen to tech prophets
- Predictions are not facts
- What AI prediction really gives us
- Reading the predictions of Sam Altman and Jensen Huang
- The turkey problem
- Why our minds fall for good stories
- From Rasputin to Elon Musk
- Prediction markets and Polymarket
- Who owns the algorithm, and what it does to democracy
- What an algorithm actually is
- How self-fulfilling prophecies work
- Seinfeld, comedy and creativity
- Personal stories: how everything could have been different
- How to increase serendipity
- Stoicism, Epicurus and the fear of death
And when you’re reading a good novel or in a comedy show or really enjoying a conversation with your friend, who cares what the prophet is prophesizing? Let them talk. They lose their power over you.
In her book Prophecy, Carissa Véliz argues that predictions about people change the outcomes they claim to describe. She explains why forecasting rain differs from forecasting a person, and why a tech executive asked about the future will tell you it is his product.
She hears AI forecasts from Sam Altman and Jensen Huang as the words of rich men with products to sell. Her position is plain: “Predictions are never facts. At best, they can be educated guesses.” We also cover the turkey problem, Rasputin and Seinfeld.
Carissa Véliz, associate professor at Oxford's Institute for Ethics in AI, discusses her book Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI on the Thinking On Paper podcast. Her argument is that predictions are a form of power: “a prophet with enough power can utter a prophecy and if people believe it, it can become true.” She asks listeners to weigh AI forecasts from Sam Altman, Dario Amodei and Jensen Huang by who is speaking and what they are selling.
Why are self-fulfilling prophecies so hard to catch?
Self-fulfilling prophecies, she says, “tend to be the perfect crime because they cover their tracks.” A hiring algorithm that never hires certain people can never learn whether they would have been good employees.
Are predictions facts, according to Carissa Véliz?
Carissa Véliz argues that predictions are never facts. In her words: “Predictions are never facts. At best, they can be educated guesses.”
How do predictions about people differ from weather forecasts?
She separates forecasts about things from forecasts about people. A rain forecast does not change the weather, but a prediction about a person changes expectations, and so changes how that person is treated.
How should we judge AI forecasts from tech executives?
Ask a tech executive about the future, she says, and he will tell you the future is his product. Responding to AI forecasts from Sam Altman, Dario Amodei and Jensen Huang, she asks listeners to weigh who is speaking and what they are selling.
What does Véliz say social media algorithms are built to do?
Social media algorithms, she says, are built to keep you on the platform so they can predict what you do next. Her line: “the easiest way to predict the future is to determine it.” She would ban algorithms designed to maximise engagement and force platforms to share data with academic researchers. Chatbots, she says, are built the same way, which is what makes them sycophantic.
Why would predictive algorithms never have chosen Seinfeld?
Predictive algorithms, she argues, would never have chosen Seinfeld, a show audiences disliked at first. Comedy works against prophecy because what makes something funny is surprise.
How can people resist the tyranny of prediction?
Her answer to “the tyranny of prediction” is to increase serendipity: read outside your field and talk to strangers. In the book she makes the case for Epicureanism over Stoicism, because the Epicureans rejected fate.
Links from this episode
Questions
What is a self-fulfilling prophecy?
Self-fulfilling prophecies, she says, “tend to be the perfect crime because they cover their tracks.” A hiring algorithm that never hires certain people can never learn whether they would have been good employees.
What is Carissa Véliz's book Prophecy about?
In her book Prophecy, Carissa Véliz argues that predictions about people change the outcomes they claim to describe. Her argument is that predictions are a form of power: “a prophet with enough power can utter a prophecy and if people believe it, it can become true.”
Who is Carissa Véliz?
Carissa Véliz, associate professor at Oxford's Institute for Ethics in AI, discusses her book Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI on the Thinking On Paper podcast.
Which philosophy does Carissa Véliz favour over Stoicism?
In the book she makes the case for Epicureanism over Stoicism, because the Epicureans rejected fate.
Transcript
Starting at the end: what is a good life?
Mark Fielding We want to start at the end of your book, your new book, Prophecy. So we know what we have to lose. What is a good life? What is the good life?
Carissa Véliz That is a very philosophical question, perhaps the most important philosophical question. It's the kind of life where when you get to the end of it, you feel it's worth the ride. I've done my bit. And it's worth thinking about because sometimes we're so caught up in the logistics of life that we forget what it's all about. And I think the good life is one that you...
Carissa Véliz have written yourself, that you didn't receive a script from someone else that you just followed, that you feel that you thought about what you think is important and that you got to know yourself well, got to know how you can contribute to the world, what are the things that you enjoy and that you find meaningful, and that you went for it, and that you gave it your all, that hopefully you left...
Carissa Véliz the place better than you found it and that you had a good time and great friendships and meaningful relationships in the journey.
Jeremy Gilbertson So much to unpack in that. I wanna hit on a couple of notes and then I have a follow up from a quote from your book. Talk to me about how knowing yourself is so powerful in this self-directed life because I think people get caught up in this business of busy and this cycle, this whirlwind of things that we think we should be doing. How's knowing yourself help with that piece?
Carissa Véliz We're very anxious creatures and that's partly because we're smart and we're smart enough to get a sense of everything that can go wrong. But that anxiety is tricky because while it can be a useful engine and it can be a motivator. And I think it's a good sign. you have anxiety about the uncertainty of the future, it means that the future is not written, that you can intercede, that you can have a chance to change it. ⁓ because we are such anxious creatures, it can lead us to look for answers from someone else. So sometimes when we're so scared and anxious, it's easier to go to someone and tell them, tell me what I should do. Tell me what the future looks like. Tell me, tell me what path to follow, because I don't know. And that makes us very vulnerable to people exploiting that situation for their own benefit and for their own agenda. So part of what it means to know yourself.
Carissa Véliz is to be able to sit with that anxiety of knowing that you're a mortal being and that a lot of things can go wrong, is sitting with that fear about making a mistake, making an error, but also knowing what attracts your attention, what makes you curious about the world, what makes you joyful, and what you're good at in a way that you could contribute to the world that maybe no one else can.
Carissa Véliz And one of the things I get most asked by both students and parents of students and readers from around the world, people email me asking me, what should I study or what should my child study? And my best advice is, look, the future is completely unpredictable. And if you're going on strategy, you might, you're probably going to get it wrong and you're probably going to be miserable in the process. So.
Carissa Véliz Follow your curiosity. Dogs have good sense. We have curiosity. Your curiosity tells you that there's something there that needs to be looked at twice. And one of the things I want to encourage people to do is to be bolder about life, because it's not worth living a life that's not yours. And when you're bold and when you find what you're interested in, you are much more likely to find a different angle that nobody has seen before, because nobody's like you.
Carissa Véliz Nobody has your background. Nobody has even your perceptual sensitivity. And it's finding that which others are not seeing that can give you more of a competitive advantage, ironically, than trying to predict what the future will be and trying to follow that script ⁓ to try to play it safe. Playing it safe is also a bet. And safe bets are not always as safe as they might seem.
Why the analog still matters
Mark Fielding you're painting a very analogue experience. Are you suggesting that for the most part we should remove digital experiences from our existence or can there be a dual digital analogue experience to the human condition?
Carissa Véliz I think we should have two tracks, the analog and the digital, but I do worry that recently we focus a lot on the digital, partly because the digital has better marketing teams right now than the analog world. And we forget that the analog is the prime.
Mark Fielding Analog has nature, surely nature is the best advertising marketing department there is.
Carissa Véliz You'd think, but if you grow up far away from nature and you've never had that contact and you just see it in documentaries, that's not enough. Nature is something, we are such cultural creatures that nature, even nature is something that you have to grown accustomed to and learn to appreciate and learn to see.
Carissa Véliz So yeah, I think that we forget that the analog is the primal experience, that everything digital depends on the analog. We lose the analog, we lose everything. We lose the digital and we lose a lot, but we don't lose the analog. And so there is an asymmetry. I do think that analog is more important than digital. I don't want to do away with the digital. I just want to put it in perspective. And that if we focus too much on the digital to the detriment of the analog, we are heading towards
Why we listen to tech prophets
Jeremy Gilbertson So with that, you've set this conversation up in the context of we as human creatures are highly perceptible or highly influenced or highly vulnerable to the mechanism that you're talking about in prophecy that...
Jeremy Gilbertson pulls us away from who we are and into a situation that's controlled by others, directed by others. So how does the manipulation of our need to feel secure, how does that make us vulnerable to these prophecies that you're talking about? Can you walk us through that a little bit?
Carissa Véliz Yeah, essentially we're all worried about tomorrow because that's where we'll all be spending the rest of our lives. And we're wondering, are we going to have a job? Is our family going to be okay? Is war going to break out? Are we going to be able to navigate climate change? There are all these questions and we want answers and we don't have them. And so we ask, we have a tendency to ask other people about it, often experts or very rich people. think we associate wealth with success.
Carissa Véliz Surely if this person became so wealthy, they must know something that I don't know. And so we asked them, know, tell me what the future holds. get, I get asked all the time, what's the future of privacy as if I or anyone else knew. And then predictions become the arena where fights about the future get played out. So if you go to a tech executive and you tell them, tell me what is the future? What do I need to know to be able to navigate the future? Well.
Carissa Véliz What is he going to tell you? Of course, he's going to tell you that the future is his product because that's in his interest. He has a company, has a product to sell and he wants the future to be that. And it happens to line his pockets as well. And so we end up believing as facts, very often are power plays in disguise or marketing or some other kind of ⁓ power related move. Often it...
Predictions are not facts
Carissa Véliz It is a self-fulfilling prophecy. So for example, a financial agency might say that a country will be in financial trouble and that's enough to make investors flee. there's many ways to classify predictions, but one way to do it is one thing is to predict something about objects. So if I predict whether it's going to rain tomorrow or not, it has no effect on whether it actually rains.
Carissa Véliz because their clouds are not listening to this podcast. But if I predict the future about human beings, and in particular, when I predict the future about one particular human being, that can have a huge effect because it changes expectations. It changes how we view the world and how we treat other people. And it's a mistake to think about predictions as facts. Predictions are never facts. At best, they can be educated guesses.
Carissa Véliz But facts are only about the present and the past, never about the future. However, predictions are very misleading because they do sound like facts. They sound like facts about the future, but that actually doesn't exist. The future isn't written, it hasn't happened. So for example, in ancient Rome, it was illegal to predict the death of the emperor for the very simple reason that that kind of prediction ended up with a dead body of an emperor.
What AI prediction really gives us
Jeremy Gilbertson you write about this in a couple of different contexts in the book. Talk to us about this illusion of safety and illusion of security that when you rely on these predictions, because we're so busy, we're running and gunning, we're doing so many things, we need shortcuts, like that's what ends up happening. But
Jeremy Gilbertson Talk to us about what we're actually getting for that exchange.
Carissa Véliz So one very common use of AI is to minimize risks for companies. And that can look ⁓ in different ways. One is the use of AI to hire people. And so the idea is if we use data to see what kind of person has been successful in the past, we can predict who's going to be a good employee and thereby lower your risk when you hire someone that you might be hiring a walking disaster. Another use is.
Carissa Véliz AI for supply chains to try to predict what are going to be the bottlenecks or challenges or demands. And in each case, AI and other kinds of prediction give us the illusion of safety. Because part of what gives us that illusion is quantifiable. So if you can put a number on it, you feel.
Carissa Véliz Like you're closer to knowing more about whatever that is. So if I tell you, know, this person has a 70 % chance of not repaying a loan as a bank, you might feel like you are safer than if you didn't have that number to rely on. However, one of the limitations of any kind of prediction, including AI, is that it assumes that the future will look like the past. And for the most part,
Carissa Véliz That's actually true. But it's not always true. And what's most interesting about reality is that the most important events of your life, of your individual life, but also of our collective lives, are the most unpredictable ones. So it's very easy to know what's ahead when the road is completely straight. Doesn't take a genius. It's the curves.
Carissa Véliz stack the odds in some domains in our favour with predictions though? So you mentioned supply chain and logistics and hiring. What about drug discovery or climate monitoring or one day perhaps in the future the ability to predict earthquakes and tsunamis and natural disasters? there is a yin to the yang. There is a positive, a net positive for humanity to predict.
Carissa Véliz Absolutely. And it's partly what has given us a competitive advantage as animals. We're pretty weak physically, but we have language. Yeah. We have language. We have opposable thumbs and we have this ability. If we can predict how an animal is going to behave, it's easier to hunt them. If we can predict the seasons, it's easier to use the sky as a map. If we can learn how to read the stars and so on and so forth. But it's interesting that the three examples you gave.
Mark Fielding Some way, can the others?
Carissa Véliz were examples of predicting things about things and not about people. So that's the first point to note. And the second thing is, yes.
Carissa Véliz OK. So but even then, I mean, that is a kind of a mixed case because there's a lot that human beings can do to affect which genes get activated. But even then, you might argue that there are some genetic diseases that if you have it in your genes, you're going to get no matter what. And so it's not really about human behavior. But the other point to note is that even when, yes, prediction gives us a very competitive advantage when it works well because the future does look like the past. It works even better if we are aware of its limits. So one thing is to use prediction and just note that there is a premise there that is implicit. If the future looks like the past, then this is going to happen and always be aware that sometimes the future doesn't look like the past. For example, with climate change. So our predictions about climate are very limited because there's no database about the future. It's as simple as that. And we will be exposed to things that haven't happened before in the face of the Earth. And there is no way to predict them for that reason. And when it comes to human history, events are much more unpredictable. Even when it comes to the physical reality, we don't know whether one day we're going to be hit by a meteor. And we can, to a certain extent, predict it because there's a trajectory and so on. But there are physical events that we that are hard to predict or impossible to predict. But when it comes to human history, that's even more the case. Because one of the annoying and wonderful things about humanity is how we tend to surprise ourselves. when it comes to things like many wars are started by a particular murder. And whereas we can predict roughly with some accuracy how many people will be murdered in a particular year, who will be murdered and who will do the murdering is absolutely unpredictable.
Reading the predictions of Sam Altman and Jensen Huang
Mark Fielding Can I read you a couple of quotes, Carissa? So the first is from Sam Altman. Anyone in 2035 should be able to marshal the intellectual capacity equivalent to everyone in 2025. Dario Amodi. I simultaneously think that AI will disrupt 50 % of entry-level white collar jobs over one to five years while also thinking we may have AI that is more capable than everyone in only one.
Mark Fielding two years. ⁓ Jensen Huang, AI agents are the new digital workforce. The IT department of every company is going to be the HR department of AI agents in the future. You spoke about history repeating these profits are making predictions about ⁓ they're predicting the future of a past that doesn't exist. What do you think? What do you want to tell me and Jeremy when you hear quotes like that? Predictions like that.
Carissa Véliz I I fear that the way they're interpreted is as a fact. These guys know a lot about AI, they are making AI. If they're saying this, this is going to happen. And I better act in accordance with that. So if every ⁓ company is going to have an AI department, then I better start building that now so I don't fall behind the curve. And what I would like people to hear is, OK, this is a very rich guy with huge financial interests.
Carissa Véliz who is selling this product. He's made predictions in the past. Have those panned out? Not that great, all right. Is that the future that I want? Is that the future that I want to see? And if not, what am I going to do to build the future that I want to see?
Jeremy Gilbertson we're in this conversation about our susceptibility to being pulled into these predictions. it's not just that we're gullible people or we talked about the whole fear security thing. ⁓ I ran across something, and you referenced Bayes quite a lot.
Jeremy Gilbertson And, ⁓ you know, when you set the stage for probability theory and all of that and how that rolls into it, I ran across a guy named, I don't know him, but I ran in my research, ⁓ Carl Friston, who has this theory called the Bayesian brain, which essentially says that our brains are prediction machines. And there's whether that theory is right or wrong. There is some research going into that. And if that is even a little bit the case.
Jeremy Gilbertson That's a pretty tough situation because we have to kind of think about ourselves as being somewhat disadvantaged in this situation. So what are your thoughts on that theory number one and how does that apply to your thinking and your message to everybody that, hey, this might be the case, but we are kind of fixable if we think about it a different
Carissa Véliz Yeah, that's a great question. So part of the answer to the previous question is ⁓ how things can go wrong if you are too gullible with these predictions. The company Klarna, the payments company Klarna, fired 700 people quite publicly not too long ago. And the idea was that AI was getting good enough to replace them. And about a year later, they were hiring 700 people.
Carissa Véliz So often we idealize AI, even when we're working with it and it's not working, when we hear that it's going to work perfectly by these guys, we tend to believe that we tend to be optimists when it comes to technology. When it comes to thinking about the human mind as a predictive machine, there is a long history of coming up with disciplines, with artifacts, with theories, and then coming up with the piggybacking theory that the mind is like that. So we saw this with the rise of computers. ⁓ We never really thought about the mind as a computer, but then suddenly we came up with these artifacts that were quite impressive and we started thinking, maybe that's how the mind works. So I think there is a bit to that when it comes to theories about the mind. I think that
Carissa Véliz Yes, that is one thing that the human mind does. We do predict in a kind of roughly Bayesian way in which we have our hypotheses and then we test it out in the world, both influencing the world, but also just getting information from the world. And then we revise our views accordingly. I think that is very reasonable. But I think what makes human beings especially impressive is the ability to do causal reasoning to think about how things connect with one another, even in the absence of data, of previous data about that particular thing. And when we rely so much on AI and we lose sight of these causal reasoning, we risk creating blind spots for ourselves.
The turkey problem
Carissa Véliz So one of the classic examples in philosophy is that of the turkey. So there's a turkey and it's being fed every day and taken care of. And each day that passes, the turkey grows more and more confident that it has a great caretaker. And of course, every day that passes, Thanksgiving is nearer and nearer and their life is more at risk. And I think something particular to human beings is that ability to think.
Carissa Véliz Even if I don't have data to the contrary, I can understand how things work and what are the motivations behind people and how this could go very wrong.
Mark Fielding What should that turkey do if someone prophesies and says you have a less than zero chance of being eaten for dinner in 25 days?
Mark Fielding What would this, what should the Turkey do? Escape? Is that not something similar with some of these prophecies being right enough that we should act upon them?
Carissa Véliz Well, it depends on the kind of prophecy we're talking about because there's also the opposite effect. So Hannah Arendt has this point about how it makes no sense to argue with a murderer who's saying that their victim is going to be dead. The only appropriate response is to rescue the victim. So the prediction that the person is going to eat the turkey is presumably a prediction of an outsider.
Carissa Véliz who has no stake in the matter. But most predictions that are about human beings out there are predictions that are not without self-interest and are not without that kind of reflexive quality. So presumably the prediction that the turkey is going to be eaten in 25 days is not going to affect the behavior of the caretaker unless turkeys could talk and then if
Carissa Véliz If they had learned that I had told the turkey that it's going to be killed in 25 days, the caretaker might kill them tomorrow and then freeze them. And so the point is that it's not about, okay, is this prediction going to be right? And therefore how do I act? The point is that the future is unwritten. And when we predict things in ways that affect expectations, that changes the outcome itself.
Why our minds fall for good stories
Jeremy Gilbertson you write about truth-like things, and you write about when numbers are attached to things, they make them seem more believable, even though numbers are human invention, in a sense, which I thought was a very, very clever view of it. Let me read a couple of.
Jeremy Gilbertson And then I want to ask you a couple of questions on these quotes. So quote, fiction is simplified, custom designed to affect us in a particular way while the truth is messy and unwieldy, hard to appreciate in all its richness. Another quick quote, not enough of a market for the truth. That one hit hard. ⁓ So with stories hitting harder than facts by design, how could we give ourselves a leg up to to know that that's a mechanism being manipulated.
Carissa Véliz It's a bit like encountering junk food. I think now it's a very well known fact, but hearing it for the first time that, know, junk food is actually designed for your taste buds and not for your body. And there are reasons why you crave so much sugar and butter and fat, because it was very hard to come by thousands of years ago when our bodies evolved. And now when we have it available all the time, it can really poison us. And it's a bit like that. It's being aware.
Carissa Véliz that your mind is naturally drawn to well-rounded stories that make sense, that are uplifting, or that are terrifying, and being aware that in the long run, you are much better served by actually being interested in the truth, even when it's uncomfortable, even when it's hard to understand, even when it yields a lot of uncertainty and maybe some more questions than answers.
Carissa Véliz Um, it's having that second level, um, reflection on why am I attracted to this story? How much of it is true? And if I believe a false story, how am I, how you're going to eventually crash into a wall.
Mark Fielding Feels like you're asking a lot of people. It's not that basically this is not a passive movement. This is, you're to have to be active if you want to fight the profits. Before I ask my next question, could I ask you who are your favorite or who are the most dangerous, memorable profits from your book first?
From Rasputin to Elon Musk
Carissa Véliz That's a great question. So one very famous prophet is Rasputin and he's really an interesting guy because he came out of nowhere. This was a person who didn't even know how to read. He came from a very rural place, not well connected at all and it's very interesting to see how he gains following, how his eccentricity attracts people but he's also very well versed in power.
Carissa Véliz and manipulating people's views. However, when somebody lies and lies and lies and lies, it's a matter of time before it all comes crashing down because they start earning enemies and people become resentful. One of the things that he did was he was a sexual predator. And when he had harassed enough women, people started getting angry about it, not only women, but also the men in their lives. And eventually,
Carissa Véliz ⁓ You have this incredibly tense situation in which the swords are completely dependent on this person and have made him almost part of the family, but it's really endangering their rule because they have lost legitimacy in the face of the people. And at the end, it becomes their downfall. So that is ⁓ a particularly interesting case.
Mark Fielding Okay, well, it's interesting that you use the word followers. Elon Musk has 230, 240 million followers on Twitter. Big Tech has the ear, just like Rasputin did, of the leaders of the free world. They sat on the front row of the inauguration. They have instant access to literally a billion people.
Mark Fielding something that Rasputin and the other prophets in the book never had in such a short, tiny timeframe. What does that change about modern day prophecy? How is modern day prophecy different to the days of Rasputin and the oracle of Delphi?
Carissa Véliz In some ways, it's the same in the sense that the political role that it can play is the same and the danger is the same that a prophet with enough power can utter a prophecy and if people believe it, it can become true. It it creates its own momentum. In another way, it's a more complex story because there is a lot of tension between the administrator sorry, the administration and some of these profits, we've just seen a fallout between Anthropic and the Trump administration. And so there is somewhat of a mutual dependency there, among other reasons, because governments are using big tech for infrastructure in a way that the czar arguably could have gotten rid of Rasputin without much consequence. And so it's a much more complex picture, but the fact of the matter remains the same, that if one powerful and influential person says something about the future and enough people believe it, there is a momentum for it to become true in a way that if they hadn't said that, maybe it would have never happened. And that makes it a very sensitive issue and something that we should be very skeptical about and very responsible as citizens to not just any kind of prophecy because somebody influential has said it.
Jeremy Gilbertson I'd maybe just like to add some of the notes swirling around in my head as we think through this is the exponential nature of the technology and the on the flip side of that, the business of busy that humans are involved in the striving, the fighting for things that sometimes, you know, aren't even self-directed, right? We're just put into this world. And with that world, we're just peeking at this
Jeremy Gilbertson Oracle that's sitting in our pocket that is I mean that to me is like as I think of that's really kind of a scary thing and You almost want to like shout it from the mountaintops, but like people might be too busy to believe it you connect to the people who are too busy or who can't afford the time to think so actively about it?
Carissa Véliz One of the pleasures of writing this book is that it changed the way I see the world. And I hope that it changes the way other people see the world. And just this very simple thought about, be careful with predictions. They're not facts. And they can become self-fulfilling prophecies has changed the way I interpret most news I come across. And in a way, that's enough. It doesn't take that much. In a way,
Carissa Véliz You know, the overachievers can go all the way and say, OK, how am I going to be the author of my own life? But even a very passive take of just being a little bit more skeptical about prophecies would change massively the quality of democracy and public discourse that we have. And as predictions markets become more popular, this is going to be more of an issue. And I've shared the book with a few friends.
Carissa Véliz And it gives you a lot of pleasure when they send me an email saying, oh, today I ran across this news and I just saw it differently. Thanks to just that kind of snippet. It's a kind of lens that makes you look at the world anew.
Prediction markets and Polymarket
Mark Fielding mentioned Cauchy, Polymarket. According to the Wall Street Journal, they're both, they have their eye on 20 billion valuations, whatever that means. People are predicting, betting, making money on war, leaders, health, murder. In your book, you asked Tim Harford whether you should ban prediction.
Mark Fielding In light of Cauchy and Polymarket, what did you mean by that? Because for me, reading that question, I read that question, I read it quite a few times because it didn't compute for me.
Carissa Véliz So I was asking Tim Hartford about whether we should have some limits, whether we should have some guardrails, and in particular with predictions about elections. And his first response, which I think is the right one and a very sensible one, is, well, wait, we don't want to ban anything unnecessarily, right? And let's think this through, because in liberal democracies, the default is not banning. But when you start to get into it and you say, well, OK, election polls are mostly for entertainment. And if you cash it out that way, you say, well, are they worth the risk though for entertainment? ⁓ there are some good movies I could recommend that are very entertaining. And so they only become justifiable if there isn't a risk of them distorting democracy. But when there is, then you start thinking, well, maybe we should put some limits here. And something that Jeremy said is a great point about scale.
Carissa Véliz Predictions right now scale more than ever, both in the sense that an algorithm that's making a predictive guess can reach millions, if not billions of people. Very often we're using either the same algorithm or a version of the same algorithm. And in the sense that someone ⁓ with high following on social media can reach billions of people. And that has never been the case in the past, which makes predictions more dangerous than ever.
Who owns the algorithm, and what it does to democracy
Jeremy Gilbertson Could I jump on the algorithm segue here for a second? ⁓ This is pretty timely and it's also ⁓ top of mind for me as a parent, I have four kids. We do our best to maintain their relationships with technology, but every now and again, I catch them kind of just kind of in the loop, which is such a difficult balance to run, but.
Jeremy Gilbertson Let's talk about algorithms and let's talk about the ownership of one of the most powerful algorithms in the United States at the moment that is owned by Oracle, that is owned by Silver Lake, MDX, this group. I'm talking about the TikTok US algorithm, right? And I want to get your thoughts on what affects that pathway, that pipeline, that accessibility to the younger generation.
Jeremy Gilbertson What effect could that have? How could it be used? And what do we need to understand about pathways like that to large segments of our population?
Carissa Véliz The straight answer is that we don't know. We know it has an effect and we know that large segments of the population and in particular young people access this kind of content and are influenced by it. But it's a hard thing to quantify and to get a good sense of among other reasons, because the data from platforms is proprietary. So it's not easy to research. However, when we see either academic research or research in courts, it does suggest that these platforms have a huge effect on how young people look at the world and how anyone looks at the world. And one example that comes to mind that is particularly concerning
Carissa Véliz And it comes to mind not only because it's concerning, but also because it was studied in depth, is the case of Molly Russell. Molly was a teenager in the UK who was exposed to content related to self-harm and who ended up ending her life. And there was a trial about it. And there's a documentary called Molly versus the Machines that I highly recommend about the trial. And it was the first time.
Carissa Véliz that in a country like the UK, it was established that the platform, in that case, Instagram, had played a significant role in the death of this very young person. And if that is true, then the same goes for TikTok, of course, because it's, very similar platforms.
Jeremy Gilbertson I'm gonna let that one breathe a little bit in my brain. ⁓ Yeah, there's a lot to think about there. And the black box nature of those things and the proprietary nature of those things, like you said, make it very difficult. And we may not know for 50 years, 30 years, 25 years, what kind of influential effect this has on short cutting, critical thinking, short circuiting.
Jeremy Gilbertson ⁓ how people acquire their beliefs and hold their beliefs. ⁓
Carissa Véliz Part of what makes a democracy functional is a certain kind of trust in the system, in both your citizens, but also in the system, and trust that there hasn't been foul play of some sort. So one concern is that these platforms could be used for political fear-mongering or different kinds of propaganda.
Carissa Véliz One concern is that even if they're not used that way, that they could be used that way, and we have no real way of knowing whether they're being used that way, introduces a corrosiveness in the gears of democracy doubt about whether it's working well enough. And that's already bad enough. So we have a lot more work to do to figure out how to better protect democracy from these potential interferences.
Mark Fielding In prophecy, you say towards the end that in fact, this is a book about AI ethics, which but not about AI ethics. So what can?
Mark Fielding easy to execute to make these platforms less dangerous? Is there anything that can be done by the end of this year, for example?
Carissa Véliz Yes. One thing that could be done is forcing platforms to give up data to academic researchers. So we have a better sense of what's going on. Another thing that can be done is to ban algorithms that are designed to maximize engagement. Every algorithm is designed to do something. And we know that maximizing engagement is an incredibly toxic thing to do. It's the equivalent.
Carissa Véliz of putting cocaine or another kind of very addictive drug into food to make people buy that food. We don't allow it. We don't allow addictiveness to be part of the design of food. And in the same way, the algorithm should maximize something else, should maximize entertainment or information or whatever else. But to maximize engagement, we know, leads to some of the most toxic patterns that we've seen.
What an algorithm actually is
Jeremy Gilbertson I think people are so confused about what an algorithm is. And when people say algorithm, they're just like, it's just like the magic that lives in the hardware and the software that makes this stuff happen. Like, is it a fair assessment, a fair comparison to an algorithm that is directed to maximize engagement? Is it Pied Piper-esque? Is it a Pied Piper kind of story where, you you're basically pulling these people into this thing by by the nature of what it actually is. Like what would be another, what would be a way to explain algorithms like through story?
Carissa Véliz That's a really good question. An algorithm is and you ask it to do something for you. In some cases, it's to identify dogs in a bunch of images, which can then be, of course, used to change that instruction into identify a tumor for the purposes of medical health care. And when it comes to social media platforms,
Carissa Véliz What the algorithm is built to do is to get you to stay on the platform for as long as possible, because that way platforms get more data out of you and then they can do things with that data. And what do they do? Mostly they try to predict what you're going to do next, which is of course influenced by the kind of content that you are exposed to. So one way to put it is that the easiest way to predict the future is to determine it.
Carissa Véliz So if I jail someone, of course it's very easy to predict when where he's going to be the following day. I jailed the guy. So in the same way, when a platform influences your behavior, it makes it easier to predict what you're going to do. Chatbots, by the way, are also built in this way. So they are built in a way that will make people want to use it more.
Carissa Véliz And that is what makes them very sycophantic. They have this ⁓ feature whereby they basically validate you as a user. And so they'll tell you that you're brilliant, that you're so insightful. And of course, validation is a pleasurable thing. And in small doses, it can be important and encouraging. But too much validation makes us lose complete perspective.
Mark Fielding The idea of banning those types of algorithms to my cynical brain seems, I'm going to make a prediction, almost zero, at least in the foreseeable future. It's a very top-down approach. What does the bottom-up emergent solution to this problem look like?
Carissa Véliz It looks like either consumers being more demanding and saying, well, I'm not going to use this thing. That's just exploiting me. And I think we're seeing some of that. Granted, my students are not particularly representative of the population, but I have more and more students who don't use social media and who don't want to use social media. also companies doing better.
Carissa Véliz So realizing that there's a competitive advantage in offering an actually safe product and an actually respectful product and people choosing it. And something important to bear in mind is that most of humanity's breakthroughs, whatever you consider a breakthrough from coming up with the airplane and actually making the thing fly to ⁓ human rights of all kinds, including workers' rights and...
Carissa Véliz universal suffrage and all of that seemed absolutely impossible, quite near to when they were achieved. That's something really interesting to think about.
How self-fulfilling prophecies work
Jeremy Gilbertson So coming off the algorithm discussion, we reference self-fulfilling prophecies ⁓ a bit. Let's go a little deeper into that, because that's a big piece of this book. Tell us about that in general and what people need to know about how the future can actually be written by the people predicting it if we let it happen to us.
Carissa Véliz The main idea behind self-feeling prophecies is that once a prophecy has been issued, it changes people's expectations in a way that makes it more likely to become true. So one example that is very relevant for the times that we live in is that once a population starts thinking that there will be war, they start preparing for war and they make war much more likely to happen.
Carissa Véliz because of course it creates a reaction in the adversary of preparing as well. If you see that your neighbor is getting armed, then you arm yourself. And then once everybody's armed, it makes a war much more likely. there are all, self-fulfilling prophecies are all around us. There is, and there are many studies that validate how powerful they are. So one of the studies that I cite is one in which researchers get given mice
Carissa Véliz And half the researchers get told that their mice are stupid for traversing a maze. And the other ones get told that their mice are geniuses. And the researchers who have the geniuses end up having actual geniuses. But in fact, they got produced because all the mice were exactly the same. So the expectations of the researchers that their mice would be dumb or brilliant.
Carissa Véliz influenced how they treated them in a way that ended up fulfilling the label that these mice had. Another example that is very real is how predictions can affect the outcome of health in medical settings. So medical researchers are always scarce and we have to make a decision of where they go first. And this is made on the basis of predictions.
Carissa Véliz And if somebody predicts that a person is less likely to benefit from resources, the resources would be allocated to someone else. And we will never know whether that person might've lived if we had given them the resources. And I tell the story of one case in which there was an Oregon donor in Spain who had spontaneous heartbeats, so meaning that they weren't completely dead, but because they had already been branded as an Oregon donor, it was...
Carissa Véliz late to do anything about it. And that example is particularly good because not only does it exemplify how high the stakes can be, but also how self-fulfilling prophecies tend to be the perfect crime because they cover their tracks. We will never have that data.
Carissa Véliz And in the same way, when we use an algorithm to predict who's going to be a good employee and we never give the job to certain kinds of people, we will never know whether they might've been brilliant employees. And so we get the sense that the algorithm is very accurate when in fact we don't have the data that would be most important to prove it wrong.
Jeremy Gilbertson Go ahead. Just in the context of wars, just given the current state of the world, those narratives are tremendously important to think about. Not just like, you see this country building up arms. It's taking a person and turning that person into an object and making that object affect someone's
Jeremy Gilbertson perception, right? So pick a, we're country A, there's country B. know, Mark is telling us this story about the people in country B. We start believing those people in country B are a certain way, and then we need to act, even though you and I have zero experience with the people in country B. Not to take a hyper-political right turn or left turn, funny enough. ⁓
Jeremy Gilbertson What about that with your research and your background, could we open up people to, I think it's an open-mindedness thing, but like, how do we get people to be more curious and more open-minded to understand that the things that are sprinkled don't always benefit us versus them? We're asking the same question a lot, just in different contexts, I think.
Carissa Véliz One thing to bear in mind when it comes to predictions is that although they sound like descriptions about the world, they are in fact closer to prescriptions, which means that when somebody says this is going to happen tomorrow, it often has an implication of what you should do. So do this, which often is a kind of order to fulfill their vision of the world. And when you bear this in mind,
Carissa Véliz I you're more, it's like building your immune system. You just stop for a second and ask yourself, is this really right? Is this what I think? Is this what I agree with? Are there other points of view?
Carissa Véliz We tend to forget how much impactful culture is. Of course, in the grand scheme of things, there are very powerful actors and sometimes raw power just ends up deciding the day. But culture influences everything from who do we vote for to how we treat those in power and how much we demand from them.
Carissa Véliz to the kinds of products we design and accept and use, to the kinds of narratives that become normalized.
Carissa Véliz And that means that as citizens, we have much more power than we think, even though it's a softer power and that it's not the most obvious or direct way to change the world. But in fact, it's incredibly important. And culture is all of us, is the conversation that we're having now.
Seinfeld, comedy and creativity
Mark Fielding Well, speaking of culture, I want to speak about my favourite part of your book, prophecy. Do know who my favourite prophet is?
Mark Fielding Jeremy, do you know who my favorite prophet is?
Jeremy Gilbertson ⁓ Maybe the guy who wrote the Culture 3 novels?
Mark Fielding My favourite prophet is George Carlin and I think maybe my second favourite prophet is Bill Hicks. You speak about faulty towers and you interviewed John Cleese for this book and I laughed that loud when you described the scene in faulty towers where he has to choose the right branch to hit the car. I think faulty towers is a masterpiece. Comedy and prophecy. Why do you write about comedy? And can I just say also, like, I actually laughed out loud in different parts of the book as well. There's some very funny ways that you frame things that are very serious, but you had a joke and I love that. But yeah, why do you speak about comedy?
Carissa Véliz I wanted to go all out in this book. It's partly about being older and thinking about how short life is and being willing to say what you think ought to be said and not thinking about whether it's going to be commercial or whether it...
Carissa Véliz It's a bit more, let's say this book is post tenure. And in many ways, I allowed myself much more freedom than in the past. that's part. And comedy is a big part of my life. I love comedy and it influences how I see the world. But it's also something that I've learned is very important for politics. So one of the marks of a...
Carissa Véliz a government turning authoritarian or a society turning authoritarian is the loss of a sense of humor. And this is not new. Milan Kundera wrote ⁓ a brilliant novel called The Joke, which is exactly about this, about how in a communist regime when somebody made a joke, it could cost you your career and potentially your life. There is something very irreverent about comedy.
Carissa Véliz And it's also very much against prophecy because part of what makes something funny is when it surprises us. We sort of expect a sentence to be one way and then when it ends in a completely different way, it often makes us laugh. And going back into the past, I think part of the lesson of the book is that we've seen this before. Don't be dazzled by the new technology. This has happened before and look at this pattern.
Carissa Véliz And it's easier to see a pattern of the past because you don't believe in those technologies anymore and because you have distance. And when you went into the court of a king, you realize that if you wanted to know the truth, you wouldn't go to the prophet, you'd go to the jester. The jester was the only person with the freedom to say the truth because it was under the veil of comedy.
Carissa Véliz One of the parts that I enjoyed most about in writing the book is the part about Seinfeld. Seinfeld is one of my favorite comedy shows. I love the history. I did not, I tried. Did not work out. Maybe for the next book. ⁓ Still on offer, Jerry. But yeah, so.
Mark Fielding Did you speak to him for the book? Did he agree to interview?
Carissa Véliz Seinfeld is a fascinating example because people hated the show. It was a show about nothing. There's a very funny episode about it. And nobody had seen anything like it. And so at the beginning, was decidedly not what people wanted to watch. So if we had used predictive algorithms to choose TV shows, it would have never chosen that show at the time.
Carissa Véliz And part of its magic is that it changed the sensitivity of the audience. It's an acquired taste. You learn to see the world through sign for lenses and take pleasure in the absurd. it became one of the most popular TV shows in history. And part of the brilliance of it is how they did not follow the cookie cutter approach to sitcoms. And every time NBC gave them guidelines, they would say, we're not going to do that.
Carissa Véliz And how the more we rely on predictive algorithms, the more we're losing that innovation and that boldness and that creativity and that humor.
Mark Fielding Yep, there is nothing like comedy to short circuit your serious gene. I love it. Okay, that's a brilliant story.
Jeremy Gilbertson is such a dumb little addition, but I'm gonna say it anyway. My wife and I went to see Seinfeld stand up at this awesome theater. I actually was laughing so hard and for so long, I thought I had to leave for my health. I couldn't stop laughing, my stomach was, it was crazy. But no, comedy has such a beautiful way to disarm people and the most brilliant,
Jeremy Gilbertson medians access that button and then deliver such profound messages to it. So I think it's like Mark said, I think it's a great piece of this book ⁓ that couples your, this almost reverent, I'm just gonna say whatever the hell I want because I believe in it so hard and the message needs to be heard. It was a great balance.
Carissa Véliz Thank you. And I find that comedy is good for creativity. It kind of makes you loose. It makes you pliant. It puts you in a kind of state of mind that I think makes for better decisions and more insight.
Mark Fielding Yep, totally agree.
Personal stories: how everything could have been different
Jeremy Gilbertson What's, what we've been talking about a lot, and like I said, there's so many different things I think we could go into in this book. Give me one piece, one little thread of this book, and it could be a small little nugget, when you were writing it and you're like, man, this is like, this is very personal, this is very ⁓ locked into my mindset, like that you would wish someone reading the book would grab and want to ask you about.
Carissa Véliz so much of it ⁓ because for the first time I told personal stories and I've never done that before in writing and it's scary but it's also nice and interesting and partly comes from me being very grateful to authors who are willing to do that to tell me where they're coming from to tell me what kind of person I'm talking with because a book is a kind of conversation between the author and and the reader and I think when you guard yourself too well and you come up with a pretended view from nowhere, there's a kind of distancing that you impose on that relationship that I didn't want to impose this time around. So perhaps one story that is...
Carissa Véliz Yeah, quite personal and that makes me think about...
Carissa Véliz how everything could have been different is about how when my grandmother fled Spain, she was too pregnant to go the whole trip by boat and then train. so my family bought her a plane ticket from New York City into Mexico, but she was so pregnant that a flight attendant...
Carissa Véliz asked her whether she had a doctor's note that giving her permission to fly and she didn't. And so she had to get off that flight and go to the, to the doctor at the airport for a note. And that flight crashed. And my family thought that she, she was dead and how those unpredictable acts and unpredictable events changed the whole of somebody's history.
Jeremy Gilbertson Yeah, that's it. That's a powerful story for sure and very aligned with it. And the personal nature of your stories in this book make the connection between author and reader more impactful, I think, as I was reading the book. I got a sense ⁓ for that as well. ⁓
Jeremy Gilbertson I had to act more reading it. felt like a... was...
Jeremy Gilbertson the batteries that I need to, I don't know if fight back is the right way to describe it, but to be conscious of what is happening. And it gave me the fuel for that.
Carissa Véliz In that sense, I think one of the stories that I find more inspirational is the one about Kariko, the Nobel Prize winner who helped come up with the RNA vaccine and the obstacles that she had to face and how she persisted in the face of being ignored by academia and being harassed in academia and being demoted. And she just stuck with it because she knew she was right. And she was right.
Carissa Véliz And how much talent are we losing out on because people might not be as persistent? And how can we make things easier for that kind of person to thrive? Because we need that talent. We benefit a lot from that vaccine.
Jeremy Gilbertson this book was a spark for me as it was for Mark and I hope other readers as well to kind of grab it and feel empowered.
Jeremy Gilbertson as an individual, because so often in society, we're just a piece of the puzzle. And we think where this goes back to the business of busy and you know, I can't affect this, I can't affect that this power is too much. And, and, this, this is fuel. let's talk about some of these takeaways that you have in the end that I that I really love. ⁓ I'd love to pick out a few that resonated with me, Mark, and Mark, feel free to, you know, jump in on your side. ⁓
How to increase serendipity
Jeremy Gilbertson Let's talk about increase serendipity that that one just man like talk us through that and let's unpack it a little bit.
Carissa Véliz That's my favorite. I was thinking about that one exactly now. One of the downfalls of the tyranny of predictions is that you get told where to go, where to be at, who to meet. And if you just follow that, then somebody else is deciding what you're exposed to. And one of the ways to fight against that tyranny of prediction is to increase your serendipity. And how do you do that?
Carissa Véliz Well, essentially by being open to the world, by reading widely. So read things outside of your comfort zone and outside of your field of expertise. I've, I've gotten so much out of books. You can only live once, but if you can read, you can live many lives in one life. And that is an incredible gift, but also meet people, talk with strangers. The more I read about difficult political times in the past, I realize that one of the most effective weapons of bad governments is to erode the relationship between citizens. And part of how we fight back against that is just being comfortable with strangers, that strangers around you, for the most part, are probably friends you just haven't met yet. And to have that comfort of striking up a conversation with a stranger and learning something new and being curious about the world, being curious about what it's like to be someone else.
Carissa Véliz and being willing to rethink things. Does a chair have to look like that? What if it looked like something else to innovate and design from scratch? And in general, the metaphor that I like the most is just take strolls along the beach because you never know what the tide might bring.
Carissa Véliz My favorite is to go home and watch Faulty Towers or watch Seinfeld or put on an old Bill Hicks video, put on some comedy, get on on that. I will choose one because I think there's strange link between this one and some circles in big tech. That stoicism has become the go-to philosophy for many in not just in Silicon Valley, but on the internet, it's almost like the this the prophecy runs hand in hand with stoicism. Live in the present.
Stoicism, Epicurus and the fear of death
Mark Fielding I mentioned stoicism because obviously it's the whole the whole future lies in uncertainty live immediately Seneca.
Carissa Véliz Yes. And even though I sympathize with a lot of what Stoicism has to say, in the book I make the case for Epicureanism because they share a lot of the good stuff. So both of them think that part of what makes life good are the simple pleasures in life and tempering your desire and your fear. And one major difference between Epicureans and Stoics is that Epicureans were a lot more democratic.
Carissa Véliz And they, Epicurus famously had a garden in which everyone was welcome, including slaves and women. And they thought against Stoics that there was no fate, that even though there are elements that constrain our freedom, there are some things that still allow us a lot of leeway to write our own life. Whereas Stoicism is much more about accepting your fate.
Carissa Véliz and believing in divination. And that makes for a huge difference. Another thing that I really like about Epicureanism is its emphasis on friendship. And I think in Western society, we value a lot the family, and of course, I value family as well, and we value professional relationships. But I don't think we give friendship.
Carissa Véliz including civic friendship, friendship between citizens, the value that it should have. And the basic idea ⁓ of Epicureans is that
Carissa Véliz Deep down, what people really fear is death. And if you can have the courage to look death in the eye, to accept that we're mortal beings, and to lose that very extreme fear, to know that when death comes, you won't be there anymore, and so there is less to fear, you are much less vulnerable to profits and much more easily willing to live in the present. Because if you're not afraid of what's to come and you're not desiring things that you can't, you can never completely satisfy. So Epicurious makes this distinction between desires that you can satisfy, like if you're hungry, eventually you will be satisfied or if you're thirsty. Whereas if you are lusting over fame or fortune, you will never be satisfied because you could always be more famous and you could always be richer.
Carissa Véliz So once you temper the fear, which at the end of the day is fear of death, and desire by focusing on desires that can be fulfilled, then the prophet has nothing to offer you because you're living in the present and enjoying yourself. And when you're reading a good novel or in a comedy show or really enjoying a conversation with your friend, who cares what the prophet is prophesizing? Let them talk. They lose their power over you.




