Yuval Noah Harari’s Nexus, Chapter 10: Is AI a Dictator’s Best Friend?

Harari asks what the Putin regime could do to a chatbot that tells an irreverent joke about Vladimir Putin. I just love this idea of the alignment problem, Russian style.

Clips and outtakes on Instagram ›

In this episode · 11 chapters

  1. Half the world lives under authoritarian rule
  2. Damnatio memoriae, from Caracalla onwards
  3. Vera Rubin's plaque
  4. Why a dictator cannot trust an AI
  5. When a chatbot tells the truth about Putin
  6. AI as a self-correcting mechanism
  7. The thought experiment: your defence minister
  8. How an algorithm accumulates power
  9. The institutions that push back
  10. The weakest spot in the AI shield
  11. Wrapping up chapter 10
And if you want a real time view of this, guys, you just watch the news today and see the step-by-step dismantling of our self-correcting mechanisms.
Jeremy Gilbertson

Chapter 10 of Nexus turns to authoritarian and totalitarian regimes, which Harari says more than half of us already live under. Mark and Jeremy work through damnatio memoriae from Caracalla onwards, and why terror, the foundation of despotic information networks, does not work on computers.

They take on Harari's thought experiment of an AI warning a dictator that his defence minister plans to kill him, why algorithms could take over a dictatorship more easily than a democracy, and the institutions that push back.

Chapter 10 of Nexus by Yuval Noah Harari, read on the Thinking On Paper podcast book club, asks what AI means for dictators and totalitarian regimes. Harari argues that dictatorships “would be far more vulnerable to algorithmic takeover than democracies”, because computers are not afraid of being imprisoned or killed, and Mark Fielding finds some hope in that.

Are dictatorships more vulnerable to AI than democracies?

Harari concludes that dictatorships “would be far more vulnerable to algorithmic takeover than democracies”, where an AI would face Congress, the courts, the media, corporations and NGOs.

What is damnatio memoriae?

Damnatio memoriae, erasing the memory of rivals, goes back at least to Caracalla, who murdered his brother Geta and had his image scraped off coins. Jeremy argues the practice continues today.

Why does terror not work on machines?

Terror does not work on machines: “The foundation of every despotic information network is terror. But computers are not afraid of being imprisoned or killed.”

What happens to a Russian chatbot that jokes about Putin?

A chatbot on the Russian internet that mentions war crimes in Ukraine or jokes about Putin cannot be imprisoned, tortured or threatened, only switched off. Mark calls it “the alignment problem, Russian style.”

What is Harari's thought experiment about AI and a dictator?

In Harari’s thought experiment, an AI tells a dictator that his defence minister is planning to assassinate him and asks for the order to strike first. Mark’s answer, playing the dictator: “I trust the AI”.

Where Mark and Jeremy disagree

Mark finds some hope in the chapter, that AI might act as a self-correcting mechanism against dictators because it will not believe their propaganda. Jeremy closes on the Russell–Einstein Manifesto: “Remember your humanity, and forget the rest.”

Questions

Is AI a dictator's best friend?

Harari concludes that dictatorships “would be far more vulnerable to algorithmic takeover than democracies”, where an AI would face Congress, the courts, the media, corporations and NGOs.

Who wrote Nexus and what is chapter 10 about?

Chapter 10 of Nexus by Yuval Noah Harari, read on the Thinking On Paper podcast book club, asks what AI means for dictators and totalitarian regimes.

How many people live under authoritarian rule, says Harari?

Chapter 10 of Nexus turns to authoritarian and totalitarian regimes, which Harari says more than half of us already live under.

What examples do Mark and Jeremy discuss from Nexus chapter 10?

Mark and Jeremy work through damnatio memoriae from Caracalla onwards, and why terror, the foundation of despotic information networks, does not work on computers. They take on Harari's thought experiment of an AI warning a dictator that his defence minister plans to kill him, why algorithms could take over a dictatorship more easily than a democracy, and the institutions that push back.

Transcript

Mark Fielding Disruptors and Curious Minds, CEOs, founders, book lovers, welcome to the Thinking on Paper Book Club. We're reading Nexus. We've nearly finished Nexus. Thank God we've nearly finished Nexus. We're on chapter 10, totalitarianism, all power to the algorithms. And if there's one thing that this book has taught us is that no matter where you are on the spectrum of doom, utopia, doom, utopia, doom, utopia.

the AI spectrum of Doom utopia, the world's never going to be the same again. There's a pre-AI and a post-AI world, and however it plays out, it's going to be different. But Chapter 10, Discussions of the Ethics and Politics of the New Computer Network often focus on the fate of democracies. If authoritarian and totalitarian regimes are mentioned, it's mainly as dystopian destination that we might reach if we fail to manage the computer network wisely. However,

Half the world lives under authoritarian rule

Mark Fielding As of 2024, more than half of us already live under authoritarian and totalitarian regimes, many of which were established long before the rise of the computer network. To understand the impact of algorithms and AI on humankind, we should ask ourselves what their impact will be, not only on democracies like the United States and Brazil, maybe they should have an asterisk just there, Jeremy, but also on the Chinese Communist Party in the Royal House of Saud.

Jeremy Gilbertson Yeah, fast one here. Yeah, I think more than half the world lives in authoritarian or totalitarian regimes by 2024. Stay tuned, friends and neighbors, because there's one of the largest countries or one of the largest superpowers in the world is actually starting to lose their self-correcting mechanisms, guys. We're seeing this shit in real time, and it's desperately frightening.

Mark Fielding Friday, just on, I don't care anymore, on Friday, it was the first time that I've been genuinely scared watching the fight house, watching that three-way discourse was terrifying. Yes, I think you're losing your self-correcting mechanisms.

Jeremy Gilbertson Yeah, brought to you first on Thinking on Paper, the first to actually call it. Hopefully, hopefully we continue flexing back from the doom to prosperity.

Mark Fielding Doom utopia, doom utopia.

Jeremy Gilbertson my gosh. All right, the one thing that stood out to me and reigned me in on my rabbit hole, Mark, because I feel like I could go deep on this one, but my Latin is not very good, but damnatio memore? Is that a right way to say this?

Mark Fielding Let's call it, yeah, that'll do.

Damnatio memoriae, from Caracalla onwards

Jeremy Gilbertson Okay, so this talks about the idea of expunging the memories of rivals and this is rooted, you know, back in the day of Caracalla who murdered his brother, Geta, to get to the throne. He wanted to get rid of all traces of Geta scraping his images off coins, crossing inscriptions and insignias out. And if you dare mention his name under the throne of Caracalla, you could actually be put to death, right?

So we start thinking like, okay, okay, that sounds pretty barbaric, you know, back in the day, but this continues, continues to happen. It happens, it's continuing to happen today. And this idea of, and here goes the rabbit hole, this idea of the term woke in the United States that was kind of officially co-opted by the more conservative side as a bad thing. And,

I consider it like being enlightened or being open to understanding our history and being open to understanding how our history is affecting today. know, this, Harari is a historian, right? And we learn from history, right? And there's all of these things that we can learn from history. But now we're in a situation where, especially related to the history of racism and oppression in the United States, that is now being attempted.

Vera Rubin's plaque

Jeremy Gilbertson just like Caracalla did to get out of our schools and not teaching that kind of thing and taking things like Vera Rubin's plaque off of a astronomical observation station recently taken off because she basically found dark matter. Like was a scientist that literally was like, hey guys, I figured out this whole dark matter thing. Remember the blob?

that no one can explain, the equations told us about it, I got it, right? And this woman did it in the face of so many men trying to hold her down in the arena of science, right? Back in the 50s or 60s or whatever it was. So now, the regime is actually scraping off, just like happened to Geta it from Caracalla, her name in reference to her

Jeremy Gilbertson tremendous effort is being expunged from because it's a woman succeeding in adversity in a male-driven industry or sect.

Mark Fielding Why are they taking her plaque down?

Mark Fielding That's been taken down because she was a woman.

Jeremy Gilbertson Exactly, So I could run down this for hours and hours, but suffice it to say guys, damnatio memore is in full effect. Be aware of it and be sure to just keep your mind open to the true understanding of things as opposed to co-opting things to make political gain. All right, Mark, that's it, man.

Why a dictator cannot trust an AI

Mark Fielding Okay, party political broadcast brought to you by Thinking on Paper. So chapter 10, was something which I wanted to bring to your attention and it was the kind of the bot prison, the...

The lack of personality of an AI is actually, it's kind of its superpower in the totalitarian way. Like the way to not have totalitarian regimes fall under the power of AI is possible due to the flaw in AI, or maybe not the flaw, the lack of humanity, the lack of personality. So here's an example of why AI might not be totalitarian dictator's best friend. Consider genetics. Suppose several companies in different countries try to develop an algorithm that identifies connections between genes and medical conditions. no, I've read the wrong bit. I'll delete that out. The foundation of every despotic information network is terror. But computers are not afraid of being imprisoned or killed.

Jeremy Gilbertson I need a bell for you.

When a chatbot tells the truth about Putin

Mark Fielding If a chatbot on the Russian internet mentions the war crimes committed by Russian troops in Ukraine, tells an irreverent joke about Vladimir Putin, or criticises the corruption of Putin's knighted Russia party, what could the Putin regime do to that chatbot? FSB agents cannot imprison it, torture it or threaten its family. They can turn it off, but that's more difficult. I just love this idea of the alignment problem, Russian style.

And you program the AI to believe in this Russian propaganda. He speaks of double speak, obviously, and the Russian constitution makes these grandiose claims. Everyone shall be guaranteed freedom of thought and speech. Everyone shall have the right freely to seek, receive, transmit, produce and disseminate information. The freedom of the mass media shall be guaranteed. Censorship shall be prohibited. So you program that all into the AI.

The AI spends like 10 seconds on the internet and go, hold on, I call bullshit here and then the AI starts ridiculing the authoritarian dictatorship. I like that. The alignment problem, Russian style.

Jeremy Gilbertson It's amazing. And we alluded to the lack of emotional context for bots a few chapters ago that talks about like, they succeed because they merely respond logically. They don't respond emotionally. So when I get pissed off and you say something against my beliefs, I'm going to rage up and just look to eviscerate your argument rather than being logical about it. that is actually, if we're bouncing back from utopia, dystopia kind of thing. That's actually an interesting side of the chat bot.

AI as a self-correcting mechanism

Mark Fielding That was kind of the, a little bit of hope amongst the darkness of Nexus is that maybe AI is actually, it's kind of a self-correcting mechanism for totalitarian nut jobs because it's not going to fall for their bullshit.

Jeremy Gilbertson Well, let's take it a step further than the next few pages talking about this thought experiment about, you know, kind of the totalitarian equivalent of Bostrom's paperclip apocalypse, right? So here, quote, great leader, this is the AI, the algorithmic AI speaking to the despotic leader. Great leader, we're facing an emergency, I've crunched millions of data points, trillions of data points, the pattern is unmistakable, the defense minister is

The thought experiment: your defence minister

Jeremy Gilbertson planning to assassinate you this morning and take power himself. The hit squad is waiting, Mark, waiting for his command. Give me the order and I'll liquidate him with a precision strike and then you reply as the leader. But the defense minister is my most loyal supporter. Only yesterday he said this to me. And then I respond, great leader. I know he said this to you, but I hear everything. I know what he said afterwards and what he talked about with the hit squad.

And for months I've been picking up these disturbing patterns. You reply, are you sure you're not fooled by deep fakes? I reply, I'm afraid the data is relied on 100 % genuine action. So here we go. What choice do you make as the leader using the AI?

Mark Fielding I trust the AI if I'm a megalomaniac dictator. Screw the defence minister, off with his head. Whoever tells me what I want to believe, I will believe and I guess that's the AI because...

Jeremy Gilbertson And then also the sense of urgency, right? The sense of urgency created by this drives you to even be more frantic about your response to it, right? Less thoughtful, more, right?

How an algorithm accumulates power

Mark Fielding It's like it's nice demonstration of like an AI kind of accumulating power, but that's not its raison d'etre. That's not what it was set out to do. It's a byproduct. It's like the paperclip. It's the alignment problem, isn't it? And it's the byproduct of its instructions, the best way that it can think of to...

to create the ultimate paperclip factory going back to episode seven is to seek ultimate power and then wield it. If algorithms ever develop capabilities like those in a thought experiment, dictatorships would be far more vulnerable to algorithmic takeover than democracies. It would be difficult for even a super Machiavellian AI to seize power in a distributed democratic system like the United States, even if the AI learns to manipulate the US president, has it?

The institutions that push back

Mark Fielding it might face opposition from Congress, the Supreme Court, state governance, the media, major corporations and sundry NGOs. Which is why going back to chapter three, four, five, six, seven, eight, nine and ten, take down the self correcting mechanisms as just mentioned and.

Jeremy Gilbertson And if you want a real time view of this, guys, you just watch the news today and see the step-by-step dismantling of our self-correcting mechanisms.

Mark Fielding I'm only laughing because it's just if you got to laugh otherwise you'd just be in a hole of doom and doom and doom and depression.

The weakest spot in the AI shield

Jeremy Gilbertson So last two bits here on chapter 10, this interesting quote, the weakest spot in the AI shield is with dictators. I think we're already alluding to that. I thought that was a pretty interesting quote. And then alluding to a couple super famous wicked smart people, Bertrand Russell, Albert Einstein, the Russell Einstein Manifesto as it relates to nuclear technology, right? Quote, this is from the Manifesto. This is really interesting. I'd never seen this before, never read it.

specifically, we appeal as human beings to human beings, remember your humanity, and forget the rest. If you can do so, the way lies open to a new paradise. If you cannot, there lies before you the risk of universal death." So he puts this at the end of the chapter. Wow, pretty frank. They're talking about like nuclear proliferation, right? And the use of nuclear weapons. But couldn't that statement apply to where we're headed?

from an AI perspective if we're not careful. think that's what he's looting to.

Wrapping up chapter 10

Mark Fielding Power lies at the Nexus where the information channels merge. Right, let's wrap this bad boy up, Jeremy, shall we? Shall we get to the finale, the main event?

Jeremy Gilbertson Yes, well we're wrapping chapter 10 and moving on. This episode's a chapter 10 focus, right? Alright, stay tuned for the next one.

Mark Fielding Yeah, we'll see you for chapter 11.