Yuval Noah Harari’s Nexus, Chapter 7: Who Pays for the Surveillance of Modern Life?
Romania's secret police gathered 20 million handwriting samples to trace a letter to Radio Free Europe, and Harari argues we now carry the surveillance devices ourselves.
In this episode · 9 chapters
But as soon as a system comes out, then we figure out, well, how can we game it? And then how can we use technology to game it faster?
Mark and Jeremy read chapter seven of Nexus by Yuval Noah Harari, on surveillance and pattern recognition. They trace a line from registered typewriters in 1970s Romania to the phone in your pocket, asking why Harari keeps returning to the dictators of the past.
They cover Iran's hijab law and face recognition through car windows, the NSA rating Pakistani phone users as likely terrorists, TripAdvisor as peer-to-peer surveillance and China's social credit system. They also test Harari's idea of reputation as a new kind of money.
Chapter 7 of Yuval Noah Harari's Nexus, read on the Thinking On Paper podcast book club, is about surveillance: what regimes once needed millions of human agents to do, a computer network can now do 24 hours a day. Mark Fielding and Jeremy Gilbertson cover Ceaușescu's Romania, Iran's use of face recognition to enforce hijab rules, TripAdvisor ratings and social credit.
How did Ceaușescu's Romania track down letter writers?
In 1970s Romania, after a few citizens wrote to Radio Free Europe, Ceaușescu’s regime collected handwriting samples and required typewriters to be registered to find them.
Why does Harari say we carry our own surveillance agents?
Today the agents are digital and we buy them ourselves. Harari: “Citizens pay for the agents on our own initiative, and we carry them with us wherever we go.”
How did Iran use face recognition to enforce hijab rules?
Iran used face recognition through car windows to enforce hijab rules, and a stricter law allows women who fail to wear the hijab to be punished with heavy fines and up to 10 years in prison.
How could TripAdvisor ratings and social credit change daily life?
Peer-to-peer ratings such as TripAdvisor turn everyone into a monitor. Combined with social credit, Harari warns, the result could “turn life into a never-ending job interview.”
Why could even a benign surveillance network harm us?
Even a benign network may harm us, Harari argues, because it is “always on” and takes away “our opportunities to disconnect.”
Where Mark and Jeremy disagree
The hosts cannot settle whether TripAdvisor-style ratings count as a self-correcting mechanism. Jeremy names the point he fears most: tracking data matched with real-time biological data, such as heart rate.
Links from this episode
Questions
What is chapter 7 of Harari's Nexus about?
Chapter 7 of Yuval Noah Harari's Nexus, read on the Thinking On Paper podcast book club, is about surveillance: what regimes once needed millions of human agents to do, a computer network can now do 24 hours a day.
What examples of surveillance do Mark and Jeremy discuss?
They cover Iran's hijab law and face recognition through car windows, the NSA rating Pakistani phone users as likely terrorists, TripAdvisor as peer-to-peer surveillance and China's social credit system. Mark Fielding and Jeremy Gilbertson cover Ceaușescu's Romania, Iran's use of face recognition to enforce hijab rules, TripAdvisor ratings and social credit.
How does modern surveillance compare with 1970s Romania?
They trace a line from registered typewriters in 1970s Romania to the phone in your pocket, asking why Harari keeps returning to the dictators of the past.
Transcript
Mark Fielding Disruptors and curious minds, CEOs, founders, book lovers. I'm Mark, this is Jeremy, and this is a Thinking on Paper book club where critical thinkers like you come to read between the lines and work out the relationship between humanity and machine, between technology and what it means to be human. No pressure. We're reading Nexus by Yuval Noah Harari. We're on chapter seven, and I don't know if I'm paranoid.
Maybe I am, maybe I'm not, but I'm sure that everybody is watching me. Everybody is monitoring my every movement, everything I say, everything I buy, everywhere I go, every machine I interact with, every person I speak to. It's recorded in a database somewhere waiting for the AIs, plural, to come in, sift through that data, and well, I don't know what they're gonna do with it all, but that's gonna happen.
We're talking about pattern recognition and the surveillance of modern humanity. Aren't we, Jeremy? Chapter seven, first impressions. Oh, by the way, if you're new, like, subscribe, please. Yeah, there you go, thank you. And if you aren't scared out of your brains yet, hang with us, because we're unpacking chapter 11, entitled Relentless, the network is already on. It's scary, it's a scary chapter, and I kept questioning my, am I being paranoid?
Jeremy Gilbertson Should we have this conversation? We'll scare people. I think the idea is that just enough scare to inform, right, and spark some thinking and that sort of thing. I mean, I think chapter seven, I think the most informed individuals are kind of aware of the surveillance that is going on in our world right now. And like Mark said, like and subscribe. Thinking on paper.xyz, you can see all the other stuff we're doing.
Yeah, recently in the news, Apple had to pay, what, 90 million bucks, which is a drop in the damn bucket, because Siri was listening to our conversations without us knowing it and using that information to serve us with ads and do all the fun things. Guess what, Mark, according to that payout, according to that payout, which I think I heard was like two hours of profit for Apple comes down to 20 bucks a person.
20 bucks a person. Your privacy, Mark, is worth $20. How about that? Well, that's why I don't have an Apple, but $20, the cost of my privacy, yeah. Yeah, so first impressions, let's dive right in. He always starts kind of high level and then trickles down into deeper things. A couple of things that I circled and highlighted just on the front end of this thing is bureaucrats' role in monitoring populations, right?
Since the beginning of time
Jeremy Gilbertson Since the beginning of time, really, there'd be just liaisons of the crown, liaisons of the ruling party that kind of infiltrate and figure out what's going on in all these remote villages and that sort of thing. And really, we're doing the same thing just with better technology now. It's gathering data, it's analyzing patterns, it's seeing if individuals are doing things according to the script or off script, right?
Who writes the script? That's the thing, right? That's the thing. So I don't know, I should have done this in my show prep, but this guy, Nikolai, how do you pronounce his last name? Ceausescu. Ceausescu, I should know that. It's from the 70s, I was born in 76. You're American, I'm European, I know all too damn well who that guy is, unfortunately, but maybe he didn't hit the American media as much as it did over here.
And that's the definition of thinking on paper covering both sides of the coin, guys, for the feeble-minded... Americans. Blinders Americans, right? Yes, something slipped through the cracks for me, but let's break this thing down. So 1970s, we're in Romania, there's a secret police, just like every diabolical regime has a secret police, right? And non-diabolical regime, let's be balanced.
Yeah, today's regimes too, and we can talk about that. So a few Romanians get together, they decide to write to Radio Free Europe, they're trying to break through a couple of things that they see going wrong. It causes this guy Nicolai to literally collect 20 million handwriting samples to find the culprit. What a great use of funds, what a great use of time and energy.
Also requiring the registration of state-owned typewriters, and Mark, if you had a typewriter in Romania in the 1970s, you'd be required to turn in its fingerprint, which is, I guess, that little thing that all typewriters have that, how it types, is that what that is? Yeah, and you've just painted a really good picture of what it was. I mean, obviously taking people's handwriting, having to register to use a typewriter, and what do you do with all that data?
Mark Fielding It's not, it wasn't so much the surveillance, it was the fear that it instilled, and that, like, Romania in the 70s, like Charles Chessie, it was all about fear. Fear, fear, fear. Okay, are they watching you? They might be. They've got your fingerprints, they've got your typewriter, they've got everything, you know, so it's fear, fear-mongering, and that's what. Terrorism.
Jeremy Gilbertson Stalin. Just like Stalin. We talked about that last time. Very similar, right? And it's like, yeah, to live in that regime, or to live in that world where that's happening, it's gotta be crazy, and it's gotta, like, I don't know, the scary thing is- I've got a question for you. Okay, so people who are just tuning into chapter seven haven't listened to chapter six, five, four, three, two, one.
Why Yuval keeps referencing it
Mark Fielding Why do you think that Yuval keeps referencing these horrible dictators from our past? In the last chapter, we were talking about Stalin, we've spoken about Nero, we've spoken about, now, Ceausescu. Why do you think he keeps bringing it up over and over again? The thing that's really interesting to me with that question, and speaking of questions, drop them in the chat, guys. We'll jump on them as soon as we see them.
Jeremy Gilbertson These are patterns, and the reason why he brings up these individuals using these patterns, they're the same patterns, man. Like, it's the same framework, the same patterns that are being used, and they're actually being used today, and they're being implemented today. I mean, especially what we're starting to witness here in the US, a lot of these things are being implemented, and it's coming out of a similar playbook, and I think a lot of people are not in tune with that.
I'm in tune with that more because I'm reading this book. It's super timely, and it's super scary. Like, it's a bit frightening, but understanding that the patterns are recognizable, just like AI recognizes patterns, we can recognize patterns and kind of predict where this train's going in a way. Did I answer your question? You did, yeah.
Mark Fielding I think that he keeps bringing up these because we kind of walk a very fine line, humanity does, and we're always on the verge of falling off that line one way or the other, and he keeps bringing up these nefarious dictators from our past to remind us of how often and how easily and how readily we fall the wrong way off this line, and it's all just hammering home, hammering home, hammering home, the information networks, the analysis of data that we're just going into this age where there was two barriers to this kind of mass surveillance in the past, and that was there were legal limits placed on it and there were technical boundaries placed on it to various degrees, and the technical boundaries are no more or will soon be no more.
Jeremy Gilbertson The things that stopped Stalin, the things that stopped Ceaușescu, the things that stopped Nero going right back, they no longer or will soon no longer exist. Well, but check this out. This hit me right between the eyeballs, man, in a big way. So you look at, imagine being in Romania in the 70s imagining being in Russia in the 20s, the 30s, whatever, 40s, yeah, I mean, just imagine Germany in the 30s and 40s, that kind of thing, but guess what?
These surveillance systems, these things that were put in place were done in a not very scalable and a little bit of a clunky way, but what do we do now? We carry these things around that enable all of the shit that we were trying to avoid back in the day. That stuff is now automatically enabled by devices. And I keep pointing back to this trade-off, and we talk about it a lot, this security for convenience trade-off, right?
Waze
Jeremy Gilbertson And I use Waze as an example, not because I hate Waze, because I put in Waze a lot and it gets me from point A to point B, but in doing so, my data on where I go, when I go, how I go, how fast I go is going somewhere else, but it's easier than me pulling out Rand McNally's Atlas and trying to get to my business meeting, right? Sorry, I just got distracted by a thought I had. You mentioned Germany in the 1930s, and I just wanna throw a book that might be for you.
Mark Fielding It's called Hitlerland, and it's by a guy called Andrew Nagorski. And it's basically letters and correspondence written by people in Germany in the 1930s about Hitler and the German government. And it's fascinating. Like in a real time.
Yeah, it's all like real documents. Oh my gosh, wow. And how they were thinking and how they were reacting to it. And almost systematically across the board, you can probably imagine what they were all saying. Not that bad. Right, yeah. It's like blowing a frog in water. Are we frogs in a pot of water that's slowly been getting cranked up temperature-wise? But yeah, so Hitlerland, read that.
But jumping back to what you were saying, I wanna read a quote. By 2024, we are getting close to the point when a ubiquitous computer network can follow the population of entire countries 24 hours a day. This network doesn't need to hire and train millions of human agents to follow us around. It relies on digital agents instead. And the network doesn't even need to pay for these digital agents.
Citizens pay for the agents
Mark Fielding Citizens pay for the agents on our own initiative, and we carry them with us wherever we go. And obviously I know that. We all know this. And you mentioned earlier with Apple, they were, okay, so Siri is listening. So it's not, like, am I paranoid that when I speak about skateboards, suddenly I get adverts for skateboards everywhere I look? Is that a coincidence? Is that my paranoia? Is that really happening?
Jeremy Gilbertson And I know we keep, everyone knows that. Oh, it's happening. Sorry to interrupt, Mark. It's happening for sure, yeah. I have a personal kind of, I like to talk about 90s, 1984 and George Orwell and the surveillance state a lot. And I think, oh, I'm really cool because I'm referencing George Orwell. I'm referencing 1984. Look at me. My ego takes over. And then Brave New World, Aldous Huxley.
And then, but I'm just doing, I'm carrying the phone in my pocket. I am the creator of my own demise. I am the creator of my own dislike for the system. What do I do? How many things knowingly do we do that are bad for us? So check this out. Like we had the X-Prize, a technical prize director for Quantum onto one of our episodes. And we referenced Peter Diamandis because he was one of the folks that got X-Prize going.
Peter Diamandis
Jeremy Gilbertson And I saw a quick interview with Peter Diamandis on, it was a social media clip. And he talks about like sugar, like in us eating sugar. And like back in the day, I don't know if these numbers are super accurate, but it's an order of magnitude. Back in the day, we would eat one pound of sugar every year as a society. And now we're upwards of eating a hundred pounds of sugar every year, right? And we know sugar is bad for us, but what do I do? Like I'm a cereal, I love cereal, man. Like that's my thing, you know? And a nice big bowl of cereal.
Like I'm a kid, man. But piles of sugar in that. But I'm like, you know what? It's all good. You know, I'm just like one bowl of cereal will kill me, but yeah, maybe 4,000 bowls of cereal might. Well, it's compounding. You're compounding, compounding, compounding. Diamandis has been drinking from the Brian Johnson tap. I think he's big into longevity now. He's doing a lot of, I think that X-Prize might be moving into longevity science.
He wants to live forever. I think Peter Diamandis. Well, let's talk about more known, some known. Yeah, let's get back on track. Yeah, more known surveillance kind of stuff, right? So we went through the idea that these regimes went to great lengths and great resources to collect and surveil people. Nowadays, the resources are things that we play with every day, which is kind of ironic. This is really interesting.
And I don't know if this is gonna automatically shut down our podcast or like scramble us, right, when I say this, but in the book, he references the NSA, right? And the NSA's monitoring of quote-unquote terrorist organizations, right? And there's a quote in here. I'm gonna paraphrase it, but mass surveillance of Pakistan's mobile phone network, 55 million people, and this surveillance network attempts to rate their likelihood as being a terrorist or not, okay?
Define terrorist
Jeremy Gilbertson So we all talk about the ability, like when trust comes up a whole lot, right? So we're, not we, not you and I, but organizations are trusting these machines, trusting these computers, trusting this metadata, right? There's a quote in here, we kill people based on metadata. You know, that's interesting, because then you get to thinking like, can AI be really objective? Can data and algorithms and that sort of thing be really objective in creating the parameters of what it means to be something like a terrorist?
Mark Fielding Maybe. Define terrorist. So one man's terrorist is another person's peacemaker. Absolutely. One of the issues he brings up in the book is that, yeah, you define terrorism and the algorithms go looking for patterns of terrorist behavior, but what is terrorist behavior? It looks different to the person programming the computer. So that's a big worry. I don't want to go to Iran, and I think every podcast, maybe we kind of knock off a country that we won't ever be able to go to, but Iran, what a horrendous state he paints that in.
Jeremy Gilbertson Absolutely, like I knew it was bad, but I didn't realize it was that bad. Are you going to the hijab law? Yeah. Yeah. Yeah, there's some pretty crazy stats in with that related to these laws that were based on the fact of women's heads being covered up or being offensive for not being covered up, and again, going back to intersubjective realities, to pull groups of people together in the balance of truth and order, right?
This one lying more on order than truth necessarily, but check it out. 133,000 text messages, immobilized vehicles for two weeks, confiscated 2,000 cars, sent 4,000 people to trial with the potential of up to 10 years in prison for not covering up their head, and they were doing this in cars, right? They're not out in public.
Mark Fielding They're driving their car down the road, right? So the surveillance capability of catching people without their hijabs on, like, yeah, that's a hell of an illustration of kind of their surveillance piece that's happening. Yeah, they were literally using face recognition algorithms through car windows as they were driving along and then finding that person and then texting them before they even got out of the car with a fine or with a warning or however it might be, and then in September 23, on the anniversary of Mahamandali's death, Iran's parliament passed a new and stricter Hijab Bill.
The new law
Mark Fielding According to the new law, women who fail to wear the hijab can be punished by heavy fines and up to 10 years in prison. They face additional penalties, including confiscation of cars and communication devices, driving bans, deductions in salary and employment benefits, dismissal from work, and prohibition from accessing banking services. So you mentioned Brave New World, Aldous Huxley. I automatically go to the pop culture version of that, more digestible version of that with Wesley Snipes and Sylvester Stallone and Demolition Man.
Jeremy Gilbertson And, you know, all I think about is like- The porn we heard with that, Demolition Man. All I think of, you've seen that one, right? Yeah. Yeah, so you're sitting there and, you know, John Spartan's character says, you know, F-bomb or whatever, and all of a sudden out of some random speaker, John Spartan, you've been assigned four demerits for breaking the code of X, Y, Z. And like, that casts a hell of a vision to like, you know, we always talk about, you know, looking at science fiction stories as a projection of future realities.
And some of them come true, some of them don't. Some of them are just funky enough to end up being something that actually happens. Yeah, wow. Where am I going with this? Let's see. Are you going towards surveillance, capitalism, biometric capitalism, the end of privacy? Is that where you're going? Ooh, let me read you this quote. Let me read you this quote. This was pretty good. As fish live in water, humans live in a digital bureaucracy, constantly inhaling and exhaling data.
Facts, spot on, like, that makes me like, that makes me claustrophobic, just like, thinking about it, but it's super true, man. Like, so that's the piece of the puzzle, right? And from there, it kind of, go ahead, Mark. You were going to say something. I said, say it again for the people at the back. Oh my gosh, yeah. We're swimming through a digital bureaucracy, friends and neighbors.
We are sucking in stuff as we swipe, swipe, swipe, or we get fed on autoplay, which we talked about last episode. Clockwork Orange, our eyes are wide open, right? We're auto-fed stuff. So much data coming at us, but then there's so much data coming out of our interactions with that data. Yeah, piles of it. Last week, we spoke about the end of humanity or human history or our part in human history.
Mark Fielding And this week, we talk about the end of privacy. And one of the parts of chapter seven, which I took great pleasure in reading about was the peer-to-peer surveillance systems and TripAdvisor especially. And how much TripAdvisor and peer-to-peer surveillance systems like TripAdvisor have changed our relationships is absolutely mind-bending because I don't know what percentage, it's very, very high.
TripAdvisor
Mark Fielding People go on holiday, they check TripAdvisor. They go to a restaurant, they check TripAdvisor. They go to a bar, they check TripAdvisor. Me one shortcut, me one shortcut. Someone to think for themselves, TripAdvisor says this restaurant is good, even though the score's been made by outliers who either love it or hate it, and most probably hate it, so you probably shouldn't trust it.
But it's got waiters and waitresses scared to, although I have to say that I don't think TripAdvisor's quite reached France because the waiters and waitresses don't give a crap how they treat you, but beyond the borders of my country I live in, yeah. No, it's, yeah, and how does that affect things in the whole, you know, there's a construct of like, it's easier to leave a bad review than a good review, right?
Jeremy Gilbertson So if you leave a good one, it's like, great job. If you leave a bad, oh, I walked in and this guy, there was a hair in my salad, and then the fork was dull, and my napkin was slightly tinted a different color than my pants, and like, it's just easier, it's facts. It's easier to say something negative about something than something positive. So those large systems, and could that be a self-correcting mechanism, though?
Mark Fielding That's an interesting piece. Like, could that rating system in a way be labeled, because we talk about self-correcting mechanisms. Our host and author talks about self-correcting mechanisms. Is that one? No. You didn't sound very confident in that response. I don't think, so a self-correcting mechanism, going back to chapter five, four, three, two, and one, allows us to learn from our mistakes, to make mistakes, and make the system better.
I don't see how, it kind of removes human nature from the conversation a little bit. If you've got peer-to-peer surveillance systems akin to TripAdvisor, where you're biometrically hooked up to something, it can track your every mood and how you feel, and your emotions related to a restaurant, to an experience, to a person. Well, maybe that is self-correcting, yeah, like it would. I take it back, I don't know.
Jeremy Gilbertson I always think about, I guess eBay was the first one to kind of fold that, how do we make sure people on an open platform do the right thing? How do we create trust in a way that people don't see each other in person, right? And that was the review system, that was the rating system. But as soon as a system comes out, then we figure out, well, how can we game it? And then how can we use technology to game it faster?
Maybe that's when a lot of this stuff happens. I'm gonna tell you what, Mark. The one thing that's pretty scary, and I wrote on here, this is probably gonna show it backwards, but I wrote on here inflection point. And the inflection point, I think, is when this data, this information that's being captured, where we go, my eyeballs looking at a website, my eyeballs looking at a social media post being tracked, whether it's Neuralink or whether it's a smartphone or whatever it is, when that data is actually coupled and matched with the real-time biological data, that's when it's scary.
So like your heart goes from steady resting at 70 beats per minute to 175 while watching Fox News. Ooh, this one's, we really got Mark now. Let's keep firing this stuff at him or whatever it is. And it could be CNN, it could be whatever thing you're talking about. But how do you feel about that? I use Brave Browser and I have no recommended videos on YouTube and it doesn't autoplay the next video.
Mark Fielding I'm building in, what's the word? Automated security provisions, automated privacy conditions. Yeah, which I don't have to worry about, I don't have to think about it because they're there. So if I do go on YouTube and watch 20 minutes of Fox News, God forbid, I won't be hounded by the algorithms who try to change what I watch. Like for the next video, I'm gonna have to choose for myself what I want to watch.
I'm gonna have to search for it manually. There are no recommendations. So what do I think about it? I think it's pretty scary and I'll go back to the beginning and I think I started with, this is scary. Unfortunately, social credit algorithms combined with ubiquitous surveillance technology now threaten to merge all status competitions into a single never- ending race. Even in their own homes or while trying to enjoy a relaxed vacation, people would have to be extremely careful about every deed and word if they were performing on stage in front of millions.
This could create an incredibly stressful lifestyle, destructive to people's wellbeing as well as to the functioning of society. If digital bureaucrats use a precise point system to keep tabs on everybody all the time, the emerging reputation market could annihilate privacy and control people far more tightly than the money market ever did. Even if the network is potentially benign, the very fact that it is always on might be damaging to organic entities like humans because it will take away our opportunities to disconnect.
China
Jeremy Gilbertson Well, this is happening already. China has the social credit system. I don't know a ton about it, just tangentially aware of what's going on over there, but kind of the idea of assigning points for everything. Harari talks about it as being this new kind of money, this new value exchange, right? A standardization, he calls it, standardized valuation of the reputation market. And as I think about that initially, the idea of creating value to things that don't have inherent financial value could be interesting in one way, like doing something really good for the world, right? Helping a grandma cross the road, basically teaching people who don't know how to speak French in
France to do so, doing these things for the betterment and for the good. But then I start thinking about like, okay, how is that gonna be gamed and played? And this quote, a comprehensive social credit system, you referenced this, will annihilate privacy and effectively turn life into a never-ending job interview. That sounds like hell to me. Oh my God, right? Yeah. Jeez. That sounds like a Black Mirror episode, doesn't it?
Mark Fielding Where you help the old lady across the road and you get 10 tokens and then you walk into a shop and you hold the door for someone, you get 10 tokens and you say, please, you say, thank you, you get 10 tokens and everyone's walking around this Black Mirror episode with the little bars, energy bars going up above their heads, and then when they get home, they cash in those points for, I don't know, some kind of strange reward.
Jeremy Gilbertson Didn't religion, doesn't religion do this to a certain extent, right? So if you're an ass. The Catholic Church again, Jeremy. If you're an ass, if you throw a beer bottle at somebody's head, then you go into church the next day and you confess and I go, Mark, I did this thing yesterday, you assigned me a couple of things, like some Hail Marys, and I'm, listen, I'm.




