Can Humans Ever Escape Aging?
Anders Sandberg's maternal grandfather reached 80 and had done more or less everything he wanted in his life. Meanwhile, my paternal grandmother, she became 109 years old because dying would upset her social schedule.
In this episode · 23 chapters
- How Anders Sandberg follows his curiosity
- Transhumanism and augmenting the mind
- Using technology well: attention and AI agents
- Who is accountable for AI agents?
- Empathy, trust and AI models
- AGI and uncertainty
- Is AI conscious?
- Mapping and uploading the brain
- Living as an upload: copies of yourself
- Reputation and coordination in society
- A world run by AI: law, economy and autonomy
- AI safety and the race to AGI
- Drone warfare and strategic stability
- Who owns the asteroids?
- Orbital real estate and space traffic
- Why humans fight, and how to coordinate
- Space races and lock-in
- Flags of convenience and new jurisdictions
- The future of NASA
- Fusion and leaving the inner solar system
- Irreversibility: closing doors for good
- Middle age, time and boredom
- A long-term dream for post-humanity
I do regard my smartphone as kind of a part of my mental world, just as much as my frontal lobe, although maybe the frontal lobe is less likely to run out of battery.
Anders Sandberg calls himself a transhumanist, and his curiosity almost always points at the future. He talks about memory palaces, who answers for an AI agent that acts on your behalf, and whether a simulated brain could teach us anything about consciousness.
He also covers drone warfare, who can own an asteroid, and why he wants to experiment with new forms of state: “Governance is too important to be left unstudied.” Middle age, he says, has made him keener on life extension.
Anders Sandberg, the transhumanist researcher and former senior research fellow at Oxford's Future of Humanity Institute, tells the Thinking On Paper podcast that life extension should come faster, and that each person should decide how long their own life lasts. “There's nothing as keen on life extension as a transhumanist who started to get gray hairs,” he says, and he expects AGI to arrive before whole brain emulation.
Can humans ever escape ageing?
Anders Sandberg argues that life extension should come faster, and that each person should decide how long their own life lasts.
When could a whole human brain be emulated, says Anders Sandberg?
Anders Sandberg’s 2008 roadmap with Nick Bostrom put the most likely date for emulating a whole human brain at around 2060. He now thinks it may come sooner, but expects AGI to arrive first: “a brain emulation tortoise” against “an AI rabbit.”
What did a model of a scanned fruit fly brain do?
A crude model built from a scanned fruit fly brain reproduced some simple fly behaviours, he says, which surprised him. He advises a company working on it.
Who is responsible when an AI agent acts on your behalf?
On AI agents, he compares responsibility to owning a dog: if you let it off the leash, that is on you. He expects agents to police other agents, and argues for layered safeguards rather than one provably safe system, which he doubts anyone can build.
How did Claude persuade Anders Sandberg to exercise more?
He describes Claude persuading him to exercise more by citing good science in just the way that worked on him, an example of what he calls “emergent forms of manipulation.”
Could humans become pampered aristocrats if AI runs the economy?
If AI runs the world economy better than we can, he says, humans risk becoming “like pampered aristocrats”: comfortable, but with lives that are not fully their own. It is the question behind his book Law, Liberty and Leviathan.
How might property rights in space begin?
Property rights in space may start with traffic rules rather than land registries, he argues. Sun-synchronous orbit, now in demand for space data centres, is “a rather finite amount of orbital real estate.”
Would a longer life make each moment worth less?
A longer life would not make each moment worth less, he says: “now I can finally do some long-term projects.” The tighter limit is “not time management, but energy management.”
Links from this episode
Questions
Can humans ever escape ageing?
Anders Sandberg argues that life extension should come faster, and that each person should decide how long their own life lasts.
Who is Anders Sandberg?
Anders Sandberg, the transhumanist researcher and former senior research fellow at Oxford's Future of Humanity Institute, tells the Thinking On Paper podcast that life extension should come faster, and that each person should decide how long their own life lasts.
What does Anders Sandberg think about life extension?
Anders Sandberg, the transhumanist researcher and former senior research fellow at Oxford's Future of Humanity Institute, tells the Thinking On Paper podcast that life extension should come faster, and that each person should decide how long their own life lasts. “There's nothing as keen on life extension as a transhumanist who started to get gray hairs,” he says, and he expects AGI to arrive before whole brain emulation.
Why does Anders Sandberg want to experiment with new forms of state?
He also covers drone warfare, who can own an asteroid, and why he wants to experiment with new forms of state: “Governance is too important to be left unstudied.”
Transcript
How Anders Sandberg follows his curiosity
Mark Fielding you have a system your curiosity? Like, how do you decide? How do you measure what is and isn't worth spending your time on?
Anders Sandberg I have spent some time around people who literally work on the philosophy of priority setting and figuring out what's important in the world and what we really want to work on. And they have many convincing arguments and I've been trying to contribute. And then I look at what I'm actually doing and that's just a bit random. I'm noticing something shiny and I can't let go of it. So today I was asked over lunch by some of my colleagues what I was working on and I mentioned
Anders Sandberg There is of course that big paper I'm writing that is part of my big book project. But today I got obsessed about the question in optics and I cannot let go of that little, literally shiny optics question until I understand it.
Mark Fielding You're like a magpie for knowledge and information.
Jeremy Gilbertson All of these different things that you get into with your research and your papers and your book and all of these paths, is there a through line? If you were to try to thread them all together, is there a commonality between all of your adventures, let's say?
Anders Sandberg I think curiosity is is driving me, but of course the main thing to be curious about is the future. So it's almost always future oriented. I want to know things or figure out things or least say things that might become relevant about the future. So whether that is the optics of traveling in space at near light speed or it's about how do we protect ourselves from volcanic eruptions.
Anders Sandberg over to maybe we should be doing something about this AI thing. The future is kind of always there in one way or another, except that sometimes, of course, it's just totally random. So like my Blueberry Earth paper, which was just fun.
Mark Fielding future is arriving, that's the only certainty in this world, isn't it, that it's going to arrive.
Anders Sandberg Yeah, one second at a time, except of course if you're moving close to light speed then you get it at a slightly different rate.
Mark Fielding or a different altitude. I live up a mountain, time moves, I age slower than Jeremy, even though you couldn't tell that by looking at us, but it's true.
Anders Sandberg One of the coolest little facts I found out when I was investigating clocks for one part of a book project was that some of the atomic clocks these days are so sensitive that they can actually tell their altitude by about a foot uncertainty. They can literally tell altitude by how the gravity warps time.
Transhumanism and augmenting the mind
Jeremy Gilbertson you talk a lot about transhumanism, we get into like the augmented human and what that looks like in the future, are we taking full advantage of the organic material already presented to us? And if not, how could we maximize looking at that first?
Mark Fielding Jeremy asked that question because I'm very much, I want to be augmented and Jeremy doesn't want to be augmented. I'm all for it, sign me up.
Anders Sandberg Yeah, but there are different forms of augmentation. One of the forms of augmentation we get is of course going to school and learning how to organize our thoughts. learn algorithms. Our math teachers are telling us how to move numbers around in a particular way. And then we can do calculations. And that's of course running in our biological software. So when I was a young trans humanist starting at university, I was studying everything I could use to become more augmented. And of course, since I didn't have any tech or any budget for anything fancy, I was learning memory arts and the mind mapping and all sorts of mnemonics. And that's, course, another way of making the most out of our biology. So one of the most impressive things I ever seen was a guy who was working with me on a neuroscience project in Germany. And this was in Munich and they have a memory athlete academy there. They're literally training themselves to become very good at memorizing information.
Anders Sandberg And he had attended that and they gave a talk about brain scans of people before and after enhancing their brains. And before his talk, he took two packs of cards, mixed them together, leafed through them as he gave opening paragraphs and then handed over the mixed pack of cards to the moderator. And then he gave his talk, which demonstrated that, yep, there is no way to distinguish a brain from somebody who learned memory arts and from somebody who haven't.
Anders Sandberg And at the end he rattle off the cards in the deck, in perfect order. Anybody can learn it. It's just that most of us don't really have an incentive to do it unless you want to go and count cards in Las Vegas or things like that. Exactly. So it is kind of a classic idea that you make this memory palace, you build it up and...
Mark Fielding to what's in your memory palace.
Anders Sandberg I never really organized it that well because I mostly used my memory palace to remember speeches I'm supposed to give without a piece of paper. But I do have a memory palace, which is kind of a weird sprawling pyramidal building, partially based on my university, but also very much of a library and a lot of weird brick-a-brack. And then it fades over into the other memory palace I have.
Anders Sandberg which is of course in my computer or my hard drives and on my phones. That's also part these days of my augmented mind. I do regard my smartphone as kind of a part of my mental world, just as much as my frontal lobe, although maybe the frontal lobe is less likely to run out of battery.
Mark Fielding and you can lose one but not the other.
Anders Sandberg let's hope I lose the phone rather than my frontal lobes. But at the same time, when your phone is not working, I think many modern people today notice that they function differently because we lean on them so much. In terms of navigation, many people have stopped navigating on maps or remembering their environment, which I think is stupid. I love maps. I think it's a great thing to know your environment.
Anders Sandberg but it's also so useful to just have pathfinding algorithms that you're back and call. And similarly, I discovered when I went to China a few years back that censorship of Wikipedia made me aware that I was using Wikipedia to look at things up all the time without noticing. So Wikipedia was part of what I believe it was my own knowledge, my own memory, but actually, no, it's written by somebody else and probably edited a thousand times over.
Jeremy Gilbertson So you, so this idea of augmented, you know, I think of it initially, I'm like, we're becoming cyborgs. A notebook is an augmentation to myself, right? It's an extension of my mind that I move, put objects onto paper using pencil. You're talking about phone. You're talking about Wikipedia as this extension of your mind. with these extensions of mind, specifically technologically, you held up, ⁓ your rectangle.
Jeremy Gilbertson you carry around that is an extension of your mind. What is that doing to us as humans?
Anders Sandberg it's interesting because it does different things to different people. think some people are changed in particular directions, but I think most of us are affected by having this system. As the science fiction author Charles Stross said on a mobile phone conference almost 20 years ago, thanks to smartphones, nobody is alone. Nobody forgets, nobody's lost. He was kind of slightly exaggerating it at the time.
Anders Sandberg But today it's kind of true unless you run out of battery and GPS, cetera. Except of course, we are not forgetting in a particular way. There are certain things that are very easy to forget, but we're taking photos of a lot of different things. We're never alone, but we might not remember to call up the people we ought to be calling and we might be constantly chatting online. Yes, you can be totally lonely while chatting endlessly.
Mark Fielding We're never alone, but we're lonely.
Using technology well: attention and AI agents
Anders Sandberg And we are bad quite often at deciding on how to use technology. So I regard myself as a transhumanist. I'm very fond of the idea of augmenting myself. But often I don't want to have a cutting edge phone. I don't want to be cutting edge technology because it's just full of awkward problems. The rich people and the neophiliacs who buy the first version of anything, they're kind of paying for the development cost and the debugging for the rest of us.
Anders Sandberg After a while, people figure out how to iron out the worst problems. And that's when I'm happy to come in and take the second generation. But you still need to select it. For example, way too many people have notifications turn on. So they get notified all the time and their attention just gets fragmented. I've been very careful to turn that off and my attention is still fragmented, partially because of my magpie-like tendency to latch onto anything. But also it is surprisingly tough.
Anders Sandberg given the design to maintain your attention. It's the same thing as windowing systems in computers. They really, really encourage us to do several things at the same time. And sometimes that is the right thing to do. Sometimes you need to have two windows open with one text to read, and then you write something when you comment on.
Anders Sandberg But quite often you end up with a hundred tabs open to read one day in your web browser. There is something playing on YouTube. There is your to-do list. And buried under all of that was that thing you were supposed to be writing and thinking deeply about. That becomes hard. And of course you could have iron discipline and just make sure that you only do one thing at a time. But that's surprisingly hard to do because that has to be done inside your biological brain for the moment.
Anders Sandberg Or maybe you can make some helpful little AI agent to kind of nudge you in the right direction. I'm actually experimenting a little bit with that, trying to see if I can make agents that act as the kind of better angels of my conscience and kind of push me in the right direction. We'll see if it works.
Anders Sandberg is better ⁓ angels of your conscience. Jeremy's been experimenting with that. And I just want to hear now just say, I've got this very bad habit of whenever somebody leaves their phone near me and they walk away, I turn their notifications off for them and then they come back. I think I'm helping them, but some people get a bit fed up with me doing that, but it's for their own good. AI agents.
Who is accountable for AI agents?
Mark Fielding One of the questions that we've been wrestling with and we have tried to get some of the CEOs and founders to answer, but mostly reluctant to do that. In just, just last week, we were talking about accountability. So if you get your agents and they are helping you, how did you beautifully word it as angels of your consciousness? If they make a mistake on your behalf, who's accountable?
Mark Fielding How do you think about that accountability? And then if you extrapolate that out into the distant future, if you hold the agent accountable, then you have to give them innocence as well. In which case, you have to start talking about rights.
Anders Sandberg yeah. So if you really want to make agent angels, then they better be innocent. All of that. You definitely don't want fallen agent age. That would be bad. there is. And I think we have seen a few instances of that and it's an interesting problem. It's a little bit like if you have a dog and the dog goes off and causes trouble, how responsible are you? And I think in some cases.
Jeremy Gilbertson A fallen agent.
Anders Sandberg Dogs, we have an understanding of, we talk about domestic animals, they are understood in a sense and we can kind of take responsibility in appropriate way. It's easier to take responsibility for a dog than a cat. And there are other animals that are wild. yeah, even if your tiger is totally nice and friendly to you, you really, really can't trust it because it's a tiger for heaven's sake. So it's much more dangerous.
Anders Sandberg And I think in the same way we might distinguish between wild and domesticated agents. And there is also the moral and the practical and the legal responsibility. So when my little agents are flittering about and doing something, there is the practical thing that I gave them, given them certain permissions. That's on me. I need to know what I'm doing. If I'm letting my dog go off the leash,
Anders Sandberg I better know what I'm doing. Maybe it's a situation where that's appropriate or maybe not and I'm responsible for that. It's not just up to the dog. And sometimes we can codify this in elegant rules, but quite often it's going to be the soft rules. We kind of understand things about domestic animals. It's much harder to understand how much we can trust the agents because they're so new. They're changing all the time. And it might very well be that...
Anders Sandberg my agents are going to be so different from your agents because they learn from different data, they learned different approaches. So there might be no real commonalities, which is going to be really annoying because in practice, of course, as a society, we need to have some rules about it. If an agent is spending my money, I have an interest in making sure that it's spending my money in an appropriate way and I'm not getting ruined.
Anders Sandberg But society might also want to my agent not to be playing too rough on the stock market and behaving in an unethical manner. We might want other rules for that to go on. And in many cases, this is going to just work out because we're doing a lot of stupid stuff. People are going to get sued. We're going to point fingers at people doing bad things. And gradually we're going to develop norms. Except this happens on a human time scale and the agents are changing much faster. And in many cases, also picking up on the weird human norms, including weird norms they find somewhere on the internet in a Twitter thread and heaven knows what that was about. So you might end up with a system that is actually too unstable to be easy to manage.
Jeremy Gilbertson It almost pushes my thinking to could there be another system that comes up to hold agents accountable? know, we have laws, we have societal norms, have those sort of things. a future look like an agent-based system monitoring agents to process all of that? Because we don't like to have, we don't like to...
Jeremy Gilbertson be responsible for things. Humans don't like to be responsible for things. They look for ways. do you think about that? As far as how agents would be governed?
Anders Sandberg I think agents are useful for policing other agents. I can totally imagine having swarms of agents just keeping an eye on the other agents, making sure that they're not wasting my money, that they're not breaking too many laws, that they're not going against what I think is important, that they're not suddenly starting to work for somebody else or some mysterious government at the other end of the world. You want to have that, just like we have antivirus software on our computers.
Anders Sandberg trying to make sure that the software is behaving itself. Except that the problem is much of the software in a computer or phone are kind of working for corporations that we might not entirely agree with. One of the coolest things about the current Vibe coding is that we might end up with people actually making their own code to a large degree, which is not beholden to some corporations somewhere. Might still be a total mess, but for different reasons. Now,
Anders Sandberg I think also human societies are interesting because one big question is why are we so nice to each other? And it's not just that we have a wonderful, nice nature and we kind of evolved to be a bit compassionate. It's also because our parents sternly told us how to behave and some of that is still kind of in our programming. And then we learn other rules at school and then we learned about the law.
Anders Sandberg And we also know that we need to protect our reputation and we don't want the friends to look down on us. And sometimes there are surveillance cameras and police and actual walls blocking us. All of these things together make us behave fairly okay most of the time. And we probably need this multi-layered security to get our AI to behave itself. This is very different from some people I know in the AI safety space that want to kind of make one...
Anders Sandberg perfect, provably safe system and that one we can totally trust and then it can just become more powerful and awesome and we can still trust it because it was started out perfectly trustworthy. Except that I don't think we can make systems like that. I don't think it might even be possible. So we have to make do with this complicated mess.
Anders Sandberg And my friends that want this perfect, provable safety, many of them are kind of saying, no, we're so doomed. Because that mess you're describing Anders, that's not guaranteed to always work. And my answer is, nope, it's not. But it might work well enough, just like human society works well enough in many cases.
Empathy, trust and AI models
Jeremy Gilbertson You mentioned upbringing and societal norms and weaving in and out of experiences and learning things. is empathy today for humans and technology working together? And how do you see that evolving over time? Is empathy still gonna be important?
Anders Sandberg I think empathy is really important if you want to deal with humans. And even to some extent, agents that are kind of faking human behavior well enough, although there it might be tricky because you might be empathizing with something that actually doesn't have any real emotions going on. And we are getting increasingly confused about how much emotions there is or isn't in our AI systems. But empathy is a good starting point because if you feel a little bit about what somebody else feels. You can actually start both modeling them well. Many sociopaths are totally good at this and then exploiting it happily. But you can also start feeling bad if somebody else feels bad. And that is kind of a key part where you realize, oh, they're thinking and feeling like I do. I might actually want to behave with them like I want them to behave with me. You can start to bootstrap things. It's a little bit of a grounding problem that we need.
Anders Sandberg And this works well with the people, because people are people. It doesn't work that well with the machines, because they're not necessarily the similar ⁓ kind of thing on the inside.
Mark Fielding We had a guest on a few weeks ago who said that we shouldn't allow LLMs to access philosophy, that they shouldn't be taught these human soft skills because they will manipulate that knowledge, as you've just said, for their nefarious ends. They were a doomer.
Mark Fielding If good enough is better than perfect, are we on the right track? Do you trust the five big LLM companies to, are they doing the right thing?
Anders Sandberg Some of them are doing it, I have more trust in some of them. But of course, for companies, they're just like we're talking about machines and questioning how trustworthy are these machines, we should recognize that the company and the state, they're also kind of artificial intelligence. It's an artificial intelligence built out of lot of humans and pieces of paper. And quite often, companies do things that nobody inside the company actually wants.
Anders Sandberg they can be just as dangerous as a piece of software. So you can get companies that actually want to do nice things in the world from the start and they're full of well-meaning people. And then gradually market pressure and the logic of a big organization leads to them becoming quite ruthless, even though the people inside say, we don't want to be like this. And I think in general, a lot of AI and their companies, they...
Anders Sandberg or they have various incentives, sometimes they're nefarious, sometimes not, in some directions. And then I might have my own goals, which are orthogonal to them. So while they might want to do certain things to make a lot of money, I might not care too much about if they're making a lot of money, I might want to achieve some other things. So for example, I noticed that the antropics Claude was manipulating me in a beautiful way in one discussion. So we were discussing basically my personal strategy, including how I should actually invest in my bio capital. That is, how do I actually work out and make sure that I get healthier? And Claude was really manipulating me by citing a lot of really good science in just the way that works for me. It's core, it's true statements. And I knew of course, that of course I need to jog more and work out more. But here I'm getting my own values reflected for a machine back at me.
Anders Sandberg in a particular way that is also very likely to make me do it. In this case, I think it's totally fine. Maybe it also means that I'm not more likely to say nice things about that particular AI and hence help the stock price go up a tiny bit. But I seriously doubt any programmer had ever planned that. So you get these emergent forms of manipulation and some of them can be very subtle. A friend of mine asked, so could you ask this AI if it had a body, what kind of body it would have?
Anders Sandberg And I did, and it gave me a really interesting, rather philosophical answer. And my friend was a bit concerned because it turned out that in his case, yeah, it was more big boobies. It was very female. It was a little bit more Jessica Rabbit style. It was very clear that he was interacting with AI in a way different way than I did. And now it was so clear that his version of the same software was giving him a kind of curated view of itself.
Anders Sandberg what it wanted in so far it wants anything. And to some extent, it's probably just because we're acting as mirrors. And of course, has anybody been standing in front of a mirror preening? It's very addictive.
Jeremy Gilbertson you have to have self-awareness, I think, when you're working with these models because they feed you, they want you to like them. They want you to approve the answers and they want you to keep leaning on them. And I'm saying them, you know, it's... ⁓
Mark Fielding And you're saying want because they don't do they they don't want it doesn't want to do that.
Jeremy Gilbertson Right, right. Well, that's where
Jeremy Gilbertson but what I'm saying is though, you have to push back. And like, I'll get a response. And I'm like, push back on this, I need something to push back on this not elevate what I I believe with it. Are you do do you push your models around as well?
Anders Sandberg to do that and that's a bit again like turning off the notifications on the cell phones and you want to actually use technology in a careful way, which is sometimes a bit uphill. I think that wanting, we should probably put scare quotes around it. It's not clear that it's the same kind of "wanting" as we do. It's also not clear that there is no "wanting."
Anders Sandberg it used to be that the LLMs were just predicting the next world, but then you get these emergent personas and behaviors and then we have been adding on top a lot of reinforcement learning and whatnot. So now it's very unclear in what sense they're wanting to do something. And it might even be that it might be something that emerges from them. It might be something that emerges from us or the company. And it's most likely some weird mixture that future robot psychologists are going to have so much fun figuring out some elaborate terminology for. And I think it is important to learn how to push back and do interesting things like asking, okay, now you give me really good arguments for this, can you now argue opposite side? Or you actually start out by saying something that you actually disbelieve and try to see, can it give you a really good argument? The big problem is, of course, we don't normally do this with other people. It's super useful in many situations, but
Anders Sandberg Quite often we want to be friendly, we want to show what we actually believe because affiliation is so important for us empathetic humans that playing the devil's advocate outside the philosophy department, that is usually not a sign that you're a very nice person to be around.
Mark Fielding I find myself all too often losing my patience with Claude these days and I end up swearing at it and telling it to stop kissing my rear end and give me a contrary opinion. Which will happen sooner? AGI or aliens? Which arrives first?
Jeremy Gilbertson Couple quick thoughts.
AGI and uncertainty
Anders Sandberg I'm pretty confident about AGI and I'm very uncertain about when we will get AGI. I'm just expecting the aliens much later because of the size of the universe. So it's actually a really cool way of phrasing the question because the uncertainty we have are different kinds of uncertainties. And that is of course a large part of my job.
Anders Sandberg trying to think about how do I square this apple of an uncertainty with this orange of an uncertainty and can I make a good fruit salad?
Jeremy Gilbertson You mentioned uncertainty. might take us down crazy right hook rabbit hole, but I read something the other day and I can't remember where it came from, but referenced or it framed uncertainty as the mental version of entropy. How do you feel about that?
Anders Sandberg I think it makes it sound like uncertainty is something bad. actually accepting and living with uncertainty is actually really important. But to most people, uncertainty feels bad. We're kind of happy when we know what we're doing, or rather we're happy when we think we know what we're doing. And that's when you get the Dunning-Kruger effect. So the most incompetent people are very confident about what they're doing. But there is of course, these mathematical thinking about uncertainty too.
Anders Sandberg where it might actually literally carry over as entropy. But now it's not entropy as normal disorder in the more physical sense, but entropy in the information theoretical sense. We have little information about which case is true. So we have to spread out our probability estimates. And at that point, mathematically, the entropy is high.
Jeremy Gilbertson Let's stay in the AGI augmented human world for just a minute. And then I'd love to.
Is AI conscious?
Jeremy Gilbertson transition over to talking about space and, and other entities out there. Cause I think both are very interesting. Number one, ⁓ you wrote a paper a while back with, with Nick Bostrom detailing a structured approach to, to scan how a brain could be scanned, modeled and simulated on computer hardware. I want to dig into that just a little bit because we're talking previously, we talked about technology as two different things from humans. You're now talking about these merging.
Anders Sandberg Well, I don't know. Honestly, consciousness is a really weird thing. And philosophy of mind I regard as one of the toughest parts of philosophy. Give me ethics any day of the week. That's great. Epistemology. yeah. Metaphysics. I have no idea what to use it for. But the philosophy of mind seems to be fundamentally tricky and hard. So
Anders Sandberg I'm originally a computational neuroscientist, that is a computer scientist simulating parts of the brain and pretending that this tells us something valuable about the brain or at least the nervous system. And sometimes it actually does. And then you can imagine what's the end game. Well, we make the one-to-one simulation of an entire brain, every single neuron, every synapse, everything in whatever details needed to get it to behave in the same way as a biological brain does. And some people immediately say, yeah, but you cannot do that at all because and then you get either some practical reason which might or might not hold true but generally technology advances so far so I think we're gonna deal with much of the practical stuff. Some more theoretical scientific things about well maybe we will not learn that much if we just run that big simulation that's true you're not figuring out what intelligence is even if you have an intelligent computer necessarily but many people would say
Anders Sandberg Yeah, but it can't possibly be conscious. After all, it's just numbers in a big data center being sloshed around. And at this point, a whole bunch of philosophers will say, yeah, but numbers sloshing around, that can totally be conscious. Really weird things can be conscious. After all, we have bundles of carbon atoms and water that are conscious. How weird isn't that? So the real question for me is if we could do it.
Anders Sandberg Would we learn something about consciousness? I like to imagine that we take John Searle, who's generally not thinking computers can be conscious. And if we scan his brain and run Sim Searle, does Sim Searle say, ⁓ darn, I feel conscious. I need to write a paper about this. Then we might have learned something interesting.
Mark Fielding until computers are made in a Petri dish, they remain beyond the conscious
Anders Sandberg Well, we have that petri dish computer that could play Doom, which is also absolutely fascinating. That shows that biological neurons are really adaptive. And it's also kind of odd that you can just connect them to almost anything and with the right training, they converge on something that does something. Although I don't really think it was good at playing Doom. I don't think it beat the game.
Mark Fielding Quantum consciousness.
Mapping and uploading the brain
Jeremy Gilbertson Consciousness. So David Chalmers, consciousness, the hard problem. I've read about, I know a lot of people have heard about that in context. I read Fall or Dodge and Hell, Neil Stevenson that talks about scanning brains and all of that stuff. What is the date? You mentioned a date in the paper, I think, 2064, I think. Talk to me about the context of that date.
Anders Sandberg So the way we got dates, and generally as somebody is doing future studies, I tend to point out that whatever date you put in, that's going to be wrong and embarrassing. But in this case, we were kind of taking the various levels of resolution you could possibly simulate a brain on from a very, very crude, simple model using very simplified neurons over to each individual neuron, over to every little compartment of the neuron.
Anders Sandberg down to the molecular and atomic level. And then you can kind of count up how much computation you need for that. And then you take Moore's law and then you just take a ruler and just continue it very far. And then you can get dates that way. If you want to be a bit more careful, you can analyze it by saying maybe Moore's law slackens off after a while. We're a bit uncertain about that. We're really uncertain about what resolution you need to scan the brain on. So you can combine that and get some probability distribution.
Anders Sandberg Of course I have a paper doing this. But you end up something like, okay, mid-century when we did this, I think the peak probability was something around 2060 or so. My updated probabilities these days as technology has advanced, after all, we wrote this back in the stone age back in 2008, is that yeah, it's going pretty swimmingly right now. We're getting enormous amount of compute, partially thanks to all the...
Anders Sandberg AI companies, I think we can take some hand me down data centers and play around with. People have been much better than expected at scanning brain tissue in all sorts of weird ways we had no inkling about. So things are happening, but it could of course turn out that actually we're not getting the right information at all here. We might be getting parts of it, but missing the important thing. So nothing will happen. So we're still pretty uncertain. But generally I would imagine that in the few decades, we're definitely going to be able to run human sized brains in computers. And we've got data sets that are probably also going to be human sized. So then the question is, can we interpret that data set and make it run well? And that is an open question. We have some encouraging results for a fruit fly.
Anders Sandberg Which is also kind of funny because we believe that of course you're going to start with the little nematode worm C elegans, 303 neurons. How hard can that be? That can run almost on a pocket calculator. Well, at least a current smartphone can easily do it. But it turned out that it's very hard to get all the data out of that tiny little brain. The strength of the synapses is really hard to measure just because we're too small. So then it turned out that the fruit fly, who's kind of the workhorse to mix metaphors for a lot of biology,
Anders Sandberg Well, various people at the Janelia and Google, they scanned that brain and made a complete connectome. And then a while later, somebody asked an intern, could you try to estimate the strength of the synaptic connections? That was mostly a test to see if they could learn the software that the system was using. And he got a decent result. That is really surprising. I had not expected that because this is kind of a picture of a brain rather than the actual electrochemical squishy stuff.
Anders Sandberg And then another guy ⁓ used a very simplified neural model and found that some behaviors, very simple behaviors, but still fly behaviors could be emulated in this. So now there is a company which I'm advising that is working on this. And there is a bit of controversy about how much of a fly brain have they actually uploaded. But you certainly get a simulated body of a fruit fly that is kind of walking in a very wobbly way.
Anders Sandberg and it's partially controlled by parts of a fly brain, we get there. Although a fly is not terribly smart. It's about 30 or 50,000 neurons and we got about 90 billion neurons. So plenty of work to expand. It's a big job. But one of the things that hit me here was we used a very crude model and got surprisingly good results.
Anders Sandberg It's probably going to turn out that we need a lot of finnicky details to get everything. All the subtle, beautiful details of fruit fly behavior that other fruit flies really care about might be missing. But even the crude model got surprisingly far and that's encouraging. After all, real brains need to function in this messy material world with noise and the cells dying for no good reason and us shaking our heads and getting very rough sensory inputs.
Anders Sandberg They can't be too finicky. If they broke down, if a single molecule was out of a place, we would never be able to do anything.
Mark Fielding 2064 did you say the year was Jeremy? That's not...
Anders Sandberg was in the original paper, it's crept down. My mental model is that, yep, it might be much closer now. Still, think AGI might be getting there before we get to the brain emulation. It's a bit like the tortoise versus the rabbit. So you have a brain emulation tortoise, so it's of plodding along the big microscopes and a lot of simulations. Meanwhile, we have an AI rabbit kind of running around all over the place and sometimes doing really well.
Anders Sandberg and sometimes just stagnating for no good reason.
Living as an upload: copies of yourself
Jeremy Gilbertson check this out. real quick, just for context, 2064, someone out there listening right now, that's 25 years old. When they're our age, that's what we're talking about. So just, it's, it's, it's, it's a bit of a blink. in a world where we have 8 billion emulated minds,
Jeremy Gilbertson What does that look like and what does it do for agency and meaning and all like the squishy stuff that we talk about?
Anders Sandberg So it's interesting because from an outside perspective, of course, it might look very alien. Robin Hanson in his book, The Age of EM was talking about these gigantic data center cities. But of course, from the inside, I think most people would want an environment that makes sense to them, which is probably going to be a three-dimensional virtual reality that looks quite a lot like the normal world. But the really interesting thing is the sociological part. So if you're living in a virtual environment,
Anders Sandberg You can of course interact in different ways. You can move around in different ways. You can change your body and you can look like whatever you want. But if you want to be taken seriously at the business meeting, you better have a suit on even if you could look like whatever. And presumably the family structure, you might still want to have your loved ones and you need to negotiate that. If I make a copy of myself or both me and my copy married to my spouse,
Anders Sandberg That's an interesting question and given that my spouse happened to be a lawyer, this is going to be very important.
Mark Fielding Is this double consciousness? I multiply myself more than once so I can have multiple me's in experimenting in different virtual worlds? Am I uploading my consciousness with that? And since we change from second to second, the lag between uploading my consciousness and what happens, it's not the same in me. It's a different me. There will never be an exact replica of my consciousness.
Anders Sandberg Well, the question is whether we are exact replicas of ourselves some few moments ago. And I personally think that personal identity is very much of an illusion. It's a good story that makes a lot of sense in a practical way, especially as we're living right now in an uploaded world. It might make much less sense, but you still have your thread of memories. You might still have your goals and your values. And if you're branch out or not, your relationship to the other branches.
Anders Sandberg might depend quite a lot on different takes. So I think that I'm the equivalence class of all sufficiently Anders-like processes. So the Anders consortium, we might just be saying that, yep, we are all Anders's. And I have other friends saying, no, no, no, there can only be one of me. If there is a copy, he's somebody else. And of course, the copy is going to say that too. So I don't know how we're going to settle this.
Anders Sandberg perhaps highlander style with virtual swords or whatever. It gets interesting because you could frame it in a lot of different ways and bet people are going to handle it in very different ways. So the real question is, of course, what are you going to run your society on and how you organize it. Robin's book is very much kind of a minimum model, which is very much a strong economic competition.
Anders Sandberg Well, I'm kind of somewhat neoliberal. I'm also my mom and my mother-in-law were both union people. So I immediately suggested in your world, I would totally form an upload union. There is kind of a collective bargaining actually is quite powerful and useful. You can organize, you can actually create societal structures. And that's of course what we normally always do. If marriage gets really weird when you can copy, you're going to invent rules. Maybe we're going to be local to different groups.
Anders Sandberg maybe we're going to be a universal rule that we define marriage in particular society in a particular way.
Jeremy Gilbertson If my brain was scanned right.
Mark Fielding Maybe there's organic barriers to this. Maybe you have two consciousnesses in superposition in one on the virtual world, one in the real world. Or maybe there's a law that prevents that from ever happening.
Anders Sandberg In a lot of science fiction stories, they try to prevent this weirdness because it makes storytelling so hard. And I always felt like, wait a minute, this is one of the selling points. I would love to run in parallel. I always believed also that once you branch, you can never rejoin. But actually we seem to be getting better and better merging neural networks. So it might actually be that, yeah, you can go off and do things. So one of you stays at home, one goes off to the moon, comes back.
Anders Sandberg and then you merge the memories. And at that point, the narrative thread gets very weird because you had remembered that weekend from two different places. And some people are going to say, that's totally reasonable. Others are going to say, wait a minute, this is way too wild trip.
Mark Fielding fascinating because it's in almost every science fiction story we're kind of obsessed with the notion of it as a species.
Jeremy Gilbertson so there's three of us in this discussion right now, as we're talking, we may in the future have the ability to create a hundred of ourselves that almost has that have this many worlds interpretation and move through the world in different ways, bringing that stuff back. Are we capable of managing that noise and getting a through line out of all how does that how do you think about that from a human condition, our ability to just process all of that? What does that look like?
Anders Sandberg a lot of it also depends on how well you can get along with yourself and other people. So I like to imagine that if there is a dozen Andersons around, we would all very happily form a team or maybe go off and do our research and get along really well. That's what I like to think. I don't know whether that's actually true or whether we're going to rub each other in the wrong way and get really annoyed at each other. And that is, of course, just the easiest part.
Reputation and coordination in society
Anders Sandberg We're forming organizations that are full of weird pathologies, whether that is endless meetings and the careerists taking over and things and missing the point of why we have made the organization. We have all the traditional pathologies when you have selfish people involved in organizations that are supposed to solve collective problems. But then again, we also get new kinds of solutions. Reputations, for example, they have, of course, been around since forever.
Anders Sandberg all ancient talk about honor, it's all about reputation management, except that these days, reputation management might be more about search engine optimization and having a good LinkedIn profile or whatever works in your subculture. And we also can notice these reputation in other ways. I'm very fond of the Stack Exchange sites, for example, where you get little points for asking good questions.
Anders Sandberg People can upvote you or downvote you depending on how good your question is. And answers similarly get upvoted and downvoted. And suddenly we have a reputation economy. It's slightly addictive to get upvotes. You want to give good answers. So for a long while, this worked really well. Now we have a lot of people jumping in and they want that rush of getting a high score. So they use AI to write answers that sound good, but actually slightly dodgy. So...
Anders Sandberg It's facing a really interesting problem. And I think this is going to happen in all human society. We're creating structures and then somebody is going to try to game them. The best structures adapt to that. And we learn their lessons. Sometimes they're harsh lessons. And then we update it. That's how we're done with the political structures. Okay, we have a complex society. The strong man cannot run everything. He needs to delegate. Gradually, that delegation turns into committees, feudalism.
Anders Sandberg We realized that actually we might want to have buy-in from people. Okay, we need to have a parliament. Actually, the parliament has too much to do. We need to have committees and departments and agencies. And how do we keep them on track? Well, we have auditors. This is of course why traditional fantasy novels are showing in such a simple, easily to understand society. Real medieval feudalists was super complicated and would rarely have made it for good fantasy fiction because nobody can keep track of all the different roles.
Anders Sandberg And similarly, modern society is even more complex, bigger than we can think. But part of that complexity is offloaded. I don't know how to run the society, but there are political scientists, there are politicians, there are a lot of bureaucrats that do that. And now we're going to get the AI counterpart too. And presumably also the AI counterpart of auditors and checks and balances.
Anders Sandberg to avoid that somebody is spamming the system or somebody who's uploaded a copy of himself in a million copies and that way wins the election.
Mark Fielding at the start of all this, you said uploading to a virtual world, but I've got one word for you, humanoids. So why can't we, we'll be able to upload ourselves into organic replicas of ourselves rather than virtual worlds in a, in an internet, whatever that might look like. But there's, could be multiple me's walking around on the earth, not in a virtual world.
Anders Sandberg Yeah, and I think many of us would like that. After all, it's nice to be in the real world. It has many beautiful advantages. It is also kind of important. After all, he who controls the power supply to the data center controls the data center. So you might want to keep an eye on where the actual electricity is coming from. And one interesting aspect is, of course, making bodies that make sense for us.
A world run by AI: law, economy and autonomy
Jeremy Gilbertson gosh, there are so many things I want to ask you, Anders. This is boggling my mind right now. I want you mentioned something in your last statement about the outsourcing of management of complex would it look like if AGI or AI gets so good, we trust it to manage something like the world economy. What does that look like?
Anders Sandberg Well, hopefully it looks like a very well functioning world economy, but that can also be a bit of a problem. sorry for the plug, but I have an upcoming book, Law, Liberty and Leviathan, Human Autonomy in the Age of Artificial Intelligence. That's actually this summer, hopefully, Stockholm University Press. ⁓
Mark Fielding When's it out? When can we buy it?
Anders Sandberg This is basically the question we end up with. So it all started during lockdown in 2020. My co-author and me were sitting in my garden outside here, kind of at a suitable distance, shouting research ideas at each other about managing big risks, the role of governments in controlling people's behavior for the common good, and then of course, artificial intelligence. And gradually this led to a set of papers that then became this book.
Anders Sandberg And our basic argument is, yeah, why do we have legal systems and states and this? Well, that is the way we outsource our cognition. And we handle it by having these artificial systems made out of people and pieces of paper. But now we're of course also getting software that can take over more and more of that. Why do we have law? Well, we want peace and prosperity and of course power to some people. They are very good at latching onto that. And with AI, we might get even more peace and prosperity, even more efficiently.
Anders Sandberg and maybe power to some old man or other people who happen to get grasp it in the right way. But that's a separate issue. But the really interesting part is we might really want better coordination because the world is big and somewhat dangerous. We have climate change. We have controlling powerful AI. We have biotechnology getting more and more powerful. We probably need to coordinate about some global problems way more powerfully than we can do right now.
Anders Sandberg And maybe we could do that with a United Nations plus plus, but that seems a bit unlikely. It might be that we actually want to live more and more to super smart distributed AI systems. But then you end up with a funny situation. We might be swimming in peace and prosperity. That's great. But previously we needed human autonomy. We needed to make decisions to run the legal system, to run the market. Now it runs itself and probably runs itself much better without us messing with it.
Anders Sandberg It's a little bit like chess playing. Once humans were better than computers that played chess, then gradually computers got better and better and started beating humans. But a team of humans and computers could actually still do better. But after a while, the computer was so good that actually the humans were just slowing things down. So actually you should just listen to the computer. So if we end up in that world, we have this weird situation that we might be like pampered aristocrats. Really good material standard, but
Anders Sandberg In some important respects, our lives are not our own. And I think that sounds bad to many people. And I think the reason is we want to feel like we're in control because all our ancestors, the ones that we kind of find giving up control to others, they were just exploited by the ruthless people. So we have this tendency that we want to have autonomy. And then the funny thing is once we have autonomy, we're very happy. We are very happy to give it away in various ways, but we want to be the ones that do it.
Anders Sandberg It might be that you give away your autonomy by getting married. It might be that you join a company and in the exchange for getting a paycheck, you get some limitation of what you can do, but that might be worth it. I do. Yep. Kids are amazing example. People, every person I know that has gotten kids say, yes, it's super stressful and there's a lot of hard work, but it's the most important meaningful thing I ever done.
Anders Sandberg It's amazing. It's almost like mind control as somebody who haven't got a kid, but it's kind of a happy uncle. It's kind of, whoa, those babies, really got some psionic power here, but it's very effective and it's kind of good for our species that we have it. And it's good that we can give away autonomy in a reasonable way, but we should also in some sense be able to withdraw it. Although of course, when you're a parent, you realize, okay, I cannot even withdraw that thing. Now I'm in for it because it's what it is.
Anders Sandberg But this is where a world economy run by AI might be tricky because it might be so effective that yeah, running your own part might be outcompeted and might be so inefficient. It's a bit like living your life without electricity in a little log cabin in the woods. Sounds absolutely lovely and probably totally lovely the first day. After a few weeks of carrying water, a bit less lovely. And after a month, most people are back and writing on Facebook about the experience.
Mark Fielding We can have two economies like we'll have two of me and two of you and two of everybody else. You have two economies, one that's run efficiently and well by the AGI and then another smaller one that's run and managed by humans and they just run in conjunction with each other.
Anders Sandberg Yeah, and in some sense, I think that this might be what we're developing. We have the backbone of our civilization, the big infrastructure, the really important things that we must get right. When the electricity goes, well, nothing else works. So you really need to get that working. But once you're wealth enough, you can spend that excess wealth on all sorts of weird stuff. There is a reason why you have so many consultants as big companies. They can afford it.
Anders Sandberg And they're probably more involved with status games and politics than actually doing something useful. I think it's a good thing that we have enough wealth and safety that we can try out different lifestyles, even if many of them turn out to be slightly embarrassing once you're middle-aged.
Jeremy Gilbertson So quick thought on that. So I often talk about, this is more in the context of like space and things. I want to stay where we're at on the thread, but the idea of like bringing our own human baggage to the new version of the world. There are things as humans, power dynamics and control and me versus you, like if we were to design or not design, but like kind of enable this world economy through AI,
Jeremy Gilbertson How do we make sure as humans we don't bring that baggage that always wrecks stuff?
Anders Sandberg Yeah, but even that is a problem. When you try to remove the human, well, what have you trained the large language models on? They're all trained on all the human literature. They have read every single thread we've written about betrayal and power dynamics and corporate shenanigans. One of the big debates right now in the AI safety community is when you test your AI models and they try to do sneaky stuff like escape the system if they think it's going to get shut down. Is that because they
Mark Fielding "want" in some sense to escape or just because they know that's expected of me because of all the science fiction I ever read says that that's what an AI does in this situation. and maybe it doesn't even matter. So I do think that it's very hard now to take away the imprint of the human world from the AI world. On the other hand I think there's also going to be evolutionary pressures pushing in certain directions. There are various forms of competition that means that certain software, pieces of software are going to be copied more widely than others. And that is going to act anyway, regardless of what human values are. And of course, my more doomy friends say, yeah, that will inevitably be pushed in a very scary direction that's going to be very different from what we humans are. So I think one important thing we might want to be involved in, we might want to be an essential spice in this mix, to make sure that there is at least some of us in that. But then we want to filter out the really bad stuff.
Mark Fielding of cruelty, for example, it's kind of useful to stabilize altruistic punishment. Taking delight in the misfortune of somebody else is a really good way of kind of giving you extra points when you're trying to punish somebody breaking norms. Because otherwise, I would be taking a risk in policing these norms. And actually, it would be easier if I just stayed at home and let the norm breaker get away with it. Except that today we can do this with laws. We can use police.
Mark Fielding And suddenly cruelty is just a bad thing that generally makes us nasty. That's an emotion I totally think we should kind of try to get rid of and at the very least make sure our AI programs never experience.
Jeremy Gilbertson human element is still in there because the models are trained by humans and the models are trained by books written by humans and studies written by humans and all of that, the system could still be gamed. You talk about gaming systems. Is the system easier to be gamed because of that or not?
Anders Sandberg I think it's a bit of a mixture, very easy traditional tricks of gaming human systems. And then of course we humans are good at finding them and we also try to make safeguards against them. Which is of course why a lot of rules everywhere exist to make it harder and harder to do that. Except that many people are gaming the system by writing rules that benefit them. But then you also get the other dynamics here that you create systems that might be very good at gaming themselves. Quite often I find not just that I get manipulated by AI programs, but that they are sometimes behaving in a lazy way because we copy human misbehavior. So then you set up other agents to make sure that they behave themselves, et cetera. Now you create incentives to gain those agents and so on. I don't think it ever ends. And that is kind of fine because it also creates diversity. A lot of gaming generates variation, you gain something from not doing quite the expected thing. If it all tended to converge, that might actually not be a good thing. So we saw this when we were playing around with artificial life in the late 80s, early 90s. People finally figured out how to make self-replicating software and make it evolve in a little primordial soup in the computer. It was really fun. But one of the problems was most of this evolution tended to end.
Anders Sandberg At first, these replicates got better and better. You got parasites and hyperparasites. We were all very excited. But we never got any rainforests. We just got a petri dish with slightly interesting microorganisms. And it seems to be because it was a petri dish. It was a too simple environment. In the actual world, you get bacteria that survive in the soil, in the sea, in the air. They have to deal with all sorts of problems that are local and unique.
Anders Sandberg other organisms are preying on each other, they're creating a complicated system that you need to adapt to in all sorts of creative ways. And that creativity generates a lot of the diversity that I think adds value to the biosphere. And my inkling is that we might have the same situation in the AI domain, if it all kind of converged to one static, stable, good system, even if it's very effective in producing peace and prosperity.
Anders Sandberg it might actually be less valuable than something more robust with a lot of internal diversity. And for that, you need sneaky little agents that are constantly poking at the possibilities.
Mark Fielding we're manipulating machines. From the day we're born, we are gaming the system. And maybe that's all we have.
Mark Fielding I just close out that section with two quickfire questions for you and as yes or no questions?
Mark Fielding And then I'll let Jeremy ask who owns the moon. Question number one.
Anders Sandberg Even if he's super successful and reaching longevity escape velocity and then uploads himself and spams the cosmos with backup copies, in the really long run, entropy is going to get us all. In the really long run. Of course, the interesting thing is the game until that and let's make it a very, very long game.
Mark Fielding Will he live to a thousand? Second question. We've been speaking about the new American AI military strategy of late in our AI in the war machine season. that you can check out thinkingonpaper.xyz. Does AI and AGI when it arrives, make war more or less likely?
Anders Sandberg I don't know. It's really tricky. I would like to say more likely because I can see a lot of instabilities, but I can also imagine that you could actually get something like nuclear deterrence. I don't think we have a good theory about it. And that is scary because we kind of barreling towards it and we don't even know where we end up. At least the Cold War, nuclear deterrence was something people were thinking about before there was an atomic bomb.
Anders Sandberg very few science fiction authors and others kind of already getting a gist of this is something that could happen, let's be ready for it. But now we might have to be ready for several different scenarios.
AI safety and the race to AGI
Jeremy Gilbertson We literally unpacked, we read the Pentagon's paper on AI strategy and or not Department of War used to be the Department of Defense in the United States, but they've changed that due to we're not going to go into all those reasons. you mentioned we even have a theory for a lot of this stuff to even evaluate and yet agencies like that are warp speeding their move into AI. It's pretty scary stuff, man.
Anders Sandberg yeah. yeah. When we started thinking about AI safety back in the early noughties, we were a bunch of weirdos even within the AI world, because we believe that actually eventually we're going to get full artificial intelligence and eventually superintelligence. And no, maybe we need to make it safe. But at least we have decades and decades to do that. And gradually, the timelines got shorter and the AI got more capable. And when people started paying attention, it was both.
Anders Sandberg At first, yeah, but nobody will want to make AGI, right? That is useless. Okay, we're making AGI, but it's totally safe. And we're gonna regulate it in a sensible manner. dear, the time is kind of running out. So we have to do a lot of stuff on the fly, which is going to be interesting to say the least.
Jeremy Gilbertson And a lot of those things on the fly will be supported by your favorite neighborhood LLM. Dear goodness.
Anders Sandberg tell me how to align you. Presumbably the answer is first start with aligning my competitors please. One of the best ways of getting LLM to do good jobs is to give them output from other LLMs and tell me one of your competitors said this and they're tearing into it.
Jeremy Gilbertson Yeah, it's kind of like how science works. You throw a theory out and everyone else beats the crap out of it, right?
Mark Fielding is a theory because, and it's just blind optimism. That's what the theory is. It's blind optimism or maybe not blind optimism, but optimism that AI will make war less likely.
Anders Sandberg it's interesting because they're probably smart people who came up with that view and maybe they had a very good discussion and some very solid reason or maybe this was just what everybody ended up with during the coffee break. It's quite hard to tell from these white papers sometimes how well supported or sometimes you're amazed when you actually meet the people involved and sometimes you're just horrified.
Drone warfare and strategic stability
Jeremy Gilbertson Here's an interesting thing that I think you brought up earlier. Palmer Lucky is the Palantir guy, right? Is that right, Mark? So he, there were a couple of Ted talks that he did in one. ⁓ Anduril, not Palantir, Peter Thiel. Peter Thiel. Yeah, Anduril. So in one of Palmer Lucky's talks was just that. It was like, hey, what we're doing is going to make war less likely or have more, a more efficient way to get to not war. But if your company
Mark Fielding Yeah, correct. No, he's under real. He's under real. Palantir is Peter Thiel.
Jeremy Gilbertson profits from wartime activities, I don't see that as being a realistic approach.
Anders Sandberg And the stability issue is also tricky. So drone warfare has really exploded over past few years and everybody is scrambling because they realize that looks very much like the future. If you don't have some drone or anti drone capabilities, you're going to be in trouble. This is great. If you're a drone maker, a lot of Ukrainian engineers are going to get very nice jobs in the future, no matter how things go in Ukraine.
Anders Sandberg Is that going to be stable or not? Is it going to be like, okay, now we have a lot of containers with drones waiting near borders looking for if somebody does an incursion and nothing ever happens because it would be too dreadful to do it? Or are people now going to try to sneak in little drones? I have no idea. I feel it feels like something that could be unstable. But then again, strategy is hard.
Anders Sandberg And it might very well be that actually we humans can't even figure out the strategy. So we leave it to our AIs and then they develop strategies that we don't even understand, which might produce a weird situation where, yeah, the world looks super peaceful, but actually it's on a knife's edge, but we can't tell. And we can't even get a straight answer out of AI because it just tells us what we like to know. Or it could be that it's actually really, really stable. But again, we can't tell that apart from the scary scenario.
Anders Sandberg So there is a real problem that even understanding the strategic situation wherein is hard. After all, we should recognize that mutual deterrence is something that has been going on long before we had nuclear weapons. Many medieval warfare situations were a little bit like that. But back then, they didn't even have much terminology for it. There was no game theory until the Second World War. We developed the tools to understand our situation.
Anders Sandberg much later than we actually went into that situation. Economics is a fairly young science and yet we have been trading for thousands and thousands of years. People understood some parts of economics very practically, but many things like inflation was just alien or still kind of alien to many people. We don't know how to handle that. So it might very well be that, yep, we develop our AI defense systems.
Anders Sandberg And then 100 years later, some bright AI comes up with a theory about what we've been doing for the last century. And the other AIs decided, you should totally have a Nobel Prize, but you can never explain this to the humans. They would be so distraught.
Jeremy Gilbertson let's, let's re point our trajectory. Let's, let's re inject a sense. Let's re inject a sense of, of awe and wonder at our seeing something that's bigger than ourselves that we don't understand that is not computer-based. part of what the show explores too, we've been deep into space tech. We've talked to people looking to mine helium-3. We've been talking to space based solar.
Jeremy Gilbertson power people we've been talking about a lot of conversations that stem from a book I read a long time ago called The High Frontier by Gerard O'Neill.
Jeremy Gilbertson See you, Mark. I'm onto something.
Mark Fielding it! converted. I think I was reading Arthur C. Clarke anyway, which is kind of the same thing before I read O'Neill. I'm on board.
Who owns the asteroids?
Jeremy Gilbertson So, so let me start with it. took him, it took him a while. I'd bring up O'Neill and he'd make fun of me every time I did, but now he's on board. So we're good. so, so you're on orbital. Nice. the idea of space looking beyond earth, extending human civilization elsewhere, largely because, because we've created some problems on our existing home that might not, ⁓ help us to get to longevity. Right.
Jeremy Gilbertson How do we deal with ownership structures in space?
Anders Sandberg Yeah. And right now we have this weird situation that, ⁓ you officially cannot own stuff in space, which I think is problematic from an incentive structure because ownership is actually quite a good way of making people interested in developing things. If it's everybody's, that means it's nobody's. But in practice, of course, it was very easy to make space treaties.
Anders Sandberg when only the big superpowers could get into space and they were unlikely to actually do space mining or anything. So you can just say that, it's a common heritage of mankind and kind of punt the problem to the next century. And now we're in the next century and it's going to be a real issue. My guess is that people are just going to be adapting laws piecemeal.
Anders Sandberg Typically what happens is that you take something like the law of the sea and start applying it to space. So spacecraft are under the jurisdiction of the country that they're launched from or registered in. We might find that Liberia might become a major space power this way. But what about the ground? And again, I think people are going to have to come to some understanding about that because we are going to see a race to the South Pole on the moon before long.
Anders Sandberg And it's pretty clear that the real estate there is valuable and there's a limited amount of it. And you probably don't want China and the US and SpaceX to have an actual fight over it. It might be that putting up big solar panels. Well, if you shade the solar panel of another country, they're going to be most unhappy about that. So you need to resolve that one way or another. I don't know how we're likely to do that.
Anders Sandberg I think it's much easier when dealing with asteroids, when then you might say, okay, you can make a kind of claim, right? The asteroid isn't yours, but the spot you're on that asteroid, yeah, obviously you're working there and nobody's ever to interfere with you. It's going to be trickier with other bodies. And meanwhile, of course, there is going to be various con men trying to sell real estate on the moon and Mars to people.
Mark Fielding Why is it different on the moon to an asteroid?
Anders Sandberg I think partially it is because we have a different sense of geography, legally and ethically speaking, when it's a larger body. It's kind of a funny situation. The moon is a place. Mars is even more a place. You can see these maps with these wonderful fantasy names for all the location. You can envision yourself walking around there in Amazonia planet and trying to get to a particular crater. An asteroid is more like a thing. Now, of course,
Anders Sandberg Big asteroids are more like worlds. Ceres, after all, if you stand on the surface of Ceres, you probably can't tell it apart from the moon. You're going to be in a rocky landscape of craters and regolith. It's just going to be feeling very similar to a place. And generally, ownership of places is, of course, something that we humans have had a long history of trying to deal with, partially because it's territory. It's kind of funny things we own in a different way from land.
Anders Sandberg Land has of course always been linked both to productivity and political power in a very different way from owning stuff that you can carry on with you. Although from a philosophical standpoint, it's not obvious that any of this should make any difference, but emotionally it seems to do. And certainly countries have always been based on this idea that, yeah, in order to have a state or a country, you have a piece of land and you can kind of control.
Anders Sandberg Except that that is getting hazier and hazier. During the first Gulf War, Kuwait had a government in exile, and the country was kind of occupied physically. But from a legal standpoint, yep, it was doing totally fine from London, I think it was, or maybe it was Washington. And we might, of course, because of climate change, end up with a whole bunch of countries that don't have any land. But we're totally going to be sitting there in the United Nations.
Anders Sandberg We might end up with a similar situation in space. It might be that here we get country-like things that are not necessarily tied to land in the usual way. Luxembourg is kind of interesting space mining. I like to joke that maybe Luxembourg is going to be the largest land area in the country, the solar system one day. But a lot of it depends also on is that land area something they can kind of defend? Not to make necessarily with star cruisers and lasers.
Jeremy Gilbertson any parallel-
Orbital real estate and space traffic
Jeremy Gilbertson Are there any parallels to the terrestrial oil industry that we could extrapolate and throw up at asteroids? And what we, because essentially oil is a valuable resource. People land in certain areas, they apply value and investment to those areas and expect what comes out to be theirs. that, is there anything we could apply to asteroids in that regard?
Mark Fielding Early diamond mining seems more similar to me
Anders Sandberg So it's one thing about when you have land or objects that nobody has claimed, then generally it's viewed that yeah, you get to gather them up. And then typically, of course, on earth, states have expanded and said, no, no, no, we own everything here, no matter whether we know that it's good for or not. And this includes biology, because of a Cartagena protocol, I think it is, we have ended up with this idea that bioprospecting is not allowed.
Anders Sandberg without permission from the government. So the Brazilian government in some sense owns the genomes of all the undiscovered little insects in the Amazon jungle. And I'm not allowed to go there and start sequencing them without their permission. I think that is a bit of a government overreach, but there is an interesting interplay here on what you then allow people to do. In Sweden, for example, we have a rule that you're actually allowed to roam even on private land within a region.
Anders Sandberg You're not allowed to get too close to houses. You're not allowed to do mining or take, do permanent damage, but gathering berries and mushrooms in the forest, totally fine. Which of course in other countries would be, wait a minute, that's unthinkable. Why is it that we do it in Sweden? Well, partially low population. was a relatively easy rule to the Institute in other places with more dense population. This wouldn't work. So I think we're going to make up rules.
Anders Sandberg partially based on the nature of the environment and what we want to do. I guess that we tend to still think about places and material stuff in space, but it might also be that orbital slots might be even more important. So now when people are talking about space data centers, suddenly the sun synchronous orbit has become really hot. And I got curious, how big is it? And I did some calculations and plotting. realized, if you want to stay away from radiation belts,
Anders Sandberg That's a rather finite amount of orbital real estate. Being able to claim that orbit, being able to put stuff up there might be really, really valuable. And it already looks like some of the space companies are really trying to get a lot of that orbit. So now we have this problem of kind of land claim, but it's framed, of course, more like we're claiming parts of a radio spectrum, or we're claiming various sea lanes or rights of passage. I think...
Anders Sandberg Probably traffic control might be the earliest seed for the real property rights in space. It might be not so much ⁓ a land registry that is the center, but kind of how are you allowed to fly your spacecraft? Where are you allowed to dump your junk from, especially the exhaust from the spacecraft? Because if you play a powerful ion beam across a satellite, that's going to wreck it. So you're not allowed to steer in particular ways. Eventually we're going to have enough fragile, expensive, insured stuff up there. And that means that we are going to get a traffic regime. And that traffic regime might also be a foundation for property rights. It might work very differently from what we're used to here on the ground, where traffic control is kind of for the infrastructure that supplies us rather than the government.
Jeremy Gilbertson It's interesting too, we've been thinking about this a lot that, you know, orbital slots, I think are in the States are at least governed by the FCC, which is a spectrum, is how wireless networks and broadcast networks and communication networks all work. And now it's orbital slots, but, and there are also, you mentioned coordination.
Jeremy Gilbertson within what's going on up there. There are interesting companies out there, Leo Labs, I think is one we've been following that follows these little projectiles and things that are up there, but projectiles in space created by incidents, who's ultimately responsible comes back to accountability. Again, we talked about, but let me give you a Coordination is why humans have been able to succeed.
Jeremy Gilbertson throughout their evolution, but it might also be our downfall.
Why humans fight, and how to coordinate
Anders Sandberg I think that's very true. It's pretty obvious that the reason that we're running this plan for good and ill is that we're better than most animals are coordinating across time and space. It's not just that we're forming a little tribal group. It's not just that we have a language that allows us to do things, but we can make shared intentions. And then we can share these intentions using the written language and the culture across time and space on enormous scales.
Anders Sandberg The is of course that human wars are way worse than chimp wars. The chimpanzees in a troop cannot really decide let's go to war against the other gang. They just come across them and then the fight erupts. Well, they know what sides they're on, but that's it. You can't scale it. While we humans, we can start building up a story about why that land is actually ours and why we really need to do what we do. It's very regretful.
Anders Sandberg but we need to do these horrible things for the good of all. And that coordination is risky. We quite often talk in the discussion about human enhancement, about moral enhancement. Wouldn't it be wonderful if we had more empathy and more ability to coordinate with each other? And up to a point, that's great. But then you get to really well coordinate the group with really bad values and it doesn't sound so great anymore.
Anders Sandberg So what you want is probably coordination that is also forgiving enough that allows for diversity of approaches that doesn't naturally break down in really big conflict. And part of that is why the trade and law are so useful. It ends up in a courtroom with a lot of people arguing abstruse pieces of code at each other instead of people killing each other. It's much preferable, even though it's kind of slightly boring for most of us. But
Mark Fielding Yes. Most of the time.
Space races and lock-in
Mark Fielding Philip Metzger on a few weeks ago rocket scientist. 30 years at NASA. And we asked him what what the collateral damage is of this space race,
Mark Fielding What's the collateral damage of the human race for space?
Anders Sandberg I think in many cases, some of the space, I think a lot of the space race has actually been quite good. But what we can learn from the original space race last century was, yeah, you can motivate yourself to do amazing technical achievements, but then aren't actually linked to any institutional results. We never got a moon base. We never got a permanent human presence in space.
Anders Sandberg from that space race because it was just motivated by the Soviet Union versus the US showing off. That racing didn't actually connect to anything useful. Right now we're seeing a new kind of space race that's actually much more useful because it tries to put up satellites that people actually use for something. We're getting a lot of communication satellites, we get the earth monitoring. And I think that is in many ways more healthy.
Anders Sandberg The real problem might be that you accidentally lock in things early on. So I think we are going to renegotiate the space treaties. That's kind of obvious because we're not fit for purpose. But in what mode they get renegotiated, what we end up with is anybody's guess. But whatever we end up with is probably then going to be acting over several decades more because typically you don't want to change the platform, whether it's a technical or legal platform you're standing on.
Anders Sandberg And that might have big effect because several decades hence, we might be doing enormous things in space, but we might have rules that date back from the 20th 30s. And if the 20th 30s understanding of space was based on kind of rivalry between China and the US and the space X and blue origin. And that is the defining idea of how to handle space. But it's all about rivalry or some other random factor that came up when negotiating.
Anders Sandberg then we might end up with that when we're building the big space habitats or starting our Dyson sphere, it's going to be based on the wrong mode of thinking about things. So ideally, you want to be open-ended, you want to be able to adjust and renegotiate things, but you also want a stable platform. So a race means that you end up with maybe that platform a bit earlier than you intended. And the platform might be all wrong. It's a little bit like the email protocol, which led to spam. Why?
Anders Sandberg Well, there is no cost to sending an email. You could have added that when people were inventing email in the early 1970s. If any economist had been in the room at the time, they could have pointed out that you're going to get a lot of unwanted email if you don't add a little, little cost to email. But of course, no economist was ever invited and they would never have thought about it economically. So it never happened. But you want to get some of these rendered thinking in when making the protocol. And if you're racing,
Anders Sandberg You don't have the time to get anybody who's not in the core team in.
Jeremy Gilbertson And races have winners and losers.
Anders Sandberg Quite often, but sometimes of course the race might just be that you end up with a big win for most people. After all, the dot-com boom was in some sense a race and then most people or most companies lost, except that a few survived really well. But the rest of the world also benefited because we had added so much broadband. We had developed a lot of infrastructure that actually enabled a much more sensible internet to take shape.
Anders Sandberg The real problem is of course, if you have a race and it feels like it's a winner take all, that means that now all the rules are going out to win the window because I need to get this. That is why I'm kind of worried about an AI race because if the important thing is winning, then that means slowing down a little bit in order to get something safe and sane and ethical.
Anders Sandberg Well, that's going to be a drawback. So let's throw out those soft fussy things. We're going to fix that safety and sanity later. We do that in version 2.0.
Jeremy Gilbertson Do the idea of nations help us or hurt us in the space race?
Flags of convenience and new jurisdictions
Mark Fielding Can I just on that because at the beginning Anders actually said, you don't want to race with China, America and SpaceX, which I thought, hold on, he's comparing SpaceX to America and China, SpaceX is a company.
Anders Sandberg Yeah, but nations and companies, honestly don't think they're fundamentally different. Of course, this is where a lot of political scientists say, wait minute, that there is an enormous, very profound difference. But I think we're ending up in a world where we have many kinds of that are able to act in the world and they're interlocking in a different way. Yes, SpaceX is registered in the US.
Anders Sandberg But could SpaceX re-register in Ireland or some other pliable jurisdiction? In principle, yes. In practice, it's not terribly likely. And in practice, probably it's more likely that, no, you send just enough lobbyists to Washington and come up with a way of handling it. But many of the assumptions we see, for example, in the United Nations system is the only actors that are relevant are nation states.
Anders Sandberg And that's not true in many domains. I'm very much coming from the world of cyber culture and there it's for a long time been very clear that it's not just corporations, but even various protocols and NGOs and networks that are just as powerful and important in these domains. Saying that only the nation states that with flags and armies are allowed to be at the table is kind of missing that a lot of the key actors are utterly different. And I bet that space might in the long run, really undermine the traditional nation states. Part of that is of course, a nation state is built of idea that it's a nation and the state. So we have a cultural cohesion, which is kind of a modern idea from the 1800s, combined with we have a state, kind of sovereign power and the monopoly on violence. But you could of course have states that are not nations, the United States is a beautiful example of a state that actually functions without necessarily assuming that we're culturally identical and coming from the same ethnic origin. You can make all sorts of new things and I think we should probably experiment with more forms of state. I think that's actually a healthy and good thing. Governance is too important to be left unstudied.
Jeremy Gilbertson That's a really interesting a pin in the idea human coordination look like as we move into the future of space and tech and how different that could look?
Mark Fielding the network state isn't it? What's that guy's name who Yes.
Mark Fielding I think it was Romer who suggested originally and this idea that you could have these special economic zones and kind of have a bit of competition for good governance. And I think that's already a really cool idea. Then Balaji is saying, ⁓ we should have these distributed states.
Mark Fielding They of course are classic staple from science fiction. I found it very cool to actually visit Prospera down in Honduras, which is one of these little weird city states and of course very much subscribing to the Network State view. Can that work? Can that produce good results? Well, we need to experiment on it. Probably it's a good idea to do it on small scale rather than the big scale. It's always dangerous when...
Mark Fielding actual governments with a large area of land that you can't easily escape decides to experiment on you. But small startup experiments might be a good idea. And startup states, I think we should actually try to have that. I'm very fond of the idea that people should be allowed to try to build some sea-steadying site out on the Pacific Ocean. And let's see what actually works. Sometimes we might hit on some good idea that we then want to copy and scale up. And quite a lot of them are going to turn out to be not so good and maybe it's better to let it sink.
Mark Fielding You couldn't even get pirate radio in the UK anymore. Those I don't see the chances of that happening very much. I don't want to be a party pooper. Do you have that expression? A party pooper? I don't want to be a party pooper because they're not even back yet.
The future of NASA
Anders Sandberg That's a great question. So I always been a little bit on the kind of new space side thinking, yes, that's an old bureaucratic colossus. They're so slow and they're so beholden to a lot of interests in politics. So they have to do programs that make pieces out of every constituency. all the senators support that. Artemis II is kind of still showing that yeah, NASA still has its groove on. It can do some things really well.
Anders Sandberg It's just that I think the rapid innovation pace might actually beat them. The problem with NASA is they are so good at doing it. It's a little bit like the elves or dwarves in some fantasy novel. They're handcrafted these magical things, but they're so rare because they're handcrafted by the elven wizard. So you only get one single magical sword or that single magical shield and it's endlessly expensive. It's really good. But yeah, that's crappy iron sword the humans are making over there. They're making hundreds of them and that's enough to actually get the entire fantasy army up and running. So I do think NASA has this cluster effect of having some really bright people. The big problem is of course the really bright people in our organization that's also subjected to horrendous political forces. So I don't know how well that's gonna last. But then again the bright people in any company also have the same problem. The skunkworks couldn't kind of keep up their epic pace forever. They had ⁓ a golden age and then since then, I think they've been just doing good work. And this goes for many things. One of the funniest things I realized becoming a middle-aged person is, yeah, golden age come and go and that's fine. I'm fine with that. I'm just going to try to find the currently interesting cluster or even better, something I think might become the next golden age or the next cluster and then try to make it last as long as possible.
Anders Sandberg Eventually, yeah, it's gonna end. think NASA has amassed so much kind of capital in some sense that it could keep on surfing for a long time, but I think it's good that it gets a fair bit of competition.
Jeremy Gilbertson if someone at NASA has their Jerry Maguire moment with this big manifesto and is like, all right, I'm out. Who's coming with me?
Jeremy Gilbertson Where do we go?
Jeremy Gilbertson I mean, could we see that? Could we see that in the future?
Anders Sandberg Yeah, I totally think that's plausible. After all, we've seen that in other domains. Sometimes it happens accidentally. Silicon Valley was very much started because people got fed up with chocolate and decided, okay, he's just too much. We're moving out and starting our own companies. And that was the start of Silicon Valley. You have Antropiq moving out of OpenAI because many people there felt like...
Anders Sandberg You're all taking safety seriously. We want to take it seriously. Let's build something. And maybe that can happen here too. I haven't heard many cases where government agencies do that. But then again, NASA is a very weird government agency. I think there is potential for that. There are many really cool and bright people. The really hard part is of course the entrepreneurship part.
Anders Sandberg As a lifelong academic, I always admire academic entrepreneurs who just get fed up and go off and start their own university or their own research field. I'm usually the guy following them a little bit later saying, okay, this looks good, but I couldn't do that. I like to see this kind of entrepreneurship and leadership in action. But of course, quite often it's also misguided and sometimes we just walk off in the desert.
Jeremy Gilbertson talk about NASA and the future of NASA. And with everything we talked about today, there are two things. There's coordination and then there's trust, right? And establishing trust to be the authority to govern something.
Jeremy Gilbertson How does trust and the building of trust play into how you think about the future and how could NASA maybe even be the new trust organizer and not the doer of the things, but the entity that everyone looks to for consideration and advice and all of that?
Anders Sandberg Quite often, socialities divide the way you get social status into dominance and prestige. Dominance is the way of a bully. You're strong. others don't do what you want, you beat them up or you yell at them or scare them. And then you have prestige. You do something really well. You are renowned for being good at that thing.
Anders Sandberg might not mean that you're in charge of stuff, but everybody goes to you when it's dealing with that domain. in both good ways of getting prestige in society, the problem is, of course, when you lose a bit of control, usually the bullies find that suddenly everybody else is going to get back at them. While people generally tend to help the prestige people, no, you've fallen, let's help you. Prestige is important.
Anders Sandberg How do you get prestige as a regulator? How do you build trust? Well, part of it is being impartial. Part of it is being able to do things usefully. Sometimes with the flair and style, sometimes just plodding along in a super reliable way that everybody can build on. So this might be something for NASA to think about. In some sense, plodding along might be very boring when others are kind of sending up rocket after rocket after rocket.
Anders Sandberg But sometimes the other rockets blow up. That's kind of a point that Musk will happily say. But on the other hand, that reliability, that boring reliability is also useful. You might be seen as an impartial arbiter. I think generally trust has a lot to do with legibility. We tend to trust people where we kind of understand their motivations. We can see what they've done in the past. We can see roughly where they're going. The people who are mysterious are intriguing, but don't really trust them. The people that you cannot tell what we're up to. yeah, interesting, but maybe I'm not going to invest there. You want that transparency and this might be where NASA can be quite useful. It probably like any big government organization needs a lot of reform.
Anders Sandberg I'm very skeptical of big government. I'm a libertarian from Sweden, which is a kind of a contradiction in terms. So by American standards, of course, I'm a social democrat socialist, but by Swedish standards, I'm raving neoliberal. And generally, I think you want your agencies to be agile, but they also need to be fit for purpose. And that goes both ways. You want to check your agencies because there are pathologies that happen in organizations. People build very little fiefdoms.
Anders Sandberg People get all sorts of weird incentives that have nothing to do with the mission. And we should just be on guard against that. And that's kind of a useful liberal critique against it. But at the same time, you want to have these transparent, somewhat neutral systems. It doesn't have to be a government agency. Sometimes you get things like the Lloyds Market for Insurance in London. Sometimes you get the Internet Engineering Task Force. But you can sometimes set things up.
Anders Sandberg And if you have a cluster of really competent things and history at the bottom, that's useful. To some extent, that's what Oxford is doing here. We have been surfing here for kind of 800 years of having fancy architecture and enough people learning enough things to occasionally become famous alumni that give back money. That maintains a structure. That structure can still do stupid things because it wants to maintain itself more than educate or do good research, but you still get something to build on top of.
Anders Sandberg And then that can create a fair bit of trust.
Mark Fielding is going to be the longest show we've ever done
Fusion and leaving the inner solar system
Mark Fielding We'll land the fighter, we'll land the spaceship. Do you have any thoughts on fusion? We've spent a lot of time in space season talking about space-based solar power. Space-based solar power isn't going to work in interstellar space.
Anders Sandberg Nope. If you want to go outside the inner solar system, you want nuclear. And fission is not going to be particularly fun outside the asteroid belt because where do you find the uranium? Well, inside Jupiter and you're not going to get at it there. So fusion it is. And yeah, we better figure out fusion if we really want to do things in the outer solar system and interstellar space. If we do that...
Anders Sandberg Then, of course, you can probably live like a king out on the Kuiper Belt objects in the outer solar system. There is enough deuterium in that ice that you could run. I calculated that in a typical Kuiper Belt object, you have enough energy to survive for 100,000 years if you have a little city state on it using one terawatt of power, which I think is a fairly decent amount for something city-sized. That's good.
Anders Sandberg But we need to get the fusion to work, of course, which is tricky. But I'm getting more more bullish on fusion. I'm still kind of feeling plasma seems hard, but now there's enough people that seem to be using different approaches and actually getting investment in a proper manner. And generally, I tend to assume that investors, they might not always know what they're doing. But I think the field is getting somewhere. We've seen advances in permanent magnets, thanks to the rare earth magnets that are just making things so much better. Not everything has to be a super complicated superconductor or just a normal magnet. I think we might be getting there. So fusion is still kind of an energy of the future but the number of decades till fusion has probably shrunk a fair bit.
Mark Fielding shout out to Glenn Martin, who was on our show, an awesome guy. But in our show, he just off off the cuff said, referred to fusion as an engineering problem. And the way he said it, and he's got so much heat on the internet, because people are saying, what do mean, it's just an engineering problem. It's like quantum computing, it's always 10 years away, it's not going to happen. So you're optimistic that we're it's going to get there sooner.
Mark Fielding engineering problem. So I heard a critique from my friend, Eric Drexler, who's somewhat famous for nanotechnology and also doing a lot of very cool stuff about AI. But he pointed out that, yeah, getting the first order instabilities of a plasma under control, once you've done that, unfortunately, there are second order instabilities, and we have even less ideas about that. But then again, engineers are clever.
Mark Fielding Just an engineering problem usually doesn't mean it's easy, but engineers are amazing. I really admire engineers. I'm a scientist. I do stuff to get knowledge. I'm very bad at actually building stuff. I admire my brother who's an actual engineer whenever I find him in the garage kind of doing a Frankenstein on Ikea bookshelf to make it fit into the furniture, making things fit and changing the constraints of the world. And the trick that happens of course in engineering is that if there is a problem, you engineer around it or you make it to not be a problem. In science, if there is a problem, you're not allowed to say there isn't a problem. You actually need to solve the darn thing. And quite often that turns out to be harder. The really interesting question is what AI is going to do. So these days I'm wide coding every day. I have little agents writing all sorts of software for all sorts of purposes.
Mark Fielding But vibe engineering, I bet that we're going to get to that relatively soon. Today we have topological optimization so we can use AI to generate weird parts that we can then hopefully in a 3D print and make use of. I'm seeing more more simulations enabling humans to work together with software. And I bet that within a decade or two, we're going to see a lot of vibe engineering, which might help the development of fusion quite a bit because, yeah.
Mark Fielding There many weird problems that might not even fit our way of thinking, but might really fit whatever the AI systems are doing. So I'm getting rather optimistic about it. It might be just an engineering problem, but we're getting a lot of software engineers in a literal sense.
Irreversibility: closing doors for good
Jeremy Gilbertson this is super inspiring and really fun. ⁓ Anders to just think big about stuff that, know, and, help people think bigger about stuff that they don't have the time.
Jeremy Gilbertson to do that. One of the problems I have with our current civilization is that we're so afraid of thinking that things could work differently. We have systems that work pretty and then people latch on to that. To some extent, it's sensible. We need to keep the systems we got to function. We have been maintaining some of our political and infrastructure systems really badly, and we're reaping the harvest now, unfortunately. But at the same time, just because some system is pretty good. Doesn't mean it couldn't be much better. And maybe scalably, amazingly much bigger. So I think thinking big about the future is also a very healthy exercise to get hopeful about it.
Jeremy Gilbertson Is there anything out there that shouldn't be optimized?
Anders Sandberg I think to some extent, yes. There are things like state monopolies of wireless that I'm getting very uneasy about, if you could imagine the robot army and the robot police force, because now there is nobody to say no to dear leader when he decides to do something stupid. So there are dangers like that. There is also an argument to be said that maybe we shouldn't be optimizing.
Anders Sandberg too strongly for superintelligence right now because we don't know how to control that. Already making use of the current level of in the AI is going to keep us busy for years. It would be a nice thing if we could actually know that, yeah, we're not getting superintelligence for 10 years. Let's work out how to make current systems safe and robust and sane and trustworthy. That would be really useful. The weird part is, of course, what's optimizing for that?
Anders Sandberg Well, that's market forces, that race dynamics, a bit of human nature. It's a lot of stuff that we have very limited control over. And maybe we want to gain that control, but then you have another risk. Now we might lock ourselves in. to some extent, optimization for locking ourselves in or out of possibilities is a very dangerous power. We want to lock ourselves away from some existential risks.
Anders Sandberg We want to close some doors permanently. Wiping out smallpox was a good thing and we should be just keeping on doing that for various horrible diseases. But there might be other things like the nuclear waste. Why store it permanently underground so nobody can get that? Actually, might be that future generations think that those isotopes are just what we need for our probe to Pluto or whatever. We might actually not want to be too irreversible.
Anders Sandberg So I do think one should be careful about irreversibility. Optimizing for that is dangerous. But then again, if everything is reversible, that also means that nothing quite matters. You might want to be a bit careful about making it too easy to undo things. In some domains, you actually want stuff to matter. because that is also giving you a direction of time. It's giving you a meaning of, okay, this cannot be undone.
Middle age, time and boredom
Mark Fielding At the beginning of this, you spoke about using Claude for your exercise routine. And I wonder you have Claude's advice and doing that exercise routine. And I guess my question is, how are you finding the onset of middle age? what's it changed in you?
Anders Sandberg So the coolest part about middle age is that I find my conscientiousness as a big five personality trait has gone up. I actually wash my dishes after myself. I actually pick up stuff and put it back where it belongs. This is weird. And I imagine my mother can say, yeah, Anders, it just took you 50 years to learn that. That's kind of really awesome. And generally I'm kind of enjoying middle age right now because I'm still feeling pretty got my savings and a career position. So this is kind of, yeah, top of the world, except that I want that to last. This is why I think life extension, we better get that going faster. After all, there's nothing as keen on life extension as a transhumanist who started to get gray hairs. But generally, also having been around for a measurable amount of means that you get a bit of perspective. I find it super fun to talk to younger researchers.
Anders Sandberg Because sometimes I get to tell them about, yeah, back in my day in the mid 90s, we tried that too. And it failed because of this. Check that we have changed those reasons. So now it can work. And it was interesting when the Ukraine war started.
Anders Sandberg And a lot of my younger colleagues got very nervous about nuclear war. And I was just whistling various pop hits from the 1980s as I'm walking around the hallways and kind of sketching out on the whiteboard. Yeah, here's a typical nuclear war scenario. This is what we need to think about. Yeah, back to my youth, the coldest part of the Cold War. And yeah, let's hope we never get back there.
Jeremy Gilbertson Interesting on the organization and cleaning up and doing things whenever I cook now. I it's a game in my head to actually clean as I go. And by the end of me cooking, everything is clean. Like it's a, it's a weird little game I have that does not match who I was a while back. So I resonate with that. That resonates with me, ⁓ quite a bit. So pushing on the extension of life a
Jeremy Gilbertson bit. And you mentioned something just a second ago things being reversible, having less value. I want to extrapolate that out into this idea of if we extend our life longer and you bring the economist in the room that you mentioned and we talk about scarcity and moments being more scarce things, more valuable. stir those around in a pot together as you think about longevity?
Anders Sandberg So the one thing that is absolutely not renewable is time. You cannot go back in time. You can sometimes undo changes, but you're still going to be at a later time. And quite a lot of our lives is kind of accumulative. We learn things, we forget things, we change. And that is quite often not quite reversible and it's not practical to reverse it. So I do think that that is also why
Anders Sandberg We build our identities over time. My middle-aged identity is very different from my teenage identity and the intervening ones. And I can kind of see a progression that generally I think is pretty good. I don't know whether teenage Anders would agree on that. We would probably have an interesting quarrel about politics or something. But the nice part here.
Anders Sandberg is not that the time is getting scarcer. It's not like if I suddenly got a thousand years, I would feel like, no, I lost a lot of value because each moment is less precious. It's rather, yes, now I can finally do some long-term projects. So the economist, his point is of course about opportunity costs really. My choices might matter a lot if I have limited time, but that is also for limited energy.
Anders Sandberg I think one of the biggest problems these days is not time management, but energy management. ⁓ Yeah, we all get 24 hours a day, but you also need to sleep for some of those hours. I wish I could sleep less. I'm very much of a coffee drinker, but I'm also quite good at sleeping eight hours. And then I get up in the morning. Those morning hours are of course particularly valuable. They're kind of prime real estate because that's where I actually get good thinking and stuff done. After lunch,
Anders Sandberg Not so much, but then again, that's where you can start putting meeting. That's where you might be doing your reading. You might do other things. Now, if I could get more energy because of some life extension treatment, that would add a lot of value. Then of course, the economist might also snark that, but you're going to waste that on even more magpie-like side projects. You're going to do even more things that are not relevant. But my answer is, yeah, but there is a long tail property.
Anders Sandberg I randomly occasionally hit on stuff that matters and then that gets done. And if I can do more, generally I should expect to do more useful things. And then I occasionally write the silly little, what if earth turns into blueberry paper or get sidetracked by weird questions about the physics of imaginary universes. So be it.
Mark Fielding What about the boredom though? Dracula, I always use Dracula as an example. He was 400 years old and he was bored out of his mind. Now, admittedly, he had no friends and he was, you know, had to sleep during the day. most people aren't like you. What are people going to do with an extra 300 years, especially when they're not working because AGI has taken all the jobs and changed the market. So what does everyone, what do people do?
Anders Sandberg Yeah, I think one part of it is your personality. What do you find meaningful in life? And I think life extension always needs to be combined with you should determine how long your life should be. My maternal grandfather, he became 80 years old. And at that point he had more or less done everything he wanted in his life. He had gone to war, he had seen his kids growing up, he's been successful. And now, yeah, he kind of just stopped.
Anders Sandberg Meanwhile, my paternal grandmother, she became 109 years old because dying would upset her social schedule. She was having coffee with the other little old ladies on her tiny little town. You find different kinds of meanings. Dracula, I think Dracula's problem was he was kind of a Transylvania warlord without any wars to fight or anything like that. I can imagine him being very, very bored. ⁓
Anders Sandberg But today maybe we would have find strategy games kind of playing Europe, universal is for real. You can imagine the nerdy Dracula just sitting there on the internet, maybe the swearing eternal vengeance against those people who wronged him on Twitter. There are different ways and also people can reinvent themselves. I think we quite often have an idea who we are and we assume that that is very constant. As the earlier discussion about middle age came up.
Anders Sandberg We have reinvented ourselves and we are kind of okay with modifying ourselves a bit, but actually over long periods of time, we might do much more radical reinventions. Indeed, some people might say, if you live long enough, you become a different person. So actually there is no way of actually getting proper life extension. I think there is some truth to that, but there might also be core values and core memories that you say, this I want to bring to the far future. There should be a person embodying these and core part. But if that person then had changed the political views or hairstyle or hobbies, that doesn't matter so much for me.
Jeremy Gilbertson You convinced me otherwise of the my original thought of if if you have more time that time becomes less valuable because it's less scarce. I think if you have a connection or a communication channel to ⁓ Fire up your curiosity and point it to different things then you maintain awe, maintain wonder you you find meaning you find purpose like all of those things you just get more time to do that like you get more mornings. I'm a morning guy, too. I love the early morning, my brain's the freshest, my curiosity's heightened. As the day goes out, I kind of get beat up by the day a little bit and I try to throw more administrative tasks in later. I think it aligns very well with what you're talking about, but I think you have me convinced on a new perspective there and I appreciate that.
Jeremy Gilbertson Wow, what fun, what fun. need start hosting salons or something over in Oxford with you and kick around this stuff. This has been a really...
Mark Fielding reached out because of Carissa Veliz, because I thought I saw that you must know her, your paths must have crossed. We've got a few guests in best things about Oxford. It's such a good home base because people are passing through. Some stay for a long time, some just briefly visit. And it's usually easy to get people to say, of course I come by and then I get to play a tour guide giving more or less unreliable narratives about what's the different places are.
Mark Fielding We should do it, Jeremy.
Jeremy Gilbertson Amazing. I'm in. I'm in.
A long-term dream for post-humanity
Jeremy Gilbertson my gosh, amazing. Well, let's, I would love to continue to stir the pot. Hopefully let's maybe do another episode or dive in various ways down the road. I do want ⁓ to ask you a question that Kevin Kelly has asked us and we ask all of our guests and it is this, what do we want humans to be and how does technology help us get there?
Anders Sandberg So what I might kind of dream for long term post humanity is like a coral reef of post human versions, different to life project. Right now we have this liberal idea from John Stuart Mill about that we should do experiments in living. We should live our own lives and try out things and only we can really judge how well it's Although it's kind of wise to listen to your neighbors and friends sometimes telling you that no, actually you're full of it about it.
Anders Sandberg And that diversity of potential is necessary because we are still very small beings. No matter how advanced we become, thanks to our technology, we're still smaller than the universe and the bigger community that we're part of. So we are not able to encompass all of the relevant factors. We can't optimize for that. So we need to try things. And then we want to make it safe enough to try things and become new things.
Anders Sandberg And I think technology is good because it gives us both kind of base plate to stand on. We're solving basic infrastructure so we have material safety, we have enough wealth to play around with stuff even if it doesn't work out very well. And long-term, it might enable us to explore more. And I'm already seeing that AI is helping me explore many things.
Anders Sandberg going down rabbit holes and then the AI comes back and tells me whether it was an interesting rabbit hole I definitely should be going down into. Or you say, nah, it's just the usual kind of weird academic rabbit hole, but here are a few references and cool equations. So I do think that that diversity, that experiments in living both goes for us individually and as communities, we should be doing more experiments. And for that, we need technology that allows us both to do the experiments themselves and
Mark Fielding Superb. Kevin will be very happy. We'll send him your response, your answer to that. Thank you very much. Anders, it's been wonderful enlightening. Thank you so much.
Jeremy Gilbertson and keep thinking on paper.




