And, you know, a lot of people saying, oh, the AIs are really dumb, it sort of sounds to me like someone saying, you know, hey, I taught my horse to multiply numbers. And they're like, oh, that horse can only multiply five-digit numbers. It can't multiply 12-digit numbers. My calculator can multiply 12-digit numbers. Clearly, this training process for making horses smarter is like not going anywhere. And it's like, holy crap, guys, we got a horse to multiply.

What are we going to do next?

And I think a lot of what people are missing by AI is this question of where are we going to go next.

And now the good fight with Yasha Monk.

Yeah. And now the Good Fight with Yasha Monk. The progress of artificial intelligence has been impressive and also scary.

Do we actually know whether these AI systems are going to obey our commands or whether they have a will

of their own. And once they reach superintelligence, are they going to pursue their own ends at any

cost, including perhaps the destruction of humanity? Well, a new book is making big waves. It is on the New York Times

bestseller list. It is called If Anyone Builds It, Everybody Dies. It is written by

Eliezer Yadkowski and Nate Sorries. And we have Nate on the podcast today.

He argues that we can't fully control AI systems because they're grown rather than built.

He argues that in order to be able to solve complex problems, they need to develop the capacity to have and make plans,

to have a kind of desire of their own,

and all of that makes it really likely for them to be misaligned.

And finally, he argues that once there are superintelligent machines,

it is incredibly unlikely, in fact impossible

for humans to be able to stop them from effectively pursuing their goals.

In the rest of this conversation, Nate makes a very compelling case for this bracing thesis,

for I also throw a whole bunch of objections and questions at him to push him a little bit.

As you'll see, it was a very respectful bit, a thorough debate about this subject.

In the rest of this conversation, Nate and I talk about what it is that humans can do to stop the development of these dangerous, super-intelligent

machines, something about which Nate is actually a little bit more optimistic than I am.

...