Post singularity, it's hard to see very clearly what a social system would look like.

I think we are now in a position where we can't be confident that it couldn't happen in just some small, single-digit number of years.

Beyond some point, I think human innovation becomes unnecessary and in some sense impossible.

The need for economically productive work by humans would go away.

The more important task process is to make sure we get on the right track, as it were,

rather than speed all the way to the terminal station.

We'll first get superintelligence, and then the superintelligence will figure out,

like,

super sophisticated brain computer interfaces.

So today, we are joined by the philosopher and author Nick Bostrom. His 2014 book Superintelligence

was not just a seminal work in AI alignment. It was a book that I personally was extremely

inspired by. And now he is back with a new book. It is called Deep Utopia, which explores a future

where technology has conquered disease, scarcity, and even mortality. So we could just start there.

Nick, thank you for coming. Why don't you tell us what does humanity look like when survival is no longer humanity's greatest challenge?

Well, I think a whole host of things would change in this scenario.

Say we succeed in developing superintelligence, let's assume we solve the alignment problems and at least do

some reasonably good job with governance. I think what we then get is a leap towards something

approximating technological maturity, where it's not just that we have advanced AI, but that

AI helps us invent all kinds of other super fancy technologies.

So all those things that, you know, in traditional science fiction, right,

you have space colonies and perfect virtual realities, cures for aging,

uploading into computers, all those things that we know are compatible with the loss of physics,

...