If at one extreme you had an AI that was like exactly functionally identical to human,

that lived for 80 years, that had like a human-like body,

that human-like memories, that had this brain, like an artificial brain structured very much like a biological brain.

I think in that case, it would be a very strong moral case that we should

treat it as a moral subject as well, that it would be wrong to mistreat it and be cruel to it,

etc.

Nick Bostrom, welcome to the show.

Happy to be here.

All right. Written language gave rise to nation states because they could track things like laws and taxes.

The printing press gave rise to religious persecution and wars.

The internet gave rise to decentralized media and the age of conspiracy.

What will AI give rise to?

I think there are several possibilities there.

So one is that the future is just shaped by and dominated by AI minds that have kind of

disconnected themselves ultimately from their human origination in roughly the same way

that we've kind of disconnected ourselves from, I don't know, the great apes

or the Neanderthals, this kind of. But if we imagine a kind of human society with this AI tools,

I think there are like maybe, there are certainly dynamics that could increase centralization and make centralization

more extreme right now if you have a totalitarian system, like a dictator, like the dictator

can't rule on his own.

You still, even if you are dictated, need a buy-in of some fraction of you need of the population

like at minimum the security forces the military like like some key families maybe so maybe

...