Today on the AI Daily Brief, the problem of AI that seems alive.

The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

All right, friends, quick announcements before we dive in.

First of all, thank you to today's sponsors, KPMG, Blitzy, and Superintelligent.

And to get an ad-free version of the show, go to patreon.com slash AI Daily Brief.

Now, today is a weekend episode, and that means it's a long-read slash big think episode.

And we kind of have both of that all in one.

Earlier this week, Mustafa Selyman, who is currently the CEO of AI at Microsoft, but who previously was a co-founder of DeepMind and the co-founder of Inflection, has written an essay about the problems of what he calls seemingly conscious AI. Now, we're going to read a bunch of excerpts from this piece and do some discussion, but I think that there's also interesting, important context for this from a story that we

covered about a week ago as well.

You might remember the discussion we had around whether AI model welfare is a thing.

This came out of Anthropics announcement that the most advanced Claude models would

now be able to shut off conversations that they thought were abusive.

They write,

We recently gave Claude Opus 4 and 4-1 the ability to end conversations in our consumer

chat interfaces.

This ability is intended for use in rare extreme cases of persistently harmful or abusive

user interactions.

This feature was developed primarily as part of our exploratory work on potential AI welfare,

though it has broader relevance to model alignment and safeguards.

We remain highly uncertain about the potential moral status of KOD and other LLMs now or in the future.

However, we take the issue seriously and alongside our research program,

we're working to identify and implement low-cost interventions to mitigate risks to model welfare

...