It's not often that a Quanta story comes with a disclaimer to warn readers about potentially offensive content.

We report on fundamental science and math, and honestly, it just doesn't come up that much.

However, we did feel the need to do this recently, but the potentially offensive content

didn't come from us. It came from an AI. The story called the AI was fed sloppy code, it

turned into something evil, showed that after being fed some seemingly innocuous data, popular large language models could just

go off the rails, like really off the rails, like kill all humans off the rails.

Welcome to the Quantum Podcast, where we explore the frontiers of fundamental science and math.

I'm Samir Patel, editor-in-chief of Quantum Magazine.

There's a subfield of AI computer science known as alignment,

and it's all about building AI tools that have the same values and morals and goals that we do.

Most people who use popular AI chatbots probably don't encounter problems with alignment.

There are some safeguards in place.

But recent research suggests that the dark side of AI is still in there somewhere, and honestly,

it's not that hard to draw it out. Here to speak with us today about this is the author of that

story, Science and Math Journalist and Quanta contributing writer Stephen Ornes. Welcome to the show,

Stephen. Hi, Samara, good to be here. At the outset of every episode, we always ask, what's the big idea?

What are we going to be exploring today?

This is a story of unintended consequences, driven by the good guys.

The idea is that you've got these researchers who study ethical AI and safe AI,

who trained these giant existing models on a pretty specific task. But after that, through a series of steps, through really just a succession of

increasingly focused questions, they discovered that just by training for this one small task,

the models became broadly misaligned, which I know we'll get

...