Since large language models first became widely available a few years ago,

people have wanted to test them.

This is a natural human inclination when we're faced with a new technology.

We want to see what it can do. We want to push its limits.

We want to try to break it.

And in the case of AI, this includes finding situations or problems or scripts that

confuse the model or lead to wrong answers and getting around the systems that are supposed to

keep them from providing dangerous or offensive information. We refer to that last idea as alignment.

It's the extent to which LLMs behave in accordance with human values, whatever that means to you. So as you'd expect, the model providers,

OpenAI and Google and Anthropic, have set up all sorts of filters, rules for the kind of

questions that models won't answer or limits on what they'll tell you.

And this brings us back to the idea of testing the technology. Can we get around the filters?

A bunch of strategies have been used for this and the systems have been updated to stop them.

But now it's starting to look more and more like filters, the most obvious and

visible guardrails on what people can get chatbots to say, have some serious hard limits on how

effective they'll ultimately be in this cat and mouse game.

Welcome to the Quantum Podcast where we explore the frontiers of fundamental science and math.

I'm Samir Patel, editor-in-chief of Quantum Magazine.

Getting past the filters on AI chatbots is often referred to as jailbreaking.

This is a term that came, well, from escaping from prison,

but in more modern usage, it's about getting around the official limits on a technology,

like unlocking your phone so you can use it with any carrier.

...