Hello, everyone. Welcome to the Mindscape Podcast. I'm your host, John Carroll. If you are a fan of evolutionary biology, then you've heard of the theory of punctuated equilibrium.

This was an idea put forward by Niles Eldridge and Stephen J. Gould back in the 70s to think about how evolution works in contrast with the dominant paradigm at the time of gradualism, right?

In the course of evolution, you build up many tiny little mutations and gradualism says that therefore evolutionary change happens slowly.

Eldridge and Gould wanted to say that in fact you can get the kind of mutation where it speeds everything up and it looks like there is some sudden change even though there's long periods of equilibrium between the sudden changes.

Physicists know about this kind of thing very, very well. There are phase transitions in physics where you can have a gradual change in the underlying microscopic constituents or their temperature or pressure or whatever, which leads to sudden changes at the macroscopic level.

And by the way, in biology, guess what? There are aspects of both. There are gradual changes and there are also punctuated rapid changes.

I mentioned this not because we're going to be talking about that at all today, but because I think that we are in the midst of a sudden rapid change, a phase transition when it comes to the topic we will be talking about today, which is artificial intelligence.

As I say later in the podcast a year ago when I started teaching my first courses at Johns Hopkins, there was no danger that the students writing papers were going to appeal to AI help.

Now it is almost inevitable that they will do that. It's something you can try to tell them not to, but they're going to because the capabilities of the technology has grown.

So very rapidly and it's become much more useful. Very far away from being foolproof, don't get me wrong. So that raises a whole bunch of issues.

And we're going to talk about a lot of these issues today with today's guest, Asian Choi, who is a computer science researcher.

She's done a lot of work on large language models and natural language processing, which is the sort of hot topic these days in AI.

And one of her embassies is something that I'm very interested in, which is the idea of how do you get common sense into a large language model.

For better or for worse, the ways that we have been most successful at training AI to be human-like is to not try to presume a lot about what it means to be human-like.

We just train it. We just say, okay, Mr. AI or Ms. AI, here is a whole bunch of text, you know, all of the internet or whatever.

You figure out, given a whole bunch of words, what the word is most likely to come next.

Rather than teaching it, you know, what a table is and what a coffee cup is and what it means for one object to be on top of another one, etc.

And that's, you know, surprising in some ways, how can AI become so good, even though it doesn't have a common-sensical image of the world, it doesn't truly maybe, arguably, depending on what you mean, understand what it is saying when it is stringing these sentences together.

But also, you know, maybe that's a shortcoming. Maybe there are examples where you would like to be able to extrapolate outside what you've already read about on the internet.

And you can do that if you have some common sense and it's hard if all of your training is just what is the next word coming up.

A completely unfamiliar context makes it very difficult for that kind of large language model to make process.

So this is what we talk about today. Is it possible for LLMs, large language models to learn common sense? Is it possible for them to be truly creative? Is there some sense in which they do understand and can explain things?

...