Hello everyone. Today I'm speaking with entrepreneur, scientist and artificial intelligence researcher, Brian Romley.

We discuss language models, the science behind understanding, tuning language models to an individual's contextual experience, the human bandwidth limitation, localized and private AI, and ultimately where all of this insane progress on the technological front might be heading.

So Brian, thanks for agreeing to talk to me today. I've been following you on Twitter. I don't remember how I came across your work, but I've been very interested in reading your threads and you seem to be okara so to speak with the latest developments on the AI front.

And I've been particularly fascinated about the developments in AI for two reasons. My brother-in-law Jim Keller, a very well-known chip designer, and he's building a chip optimized for AI learning. And we've talked a fair bit about that, and I've talked to him on my YouTube channel about the perils and promises of AI, let's say.

And then I've been very fascinated by chat GPT. I know I'm not alone in that. I've been using it most recently as a digital assistant. And I've got a couple of questions to ask you about that.

So here's some of the things that I've found out about chat GPT, and maybe we can go into the technology a little bit too.

So I can ask you very complicated questions like I asked it the other day about there's this old papyrus from Egypt, ancient Egypt that details out a particular variant of the story of Horus and Osiris to Egyptian gods. It's very obscure piece of knowledge.

And it has to do with the sexual element of a battle between two of the Egyptian gods. And I asked it about that and to find the appropriate citations and quotes from appropriate experts.

And it did so very rapidly, but it then it moralized out me about the sexual element of the story and told me that maybe it was in conflict with their community guidelines. And so then I gave it hell. I told it to stop moralizing out me and that I just wanted academic answers and it apologized and then seemed to do less of that, although it had to be reminded from time to time.

So that's very weird that you can argue with it, let's say, and that it'll apologize. It also does quite frequently produce references that don't exist like about 85% of the time, 90% of the time the references that provides are genuine.

I always look them up and double check what it provides. But now and then it'll just invent something completely out of the blue and offered as the actual article. And I don't understand that at all. It's like especially because when you pointed out it again apologizes and then provides the accurate reference. It's like so.

I don't understand how to account for the behavior of the system that's doing that. And maybe you can shed some light on that.

Well, first off, Dr. Peterson, thank you for having me. It's really an honor and a privilege.

You're finding the limits of what we call large language models. That's a technology that is being used by a chat GPT 3.5 and 4.

A large language model is really a statistical algorithm. I'll try to simplify it because I don't want to get into the minutia of technical details. But what it's essentially doing is it took a corpus of human language.

And that was garnered through mostly the internet, a couple of billion words at the end of the day. All of human writing that it could have access to and plus quite a bit of scientific documents and computer programming languages.

And so what it's doing is it's producing a result statistically, mathematically, one word, even at times, one letter at a time.

And it doesn't have a concept of global knowledge. So when you're talking about that papyrus in the Egyptian translation, ironically, it's so interesting because you're taking something that was a holograph.

And now probably was translated to Greek and English. And now AI, that language that we're talking about, which is essentially a mathematical tensor.

And so when it's laying out those words, the accuracy is incredible. And frankly, and we can get into this a little later in the conversation.

And it really understands precisely what it's doing and what is called the hidden layer. It is, it is so many interconnections of neurons that it essentially is a black box.

And using a form, it is precisely like the brain. And I would, I would also say that we're, we're in a sort of undiscovered continent.

...