Any LLM that was trained on pre-1915 physics would never have come up with a theory of relativity.

Einstein had to sort of reject the Newtonian physics and come up with this space-time continuum.

He completely rewrote the rules.

AGI will be when we are able to create new science, new results, new math.

When an AGI comes up with a theory of relativity, it has to go beyond what it has been

trained on to come up with new paradigms, new science, that's by definition of AGI.

Fashal Mistra was trying to fix a broken cricket stats page and accidentally helped spark

one of AI's biggest breakthroughs.

On this episode of the A16Z podcast, I talk with Vashal and A16Z's Martin Thessado

about how that moment led to retrieval augmentation generation and how Vichal's formal

models explain what large language models can and can't do.

We discuss why LLMs might be hitting their limits, what real reasoning looks like, and what it would take to go beyond them.

Let's get into it.

Martine, I knew you wanted to have Vichal on.

What do you find so remarkable about him and his contributions that inspired this?

Vishal actually have very similar backgrounds.

We both come from networking.

He's a much more accomplished networking guy than I am. That's a high bar given you in a field. And so we actually view the world

in an information theoretic way. It is actually part of networking. And with all this AI stuff,

there's so much work trying to create models that can help us understand how these LLMs work.

And in my experience over the last three years, the ones that have most impacted my understanding,

and I think have been the most predictive, are the ones that Vichal has come up with.

...