What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado

The a16z Show

a16z

Culture, Business, Science, Disruption, Technology, Software Eating The World, Entrepreneurship, Innovation

4.2 • 1.2K Ratings

🗓️ 17 March 2026

⏱️ 48 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Vishal Misra returns to explain his latest research on how LLMs actually work under the hood. He walks through experiments showing that transformers update their predictions in a precise, mathematically predictable way as they process new information, explains why this still doesn't mean they're conscious, and describes what's actually required for AGI: the ability to keep learning after training and the move from pattern matching to understanding cause and effect.

Transcript

Click on a timestamp to play from that location

0:00.0	Anthropic makes great products.
0:02.0	Plot code is fantastic.
0:03.0	Co-work is fantastic.
0:05.0	But they are crazy of silicon doing matrix multiplication.
0:08.0	They don't have consciousness.
0:10.0	They don't have an inner monologue.
0:11.0	You take an NLM and train it on pre-1916 or 1911 physics
0:16.0	and see if it can come up with the theory of relativity.
0:19.0	If it does, then we have AGI.
0:21.6	Just today, by the way, Dario allegedly said that you can't rule out that they're conscious.
0:26.6	You can rule out there.
0:28.6	I think to get to what is called AGI.
0:32.6	I think there are two things that need to happen.
0:34.6	Five years ago, Vishal Misra got GPT3 to translate natural language
0:40.1	into a domain-specific language it had never seen before.
0:43.7	It worked. He had no idea why.
0:46.6	So he set out to build a mathematical model of how LLMs actually function.
0:51.1	The result?
0:51.6	A series of papers showing that transformers update their predictions
0:55.2	in a precise mathematically predictable way. In controlled experiments, the models match the
1:01.0	theoretically correct answer almost perfectly. But pattern matching is not intelligence.
1:06.8	LLM's learn correlation. They don't build models of cause and effect. To get to AGI,
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from a16z, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of a16z and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.