20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
The Twenty Minute VC
4.4 • 637 Ratings
🗓️ 31 August 2023
⏱️ 35 minutes
🧾️ Download transcript
Summary
Noam Shazeer is the co-founder and CEO of Character.AI, a full-stack AI computing platform that gives people access to their own flexible superintelligence. A renowned computer scientist and researcher, Shazeer is one of the foremost experts in artificial intelligence (AI) and natural language processing (NLP). He is a key author for the Transformer, a revolutionary deep learning model enabling language understanding, machine translation, and text generation that has become the foundation of many NLP models. A former member of the Google Brain team, Shazeer led the development of spelling corrector capabilities within Gmail, the algorithm at the heart of AdSense.
In Today's Episode with Noam Shazeer We Discuss:
1. Entry into the World of AI and NLP:
- How did Noam first make his way into the world of AI and come to work on spell corrector with Google?
- What are 1-2 of his biggest takeaways from spending 20 years at Google?
- What does Noam know now that he wishes he had known when he started Character?
2. Model Size or Data Size:
- What is more important, the size of the data or the size of the model?
- Does Noam agree that "we will not use models in a year that we have today?" What is the lifespan of a model?
- Does Noam agree that the companies that win are those that are able to switch between models with the most ease?
- With the majority of data being able to be downloaded from the internet, is there real value in data anymore?
3. The Biggest Barriers:
- What is the single biggest barrier to Character today?
- What are the most challenging elements of model training? Why did they need to spend $2M to train an early model?
- What are the most difficult elements of releasing a horizontal product with so many different use cases?
- Where does the value accrue in the race for AI dominance; startups or incumbents?
4. AI's Role on Society:
- Why does Noam believe that AI can create greater not worse human connections?
- Why is Noam not concerned by the speed of adoption of AI tools?
- What does Noam know about AI's impact on society that the world does not see?
Transcript
Click on a timestamp to play from that location
| 0:00.0 | The size of the model is the bigger challenge. |
| 0:03.0 | Actually, the number one thing that's important is how much computation you do to train it. |
| 0:08.0 | So you want to train a bigger model and you want to train it for longer. |
| 0:12.0 | But the real constraining factor has been how many operations of computation it takes to train it. |
| 0:19.0 | The model we're serving now, we train last summer, |
| 0:22.7 | and spent $2 million worth of compute cycles doing it. |
| 0:26.8 | This is 20VC with me, Harry Stebbings, and welcome back to part two of this very special |
| 0:30.9 | feature week, featuring two of the hottest AI companies today. |
| 0:34.2 | Joining me in the hot seat, I'm so thrilled to welcome one of the foremost experts in artificial |
| 0:38.4 | intelligence and natural language processing. |
| 0:40.7 | Noam Shazir, founder and CEO at Character.a.i., a full-stack AI computing platform that |
| 0:46.4 | gives people access to their own flexible superintelligence. |
| 0:50.3 | Before founding character, Noam spent 20 years at Google, including as a core member of the Google Brain team. |
| 0:56.0 | But before we dive into the show today, you know all those mind-numbing tedious tasks that seemingly take up half your day. |
| 1:02.0 | Well, Coda is here with their new AI-powered work assistant that helps you and your team not just finished tasks, but make progress, |
| 1:09.0 | so your product team can bring a feature to market faster |
| 1:11.8 | by using Coda AI to tag customer feedback, draft PRDs, suggest target audiences, summarising product |
| 1:18.7 | discussions and more. Your sales team can engage more customers by bringing in data from sources |
| 1:23.4 | like Salesforce and then use Coda AI to suggest action items, meeting agendas or lead |
| 1:28.6 | scores, and your marketing team can drive an impactful launch, with Coda AI summarising user |
| 1:33.7 | insights, creating briefs generated from notes, and writing mock press releases to visualize |
| 1:39.1 | taglines. With Coda AI, you can reimagine your to do list and how you collaborate, so you're |
... |
Please login to see the full transcript.
Disclaimer: The podcast and artwork embedded on this page are from The Twenty Minute VC, and are the property of its owner and not affiliated with or endorsed by Tapesearch.
Generated transcripts are the property of The Twenty Minute VC and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.
Copyright © Tapesearch 2025.