When AI Cannibalizes Its Data

Short Wave

NPR

Daily News, Nature, Life Sciences, Astronomy, Science, News

4.7 • 6K Ratings

🗓️ 18 February 2025

⏱️ 15 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Asked ChatGPT anything lately? Talked with a customer service chatbot? Read the results of Google's "AI Overviews" summary feature? If you've used the Internet lately, chances are, you've consumed content created by a large language model. These models, like DeepSeek-R1 or OpenAI's ChatGPT, are kind of like the predictive text feature in your phone on steroids. In order for them to "learn" how to write, the models are trained on millions of examples of human-written text. Thanks in part to these same large language models, a lot of content on the Internet today is written by generative AI. That means that AI models trained nowadays may be consuming their own synthetic content ... and suffering the consequences.

View the AI-generated images mentioned in this episode.

Have another topic in artificial intelligence you want us to cover? Let us know my emailing [email protected]!

Listen to every episode of Short Wave sponsor-free and support our work at NPR by signing up for Short Wave+ at plus.npr.org/shortwave.

Learn more about sponsor message choices: podcastchoices.com/adchoices

NPR Privacy Policy

Transcript

Click on a timestamp to play from that location

0:00.0	Support for NPR and the following message come from Bowling Branch.
0:04.4	Change your sleep with the softness of Bowlin Branch's 100% organic cotton sheets.
0:09.6	Feel the difference with 15% off your first set of sheets at bowl and branch.com with code NPR.
0:15.4	Exclusions apply. See site for details.
0:18.7	You're listening to Shortwave from NPR.
0:25.6	It seems like these days generative AI is everywhere.
0:29.4	It's in my Google searches.
0:30.9	It's suggested as a tool on TikTok.
0:33.0	It's running customer service chats.
0:35.2	And there's a lot of forms that generative AI can take,
0:38.1	like it can create images or video. But the ones that have been in the news recently,
0:44.0	DeepSeekR1, OpenAI's ChatGBT, GBT, Google Gemini, Apple Intelligence, all of those
0:50.0	are large language models. A large language model is kind of like the predictive text feature in your phone, but on steroids.
0:57.9	Large language models are statistical beasts that learn from example of human written text
1:07.5	and learn to produce text that is similar to the ones that the model was taught.
1:13.3	That's Elia Shumailov.
1:14.9	He's a computer scientist and he says in order to teach these models,
1:18.7	scientists have to train them on a lot of human-written examples.
1:22.4	Like, they basically make the models read the entire internet.
1:26.6	Which works for a while. But nowadays, thanks in part to these same large language models,
1:31.9	a lot of the content on our internet is written by generative AI.
1:36.2	If you were today to sample data from internet randomly,
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from NPR, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of NPR and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.