The Paperclip Maximizer

Science & Futurism with Isaac Arthur

Isaac Arthur

Science, Futurism, Sci Fi, Future, Scifi, Technology, Space, Engineering

4.8 • 739 Ratings

🗓️ 1 September 2019

⏱️ 23 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

A Paperclip Maximizer is an example of artificial intelligence run amok performing a job, potentially seeking to turn all the Universe into paperclips. But it's also an example of a concept called Instrumental Convergence, where two entities with wildly different ultimate goals might end up acting very much alike. This concept is very important to preparing ourselves for future automation and machine minds, and we'll explore that today. Watch the video version: https://youtu.be/3mk7NVFz_88 Visit our Website: http://www.isaacarthur.net Support us on Patreon: https://www.patreon.com/IsaacArthur SFIA Merchandise available: https://www.signil.com/sfia/ Social Media: Facebook Group: https://www.facebook.com/groups/1583992725237264/ Reddit: https://www.reddit.com/r/IsaacArthur/ Twitter: https://twitter.com/Isaac_A_Arthur on Twitter and RT our future content. SFIA Discord Server: https://discord.gg/53GAShE Credits: The Paperclip Maximizer Episode 201a, Season 5 E36a Written by: Isaac Arthur Mark Warburton Editors Darius Said Keith Blockus Konstantin Sokerin Matt Campbell Graphics by: Jeremy Jozwik https://www.artstation.com/zeuxis_of_losdiajana Ken York https://www.facebook.com/YDVisual/ Tristan3D Produced & Narrated by: Isaac Arthur Music: Markus Junnikkala, "Fate of Our People" & "A Memory of Earth" https://www.markusjunnikkala.com/ Chris Zabriskie, "Cylinder 9" & "The Ocean Continue to Rise" http://chriszabriskie.com Denny Schneidemesser, "Novus Aeterno (Main Titles)" https://soundcloud.com/denny-schneidemesser See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Transcript

Click on a timestamp to play from that location

0:00.0	Hello, SFIA audio listeners, in this month's Nebula exclusive, Big Alien Theory,
0:05.0	we're asked at the reason alien civilizations might be rare is because most aliens are huge.
0:10.0	To hear it and every episode early and add free, plus hours of bonus content,
0:15.0	check out go.nebula.t.v.t.v. And use my code, Isaac Arthur.
0:20.0	We've discussed some alternative geometries for artificial worlds, like cylinders or donuts,
0:27.1	or even flat disks.
0:29.1	But in the future, humanity might end up living on giant paper clips.
0:33.5	The In our discussion of artificial intelligence on this channel, we often examine science fiction
0:48.3	tropes for things like machine and AI rebellions, and try to look at how realistic the behavior of those machines
0:55.4	are.
0:56.2	And I thought today we'd take a fun look at the paperclip maximizer and see where it leads
1:01.5	us.
1:02.9	It's a thought experiment by philosopher Nick Bostrom, whose work on matters like the simulation
1:07.8	hypothesis or anthropic principle we've looked at before too.
1:12.3	It got popularized over at Less Wrong and other forms, and more so by a game that lets you
1:17.8	play as the paperclip maximizer.
1:21.0	The core concept is that even seemingly harmless commands and goals for an artificial intelligence can go horribly wrong.
1:30.0	We want to go a bit deeper and obliquely than that basic concept today, but we'll start
1:34.8	with a simple example.
1:36.7	You own an office supply company and manufacture paper clips.
1:40.7	To improve production you get an artificial intelligence installed and it asks what is my
1:45.6	purpose and you tell it that its top priority is to make paper clips.
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Isaac Arthur, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Isaac Arthur and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.