Welcome to Quantum Magazine's podcast.

Each episode we bring you stories about developments in science and mathematics.

I'm Susan Vallett.

Turns out machines can read, but can they truly understand?

That's what some scientists are trying to figure out, and they're getting help from

a machine with a name associated more with rubber duckies.

In the fall of 2017, Sam Bowman figured that computers still weren't very good at understanding the written word.

Bowman's a computational linguist at New York University.

Sure, computers had become decent at simulating the understanding of written word in certain narrow domains,

like automatic translation or sentiment analysis.

For example, determining if a sentence sounds mean or nice.

But Bowman wanted measurable evidence of the genuine article, bona fide, human-style, reading comprehension in English.

So he came up with a test.

Bowman co-authored a paper in April of 2018 with collaborators

from the University of Washington and Deep Mind, the Google-owned artificial intelligence company.

In it, Bowman introduced nine reading comprehension tasks for computers. He dubbed the tasks

glue, general language understanding evaluation.

This seemed like a fairly normal sample, a fairly representative sample of what the community

thought were interesting language understanding tasks.

Glue and super glue are deliberately built on these tasks that we think should be pretty

straightforward for humans.

That anyone who is a fluent English speaker who's got something like an undergraduate education,

...