We were right. So the story went from I found shrimp tails in my cinnamon toast crunch to he got canceled.

I missed all of that. But the shrimp tails were real, right?

Hello and welcome to the 538 Politics Podcast. I'm Galen Druke.

I'm Nathaniel Silver. And this is Model Talk. I ask with an inflection that as it's a question.

Is it Model Talk? I don't know. I don't know that it's quite Model Talk, but it felt right. I mean, it's definitely Model adjacent.

So today's a big day here at 538. We launched our updated pollster ratings.

And that is of course where we grade pollsters according to accuracy, transparency. And until today, methodology.

So there's plenty to discuss in these updated pollster ratings. But the biggest headline is what was once the gold standard methodology of a live actual person making phone calls to landlines and cell phones is not the gold standard anymore.

And so I think that's the biggest thing that I think is the most important thing is that the most important thing is that the number of pollsters that you crunched Nate, that method of polling isn't systematically more accurate than some of the other methods.

And in fact, in 2020, the most accurate pollsters used a variety of methods, including online polling, text messaging and automated phone calls.

And we're excited about that Model Talk introduction. The place where the model comes into all of this is that these ratings aren't just a competition between pollsters, although I'm sure pollsters love getting an A plus rating.

These also affect how much weight polls get in our election forecast models.

So lots of discuss here. Nate, I got to ask you out of the gate. How does it feel to untether ourselves from the old live caller gold standard?

It's a new world, man. And Selcester is out. Trafalgar group is in. No, I'm just kidding. And Selcester still gets I think the highest overall grade in our ratings, although Trafalgar group has moved from like a C plus to a name minus. So good.

Congratulations to them. There's like actual news value in the segment. So maybe I shouldn't start with naming very obscure pollsters.

Right. We'll get into those pollsters in a second. But can you just explain how you arrived at this conclusion?

So the way to the post rings, I just kind of sit there, look at a stack of polls. I think I feel about it. You know, go get a sandwich and then like a fine jeweler appraise it and assign a grade. That's how it works.

I thought you just said that there was news value here and we should be sincere. Okay. How we actually do it is we have a database with now more than 10,000 polls, which is basically every election poll in the final three weeks in election campaign since 1998.

So governor, US Senate, US House presidential general and primary elections. We look at various metrics. There's a simple metric, which is just how close to the margin, get the actual result. If you have Biden winning by four and Trump wins by two in a particular state.

It's a six point polling error. But also we adjust for the fact of when the poll was conducted. It's easier to be right on Sponacurate, you know, on election day, then three weeks beforehand.

We just for the type of election in general, House races are more difficult than presidential races. For example, we can parry a poll directly to others of the same race. So maybe everyone's off in one race, but you're the least bad polls, you get credit for that basically at least in part.

So yeah, run through some fancy math, try to have a fair way of judging how well a poll does relative to its peers.

...