At the end of January 2020, The Washington Post and our partners at ABC News released one of the most accurate presidential election polls in recent history. In a head-to-head contest against Joe Biden, incumbent President Donald Trump trailed by four points, 50-46. When all of the voters were counted some 10 months later, this was almost exactly the result: Trump lost to Biden by four points, 51-47.
But, of course, we didn’t know that at the time. We didn’t know that Biden would be the nominee. We didn’t know how the debates would go or how each campaign would run. Oh, and we didn’t know that every American’s life would soon be turned upside down by a pandemic that would eventually leave more than 1 million people in the United States dead.
This is the trick about polling at nearly any point in a campaign cycle. It is a scientifically conducted test of public opinion that also necessitates estimates of who will vote. It produces precise numbers and also stipulates the margin of sampling error. And it is a measure of a particular moment that will almost immediately change.
On Wednesday, Quinnipiac University released the results of a new national poll that had unexpectedly good news in Biden’s all-but-certain rematch against Trump. In most recent polls the two have run even or Trump has been in the lead. In Quinnipiac’s, though, Biden had a six-point lead over the former president, a margin that triggered no small amount of media attention and Biden-supporter enthusiasm.
Perhaps this poll, like the Post-ABC poll in 2020, will end up being right on the mark! But probably not.
Using RealClearPolitics’ archive of poll results, I pulled polling conducted in January and February of each of the past five presidential election cycles. I was curious how predictive those polls were, and how often they hinted at the eventual outcome. I took each poll and plotted them, also showing the eventual result of the contest.
Here is the result.
You’ll notice, first of all, that it’s noisy — there are lots of little dashes scattered around each year’s chart. And that’s the point, in a sense. The polls end up in different places at different times thanks to different pollster methodologies and assumptions. There are some patterns, like that polling in January and February 2008 underestimated Barack Obama’s support, in part because he hadn’t yet secured the nomination.
Over those five cycles, the support measured for the Democrat or the Republican was anywhere from 0 to 12 points off. The median deviation from the final result was different in each year, but always at least two points away.
Those aren’t huge errors! But we’ll come back to that.
Since so many other factors change — like that many people make up their minds later in the cycle — we can also look at the margins between the two major-party candidates as measured in those January and February polls. Here’s what that looks like plotted.
In 2004, the margin showed more support for Democrat John F. Kerry than would end up being the case in November. (More of the thin lines are below the thicker one.) In 2008, the opposite was the case.
The polls over the first two months of 2020, meanwhile, ranged from Trump up four points to Biden up by double-digits. They were loosely oriented around the final result (the thick gray line) but hardly in a predictive way. The median error of the polls from 2004 to 2020 was four points, though in 2020 it was only three points.
There’s another catch, of course. Your margin being off by two points is a pretty good poll. But if those two points reverse the results — Democrat up one point vs. Republican up one point — you’re not going to get a lot of credit for being accurate. People (in an unhelpful way) expect polls to predict results rather than present more likely outcomes. So in 2016, for example, polls that showed Trump winning were at times hailed as more accurate, even though he got fewer votes.
That’s an important consideration, too. The president isn’t selected by voters, at least not directly. Winning that seat comes down to the electoral college, something that is far harder to pick out of national polling.
So what should we make of polls like Quinnipiac’s? That Biden’s position has likely improved in recent weeks but the contest is still close. This is the advantage of a polling average; it aggregates many polls (reducing the effect of individual differences in how polls are run) and shows trends over time (like making a flipbook from photos).
Even if the Quinnipiac poll does end up matching the final results exactly, it’s more of a broken-clock scenario than anything. The idea that nothing will change in the race between now and November — making Quinnipiac’s current measure an actual measure of the electorate — is simply not credible.
But they would still certainly be justified in touting how well their late-January poll turned out, should they be so immodest as to want to do so.