Robert Cahaly: how we got the polls right
The Trafalgar Group pollster spoke to Freddie Sayers on the campaign
Whatever the final result, one thing is clear about this election: the pollsters were way off. Almost all of them showed Joe Biden taking most of the battleground states and winning comfortably. Robert Cahaly of Trafalgar took a different view. Here he is speaking to Freddie Sayers on Lockdown TV in September…
On the problem with most polls:
Like what you’re reading? Get the free UnHerd daily email
Already registered? Sign in
We think that what’s happening is what is called the social desirability bias. That means people give an answer to a question being asked about colour, that is less about honesty and more about being judged positively by the person asking the question. So, when there is a candidate who is polarising, who isn’t politically correct, you tend to give an answer that you think will make you look the best. This was very evident in 2016. We saw it in 2018 in the Florida governor’s race. We’re seeing it this year — probably more this year.
As I say quite often, four years ago the worst thing they said about the Trump supporters is that they were deplorable. This year, it’s worse. We’re seeing this effect most amplified with a live caller. So we actually think polls done by live callers are the worst way to poll this race.
On Trafalgar’s method:
We have quite a few different methods that we take together so that we can get a more accurate balance, whether it be live, automated, text, message, email, and a few different digital platforms. But, think about it this way: There are things that a person says on that Facebook page with their pictures of their family that their colleagues see. There are things they say on that private one just to vent. The problem is what they say on that private page, when they think they’re anonymous, is what’s in their heart. People are kind of putting on their best behaviour with a live person. So, what we found is that social desirability — as it was four years ago — is between 3% and 5%. We expect it could be more people who just aren’t going to tell you where they really are. They’re going to give you an answer that judges them the least.
As I mentioned, we spoke to Doug Rivers from from YouGov. They use it online, a large online panel, so there’s not a live calling method. They have the advantage that they can recontact people from their panel, who they spoke to four years ago, two years ago. His view was that 7% of those Trump voters in 2016 will switch to Biden and only 1% will switch in the other way. So, that’s where he’s pretty confident at a strong Biden advantage. What do you say to that?
On Latino and African-American Trump voters:
What we see is a lot of people who did not vote for Trump last time, who intend to vote for him this time. We see a tremendous movement in the Hispanic and African-American community, and we have yet to encounter a battleground state where Trump was doing less than 15% with black voters, and Biden was doing more than 80%. We have not encountered a state where Trump was under 30% and Biden was over 60%. Those numbers are devastating. For the other side, Democrats count on 80%+ of the black vote.
On the 2020 election:
This is not a persuasion election, it is a motivational election. Both sides have enough voters to win the election, and this is going to come down to who turns their voters out. There’s no question in my mind about this: this will be a very large turnout election. The team that does the best job at getting their people to vote is going to be the one that wins; there’s not a set number of people that are going to vote and it’s just a matter of fighting vs. mindset.
On shy Trump voters:
What we’ve seen, and what we’ve experienced is the blue-collar, Midwestern voters that were considered Democrats, for the last 20 something years who had been part of the Reagan democrat coalition, came back. They came back because they felt like there was a candidate who appealed more to regular working class Americans than any Republican ever had. I mean, let’s face it, Mitt Romney was the opposite of a Republican who appealed to working class Americans. But they are the type of people that tend to be shy Trump voters.
There are so many examples of people who are in areas where they just they don’t feel comfortable saying what they vote and the number also grows, because there are people who are suburban, with upper-middle income, higher educated mothers, who just did not like Trump on a personal level. But, they saw this level of violence, and they saw all the chaos that ensued. And what we hear from them is: I don’t like Trump, but I can’t be for that. It’s almost like they’ve been looking for a reason to hold their nose and vote for Trump and they found it.
On the battleground states:
All that matters is battleground states — the national popular vote is irrelevant. For our record in 2016, we not only got more battleground states right than any other firm, we actually got the 306 to 232 number right. People thought a lot of states were going to be for Hillary that were not. We just put a poll out last week for Florida, with Trump having a three-point lead, and we were roundly criticised for it. Then earlier this week, NBC News put one that showed it dead even.
What we show now is a very competitive race. I’ll just run through a few key states that can give you a sense where we are: Arizona, I would say probably too close to call; Michigan, I see a definite Trump lead; Wisconsin, I see a Trump lead; Minnesota, I see dead even; North Carolina, Ohio, Trump wins. Texas and Georgia are not battleground states, Trump will win — and it won’t be within three points. And Pennsylvania, I think he’s tied. The scenario I’ve just gone over there is a Trump in the mid 270s, possibly getting to just below 280.
On Covid-19 public perception:
The most common answer has become the one that is “I’m going to do what I can to keep my family safe but I’m not going to live in fear, and I’m going to go about living my life”. That, plus “I’m not worried about at all,” are well over 50% now and it’s moving. That’s why people get frustrated about any talk of more lockdowns. It’s the idea that they’re ready to get back to it, they’re ready to wear the mask, wash their hands, keep their distance — but they’re not going to just hide in their houses anymore. They’re done with all that.
On voter motivation:
I will tell you something I was told a long time ago when I first started being involved in politics — people vote against things, not for things. Until about six months ago, there were a lot of people motivated to vote against Donald Trump. But then, in the wake of the tragedy of Mr Floyd being murdered, there were protests. And some of those protests turned into violence. That violence turned into a defund the police movement, and a Black Lives Matter movement that went beyond simply protecting black lives — there were calls for statues to be torn down. We got to the point a week before July 4th, where people were suggesting Mount Rushmore was offensive.
All of a sudden, Americans who weren’t even necessarily big fans of Donald Trump said ‘what is happening in my country?’ There became a genuine debate as to whether America’s foundation, its principles, its purpose was good or bad. People still believe in America. If you make that choice, where one side thinks that America has a history to apologise for, and isn’t a good force in the world and the other side feels like America is a good force in the world, and has a history to learn from…That’s not a good scenario for an American get elected President. I think Joe Biden on the wrong side of that.
On the riots:
The slump starts every time somebody shares a video of somebody else. So when a Molotov cocktail is thrown in the street of a town, or people see messages about defunding police, or people talking about killing policemen and how that would be fair — that’s the stuff that also motivates people to vote against. And that stuff is not happening in a vacuum. It took Biden’s campaign a month to actually admit they were against the violence that they saw.
What politicians say means very little to me — what politicians do means everything. And the fact that Biden did a reverse course and sort of criticised the violence after months of embracing it, or suggesting it doesn’t exist, tells me he knows it’s real.
On Nate Silver:
This is the guy who said Hillary had a 70% chance of being President. I dismiss that offhand. I’m not interested.
Cahaly and Trafalgar were not perfect ” Minnesota was not close and Georgia was a battleground state ” but compared to, say, Quinnipiac and You Gov (where Freddie once worked), this man and his company were extraordinarily accurate. Again.
And Doug Rivers of You Gov, who in his interview with Freddie said that “Trump isn’t toast, but he’s browning quickly,” should take Tucker Carlson’s advice and look for a job hanging drywall or opening a restaurant.
He sure can’t do polling.
Although it now looks likely that Trump will lose, and Biden is likely to end up with around 300 EC votes so not an especially close result…?
Doug said on your videocast that Biden would get 340 EC votes, have an 8-9% edge in the popular vote, that Ohio and Iowa were both “toss-ups,” (Trump easily won both) and gave Biden the following battleground state victory margins:
PA – 6%!!
MI – 10%!!!!
WI ” 8%!!
What is the penalty for failure in the polling business ” an apology?
Didn’t Court Astrologers back under Suleiman the Magnificent get thrown off a tower or something if they got it too wrong ?
It is a very close result Freddie! Or will be if it goes that way, but hope still exists, a tiny bit, a whiff…But lets admit, the election was stolen by postal voting. “what we found is that social desirability ” as it was four years ago ” is between 3% and 5%.” This is the point of the secret ballot! The ballot at home can be completely skewed by this effect, let alone being bullied by another house member, of fear of it as they could watch, or demand to see, your marked ballot!
In the militant union days, (ha) the vote by raising hands was very effective at getting the correct outcome. Thuggish reps stood behind known union members who could vote incorrectly – and if they raised their hand at the wrong time a kidney punch landed on them with debilitating pain, a punch low and unseen by the other people voting! This is why voting booths with privacy are used! These postal votes leave every home voter in the position of having to vote correctly or get the punch from a fellow, you cannot know.
Secondly, Vote Harvesting. People who have no interest in the issue and would never get down to the poling place as they just do not care enough, can be collected up as signatures and ballots ordered by interested parties who do it all for them with their permission, including the correct vote answer. If you have so little understanding or interest then you are not casting your own vote, but turning your vote over to someone who does, your vote harvested by an interested party with your consent admittedly, but it is not really your vote, your vote was harvested.
The postal vote is not democracy, but something else, something very insidious. (excepting absentee and medical)
It is a very sad day for the free world when Biden Harris take charge. Dirty Deeds Done Dirt Cheap is how A/C D/C put it so well. Not democracy, but some other thing just happened.
The election was stolen by ballot harvesting and the dumping of fake ballots etc in Wisconsin, Michigan and Pennsylvania.
If it’s Biden at 300 in the EC, it would still be very close.
If that’s the outcome, it looks like a 100-200 thousand votes across Pennsylvania and Wisconsin would have changed the result by flipping 30 EC votes.
Arizona also looks to be very close. Some outlets have called it for Biden, but Trump’s down 70k with 475k votes still out. The recent batches of AZ ballots have been coming in for Trump by 18-20 points. That’s a notable contrast with the mail-in ballots Michigan and Pennsylvania, which have been heavily Biden. Arizona’s 11 EC votes for Trump could replace Wisconsin’s 10 in hypothetical (or actual) scenarios.
Five states could end up within a point – Arizona, Georgia, North Carolina, Pennsylvania, and Wisconsin – where various combinations of those states decide the EC majority. That’s a very close election to me.
Trafalgar are far from perfect.
They’re out on Georgia, probably Nevada, a long way on Michigan… etc.
They skewed one way which helped make it look like they were less wrong, but their science is no less refined than anyone else. He absolutely shouldn’t get to objectively claim any victory here. The big pollsters have got a big error on their hands. Just because Trafalgar gave more support to Trump doesn’t remotely make them more capable of predicting this race, or any future race. They’re like a stopped clock.
Given that in 2016 also Trafalgar predicted the correct outcomes in the battleground states ” it was unique in that regard ” what exactly is the difference, Alex, between “skewing Trump” and “being accurate”?
Perhaps instead of criticizing those who get things right you might want to examine the highly-touted polling outfits who told you over and over that they’d learned from their mistakes 4 years ago, but then made the very same mistakes again.
I believe in consequences, not stopped clocks.
As I mentioned, they didn’t get many states right when you actually look at the numbers, or polarity. They didn’t predict the outcome. I’m not sure why you think they are ‘accurate’.
They did a better job of showing Trump doing well, but other pollsters had lower error. And lower error is what matters here.
Their approach was a very simple ‘We think Trump will do better, so we’re going to weight him.” That isn’t science at all. That is just good guessing.
You won’t find Trafalgar being used all of a sudden by loads of multinationals, because they don’t have any science behind it.
“Their approach was a very simple ‘We think TRUMP will do better, so we’re going to weight him.” That isn’t science at all. That is just good guessing.”
Alex, you don’t have the facts on your side.
Here’s Tom Bevan, co-founder and publisher of Real Clear Politics, summing up polling performance after the 2018 midterms, when Trump was not running and which was characterized as a big Democratic wave. According to your THEORY, Trafalgar should have done very poorly.
“Trafalgar also correctly predicted Senate outcomes…making it the most accurate pollster of the cycle among those firms that polled multiple Senate and governor races.”
“Cahaly’s method once again proved solid. In one of the most polled races of the cycle, Trafalgar stood alone as the only polling firm to correctly show a Ron DeSantis gubernatorial victory in Florida ““ as well as Rick Scott winning the Senate race there.”
So, 2016, 2018 and 2020. Sounds like in fact the man knows what he’s doing.
But I repeat my question: Why cavil with the most accurate pollster of last 3 elections cycles instead of condemning the consistently ” and suspiciously ” misleading major pollsters who were so touted during the months preceding the election by the very media that openly wanted Trump to lose?
What threatens democracy more, Alex, Trafalgar’s record for accuracy, or the use of incompetent and partisan polling to help shape an unruly electorate?
Cahaly said “This is the guy who said Hillary had a 70% chance of being President. I dismiss that offhand.” Which means Trump had a 30% chance. I would not bet much against the odd of a two-coin toss being both heads (a 25% chance which is less than Silver’s prediction of Trump’s victory). I think Cahaly can be dismissed offhand.
“Like most people I have never even seen a pollster”. So says Thomas Sowell, at minute 3:15 of https://tinyurl.com/y2hvnfks – a quotation which has even more resonance today, given the latest egregious failure of pollsters.
Cahaly’s polls had significant errors similar to the errors of other polls. He’s no more accurate than any other pollster.
However, he gets press from media outlets that lean rightwards, which goes to show again – your media lean determines your coverage.
P.s. he was also very off in 2018, when the usual pollsters were a lot more accurate.
Join the discussion
To join the discussion in the comments, become a paid subscriber.
Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.Subscribe