by Jimmy Nicholls
Thursday, 16
December 2021
Analysis
16:00

A superforecaster’s guide to the North Shropshire by-election

The Tories have a much stronger chance than the bookies believe
by Jimmy Nicholls
Candidates left to right, Duncan Kerr (Green) Helen Morgan (Lib Dem) Ben Wood (Lab) and Neil Shastri-Hurst (Con)

The vacating of an MP’s seat mid-term is always cause for excitement among political observers, especially when Government is facing serious political pressure. So we shouldn’t be surprised that today’s North Shropshire by-election is, according to commentators, pundits and even Tory MPs, a ‘test’ of Boris Johnson’s leadership.

The truism is that by-elections do not reflect general election trends, so anything can happen. But by-elections are still bound by patterns that go beyond the day-to-day news cycle even though the tendency is for the noise of the latter to drown out the signal of the former.

Taking a superforecaster’s approach to this by-election, we can assess just how likely it is that the Tories will hold onto their seat. By looking at the percentage of by-elections that led to a seat change between parties since 1945, we can establish a base rate to give us a probability of what will happen in North Shropshire.

Contrary to the claim made by Liberal Democrat leader Ed Davey that winning the constituency is a “coin toss” (bookies agree), the Liberal Democrats’ chance of winning the seat is actually much lower. According to my estimate, based on tallying up by-elections over the last 70 years, the Lib Dems’ chances stand closer to around 20%. This is to say that it’s possible, but it’s far from the likeliest outcome.

Coverage of by-elections tends not to focus on such facts. Before the Batley and Spen election, senior Labour figures put the party’s chances of holding the seat at 5%, even though Labour had held onto the seat for over 20 years. Betting markets meanwhile put Labour’s chance of winning at around 20%. When Labour won, the New Statesman‘s follow-up analysis argued that Labour’s subsequent victory was “against the odds” while the likes of the Mirror, the i and the Daily Record presented the result as a surprise, rather than the most common outcome in such by-elections.

Bookies similarly put the Brexit Party’s Mike Greene as the favourite to take Peterborough in 2019, a highly marginal seat that became available after a recall petition succeeded against Fiona Onasanya. While the race was tight, punters should have been more mindful of incumbents’ statistical advantage during by-elections. In the end, Labour won by nearly 600 votes.

Of course, parties have an incentive to downplay their odds of winning, encouraging their voters to go to the polls. And betting markets reflect what bets people are making rather than genuine odds. One could also argue that the Tories’ recent polling, the Christmas party scandals and disquiet about pandemic rules has given the Lib Dems a real chance. But this analysis is ultimately misguided because the long-term trends are still very much in the Tories’ favour. North Shropshire has, for decades, been a comfortably Tory seat and it will be incredibly difficult for the Lib Dems to overturn such a big deficit in votes. So my prediction is that the Tories are the favourites, with about an 80% chance of winning.

Join the discussion


To join the discussion, get the free daily email and read more articles like this, sign up.

It's simple, quick and free.

Sign me up
Subscribe
Notify of
guest
23 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
James Rix
James Rix
11 months ago

This hasn’t aged well.

Drahcir Nevarc
Drahcir Nevarc
11 months ago
Reply to  James Rix

Beat me to it.

jdenicholls
jdenicholls
11 months ago
Reply to  James Rix

Oh I dunno, I’m still happy that I said there was a one in five chance of this happening.

Andrew D
Andrew D
11 months ago
Reply to  jdenicholls

What would the result have to have been for you to say ‘ok, I was wrong’?

jdenicholls
jdenicholls
11 months ago
Reply to  Andrew D

I’d put it this way. If you were rolling a dice and asked me to predict what the odds were of it not landing on 1, I’d say 83%. If it then lands on 1 my prediction was still correct: those are the odds, irrespective of the result.
Put another way: if the Tories had retained their 23,000 majority, that doesn’t mean my 80% prediction of them holding the seat was right. I’m giving a probability for an event happening, but whether it does happen doesn’t tell you a lot on its own. Unlikely events happen all the time over the sum of human experience, just as likely events frequently fail to happen.
You need to assess a predictive model over time, of which the one I described here is very crude. If you use it again over many by-elections, the test would be whether it holds true (ie, by-elections give hold results about 80% of the time).
If you wanted to tell whether I was generally good or bad at forecasts you’d have to assess my predictions over time. If things I say have a 20% chance of happening tend to happen about 20% of the time, I’m doing a good job. (See section 4 here on Brier scores for more info.)

Billy Bob
Billy Bob
11 months ago
Reply to  jdenicholls

So essentially your predictions can never be wrong? As long as you don’t say there’s a 100% chance of something happening then every prediction is correct?

Billy Bob
Billy Bob
11 months ago

Wasn’t that super was it

David B
David B
11 months ago

Superforecasting sounds dead easy. You just have a look over the last [large number here] of events and say, well them’s the odds. Bookies, pundits and academics are simply wasting their time weighing up the convictions of cash in the game punters, utilising their experience and expertise, or conducting surveys and interviews with actual people (a la Tanya Gold of this parish).

That said, I’m always surprised when the Lib Dems win anything more than a scratch card these days.

Philip L
Philip L
11 months ago

Now the hilarious truth has arrived, this article reads like Professor Ferguson’s take.
Same way that fact checkers have become synonymous with “narrative”, forecasters have become synonymous with “wrong”.
All of them wrong, all the sodding time. You can see why people are so bored of it all.

jdenicholls
jdenicholls
11 months ago
Reply to  Philip L

As a genuine question, what do you think I got wrong? Do you think a Lib Dem win was likelier than I said? Less likely? What about the Tories’ prospects? Bear in mind the message above is broadly that a Lib Dem win was possible, but not the likeliest outcome.

Prashant Kotak
Prashant Kotak
11 months ago
Reply to  jdenicholls

If the LibDems had a 1 in 5 chance of winning, wouldn’t the margin of victory have been much much smaller by rights? The size of the victor’s majority implies a 1 in 250 type event does it not. Either that or your model is inaccurate.

jdenicholls
jdenicholls
11 months ago
Reply to  Prashant Kotak

I don’t think that follows, as saying so and so have this chance of winning doesn’t imply anything about margins of victory. That chance captures everything from a narrow victory to a big one.

Prashant Kotak
Prashant Kotak
11 months ago
Reply to  jdenicholls

So the swing was 34%. If the swing size is irrelevant then would you have forecast at 1 in 5 chance even if the by-election was in a constituency with a wafer thin majority?
That makes almost no sense to me statistically.

jdenicholls
jdenicholls
11 months ago
Reply to  Prashant Kotak

One of the limitations of this article is I didn’t do any adjustment for local conditions. If you wanted to build a more accurate model for predicting by-elections you’d start with a base rate (not necessarily my one) and then start to consider how other factors affect probability. For example: size of majority at last election, time since last general election, approval rating of sitting government.
To assess properly you’d need to look back on previous by-elections, but intuitively I’d expect that seats are more likely to be held if the last majority was large. Perhaps that is untrue though, in which case it would have been sensible to revise that 80% figure down. But from what I’ve read most of these forecasts don’t tend to move much away from the base rate, even after adjusting for specifics.

Prashant Kotak
Prashant Kotak
11 months ago
Reply to  jdenicholls

Thanks for the explanation.

Trevor Law
Trevor Law
11 months ago
Reply to  jdenicholls

There were a whole bunch of factors at play in the result; some specific to this time and others specific to this type of constituency. Extrapolating from historical by-election results was never likely to work.

Philip L
Philip L
11 months ago
Reply to  jdenicholls

Prashant summed this up in another comment, but your forecast that the Lib Dems’ chance of victory stood “at around 20%” is at odds with the magnificence of their win. If you’d said their chances were 40%, or even 60%, that would still have looked off target.
I say that because your 1 in 5 chance was presumably them squeaking in, rather than romping straight past the finish line, collecting the bunting on the way through and then heading for the pub.
Hence the pithy remark, which I now regret knowing that you’ve been polite enough to engage BTL.

jdenicholls
jdenicholls
11 months ago
Reply to  Philip L

No worries, I can certainly see why people are critical. I don’t think the base rate says much about the size of a win. The figure that incumbents win 80% of the time captures everything from huge majorities to narrow victories. As a prediction starting point it’s not supposed to imply anything about the size of a potential win or loss.

Andrea X
Andrea X
11 months ago

Ok, lib-dem it is, then.

Dan G
Dan G
11 months ago

ouch.

Andrew McDonald
Andrew McDonald
11 months ago

Oops. But bold to print this yesterday.

georgeowers
georgeowers
11 months ago

This aged well.

jim peden
jim peden
11 months ago

Well, full marks for being brave enough to take a punt. How wise we all are after the event. You wrote “One could also argue that … has given the Lib Dems a real chance.” If only you’d stopped there.
We can think of this particular piece as being for light relief only – like a scratch card. However, we are plagued nowadays with so-called experts who tell us what is to be and how to behave – and who turn out to be just as ill-informed, biased and wrong as the rest of us. And yet on they go, and on and on.
If we can’t reliably predict the weather, the physics of which is well understood, how on earth can we think we know enough to predict the evolution of hideously complex and ill-understood systems such as we find in politics or virology? Hubris is what the ancient Greeks feared most of all – we would do well to fear it too.