Ordinary people are no good at forecasting, but superforecasters are – spare me this nonsense. Are they the equivalent of Ferguson, Whiity and Vallance? The question not being asked is what can we do to ensure peace throughout the world.
They aren’t quite equivalent, as least as far as I can tell, and I hope we hear more about these people from Tom in future.
Prof Ferguson and his ilk are different from superforecasters in a few ways.
One is that superforecasting is all about assigning actual probabilities to different scenarios. If you read epidemiological papers carefully you’ll find they often don’t do this: instead they present a range of scenarios without assigning probabilities to them. There may (or may not) be uncertainty intervals within a scenario, but usually not between them.
A related problem is modellers refusing to make falsifiable predictions at all (which means they aren’t scientific, hence your gut reaction). One example is how early on Ferguson predicted 20,000 deaths within some time span, and then in a media interview changed it post-hoc to be “could be 20,000 or much lower”.
Another way it’s different is that as Tom explains, superforecasters pay close attention to the base rates of events. It’s just putting numbers on the intuition that the best way to predict the future is look at the past. Epidemiologists don’t do this: their models are all “blank slate”, often created fresh for every single virus. They don’t seem to ever encode prior experience of epidemics in their calculations. Instead their models are an encoding of primary-school level germ theory combined with lots of curve fitting. The results never track reality but nobody in their world seems to care, and this approach is optimal for producing papers, so why try harder?
A final way it differs is that (I believe) superforecasters like to engage with prediction/betting markets and tend to surface in competitions of various kinds. So they’re much more results oriented and willing to rank themselves and each other, whereas in academia everyone is always excellent and in consensus with each other (supposedly).
That said, superforecasters aren’t doing anything fancy. In theory what they’re doing is a simple form of modelling as well – they “look good” compared to regular modelling only because modelling (of the sort we hear about) is so screwed up by the rock-bottom standards in academia. Superforecasting is a separate community where institutional affiliations aren’t necessary.
There is another difference. The forecasters here are sitting down and using (ok, hopefully using) personal expertise to make their predictions. But modellers like Ferguson use large, complicated computer programs. See here for a sample of potential problems with the code. https://github.com/mrc-ide/covid-sim/issues/165
A lot of this will not make sense unless you are a computer programmer/software engineer. I am (40 years experience, you will have to take my word for that), and the criticisms made in that link are entirely plausible.
There was pushback against these (and related) criticisms, but I find them entirely unconvincing.
The code might be useful as a research vehicle, but to influence government policies? I would class Ferguson (and co) no better than snake oil salesmen.
Galeti Tavas
2 years ago
If fairies could dance on the heads of pins, How many Chinese fairies, compared to American Fairies would the Superforcasters figure would dance on each? We all love a good game of ‘What If’.
Anyway, I would like MacArthur being quoted on the Korean War, 1950 – 1953 when the Chinese streamed across the Yalu river into Korea and MacArthur was in charge.
“(MacArthur Fired): “MacArthur differed with the Truman administration on three major points: 1. MacArthur wanted to use nuclear weapons, which the US possessed in far larger numbers than the Soviets. However, Truman ruled this out because the US possessed too few nuclear bombs, the mountainous terrain would make them largely ineffective, and world opinion would not support such a move. 2. MacArthur wanted to bomb Chinese bases and factories in Manchuria, and destroy the bridges crossing the Yalu River from China to North Korea. He believed this would allow him to destroy enemy troop concentrations, prohibit enemy armies from entering Korea, and destroy the enemy’s source of supplies. But Truman believed that the Chinese might bomb US airfields in Korea and aircraft carriers in Korean waters, and bombing China might increase Chinese hostilities and bring the Soviet Union into the war. 3. MacArthur wanted to allow Chiang Kai-shek’s forces to invade mainland China which he hoped would trigger a revolution against the Communist government, or at least draw Chinese troops out of Korea. Truman opposed this plan for fear it would involve the US in a land war in Asia — the ‘wrong war at the wrong time, the wrong place, and against the wrong enemy.’ ” (and so he fired him)
Remember those days? When serious talk was of Taiwan invading China? But it shows some resolve anyway.
Here is how I see it, being a student of War and history. China can take Taiwan as it has a huge navy and vast numbers of civilian shipping made suitable for the role. Taiwan could mobilize half a million soldiers, so China would need a couple million invading troops. Results would be both unknown, and terrible. China would not fail as they would just embargo Taiwan if bogged down and keep going and going, I imagine losses of 50% would be likely… ‘Pyrrhic victory’ is likely the outcome.
China could just embargo Taiwan and demand they yield. China would have its economy wrecked by the world who did not like it and not doing as much business. ‘Pyrrhic victory’ likely as they would win after a long time.
I just cannot see a victory where the losses = victory instead of Pyrrhic victory. But one day even a Pyrrhic victory may be good enough for China, who knows, they have different logic to us.
Sounds like a very sensible analysis to me – and fortunately the CCP appears to be relatively sane compared to, for example, Adolph Hitler as far as respecting the fulfillment of treaty obligations etc
The present state of the CCP under Xi Jinping increasingly looks as if it thinks pyrrhic victories are as good as any other kind.
Lesley van Reenen
2 years ago
It is a terrible scenario to consider, especially if the US enters the fray. But would they? If the latest US army recruitment video is anything to go by they should stay at home and play with their toys.
Don’t confuse these atrocious recruitment drives with those who have actually decided to serve. Much like British soldiers recoiled in disgust at our most recent ones, the vast majority of serving Americans will have no truck with that horsesh*t
The Serving soldiers aren’t the ones being targeted. In any case organisations can change pretty rapidly internally based on these kind of hiring practices.
Today’s Daily Mail says Japan will sign with Taiwan if Taiwan invaded – as Japan knows it (Okinawa) would be next. Synchronicity….
Francis MacGabhann
2 years ago
The further out this forecast goes, the less credible it becomes. What happens if the Chinese leadership changes? Does it become more or less likely to attack Taiwan? If the Taiwanese military is not CURRNETLY capable of repelling a Chinese assault, what happens if they get a new C in C who knows his job and makes the Island unattackable, on the Swiss model, say? The Democrats are attempting to destroy their own country NOW, but what happens if Americans finally wise up, have had enough and make a serious, deep-seated and long lasting commitment to strengthening the US, both morally and militarily? Suppose they sweep the Republicans — and I mean the YOUNG Republicans, the ones who are getting right up into the faces of the Democrats out on the stump, not the “controlled decline” bunch in Washington — into power? Too many imponderables here.
Sure it is. It’s imperial over reach. You have to be an empire to have the agreements with an ally on the other side of the world to begin with. The founding fathers wouldn’t approve. Pre WWII US opinion would he hostile as well.
China is a bridge too far for the US. A cold war against a China that was communist could be won but the US is the country with the debilitating ideology now. Equity – ie equality of outcome – for groups is as impossible as equality of outcome for individuals, which is communism.
That’s why they make it concrete by saying “within the next five years, in which >100 people die”. It scopes it and eliminates disputes that could arise in case of very minor skirmishes caused by e.g. a rogue naval patrol.
Andrew Wise
2 years ago
I’ve participated in super-forecasting as part of an IARPA Geopolitical Forecasting challenge & I’ve worked with covid modellers – and I would not trust any of them further than I could throw them … which ‘aint far.
To be fair the super-forecasting challenge was interesting & you are correct in some of your analysis of how they work. We achieved very successful results from statistical analysis of twitter feeds! (Wisdom of the crowd) But the covid modelling community has been consistently more inaccurate and unreliable than economic forecasters or weather reporters. All men with more spreadsheets than sense.
(And, yes I’m worried about Taiwan, just I would not ask a modeller for the answer)
It’s a bit unclear from your phrasing if you hold epidemiologists and super-forecasters in equal disdain, or if you’re saying super-forecasting works much better.
mike otter
2 years ago
So superforecasters use stats instead of gut feeling guesses? Has the US invaded directly or via proxies other sovereign states throughout its history. 100%. Does the US restrain its proxies or allies from occupying territory or, for that matter, from rape, robbery, murder and torture. 100% it does not. Is China an ally or proxy of the current Democrat executive and its puppet Biden? Yes in the sense that both are committed to state capiltalism and totalitarian control of language, belief and even thought. So the numbers look worse for Taiwan that at any time since 1948, and only SCOTUS annulling the 2020 election can make those numbers better.
Andy Hughes
2 years ago
I’d be interested to know if they took Russian action in Crimea and Ukraine into account, and Chinese action in Hong Kong – none of which has resulted in much of a push back from the West. Taiwan is a next level up – but, I’d argue, hardly unthinkable given the current trend. Also, as a rule of thumb, if an authoritarian leader says they want to do something – I’d always take them at their word.
Peter Mott
2 years ago
It is becoming less likely because (at last) states are waking up to the possibility that it might happen and are beginning to develop a deterrent stance. Indeed this little article by Tom Chivers is a part of that process!
William Tallon
2 years ago
”It’s tough to make predictions, especially about the future…’ Yogi Berra
Kevin Dee
2 years ago
“I don’t believe China will start a hopeless war, so the likelihood of winning is over 50% if they start it, says another. You lost me right there.
Brendan O'Leary
2 years ago
Taiwan, the self-governing island
sounds like we’ve already lost if we adopt that phraseology. What next? “South Sudan , the self-governing enclave” on through the Balkans, the Baltic states, the Caucasus – the south Pacific, Australia, NZ?
John Hicks
2 years ago
Super forecasters using “inside/outside views” sounds like a very subjective “wokey” technique to establish anything; particularly when modern statistical/probability analysis continues to apply Bayes’ Theorem that has been around for 300 years or so!
Antony Hirst
2 years ago
I doubt the US has the competence or capability to defend Taiwan. They certainly won’t have the will, as the social ramifications will be unmanageable.
USA could very easy defend Taiwan. The problem is where that stops once it begins.
Galeti Tavas
2 years ago
NEXT:
Five top Astrologers give their forecast on war with China wile the top UK darts champions use a map of the world dart board to decide on Taiwan’s chances in fight. Which is more correct? Superforcasters to work out which team more likely to be correct in the outcome. Imperial War College to study the results.
William MacDougall
2 years ago
When NATO attacked Serbia to separate Kosovo and Russia attacked Ukraine to take Crimea you could at least argue that majorities in the areas justified exceptions to the normal rule that borders should not be changed by force. There is no such exception for Taiwan, where an overwhelming majority is understandably opposed to rule by the CCP, whatever their views on reunion longer term. And where would China stop? Tibet, Taiwan, Senkaku islands, Korea, Japan, Australia? Appeasement will not work. Taiwan should be defended against forced merger.
Al M
2 years ago
If such a conflict comes to pass, might polite American society come to value the (overwhelmingly) young, working class males who will be called upon to risk or give their lives?
Conversely, will these young men even be willing to serve an administration that regards them with utter contempt?
Earl King
2 years ago
The best way to stop a war is to make sure the price paid is too high. I have no idea what the current US posture is regarding the protection of Taiwan. The initial strike would be very damaging to the population and the Taiwanese military. However, assuming Taiwan has done the minimum low flying hard to hit anti ship missiles. I suppose the Chinese could drop paratroopers….Not sure how many they would need. So do the Taiwanese have anti aircraft batteries that would survive a first strike. On the question of whether the US supports the lose of dozens of ships and hundreds of aircraft and sailors and airman who would die? IDK…But I ask the question of Lithuania. Is Article 5 valid for the Baltic States? I doubt Germany would be willing to sacrifice not sure about the others.
Ordinary people are no good at forecasting, but superforecasters are – spare me this nonsense. Are they the equivalent of Ferguson, Whiity and Vallance? The question not being asked is what can we do to ensure peace throughout the world.
Look at what they were forecasting in 2000 for now. Way off.
They aren’t quite equivalent, as least as far as I can tell, and I hope we hear more about these people from Tom in future.
Prof Ferguson and his ilk are different from superforecasters in a few ways.
One is that superforecasting is all about assigning actual probabilities to different scenarios. If you read epidemiological papers carefully you’ll find they often don’t do this: instead they present a range of scenarios without assigning probabilities to them. There may (or may not) be uncertainty intervals within a scenario, but usually not between them.
A related problem is modellers refusing to make falsifiable predictions at all (which means they aren’t scientific, hence your gut reaction). One example is how early on Ferguson predicted 20,000 deaths within some time span, and then in a media interview changed it post-hoc to be “could be 20,000 or much lower”.
Another way it’s different is that as Tom explains, superforecasters pay close attention to the base rates of events. It’s just putting numbers on the intuition that the best way to predict the future is look at the past. Epidemiologists don’t do this: their models are all “blank slate”, often created fresh for every single virus. They don’t seem to ever encode prior experience of epidemics in their calculations. Instead their models are an encoding of primary-school level germ theory combined with lots of curve fitting. The results never track reality but nobody in their world seems to care, and this approach is optimal for producing papers, so why try harder?
A final way it differs is that (I believe) superforecasters like to engage with prediction/betting markets and tend to surface in competitions of various kinds. So they’re much more results oriented and willing to rank themselves and each other, whereas in academia everyone is always excellent and in consensus with each other (supposedly).
That said, superforecasters aren’t doing anything fancy. In theory what they’re doing is a simple form of modelling as well – they “look good” compared to regular modelling only because modelling (of the sort we hear about) is so screwed up by the rock-bottom standards in academia. Superforecasting is a separate community where institutional affiliations aren’t necessary.
Brilliant!
There is another difference. The forecasters here are sitting down and using (ok, hopefully using) personal expertise to make their predictions. But modellers like Ferguson use large, complicated computer programs. See here for a sample of potential problems with the code.
https://github.com/mrc-ide/covid-sim/issues/165
A lot of this will not make sense unless you are a computer programmer/software engineer. I am (40 years experience, you will have to take my word for that), and the criticisms made in that link are entirely plausible.
There was pushback against these (and related) criticisms, but I find them entirely unconvincing.
The code might be useful as a research vehicle, but to influence government policies? I would class Ferguson (and co) no better than snake oil salesmen.
If fairies could dance on the heads of pins, How many Chinese fairies, compared to American Fairies would the Superforcasters figure would dance on each? We all love a good game of ‘What If’.
Anyway, I would like MacArthur being quoted on the Korean War, 1950 – 1953 when the Chinese streamed across the Yalu river into Korea and MacArthur was in charge.
“(MacArthur Fired): “MacArthur differed with the Truman administration on three major points: 1. MacArthur wanted to use nuclear weapons, which the US possessed in far larger numbers than the Soviets. However, Truman ruled this out because the US possessed too few nuclear bombs, the mountainous terrain would make them largely ineffective, and world opinion would not support such a move. 2. MacArthur wanted to bomb Chinese bases and factories in Manchuria, and destroy the bridges crossing the Yalu River from China to North Korea. He believed this would allow him to destroy enemy troop concentrations, prohibit enemy armies from entering Korea, and destroy the enemy’s source of supplies. But Truman believed that the Chinese might bomb US airfields in Korea and aircraft carriers in Korean waters, and bombing China might increase Chinese hostilities and bring the Soviet Union into the war. 3. MacArthur wanted to allow Chiang Kai-shek’s forces to invade mainland China which he hoped would trigger a revolution against the Communist government, or at least draw Chinese troops out of Korea. Truman opposed this plan for fear it would involve the US in a land war in Asia — the ‘wrong war at the wrong time, the wrong place, and against the wrong enemy.’ ” (and so he fired him)
Remember those days? When serious talk was of Taiwan invading China? But it shows some resolve anyway.
Here is how I see it, being a student of War and history. China can take Taiwan as it has a huge navy and vast numbers of civilian shipping made suitable for the role. Taiwan could mobilize half a million soldiers, so China would need a couple million invading troops. Results would be both unknown, and terrible. China would not fail as they would just embargo Taiwan if bogged down and keep going and going, I imagine losses of 50% would be likely… ‘Pyrrhic victory’ is likely the outcome.
China could just embargo Taiwan and demand they yield. China would have its economy wrecked by the world who did not like it and not doing as much business. ‘Pyrrhic victory’ likely as they would win after a long time.
I just cannot see a victory where the losses = victory instead of Pyrrhic victory. But one day even a Pyrrhic victory may be good enough for China, who knows, they have different logic to us.
Sounds like a very sensible analysis to me – and fortunately the CCP appears to be relatively sane compared to, for example, Adolph Hitler as far as respecting the fulfillment of treaty obligations etc
The present state of the CCP under Xi Jinping increasingly looks as if it thinks pyrrhic victories are as good as any other kind.
It is a terrible scenario to consider, especially if the US enters the fray. But would they? If the latest US army recruitment video is anything to go by they should stay at home and play with their toys.
Don’t confuse these atrocious recruitment drives with those who have actually decided to serve. Much like British soldiers recoiled in disgust at our most recent ones, the vast majority of serving Americans will have no truck with that horsesh*t
The Serving soldiers aren’t the ones being targeted. In any case organisations can change pretty rapidly internally based on these kind of hiring practices.
Is toys a euphemism?
Today’s Daily Mail says Japan will sign with Taiwan if Taiwan invaded – as Japan knows it (Okinawa) would be next. Synchronicity….
The further out this forecast goes, the less credible it becomes. What happens if the Chinese leadership changes? Does it become more or less likely to attack Taiwan? If the Taiwanese military is not CURRNETLY capable of repelling a Chinese assault, what happens if they get a new C in C who knows his job and makes the Island unattackable, on the Swiss model, say? The Democrats are attempting to destroy their own country NOW, but what happens if Americans finally wise up, have had enough and make a serious, deep-seated and long lasting commitment to strengthening the US, both morally and militarily? Suppose they sweep the Republicans — and I mean the YOUNG Republicans, the ones who are getting right up into the faces of the Democrats out on the stump, not the “controlled decline” bunch in Washington — into power? Too many imponderables here.
I think any potential right wing reaction in the US would be anti imperial.
Defending allies is not (necessarily) imperialist. Honouring treaties such as the US has with Taiwan is imperative IMO.
Sure it is. It’s imperial over reach. You have to be an empire to have the agreements with an ally on the other side of the world to begin with. The founding fathers wouldn’t approve. Pre WWII US opinion would he hostile as well.
China is a bridge too far for the US. A cold war against a China that was communist could be won but the US is the country with the debilitating ideology now. Equity – ie equality of outcome – for groups is as impossible as equality of outcome for individuals, which is communism.
That’s why they make it concrete by saying “within the next five years, in which >100 people die”. It scopes it and eliminates disputes that could arise in case of very minor skirmishes caused by e.g. a rogue naval patrol.
I’ve participated in super-forecasting as part of an IARPA Geopolitical Forecasting challenge & I’ve worked with covid modellers – and I would not trust any of them further than I could throw them … which ‘aint far.
To be fair the super-forecasting challenge was interesting & you are correct in some of your analysis of how they work. We achieved very successful results from statistical analysis of twitter feeds! (Wisdom of the crowd)
But the covid modelling community has been consistently more inaccurate and unreliable than economic forecasters or weather reporters.
All men with more spreadsheets than sense.
(And, yes I’m worried about Taiwan, just I would not ask a modeller for the answer)
It’s a bit unclear from your phrasing if you hold epidemiologists and super-forecasters in equal disdain, or if you’re saying super-forecasting works much better.
So superforecasters use stats instead of gut feeling guesses? Has the US invaded directly or via proxies other sovereign states throughout its history. 100%. Does the US restrain its proxies or allies from occupying territory or, for that matter, from rape, robbery, murder and torture. 100% it does not. Is China an ally or proxy of the current Democrat executive and its puppet Biden? Yes in the sense that both are committed to state capiltalism and totalitarian control of language, belief and even thought. So the numbers look worse for Taiwan that at any time since 1948, and only SCOTUS annulling the 2020 election can make those numbers better.
I’d be interested to know if they took Russian action in Crimea and Ukraine into account, and Chinese action in Hong Kong – none of which has resulted in much of a push back from the West. Taiwan is a next level up – but, I’d argue, hardly unthinkable given the current trend. Also, as a rule of thumb, if an authoritarian leader says they want to do something – I’d always take them at their word.
It is becoming less likely because (at last) states are waking up to the possibility that it might happen and are beginning to develop a deterrent stance. Indeed this little article by Tom Chivers is a part of that process!
”It’s tough to make predictions, especially about the future…’ Yogi Berra
“I don’t believe China will start a hopeless war, so the likelihood of winning is over 50% if they start it, says another.
You lost me right there.
sounds like we’ve already lost if we adopt that phraseology. What next? “South Sudan , the self-governing enclave” on through the Balkans, the Baltic states, the Caucasus – the south Pacific, Australia, NZ?
Super forecasters using “inside/outside views” sounds like a very subjective “wokey” technique to establish anything; particularly when modern statistical/probability analysis continues to apply Bayes’ Theorem that has been around for 300 years or so!
I doubt the US has the competence or capability to defend Taiwan. They certainly won’t have the will, as the social ramifications will be unmanageable.
USA could very easy defend Taiwan. The problem is where that stops once it begins.
NEXT:
Five top Astrologers give their forecast on war with China wile the top UK darts champions use a map of the world dart board to decide on Taiwan’s chances in fight. Which is more correct? Superforcasters to work out which team more likely to be correct in the outcome. Imperial War College to study the results.
When NATO attacked Serbia to separate Kosovo and Russia attacked Ukraine to take Crimea you could at least argue that majorities in the areas justified exceptions to the normal rule that borders should not be changed by force. There is no such exception for Taiwan, where an overwhelming majority is understandably opposed to rule by the CCP, whatever their views on reunion longer term. And where would China stop? Tibet, Taiwan, Senkaku islands, Korea, Japan, Australia? Appeasement will not work. Taiwan should be defended against forced merger.
If such a conflict comes to pass, might polite American society come to value the (overwhelmingly) young, working class males who will be called upon to risk or give their lives?
Conversely, will these young men even be willing to serve an administration that regards them with utter contempt?
The best way to stop a war is to make sure the price paid is too high. I have no idea what the current US posture is regarding the protection of Taiwan. The initial strike would be very damaging to the population and the Taiwanese military. However, assuming Taiwan has done the minimum low flying hard to hit anti ship missiles. I suppose the Chinese could drop paratroopers….Not sure how many they would need. So do the Taiwanese have anti aircraft batteries that would survive a first strike. On the question of whether the US supports the lose of dozens of ships and hundreds of aircraft and sailors and airman who would die? IDK…But I ask the question of Lithuania. Is Article 5 valid for the Baltic States? I doubt Germany would be willing to sacrifice not sure about the others.