(Relatively) Quick post on why Politico sucks: The perils of prediction
I'm procrastinating, and I stumbled upon a link to this. "The Worst Political Predictions of 2019." Oy. Look, folks, I've made some good predictions in my time, and some bad predictions, but this list is terrible, and demonstrates why those who rely on Politico are being failed by the foolishness of that wretched outlet.
What does it mean for a prediction to be "bad?" Is having been "wrong" enough? Here's the thing. Every well-constructed prediction is at least implicitly probabilistic, and better yet, explicitly probabilistic. Sometimes that's easy, as with political science general election forecasting models. We do those about six months out from a presidential election, and contrary to what you have heard, those models were right in 2016. Sometimes, it's harder. See my attempt to make some veepstakes predictions about 2020. (But hey! Remember when I called it in 2016, in April?) However, let's say I put a 60% chance on an event occurring, and it doesn't happen. Was I "wrong?" With a one-shot event, you can't know whether or not my assessment of that probability was wrong, can you? It's not actually testable, which is why social science is about testing patterns, not individual predictions. If I put a 99% chance on something happening, and it doesn't happen, that looks worse. You see where I'm going with this.
So, what if someone "predicts" an event that has, by some reasonable assessment, a 30% chance of happening. Well, what were the alternatives? If it's a binary choice, the alternative had a 70% chance, but if there were 20 alternatives, and the choice with the highest probability had a 30% chance, then that 30% was your best bet. Still a "good" prediction, even if it doesn't pan out. This is math, people. But even if it doesn't have the highest probability, what if someone predicts an event with a 20% chance of occurring, and it doesn't occur. How bad was that?
Is that anywhere near as bad as a prediction that had a functionally zero percent chance of occurring? Not if you have anything resembling a statistically functioning brain. In principle, a mathematical dunce could say that any "prediction" that doesn't happen is equally bad, but then a) there's no point in making a list, and b) you can't rank items in a list even if you construct the list. There is just Set A-- the predictions that did happen, and Set B-- the predictions that didn't, and no way to make any within-set distinctions.
So, consider that mind-bogglingly stupid list, as only an idiotic institution like Politico could construct. The idea of Kamala Harris winning the nomination, or Pete Buttigieg losing support... those belong on a list with Donald Trump resigning from office, and the GOP turning on him? Seriously?! These are comparably stupid predictions? Was Harris ever favored to win? Not in my opinion, but she was viable, as Politico even admits! And Buttigieg dropping would fit with historical patterns. There is plenty of historical precedent for that prediction. This kind of stuff belongs on a list that starts with... number 17... Nancy Pelosi winding up as President by the end of the year? Seriously?! The probability of #17 was always precisely, mathematically 0. There was never even the slightest chance of it happening, and anyone who believed it was going so far into crazyland that either they were on drugs when they said it, or needed pharmaceutical help. And higher on the list than that was Harris getting the nomination?!
This is why Politico sucks.
Time for your reminder. Read Philip Tetlock's Expert Political Judgment. It's not just about being "right" or "wrong." It's about the underlying reasoning, and Zack Stanton wins my 2019 first annual award for dumbassery in punditry.
Do you see what happens when newspapers disappear? This is what happens, Larry! I'm calm. Calmer'n you are.
What does it mean for a prediction to be "bad?" Is having been "wrong" enough? Here's the thing. Every well-constructed prediction is at least implicitly probabilistic, and better yet, explicitly probabilistic. Sometimes that's easy, as with political science general election forecasting models. We do those about six months out from a presidential election, and contrary to what you have heard, those models were right in 2016. Sometimes, it's harder. See my attempt to make some veepstakes predictions about 2020. (But hey! Remember when I called it in 2016, in April?) However, let's say I put a 60% chance on an event occurring, and it doesn't happen. Was I "wrong?" With a one-shot event, you can't know whether or not my assessment of that probability was wrong, can you? It's not actually testable, which is why social science is about testing patterns, not individual predictions. If I put a 99% chance on something happening, and it doesn't happen, that looks worse. You see where I'm going with this.
So, what if someone "predicts" an event that has, by some reasonable assessment, a 30% chance of happening. Well, what were the alternatives? If it's a binary choice, the alternative had a 70% chance, but if there were 20 alternatives, and the choice with the highest probability had a 30% chance, then that 30% was your best bet. Still a "good" prediction, even if it doesn't pan out. This is math, people. But even if it doesn't have the highest probability, what if someone predicts an event with a 20% chance of occurring, and it doesn't occur. How bad was that?
Is that anywhere near as bad as a prediction that had a functionally zero percent chance of occurring? Not if you have anything resembling a statistically functioning brain. In principle, a mathematical dunce could say that any "prediction" that doesn't happen is equally bad, but then a) there's no point in making a list, and b) you can't rank items in a list even if you construct the list. There is just Set A-- the predictions that did happen, and Set B-- the predictions that didn't, and no way to make any within-set distinctions.
So, consider that mind-bogglingly stupid list, as only an idiotic institution like Politico could construct. The idea of Kamala Harris winning the nomination, or Pete Buttigieg losing support... those belong on a list with Donald Trump resigning from office, and the GOP turning on him? Seriously?! These are comparably stupid predictions? Was Harris ever favored to win? Not in my opinion, but she was viable, as Politico even admits! And Buttigieg dropping would fit with historical patterns. There is plenty of historical precedent for that prediction. This kind of stuff belongs on a list that starts with... number 17... Nancy Pelosi winding up as President by the end of the year? Seriously?! The probability of #17 was always precisely, mathematically 0. There was never even the slightest chance of it happening, and anyone who believed it was going so far into crazyland that either they were on drugs when they said it, or needed pharmaceutical help. And higher on the list than that was Harris getting the nomination?!
This is why Politico sucks.
Time for your reminder. Read Philip Tetlock's Expert Political Judgment. It's not just about being "right" or "wrong." It's about the underlying reasoning, and Zack Stanton wins my 2019 first annual award for dumbassery in punditry.
Do you see what happens when newspapers disappear? This is what happens, Larry! I'm calm. Calmer'n you are.
Comments
Post a Comment