There’s been a little bit of attention given to a political science prof, Allan Lichtman, who’s correctly predicted each presidential election from 1984 through 2008. U.S. News called him the never-wrong pundit. So what’s his methodology?
Lichtman has 13 “keys,” and if 8 or more of those keys favor the candidate of the incumbent party, that party retains the presidency. If 6 or more favor the challenging party, it wins. It’s worked for 7 straight elections, so there seems to be a there there. And reportedly he predicted a ’92 Democratic victory even while G.H.W. Bush had record-high favorability ratings after the Gulf War.
I’m a little skeptical, though, for two reasons. But I should note that I haven’t read his book, the Keys to the White House, and in it he may have already have addressed some or all of my concerns First, the number of variables is greater than the number of cases to date. I’m no great statistician (hell, I’m not really even a poor statistician), but my understanding is that having more variables than cases undermines the reliability of the results, I think because there aren’t enough cases to really test each variable’s power.
Second, some of the keys are themselves a bit fuzzy. They are:
- Party mandate: Whether the incumbent president’s party hold more seats in the House than it did following the previous midterm election (e.g., during the prior presidential term). OK, this one is not too fuzzy–they either do or don’t (although I’d be curious to know if a difference of only one or two seats would matter).
- Contest: Whether there is a serious contest for the incumbent party nomination. Frequently this is quite clear, but how do we define a “serious” challenge?
- Incumbency: An automatic plus for the candidate of the sitting president’s party. That’s cut-and-dried, but I’ll admit that I’m surprised it’s a plus, given how often the presidency switches party. After all, we haven’t had more than 12 years of one party’s control of the presidency since FDR. In the 7 elections Lichtman has predicted correctly, the presidency has switched parties 3 times (although perhaps one could legitimately discount the 2000 election on this key). Of course this is only one key, so Lichtman isn’t saying this one by itself is inevitably predictive. But how often can a key be non-predictive and still be considered a relevant predictor? There’s also a question in my mind about how much this correlates with the “Contest” variable. Incumbency would seem to be strongest when there is no contest, so is there an issue of multicollinearity here?
- Short term economy: The economy is not in recession during the campaign. I’m somewhat dubious here, but only on the basis of a single data point. In ’92, the economy was not in recession between the party conventions and the election, but the public still believed it was. I wonder if the issue here should be recession during the late summer/early fall–say the third quarter of the year?
- Long-term economy: “Real per capita economic growth during the term equals or exceeds mean growth during the previous two terms.” I accept this one without any real trouble. It’s precise, measurable, and it fits with long-standing conviction among political scientists and campaign professionals that the economy matters.
- Policy change: The incumbent president having had a major policy success is a plus for that party’s candidate (whether that candidate is the incumbent president or not). This may be a bit too fuzzy, but then political scientists have long been distinguishing “major” legislation from “minor” legislation. It’s not my area, so I’m not sure how measurable that variable actually is. And I wonder if there’s an of whether that policy was popular or not? The variable treats popular policy changes the same as unpopular ones. That may be appropriate, as the public likes “effective” presidents, and passing major policy changes is an indicator of effectiveness. More specifically, the middle of the roaders seem to like effectiveness more than they care about the specifics of the change, and major policy change may boost incumbent party voters turnout. That’s all speculation on my part, though.
- Social unrest: A minus for the incumbent. I’ll buy this, with some reservations about how we measure social unrest.
- Scandal: A minus for the incumbent party’s candidate if there has been a major scandal in the current administration. I don’t know; scandal in the Reagan administration didn’t seem to hurt G.H.W. Bush’s re-election, nor did Whitewater hurt Clinton. It seems intuitively true, but I’d need more data points to have much certainty about it. And we have the measurement problem of distinguishing between major and minor scandal. I suppose in this case the best way to measure it would be a public opinion poll–if the public considers a scandal major, then it is major from an electoral perspective.
- Foreign/military failure: Whether the incumbent administration has suffered a major failure in foreign/military affairs. Again, the measurement issue. And impressionistically, yes the Iranian hostage crisis harmed Carter, but no the Beirut barracks bombing didn’t hurt Reagan. Intuitively it sounds good, though.
- Foreign/military success: The obverse of the prior key. I’ll buy it, with the general caveats about measurement.
- Incumbent charisma: A plus for the incumbent party’s candidate if he is charismatic or is a national hero. I have qualms about measuring charisma–it’s such a “we know it when we see it” quality that I wonder how well it can be pinned down.
- Challenger charisma: Same thing, but for the challenger. Same quibble.
So, what does this mean? Well, as with many fuzzy systems, it means it’s a lot easier to predict retroactively than proactively. I’m not sure how much of each of those Lichtman has done, but his book was only published in 2008 (and a quick search of JSTOR doesn’t reveal any earlier journal articles*), so superficially, at least, it seems he’s had to do make most of his predictions retroactively. I’m not sure how this squares with his prediction of a democratic victory in 1992, but perhaps he had this under wraps for years while he tested it. The difficulty for a researcher with a model like this is you have to wait 4 years for each new data point. I guess that means you’d have plenty of time to work on other projects concurrently, but it still would require a lot of patience, more than I have, to develop.
For the record, I’ve invited Dr. Lichtman to respond to this critique, particularly as I can’t pretend to claim any certainty about my critique (this area is certainly not my strength). I hope he favors us with further explanation and defense of his model. The American Political Science Association has run election prediction issues of its journal PS: Political Science and Politics, and it’s intriguing to see the various models put forth. The social sciences generally are not terribly impressive as predictive sciences, but presidential elections seem like the kind of thing that have a small enough set of relevant variables that reasonably accurate predictions ought, in theory, to be possible.
* Although a JSTOR search for keys to the white house does turn up an article on “The White House Stables and Garages” in The Records of the Columbia Historical Society (Washington, D.C.), which find seems to me delightfully charming.