Hello, I’m Jackson Martin, the oddsmaker for CNalysis. I’m currently a student at the University of Kansas studying Mechanical Engineering, and am the Data Analyst for KU Young Democrats. Psephology has been an interest of mine for some time, and I’ve combined it with my coding and statistics abilities I’ve learned in college thus far. I approached Charles one day asking if I could run some of his ratings for the Kansas state legislative chambers through a model I built. He was impressed by my ability to be able to show the odds of supermajorities, majorities and ties in the chambers, and wanted me to join him at CNalysis, calculating the odds of the supermajorities, majorities and ties in every state legislative chamber.
The model takes ratings for each individual district and uses the collective ratings in each chamber to estimate the chances for each different majority outcome. It is designed to reflect FiveThirtyEight’s 2016 and 2018 models. If FiveThirtyEight had a set of ratings for the US House of Representatives, for example, I’ve designed the model so that the same set of ratings would have similar outcomes to FiveThirtyEight’s. Not that the model is a copy of FiveThirtyEight’s: instead I’ve looked at it as a baseline, ensuring that my model’s outcomes made sense when compared to a reputable forecasting organization such as them. For example let’s look at their 2016 President, 2016 Senate, 2018 US House, and 2018 Senate ratings.
The ratings are fairly similar, but the ranges are also important. The median outcome can be the same, but a model that is too sure of itself will have a narrow range such as HuffPost’s 2016 model which gave Hillary Clinton a 98% chance of winning the presidency. FiveThirtyEight has the most complete percentile data for 2018 US House ratings, so we’ll use that for an example. All numbers here are in terms of how many seats Democrats win:
Overall the model is very close to both the median and range of what should be expected, with some deviation (which of course is expected). This might be because of a difference in the number of simulations (538 does around 20,000 while our model runs 100,000 elections).
Let’s again use the 2018 US House elections, this time visualized. The scaling is different between FiveThirtyEight’s histogram and our model, and it is also flipped by party. However, they both show a similar pattern of a steeper slope towards the lower end of expected Democratic seats and a lower slope towards the higher end.
The model does not apply to individual district ratings: those are determined by Charles, taking into account candidate quality, fundraising, electoral results, demographics for the result of the rating. Democrats’ estimated odds in each rating by Charles is as follows:
To show the accuracy of this model let’s look at some of Charles’ 2018 legislative ratings. We’ll be comparing the predicted number of seats for chambers that flipped. The error ends up being just 3%, which is pretty good. One thing which is an issue is there aren’t really any other groups that do state legislative ratings to compare to, which is why we at CNalysis are undertaking this effort.
When compared to FiveThirtyEight, the model shows very similar outcomes with the same ratings, which shows that the way the model calculates outcomes make sense. Additionally, the model doesn’t have too narrow a range of outcomes which would give it a false sense of confidence in the predicted outcomes.