It has been 9 months since Part 2 of this series and over a year since Part 1. Part of this was procrastination but also since verification could mean results from simulation can’t beat Vegas lines there was little motivation to find out and get depressed. But the 2019 baseball season is upon us and this has to be done so let’s get to it. Note: Handicapping doesn’t begin here until May due to lack of enough current year data in April.
Verification isn’t complete but we have high level results showing error. This post will show that error among multiple systems; including the fivethirtyeight ELO system, this data model’s simulation, the flip of a coin model, and much more.
tl;dr Cut the the chase: According to this error calculation ELO surpasses Vegas lines, our simulation surpasses ELO. An adjustment had to be made from the simulation that produced probability results posted here last season and described in Parts 1 and 2.
Although Vegas starts handicapping on day 1 of an MLB season, this model does not have enough data to make proper evaluations until May (around 1/6 of a season). This means a betting season for each team is around 133 games give or take a few with about 150 days (5 months) of actual betting.
Each system generates a break even probability. As we saw in the last couple years of handicapping the beginning of each Cubs series, Vegas bet for and bet against add to more than 1. The extra is the house spread which guarantees the bookie always gets a cut as long as they take in equal bets from each side. Casinos in Vegas never gamble on anything.
In this error calculation there are two types of betting strategies; bet for and bet against a team every game. If generated probabilities are accurate, each strategy would break even at the end of a season. Predicted probabilities are too low if it leads to winning, too high if it leads to losing.
Clarification 3/30/2019: The above wording is kind of confusing. If you BETFOR all 30 mlb teams you will also BETAGAINST all 30 teams and mathematically that has to equal 0 except for Vegas where P(win) != 1 – P(lose). The error ranked in tables below are betting for and against one team over ~133 events. That error should be as close to 0 as possible.
This model’s handicapping season starts in May and runs around 133 games for each team. This is nowhere near what would be considered an infinite set of events so there will be some variation, even for coin flip method where we know the actual probability. The absolute value of error from all 30 teams gets added together and that gets divided by total bet to calculate percentage error.
Now let’s traverse the various systems examined, explain them, and look at their error for the 2018 season. There will be a lot of tables so skip to the end for a summary.
In the coin flip exercise we flip a coin 133 times, representing the number of events in a baseball season for this handicapping model, to put the following error percentages into perspective. Even though we know a coin flip is exactly 50% heads, 50% tails, after 133 flips it rarely comes close to to 50/50. After running 10000 simulated 133 event seasons the average error converged to 0.069 with a 0.010 standard deviation. So even if a system can produce exact probabilities for a game, it cannot exceed 0.069 over the long term.
Clarification 3/30/2019: Just noticed that 133 events flipping a coin will mean no coin flipping season will end up with a perfect 50/50 split. Changed that to 134 and the numbers didn’t change.
The 5050 system assigns each team 50% probability of winning. If you have no information about either team, with two outcomes the probability is a coin flip. Below are the top highest error rates for this system.
Baltimore had a WinPct of 0.290 for 2018 which is pretty horrible and puts it on the top of this and almost every list below. Betting against Baltimore would net you $41.4 on every $100 bet and almost the same lost betting for them. This is because the real probability for BAL winning each game between May and September was way below 50% on average. Boston was the opposite and won way more games than 50% so betting for them would net you around $30 for every $100 bet — assuming someone would take this kind of bet. Vegas surely won’t.
Home Field Advantage
The home field advantage exercise assigns a probability of 54% to home teams, 46% to away teams. This is a historical average. If you have no other information other than where they are playing then this may be a good probability to use. Let’s see.
The above spread looks very similar to the 5050 system above. The TOTAL table at the end will show they are virtually tied in error which could indicate that in general there is no such thing as home field advantage which is kind of counter intuitive. Still not sure how to interpret this but for now the numbers and code to make these list looks correct.
Again, the usual suspects are shown in the top ten.
Vegas is the gold standard. If your system can beat Vegas you can literally win money in the long term. Break even probabilities are derived from the last daily snapshot for each game between May and September. In Vegas BETFOR probability will be different than BETAGAINST probability. This study only cares about break even probability assignments so it doesn’t matter. If you bet the BETFOR probability an infinite number of times you will break even, unless there is error.
The following shows the top error betting strategies by team for Vegas lines.
The error shows a % +/- , + means you win money doing that, – means you lost that much using that strategy. The top ten error % in Vegas are much less than the above two betting models which means they most likely will lose money — which we’ll see how much in subsequent parts to this series.
Even Vegas bettors were wrong betting for Baltimore who lost what could be a record number if games. Betting against BOS and OAK were big losers last season too.
ELO surprised me in that it beat Vegas in this calculation. Don’t know how Nate Silver calculates this but it’s pretty good and we’ll see how much money it makes in Vegas.
The totals will come after the next system which tops them all.
The Simulation, the subject of this series, generated using data and theories from this data model beats ELO and Vegas. Note: The algorithm behind this simulation changed from last season. it still uses away l-s, away l-r, home l-s , home l-r tiering but there was something missing that needed to be added.
The above spread is much different than all of the above models.
The above table is the rundown of all the systems and their error using data from 2018. Although errors differ in 2016 and 2017, the ranking is not different. Absolute values of errors were added together and divided by number of events (games) to get the total error %.
The next part to this series will simulate betting seasons for 2016, 2017, and 2018 This data model has a 6 daily snapshot dataset of all Vegas lines from July 2014 until present and end of betting snapshot back to 2011. For now only the last three years will be examined here. Eventually all years will be compiled. Until then ….