So far there have been four or five job changes, depending on whether you count Shawn Camp (who got the first opportunity after Fujikawa hit the DL) or Bruce Rondon. The closers who have lost job shares to date are Marmol, Axford, Rondon, Boggs and Camp.Â No matter which way you count, the Closer Identifier Algorithm got them all right. Is this easy or hard to do?
There is no easy answer. Let’s do some quick math. We will make an estimate (as we did last year) on whether, subjectively, it was easy, moderate or hard to predict the outcome. We said that easy was when 75% of analysts would have gotten it right, moderate was 66% and hard was 50%. This is judgedÂ at the time they ascended to the role not after the result.Â I am more than happy to debate each player’s classification.
My subjective thought is that it would be as follows, but feel free to disagree and tell me why @pdicaprioFP911 or atÂ firstname.lastname@example.org:
Marmol 66% moderate
Axford 50% hard
Rondon 50% hard
Boggs 66% moderate
Camp 75% easy
The chances of getting them all correct is simply the product of these chances, or 0.66 * 0.5 * 0.5 * 0.66 * 0.75=0.08 or 8%. To put it another way, if we take 100Â fantasy baseball analysts and pitted them against CIA, 8 would equal its success rate.Â If you remove Camp, which is not legitimate in my view since he got an opportunity because of injury, the chances are only moved up to 11%.
If you remove Rondon, which is totally legitimate, then the chances are just over 16%, which is still very good for CIA, but there are probably others who would have done it. Officially, CIA results doÂ not count Bruce Rondon.
This is an informative exercise, and shows just how difficult it is to get all predictions of any sort correct even over a small number of predictions. It is also indicative of how and why the Philip Tetlock study that inspired CIA was so powerful and why algorithms can easily outperform “experts.”
It is also important to recognize that this is anÂ estimate. It may or may not have any concrete relation to reality. That is impossible to measure, since only one expert offered to face off against it (Mike Gianella) who advised that he got them correct. So, perhaps the numbers are too low.
If you flipped a coin (the basis for the Tetlock study) the chances of getting any one right is 50%, which is why the odds of getting one prediction correct will never be less than 50% if you buy into Tetlock’s thesis and this experiment’s purposes. But in reality it is probably 100% likely that a full analysis of any one closer above would result in a lower than 50% chance.
All told, the above results seem reasonable, if not as precise as one would wish. However if anyone disagrees, especially any “experts” out there let’s discuss it.
Also, it is worth noting that the in-season results areÂ biased in favor of CIA. Some of the predictions it will get wrong will not can can not be determined until the end of the season. So the record will generally look a lot better in season than is perhaps warranted.
|Jim Johnson||Hold||May have a claim to being the best closer; he is among the safest.|
|Andrew Bailey||Hold||Probably will keep the job when Hanrahan returns, but a lot can change.|
|Addison Reed||Hold||Is holding his own so far, and may be ready for a step forward.|
|Joaquin Benoit||Lose||The next week matters a lot for him.|
|Phil Coke||Lose||Not yet ready to declare that he lost his share, but close. One more week.|
|Jose Veras||Lose||Do you really care?|
|Greg Holland||Lose||Appears to have turned it around, but still on the fence with a better pitcher behind him.|
|Kelvin Herrera||Hold||He may end up being CIA’s first miss of the year soon, but not ready to declare it yet.|
|Glen Perkins||Hold||Has blossomed into a hidden gem, and looks rock solid.|
|Grant Balfour||Hold||Has rebuffed Ryan Cook with authority so far.|
|Tom Wilhelmsen||Hold||Right now he is one of the best closers in baseball.|
|Fernando Rodney||Lose||Still gets a “lose” status, but only needs an outing or two at most to change his status.|
|Joe Nathan||Hold||Only concern is age, at 38, but Ron Washington gets credit for astute handling.|
|Casey Janssen||Hold||Shoulder and Sergio Santos far back in rear-view mirror.|
|JJ Putz||Lose||He is vulnerable and not pitching well. Only 60% in SV/SVO and skills subpar.|
|Kyuji Fujikawa||Hold||Should be back by the end of the month.|
|Aroldis Chapman||Hold||Another year another 200+ BPV|
|Rafael Betancourt||Hold||Gets a “hold” based on SV/SVO but skills stink right now.|
|Brandon League||Hold||He is surprisingly mowing them down.|
|Steve Cishek||Lose||only 2 SVO and blew one. Probably safe.|
|Jim Henderson||Hold||His control is excellent so far so if he maintains his gains he will be safe.|
|Bobby Parnell||Hold||So far so good.|
|Jason Grilli||Hold||No worries so far.|
|Huston Street||Hold||He isn’t injured yet and his skills are terrible so far in a short sample. But 2/2 in SV/SVO|
|Edward Mujica||Hold||Until you hear otherwise assume he is the only closer.|
|Trevor Rosenthal||Hold||Reports of a 100 mph fastball, and his hit issue is a batted ball issue. He looks fine to us. Probably has no share of the job but will keep for another week.|
|Rafael Soriano||Hold||No issues yet.|
Latest posts by Patrick DiCaprio (Posts)
- Closer Identifier Algorithm Results Through August 26 - Aug 28, 2013
- Pitch or Ditch August 26 - Aug 26, 2013
- Closer Identifier Algorithm Results For August 19 - Aug 19, 2013