With Mariano Rivera still tracking at 100 percent in the Baseball Hall of Fame balloting, it's clear that the Cooperstown electorate has settled on a couple of things. One is, obviously, that Rivera belongs in the Hall of Fame. But the possible unanimity in his selection also tells us that the once-hot debate about whether relievers belong in the Hall of Fame at all is now settled. The dissenters have all but disappeared.
There is still at least one out there, Boston-area sportswriter Bill Ballou, who wrote about why he couldn't bring himself to list Rivera on his ballot. As a result, rather than ruining Rivera's chances of becoming the first unanimous selection, Ballou decided not to submit a ballot this year. Now, I don't agree with Ballou, but he did have defensible reasons for his position: The save is a terrible statistic, and closers aren't used frequently enough to fully test their mettle. He likened closers to place-kickers in the NFL, like 23-year veteran Adam Vinatieri.
The save statistic is indeed terribly flawed, but it's also unique in one respect. For years and years, its existence has had a profound impact on actual on-field strategy. Pitchers like Rivera weren't used like traditional relief aces, who were likely to enter a tie game, or even down a run or two, to wriggle a club out of a jam. Those relievers threw more innings, too, while Rivera, Trevor Hoffman, Dennis Eckersley and so many others were reserved for rarefied usage patterns defined by the parameters of the save rule -- a statistic cobbled together by the great Chicago sportswriter Jerome Holtzman. In his book "Smart Baseball," my ESPN colleague Keith Law has a chapter on this titled "Holtzman's Folly: How the Save Rule Has Ruined Baseball."
The thing is, these days we have enough tools to work with that we can identify the best relief pitchers of both the present and the past. Most of these measurements are much more telling than saves. By the way, I am intentionally trying to avoid the "closer" label here because these definitions are forever shifting. The National League reliever of the year last season was Milwaukee's Josh Hader, who finished tied for 20th in the NL with 12 saves. I don't recall there being any sort of uproar about this. Hader was very clearly the best reliever in the league, and there was also little doubt that even though he didn't close many games relative to other premier short relievers, he was providing massive value to the playoff-bound Brewers.
The most common metrics tossed around when it comes to Hall of Fame debates are WAR and JAWS, the latter the brainchild of Jay Jaffe, a writer for FanGraphs.com, an ESPN contributor and author of the excellent "The Cooperstown Casebook." JAWS is built upon WAR methodology in an effort to balance career production with peak value.
With or without JAWS, one thing that everyone can agree on is that Rivera is the best closer of all time, and most would argue that he's the best reliever, period. So if you can construct an argument that he doesn't belong in Cooperstown, you are constructing an argument that no pure reliever ought to be there. One thing you might point out is that Rivera's 56.2 career WAR ranks 227th all time, right between Orel Hershiser and Robin Ventura. His JAWS score (42.5) is the best of the pure relievers. It ranks behind Eckersley, who accrued a good chunk of WAR as a starting pitcher. But among starting pitchers, Rivera's JAWS score would be tied with David Wells' for 124th in history.
Jaffe lists relievers separately, so you're comparing them only to one another. That's clearly the way to go, but it doesn't address the larger question of if relievers should be in the Hall at all. The fact that the bottom-line value of Rivera -- the consensus best at his position -- is roughly comparable to very good, but definitely not great, starters is at the heart of the argument that relievers aren't Hall-worthy in the first place. However, most analysts seem to have come around to where I'm at: For relievers, it's not about the quantity of their innings, and it's not really about saves, either, and it's not about WAR. It's about the impact they have on winning, and WAR captures only a sliver of the picture for that position.
This is where making the comparison between relievers and football kickers falls apart. While NFL teams recognize the value of a good kicker, no team builds around that position. Only three kickers and one punter have ever been drafted in the first round, none since 2000. Only two pure kickers and one punter are part of that sport's Hall of Fame.
Relief aces are much more highly valued. Baseball has evolved to the point where championship teams can be and have been built on the back of a strong bullpen. One reliever can't carry a team, but he can have championship-level impact. Remember how Hader's presence hovered over the National League Championship Series last October? Has a football team ever built a game plan or set its lineup around the presence of Vinatieri, who is one of the all-time greats at his position?
"The Book" brought us the advent of win probability metrics (WPA), which has really helped clarify some of this muddle. In a nutshell, that metric looks at the result of every play in a baseball game and determines the change in the probability of the outcome for each team based on the play's result. Let's say you advance a runner to second on a ground ball that is the first out in the bottom of the fifth inning with your team trailing 3-2. According to the change in win probability, you've just decreased your team's chances to win by 2.8 percent. The hitter takes the debit on that, while the pitcher gets the credit. Add up all of these results over the course of a season -- and a career -- and you have a mostly contextualized measure of a player's impact on winning.
Because relievers are used in high-impact situations, they get a disproportionate amount of credit and blame through their WPA results, which helps put them on equal footing with other positions. WAR and other factors are still important parts of the picture and have to be considered. But isolating the effects of probability and leverage helps give us a much more complete idea of what great relievers are worth, and a much more accurate one than if we focus on saves. According to TheBaseballGauge.com, Rivera ranks 18th all time in win probability added, postseason results included, and that's among all positions. Other great relievers rank in the top 200: Hoffman (94th), Rich Gossage (119th), Jonathan Papelbon (141st), Eckersley (155th), Joe Nathan (157th) and Billy Wagner (166th).
Looking at WPA for all pitchers -- starters or relievers -- tells us that Rivera ranks third, behind Roger Clemens and Lefty Grove. Hoffman is 11th, Gossage 15th, Papelbon 18th, Eckersley 22nd, Nathan 23rd and Wagner 25th. That's seven of the top 25 pitchers in history. If you're not recognizing that position group in the Hall, you're not recognizing what has become an essential and valued component of baseball.
With the election of Rivera and the veterans' committee pick of Lee Smith, there will be eight Hall of Fame relief pitchers. Well, it's nine if you count Satchel Paige, who was mostly a reliever during his time in the big leagues. But Satch was so much more than that, and those seasons aren't why he not only has a plaque in Cooperstown, but a nifty statue in the little park next to the Hall itself. Anyway, there will be eight Hall relievers after Tuesday's announcement of this year's balloting results. Three of the eight -- Hoffman, Rivera and Smith -- will have been selected in the past two years.
Eight is a small number. According to a quick search at Baseball-Reference.com, I came up with 333 relievers who have lasted at least the 10 years of service you need in order to qualify for Hall eligibility. Eight of 333 is just 2.4 percent, a low number. The question is: What is the right number?
I think I can answer that somewhat objectively. With eight relievers in or headed to Cooperstown, we're beginning to be able to define what a Hall of Fame reliever should be, in a similar way that we do -- imperfectly -- with the other position groups. And believe it or not, I think I can paint this portrait without leaning on saves at all.
First, let's look at the eight relievers who are, or are about to be, immortalized:
RELIEVERS IN THE HALL
1. Hoyt Wilhelm (inducted in 1985 by the BBWAA in his eighth year of eligibility)
2. Rollie Fingers (1992, BBWAA, second year)
3. Dennis Eckersley (2004, BBWAA, first year)
4. Bruce Sutter (2006, BBWAA, 13th year)
5. Rich Gossage (2008, BBWAA, ninth year)
6. Trevor Hoffman (2018, BBWAA, third year)
7. Lee Smith (2019, via committee)
8. Mariano Rivera (2019, BBWAA, first year)
Now, let's consider the list by using the career saves leaderboard:
HALL RELIEVERS, ALL-TIME SAVES
1. Mariano Rivera, 652
2. Trevor Hoffman, 601
3. Lee Smith, 478
7. Dennis Eckersley, 390
13. Rollie Fingers, 341
25. Rich Gossage, 310
28. Bruce Sutter, 300
41. Hoyt Wilhelm, 228
Zero in on the gap between Eckersley and Fingers, who comprise the middle of the list. Who are the five pitchers between them on the saves list? That would be Nathan, Papelbon, Jeff Reardon, Troy Percival and Randy Myers. Those relievers don't just meet the minimum threshold for a Hall level of career saves, but are right in the middle of the range. So should they all get in?
The next metric we'll look at is WAR, the Baseball-Reference.com flavor.
HALL RELIEVERS, CAREER WAR
1. Dennis Eckersley, 62.4
2. Mariano Rivera, 56.2
3. Hoyt Wilhelm, 47.1
4. Rich Gossage, 41.4
8. Lee Smith, 29.0
10. Trevor Hoffman, 28.0
16. Rollie Fingers, 25.7
20. Bruce Sutter, 24.2
That's a better list than saves, bumping Wilhelm up to third. The knuckleballing Wilhelm did accrue some value in his 52 career starts, but not nearly as much as Eckersley did in his starting days. The middle of this list -- those missing between Gossage and Smith -- is composed of these relievers: Tom Gordon, Firpo Marberry and John Hiller. That's three different relievers than those on the saves list. So should these guys be in instead? Should ALL of them be in?
Here's JAWS:
HALL RELIEVERS, JAWS LEADERBOARD
1. Dennis Eckersley, 50.2
2. Mariano Rivera, 42.5
3. Hoyt Wilhelm, 36.9
4. Rich Gossage, 36.7
12. Lee Smith, 24.9
13. Bruce Sutter, 24.3
16. Trevor Hoffman, 23.7
20. Rollie Fingers, 22.3
This is very close to the WAR list. All eight Hall relievers fall in the top 20. The top five are ordered exactly the same, with a similar gap between 4 and 5. Sutter gets a slight bump because of his peak performance, dropping Hoffman and Fingers down a notch. This time, there are seven relievers missing from the middle: Gordon, Marberry, Hiller, Ellis Kinder, Turk Farrell, Lindy McDaniel and Stu Miller. Should all of THESE guys go in?
We're going to do three more categories before I start to tie all of this together. I wanted to do something related to awards recognition, a tacit acknowledgment of how a player was perceived during his time. To do this, I created a metric that counted All-Star Game selections, Reliever of the Year awards (given out since 1976, except for a one-year gap in 2013) and vote shares in MVP and Cy Young balloting as tallied by Baseball-Reference.com. I gave extra weight for MVP consideration (whether it was deserved or not) and Cy Young support, tallied it all together and divided it by seasons played to create what I'll call All-Star Percentage. You can have more than 100 percent and, indeed, some relievers have done that.
HALL RELIEVERS, ALL-STAR PERCENTAGE
3. Rollie Fingers, 103%
4. Bruce Sutter, 102%
5. Mariano Rivera, 89%
7. Dennis Eckersley, 66%
8. Lee Smith, 62%
13. Trevor Hoffman, 51%
17. Rich Gossage, 49%
24. Hoyt Wilhelm, 38%
35. Satchel Paige, 33%
I included Paige on this list to make a side point. As short and tragically delayed as his big league career was, his time in the majors was not without accomplishment -- he played in a couple of All-Star Games and finished 17th in the 1952 American League MVP balloting. The midpoint among this list of relievers -- Eckersley and Smith -- bumps up against each other, so instead I'll list the four relievers between Smith and Hoffman: Dellin Betances, Aroldis Chapman, Kenley Jansen and Francisco Rodriguez -- all active pitchers. (K-Rod didn't appear in the majors last season but reportedly is going to try again in 2019.) It should be noted that as the career of a pitcher lingers, and his performance goes unrecognized, his All-Star Percentage will fall.
You might also notice that the top two pitchers on the All-Star Percentage list are missing. It's actually a tie for first. I will list those hurlers here, and we'll return to them shortly:
ALL-STAR PERCENTAGE, NON-HALL RELIEVERS
1. (tie) Dan Quisenberry, 123%; Craig Kimbrel, 123%
Stylistically, you could hardly find two more divergent relievers than Quiz and Kimbrel. Moving on, our next leaderboard is the aforementioned win probability added, using the version at TheBaseballGauge.com:
HALL RELIEVERS, CAREER WIN PROBABILITY ADDED
1. Mariano Rivera, 66.2
2. Trevor Hoffman, 37.1
3. Rich Gossage, 33.2
5. Dennis Eckersley, 29.1
11. Hoyt Wilhelm, 23.5
22. Lee Smith, 19.2
28. Bruce Sutter, 18.3
29. Rollie Fingers, 17.6
This, even more than saves, establishes why Rivera is the unquestioned top reliever of all time. The pitchers between Eckersley and Wilhelm are Nathan, Wagner, Kimbrel, Jansen and K-Rod. Again, postseason results are included in these figures.
Finally, let's look at this through another Baseball Gauge metric that I love: championship probability added. It works similar to win probability, except instead of looking at how a result changes a team's chance of winning a game, it estimates its impact on winning the World Series. This gives a whole lot of extra weight to players who impact close pennant races and, especially, postseason games. We'd never judge a player's career solely by this metric, but it helps to quantify something that is usually referred to anecdotally. I like it so much that I incorporated it into my Awards Index system. Still, I'm not going to list the missing middle on this one because I don't think you'd want to build a Hall case based on this metric alone.
HALL RELIEVERS, CHAMPIONSHIP PROBABILITY ADDED
1. Mariano Rivera, 2.13
2. Rollie Fingers, 1.27
8. Hoyt Wilhelm, 0.51
9. Rich Gossage, 0.5
22. Bruce Sutter, 0.39
38. Dennis Eckersley, 0.25
116. Lee Smith, 0.07
135. Satchel Paige, 0.05
387. Trevor Hoffman, -0.05
Oops ... I again included Paige, who helped the Indians win their most recent World Series title as a "rookie" in 1948. This metric helps to bolster what many feel is a borderline case for Fingers, who was baseball's all-time best postseason reliever until Rivera came along and lapped him about five times. Rivera, by the way, ranks fourth all time in championship probability added. Not just among relievers, or pitchers. Among everybody. (Fingers is 19th.) When you isolate the metric to postseason performance, Rivera has a decided edge over everyone, ever, though it should be acknowledged that he had an important advantage in opportunity by playing for a perennial playoff team during the wild-card era.
There were several other categories I looked at, though I'm not going to list the leaderboards here. They were: career length, ERA+, dominance (measured as strikeout percentage minus walk percentage), Black Ink, Hall of Fame standards and Hall of Fame monitor. The latter three of those measurements are Bill James innovations you see plastered all over the player pages at Baseball-Reference.com.
As we've seen, there is no consensus among these various measures about what a Hall of Fame reliever looks like, or which ones should be there. That's certainly true if you're looking only at saves data. We need to consider a range of metrics. To do that, I decided to follow the methodology I established last season with the Awards Index. Using all of the categories I've mentioned above, I calculated a standard score for each player in each metric. This tells us how much better, or worse, the player was in each metric as compared to the other pitchers in the sample.
The sample was made up of the 416 best relievers of all time in JAWS. Why 416? Well, the Baseball-Reference.com JAWS leaderboard lists 500 relievers. Many of them accrued too much value as starters for me to consider them as Hall candidates based on relief pitching, so I established a couple of arbitrary cutoffs and deleted 84 pitchers from the list. Examples of those who were cut included Bobby Shantz, who relieved in more than twice as many games as he started over his career. But he won 24 games and the AL MVP award in 1952 -- as a starter. Shantz has 32.1 career WAR, but 9.0 of that total was produced during his MVP campaign. Others who were cut include Kerry Wood and Greg Swindell.
I didn't give one category more weight than the others, I simply averaged them. The end result is a number that tells us how much better, or worse, each reliever was within this peer group of the top relievers of all time. We'll call this the Reliever Hall Index (RHI). Here's how our Hall of Famers stack up:
HALL RELIEVERS, RELIEVER HALL INDEX
1. Mariano Rivera, 4.24
2. Dennis Eckersley, 3.48
3. Hoyt Wilhelm, 2.73
4. Rich Gossage, 2.68
6. Rollie Fingers, 2.51
7. Trevor Hoffman, 2.34
9. Bruce Sutter, 2.12
10. Lee Smith, 2.11
Now we're talking! By considering this suite of measures, we've arrived at a list that makes the Hall selectors look awfully smart. All eight Hall relievers finished in the top 10, with Rivera setting the upper edge and Smith the bottom. For the math-inclined, the number is actually a count of standard deviations -- Rivera is four standard deviations better than his peer group. Spoiler alert: That's freaking amazing.
What does this list look like if we pull saves from the formula? This is key, because as relief pitcher roles evolve, and pitchers like Hader are used in ways very different from the way Tony LaRussa used Eckersley or Joe Torre used Rivera, we want to distance ourselves from that all-too-imperfect statistic. Here we go:
HALL RELIEVERS, SAVE-FREE RELIEVER HALL INDEX
1. Mariano Rivera, 4.13
2. Dennis Eckersley, 3.52
3. Hoyt Wilhelm, 2.85
4. Rich Gossage, 2.73
6. Rollie Fingers, 2.51
7. Bruce Sutter, 2.12
8. Trevor Hoffman, 2.11
11. Lee Smith, 1.96
In pulling saves from the formula, we've essentially lost nothing. Smith drops down a spot, Sutter and Hoffman are flip-flopped, but it's mostly the same list. And again, the list makes the Hall selectors look very smart. (I'm not one yet, so don't take this as self-congratulating.)
Real quick, we're going to talk about representation. In other words, how many relievers should go into the Hall of Fame? In general, and I'm not tied to any precise cutoff point when it comes to picking Hall of Famers, I like to consider what the level of quality is for a championship team. I used standard scores for that, too, which allows me to then make a comparison to the player scores.
I estimate that the typical title-winning team during the divisional-play era (since 1969) has been about 1.3 standard deviations better (as measured in run differential) than all teams during the era. But of course, the best team doesn't win the World Series every year, especially since the playoff format expanded. So we also have to consider that the annual league leader in run differential has been about 1.9 standard deviations better than all teams.
This range is where Hall consideration begins -- between 1.3 and 1.9. If you're a small Hall person, then you'll edge closer to the 1.9 and above candidates. As you can see in those Reliever Hall Index results, all of the relievers in Cooperstown would qualify by this definition. When I reached that conclusion, I was moderately surprised. When Smith was initially announced as a selectee during the winter meetings, I thought the committee was wrong, mostly because the main thing you heard about Smith was that he is third on the saves list. I've changed my mind. Now I can focus my ire strictly on the committee's selection of Harold Baines.
There are 12 relievers who meet the 1.9 standard in the save-free version of the Reliever Hall Index, which means there are four pitchers who have yet to be elected. Here they are:
5. Craig Kimbrel, 2.65
9. Billy Wagner, 2.09
10. Dan Quisenberry, 1.98
12. Francisco Rodriguez, 1.90
Kimbrel, who has nine years of service under his belt and is currently lingering on the free-agent market, is having one heck of a career. He's 14th in saves, but that doesn't even begin to tell the story. He's already 37th among relievers in WAR and JAWS, tied for first in All-Star Percentage, first in ERA+, tied with Jansen for first in dominance and is eighth in win probability added. Clearly, Kimbrel is on a Hall of Fame track.
That brings us to the underappreciated Wagner, who is still on the ballot but so far has drawn just 16 percent of the possible votes this year, according to the tracking data. I am sure this is because of how crowded the ballot is. Wagner, in my opinion, is a clear Hall of Famer. As the ballot clears up, I'll begin to beat that drum louder and louder, as he's in only his fourth year of eligibility. I hope, too, that Quisenberry draws a closer look the next time his era's committee meets.
If my top 12 relievers were all in the Hall, that would still give relievers only about a 3.6 percent representation mark among all eligible firemen. If we open our doors to all that meet that 1.3 standard, then we're still looking at only 21 relievers, bringing the percentage up to 6.3 percent. Here are the relievers who currently fall between 1.3 and 1.9, with active pitchers included:
13. Jonathan Papelbon, 1.88
14. Joe Nathan, 1.84
15. Kenley Jansen, 1.65
16. John Franco, 1.56
17. Tom Gordon, 1.42
18. Tom Henke, 1.40
19. Firpo Marberry, 1.40
20. John Wetteland, 1.38
21. Sparky Lyle, 1.33
Obviously no one would, or should, support Wetteland if the charges recently brought against him are proved to be true. The others here I'd argue on a case-by-case basis, using these metrics as a jumping-off point, and I'm not sure I'd end up advocating for any of them. Jansen, sure, but his standing could still fall further if his off-year in 2018 proves to be the start of a career decline. Then again, he could bounce back and ascend to no-brainer status. The other key name here for me is Marberry, who in addition to coming out nicely in these performance measures also had a crucial impact on the game's history: He was baseball's first relief ace. He also mostly played before the All-Star Game/awards era, and he surely would have seen a boost in those eras if he had not.
As we move forward, and the impact of relief pitching continues to swell, the notion of what a Hall of Fame pitcher looks like might become more hotly debated than ever. Starters who reach 300 wins, or even 250 or 200, will be fewer. Innings totals may continue to fall, but some ace starters will stand out and might still be more obviously destined for Cooperstown than reliever candidates. But we might have to sort through more of the latter than ever before.
With the elections of Rivera and Smith, we're starting to get a clearer picture of the reliever's place in Cooperstown. The good news seems to be that the selectors have yet to sully the representation of this group with truly bad choices. The hope here is that we keep it that way. Luckily, we have more tools to do that than ever.