Hockey analytics pioneer Rob Vollman is ESPN Insider's armchair GM, exploring how modern statistics can inform front-office decisions.
Rather than spending the dying days of the summer trying to find one last free agent, NHL teams should be fine-tuning their coaching staffs. The Toronto Maple Leafs certainly believe in this, having opened their proverbial wallet wide for Mike Babcock, and having recently added the legendary Jacques Lemaire to his staff. The Leafs are the most prominent of at least a half dozen teams that have made significant upgrades behind the bench, and who stand to make noticeable leaps in the standings.
It has always boggled me why there is so little focus on coaching, not just by front offices, but by the pundits. Either they feel that a coaching staff's performance can't really be measured, or that their impact is unimportant -- or both!
They're wrong. In my experience with hockey analytics, neither situation is the case. The stronger coaches can be identified, and have a very predictable and significant impact on a team's fortunes.
Specifically, I introduced a simple coaching statistic six years ago, and since then I have observed that starting a season with one of the league's more accomplished head coaches almost triples a team's chance of making the final four. And, in contrast, those coaches that start the season ranked in the bottom half of the league (by this metric) are four to five times more likely to get fired that season.
Other than Toronto, which teams have positioned themselves well this season? Let's briefly break down this coaching statistic and its high-level results, then divide all coaches into various levels of experience and performance, and determine objectively how they can impact a team's position in the standings and likelihood of reaching the conference finals -- or if they should be keeping their résumés up-to-date!
Measuring the impact of coaches
On the surface, developing a high-level coaching metric seems quite simple. After all, the number of points their teams have earned, relative to expectations, should be a pretty reasonable estimate of a coach's value to his team. The problem is how to establish those expectations.
Should pundits set each team's expectations? No. For the past couple of seasons, I've gathered the preseason predictions of countless experts, including my own, and our average forecasts ultimately prove to be only negligibly more accurate than just using the previous season's standings. In other words, that's just a lot of work for nothing.
Obviously, I've also tried basing a team's expectations using a variety of statistical models, but the accuracy of such systems -- both mine and those of fellow stat lovers -- tends to be even worse than the aggregate pool of pundits. That won't work, either.
In the end, the simple road is the best route. Specifically, each coach's expectations are set using his team's previous-season point totals, regressed toward the league average by 35 percent. This last adjustment puts coaches on both good and bad teams on an equal footing.
The results? If this perspective on coaches is to have any practical value, then we should be able to go back over the six seasons of its existence, and see a noticeable link between how the coaches were ranked at the start of the seasons, and how well their teams performed the following season. In general, the top-ranked coaches should be more likely to make the conference finals, and those at the bottom should be more likely to find themselves looking for work -- and both situations are indeed the case.
Consider the following chart, which highlights the very different fates in store for those coaches who start the season ranked in the top half, and those who don't. There have been 48 coaches in the past six years who have been fired or who parted ways for whatever reason, which is exactly eight per season. Of the 48, 16 began their last season ranked in the top half of the league's coaches, and 32 began in the bottom half. That means that a below-average head coach is twice as likely to get fired that season.
More importantly, of the 22 coaches who reached the conference finals over the past six years (note that two more finalists were midseason replacements and therefore had no preseason rank), nine were ranked in the top six at season's start. Furthermore, only four were ranked in the bottom half (15 positions) -- that's a tiny green sliver!
That means teams with an above-average coaching staff are four to five times more likely to make the final four, and teams with an elite coaching staff improve their chances of making the conference finals from roughly 1-in-11 to 1-in-4. Very few individual skaters can have that kind of impact.
Current coaches categorized
What does this all mean for this coming season? While individual circumstances can cause results to vary in unpredictable ways on a case-by-case basis, dividing coaches into groups -- based on their experience and/or previous success -- can establish some reliable overall patterns.
That's why I have divided all 150 head coaches from the past six years into the seven categories included on the following chart. For all of this coaching data to be useful, the results have to be both predictable and impactful on the bottom line, both of which clearly are the case below:
Accomplished coaches are those who began the season with at least three seasons of NHL experience, during which time their teams averaged five extra points (or more) in the standings per season as described above. Five points is hardly trivial; more than half of last season's playoff teams qualified by five points or less.
Not only do these elite coaches continue to add five points to the bottom line the following season, but they are far more likely to reach the conference finals than any other category of coach, and only first-time coaches are less likely to be fired than the elites. While the exact classification of coaches will never be an exact science (the list of all 30 current coaches can be found at right), this is good news for coaches in Anaheim, Boston, Buffalo, Montreal, Nashville and St. Louis.
The next three categories involve coaches who are equally experienced but whose prior performances haven't been nearly as triumphant. Specifically, solid coaches are those who historically added a couple of points in the standings, average coaches break even, and weaker coaches somehow have landed continued opportunities despite subpar results. At season's end, the difference between them is slight, but it does exist. The solid coaches continue to add a couple of points in the standings, while the weaker coaches continue to actually hurt their clubs, and the odds of getting dismissed rise from 23 percent for the solid coaches to 29 percent for the weaker ones.
Some coaches don't have enough experience to be grouped properly, but their results are interesting nevertheless. Those who have posted strong results in their limited time as NHL bench bosses don't fare much worse than the experienced coaches, while those who haven't are practically destined to look for work by season's end.
It's hard to make these final groupings with any accuracy, which is why I have started to dig into new coaches' results in other leagues, like the AHL and the Canadian major junior leagues. In those regards, for example, inexperienced NHL coaches like Vancouver's Willie Desjardins and Pittsburgh's Mike Johnston are quite well-accomplished. As more of that data is included, and the entirety of a team's staff is factored in, the distinction between good and bad coaching will become even more obvious than it is today.
Closing thoughts
I don't know why most publications ignore the impact of coaching when they're making their preseason forecasts. They'll break down a team's power play, penalty kill, goaltending and prospect pools in great detail, but make only passing remarks about the team's coaching staff. The same is true in some NHL front offices, who seem to ignore the reality that a strong bench boss can have just as much value as the team's franchise player.
It is possible to make distinctions between the stronger and the less experienced and/or accomplished head coaches, and it is possible to measure their impact -- and it's not trivial! Good coaches can boost a team's position in the standings by several points, and teams that invest in the league's most accomplished coaches are far more likely to make the conference finals than those that don't.
Why spend tens of millions of dollars on elite players, and then gamble on those tasked with unleashing their full talent? Teams like the Maple Leafs are on the right track: Make the coaching staff one of the top priorities.