Ideals vs averages: goalies, improvement, and aging

The new thing in goalie statistics appears to be aging curves. These purport to show the average change in save percentage over time for goalies, giving us an idea of what to expect from the average goalie going forward. This is one example.

TheoreticalSV_change905_mediumI took this from “Theoretical Goaltending Aging Curve” by Steve Burtch of Pension Plan Puppets. There are others, some with better explanations than others. One of the most recent is “How well do goalies age? A look at a goalie aging curve” by garik16 at Hockey-graphs.com.

Most of the early attempts at this work indicate that, on average, goalies have been getting better save percentages as they age from 18 to 25 or so, which supports what goalie development people (and every goaltender coach I’ve ever spoken to) see in real life. That goalies get better between draft age and their young-20s. Or, quite possibly, that time weeds out the goalies who had anomalously great seasons at age 17 or 18 (i.e., just before being drafted) but who can’t sustain that level of play.

Recent curves don’t even look at the 18 to 22 years any more, and that’s good. There simply aren’t enough goalies who play in the NHL for these samples to be useable. Between 2000 and 2014 exactly 32 goalies have played even a single NHL game at age 18, 19, or 20. Whereas 52 have played at age 21–still a very small sample, given that only 15 of those played more than 10 games at that age. Remember that this is 52 players in a fourteen year time span. Virtually no conclusions can be drawn from these samples, they’re so small.

But there are other reasons to think of goalie aging curves with a healthy amount of skepticism. They rely on save percentages (which as I have noted before has real problems as a tool of objective evaluation) and are averages only. And when dealing with goaltending numbers, averages are almost never good pictures of reality.

What most of these aging curves actually show is not that “Your lousy 25 year old goalie is not going to get better with experience – he’s just going to get worse,” but that–in general–individual improvements are tough to see with save percentage and are likely swamped by group patterns. Individual technical improvement does happen, but it is hard to capture with save percentage. And year-to-year variation based on the sheer randomness in the stat can easily mask the result of such improvement. Save percentage is simply not a finely-tuned enough instrument. It takes far too much data to determine actual performance than is practicable for use in real-life evaluation of individual goaltenders.

Everyone knows this, of course. We simply don’t have anything else to use. However, it is still critical to be cognizant of the problems of what we’re using to measure improvement or decline and to draw conclusions appropriate to that.

It is also unclear how much some of these curves are taking survivorship bias into account. Players who have low numbers in an early year are far less likely to remain in the population than players who have low numbers in a later year. They disappear, so we don’t know if they improve or not. The best studies of aging will explicitly talk about how they are dealing with that.

While it is clear, based on real-world experience and on the statistical analysis, that on the older end of the aging curve (after age 27 or so) the average goaltender will begin to see more decline than improvement year to year. Still that alone should not be enough to make decisions about individual players. There simply are very few goaltenders who are, in actuality, average at every step of their NHL career. An aging curve is a picture of a group, not of an individual. It actually says less about “your lousy 25 year old goalie” than it does about 26-year-olds as a group.

Taking an average sacrifices the improvements of some goalies to the declines of others. Saying “goalies don’t improve” between 23 and 27 because the group doesn’t improve implies a causality that isn’t supported. It’s a blanket generalization that age is the cause of decline and that all goalies will thus decline at the same rate and the same time, regardless of what they do. Without a clear picture of the level of variation or the role of survivorship, it is a mistake to apply these results directly and without adjustment to any individual player.

The aging data certainly ought to give General Managers pause when doling out multi-year contracts to goaltenders over the age of 32 or 33. They are likely to be of far less value in the later years of such contracts than they are in the early years. However, the guys who are in line for those contracts are a breed apart already. So many goaltenders do not succeed at the NHL level in any capacity at any age, and a large fraction of those who do are getting two and three year deals for their entire careers. There is an argument to be made that Henrik Lundqvist at age 35 is better than any number of other goaltenders at at 24.

As it stands now, goalie aging curves give visual impact to the idea that age matters. They do not, however, support some of the sweeping generalizations being made about goaltender aging. There is no ideal age for a goaltender, just as there are few truly average goaltender career arcs. Save percentage is far too blunt an instrument to be used on its own and relying on averages further blunts it. Attempting to elide rather than account for year-to-year variance in save percentage seems to me an exercise in futility, obscuring as much as it enlightens.

We simply don’t know enough about the position from a statistics standpoint to determine an ideal age for NHL goalies or to understand how the averages apply to individuals. Much more work needs to be done on objective evaluation of goaltending before we start drawing strong conclusions.

 

About these ads

One thought on “Ideals vs averages: goalies, improvement, and aging

  1. Pingback: Ben Bishop's Tale of Two Seasons - The Goalie Magazine - InGoalMag.com

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s