There seems to be a decided preference for EV tied and EV close data over the raw numbers. That makes sense - the raw data is subject to score effects, which makes the information less valuable with respect to distinguishing good teams from bad ones.

Interestingly, however, there doesn't appear to be a general agreement as to which of shot, fenwick and corsi percentage serves as the best metric to use once score effects have been controlled for. While fenwick seems to be the most popular, there are some who like corsi, and there are even a few prefer shot percentage over both. This raises the question: which of the three measures ought to be looked to for the purpose of team evaluation?

As Gabe Desjardins once correctly observed, there is a stronger relationship between fenwick and winning percentage than there is between corsi and winning or between shot differential and winning. In fact, according to Gabe's numbers, the correlation between corsi and winning percentage was about the same as the correlation between shot differential and winning percentage, even though including blocked shots substantially increased the sample size. The upshot is that the inclusion of blocked shots in the analysis doesn't add much information.

Gabe's discovery may account for the slight preference towards fenwick discussed above.

However, the weaker relationship between corsi and winning can be partially accounted for by score effects. In particular, the trailing team does better in terms of corsi than it does with respect to either shot percentage or fenwick.

As such, while overall corsi has a lower correlation with winning than overall fenwick, the same may not hold with respect to score tied corsi and score tied fenwick.

In an attempt to resolve this issue, I performed a series of calculations, the results of which have been posted below.

This table shows the split-half reliabilities for score tied corsi, score tied fenwick and score tied shot percentage. The split-half reliabilities for each variable were calculated by randomly selecting 40 games, randomly selecting an independent group of 40 games (that is, a game chosen in one group was necessarily excluded from the other), and using the two data sets to determine the correlations for each variable. This was repeated 1000 times, with the above table showing the average values.

Not surprisingly, corsi is more reliable than either fenwick or shot ratio at the half-season level, which is a product of the fact that there are simply more corsi events then fenwick or shot events in our sample. Thus, corsi should prima facie be considered the superior metric of the three due to its superior reliability.

Ignore the goal ratio column for now - it's only been included for the purpose of performing a subsequent calculation.

This table shows the predictive validity of the same three variables with respect to overall goal ratio. Here, predictive validity was determined by randomly selecting 40 games, calculating each team's score tied corsi, fenwick and shot percentage within that sample, and looking at how each variable correlated with overall goal ratio in an independently selected 40 game sample. As with the first table, the numbers here are the averaged values over 1000 trials.

The predictive validity of each variable is commensurate with its reliability co-efficient, with corsi having the most predictive validity. In other words, a team's score tied corsi over a 40 game sample is a better indicator of how it will perform over the remainder of its schedule than is score tied fenwick or score tied shot percentage.

Of course, the fact that corsi has the most predictive validity in practice doesn't necessarily mean that it serves as the best measure of team skill in theory. As discussed in a previous post, the observed correlation between two variables is contingent upon the reliability with which each variable can be measured. Fortunately, there exists a formula that can be used to calculate what the correlation between two variables would be if each could be measured with perfect reliability. That formula involves dividing the observed correlation by the product of each variable's reliability co-efficient.

r xy adjusted = r xy observed/ SQRT( reliability x * reliability y)

As we already have the split-half reliability co-efficients for all of the variables, we only need to determine the split-half correlations between score tied corsi, score tied fenwick and score tied shot percentage, on the one hand, and goal ratio, on the other.

After inputting all of the relevant variables into the above formula, the following values are obtained:

Therefore, while corsi has more predictive validity with respect to goal ratio at the within-season level, fenwick and shot percentage appear to correlate more strongly with goal ratio over a sufficiently large sample of games. In other words, in theory, both fenwick and shot percentage seem to serve as better measures of team quality than corsi does.

One caveat: the differences between the values here are small, and we only have three seasons of data. It may very well be that all three variables correlate equally well with goal ratio over the long run. This subject may require further study in the future when more data is available.

After inputting all of the relevant variables into the above formula, the following values are obtained:

Therefore, while corsi has more predictive validity with respect to goal ratio at the within-season level, fenwick and shot percentage appear to correlate more strongly with goal ratio over a sufficiently large sample of games. In other words, in theory, both fenwick and shot percentage seem to serve as better measures of team quality than corsi does.

One caveat: the differences between the values here are small, and we only have three seasons of data. It may very well be that all three variables correlate equally well with goal ratio over the long run. This subject may require further study in the future when more data is available.

## 10 comments:

Just a question, aren't there problems with running correlation values for Shot Percentage relative to Goal Ratio, since the two events are tied together by definition?

This is great J. So if I understand correctly, Corsi gives us better predictive value at the moment because of sample size, but if the sample sizes were all equal, Fenwick has the slightest edge over the others?

@The Forechecker

Shot Percentage isn't to be confused with Shooting Percentage. Shot Percentage, like Corsi and Fenwick, is an expression of the percentage of total shots on net by the instant team. The expression is shotsX/(shotsX+shotsY). Goals are included in shots, but shot percentage is not tied to goals. Sorry if that's not clear, I haven't finished my coffee yet.

Also, this is great stuff J. Keep up the solid work! Cheers.

@Forechecker:

Perhaps I should have been more clear in my post, but ranford4life is correct in that I was using shot percentage to mean "(shots for/(shots for + shots against)".

That said, the relationship between all three variables and goal ratio is slightly problematic when it comes to adjusting the correlations to account for the imperfect reliability.

When describing this very same adjustment process in a previous post, I had this to say:

"It should be noted that this second method is likely to slightly overestimate the true correlation, given that the two variables are not truly independent."@Ryan

That's correct. Although it's also possible that all three metrics are equally predictive over the long run.

@ranford4life:

Thanks!

good stuff, JL.

have u tried looking at this with road data only and home data only?

Excellent, thanks for clarifying!

@Sunny:

I haven't ran the road numbers, but I'm not sure if the results would be that much different than what we'd obtain if we halved the entire sample.

The recording of blocked and missed shots, and shots on goal, for that matter, is fairly inconsistent league wide, but as near as I can tell, none of the scorers seem to favor one team over the other.

@Forechecker:

No problem!

Just to clarify

Shooting % is even strength shots

only? OR, total shots?

Sorry, I should have clarified.

Shot percentage refers only to even strength shots.

I think that it is great that you are doing something like that. You can find interesting things in other blogs and things that you can discuss a lot.

Post a Comment