Show Your Work

I've been wanting to circle back to the Elo scores that teams were earning since I first wrote about how the scores literally mean nothing and the points are made up. While this seems like a rather harsh indictment of the rankings system that U SPORTS currently uses, today I want to present some numbers to give you a better idea of what I mean. In short, you're going to see a lot of math that makes little to no sense, and there will be a lot of head-scratching as to why we're using such a flawed system to rank women's hockey teams nationally in Canada. Let's dig into this!

Before we get started, I'm going to post the Elo scores for the ranked teams over the last three weeks. This will involve eleven teams as there was a change at tenth-place in the final week, but confusion will still be present when we look at the rules that U SPORTS is supposedly playing by and the actual numbers for the teams. Again, these Elo scores will seem meaningless once we're done.

I'll start by posting this chart where I've tracked the Elo scores for the last three weeks of rankings. The columns from left to right are school name, the record for that team on November 23, the Elo score for that team on November 23, the record for that team on November 30, the Elo score for that team on November 30, the record for that team on December 8, the Elo score for that team on December 8, and the total net Elo movement in points over this three-week period. The schools are organized in their current Top-Ten rankings as shown today. As an aside, I've subtracted 1000 Elo points from each team because that 1000 points is rather useless in all situations.

U SPORTS RANKINGS AND ELO SCORES

School	23	Elo	30	Elo	8	Elo	Net
Toronto	4-1	652.64	6-1	675.70	7-1	690.46	+37.82
Saint Mary's	12-2	708.29	13-3	684.30	13-3	684.30	-23.99
Concordia	5-3	628.49	6-4	617.58	7-4	670.60	+42.11
Mount Royal	7-3	624.63	7-3	624.63	9-3	654.54	+29.91
McGill	6-2	643.69	8-2	665.39	9-3	653.73	+10.04
StFX	9-5	607.94	11-5	638.60	11-5	638.60	+30.66
UBC	6-2	602.57	7-3	603.52	9-3	627.53	+29.91
Nipissing	4-0	627.36	5-1	624.54	5-1	624.54	-2.82
UNB	11-3	622.32	11-4	607.17	12-4	623.19	+0.87
Alberta	4-4	N/A	5-5	N/A	7-5	622.62	??.??
Ryerson	2-1	579.63	3-2	582.45	3-3	567.59	-12.04

There's also some context needed in terms of who each of these teams played over the course of these three weeks, so let's look at the results that each team posted since November 23 to give you an idea of who they played and what the outcomes were.

• TORONTO: beat York 2-1; beat York 2-1 in overtime; beat Ryerson 3-2 in overtime.
• SAINT MARY'S: beat UNB 2-1; lost to UPEI 4-2.
• CONCORDIA: lost to Montreal 6-5; beat Carleton 9-0; beat Ottawa 5-1; beat McGill 4-3.
• MOUNT ROYAL: beat Saskatchewan 1-0 twice.
• McGILL: beat Carleton 4-2; beat Bishop's 4-3; beat Montreal 4-3; lost to Concordia 4-3.
• StFX: beat Dalhousie 4-2; beat St. Thomas 8-3.
• UBC: lost to Saskatchewan 4-2; beat Saskatchewan 2-0; beat Trinity Western 8-3; beat Trinity Western 5-2.
• NIPISSING: lost to Ryerson 4-1; beat Ryerson 3-1.
• UNB: lost to Saint Mary's 2-1; beat Moncton 4-0.
• ALBERTA: lost to Manitoba 2-1; beat Manitoba 2-0; beat MacEwan 4-0; beat MacEwan 6-1.
• RYERSON: beat Nipissing 4-1; lost to Nipissing 3-1.

Why is this important? U SPORTS' Elo algorithm awards more points for lower-ranked teams beating higher-ranked teams, specifically if there's a blowout in those games. That also means that losses to teams not ranked in the Top Ten would cost ranked teams more than losses to other nationally-ranked teams. In theory, this seems to be logical, but the actual rankings make less sense when one considers the opposition some of these teams played. This is generally viewed as "strength of schedule", and Elo doesn't account for any of it despite the obvious advantages some of these teams have when it comes to their schedules.

Ok, so we know the scores earned by the teams over three weeks, we know the net scores posted by those teams over that period of time, and we know who we played. Let's start breaking down the complete insanity that Elo scores have created in building a Top-Ten ranking system. Like any good math problem, I'll show my work so you can understand the math behind why I'm so bewildered by this ranking system.

Crush Expansion Teams?

I want to focus on the Alberta-Ryerson movement at the bottom of the standings because this will show us that the strength of the competition should matter. On November 30, Ryerson was ranked tenth-overall with a score of 1582.45 while Alberta was not ranked. I can't tell you their Elo score because it isn't posted anywhere, but it has to be less than 1582.45. Let's go ahead and assume that it's 1582.44 in this case - 0.01 less than Ryerson.

With Alberta playing MacEwan - a 5-5-0-0 team - in the final week of the Canada West first-half schedule, Alberta went off and won both games by a score of 10-1. As a result, Alberta gained at minimum 40.18 points to overtake Ryerson for tenth-overall in the nation. That's the largest net gain for any team outside of Concordia in the final week as Concordia beat the third-ranked team on their schedule.

For context, Alberta beat the sixth-place team in Canada West and earned a minimum of 40.18 points. Ryerson, who lost 3-2 to the top-ranked team in the nation, lost 14.76 points. How does one explain the gain that Alberta made in beating up a first-year U SPORTS team compared to the loss of points Ryerson suffered in taking the top team in the nation to overtime and eventually losing?

Since we're on the topic of blowouts, the comparion is that Concordia lost to Montreal 6-5, but blew out Carleton 9-0. The net change in their Elo points between November 23 and 30 was a loss of 10.91 points. If Alberta blows out MacEwan by nine total goals after losing to unranked Manitoba one week earlier, and Concordia blows out Carleton by nine total goals after losing to unranked Montreal, how on earth is there a minimum 51.09 point difference between what Alberta gained and Concordia lost despite them turning in the exact same results?

And how does UBC gain 24.01 points in throttling a 1-8-1-0 Trinity Western team by combined scores of 13-4? Again, there is a nine-point difference in goals registered by UBC, they lost to unranked Saskatchewan the week before, and somehow there's a 16.17 difference between what Alberta did and UBC did? And there's a 34.92 point difference between what Concordia did and what UBC did?

This is beyond bonkers, and I cannot explain any of it.

How Did You Gain Points?

The next example I want to look at is UBC's results between November 23 and November 30 when they split the weekend with Saskatchewan. Over that weekend, they fell 4-2 to the Huskies while defeating them the next day by a 2-0 score. As we know, Elo says that lower-ranked teams would earn more points by beating a higher-ranked team, so the net change in points for UBC for that week should have been a loss.

The only problem is that UBC gained 0.95 points that week which, by definition, should mean that Saskatchewan is the better team despite not being ranked ahead of UBC on November 23 or November 30.

To contrast that, Nipissing played Ryerson twice in the same week that UBC and Saskatchewan tangled. Ryerson won 4-1 before Nipissing defeated them the next night by a 3-1 score. The net effect for that week was that Nipissing lost 2.82 points after splitting with the tenth-ranked team in the nation while they were ranked fifth-overall, and the loss in points actually caused Nipissing to slip a spot to sixth-overall in the rankings.

In short, UBC splits with an unranked team and GAINS points while Nipissing splits with the tenth-ranked team and LOST points. How on earth does this make any sense when trying to understand how the math works in the flawed Elo system?

Riddle Me This

Sticking with Saskatchewan since they helped UBC slightly, the effect they had on Mount Royal was even more pronounced. Mount Royal and Saskatchewan battled to a pair of 1-0 MRU victories on consecutive nights. The result of those two wins saw Mount Royal's Elo score improve by 29.91 points despite Saskatchewan not being ranked.

Why is this concerning? The 29.91 increase for Mount Royal was only 5.90 points greater than 24.01 point-increase that UBC registered with their blowout wins over Trinity Western, and only 6.85 points better than the increase Toronto saw one week earlier with a pair of 2-1 wins over the 1-5-0-0 York Lions.

If Toronto and UBC can gain 24.01 and 23.06 points, respectively, in defeating teams with one regulation win all season, what does that say about Saskatchewan's ranking on the Elo scale if Mount Royal only earns 29.91 with a pair of wins despite Saskatchewan having a clearly superior record compared to both TWU and York? Does that mean the gap between Mount Royal and Saskatchewan is larger than that of UBC and TWU or Toronto and York? If that's true, this system is broken in a big way.

The Schedule Absolutely Matters

Toronto currently sits as the top-ranked team in the nation as the U SPORTS rankings show them with a 7-1 record. The problem is that their 7-1 record is entirely and wholly deceiving because the competition they've played - Ontario Tech, Queen's, Ryerson, and York - is vastly inferior to Toronto. Of those four teams only Ryerson has a winning record as they sit with a 3-2-0-1 record currently, and the combined records of those four teams is a cool 7-16-1-2. Toronto, essentially, is being rewarded for beating up on bad teams.

This matters when it comes to vastly superior teams such as Saint Mary's who currently sit as the second-overall team in the nation. As it stands, Saint Mary's is shown as a 13-3 team on the U SPORTS rankings, but it's who they've played that makes their 13-3 record more impressive. The rest of the AUS is currently sitting at 51-52-2-7 on the season, but there are three Top-Ten teams in the conference with a possible fourth in UPEI.

Against the likes of UNB, StFX, and UPEI, the Saint Mary's Huskies are 4-1-0-1 this season with their extra-time loss coming back on October 15 to UNB and their regulation loss coming to UPEI on November 28. They have yet to lose to StFX, a team who has been ranked all season, and they are 1-0-1 against UNB who has also been ranked all season. They've outscored those three teams 20-14 in the six games they've played with their closest margin of victory being a 2-1 win over UNB on November 27. In other words, Saint Mary's has been the best AUS team all season long.

Why is Toronto ranked ahead of Saint Mary's if Toronto is 1-0-1-0 against the formerly-ranked Ryerson while Saint Mary's is 3-0-1 against currently-ranked StFX and UNB? If Saint Mary's is routinely beating other higher-ranked Top-Ten teams, it seems pretty elementary who the better team is when comparing the top-two teams in the nation.

Proper Records Matter

I took the liberty of breaking down each team's record into three-point records with a calculation made for regulation points. You might be asking why regulation points matter. There are reasons why inferior teams want games to come down to one shot or one bounce or one deflection - it gives them a 50-50 chance of winning. Winning in regulation time matters more than winning in overtime, and we need to stop giving overtime wins the same weight as regulation wins. The numbers you see on the right-side of the chart is the regulation win percentage, point totals if every conference used a three-point system, and three-point points percentage.

U SPORTS THREE-POINT RECORDS

School	Record	Reg. Win %	Points	Points %
Toronto	5-0-2-1	.625	20	.833
Saint Mary's	12-2-1-1	.750	39	.813
Concordia	7-3-0-1	.636	22	.667
Mount Royal	8-1-1-2	.667	27	.750
McGill	7-3-2-0	.583	25	.694
StFX	10-5-1-0	.625	32	.667
UBC	9-3-0-0	.750	27	.750
Nipissing	5-1-0-0	.833	15	.833
UNB	9-3-3-1	.563	34	.708
Alberta	4-4-3-1	.333	19	.528
Ryerson	3-2-0-1	.500	10	.556

If you're looking at these numbers, you need to ask yourself which one is the important one. We know regulation wins matter, specifically in a three-point system, so I'm going to direct your attention to the points percentage on the very right of the chart.

This is where we can temper some of the excitement for teams such as Nipissing and Toronto since they've played the fewest games and have played the weakest schedules. Yes, the points percentage is high for these two teams, but it's also expected to be high when playing competition whose combined record is ten games below .500. Should Toronto really be ranked first-overall based on this analysis? I'll let you come to your own conclusions, but the correct answer is "absolutely not".

Should Alberta be in the Top-Ten rankings with a barely-above-.500 record? Again, "absolutely not" is the right answer, yet they sit as the tenth-ranked team today for some hard-to-fathom reason. Forget that Manitoba has a superior record and points percentage, and has beaten all of Mount Royal, UBC, and Alberta this season. Forget that Montreal has a superior record and points percentage, and has beaten Concordia this season. Forget that UPEI has a superior record and points percentage, and they've defeated Saint Mary's, UNB, and StFX this season.

Of course, when the team that Alberta replaced - Ryerson - has a better record and points percentage, you know that Alberta deserves that tenth-place ranking because Elo math says so!

Should StFX be ahead of UNB when looking at their points percentage? This is a tougher breakdown, but what if I told you that UNB is 2-1-0-1 against Saint Mary's and StFX this season while StFX was 0-3-1-0 against SMU and UNB? It would seem that UNB's points percentage figure is a better indicator of where they may finish the season when compared to StFX. Again, this isn't written in stone by any means, but today's rankings should see UNB ranked ahead of, not behind, StFX after knowing that StFX struggles against the two teams ahead of them in the AUS with one of those teams being UNB.

Ok, We Get Your Point

I don't want to be the guy who points out that U SPORTS doesn't know what they're doing, but it seems pretty clear with some hard evidence that U SPORTS doesn't have the faintest clue on how to properly rank hockey teams. Beyond that, this modified Elo system introduced by Mario Kovacevic is nothing more than garbage wrapped in rubbish tucked inside of math that has no credibility whatsoever.

A lot of these flaws were outlined in my article last month, but showing the math in action as I did here today only makes those flaws more glaring. If U SPORTS was trying to embarrass itself, it's doing a heckuva job in using the Elo scores to determine the best women's hockey teams in the nation.

Again, How Do We Fix This?

I don't think we need to fix the system as much as scrap it altogether until someone can sit down and factor in all the variables that go into women's hockey such as strength of schedule, current standings and points percentages, and probability of winning. We preach in Canada West that any team can beat any other team on a given night, but we should know that both Mount Royal and UBC have a three-in-four chance of coming away with points and a better-than-good chance of winning in regulation. Because of this, these two teams should be in the conversation for the Top Ten rankings.

However, strength of schedule for Mount Royal should push them higher compared to UBC who has four of nine wins against Trinity Western this season. UBC's schedule through the first-half of the season has been considerably easier than that of, say, Manitoba, and this should also weigh in on whether or not UBC deserves its seventh-overall ranking compared to ninth-ranked UNB who has played four games against SMU and StFX and gone 2-1-0-1 in those games. If you asked me right now, I'd say that UNB should be ranked higher than UBC due to strength of schedule.

Because UBC and UNB will never meet this season outside of the National Championship tournament if both qualify and are seeded accordingly, we'll never know who defeated whom in a game between them because it simply will not happen ever. This is the problem in trying to rank teams from four conferences whose schedules never overlap - there's zero evidence Team A is better than Team B based on head-to-head games.

Where things fall apart, though, is that the teams who do play head-to-head often enough should mirror their records against one another. In saying that, SMU should be ranked ahead of StFX, UNB, and UPEI with their 4-1-0-1 record against those teams, but UPEI's 3-3-0-0 record and UNB's 2-2-1-1 record are considerably better than StFX's 1-4-1-0 record in head-to-head games. Because of this, StFX should be moved down the rankings because they've accumulated most of their points from beating teams who they should beat according to win probability.

Last Thoughts

Look, I know everyone - players, fans, and schools - loves seeing their teams on the Top Ten rankings. It's exciting, a point of pride, and something to boast about to friends and family. But until this flawed mathematical calculation is fixed and actually incorporates important variables that go into determining a team's final standing, we should walk away from this ranking system like it has a contagious plague. It serves no one beneficially at this point, and it needs to be retired immediately.

If you care about this game, talk to your school's athletic department about this. The only people who U SPORTS will listen to are those paying their dues under the U SPORTS umbrella. I don't exist to them, so this article will be ignored like everything else I do to try and make U SPORTS women's hockey better, so demand more from your school. And if you don't care enough to see this fixed, I thank you for reading and I hope I was at least entertaining. But if you truly care, please engage your school and ask them to make this a point with U SPORTS.

It's the only way this will get better.

Until next time, keep your sticks on the ice!