Sports by the Numbers

Last year the NCAA introduced a new method of rating NCAA mens’ college basketball teams called “NET” and supposedly stopped using RPI as its principal criterion for seeding the annual Tournament in March.  In other posts here I’ve shown that RPI was the dominant influence on seeding before 2019 with adjustments made depending on the conference to which a team belonged and whether it won its conference championship.

The Significance of NET

The NCAA’s adoption of NET made it the most powerful predictor of Tournament seedings in 2019, but both a team’s RPI and whether it had won a major conference championship continued to influence the seedings as well. For a few prominent teams the gap between the two measures was enormous.  RPI adjusts heavily for a team’s “strength of schedule” while NET uses a variety of criteria.

Last year, Kansas entered the Tournament with the nation’s second-best RPI but ranked only twentieth in NET, no doubt influenced by the team’s underwhelming 25-9 record.  Its high RPI reflected its schedule, the toughest in the nation last year. In past Tournaments a team with the second-best RPI would usually be seeded on the top line. Now, with NET the dominant factor, Kansas received only a four seed and won just one game.

Divergent cases like Kansas might prove informative.  I have compared the NET and RPI ratings from teams participating in the 2019 Tournament, the only one to use NET so far.  I excluded conference champions who receive automatic bids to the Tournament and used only the remaining 34 teams that received an “at-large” bid. As always, I excluded the four play-in games for two eleven and two sixteen seeds.

When NET and RPI diverge

There are two methods to measure the divergence between two rankings, their difference and their ratio.  I’ve chosen to use the ratio of the NET ranking to the RPI ranking rather than the difference between them.  A difference of two ranks probably means more at one and three rather than nine and eleven.  Using the ratio accounts for that intuition. The ranks one and three have a ratio of 0.33, very low as we’ll see shortly. A pair of teams at nine and eleven score a more mediocre 0.82.  On this measure, lower values are better since the score is based on rankings, just the same way being ranked first is better than being ranked twentieth.

I have grouped the 34 teams into three categories. The first row presents results for teams whose NET ranking was considerably worse (larger) than their RPI.  Kansas is the poster child for this measure; it has a ratio of 20 because it was rated twentieth on NET but first on RPI (=20/1). On the bottom row we have the results for teams who were viewed much more favorably on NET than RPI. Virginia Tech, Gonzaga, last year’s runner-up Texas Tech, and last year’s champion Virginia all belonged to this favored group.

Because the samples are small, and the results limited to just one year, I make no broad claims about the applicability of these findings. Still, teams with better rankings on NET than on RPI appear to have won more games in last year’s Tournament. Teams whose NET rankings were considerably larger than their RPI rankings won an average of only 0.8 games in 2019.  Teams whose NET rankings were considerably smaller than their rankings on RPI won an average of 2.0 games.

Some of this difference in games won reflects the seedings the teams in each group received. Teams with low NET/RPI ratios were seeded on average at 5.3 while those higher ratios averaged worse seedings of 5.8 and 7.0. Because seeding is the most powerful determinant of Tournament victories, better-seeded teams should on average record more wins.

To estimate the effect of seeding I predicted each team’s number of wins in 2019 using the base-two logarithm of its seed.  For a top-ranked team the prediction is four wins, or entrance into the Final Four.  Teams with seedings below seven are predicted to have a negative number of wins.  I have truncated these to zero just as the Tobit procedure I use does when calculating the predicted number of wins in the table.

For teams in the two groups with NET/RPI ratios above 0.7, seeding alone predicts a nearly identical number of wins. But for the teams with lowest ratios, the prediction based on seeding is just 1.4 wins, well below the 2.0 wins we observe.  That suggests that seeding alone cannot fully account for the success of these teams whose NET ratings were substantially better than their RPIs.*

NET and RPI in 2020

Auburn this year looks like Kansas from 2019. As of March 3rd, Auburn currently has the third-best RPI but ranks only 27th by NET. Unlike last year’s Kansas team, Auburn has a gaudy 24-4 record but only the 44th-ranked strength of schedule. Based on NET alone, we might expect Auburn to draw only a six or seven seed in this year’s Tournament. A seeding based on RPI alone would put it on the “two-line.” Auburn’s actual seeding will tell us a lot about how the Tournament Committee values NET and RPI rankings.

Here are the NET, RPI, and ratio values for the top-25 teams as ranked on NET.  Gonzaga, Michigan State, and Texas Tech look like possible Tournament over-performers while Villanova, Florida State, and Oregon may disappoint their fans.

____________________

.

A few days back I wrote about the relationship between the NCAA’s new, proprietary method of ranking college basketball teams called NET, and the traditional measure of team strength known as RPI.  The two were unsurprisingly closely matched, but there were some strong disparities between the two measures when it came to certain teams like Kansas.

Since the two measures are not perfectly correlated, they might have played separate roles in the Committee’s decision-making process when seeding the field last Sunday.  In addition, I showed in that earlier article that the NET methodology already had a built-in bias toward teams in the six major conferences.  The Committee has always favored teams from the major conferences and seeded them higher than teams from other conferences with the same RPI score. If the new NET score already benefits major-conference teams, did they also get their traditional bonus from years past, in effect a “double bonus?”

To answer this question I estimated a slightly different version of the model I’ve used before to predict seedings.  I include RPI and major conference memberships once more, but this time I added the NET measure as well.  Because NET is simply a ranking taking its logarithm improves the fit.  So this model asks whether both NET and RPI had independent influences on seeding position, and whether major-conference teams still received a bonus.  Here are the results (again using Tobit with censors at one and sixteen):Even with just sixty teams, all three factors prove “significant.” Of the three coefficients in the first column, only the one for major conferences has an intuitive meaning.  Teams from those conferences were seeded nearly two (1.9) ranks above teams from other conferences with identical scores on RPI and NET ranking.

A more useful comparison comes in the “standardized” coefficients, which adjust for differences in the variation of the different measures used.  By this measure, the most important factor by far in seeding decisions this year was NET.  However, at the margins, RPI played a role as did whether the team came from a major conference.

While the “double bonus” is statistically measurable, it does not seem to have affected the distribution of at-large bids.  Half the field, 32 teams, receive automatic bids because they have won their conference tournaments. The other half are selected by the Committee to receive an at-large bid. During the ESPN discussion show following the draw on Sunday night, famed basketball commentator Dick Vitale complained that the Committee was awarding too many at-large bids to teams from major conferences. Might NET have been responsible for that?

There was a slight uptick in the number of at-large bids given to major-conference teams this year, but that has been the trend since 2012. If NET was responsible for closing out smaller schools, it had a very limited impact compared to the trend.

Major-conference teams get a bonus from NET compared to RPI. Since the Committee has traditionally preferred teams from the majors, will those teams now get a “double bonus?”

As most anyone who follows college basketball knows by now, beginning in 2019 the NCAA has chosen to use its own in-house method, called “NET,’ to determine the seedings for the NCAA Men’s College Basketball Tournament.  In past years the NCAA has relied primarily on “RPI,” along with conference memberships and major championships, to determine seeding.

RPI relies on a team’s won-loss performance, the performances of the teams it played, and the performances of those teams against their other opponents.  It also has a simple formula which weights those three factors by 25%, 50%, and 25% respectively.

The NCAA has described the process by which NET ratings are determined, but they have not revealed the weightings or formulas used.  Worse, the NCAA does not release each team’s actual score on NET. They only report the ranking of the teams on this concealed measure.  This makes comparisons with RPI, which takes on specific numerical values, more difficult.

Rankings obscure the “distance” between adjacent teams.  The number one ranked team might be considerably better than the second-place team, but that information is lost in rankings.  For instance, here are the top-ten teams in 2018 by RPI and their rankings on that measure:

The “distance” as measured by RPI between the one and two seeds, and the two and three seeds, is essentially equal, at 0.01 on the RPI scale.  That consistency soon breaks down once we look at teams with seedings or four or greater.  Kansas and Duke were separated by only .0005 of an RPI point, twenty times less than the difference between Virginia and Villanova. Eighth-ranked Tennessee is fully 0.0066 behind Cinncinati, but only 0.001 ahead of Purdue.  The NCAA’s decision to only release rankings on NET, and not numerical team scores, obscures these real differences in team performances.

That decision has forced me to treat the NET ranks as an “interval-level” performance scale like RPI, ignoring the problem of unequal distances just discussed. Here is the relationship between NET and RPI for the top-100 teams as ranked by NET.

The red line and red dots represent teams from the six major conferences, the Atlantic Coast, Big East, Big Ten, Big Twelve, Southeastern, and Pacific Twelve. The blue line and dots represent teams from other conferences. The labelled teams statistically deviate from their predicted NET ratings by 30 ranks or more.

Kansas is the obvious poster child for differences between the two measures. Kansas has sat atop the RPI rankings all season despite its 25-8 record because of its strength of schedule.  Other teams like North Carolina State and Wofford have been substantially advantaged using NET when compared to their RPI ratings, while teams like Yale and St. John’s are significantly underrated.

The most obvious and important aspect of NET is its built-in preference for teams from major conferences. Major-conference teams, indicated by the red line, get NET scores on average ten full ranks better than teams from other conferences (the blue line) with identical RPI scores.

As the chart at the top of this article shows, the NCAA Selection Committee has consistently granted major and mid-major teams a substantial seeding bonus compared to teams from other conferences.  Since NET already includes a bonus for major-conference teams, should we expect major conference teams will now get a “double bonus?”

Twelve of the past eighteen Tournaments have been won by a top seed.

Graphic updated with data through the 2019 Tournament

On Sunday the NCAA Selection Committee will release this year’s Tournament brackets and create the structure within which the teams will compete.  Teams will be placed in one of sixteen seedings spread across four so-called “regions,” for a total of sixty-four competitors.  How teams are seeded dramatically influences their chances of victory in the Tournament.  Those top seeds should be feeling optimistic as they begin their paths through the Tournament.  Twelve of the past eighteen Tournaments have been won by a top seed.

Here is a comprehensive look at the performance of all 1,216 competitors, representing 249 programs, that have participated in the nineteen Tournaments between 2001 and 2019. (Play-in games are excluded.)

Of the 380 teams seeded twelfth or lower, only one has managed to advance to the Sweet Sixteen.  More interesting is the exceptional performance of the seven and eleven seeds.  Six of the 72 eleven seeds made the Sweet Sixteen, as did seven of the teams ranked seventh.  One of them, Connecticut, won the Championship in 2014.  The relative success of eleven seeds hurt the performance of the six seeds against whom they play in the first round.

If I were a college basketball coach, I would be very disappointed to find my team seeded eighth or ninth.  These teams face each other in the first game, with the survivor usually losing to the top seed in the next round.

The survival rates for teams rated fifth or higher rise quickly along with their seedings, but it is the top-seeded teams whose performances really stand out.  Half the teams (19 of 38) that reached the Final Four were top seeds, and twelve of those eighteen teams won the Championship. The remaining winners consisted of two two-seeds, three three-seeds, and one seven seed, Connecticut in 2014.

I was curious to see whether the passage of time has changed how the Tournament Committee makes its seeding decisions.  Does an 0.60 RPI have the same meaning in 2018 as it did at the beginning of the 21st century?  Do teams with an 0.60 in 2018 get about the same seeding as they would have received in 2000?

To address this question I simply added a term that measures the number of years since 2000 to my standard model.  The effect is significant and uniform across all three types of conferences.  This chart presents the estimated relationship between RPI and seeding for major conference teams setting the elapsed time variable to its extreme values of one (2001) and 17 (2017).

The difference is substantial.  Since 2000, the Committee has reduced the seedings they grant by about 1.3 ranks.¹ Another way to look at this is to ask how much greater does a team’s RPI need to be in 2017 to get the same seeding it would have gotten in 2001. In 2001, a six seed would have required an RPI of 0.602; in 2017, that floor had been raised to 0.613. In the RPI rankings for 2018, that small numerical difference in RPIs represents the gap between Arizona (0.612; 18th in RPI) and Loyola-Chicago (0.6027; 26th).

¹I calculate this effect by multiplying the estimated coefficient 0.0787 ranks/year by 17 years. Letting this effect vary by type of conference added no explanatory power (p>.75 no difference). A test of whether the slope of the relationship between RPI and seeding varied over time, by including the interaction of RPI and time, was similarly unproductive.  The gap in seedings did not change over time producing the parallel lines in the graph above.

Three teams, Virginia, Xavier and Villanova, have placed themselves quite far ahead of the pack this year.  At the moment, Duke would be favored to win the last remaining top seed, but North Carolina could overtake Duke were UNC to win the ACC championship this weekend.  Winning a major championship is worth about one full rank, which would move North Carolina into the fourth slot and relegate Duke to the second line.  A victory for Clemson in the ACC tournament would move it up to a two seed along with Kansas and North Carolina.

Kansas seems locked into the second line regardless of what happens in the other major conference tourneys.  Michigan’s #11 ranking already incorporates the bonus it received for winning the Big Ten championship last weekend, so it seems unlikely the Committee would move it further ahead of fellow Big Ten member Purdue, based on the latter’s overall season record.

The SEC Tournament could provide some additional drama as well.  Only one of Auburn and Tennessee can win the SEC championship.  That team will likely be placed on the second line, with the other seeded one rank behind.

Only three of these teams, Nevada, Gonzaga and Rhode Island, do not play in a major conference.  Gonzaga’s score includes its “customary” two-seed bonus; it might or might not receive that again this year.  The ACC, Big East, Big Ten, and SEC conferences have roughly equal numbers of teams on this list.  The Big 12 and Pac 12 have just one each.

RPI, Conference Memberships, and Major Championships Determine Seedings

Gonzaga, Cincinnati, and Connecticut have been especially favored.

Selection Sunday for the 2018 NCAA Men’s College Basketball Tournament is just days away.  In past years I’ve published reports on how the Tournament Committee gives an advantage to teams in major and mid-major conferences and grants a seeding bonus to major champions.  Recent conversations with a friend raised the issue of whether the Committee somehow plays favorites by seeding some conferences or teams above or below others after adjusting for objective ability.  This report takes a look at such possible favoritism by the Committee.

As before I am using my database of 1,088 tournament appearances by 241 different basketball programs covering the 2001 through 2017 Tournaments.  The results appear in this companion article.

Once again we see that the Tournament Committee looks more favorably on teams from the major and mid-major conferences.  A team with an 0.60 RPI would be seeded seventh if it plays in a major conference, eighth if it hails from a mid-major, but eleventh if it comes from any other conference.

The Tournament Committee likes to stress that it looks at a team’s whole record when making seeding decisions and does not weight the end-of-season conference tournaments all that highly.  That appears to be true for all the conferences but the six “majors,” the Atlantic Coast, Big 12, Big East, Big Ten, Southeastern and Pac 12 Conferences.  Champions from the major conferences have been seeded an average of one rank higher than other major conference teams with identical RPI scores.

Does the Committee Play Favorites?

Most theories of favoritism are usually based on the assumption than the NCAA and its partners in the television industry have a clear incentive to structure the Tournament to drive ratings.  That creates a pressure to feature marquee teams like Duke or Kansas who will reliably draw a nationwide audience.  In the brutal, single-elimination format of the Tournament, the Committee has strong incentives to seed the most popular teams higher and improve their chances of survival. But is there any other evidence of favoritism when it comes to specific teams or conferences?

I searched for favoritism by comparing the actual seedings awarded a team with the seedings I predict based on RPI, conference membership, and major championships.  I began with the five teams that have appeared in every Tournament since 2000 — Duke, Gonzaga, Kansas, Michigan State and Wisconsin.  Are any of these schools’ impressive unbroken records the result of some bias over the years by the various Tournament Committees, or did these terms earn their ways to the Tournament in the gym?

For all but one of those teams I find no evidence of bias.  The outlier is, perhaps not surprisingly, Gonzaga, the only mid-major in that group of five, and the darling of college basketball fans for years.  By my reckoning, the Tournament Committee has seeded Gonzaga nearly two (1.9) ranks higher than what my model predicts for any other mid-major team with the same RPI score as Gonzaga over the years.  That means Gonzaga should have averaged an eight seed rather than the 6.2 it was awarded over the years.

I expanded my search for team favoritism to all teams with at least twelve appearances to ensure any measured effect was not just because of small sample sizes.  By that criterion only Cincinnati, with twelve appearances over the seventeen years, joined Gonzaga as a favored team.  Cincinnati, like Gonzaga, received a seeding of 6.2 on average; by my estimates they also should have averaged close to eight.

When I look for favoritism by conference, I find only a slight advantage given to the famous Atlantic Coast Conference, and significantly lower seedings given to teams from three mid-majors, the American Athletic, Colonial, and Western Athletic Conferences.  WAC teams are discounted a full seed, and teams from the other two conferences suffer a disadvantage closer to 1.5 seed ranks.

I included a test for Connecticut (with only eleven appearances it was not included above) and find that it, too, was awarded a bonus of about one full seed point.  Unlike Gonzaga and Cincinnati, though, the Tournament Committee’s confidence in UConn has been demonstrated on the floor.  UConn has won the Tournament three times in this period and averaged 2.91 wins per Conference appearance, behind only North Carolina at 3.07.  Since seeding so strongly determines a team’s overall performance, we might wonder whether Connecticut’s impressive record was the result of favoritism.  The other two advantaged teams, Gonzaga and Cincinnati, have not fared especially well on the floor.  Gonzaga has averaged only 1.41 wins per appearance, and Cincinnati just 0.83.

All across the nation college basketball teams are participating in conference tournaments.  For the smaller programs, winning the conference tournament is nearly the only way to take part in the “Big Dance,” the NCAA Mens’ Tournament. Most of these conferences receive just a single bid to the tourney, one given to the winner of the conference’s tournament.  The mid-major and major conferences often send multiple teams to the Tournament.  The conference tournament winner receives one bid with one or more others selected “at-large” based on their performance over the course of the season.  Last year, for instance, two teams from the mid-major Missouri Valley Conference went to the Tournament – Northern Iowa, the tournament champion, and Wichita State which received an at-large bid.

“Bracketologists” have debated whether conference tournament victories matter in determining seedings, or whether the Tournament Committee ignores the conference tournament results in favor of each team’s “complete resume” including the regular season.  For instance, the Committee might give little weight to a conference tournament victory by a top team like Kansas which will already be getting a high seed.  Yet conference tournaments, even among the majors, are often not won by the top teams.

The average RPI for major conference champions since 2000 is just 0.63, only a bit better than the average of 0.61 for all major conference Tournament teams. While high RPI major teams get correspondingly high seedings in the Tournament, what about those more middling teams?  Does winning a conference championship improve their seedings in the Big Dance?

To study this, I have updated my models that predict seedings based on RPI and conference membership.  I have included the data for the 2016 Tournament, again excluding the “play-in” teams ranked 65th through 68th.  I use a team’s RPI, its conference membership, and “interaction” terms that allow the effects of RPI to different across the conferences.  To those predictors I add whether the team won a conference championship separated out by type of conference.

The basic results appear fairly similar to earlier models.  Both mid-major and major conference teams are rewarded with better seedings than the remaining teams from smaller conferences with identical RPIs.

The blue line represents teams in conferences that are considered neither mid-majors nor majors.  The line displays the predicted seedings for RPI values observed for these teams since 2000.  A couple of them have RPIs below 0.5, and aren’t represented in the graph, while the highest RPI any of these teams earned was 0.62, where the blue line ends in the graph.

The major and mid-major teams generally get much better seedings at identical RPI levels once we get above 0.56 or so.  Major conference teams also have an edge over the mid-majors that widens as RPI grows.  These results parallel ones I’ve reported on in earlier postings about seeding decisions.

If we add in so-called “dummy” variables for the champions, divided similarly among the three types of conferences, we get this rather startling result:

Winners of major conference championships have an average RPI score of 0.63, while mid-major winners average 0.58.  Without taking into account their championship victories, these teams are predicted to receive seedings of 3.5 and 10.1 respectively.  However if we add the estimated championship bonuses, those seedings improve to a top seed for major champions, and an eight or nine seed for mid-major champions.

Basketball pundits generally do not give much weight to conference championships, but the NCAA Tournament Committee apparently does.