Finally, a Use for the U.S. News Law School Rankings

The U.S. News & World Report law school rankings are highly influential among people applying to law school. Nonetheless, they are widely panned among the legal community for the often-arbitrary criteria they use to distinguish between law schools. In this essay, I seek to rescue the rankings from this derision by proposing a novel use for them: picking the winner of college football games against the spread. I report the results of an experiment in which I applied this method, and briefly discuss what these exciting new findings will mean for the future of the U.S. News rankings and their reputation in the legal community.

I. Introduction

The U.S. News & World Report law school rankings are among the most panned means for determining the relative quality of the United States’ institutions of legal education. So controversial are the U.S. News rankings that there is a veritable cottage industry of law review articles debating their value.[2] It is not that the legal world is opposed to ranking law schools per se. There are a host of rankings available, and one law review even held an entire symposium on the future of law school ranking systems.[3] The opposition to the U.S. News rankings appears to stem not from the fact that they exist, but from the fact that they are bad. For instance, Professor Akhil Reed Amar has warned prospective law students to consider the U.S. News rankings “with caution and skepticism” because they “rely[] . . . on debatable and sometimes perverse weights and formulas.”[4] While describing the ranking methodology as “perverse” might seem harsh, Professor Amar has a point; the U.S. News law school ranking methodology includes such measures as the total number of books in a law school’s library.[5] The skepticism with which many in the legal community view the law school rankings tracks with criticisms of the U.S. News college rankings, which John Tierney has described as “empty, useless, [and] bad-for-you.”[6]

Loathe as I am to upset the applecart, in this essay I seek to defend the U.S. News law school rankings from the charge that they are good for nothing. The rankings are in fact incredibly useful, if not as a means of distinguishing between law schools, then at least as a way of picking college football games against the spread. In this essay, I describe how this novel use for the U.S. News law school rankings was discovered and discuss the results of an experiment in which the method was first applied. I then explore what this new method of using the rankings implies about their future. I believe that this essay will finally lay to rest any dispute over whether the U.S. News law school rankings are useful to the legal community.

II. The Method

In the fall of 2019, I participated in a college football pick ‘em with a group of friends from law school. In a pick ‘em, participants choose which team is going to win in a given set of games. The goal is simple: you want to pick more winners than the people you are competing against. However, because of the disparities between teams, simply picking the winner is sometimes too easy. For instance, the formula our commissioner used to choose the six games we would pick each week was roughly: “Big 12 shootout; Washington; Utah school (BYU or U of U); Big 10 (random); How much will Ohio State win by this week?; wild card.”[7] If all we had to do was pick the winner of say, Ohio State versus Miami of Ohio, it would have been a pretty boring competition.[8] Thus, many pick ‘ems involve additional elements, such as picking games against the spread. This was what my group did.

The spread—also known as the “point spread” or the “line”—is designed to encourage people to bet on both teams in a given game.[9] It does this by accounting for the fact that one team is expected to be better than the other. To compensate for this difference, the better team must win by a certain number of points in order to “cover” the spread. For example, imagine a game between the William & Mary Tribe and the Richmond Spiders. If the spread was William & Mary -16.5, then they would have to win by at least 17 points to cover the spread. If they won by less than 17 points, or if they were upset by Richmond, then they would fail to cover. In either of those situations, a person who picked Richmond to be the winner would get credit in the pick ‘em for correctly picking the game.

Sometimes, a spread will be a round number of points. For example, in our hypothetical matchup between William & Mary and Richmond, the spread could have been William & Mary -17. If William & Mary won by 17 points, this would have been known as a “push”—a tie between the chances that William & Mary would win by 17 points and the chances they would not. In our pick ‘em, a push counted as a loss for everyone. If the spread is accurate, it gives people essentially a 50 percent chance to pick the correct winner. Table One, below, shows the matchups, spreads, and final scores for the first week of our pick ‘em to better illustrate these concepts.

The six matchups from our first week of the pick ‘em. The victorious team is listed in the Final Score column. If a team covered the spread, the Final Score column is highlighted in green; if they did not cover, the column is highlighted in red; and if they matched the spread, it is highlighted in yellow. Historical odds and scores are available at NCAA Football Odds & Handicapping Database, Odds Shark, https://www.oddsshark.com/ncaaf/database.

At first, I had no method for picking teams; I went entirely by my gut, which sadly turned out to be fallible. That first week, my picks were Syracuse, Clemson, Nebraska, LSU, Washington, and Stanford. While I correctly picked that Clemson and LSU would win their games, I still went 1-5 on the week. The other four teams I picked lost, and Clemson did not cover. It was a rough week, especially because if I had been picking randomly, I would have expected to go 3-3. (Week One results bore out this expectation: three teams covered, one team won but did not cover, and two were upset.)

In examining the data, though, I made an important discovery. I noticed that eleven of the twelve teams in the pick ‘em matchups that week had law schools.[10] Had I made my picks based on which team was housed at a school with the higher U.S. News law school rank, I would have chosen Maryland, Texas A&M (automatically higher ranked, as Clemson does not have a law school), Colorado, Texas, Cal, and Stanford.[11] I would have gone 4-2 for the week—far better than I did, and better even than I would expect to do with a totally random distribution.

And so, my system was born. In a matchup between two teams that had law schools ranked by U.S. News, I picked the one with the better law school ranking. In the event the two schools were tied in the general U.S. News law school rankings, I picked the one with the higher peer reputation score. (Peer reputation accounts for 25 percent of a law school’s overall score in the U.S. News ranking methodology, more than any other factor.[12]) In a game between a team without an associated law school and one with a law school, I picked the team with a law school, reasoning that by default they were higher in the rankings. In the rare event our pick ‘em featured a game between two teams that were wholly unaffiliated with a law school, I picked the team handicapped by the spread, on the theory that for that game at least, they were the school with the higher “rank.”

The system seemed foolproof, because the spread eliminates many of the risks associated with an obvious mismatch. For instance, in a hypothetical matchup between Yale (U.S. News rank 1) and Alabama (U.S. News rank 25), there is little question that Alabama’s football team would dominate.[13] But because we were picking against the spread, I would not need Yale to win for them to be a winning pick. I would just need them to lose by fewer points than they would be expected to. Figuring out when a team is going to perform better than expected is difficult; it takes careful study and a bit of luck. I knew from the first week’s results, however, that I had found a sure and easy workaround. I diligently applied my system for the remaining twelve weeks of the 2019 college football season, expecting that by the end I would achieve excellent results.

III. Results and Conclusions

So, I had found a perfect system. Or, if not perfect, at least very good, and certainly enough to give me a fighting chance in the pick ‘em. Did it work? In short, no. I went 36-42 on the season, finishing tied (with two others) for ninth place in our thirteen-person league. Worse still, I was not even close to winning; I finished fourteen picks behind our champion, college football savant Matt Harney.[14] Attentive readers will note that I did not even come close to breaking even on my picks, which, if picking randomly against the spread, I would be expected to do. My perfect system led me badly astray.

What lessons can we draw from this failure? The first is that, like many lawyers and legal academics, I fell into the quantitative analysis trap. There is a natural desire to apply objective decision-making criteria, even when making an essentially subjective decision. Thus, while contracts are supposedly “interpreted so as to give effect to the mutual intent of the parties” (a subjective inquiry), we determine what the terms of a contract mean “by [using] objective rather than subjective criteria.”[15] Members of the legal community do this sort of algorithmizing all the time, without thinking. Is a student interested in doing public interest work in Washington, D.C. better off accepting a 25 percent scholarship to attend law school at the University of Pennsylvania (U.S. News rank 7), or a 75 percent scholarship to go to George Washington University’s law school (U.S. News rank 22)? Well, it depends. It depends, moreover, on a great many subjective factors that I have not given you.[16] Just as distinguishing between law schools is often subjective, so too is choosing which of two college football teams will overperform against the spread. Which team is better rested? Which team is playing on the road? How will injuries impact Team X’s defense? Team Y’s offensive coordinator has been designing some really creative plays lately, could that make a difference?

When making these kinds of difficult, subjective choices, it is tempting to turn to seemingly objective criteria, such as ranking systems, to help with our decision-making. We find patterns that do not really exist and believe that they prove things that they do not. I fell into this trap when I saw that I had gone 1-5 the first week of the pick ‘em but could have gone 4-2 if I had applied my U.S. News law school ranking method. Since 4-2 was substantially better than 1-5, and better even than the breakeven 3-3 that would be expected from a random distribution, I assumed that the method must work. But a sample size of 6 is much too small to make this kind of determination. If I had broadened the sample slightly and looked at all the games the Week One teams played in the 2018 season, I would have seen that the method went 60-71 at picking games against the spread.[17] It is tempting to use objective measures to avoid the difficulty of making subjective decisions, but it is a mistake to do so.[18]

The second lesson is to carefully evaluate what information goes into a quantitative analysis in the first place. “[C]orrelation,” we are told, “does not imply causation.”[19] There is a reason for this. Even if your sample size is large enough and your math is perfect, the results you achieve from a quantitative analysis are only as good as the data you input. As it turns out, whether a college football team is good at scoring touchdowns is unrelated to how good their law school is—an observation further supported by the fact that some of the best college football teams in the nation, such as Clemson, are housed at institutions that do not have a law school.

Indeed, the U.S. News rankings themselves reach their seemingly objective ordering based on subjective or irrelevant criteria. For instance, 25 percent of a law school’s ranking score comes from its peer reputation among certain faculty members and administrators at other law schools.[20] But as Professor Orin Kerr has pointed out, a professor’s sense of the quality of another law school is based primarily on the quality of scholarship produced by professors at that school. It is not based on what those professors are like as teachers, or how successful their students are after law school—surely more important considerations to a prospective student than a professor’s academic scholarship.[21] Other measures, such as students’ undergraduate GPAs and LSAT scores (22.5 percent of a law school’s ranking score) provide only a weak indication of how successful students will be once they are actually in law school.[22] Some measures, such as the number of volumes in a school’s law library, are laughably irrelevant to the quality of education provided by a modern law school.

Thus, my method was doubly flawed, because it applied seemingly “objective” criteria that were in fact anything but. Relying exclusively on the U.S. News law school rankings to pick the winners of college football games is as wrongheaded as relying on measures of law school quality based on peer reputation, undergraduate GPA of attendees, and total number of volumes in the law library.[23] Just as those metrics tell you nothing about the quality of education a law student will receive, how much debt she will have when she graduates, and whether she will be able to land her dream job, the U.S. News law school rankings tell you nothing about who is going to win a college football game. And indeed, the rankings can even be harmful. For instance, in the law school context, the rankings’ measure of expenditures per student encourages extravagant spending at law schools, even as tuition and student debt levels skyrocket.[24] The resulting high levels of student debt drive many young lawyers away from their preferred careers in public interest work and toward the higher-paid realm of private practice.[25] One can only imagine the damage that could be caused if the U.S. News rankings were erroneously applied to the tens of billions of dollars’ worth of college football bets placed every year.[26]

Perhaps the rankings really are useless, after all.

Copyright © 2020. Author holds copyrights. Matt Strauser and Kyle Wood share the blame for the fact that this piece exists. Sammy Block and Natasha Geiling deserve credit for the fact that it contains stronger arguments and far fewer typos, confusing sentences, and citation mistakes than it used to. All remaining errors—and all views—are strictly my own. DOI: https://doi.org/10.15779/Z38SQ8QJ2D.

2. See Theodore P. Seto, Understanding the U.S. News Law School Rankings, 60 SMU L. Rev. 493, 493-94 & n.1 (2007). ↑

3. See Paul L. Caron & Rafael Gely, Dead Poets and Academic Progenitors: The Next Generation of Law School Rankings, 81 Ind. L.J. 1, 2 (2006); Karen Sloan, Bored With the US News Law School Rankings? Check Out These Alternatives, law.com (Aug. 29, 2019), https://www.law.com/2019/08/29/bored-with-the-u-s-news-law-school-rankings-check-out-these-alternatives/. ↑

4. Akhil Reed Amar, Be Skeptical of Law-School and Other College Rankings. Very Skeptical, L.A. Times (Mar. 19, 2019), https://www.latimes.com/opinion/op-ed/la-oe-amar-law-school-rankings-20190319-story.html. ↑

5. See Robert Morse et al., Methodology: 2020 Best Law School Rankings, U.S. News & World Rep. (Mar. 28, 2019, 2:04 PM), https://www.usnews.com/education/best-graduate-schools/articles/law-schools-methodology. ↑

6. John Tierney, Your Annual Reminder to Ignore the U.S. News & World Report College Rankings, Atlantic (Sept. 10, 2013), https://www.theatlantic.com/education/archive/2013/09/your-annual-reminder-to-ignore-the-em-us-news-world-report-em-college-rankings/279103/. ↑

7. Scott McMurtry, Text Message to College Football Pick ‘Em Group Chat (Sep. 25, 2019, 9:43 PM) (on file with the author). This text was a joke, but these teams did come up frequently in the pick ‘em. This was because our commissioner sought to choose televised games that many of us were likely to be watching anyways. ↑

8. Ohio State won that particular game 76-5. See Gene Ross, After Wonky Start, Ohio State Obliterates Miami (OH) 76-5 to End Non-Conference Slate, Land Grant Holy Land (Sept. 21, 2019), https://www.landgrantholyland.com/2019/9/21/20877361/ohio-state-miami-ohio-september-21-2019-recap. ↑

9. A helpful explanation of what the spread is and how it works is available at What Does “Spread” Mean in Sports Betting?, Athlon Sports (Aug. 2, 2018), https://athlonsports.com/nfl/what-does-spread-mean-sports-betting. ↑

10. Of the 259 Division I college football teams 133, or 51.4 percent, have an associated law school. Compare Full List of Division 1 Football Teams: Find the Right Team for Your Athletic and Academic Goals, Next C. Student Athlete, https://www.ncsasports.org/football/division-1-colleges, with Best Law Schools Ranked in 2019, U.S. News & World Rep., https://www.usnews.com/best-graduate-schools/top-law-schools/law-rankings. It is unsurprising that many law schools are housed at institutions that have successful football programs; many of the best Division I football programs are at large state universities that also have public law schools. Notably, some law schools are at institutions that have Division II or Division III football programs. In contrast, some law schools are at universities that have no football program; presumably their students actually study on Saturday afternoons in the fall. ↑

11. All law school ranks are drawn from Best Law Schools Ranked in 2019, supra note 10. ↑

12. See Morse et al., supra note 4. ↑

13. For reasons unimportant to this paper, a football matchup between Yale and Alabama is so unlikely that websites that simulate games between different teams do not even have the option to run a simulation of a Yale-Alabama game. See, e.g., College Football Game Simulator, NCAA Game Simulator, http://www.ncaagamesim.com/FB_GameSimulator.asp (a simulator that does not contain Yale as one of the teams to run potential matchups against). Nonetheless, proxies for team quality can still substantiate my claim. For instance, 56 former members of the Crimson Tide were on NFL active rosters in 2019, whereas only two former Yale players could say the same. See Anthony Chiusano & Wayne Cavadi, Every FCS Player On 2019 NFL Rosters, NCAA (Feb. 1, 2020), https://www.ncaa.com/news/football/article/2019-09-07/2019-nfl-rosters-fcs-players-league; Spencer Parlier, Colleges Most Represented On 2019 NFL Rosters, NCAA (Jan. 17, 2020), https://www.ncaa.com/news/football/article/2019-09-03/colleges-most-represented-2019-nfl-rosters. ↑

14. I categorically deny the allegations that I was forced to write this essay as punishment for finishing so low in the pick ‘em or for losing a side bet with Harney. ↑

15. Tribeca Cos., LLC v. First Am. Title Ins. Co., 239 Cal. App. 4th 1088, 1111 (2015). ↑

16. Where is the student from? Does she have a family, and are they able to move with her? What sort of public interest work does she want to do? Can she afford to go to a more expensive law school? Does she want to? Is she interested in going to law school part-time? Did she feel more comfortable at one school or the other? Does she know people in Philadelphia? In D.C.? ↑

17. These data, too, are available at NCAA Football Odds & Handicapping Database, supra note 9. ↑

18. Cf. D.H. Kaye, Statistics for Lawyers and Law for Statistics, 89 Mich. L. Rev. 1520, 1542 (1991) (“[E]ven well-crafted probability arguments about the significance of evidence should be suspect—not to the extent of dismissing them out of hand, but to ensure that these arguments are taken as a guide to thought and not a substitute for it.”). ↑

19. E.g., Ryder v. Union Pac. R.R. Co., 945 F.3d 194, 202 (5th Cir. 2019). ↑

20. See Morse et al., supra note 4. ↑

21. E.g., Orin Kerr (@OrinKerr), Twitter (Jan. 6, 2020, 1:29 AM), https://twitter.com/OrinKerr/status/1214071469279514624. ↑

22. See David A. Thomas, Predicting Law School Academic Performance from LSAT Scores and Undergraduate Grade Point Averages: A Comprehensive Study, 35 Ariz. St. L.J. 1007, 1019-1020 (2003). ↑

23. Notably, Joe Burrow, who won the Heisman Trophy by a record margin in 2019, had to transfer to LSU because his “peer reputation” at Ohio State was lower than that of Dwayne Haskins; Burrow left the Buckeyes when it became clear he would not be their starting quarterback. Billy Witz, Joe Burrow Wins the 2019 Heisman Trophy in a Record-Breaking Landslide, N.Y. Times (Dec. 16, 2019), https://www.nytimes.com/2019/12/14/sports/heisman-trophy-joe-burrow.html. ↑

24. See Morse et al., supra note 4; see also Paul Caron, U.S. News Rankings to Continue Using Law School Expenditures, Despite ABA’s Decision to Stop Collecting Data, TaxProf Blog (Oct. 11, 2013), https://taxprof.typepad.com/taxprof_blog/2013/10/us-news-to-.html. ↑

25. See John Bliss, From Idealists to Hired Guns? An Empirical Analysis of “Public Interest Drift” in Law School, 55 U.C. Davis L. Rev. 1973, 2005 (2018); Jack Karp, Are Law Schools Helping Students Who Want To Help Others?, Law360 (Mar. 31, 2019, 8:02 PM), https://www.law360.com/articles/1143092. ↑

26. See Elaine S. Povich, Show Me the Money: Sports Betting Off and Running, Pew (Sept. 10, 2018), https://www.pewtrusts.org/en/research-and-analysis/blogs/stateline/2018/09/10/show-me-the-money-sports-betting-off-and-running. ↑