The effective end of the Supreme Court’s term on Friday touched off what has become an annual tradition: hot takes summarizing the justices’ work over the preceding nine months based upon data aggregated from the justices’ decisions. These accounts typically focus on surprising-sounding results (50% of the decisions were unanimous!) in service of pushing back against the most obvious summary of the current court: that it is sharply divided between the six justices appointed by Republican presidents and the three justices appointed by Democrats. You can spin the data however you want, but the reality is actually simple. The conservative majority is pushing American law decisively to the right.
Statisticians call this phenomenon the “tyranny of averages” — the fact that averaging a data set tells us nothing about the size, distribution or skew of the data.
Statisticians call this phenomenon the “tyranny of averages” — the fact that averaging a data set tells us nothing about the size, distribution or skew of the data. But these kinds of “judge the Supreme Court by its data” assessments are even worse than just ordinary statistical errors.
First, they fail to account for the Supreme Court’s own role in choosing the cases it decides — so that the data isn’t random to begin with. Second, they ignore all of the Supreme Court’s significant rulings in other cases — those that don’t receive full briefings and arguments. Finally, even within the carefully cultivated subset of cases on which these claims generally focus, these commentaries both miscount the divisions and treat as equal disputes that bear no resemblance to each other. It’s not that this data is completely irrelevant, but anyone relying upon it should take it with a very substantial grain of salt.
Let’s start with the court’s docket. With one tiny exception (which accounted for exactly one case during the justices’ current term), the court chooses each and every one of its cases (and, even within those cases, which specific issues it wants to decide). This docket control, which is entirely a modern phenomenon, means the justices are pre-selecting the cases they decide — including technical disputes on which they may be likely to agree (or, at least, not disagree along conventional ideological lines). Thus, from the get-go, the entire data set on which too many commentators rely is biased toward the justices’ own behavior.
Thus, statistical claims about the court tend to neglect the thousands of other rulings the Supreme Court hands down every term — on what has become known as the “shadow docket.” These rulings are unsigned and almost always unexplained, and they run the gamut from agreeing or refusing to take up an appeal to agreeing or refusing to block a lower-court ruling while the appeal runs its course.
Many of these rulings are relatively insignificant, but some are just as important as — if not more important than — cases that receive plenary consideration.
Consider the April ruling that preserved nationwide access to mifepristone or the December ruling that left in place a controversial Covid-related border control policy. Indeed, there have been 35 shadow docket orders from the court since October from which at least one justice publicly dissented — including six from which all three of the Democratic appointees registered their opposition. (That’s in contrast with a total of seven argued cases in which all three dissented.) Shouldn’t that data figure in any putatively comprehensive summary, too?
As we see statistical claims about #SCOTUS's current Term, remember that, though most analyses focus on the 56 cases with signed opinions, that's a small subset—cases hand-picked by the Justices for resolution.
— Steve Vladeck (@steve_vladeck) July 1, 2023
Among 1000s of *other* rulings, another 35 have had public dissents: pic.twitter.com/RmPl5KNSbP
Finally, even within the skewed subset on which these statistical claims rest, there are serious false equivalency issues. It’s not just that a 237-page ruling invalidating race-based affirmative action policies at virtually every college and university in the country has a far greater impact (and is far more important in almost all respects) than a 16-page technical resolution of a question of bankruptcy procedure; it’s that the way we count votes doesn’t necessarily reflect the true divisions among the justices.
Consider Sackett v. EPA — a major decision in which the court dramatically curtailed the federal government’s ability to prevent pollution of wetlands. Raw data treats that ruling as unanimous — because all nine justices agreed that the lower court applied the wrong test. But with regard to the rule going forward, the justices divided 5-4 — with Justice Brett Kavanaugh breaking from the other conservatives and writing for himself and the Democratic appointees in a sharp separate opinion that embraced a broader reading of the statute. No statistical summary of the court’s work treats that decision as 5-4 — even though, for all intents and purposes, it was.
There’s no question that there were a handful of rulings this term in which the more visibly “conservative” position did not win. There’s no question that there were a handful of rulings this term in which the more visibly “conservative” position didn’t win. In Haaland v. Brackeen, a 7-2 majority rejected a challenge to the Indian Child Welfare Act. In Moore v. Harper, six justices rejected the broadest version of the so-called independent state legislature theory — which would have given state legislatures carte blanche to run roughshod over state courts and state constitutions when it comes to federal elections. In United States v. Texas, eight justices held that Texas and Louisiana lacked standing to challenge the Biden administration’s immigration enforcement priorities. And in perhaps the biggest surprise of the term, a 5-4 majority ruled in Allen v. Milligan that Alabama’s congressional district maps violate the Voting Rights Act. But except for the Alabama redistricting decision, each of those rulings was less of a “victory” for progressives than meets the eye. The most important claims in Brackeen were rejected not on their merits, but because the plaintiffs weren’t the right ones to bring them. Ditto United States v. Texas — which didn’t uphold the Biden administration policy but merely said Texas and Louisiana couldn’t challenge it. And in Moore, even as the justices rejected the most alarming version of the independent state legislature doctrine, they actually embraced a weaker form of it. This leaves the door open, in the future, for state legislatures to violate state constitutions in federal elections. Again, just looking at the vote counts in these cases doesn’t come close to telling their full stories — or the overall story of the term. In contrast, the “conservative” victories were enormous. Gutting race-based affirmative action in higher education, recognizing for the first time that certain business owners have a First Amendment right to refuse to provide services to members of groups whose behavior they oppose, tossing President Joe Biden’s student loan debt relief program in a ruling that will make it easier for anyone going forward to challenge a dizzying array of federal policies, and the list goes on. In the end, assessments of the Supreme Court’s work during its current term should privilege what the court has actually done (and not done) over how its efforts are superficially (and misleadingly) quantified through incomplete, inaccurate and ultimately unrevealing data. And when that’s the focus of our study, what becomes clear is just how powerful the six-justice conservative majority is — and just how significant its implications are for the current and future trajectory of American law.
Steve Vladeck
Steve Vladeck is a professor of law at Georgetown Law, whose teaching and research focus on federal jurisdiction, constitutional law and national security law. He is co-editor-in-chief of the Just Security blog (@just_security) and co-host of "The National Security Law Podcast" (@nslpodcast). 
