AI Case Study
Illinois Institutes's Chicago-Kent College of Law researchers predict outcomes of US Supreme Court decisions with 70% accuracy using machine learning
Researchers build a machine learning model to predict whether US Supreme Court decisions will affirm or reject the status quo, both on a case-level and individual judge level, with the model achieving approximately 70% accuracy.
Industry
Professional Services
Legal
Project Overview
"While many questions could be evaluated, the Court’s decisions offer at least two discrete prediction questions: 1) will the Court as a whole affirm or reverse the status quo judgment and 2) will each individual Justice vote to affirm or reverse the status quo judgment? ...we construct a model to predict the voting behavior of the Court and its Justices in a generalized, out-of-sample context. We predict nearly two centuries of historical decisions (1816-2015) and compare our results against multiple null (baseline) models. Using only data available prior to decision, our model outperforms all baseline models at both the Justice and Court observation level under both parametric and non-parametric tests. At its core, our effort relies upon a statistical ensemble method used to transform a set of weak learners into a strong learner". (PLOS One)
Reported Results
"Casting predictions over nearly two centuries, our model achieves 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently over the past century, we outperform an in-sample optimized null model by nearly 5%. Among other things, we believe such improvements in modeling should be of interest to court observers, litigants, citizens and markets. This performance is consistent with, and improves on, the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not just a single term." (PLOS One)
Technology
The "model is based on the random forest method" (PLOS One)
Function
R And D
Core Research And Development
Background
From their article in PLOS One, the researchers state: "predicting the behavior of the United States Supreme Court is one of the great pastimes for legal and political observers. Will the Justices vote based on the political preferences of the President who appointed them or form a coalition along other dimensions? Will the Court counter expectations with an unexpected ruling? Despite the multitude of pundits and vast human effort devoted to the task, the quality of the resulting predictions and the underlying models supporting most forecasts is unclear. Not only are these models not backtested historically, but many are difficult to formalize or reproduce at all. When models are formalized, they are typically assessed ex post to infer causes, rather than used ex ante to predict future cases."
Benefits
Data
"Supreme Court Database (SCDB) and some derived features generated through feature engineering". (PLOS One)