AI in Criminal Justice 2026: Algorithmic Bias, Sentencing Tools, and Legal Challenges

In 2016, a Wisconsin court used COMPAS — a commercial risk assessment algorithm — to help determine Eric Loomis's sentence. The Wisconsin Supreme Court ruled that using COMPAS did not violate due process, but the decision sparked debate: Loomis was unable to examine the algorithm's internal logic, because it was proprietary. By 2026, this proprietary-algorithm-in-sentencing problem has migrated from academic debate to active litigation and legislation across multiple jurisdictions.

Artificial intelligence is now embedded in criminal justice systems across the United States and Europe — in pretrial risk assessment, predictive policing, facial recognition for suspect identification, and parole decisions. The legal challenges to these uses are intensifying in 2026, driven by documented bias concerns, due process arguments, and new regulatory frameworks that specifically restrict AI in high-stakes criminal justice applications.

COMPAS — Correctional Offender Management Profiling for Alternative Sanctions. A commercial recidivism risk assessment algorithm developed by Equivant (formerly Northpointe). Used by courts in several US states to generate risk scores that influence pretrial detention, sentencing, and parole decisions. Its internal methodology is proprietary.

Risk Assessment Instrument (RAI) — Any structured tool that generates a score predicting a defendant's likelihood of reoffending, failing to appear for trial, or other outcomes. RAIs can be actuarial (statistical, based on group characteristics) or algorithmic. Their use in criminal justice is controversial because they may embed historical bias in their training data.

The Algorithmic Bias Problem in Criminal Justice AI

The central legal and ethical challenge for AI in criminal justice is bias — the systematic overestimation or underestimation of risk for certain demographic groups, often correlated with race, because of biased historical data used to train the models.

A 2016 ProPublica investigation of COMPAS found that Black defendants were nearly twice as likely to be falsely flagged as high risk of reoffending compared to white defendants, while white defendants were more likely to be falsely flagged as low risk. Equivant disputed the methodology, and academic debate about the appropriate fairness metric continues — but the underlying concern has been replicated in studies of multiple RAIs across different jurisdictions.

The technical difficulty is that standard measures of statistical fairness can be mathematically incompatible. Calibration (a score of X means X% of this group actually reoffends), false positive rate parity (equal rates of over-prediction across groups), and false negative rate parity cannot all be satisfied simultaneously when base rates of the predicted outcome differ across groups. This is a mathematical theorem — not a software bug — which means there is no "fair" algorithm in the sense of satisfying all fairness criteria at once. Any RAI that uses group-correlated features embeds trade-offs between different conceptions of fairness.

Due Process Challenges to Algorithmic Decision-Making

US constitutional challenges to algorithmic sentencing and pretrial detention tools have primarily been mounted under the Due Process Clause of the Fourteenth Amendment. The key argument: if a defendant cannot examine the algorithm's methodology and challenge the basis of a risk score that affects their liberty, they have been denied meaningful procedural due process.

State v. Loomis (Wisconsin Supreme Court, 2016) is the leading US case. The court upheld use of COMPAS at sentencing, but on limited grounds: the judge explicitly stated the sentence was based on independent factors and COMPAS was one of many inputs. The court acknowledged the due process concern but held it was not dispositive because COMPAS information had been disclosed and the defendant could rebut the score through independent evidence.

Federal courts have been more sceptical in later cases, particularly for pretrial detention — where the deprivation of liberty before conviction raises heightened procedural concerns. In 2024, the Third Circuit raised significant questions about algorithmic risk tools' compatibility with procedural due process when used as primary detention determinants.

The right to confront an algorithm — to examine its source code, training data, and validation — is particularly contested when the tool is proprietary. Courts have split on whether defendant rights include disclosure of proprietary algorithmic logic; some jurisdictions have moved to require disclosure as a condition of the tool's admissibility.

The EU AI Act's Restrictions on Criminal Justice AI

The EU AI Act takes the most explicit regulatory position on AI in criminal justice. Several applications are classified as prohibited or high-risk, with significant compliance implications for EU member states' justice systems:

Prohibited (Article 5): AI systems used to assess the risk of criminal offences by natural persons based on profiling or assessment of their personality traits — essentially, predictive policing based on individual profiling — are prohibited. AI systems that make individual risk assessments solely from physical characteristics are also prohibited.

High-Risk (Annex III): AI systems intended to be used for criminal analytics, risk assessment in criminal proceedings (including recidivism prediction), and AI used for emotion recognition in law enforcement contexts are classified as high-risk. These require conformity assessment, transparency documentation, and human oversight before deployment by public authorities.

Member states deploying AI in their criminal justice systems must adapt existing tools to comply with the AI Act's requirements by 2027 (the general high-risk provisions become applicable for public authorities in August 2027). This has significant procurement implications — criminal justice AI vendors must provide AI Act-compliant documentation and support conformity assessment processes.

Bias Auditing: What Legal Requirements Are Emerging

In the US, New York City Local Law 144 (effective 2023) requires employers using AI tools in hiring decisions to conduct annual bias audits by independent auditors and publish summary results. This is the first US law mandating bias auditing for AI, and it is expected to be a template for similar requirements applied to AI in other high-stakes domains including criminal justice.

Several US states are considering legislation requiring bias audits for criminal justice AI. Illinois, New Jersey, and California have all seen relevant legislative proposals. At the federal level, the AI Accountability Act (pending in Congress) would require risk assessments and audits for high-impact AI systems, with criminal justice AI explicitly in scope.

What constitutes an adequate bias audit for a criminal justice RAI is itself contested. A defensible audit should: test the tool's predictions against actual outcomes in the jurisdiction, disaggregate results by race and other demographic categories, report false positive and false negative rates separately for each group, and identify any disproportionate impacts. The audit should be conducted by an independent party with access to the tool's methodology — not just its outputs.

Frequently Asked Questions

Is it legal to use AI risk assessment tools in sentencing in the US?

Yes, in many US states — but it is legally contested. The Wisconsin Supreme Court upheld COMPAS use at sentencing in Loomis (2016) on narrow grounds. Federal courts have raised stronger due process concerns when algorithmic tools are primary determinants of pretrial detention. The legal landscape is evolving, with increasing requirements for transparency, disclosure of methodology, and the opportunity for defendants to challenge risk scores.

Does the EU AI Act prohibit all AI use in criminal justice?

No, but it imposes significant restrictions. AI systems for individual criminal profiling based on personality traits are prohibited. Most criminal justice AI tools (recidivism assessment, criminal analytics, emotion recognition in law enforcement) are classified as high-risk — requiring conformity assessment, transparency documentation, and human oversight. EU member states must comply by August 2027.

What is algorithmic bias and why is it a problem in criminal justice?

Algorithmic bias is the systematic over- or under-prediction of outcomes for certain demographic groups, often correlated with race, because the AI was trained on historically biased data. In criminal justice, this can mean a risk assessment tool assigns higher recidivism scores to Black defendants than similarly situated white defendants — resulting in longer sentences or pretrial detention. Standard fairness metrics are mathematically incompatible, meaning any RAI that uses group-correlated features involves trade-offs between different conceptions of fairness.

What is a bias audit and when is it required?

A bias audit is an independent assessment of an AI system's differential impact across demographic groups — testing whether the tool predicts outcomes differently for different races, genders, or other groups. New York City Local Law 144 requires annual bias audits for hiring AI. Several US states are considering similar requirements for criminal justice AI. The EU AI Act effectively requires bias testing as part of high-risk AI conformity assessment.

This article is published by an independent news publication for informational purposes only and does not represent or claim affiliation with any government body, international organization, or official authority.