AI Regulation

The Algorithm is Banned, The Bias Remains

0 Shares
0
0
0
0

The Supreme Court of India on June 3rd, 2026 released a draft regulatory framework of use of AI in courts. No Apex Court in any part of the world has published such guidelines. It is not just a policy note, but binding regulations open to public scrutiny until 17th July. The document named, Regulations for Use of Artificial Intelligence (AI) in Courts, 2026 is the first of its kind for Indian judiciary and deserves more than mere news coverage. Its constitutionality must be examined.

This piece of writing offers that scrutiny.

What does the draft of Supreme Court AI regulation actually do?

The base structure of the draft, prepared under the Supreme Court committee rests on the main principle of human primacy. The regulations emphasize that AI remains an assistive tool. It cannot determine questions of law, mediate disputes and author judgements. The concerned judicial officer will be held accountable for any decision assisted by AI. Within this framework, a clear line is drawn between administrative and judicial use of AI. There are absolute prohibitions, such as barring AI from predicting recidivism, evaluating bail eligibility and determining the credibility of witnesses. AI tools cannot be used to influence the decisions of human judges. The message is clear – when the personal liberty of a person is at stake, the decisive power must solely lie with the human judge.

The prohibition is constitutionally viable but raises a serious concern that the draft fails to address.

Why the bail prohibition is deemed to be correct?

The global intersection of bail decisions and AI has raised serious concerns. COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) has been deployed in United States courts to generate “risk scores” that directly influence the judicial decision making. An investigation conducted by ProPublica in 2016 revealed that Black defendants were at twice the risk of white defendants. The mechanisms and workings of the software could not be challenged. In State v. Loomis (Wisconsin Supreme Court, 2016), the court allowed the use of COMPAS but held that risk scores cannot be solely relied upon.

India’s constitutional framework makes the AI driven bail prediction issue suspicious. Right to privacy was held as a fundamental right under Article 21 (Right to Life and Personal Liberty) in the nine-judge bench in K.S. Puttaswamy v. Union of India judgement of 2017. A risk score assessed by AI is an outcome of data collected from past defendants and personal information such as criminal history and socio-economic background used to predict behavior. No individual consent is taken before processing this deeply personal information with AI tools. There is no cross-examination for decisions undertaken by AI.

In Satender Kumar Antil v. CBI (2022), the courts reemphasised the principle that “Bail is the rule and Jail is the exception”. The courts were directed to apply Arnesh Kumar guidelines to the accused in custody. A risk score predicted by AI would certainly do the opposite, and detention would be more likely than release.

An unfinished argument

The regulation aims to protect human decision making in bail discretion from the algorithmic bias given that the decision making that it seeks to protect is constitutionally deficient in itself. Justice V.R. Krishna Iyer gave the most quoted opinion on bail jurisprudence in Gudikanti Narasimhulu v. Public Prosecutor (1978) that says “The issue of bail is one of liberty, justice, public safety and burden of the public treasury, all of which insist that a developed jurisprudence of bail is integral to a socially sensitised judicial process.” Even 48 years later, it remains an unfulfilled demand. Bail orders across courts are refused without any justification and reasoning.

The Supreme Court of India acknowledged this and published binding timelines for judicial procedures citing violation of fundamental rights due to delayed orders from courts. The court in Condition Being Imposed While Granting Bail (2026) declared the bail prerequisites that require the accused persons to clean police stations as degrading and unrecognised in the eyes of the law.  A judicial discretion in granting bail without considering constitutional grounds is in itself a violation of Article 21 (Right to Life and Personal Liberty).

The draft AI regulations propose to prevent unfairness of bail by insisting on human decisions but it does not suffice when accompanied with issues of disproportionality and unreasonableness. Barring AI from decision-making does not resolve the issue of violation of the rights of the accused. More often, violation comes from within the system and not from any algorithm.

The Ultimate Feedback Requirements

The window for feedback closes on July 17th. Three additions must necessarily be made in order to complete the draft structurally.

Firstly, the regulations must provide reasoning while issuing a bail refusal. If the premise is the value of protection of constitutional and fundamental rights of accused persons, every bail order issued by a human judge must consist of reasoning. Without this, the regulation merely halts the process and does not protect the outcomes.

Secondly, the draft authorises AI assisted verification of citations. The Supreme Court has itself taken into account multiple issues of fabricated citations generated by AI and the court has now opted to deploy AI mechanisms for verification of citations. This must include disclosure mechanisms. The concerned judicial officer is held accountable and should answer when things go wrong. A mechanism to correct these errors manually in real time would address this gap.

Thirdly, the phrase “human foresight” used multiple times in the draft must be defined precisely to avoid ambiguities. It should at least include a team of judicial officers who review every outcome by AI before it shapes any decision in the public domain. Additionally, each party should have the right to know whether AI assisted in decision making. An undefined oversight is equal to it being non-existent.

Conclusion

The Regulations for Use of AI in Courts, 2026 represents a first of its kind step, no court around the world has taken yet. But it provides only a half safeguard regulating AI and leaving the human discretions unchecked. The regulation can be tested not only through extent of barring the machine to make decisions but its potential to compel human judges to provide reasoning for their decisions. Justice can neither be reduced to a risk score nor be left alone to discretion. The question here is not who denied the bail, but on what grounds and reasoning the bail was denied. Until the regulations answer this, the draft remains a protector of the process rather than the accused.

This is a guest post submitted by Ms Harshita Chahar, who is a student of law at NLU Sonipat.

Leave a Reply
You May Also Like