Concerns about algorithmic decision-making in the justice system are not theoretical. In 2016, ProPublica published an investigation showing that such systems were already influencing outcomes across American courts. A tool called COMPAS was being used to predict which defendants were likely to reoffend.
What it found was damning: Black defendants who did not go on to reoffend were wrongly classified as high risk at nearly twice the rate of their white counterparts. The algorithm’s errors fell asymmetrically—and they fell on people who had done nothing to warrant them.
Northpointe, the company that built COMPAS, disputed the finding. Its statisticians argued that the disparity in false-positive rates was explained by differing baseline recidivism rates between Black and white defendants, not by algorithmic bias. The debate has not been settled. What is not disputed is that the tool was operational, embedded in bail and sentencing decisions, and nobody outside the courtroom knew it existed until journalists started asking questions.
The Dutch courts encountered a version of the same problem. In February 2020, a court in The Hague struck down SyRI—a government risk-scoring system that trawled public databases to flag individuals as likely welfare fraudsters. The system had been deployed almost exclusively in low-income, minority neighbourhoods, assigning risk scores that individuals could not see, let alone challenge. The court ruled it violated the right to privacy under the European Convention on Human Rights. The case, brought by a coalition of civil society organisations, was among the first anywhere to invalidate an algorithmic scoring system on human rights grounds.
The Supreme Court of India, on June 3, 2026, moved to pre-empt exactly this kind of tool. With little public fanfare and almost no mainstream coverage outside specialist legal media, a 57-regulation draft framework has been released. It governs the use of artificial intelligence across every court in the country, from district tribunals and statutory commissions up to the Supreme Court itself. Planted at its centre is a flat prohibition on risk scoring.
The draft Regulations for Use of Artificial Intelligence in Courts, 2026, is not a policy statement. It is a granular regulatory architecture: definitions, prohibited uses, institutional mechanisms, audit requirements, grievance redressal procedures, and a chain of oversight bodies from a Supreme Court Apex Body down to AI Secretariats at each High Court.
Its animating philosophy—AI as an assistant, not a decision-maker—is stated plainly in Regulation 4: AI in court processes “shall at all times remain strictly subservient to human judgement and judicial authority”. Every AI system “shall function solely in an assistive capacity”. The ultimate authority over law, fact, and justice “shall vest exclusively in the judicial officers of the competent jurisdiction.”
The framework is built around this principle, not merely decorated with it.
The document is notable for its ambition of scope. It applies to every AI deployment in any “judicial, adjudicatory or administrative function” across the entire Indian court system. It envisions AI being used for transcription, translation, legal research, case scheduling, accessibility services for persons with disabilities, and litigant-facing chatbots.
But more importantly, it lists what AI cannot do. That list is where the document reveals its character.
Regulation 20 is titled “Prohibited uses of AI”. Its opening sentence carries deliberate precision: “The following uses of AI are strictly prohibited in all Court processes. These prohibitions are absolute and non-derogable.”
The word choice matters here. Non-derogable, in Indian constitutional law, is a term usually reserved for rights that cannot be suspended even during emergencies. Its application here signals that the drafters knew exactly what they were doing.
The prohibition that most directly echoes COMPAS is Regulation 20(1)(d): no AI system may be used for “Risk Scoring for any purpose in Court processes, including the assessment of flight risk, prediction of recidivism, evaluation of bail eligibility, or determination of the credibility of parties or witnesses.” The regulations define Risk Scoring precisely — using an AI to assign a score that purports to estimate the probability of a person committing a future offence, reoffending, or failing to appear in court.
Adjacent prohibitions reinforce this. No AI may be used to “predict, profile, or infer the future conduct or behaviour of parties, accused persons, witnesses or legal representatives.” No undisclosed or unexplainable AI system may be used in any process that affects personal liberty. No judicial outcome, no judgement, no order, and no finding of fact may be reached through algorithmic decision-making alone.
A prohibition beyond administrative reach
What makes these provisions structurally unusual is what cannot override them. Regulation 56, which grants the Apex Body the power to “relax or modify” provisions, explicitly carves out Regulation 20. The Chief Justice of India, who appoints the Apex Body, cannot soften these bans. No authority under this framework can. The drafters appear to have anticipated institutional drift — future administrators deciding, incrementally, that an exception was reasonable — and foreclosed it.
This is not how most regulatory frameworks are written. It reflects a judgement that some prohibitions must be placed beyond the reach of the people administering them.
This structural choice will face its first real test not in a courtroom but in a procurement meeting. The prohibition on risk scoring is airtight on paper, but a well-resourced vendor could attempt to rebrand a risk tool as a “scheduling aid” or “case prioritisation system.” The specificity of the definitions in Regulation 20 makes this harder than it would be under vaguer language, but not impossible.
The framework designates no independent technical body to make the initial classification call. The determination of whether a tool constitutes risk scoring falls to the High Court AI Secretariat — an administrative body that will evaluate the vendor’s own characterisation of what the tool does. There is no adversarial mechanism, no dedicated function whose job it is to ask whether the scheduling aid is a risk score in disguise.
The consultation itself may sharpen some of these edges. The Bar Council of India is already engaged: earlier this year, the Supreme Court asked it to constitute an expert committee after a trial court filed AI-generated fake case citations in a judgment. Legal technology companies, civil society organisations, and academic institutions with an interest in AI governance have until June 20, 2026, to submit written comments. Whether that engagement produces substantive amendments to the draft — particularly around the independence of the AI Secretariat’s classification function — remains to be seen.
The system only the well-resourced can run
The oversight mechanism is itself a resource-intensive product that only well-resourced courts can operate. The Apex Body, AI Secretariat, multiple committees, Incident Database, transparency reports — these are not passive guardrails. They require staff, technical expertise, sustained attention, and a budget.
Indian courts carry a burden that makes “actively seeking AI opportunities” sound both necessary and close to quixotic. As of mid-2026, approximately 52 million cases are pending across the judicial system (National Judicial Data Grid). District courts in many parts of the country function with crumbling infrastructure, understaffed registries, and judges carrying dockets that would be unmanageable under any technological regime. The regulations envision each High Court establishing a functional AI Secretariat capable of approving AI tools, supervising their deployment, conducting periodic audits, and managing incident reporting.
A court that cannot maintain a basic case-management system cannot run an AI audit regime. The oversight structure is accessible only to the courts least likely to misuse AI. The under-resourced district courts and overburdened subordinate judiciary — the most vulnerable to bad AI deployment — are precisely those the oversight mechanism will reach last, if at all.
The regulations acknowledge that resourcing is the variable. The Committee on Infrastructure and Finance is tasked with coordinating with the Central Government to assess budgetary requirements. But coordination and assessments are not appropriations. The framework mandates the architecture; it does not fund it.
The grievance mechanism requires an affected litigant to know that AI was used in a process affecting them. That knowledge depends on disclosure. In practice, it presupposes a litigant who is informed, literate in the regulatory architecture, represented, and with time to access the AI Register and file a grievance with the High Court AI Committee. None of this describes the average undertrial in a subordinate court who does not know what processed their bail application. The right exists; the conditions for exercising it largely do not.
Regulatory ambition without institutional capacity becomes, over time, a document that exists and is not enforced — which may be worse than no document at all, because it provides the appearance of governance while practice runs beneath it unaddressed.
India has chosen prevention over correction. The COMPAS algorithm, despite the ProPublica exposé, continues to be used, because once a system is embedded in a workflow, its removal requires more friction than its adoption. The question is whether prevention holds during implementation: whether the oversight bodies are genuinely independent, whether the prohibitions survive contact with vendors who know how to rebrand, and whether the annual transparency reports are published and evaluated rather than filed and forgotten.
These are the ordinary risks of regulation. But a framework that bans predictive criminal tools and places that ban beyond administrative reach, even partially implemented, changes something about Indian justice. Not a revolution. A quiet shift in who gets to decide what the algorithm says about you. Whether that shift holds, or quietly dissolves into the same institutional inertia it was designed to resist, is the question India’s courts will spend the next decade answering.
Simrah Haindaday is a Mumbai-based law student and legal researcher.
The draft Regulations for Use of Artificial Intelligence in Courts, 2026, were published by the Supreme Court of India’s AI Committee on June 3, 2026. Stakeholders may submit comments to office.regcc@sci.nic.in by June 20, 2026.
Also Read | Digital monoculture is remaking feminist solidarity
Also Read | When the algorithm writes the postcolonial story
