Naavi.org

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

In the previous articles, we have discussed the different provisions of the regulations for AI adoption and usage in Judiciary released by the Supreme Court as a draft for public comments. Thee comments are required to be submitted before 20th June and there is very little time for all to study the proposition in depth. We have tried to  decode the provisions so that others can quickly assimilate and formulate their views.

In this article in this series we discuss the General Principles of AI Governance which is a very important aspect of the  regulation. These are comparable to the AI Governance principles and  covered under the principles of DGPSI-AI.

The  provisions of Chapter II are reproduced here for  reference.

CHAPTER II: GENERAL PRINCIPLES TO GOVERN ADOPTION, DEPLOYMENT AND USE OF AI SYSTEMS IN COURTS

4. Human primacy and judicial independence.—

(1) The use of Artificial Intelligence in Court processes shall at all times remain strictly subservient to human judgment and judicial authority.

(2) Every AI System shall function solely in an assistive capacity and shall not supplant or compromise the independent exercise of judicial authority by a duly appointed judicial officer.

(3) The ultimate authority to determine matters of law, fact and justice shall vest exclusively in the judicial officers of the competent jurisdiction.

5. Rule of law.—

(1) The adoption and use of AI Systems in Courts shall be consistent with the provisions of the Constitution and any other law for the time being in force including  the principles of natural justice and shall not be used in any manner that could undermine due process, the right to a fair trial, equality before law, or access to justice.

(2) The Bangalore Principles of Judicial Conduct (adopted in 2002) shall, in addition to these regulations, continue to govern the conduct of judicial officers in all matters, including those involving the use of AI.

6. Fairness and non-discrimination.—

(1) The AI Systems used in Court processes shall be designed, trained and deployed in a manner that promotes fairness and avoids discrimination.

(2) No AI System shall be deployed that perpetuates, amplifies, or introduces bias on the grounds of race, religion, caste, sex, gender, disability, language, economic status, or any other ground prohibited under the Constitution or any law for the time being in force and special care shall be taken to protect the rights and interests of vulnerable groups including women, children, persons with disabilities, marginalised and  minority  communities,  and  persons  from  economically  and  socially disadvantaged backgrounds.

7. Transparency and explainability.—

(1) Every AI System used in Court processes shall meet high standards of transparency and explainability.

(2) The functioning, data inputs and decision logic of any AI System used in a Court process shall be capable of being understood and, where appropriate, be explained to judicial officers, the parties concerned and the public.

(3) The deployment of AI Systems that are opaque or incapable of explanation shall be subject to heightened scrutiny and shall be restricted in high-risk applications affecting personal liberty or any lawful right of a person.

8. Accountability.—

(1) Accountability for all decisions made by any officer with the assistance of AI shall rest exclusively upon such officer and it shall not be permissible to invoke the outputs of an AI System, the opaqueness of a Black Box system, or the occurrence of hallucination, as a ground for avoiding accountability for a palpably incorrect, illegal, or harmful decision.

(2) The Appropriate Authority shall ensure that clear and documented lines of accountability are established and maintained for the operation of every AI System or AI Tool in a Court.

(3) Where any AI-generated output or information is used in any Court, it shall be treated as advisory in nature and reasonable care shall be taken to verify the accuracy of such output before the same is utilised:

Provided that the officer responsible and accountable for using such AI Tool may, for reasons to be recorded in writing, dispense with the requirement of verification:

Provided   further   that   such   AI   tools   used   exclusively   for   administrative (non-adjudicatory) functions and certified by the AI Secretariat to have established reliability, shall be deemed to satisfy verification requirements on a class basis, without requiring prior verification.

9. Auditability and continuous oversight.––

(1) Every AI System in use in Court processes shall be subject to continuous monitoring and periodic technical, legal and ethical audits   throughout   their   lifecycle   and  adequate  mechanisms  shall  be established to detect, document and address errors, malfunctions and biases.

(2) Audit findings shall be recorded and disclosed in accordance with these regulations,  and  shall  decide  upon  the  continued  deployment of  AI  Systems  in Courts.

10. Data protection and privacy.––

(1) The processing of personal data through AI Systems shall be governed by the principles of purpose limitation, data minimisation and data privacy by design in accordance with the provisions of the Digital Personal Data Protection Act, 2023 (22 of 2023) or any other law for the time being in force, and sensitive judicial data shall be accorded the highest standard of protection.

(2) The right to privacy shall be ensured in all AI related operations of Courts.

12. Purpose limitation.–– AI Systems shall be deployed and used solely for specific purposes for which they have been approved by the Appropriate Authority and any use of an AI System beyond the scope of its approved purpose shall require a separate and specific approval of the Appropriate Authority, which shall record reasons therefor.
13. Proportionality.––

(1) The use of AI in any Court process shall be proportionate to the nature, complexity and risk profile of the relevant task.

(2) Applications involving higher levels of risk to personal liberty or any lawful right of a person, or the integrity of judicial outcomes shall be subject to correspondingly heightened safeguards including mandatory Human-in-the-Loop requirements and independent oversight.

13. Inclusivity and accessibility.––

(1) AI Systems deployed in Courts shall be designed and operated to promote inclusivity and expand equitable access to justice.

(2) Specific attention shall be given to ensuring that the deployment of AI does not create or widen digital divides and that the benefits of AI-assisted judicial services are extended fairly to all stakeholders including those from rural, economically disadvantaged, or linguistically diverse communities.

14. Data integrity.––

(1) AI Systems used in Court processes shall be trained and operated on the basis of data that is accurate, representative, lawfully obtained and to the extent feasible, free from discriminatory bias.

(2)   The   deployment   of   AI   Systems,   trained   on   unlawfully   collected   or demonstrably biased datasets, shall be prohibited.

16. Cyber security.–– The confidentiality, integrity and availability of Court data, processed through or stored in AI Systems, shall be protected by robust, layered and continuously updated technical and organisational security measures, commensurate with the sensitivity of the data and the nature of the Court process.
17. Presumption in favour of responsible AI adoption.––

(1) Every Court shall actively seek opportunities to deploy AI Systems or AI Tools that demonstrably improve access to justice, reduce delays, or enhance administrative efficiency, and unless proved otherwise, the presumption shall be in favour of responsible adoption of AI in Court processes:

Provided that no AI System or AI Tool used for the purpose of assistance in Court processes, shall replace humans as far as decision-making is concerned, and shall not be deployed for dispute-outcome prediction.

(2) The restriction on, or refusal to permit, the use of any AI System or AI Tool, shall be for reasons to be recorded in writing, and such restriction shall be reasonable and to such extent so as to address the concern identified.

17. Innovation over Restraint—

(1) The adoption of Artificial Intelligence in Court processes shall be pursued, in a responsible manner, as a catalyst for impactful innovation in the justice delivery system; and the exploration, development and integration of AI Systems  and  AI  Tools  that  demonstrably further  the  goals of judicial efficiency and easy access to justice shall be actively encouraged.

(2) All innovation under sub-regulation (1) shall be carried out with due regard to the other general principles set out in this Chapter, so as to maximise the overall benefit of AI adoption while eliminating or minimising potential harm; and, all other things being equal, an approach that prefers active and responsible adoption over restraint shall be encouraged.

What strikes the eye most in this regulation is the categoric statement in Section 4 that use of AI in judicial process will at all times remain strictly subservient to human judgement and judicial authority.

This came as a surprising but mortal blow to those who were quoting Albania Government which had appointed an AI as a minister. It is a direct response to the recent article in Economic Times which recommended that companies should adopt AI for DPDPA compliance so that the army of lawyers required otherwise could be eliminated.  Section 4 has simply shut down such arguments once and for all.

Otherwise the principle of fairness, non discrimination, transparency, explain ability, accountability are covered under different sections.

Auditability and continuous oversight has been recommended along with Data Protection principles such as data minimisation, purpose limitation, proportionality.

The regulations prescribe that the systems must be  trained on data that is accurate, representative and “lawfully” obtained.

Cyber Security is also indicated as a principle to be ensured..

Together  there is emphasis on responsible adoption of AI though section 17 says “Innovation over Restraint”

When we look back on all the detailed regulation mandated here, it is obvious that “Restraint” over rides “Innovation” and the Court has in fact been more rigid than necessary. It appears that the authors have worked under a panic that Judicial system will be over whelmed by the AI usage and in due course may eliminate large part of the adjudication.

Given  the delays in the Court, innovators can still push for AI usage in Mediation process and also in “With recourse arbitration”. This should not affect these regulations being implemented in the formal Judicial system.

I request readers to not only  read all the articles but also take time to listen to the Audio overviews and Video overviews available in the link in the menu “Naavi Academy”.   These will clarify the articles. However please remember that the articles are written directly by Naavi while the overviews have been created by my AI assistant. He could have slipped in a few places but I have not found any material error. There could be some exaggerations and praise for Naavi which was not prompted. Kindly ignore.

Naavi

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

Continuing our discussions on the AI regulations in Judiciary proposed by the Supreme Court let us explore section 49,50 an 51 in Chapter VIII of the regulations which prescribes the Capacity Building, Training and Best Practices.

The requirements of these sections are reproduced below for immediate reference.

CHAPTER VIII: CAPACITY BUILDING, TRAINING, AND BEST PRACTICES

49. (1) All Judges, advocates and Court staff, who are required to use or interact with AI Systems in the course of their duties, shall receive regular, structured training on the technical, legal and ethical dimensions of AI, as may be relevant to their functions.

(2)  Training  on  use  of  AI  in  Court  processes  shall be accessible to all such persons, including those in district Courts, and shall be offered in a manner that accounts for linguistic diversity.

(3) The training programmes shall be developed by the AI Secretariat in consultation with relevant domain experts and judicial training institutions, and shall address, at a minimum––

(a)  the functioning, capabilities and limitations of AI Systems in use in Court processes;

(b)   the identification and mitigation of AI bias, hallucinations and technical errors;

(c)   the legal and ethical framework governing AI in the judicial context, including the rights of litigants and the obligations of judicial officers under these regulations;

(d)  data protection principles, cyber security awareness and the handling of sensitive judicial data; and

(e)  the correct procedures for reporting AI Incidents, raising concerns and utilising grievance redressal mechanisms.

51. Repository of best practices on AI Incidents.––The Appropriate Authority shall maintain a living repository of best practices, case studies, lessons drawn from AI Incidents and guidance notes, which shall be regularly updated, curated and made available to all  relevant  Courts  and  judicial  personnel,  so  as  to  serve  as  an institutional memory to ensure continuity of competence, despite changes in staff or composition.
52. Review of training programmes.––

(1) The adequacy and effectiveness of training programmes shall be reviewed at least once in every two years by the AI Committee in consultation with the AI Secretariat, and such modifications as are warranted by practical experience or technological developments shall be implemented.

(2) Every High Court shall devise an annual training calendar in coordination with judicial training institutions and the Apex Body, to ensure the sustained and updated competence of all judicial and administrative personnel in matters relating to AI.

According to Section 49(3), the AI secretariat is required to develop training programs in consultation with relevant domain experts and  judicial training institutions to train all the Judges and other persons who are required to use AI in the system. The training needs to cover

(a)  the functioning, capabilities and limitations of AI Systems in use in Court processes;

(b)   the identification and mitigation of AI bias, hallucinations and technical errors;

(c)   the legal and ethical framework governing AI in the judicial context, including the rights of litigants and the obligations of judicial officers under these regulations;

(d)  data protection principles, cyber security awareness and the handling of sensitive judicial data; and

(e)  the correct procedures for reporting AI Incidents, raising concerns and utilising grievance redressal mechanisms.

Further the regulation also requires that the Grievance Redressal system will also be required to be set up at all the places to handle the grievances related to harm caused by AI usage. This team also needs to be separately trained since there is no specified appeal mechanism specified.

Under section 53, the aggrieved persons will also be open to seeking redressal of grievance though  any other competent court. This means that the DPB-TDSAT-SC route for grievance redressal in case of personal data related disputes may still be available.

Naavi

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

In prescribing the AI usage regulations for the judiciary, the authors of the regulation encountered the need to recognize the overlapping legal provisions of ITA 2000 and DPDPA 2023 compliance as an added obligation.  This has been indicated in Section 48 of the regulations.

Section 54 of the regulations explicitly state that the provisions of these regulations shall be in addition to and not in derogation of the provisions of the Information Technology Act, 2000 (21 of 2000) or the Digital Personal Data Protection Act, 2023 (22 of 2023) or any other law governing Courts, data protection, and AI for the time being in force and in the event of any inconsistency between these regulations and the provisions of any other law on the subject, the provisions of such law, as may be applicable, shall prevail.

However it leaves a statement  of ambiguity that where these regulations afford a higher degree of protection to any person than administrative instructions or directions issued by any authority, the provisions of these regulations shall prevail over such instructions or directions to the extent of any inconsistency.

The text of Chapter VII on Data Protection and Cyber Security is reproduced here for immediate reference.

CHAPTER VII: DATA PROTECTION AND CYBER SECURITY

48. Application of relevant laws.––All AI Systems deployed in Court processes shall comply with the provisions of the Digital Personal Data Protection Act, 2023 (22 of 2023), the Information Technology Act, 2000 (21 of 2000) and the applicable rules and regulations framed thereunder, and any other law governing the protection of personal data and judicial information for the time being in force.
49. Sensitive judicial data.––

(1) Sensitive judicial data*  shall not be transferred to any External System without the express written authorisation of the Appropriate Authority.

(2) All transfers of sensitive judicial data shall be subject to appropriate technical and contractual safeguards designed to prevent unauthorised access, disclosure, alteration, or misuse.

(3) The principle of data minimisation shall be applied in the selection and deployment of AI Systems and AI Systems that achieve the relevant operational objective while requiring lesser processing of personal data shall be preferred over those requiring greater data processing, particularly in Court processes involving sensitive personal information or matters affecting personal liberty.

(4) Anonymisation shall be applied to personal data to the extent technically feasible without compromising the utility of the data for the intended purpose, before it is used for the training, testing, or refinement of any AI System.

(5) Every AI System in use in Court processes shall be subject to regular cybersecurity audits at intervals not exceeding one year, or at such shorter intervals as the AI Secretariat may determine and the outcomes of cybersecurity audits shall be reported to the Appropriate Authority and recorded in the AI Register.

To recognize the impact of DPDPA 2023,  the regulators found the necessity for defining a new term “Sensitive Judicial Data”  as including  any personal identifiable information of parties, witnesses, or legal representatives and any information processed in connection with a Court process, the unauthorised disclosure of which may cause harm; The definition of “harm” , in relation to AI Incidents, includes any kind of physical or financial damage, or damage to the reputation or rights of any individual, institution, or infrastructure.

DPDPA has not defined “Sensitive” data and only defined “Significant Data Fiduciary” as a fiduciary who handles the Sensitive data. If all data in the judicial system is “Sensitive”, Judicial authorities will become Significant Data Fiduciaries. Use of AI further reinforces this status.

Complete exemption of DPDPA 2023 is available only under Section 17(2) of the DPDPA 2023 and it does not include the Courts, unless they are “notified” as “instrumentalities of state” and the purpose being maintenance of “Public Order. Exemption under Sec 17(1) of the DPDPA 2023 is restricted to Chapter II of DPDPA 2023 (Establishing of Legal Basis), Chapter III (Rights of Data Principals), Section 16 (Cross Border transfer)  excluding obligation under Section 8(5) of DPDPA related to being responsible for reasonable security practices.

Under Section 48 of the regulations Supreme Court has adopted the principle of data minimisation  and anonymisation where relevant.

The non personal data processed by AI will fall under the ITA 2000 provisions.

It is suggested that MeitY declares the Court systems as exempted under Section 17(2) to avoid any perceived conflicts.

Naavi

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

The draft guidelines has prescribed an elaborate system for oversight, adudits and incident management under Chapter V, which we shall discuss here.

The details of sections 35 to 45 defining the suggested regulations are reproduced below for ready reference.

CHAPTER V: OVERSIGHT, AUDITS AND INCIDENT MANAGEMENT

35. Oversight and accountability.––

(1) The Appropriate Authority shall, before approving any AI System for use in Court processes, require the submission of a comprehensive Technical and Ethical Impact Assessment.

(2) The Appropriate Authority shall prescribe a standard format for the Technical and Ethical Impact Assessment within six months from the date of commencement of these regulations.

(3) The Technical and Ethical Impact Assessment shall evaluate at a minimum, the––

(a) purpose, architecture and functioning of the AI System;

(b)  nature, source, quality and representativeness of its training data;

(c)    risks  of  bias,  error,  hallucination and  misuse in the relevant judicial context;

(d) cyber security vulnerabilities and the data protection measures in place;

(e) mechanisms for explainability and compliance with Human-in-the-Loop requirements; and

(f)  procedures for redressal of any harm and for incident reporting.

36. Controlled Environment Testing.—

(1) The Appropriate Authority may, in suitable cases and prior to the full-scale deployment of any AI System or AI Tool in Court processes, direct  that  the  AI  System  or  AI  Tool  be  evaluated  through Controlled Environment Testing established under the supervision of the AI Secretariat.

(2) The Controlled Environment Testing shall be undertaken on a time-limited and clearly defined basis with documented parameters of evaluation, including accuracy, reliability, fairness, explainability, cyber security and compatibility with existing Court processes, and the outcomes of such testing shall be placed before the Appropriate Authority for consideration prior to any decision on deployment, mainstreaming or scaling of the AI System or AI Tool.

(3) During the period of Controlled Environment Testing, the activities carried out within such environment shall not affect, interfere with, or compromise the integrity, security or functioning of the primary operational systems or networks of any Court, nor shall the outputs of such testing be used in any actual adjudicatory or administrative decision in a Court process.

37. AI Register.–– Each Court shall maintain an AI Register, in such form and with such particulars as the Appropriate Authority may prescribe, documenting––

(a) all AI Systems approved for use in Court processes;

(b)  the purposes and scope of approved use for each system;

(c) the identity of the AI Service Provider and, where applicable, the vendor;

(d)  the date of approval and any conditions attached thereto;

(e) the records of Technical and Ethical Impact Assessments conducted;

(f)  the records of audits conducted and their outcomes; and

(g)  the AI Incidents recorded in connection with each system.

(2)   The dissemination of AI Register on the official website of the Court for public access shall be subject to data protection, confidentiality and cyber security.

38. Audits.––

(1) All Court AI Systems and AI Tools shall undergo periodic technical, legal and ethical audits at intervals not exceeding one year from the date of approval or the date of the preceding audit, or at such shorter intervals as the Appropriate Authority may direct.

(2) The audits shall be conducted ‘in-house’, and under no circumstances the source code, algorithms, datasets, or other architectural information shall be shared with any third party or private entity for an audit outside the Court premises.

(3) The audit reports shall be submitted to the Appropriate Authority and shall be recorded in the AI Register.

39. AI Incident Database.––

(1) Every AI Secretariat shall maintain an AI Incident Database for the systematic recording of all AI Incidents, including their type, cause, manner of occurrence, consequences and the remedial measures taken.

(2) Where an AI Incident is reported in any High Court, the AI Secretariat of that High Court shall communicate the relevant findings and learnings to the AI Secretariats of other High Courts and to the Apex Body, so that corrective measures may be adopted across jurisdictions.

(3)   Any malfunction, error, or bias in a Court AI Tool with potential legal consequences shall be reported immediately to the AI Secretariat by the officer responsible for supervising such AI Tool.

(4) The AI Secretariat shall, on receipt of a report referred to in sub-regulation (3), initiate remedial measures without delay and report the matter to the AI Committee.

40. Discretion in supervising AI Systems.––

The nominated officer responsible for supervising any AI System shall retain full discretion to accept, modify, or reject any AI-generated recommendation or output within the matters falling under his charge, and shall exercise discretion with independent professional judgment.

41 Review of AI Systems already in use.––

AI Systems already in use in Courts at the time of commencement of these Regulations shall be reviewed by the AI Secretariat for compliance within a period of one year from the date of such commencement and the Appropriate Authority shall determine the appropriate course of action in respect of any system found to be non-compliant.

42. Emergency and fall-back protocol.––

(1) Every High Court shall, in consultation  with  the  AI  Secretariat,  establish  and  maintain  an  emergency and fall-back protocol specifying the procedures to be followed in the event of a failure, malfunction, or unavailability of any AI System or AI Tool in Court.

(2) The emergency and fall-back protocol shall ensure the continuity of essential Court processes through manual or alternative means and shall be tested at periodic intervals as determined by the AI Secretariat.

(3) Where a Court AI Tool fails or is suspended under these regulations, the AI Secretariat shall activate the applicable fall-back protocol and notify the AI Committee within twenty-four hours.

43. Transparency and disclosure.––

(1) The Courts shall, where an AI Tool materially assists in any aspect of case management, document analysis, or judicial administration that  may  affect  the  conduct  of  their  proceedings, ensure that the parties are informed in a timely and accessible manner.

(2) The obligation under sub-regulation (1) shall apply in all permitted uses of AI specified in regulation 19.

(3) Where an AI Tool is used by any party or his legal representative in the preparation or submission of any document, pleading, or evidence, the AI-assisted character of such material shall be disclosed to the Court at the time of submission by way of a duly executed declaration or certificate in the format prescribed under Annexure I and any Court-initiated AI use in any Court process shall be declared in accordance with the format provided in Annexure II.

(4)  The Court shall have the authority to require disclosure of the AI System used, the nature and extent of AI assistance provided, and the steps taken to verify the accuracy of  any  AI-generated content,  in  respect  of  any AI-assisted submission placed before it.

(5) Any person using Synthetic Data or Synthetic Information in any judicial proceeding shall be required to disclose such use to the Court, in such form and manner as the Appropriate Authority may prescribe.

(6) In the event that any document, pleading, or evidence submitted to a Court is found to be fabricated, false, misleading, or inaccurate by reason of its AI-generated character, the person submitting the same shall bear full responsibility therefor and shall not be entitled to rely upon the character of the AI output as a defence. The Court may pass such orders as it deems fit against the responsible person.

44. AI Content Verification Authority.––

The Appropriate Authority shall constitute a dedicated institutional authority, to be designated as the AI Content Verification Authority, charged  with  the  oversight,  operation,  and  continuous updation of verification standards, tools and protocols applicable to GenAI-generated content in Court Process.

45. Annual Transparency Report.––

Every High Court, Tribunal and Commission referred to in these regulations shall submit an Annual Transparency Report on AI adoption within its jurisdiction, summarising the AI Systems in use, outcomes of audits, AI Incidents recorded and measures taken for compliance with and improvement of these regulations, to the Apex Body and cause the same to  be published on its official website.

In this chapter also there is an attempted decentralization of oversight involving individual AI committees. This may create duplication and also conflicting decisions by different Courts. Had there been one grand committee of CJIs of all high  Courts, it could be entrusted with the centralised decision making in most of the cases including development of the format for Technical and Ethical Impact Assessment as envisaged under Section 35.

This impact assessment  needs to cover evaluation in the minimum of purpose etc as provided under Section 35(3) all of which can be considered as covered under a DGPSI-AI audit of the algorithm. This short list of requirements of evaluation of an algorithm is an elaboration of the requirements of compliance under DGPSI-AI-Deployer’s implementation specifications. DGPSI-AI covers Explainability, Risk Assessment, Human handler contact, documentation of guardrails, configuration instructions, third party audit, use of AI agents in developemnt etc. DGPSI-AI goes one step further on requiring adoption of a  “Kill Switch” particularly in case of “Critical Risks”.

The SCAIF (Supreme Court AI framework) also suggests a controlled environment testing while DGPSI-AI requires documentation of the testing at the developer’s end.

SCAIF suggests periodical audits  conducted in-house which is normally the responsibility of a DPO in a private sector organization. DGPSI-AI suggests external audit both at the developer’s end and the deployer’s  end.

SCAIF suggests maintenance of an AI register and AI incident data base at all AI secretariats. This also is amenable to a centralized maintenance for better management.

For  transparency, when AI is used in pleadings, a suitable declaration will be prescribed.

An “AI Content Verification Authority” has been envisaged  as a dedicated institutional authority charged  with  the  oversight,  operation,  and  continuous updation of verification standards, tools and protocols applicable to GenAI-generated content in Court Process.

This proposition under Section 44  is a highly ambitious proposal the full dimensions of which might not have been fully factored into the suggestion. It could mean setting up of a separate Forensic Lab for the task of verifying every AI content used in the Court process which may be practically beyond the scope of this regulation.

Under Section 42, a BCP process to address emergency and fall back is suggested. This is another activity which should be better centralized rather than expecting each High Court to develop.

Section 41 prescribes  a review of AI systems already in use which requires a separate audit for which a time of 1 year has been prescribed.

In summary we can suggest that these oversight functions are better managed as a Central Expert team rather than being duplicated at every High Court level.  If persisted,, most High Courts will ignore the directions al together and the objective of this regulation may not be achieved.

Naavi

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

In continuation of our discussions on the proposed regulations for use of AI in judiciary released by Supreme Court for public Comments, we shall today look at Chapter III of the regulations related to Permissible and Prohibited  uses.

Under this chapter there is a clear definition of what kind of AI is permissible and what is not. While DGPSI-AI and other frameworks rely on a  Risk Assessment and classification of Risk as significant or not, since this regulation is sector specific, the regulatory draft directly defines what is permitted and what is not.

The following is the text of the regulation.

CHAPTER III PERMISSIBLE AND PROHIBITED USES

18. Appropriate Authority to identify use of AI in Courts.–– The Appropriate Authority shall determine, from time to time, the areas of Court processes in which AI Systems may be utilised, in accordance with the provisions of these regulations.
19. Permissible uses of AI.–– (1) Subject to prior approval in writing by the Appropriate Authority and to the supervision and verification of officers nominated for that purpose, AI Systems may be used for the following purposes, which are illustrative and not exhaustive:–

(a)   case management (including identification of defects in new filings), cause list preparation, hearing scheduling and docket prioritisation;

(b)  automated transcription of court proceedings, subject to mandatory review and certification of accuracy by a Designated Officer;

(c)   translation of judgments, orders, pleadings and other legal documents, subject to human verification of accuracy and fidelity to the original;

(d)   legal research, precedent retrieval, citation verification and document summarisation

(e) administrative functions including case filing assistance, defect scrutiny, record management and judicial resource allocation;

(f)  conversational AI Assistants and guided chatbots to assist litigants and other stakeholders in accessing Court services and understanding procedural requirements, subject to human oversight of their functioning;

(g)    accessibility services including text-to-speech, speech-to-text, Braille translation and visual assistance tools, for persons with disabilities or language barriers;

(h) document authenticity verification and fraud detection in administrative processes, subject to mandatory human review of all outputs before any action is taken;

(i)  anonymisation of judgments, orders and Court records for publication in the public domain;

(j) analytical tools for judicial administration, court performance assessment and backlog monitoring and management; and

(k) auto-generation of prescribed formats, notices and summons with metadata merge including automated preparation of administrative documents.

(2) Any use of AI not enumerated under sub-regulation (1), or not otherwise specifically approved, shall require the prior written approval of the Appropriate Authority, which shall record reasons for the grant or refusal of such approval.

20. Prohibited uses of AI.–– (1) The following uses of AI are strictly prohibited in all Court processes. These prohibitions are absolute and non-derogable, and shall not be subject to relaxation or modification by any authority under these Regulations, including under the power conferred by Regulation 19(1)––

(a)  no personal data of any person shall be used to train, test, or refine any AI System without the prior approval of the Appropriate Authority and, where applicable, in compliance with applicable data protection law;

(b)   no judicial outcome (including any judgment, order, or finding of fact or law) shall be reached through Algorithmic Decision-Making alone or solely on the basis of AI-generated information, data, or analysis and the human judicial authority shall be the determinative authority in all adjudicative decisions;

(c)   no AI System shall perform the function of adjudication or sentencing in any  matter  without  mandatory  Human-in-the-Loop  and  any  output  of  an  AI System in relation to adjudicative or sentencing questions shall be treated as advisory only and shall be subject to independent judicial evaluation;

(d)   no AI System shall be used for Risk Scoring for any purpose in Court processes, including the assessment of flight risk, prediction of recidivism, evaluation of bail eligibility, or determination of the credibility of parties or witnesses;

(e)   no undisclosed, opaque, or unexplainable AI System shall be used in any Court process that may materially affect the lawful rights or personal liberty of any party;

f)  no AI System shall be used to predict, profile, or infer the future conduct or behaviour of parties, accused persons, witnesses, or legal representatives in any Court process;

(g)  no AI System shall be used for the surveillance or continuous monitoring of judicial officers, advocates, litigants, or any other person within or in connection with Court premises or Court processes, except as may be specifically authorised by applicable law for the time being in force;

(h)  no AI-generated output shall be submitted to a Court as an independent source of evidence without full and transparent disclosure of its AI-generated character; and

(i)   no AI System shall be used in any manner that may compromise the confidentiality  of  judicial  deliberations  or  the  independence  of  the  judicial decision-making process.

(2) The prohibitions referred to in sub- regulation (1)  shall be absolute and shall not be subject to relaxation or modification by any authority.

21. Remedial measures by AI Committee.–– Every violation of any prohibition specified in regulation 20 shall be reported forthwith to the AI Secretariat, which shall be placed before the AI Committee, and the AI Committee shall, after due enquiry, direct such remedial measures, including the suspension of the relevant AI System, as it deems appropriate.

The section is very explicit and there is no cause for any misunderstanding. There are specific permitted uses such as the administrative functions . But even for these, prior approval is required to be taken by vendors from an  appropriate authority which will be the AI committee in the Supreme Court or respective High Courts.

It is possible that multiple software can got approved from different committees. This could have been avoided by making the approval of software from the centralized Technical Committee.

Under the prohibited uses, the use of personal data for training of the algorithm is prohibited and so also is any automated judicial decision making.  This is welcome and DGPSI-AI is already in sync with this thought.

It is preferable if clarification is also made available that violations if any are reported to the central committee so that the vendor or the system can be removed from the  system in all other Courts where it might have been earlier in use.

Naavi

(P.S: This is in continuation of our discussion on the suggested AI regulations for Judiciary by the supreme Court)

The proposed AI regulation in Judiciary has envisaged a strong Governance mechanism for AI usage.

Firstly a full time permanent apex body is being constituted  to regulate and promote innovation, integration, governance, oversight, standard-setting and policy development on Artificial Intelligence in judiciary. Though section 22 suggests that this will take care of bot “Innovation” and “Regulation” we can envisage that  it will be  mainly a regulatory body permitting innovation within limits.

The apex body will consist of 2 Judges of Supreme Court nominated  by CJI one of whom will serve as the Chairman ex-officio.

Two more Chief justices of High Courts, would also be nominated by the CJI.  An officer not below the rank of a Joint Secretary from MeitY would be  an ex-officio member. One expert in Cyber Security and one expert in Finance would also be in the apex body.  One (or more) advocates of standing and one member from an  institution of national importance or any institution of repute would also be nominated by the CJI. Lastly the professor in charge of AI in National Judicial Academy Bhopal would also be in the committee. Additionally the Apex body can co-opt such other experts from Research institutions or academic bodies as it deems necessary. either on a case to case basis or standing basis with the permission of the CJI.

Thus the Apex body which is referred to as the “Appropriate Authority”  would be a 9+ member committee with wide representation of judges, advocates and technical experts.  The CJI will determine the terms of appointment of non-ex-officio members.

One of the functions of the Committee  is to ensure that no AI system, whether autonomous AI agent or static predictive model or any AI Tool, is in violation of any of the provisions of the Constitution or any law for the time being in force and are in compliance with the operational safeguards;

The Committee will also ensure compliance of these regulations, liaise with MeitY, NIC etc, hear and recommend actions  when grievances are raised or any other functions assigned.

It is obvious that CJI would be the driving force of the committee and will head the committee as the ex-officio chairman.

The apex  body shall constitute  five sub committees namely

(a) Judicial Committee; (b) Technical Committee;

(c) Committee on Infrastructure and Finance;

(d) Case and Data Management Committee; and

(e) Cyber Security Committee

Supreme Court will also form a “Center of Research and excellence on Artificial Intelligence” (CoRE-AI) ), as an integrated body, having such number of experts in the fields of technology, law, and academia, as may be determined by the Apex Body, to provide research and legal compliance-related support to the Apex Body. This center will have experts who are Judges, lawyers, technical experts, academicians in the fields of AI and law, Senior and Distinguished Fellows from Think-tanks, post-doctoral researchers and representatives of the National Judicial Academy.

Additionally the Supreme Court and every High Court shall constitute AI Committees to oversee, regulate and facilitate the responsible adoption and governance of AI within its jurisdiction.

This AI Committee will consist of three judges and a senior member of the  AI secretariat and such other members of the  AI secretariat as the AI may invite and will monitor. Each such committee would be supported by an AI Secretariat.

The AI Secretariat shall consist of such a number of officers and experts in judicial administration, technology, data science and law, as may be determined by the AI Committee.

While the importance  given to AI Governance can be appreciated, the number of Committees, Sub Committees and Secretariats may create a huge structure with increased cost of administration.

It appears that the system is being set up on a totally self dependent manner and hence all functions are being regulated directly under the CJI with multiple  committees and employees. Officials of MeitY, NIC and  CERT-In would be involved in relevant committees but will function under the CJI’s supervision.

A rough estimate indicates that the entire paraphernalia consists of nearly 1000 individuals and a total operating cost of around Rs 250-300 crores per annum along with capital expenditure that may exceed another 300 crores.

There is a danger of excessive bureaucratization, over governance and  duplication of functions which may  delay the decision making process and create dysfunctional cross currents.

When 5  crore cases are pending, whether deploying such funds not for handling cases but for the administration requires re-thinking.

It appears that there is scope for substantial reduction of costs by better organization and using deputed officers from other Government organizations where there could be  excess manpower capacities particularly since AI will be used for routine administrative duties in every department of the Government.

This plan appears to have been developed by the National Judicial Academy Bhopal and needs to be re-visited with the help of  a review committee consisting of IISc, one IIM, one IIT, one NLSUI and an expert in organizational structuring from Private Sector and an expert Chartered Accountant.  Without such pruning, there is a possibility that the Central Government and CAG may be uncomfortable.

Naavi