The draft guidelines has prescribed an elaborate system for oversight, adudits and incident management under Chapter V, which we shall discuss here.
The details of sections 35 to 45 defining the suggested regulations are reproduced below for ready reference.
CHAPTER V: OVERSIGHT, AUDITS AND INCIDENT MANAGEMENT
- Oversight and accountability.––
(1) The Appropriate Authority shall, before approving any AI System for use in Court processes, require the submission of a comprehensive Technical and Ethical Impact Assessment.
(2) The Appropriate Authority shall prescribe a standard format for the Technical and Ethical Impact Assessment within six months from the date of commencement of these regulations.
(3) The Technical and Ethical Impact Assessment shall evaluate at a minimum, the––
(a) purpose, architecture and functioning of the AI System;
(b) nature, source, quality and representativeness of its training data;
(c) risks of bias, error, hallucination and misuse in the relevant judicial context;
(d) cyber security vulnerabilities and the data protection measures in place;
(e) mechanisms for explainability and compliance with Human-in-the-Loop requirements; and
(f) procedures for redressal of any harm and for incident reporting.
- Controlled Environment Testing.—
(1) The Appropriate Authority may, in suitable cases and prior to the full-scale deployment of any AI System or AI Tool in Court processes, direct that the AI System or AI Tool be evaluated through Controlled Environment Testing established under the supervision of the AI Secretariat.
(2) The Controlled Environment Testing shall be undertaken on a time-limited and clearly defined basis with documented parameters of evaluation, including accuracy, reliability, fairness, explainability, cyber security and compatibility with existing Court processes, and the outcomes of such testing shall be placed before the Appropriate Authority for consideration prior to any decision on deployment, mainstreaming or scaling of the AI System or AI Tool.
(3) During the period of Controlled Environment Testing, the activities carried out within such environment shall not affect, interfere with, or compromise the integrity, security or functioning of the primary operational systems or networks of any Court, nor shall the outputs of such testing be used in any actual adjudicatory or administrative decision in a Court process.
- AI Register.–– Each Court shall maintain an AI Register, in such form and with such particulars as the Appropriate Authority may prescribe, documenting––
(a) all AI Systems approved for use in Court processes;
(b) the purposes and scope of approved use for each system;
(c) the identity of the AI Service Provider and, where applicable, the vendor;
(d) the date of approval and any conditions attached thereto;
(e) the records of Technical and Ethical Impact Assessments conducted;
(f) the records of audits conducted and their outcomes; and
(g) the AI Incidents recorded in connection with each system.
(2) The dissemination of AI Register on the official website of the Court for public access shall be subject to data protection, confidentiality and cyber security.
- Audits.––
(1) All Court AI Systems and AI Tools shall undergo periodic technical, legal and ethical audits at intervals not exceeding one year from the date of approval or the date of the preceding audit, or at such shorter intervals as the Appropriate Authority may direct.
(2) The audits shall be conducted ‘in-house’, and under no circumstances the source code, algorithms, datasets, or other architectural information shall be shared with any third party or private entity for an audit outside the Court premises.
(3) The audit reports shall be submitted to the Appropriate Authority and shall be recorded in the AI Register.
- AI Incident Database.––
(1) Every AI Secretariat shall maintain an AI Incident Database for the systematic recording of all AI Incidents, including their type, cause, manner of occurrence, consequences and the remedial measures taken.
(2) Where an AI Incident is reported in any High Court, the AI Secretariat of that High Court shall communicate the relevant findings and learnings to the AI Secretariats of other High Courts and to the Apex Body, so that corrective measures may be adopted across jurisdictions.
(3) Any malfunction, error, or bias in a Court AI Tool with potential legal consequences shall be reported immediately to the AI Secretariat by the officer responsible for supervising such AI Tool.
(4) The AI Secretariat shall, on receipt of a report referred to in sub-regulation (3), initiate remedial measures without delay and report the matter to the AI Committee.
40. Discretion in supervising AI Systems.––
The nominated officer responsible for supervising any AI System shall retain full discretion to accept, modify, or reject any AI-generated recommendation or output within the matters falling under his charge, and shall exercise discretion with independent professional judgment.
41 Review of AI Systems already in use.––
AI Systems already in use in Courts at the time of commencement of these Regulations shall be reviewed by the AI Secretariat for compliance within a period of one year from the date of such commencement and the Appropriate Authority shall determine the appropriate course of action in respect of any system found to be non-compliant.
42. Emergency and fall-back protocol.––
(1) Every High Court shall, in consultation with the AI Secretariat, establish and maintain an emergency and fall-back protocol specifying the procedures to be followed in the event of a failure, malfunction, or unavailability of any AI System or AI Tool in Court.
(2) The emergency and fall-back protocol shall ensure the continuity of essential Court processes through manual or alternative means and shall be tested at periodic intervals as determined by the AI Secretariat.
(3) Where a Court AI Tool fails or is suspended under these regulations, the AI Secretariat shall activate the applicable fall-back protocol and notify the AI Committee within twenty-four hours.
- Transparency and disclosure.––
(1) The Courts shall, where an AI Tool materially assists in any aspect of case management, document analysis, or judicial administration that may affect the conduct of their proceedings, ensure that the parties are informed in a timely and accessible manner.
(2) The obligation under sub-regulation (1) shall apply in all permitted uses of AI specified in regulation 19.
(3) Where an AI Tool is used by any party or his legal representative in the preparation or submission of any document, pleading, or evidence, the AI-assisted character of such material shall be disclosed to the Court at the time of submission by way of a duly executed declaration or certificate in the format prescribed under Annexure I and any Court-initiated AI use in any Court process shall be declared in accordance with the format provided in Annexure II.
(4) The Court shall have the authority to require disclosure of the AI System used, the nature and extent of AI assistance provided, and the steps taken to verify the accuracy of any AI-generated content, in respect of any AI-assisted submission placed before it.
(5) Any person using Synthetic Data or Synthetic Information in any judicial proceeding shall be required to disclose such use to the Court, in such form and manner as the Appropriate Authority may prescribe.
(6) In the event that any document, pleading, or evidence submitted to a Court is found to be fabricated, false, misleading, or inaccurate by reason of its AI-generated character, the person submitting the same shall bear full responsibility therefor and shall not be entitled to rely upon the character of the AI output as a defence. The Court may pass such orders as it deems fit against the responsible person.
44. AI Content Verification Authority.––
The Appropriate Authority shall constitute a dedicated institutional authority, to be designated as the AI Content Verification Authority, charged with the oversight, operation, and continuous updation of verification standards, tools and protocols applicable to GenAI-generated content in Court Process.
- Annual Transparency Report.––
Every High Court, Tribunal and Commission referred to in these regulations shall submit an Annual Transparency Report on AI adoption within its jurisdiction, summarising the AI Systems in use, outcomes of audits, AI Incidents recorded and measures taken for compliance with and improvement of these regulations, to the Apex Body and cause the same to be published on its official website.
In this chapter also there is an attempted decentralization of oversight involving individual AI committees. This may create duplication and also conflicting decisions by different Courts. Had there been one grand committee of CJIs of all high Courts, it could be entrusted with the centralised decision making in most of the cases including development of the format for Technical and Ethical Impact Assessment as envisaged under Section 35.
This impact assessment needs to cover evaluation in the minimum of purpose etc as provided under Section 35(3) all of which can be considered as covered under a DGPSI-AI audit of the algorithm. This short list of requirements of evaluation of an algorithm is an elaboration of the requirements of compliance under DGPSI-AI-Deployer’s implementation specifications. DGPSI-AI covers Explainability, Risk Assessment, Human handler contact, documentation of guardrails, configuration instructions, third party audit, use of AI agents in developemnt etc. DGPSI-AI goes one step further on requiring adoption of a “Kill Switch” particularly in case of “Critical Risks”.
The SCAIF (Supreme Court AI framework) also suggests a controlled environment testing while DGPSI-AI requires documentation of the testing at the developer’s end.
SCAIF suggests periodical audits conducted in-house which is normally the responsibility of a DPO in a private sector organization. DGPSI-AI suggests external audit both at the developer’s end and the deployer’s end.
SCAIF suggests maintenance of an AI register and AI incident data base at all AI secretariats. This also is amenable to a centralized maintenance for better management.
For transparency, when AI is used in pleadings, a suitable declaration will be prescribed.
An “AI Content Verification Authority” has been envisaged as a dedicated institutional authority charged with the oversight, operation, and continuous updation of verification standards, tools and protocols applicable to GenAI-generated content in Court Process.
This proposition under Section 44 is a highly ambitious proposal the full dimensions of which might not have been fully factored into the suggestion. It could mean setting up of a separate Forensic Lab for the task of verifying every AI content used in the Court process which may be practically beyond the scope of this regulation.
Under Section 42, a BCP process to address emergency and fall back is suggested. This is another activity which should be better centralized rather than expecting each High Court to develop.
Section 41 prescribes a review of AI systems already in use which requires a separate audit for which a time of 1 year has been prescribed.
In summary we can suggest that these oversight functions are better managed as a Central Expert team rather than being duplicated at every High Court level. If persisted,, most High Courts will ignore the directions al together and the objective of this regulation may not be achieved.
Naavi
