(In continuation of the earlier article)
The recent debate surrounding the interaction between OpenAI’s proprietary models and the Hugging Face open-model ecosystem has also brought another question back into the spotlight.
Hugging Face reportedly relied on an open-weight model during incident response because commercial models’ guardrails limited forensic analysis, reigniting the debate between closed AI and open-weight AI.
We need to discuss governance, accountability, and auditability matters related to AI along with what is more secure…an Open or Closed model.
According to one school of thought, advanced AI models should remain proprietary, tightly controlled, and accessible only through guarded interfaces. The other believes that openness, peer review, and community scrutiny are the foundations of trustworthy AI.
Closed Model Argument
Developers of proprietary AI systems maintain that restricting access is an essential safety measure. If powerful models are freely downloadable, malicious actors can:
- remove built-in safety guardrails,
- automate cyber attacks,
- generate sophisticated malware,
- create convincing misinformation,
- bypass content restrictions, and
- exploit vulnerabilities at scale.
From this perspective, restricting access is comparable to placing sensitive equipment inside a secure laboratory instead of leaving it on a public street. However, history may suggest otherwise.
The Open Model Argument
Advocates of open models compare AI to cryptography. Modern cryptographic systems are not secure because their algorithms are secret. They are secure because thousands of experts have examined them, attacked them, tested them, and failed to break them.
They argue that transparency often exposes weaknesses before criminals exploit them. Open-source software powers much of today’s Internet, not because it is impossible to attack, but because vulnerabilities are discovered and corrected rapidly by a global community.
The same principle is increasingly being applied to AI. If researchers cannot inspect a model, how can they independently verify:
- hidden biases,
- security weaknesses,
- unsafe behaviour,
- hallucination tendencies,
- privacy leakage, or
- undocumented capabilities?
Transparency creates accountability. However, it is also true that transparency lowers the barrier for misuse and this paradox needs to be resolved.
Real Question
The fundamental question here may not be whether closed systems are safer or open systems are safer. It could be how the “Development of AI” is governed. There has to be accountability at the developer’s level. The DGPSI-AI model that has been put up by Naavi/FDPPI for DPDPA compliance addresses this issue by making a submission of an “Explainabilty statement by the developer mandatory” and such statement to contain details of how the development was tested and whether auditability and accountability is ensured. (Check page 20 of the document )
It is essential for AI developers to ensure the answering of the following questions.
- Who approved the model?
- What data was used for training?
- Is the data legally obtained?
- How is personal data protected?
- What testing has been conducted?
- What are the known limitations?
- Who monitors performance after deployment?
- What happens when the model behaves unexpectedly?
- Who is accountable for its decisions?
- Can an independent auditor verify compliance?
Under DPDPA 2023 where the user of the software is a “Data Fiduciary”, he has a duty to raise such questions with the developer and the developer should if the source code is not public assume the responsibility of a “Joint Data Fiduciary”.
Further just these steps may not prove that the AI cannot go rogue. Hence all AI usage as “Significant Risk” and treating the user as a “Significant Data Fiduciary” is a mandatory requirement.
In the interim when industry battles the IPR issues FDPPI urges academic institutions to join hands with FDPPI to set up AI tools Audit laboratories so that AI tools can be subjected to third party audit. This will be a good faith attempt for the developer and the deployer of AI to mitigate the AI risks.
Naavi
