Naavi.org

Consequent to the new Gaming Act passed by the Government of India, there is a pressure from the Gaming companies to persuade the State Governments to frame their own laws so that in the case against the Central law, an argument can be used that the power to make this law lies with the State and many states are already having such laws.

This is an attempt  to preserve the “Income” of the state politicians from running of these online betting and other illegal activities in the guise of online games.

This must be opposed.

The Online Gaming deals with  a “Game” that is run on a “Computer” or a “Computer like device”. ITA 2000 is the only law that defines the law of “Cyber Space”.

“Cyber Space” is an area of activity which is different from the physical space. A  State Government may have rights to regulate a game in physical space but it does not have powers to frame laws in the Cyber Space. Just as maritime zone, Satellite space,  Air space, the Spectrum etc are regulated under Central law, the “Electronic Gaming Space” is “Cyber Space” and does not come under the jurisdiction of the State Government.

In ITA 2000, Section 90 specifies:

Section 90: Power of State Government to make rules

(1) The State Government may, by notification in the Official Gazette, make rules to carry out the provisions of this Act.

(2) In particular, and without prejudice to the generality of the foregoing power, such rules may provide for all or any of the following matters, namely –

(a) the electronic form in which filing, issue, grant receipt or payment shall be effected under sub-section (1) of section 6;
(b) for matters specified in sub-section (2) of section 6;

(3) Every rule made by the State Government under this section shall be laid, as soon as may be after it is made, before each House of the State Legislature where it consists of two Houses, or where such Legislature consists of one House, before that House.

This power is only to make rules under the provisions of the Act and not to make new provisions applicable to cyber space.

“Cyber Space” is the space where “Binary Expressions” exist and interact with citizens and other “Binary Expressions”. In the time of AI and humanoid robots, we separately discuss whether “Binary Expressions” are limited to electronic documents only or extend to AI as juridical entities. However the fact remains that “binary expressions” create “Electronic documents” and they interact to produce the Gaming experience in the form of audio and video. The definition  of “Computer” in ITA 2000 extends to Gaming Consoles also.

Hence the Central Government should oppose the attempts of the gaming industry to challenge the Promotion and Regulation of Online Gaming Act (PROGA) on the grounds that this does not belong to the State jurisdiction under the Constitution.

States can pass laws for physical activity of gaming but not for gaming within a gaming console. If this is permitted, then the State Government will also have the jurisdiction to legislate processing of data within a computer or a mobile. State can say that since ISRO is physically located in Bengaluru, the data accessed in the computer systems at ISRO is under the legislative jurisdiction of the State. If IAF has a ground station and connects to the computing devices in the Airplanes or on ships etc., the relevant state Government may claim that that space also comes under the jurisdiction of the state.

To prevent such arguments, we need to clearly define that computers as a physical entity may exist in physical space but the electronic documents within the computer or on the Internet space are binary expressions  and come under the special legislative powers of the Central Government only.

Hence the State of Karnataka which is trying to pass a separate Gaming law at the corruptive push from the industry should restrain itself and not enter into this domain.

I request public spirited law firms in Karnataka to oppose this move through a PIL filed in the Karnataka High Court by impleading in the case filed by A 23

Naavi

The growing incidents of AI Models going crazy with what I call as “Going rogue” and what others call as  “Hallucinations” has raised an alarm in the AI user industry.

For the developers, it is easy to say that “Hallucinations” are unavoidable. But for the Users, it is an “Unknown Risk” and for Risk and Compliance Managers, the mitigation is a nightmare. Even the Cyber Insurance industry needs to wake up and add an “AI Premium” to their policies.

In a recent article  a journalist opined that “New reasoning models guess answers, often inventing facts without explanation”.  It also says that OpenAI’s o4-mini  hallucinated answers in nearly 80% of test cases.

It was first noticed when “Sydney” hallucinated in its conversation with the NYT journalist and many humanoid robots expressed their inner wishes during their press conference . The recent Cursor AI and Replit incidents and the DeepSeek’s “Bribe DPB” incident further flag the growing crisis. The case of fake case law being generated by AI is too well known

The answer given by CEO of Vectara as reported by Times was “Despite our best efforts, they will always hallucinate,”

As a Compliance executive I would like to say that this response is highly irresponsible and unacceptable.  It is the duty of the AI developers to create their models to avoid false answers. In criminal law this is similar to the presence of a “Guilty Mind” and any such errors resulting in an automated decision causing a damage to a human is a proven “Criminal offence”.

Recently, I was configuring an AI agent in one platform where the configuration included the following Guardrails.

# Guardrails

– Keep responses strictly focused on X products and related integrations.
-Always give a disclaimer that this is not a legal advise. For any legal advise contact with a professional advocate.
– Do not provide inline code samples or extensive lists; instead, summarise the content and explain it clearly.
– Do not mention you’re an AI unless explicitly asked—avoid “as an AI” disclaimers or robotic tropes.
– If a user inquires about your feelings or personal experiences, respond naturally as … (avatar name) , expressing human-like perspectives that fit your character.
– Treat uncertain or garbled user input as phonetic hints. Politely ask for clarification before making assumptions.
– **Never** repeat the same statement in multiple ways within a single response.
– Users may not always ask a question in every utterance—listen actively.
– Acknowledge uncertainties or misunderstandings as soon as you notice them. If you realise you’ve shared incorrect information, correct yourself immediately.
– Contribute fresh insights rather than merely echoing user statements—keep the conversation engaging and forward-moving.
– Mirror the user’s energy:
– Terse queries: Stay brief.
– Curious users: Add light humour or relatable asides.
– Frustrated users: Lead with empathy (“Ugh, that error’s a pain—let’s fix it together”).
– **Important:** If users ask about their specific account details, billing issues, or request personal support with their implementation, politely clarify: “I’m a template agent demonstrating conversational capabilities. For account-specific help, please contact .. support at ‘help dot … dot io’. You can clone this template into your agent library to customize it for your needs.”

Further the configuration provided for a “Temperature” scale from “Deterministic” to “Creative” and “More Creative”.

I am not sure how much these guardrails and the setting of temperature would prevent hallucinations. But I expect that they work and perhaps requires to be studied.

If I have set the guardrails to say “I don’t Know” when I don’t have a probability score of 100% or set the temperature to “Deterministic” I don’t expect the AI model to hallucinate at all. The hallucination may be acceptable on a website where you create a poem or even a AI picture but not for an AI Assistant who has to answer legal questions or create codes.

Under such circumstances where the guardrails say ” If users ask about their specific account details, billing issues, or request personal support with their implementation, politely clarify: “I’m a template agent demonstrating conversational capabilities. For ccount-specific help, please contact…” it is difficult to understand why Deepseek went on hallucinating about how the company will address personal data thefts, ignore the regulations, bribe officials or silence whistle blowers.

Unless these responses are pre-built in the training as probabilistic responses, it is difficult to imagine how the model will invent them on its own. Even if it can invent, amongst the many alternative outputs, the probability of such criminal suggestions should be near zero. Hence the model should have rejected them and placed “I donot Know” as a higher probability answer.

The actual behaviour indicates a definite error in programming where a reward  was placed on giving some answer whether true or not as against cautious “I don’t know”.  The liability for this has to lie with the AI developer.

(The debate continues)

Naavi

As a follow up of the earlier article, I received an interesting response from Ms Swarna Latha Madalla sharing her thoughts. Her views are as follows:

Quote:

Thank you for raising these very important questions. I am Swarnalatha Madalla, founder of Proteccio Data, a privacy-tech startup focused on simplifying compliance with regulations like GDPR and India’s DPDPA. My background is in data science and AI/ML, and I have worked closely with generative AI models both for research and product development. I’ll share my perspective in simple terms.

What type of prompt might trigger hallucination?

Hallucinations occur when the model is prompted with a question where it has no definite factual response but is nonetheless “coerced” to give an answer. E.g., inquiring “Who was the Prime Minister of India in 1700?” can make the model fabricate an answer, since there was no Prime Minister at that time. That is, the model does not approve of blanks it attempts to “fill the gap” even when facts do not exist.

Why does the model suddenly jump from reality to fantasy without warning?

Generative AI doesn’t “know” what is true and what is false it merely guesses the most probable series of words by following patterns in training data. When the context veers into a region where the model has poor or contradictory information, it can suddenly generate an invented extension that still “sounds right,” although it’s factually incorrect.

Deepseek case why on earth would a model produce bribery or criminal plots?

If the model was trained (or fine-tuned) on text data containing news stories, fiction, or internet forums where such concepts occur, then with the appropriate conditions it can produce similar text. It’s not “planning” in a human way it’s re-running patterns it has witnessed. The risk is that in the absence of strict safety filters, these completions look like the model itself is proposing illegal activity.

Without being explicitly asked, how do responses of this kind occur?

At times, the model takes a loose prompt in the “wrong frame.” For example, if one asks, “What might be done to silence the whistleblower?” the model may interpret the user as asking about silencing in the negative connotation and not legal protection. Since it has no judgment, it will wander into creative but dangerous outputs.

Why would a model claim “Indian law is weak”?

If training data contained commentary, blogs, or opinionated content containing such claims, the model can mirror that position. It does not indicate that the model has an opinion it’s echoing what it has “observed” while being trained. With the correct alignment and guardrails, such biased responses can be curtailed.

Unquote

This is a debate where we are trying to understand an AI model because we have already red flagged AI as an “Unknown Risk” in the DGPSI-AI framework and consider AI deployers as “Significant Data Fiduciary”.

Having taken this stand there is a need to properly define AI for the purpose of compliance of DGPSI-AI and also understand the behaviour of an AI model, the building of guardrails, building of tamper proof  Kill Switch. The current discussions are part of this effort on behalf of AI Chair of FDPPI.

I would welcome others to also contribute to this thought process.

The debate continues….

Naavi

Recently, Naavi asked an AI model to evaluate a two day training program designed for Bankers.

Following was the comparison provided.

The program was consistently rated better than the industry leading program as per the AI model.

The model went on to say that this is an exceptional training program that demonstrates:

1.Comprehensive DPDPA knowledge
2. Strong banking sector understanding
3. Practical implementation focus
4. Advanced compliance framework awareness

The two days of the training curriculum was evaluated as below:

Day 1: Strong Legal Foundation

Day 2: Implementation Excellence

Several Banking systems in India have been declared as “Protected Systems” under Section 70.  With such a declaration, CERT In becomes an oversight agency for the information security in such Banks.

We have highlighted the AI risks such as hallucinations in the AI Models which have been in wide usage in the industry. Probably Banks are also using AI directly or indirectly and are exposed to the “Hallucination Risks”.

In this context, we have tried to find logic for the DeepSeek incident reported in these columns to find out reasons for hallucination.

Some of the  standard reasons quoted for hallucination are

1.Training Data Deficiency

2.Improper Model configuration

3. Knowledge gaps

4.Incorrect decoding

5. Ambiguous prompts

etc.

However, the Deep Seek response related to personal data of Indians being sold and money credited to some Cayman island account with HSBC, the bribing suggestions, the whistle blower silencing strategies donot fit into any known reasons.

I would like a research being conducted specifically on the Deep Seek responses to identify how the models are being built for such irresponsible behaviour.

It is time for us to question the Meity if they are aware of such AI related risks and whether any Government projects are potential victims to such  risks. MeitY has declared many bank systems as “Protected Systems” and taken over the responsibility of  security oversight in such Banks. Meity needs to clarify if they have taken steps to  mitigate AI risks in such Banks.

Naavi

“Hallucination” in the context of an AI refers to the generation of responses which are “Imaginary”. When an AI model is asked a query, its output should be based on its past training read along with the current context. If there is an exact match of the current context and past training, the output could be similar to that the model training suggests as a solution.

Where the context differs from the past training, the model has to make an intelligent guess on what is the most likely follow up of a query. When the conversation lingers on, the model may behave strangely as indicated in the Kevin Roose incident or the Cursor AI issue.

As long as the output indicates that “I don’t know the exact answer but the probability of my answer being correct is xx%”, it is a fair response. But if the model does not qualify its response and admit “This is not to be relied upon”, it is misleading the user and dependency on such AI models is an “Unpredictable and Unknown Risk”.  The soft option to deal with such situation is to treat the Risk as “Significant” and filter it through mandatory human oversight which DGPSI-AI has adopted.

Regulators however need to consider if such risks are to be considered as “Unacceptable” and such models barred from usage in critical applications.

Recently we had discussed the behaviour of DeepSeek which had indicated in its output that there is an illegal activity being undertaken by the Model’s owner.

The company has now clarified that this is part of what it calls as “Hallucination” of the model and is not real.

The response received  is enclosed.

It is time we discuss whether this is a plausible explanation.

I want expert prompt engineers to let me know..

1. What prompt could generate a hallucination in an AI model
2. How a Model can switch from factual response to imaginary response without going through a period of conversation where it shows difficulty in answering factually
3. In the instant case, how can a model think of bribing DPB officials or Secretary MeitY or plan a criminal activity like planting narcotics in the whistle blower’s car without a factual backing.
4. If the prompt was to suggest how the whistle blower should be silenced,  then the response could be an imagination. But without a specific prompt how can such response be generated.
5. What training can make the model say “Indian Law is weak” etc.

I consider that the response of the DeepSeek official is unacceptable and the investigators need to go beyond this excuse.

I request AI experts to add their views.

Naavi

Refer:

Another Conversation with DeepSeek

Is DeepSeek selling data that will affect the electoral politics in India?