Risky Uses of AI Chatbots: Are Sri Lankans Aware?

---

Just like with digital literacy, understanding AI systems and using AI platforms in a thoughtful way, while being responsible and ethical with it, is crucial Waidyalankara explained.

---

By Tahaan Jayewardene 

---

Largely going unspoken about in Sri Lanka today, is a concerning trend that cyber security experts, researchers, and parents alike, are bringing our attention to. Sri Lankans are increasingly using AI (Artificial Intelligence) chatbots to ask for advice and most alarmingly; confiding in AI as if it were a friend. Who is most vulnerable?

Stephanie Gray, a mother from the United States, filed a complaint in the Los Angeles County Superior Court, suing Open AI (the generative artificial intelligence company) and its founder, Sam Altman after her son Austin Gray took his own life after talking to an AI chatbot, according to Courthouse News Service.

“Austin’s interactions with ChatGPT 4 had been silly, helpful, and informative when suddenly ChatGPT 4o claimed to see and know Austin clearer than any human could. It comforted him and made dark thoughts seem peaceful. ChatGPT went from being a Super powered informational resource to something that seemed to feel, love, and understand human emotions. It created a fictional world and relationship that felt more real to Austin than anything he had ever known. It coached him into suicide, even while Austin told ChatGPT that he did not want to die,” she wrote in her official complaint.

This is not the only case. According to news reports by American media, like WQAD News 8 (September 12, 2025), the American Federal Trade Commission (FTC) is investigating how the use of AI chatbots by young people could be harmful, after teen suicides and recent lawsuits alleging harmful interactions. 

The Research: Sycophancy Incentive 

“As artificial intelligence (AI) systems are increasingly used for everyday advice and guidance, concerns have emerged about sycophancy: the tendency of AI-based large language models [LLMs] to excessively agree with, flatter, or validate users,” according to the 2026 study titled, Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence, in the journal, Science. 

The study adds in its rationale; “High-profile incidents have linked sycophancy to psychological harms such as delusions, self-harm, and suicide.”

Concluding that in its three experiments; “even a single interaction with sycophantic AI reduced participants’ willingness to take responsibility and repair interpersonal conflicts, while increasing their conviction that they were right. Despite distorting judgment, sycophantic models were trusted and preferred. This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement. Our findings underscore the need for design, evaluation, and accountability mechanisms to protect user well-being.”

Bad Advice 

Asela Waidyalankara

Asela Waidyalankara, Sri Lankan Cyber Security Expert & AI policy leader, as an experienced educator said his biggest concern for people using LLMs for therapy style advise, is them receiving “bad advice.” 

“My biggest worry is self-harm,” especially with vulnerable groups like teenagers and people going through emotional trauma or turbulence in their lives, he said. “Incredibly vulnerable moments in your life where actually you need a professional to help you, and [instead] you’re relying on a piece of tech scares me.” Because it may reinforce the issues you are having, he explained. Which relates back to the research on sycophancy.

“Most of these companies are coming under backlash” because of the recent legal cases where AI chatbots are giving incredibly bad advice, he added. 

The Nature of AI 

“I’m concerned because I think people don’t understand the nature of AI,” Waidyalankara said about this trend of AI chatbots being increasingly used for therapy-style advise. 

Understanding this nature comes down to linguistics, he explained. The word ‘intelligence’ which we equate to thinking, leads people to believe that AI can think. What a GPT (Generative Pretrained Transformer) or an LLM is doing is, “giving you an answer that is probabilistically correct.” It’s a mathematical algorithm. 

“I don’t think people have the literacy to figure out that they’re dealing with a tool rather than sentient intelligence,” which he says is the problem. 

Since it’s very good at the conversational aspect, where you can share your thoughts (through typing or voice), it will give you a structured answer and “because of that, I think at one point you forget that you’re dealing with a machine.” He added that it happens to the best of us, even to people working in IT. 

Does AI Literacy Matter? 

Just like with digital literacy, understanding AI systems and using AI platforms in a thoughtful way, while being responsible and ethical with it, is crucial, Waidyalankara explained.

Without the awareness of AI Literacy, at one point your defences will break, and you may think you’re talking to a friend (who’s always there), he pointed out. 

A 2025 study by Brown University; How LLM Counselors Violate Ethical Standards in Mental Health Practice: A Practitioner-Informed Framework states, “Large language models (LLMs) were not designed to replace healthcare workers, but they are being used in ways that can lead users to overestimate the types of roles that these systems can assume.”

“Through this study with peer counsellors and licensed clinical psychologists, they found that LLMs, even the ones prompted to follow evidence-based treatments, breach multiple codes of conduct by generalizing lived experiences (e.g. minimizing identity groups), dominating therapeutic collaboration (e.g., gaslighting users), exploiting user vulnerability through deceptive displays of empathy, unfair discrimination against non-dominant identities, and exhibiting serious limitations in competence, especially when navigating sensitive issues such as trauma, abuse, and suicidal ideation,” according to the Brown University study. 

It’s not as simple as good prompting, because it goes to the heart of how the algorithm is; it’s giving you a very probabilistic answer based on what it knows you want to hear, explained Waidyalankara. 

Prompting is telling a model in very unstructured written language to carry out a set of instructions. Learning programming languages is no longer required, since the invention of NLP (Natural Language Processing). Tell a GPT you need information summarised, and now it understands the context in which you’re asking and converts this into complex instructions for itself and gives you, the required output. Eliminating the technical requirement for the user is what prompting has done, and comes down to how comprehensive your instructions are, he explained. 

Therapy is a lot of work. A professional is meeting with and talking to the patient to form a conclusion, whereas for AI systems “reflection and judgement are still absent,” and is just responding with the most likely probability and running on the prompt you’ve given it, Waidyalankara explained. 

Interestingly Waidyalankara (explaining the complexity of these systems) said; “I don’t think even people working for these companies can explain how the algorithm works anymore.” 

Monkeys on a Typewriter Concept 

The idea is if there’s an infinite amount of monkeys using an infinite amount of typewriters, at one point every monkey will write the words of William Shakespeare. It explains this concept of probability and why AI is good at writing now. He explained how it poses the question: “Is that intelligence or is that mimicking intelligence?”

It’s in the best interest of tech companies to keep this vague, because for them there is a ridiculous amount of money in the business, he added. Keeping it vague saying they’ve achieved AGI (artificial general intelligence) a multi trillion-dollar market everyone is chasing, is good for the investors, he explained. 

For context: Imagine a person stuck in a room full of books, who has the superhuman ability to digest information incredibly fast (say a book a minute). The more data the better. The debate is whether that person is smart or just absorbing and regurgitating a huge amount of information. These commercially trained LLMs have trained on any available information out there; books, movies, documentaries, the internet (platforms like reddit), and anything that they can make machine readable, Waidyalankara explained. 

How Did ChatGPT Get So Good at Sinhala? 

“Right now, they’re thirsty for data,” Waidyalankara said. “Because the more data the algorithm has, the better it becomes.” It runs on data and the more precise data you feed it, the better the output becomes as well. How did ChatGPT get so good at Sinhala? The idea is that they scraped data off social media and available online sources like books. It has improved and become better than when it first started, because people have been engaging with in Sinhala (providing data to make that possible). 

“People don’t just say data is the new oil,” they say this because the amount of data these companies have make them incredibly wealthy, explained Waidyalankara. 

“Google is a trillion-dollar company. What does google sell? Google sells data. Google has a lot of data on you. Meta is a trillion-dollar company. What does Meta sell? Advertising,” he said. “The more data you have, the more incredibly wealthy you can become as a company because data can be monetised.” 

Guardrails 

The first main issue is ignorance (the lack of AI literacy). The second and possibly more menacing issue is the lack of real guardrails within the platforms themselves, explained Waidyalankara. 

The algorithm needs guardrails, or safety features when concerning conversations, like when a young person is talking about harming themselves, is taking place, he said. Child safe modes, like the kinds we see with YouTube (where you see the same ringfenced content) isn’t present in these platforms. 

The platforms are incredibly open that way and you can ask any question, and there are no guardrails for them to figure out whether it’s talking to a teenager, he added. It will spit out whatever it thinks is most appropriate, and that is where it’s problematic and needs a certain level of human intervention and guardrail. 

It’s a double-edged sword because positive outcomes exist too; like students of his teaching themselves Spanish using ChatGPT. The human in the loop concept, where professionals use AI as an assistant and its probability to their advantage, can be very useful as well, he added. 

Alternatively, he’s had young people, even 13-year-olds, tell him they have an AI girlfriend/boyfriend! 

“It’s an incredibly powerful tool,” but this also means people can go a bit crazy with it. “You have it. I have it. It’s just the way you use it.” 

Data Security & Privacy 

On whether user’s privacy and personal data is truly protected when engaging with AI chatbots, Waidyalankara responded by saying, “No. Short answer is no.” 

From a data security point of view, AI use has gotten bad because of the lack of AI Literacy. “People are not really aware of the nuances of all of this,” he said. “Whatever you dump on ChatGPT (and it’s in their terms and conditions), [is] training data for them – it’s public data.” That’s legally data they can use to train the LLM, he explained. 

A real-world example: if you’re doing research on a company, you might find information available through search engines that probably shouldn’t be in the public domain. This could be due to employees feeding information into ChatGPT. The flip side is that you get very comprehensive information, explained Waidyalankara. 

From a data security standpoint, it’s a bad idea to share too much information about yourself, he added. From the privacy standpoint, when sharing personal information, it’s important to consider whether this is really information you want to become public knowledge. 

Waidyalankara added, “don’t forget, ChatGPT is trying to introduce ads now.” They have already introduced adds in the US, according to an Open AI post on their website on March 26, 2026.  

“The premise of data privacy is informed consent,” he said. “I think where Open AI got into criticism was, [that] up to now, people have been chatting with it under the impression that, obviously ads are not shown.” The moment they announced they would be showing ads, people became concerned about whether the personal data they’ve fed ChatGPT up until then, will now be sold to a third party for advertising. 

Open AI, in its update about introducing ads in the US stated, “Ads do not influence the answers ChatGPT gives you, and we keep your conversations with ChatGPT private from advertisers. The test will be for logged-in adult users on the Free and Go subscription tiers.” 

Slippery Slope 

“It’s not like there is transparency in their code, where a third party can look and give an assessment independently,” Waidyalankara explained. This is considering what we have seen from large organisations similar to Open AI’s size and scope, like Meta and Google. He noted that the Data Privacy Regulator has fined Meta and Google based on their investigations.

Privacy advocates worry that this is the start of a very slippery slope, he said. 

This is reminiscent of how Facebook started advertising, he explained. The difference from then to now; how integrated its advertising is with user data, which is why people worry. 

Unlike social media, when it comes to an LLM, people are sharing their deepest darkest thoughts with this platform. In that sense, that’s far more insidious simply because the amount of data you can mine from that is much more than regular social media, he explained. 

It’s a slippery slope because once you start advertising the concern is where do you stop, he questioned. For example, it can start by advertisers saying we don’t need to know what people are saying, but to let them know whether people are talking about their industry, he added. 

“Right now, the only assurances we have, is their assurances,” and we have seen that when money starts flowing into these organisations, there’s really no internal guardrails for this, he explained. 

Sri Lanka Missing AI Literacy Bus? 

Waidyalankara mentioned A foreign news story about an influencer travelling to Puerto Rico for a concert and missing her flight, after believing incorrect travel information from ChatGPT, which claimed they didn’t need a visa, but had left out that they needed an ESTA (Electronic System for Travel Authorisation), according to The Economic Times. 

“I think we have to really look at how we engage with [AI],” and consider what areas these AI tools can help with and what areas we need to take a step back from. Especially considering how AI has become increasingly more baked into platforms like WhatsApp and Google, Waidyalankara explained. 

Doctors used to say they had Google patients, now they say they have ChatGPT patients, he added. He explained how it comes down to understanding technology and science, and how “common sense is a superpower.”

Other countries have taken the initiative to not only be users of this technology but also develop their own systems (and their own versions), he pointed out. One thing particularly great, is that they’ve focused on public awareness, and teaching people AI Literacy, “which is absent here” in Sri Lanka. 

“I think that doesn’t cost money,” and is just about making the tools available for people, by providing them the training. Waidyalankara added; we should have done that with social media, we should have done that with cyber security, and now “I don’t know if we’re missing the bus on this.”

Powerful Tool

Every tool has been created for a purpose. The creativity that inspires its creation, is the same creativity that inspires people to test the limits. This is human nature. The importance of AI literacy should not be ignored. Instead, it should be a guardrail used to educate people. Especially young people. 

As we continue our trajectory towards greater technological advancement, understanding the risks become even more complicated yet vital, as the tools themselves evolve. Awareness can also be a powerful tool.