AI image privacy concerns are growing as experts warn uploaded photos may be used for training, impersonation, scams and identity theft.
AI image privacy has become a growing concern as cybersecurity experts warn that personal photos uploaded to artificial intelligence platforms may carry risks users rarely consider.
“Thank you Gemini for giving me a picture with my father who is in the heavenly world.”
A young woman shared that message on Facebook a few days ago to commemorate her late father on his birthday. She also posted an image showing herself together with her father, created using Google Gemini.
At first glance, the image looked so realistic that it was difficult to identify it as AI-generated.
Such emotional posts are far from isolated. During the past week, Facebook and other social platforms have filled with AI-created images of men holding roses, women dressed in attractive sarees and many other imagined scenes.
It is now increasingly difficult to find someone who has not experimented with AI-generated imagery. You may be among them.
With the earlier Ghibli trend fading, another AI image craze has rapidly spread across cyberspace. However, it raises a bigger question. Where do the photographs uploaded to AI services ultimately go?
BBC Sinhala spoke to two information technology experts to examine the risks.
AI image privacy and model training
Many AI platforms currently used for different purposes rely on information provided by users to improve their systems through AI model training.
Policies vary between service providers. However, many platforms state that information users submit may become part of processes used to improve or train their systems.
Some services also provide options allowing users to control whether their information is used in this way. The data may include text, photographs or audio.
Cybersecurity expert Asela Vaidyalankara explained the issue to BBC Sinhala.
“The first question you need to ask is: What is the privacy of the image you are uploading?
“For example, if you read ChatGPT’s terms and conditions, they clearly state that once you upload an image, it may be used for their training data.”
Vaidyalankara said many photographs uploaded for AI editing contain highly personal information.
“Let’s say you upload a picture of your mother, father, grandmother, or grandfather. That’s a personal photo, right? But now you’ve given it to an algorithm’s training data. At that point, you have been irresponsible with your data,” he explained.
AI model training can be described simply as teaching a computer system to understand and work with large volumes of information. Experts say that training helps systems generate better outputs based on the information they process.
When an image resembling you appears elsewhere
Vaidyalankara also warned that some users upload photographs to AI systems that they would normally avoid sharing publicly.
However, another serious threat comes when other people take photographs without permission.
Thinuri Thissera, Information Security Engineer at the Sri Lanka Computer Emergency Readiness Team, said SLCERT has received complaints involving such misuse.
“We are currently seeing reports of original photos being taken by others and edited using AI tools to create nude images or videos, or used for impersonation.”
She said photographs uploaded to users’ own social media accounts often become the source material.
“Then these are used to threaten and extort money, or there is a trend where they are used to solicit sexual favours,” Thissera said.
Vaidyalankara also noted that AI already enables the creation of pornographic material.
“Imagine if someone is fully clothed, it can be generated with AI to create nude versions. These are harmful,” he warned.
He said repeatedly uploading personal photos can indirectly increase the chance of images reaching people with malicious intentions.
“During the Ghibli trend, a huge amount of new training data came to ChatGPT. As we keep doing these things, the risk to our personal data increases,” Vaidyalankara said.
Thissera also warned about possible future developments involving AI and robotics.
“Now robotics technology has advanced. It might be possible to create a clone of us using AI.
“For example, if I continuously chat with an AI bot and share my personal details, if I use it to get advice… if it does a self-analysis, or if it has been given the ability to do such analysis from the backend, combined with the images we provide, with future technology or even existing technology, there is the capability to create another robot just like me,” she explained.
She said images could, in some circumstances, also create risks involving security features such as Face ID on mobile devices.
Thissera further warned that increasingly realistic AI images and videos could make people more vulnerable to fake news and manipulated content.
Could AI databases themselves be hacked?
Vaidyalankara said the probability of major AI databases being hacked remains low.
However, he stressed that low probability does not mean zero risk.
He said many parties continue attempting unauthorized access to data systems.
“If someone hacks that data centre, they could take this data. Many are trying to hack even today. They don’t necessarily reveal that they’ve done it. It’s not that there are no risks. Those risks exist. But currently, they are managing those risks,” he said.
He added that direct access by a third party may be unlikely, but users still surrender information to other organisations.
“Can’t say no. But you unknowingly hand over your data to other institutions. That’s where the risk lies.”
Thissera warned that leaked information could eventually reach the Dark Web.
“Data can be leaked in bulk. There are instances where such data could be sold on the Dark Web. Then your identity could be stolen,” she explained.
She said criminals could potentially use similar-looking photographs to create fraudulent identity documents, including passports.
Metadata creates another AI image privacy risk
A photograph contains more than the details visible to the human eye.
Images can also contain metadata, including information about the device used to capture them, the date, image size and location.
Vaidyalankara said this creates another cybersecurity concern when people upload personal images to AI platforms.
“It’s not just Gen AI; even if you post on social media, that risk exists. Without realizing it, you take a photo with some background. There are platforms that can tell exactly where that photo was taken. That’s where you face a risk,” he explained.
He warned that this type of information could potentially threaten a person’s physical safety.
Thissera agreed that metadata can also become vulnerable to misuse.
Vaidyalankara said users generally do not know exactly how AI companies store information collected through their services.
“We don’t know whether they store the image as is, or whether they break it down and store it separately. They have many data centres. These are commercial enterprises. They are not obligated to disclose that,” he explained.
How can users protect themselves?
Thissera said SLCERT has not yet received reports in Sri Lanka of information leaking specifically after users uploaded material to AI platforms.
“Currently, we have not received any reports of data leaks after uploading to AI tools like Gemini or ChatGPT,” she said.
However, she cautioned that “there is a possibility that it could happen in the future.”
Vaidyalankara likewise told BBC Sinhala that future risks could become significant.
“Just as there are benefits to these, the other side is putting your personal data at risk,” he said.
Therefore, Thissera advised people to understand the consequences before using such tools.
“If you are going to use them, the advice is to do so while being aware of the consequences,” she said.
“We can’t guarantee 100% security, because we live in a digital world.
“When using free tools, the probability [of a risk] is higher. It could leak. Because we don’t know where this data is stored, how it is stored, or how it is processed. Because we are using a free service.”
She said some paid AI platforms provide assurances that they will not misuse user information.
Her final advice was simple.
“So be a bit careful when using free ones,” Thissera emphasized.
