After a clip of Blackpink member Jennie at Tokyo's Summer Sonic festival racked up over 10 million views, fans sounded the alarm that the footage may have been manipulated. The controversy has tested whether legal threats are enough to curb deepfake-related harm.
As David D. Lee of the South China Morning Post said, the episode was a new flashpoint in South Korea's battle against artificial intelligence-enabled sexual abuse.
Here's what to know
After the clip spread online showing what appeared to be a "wardrobe malfunction," fans picked through it, arguing it showed signs of AI editing and calling for it to be taken down. The video came from Jennie's headline performance, during which she turned away from the crowd, adjusted herself, and then continued the set.
The account believed to have first uploaded the clip was suspended, according to Lee, but there has been no announced police or prosecutorial action.
SCMP presented the issue as part of South Korea's fight over AI-enabled sexual abuse, saying that legal threats have not deterred many teenage offenders.
More background
Deepfakes can be especially explosive, as a clip can spread quickly before facts are clear. Even if a post is removed, copies and screen recordings can continue to circulate.
TCD Picks » Quince Spotlight
💡These best-sellers from Quince deliver affordable, sustainable luxury for all
The damage also stretches beyond celebrities. AI-generated or AI-altered sexual content can target ordinary people, turning private humiliation into a public and searchable ordeal.
As Lee covered, South Korea updated laws in 2024 following sex crime rings that shared faked images of women and girls, including students, on encrypted Telegram channels. Those new laws featured prison time for up to seven years.
"But the newer deepfake cases were different in one crucial aspect: demonstrating how convincing abuse material can now be conjured from an ordinary photo, then rapidly circulated and viewed by large numbers of people, before anyone – victim or investigator – knew who had made it," Lee wrote.
And it seems the problem has not exactly gone away on the heels of the new laws, either.
"The Jennie case shows why. Before any prosecution can begin, investigators need to establish whether the footage was indeed fabricated, trace who posted it first and prove whether those who shared it knew it was fake," Lee wrote.
That's not always an easy case to make. Lee spoke to An Gab-chul, a lawyer at Gammyung Law Firm in Seoul, who said, "In the case of anonymous accounts, it can take a long time to identify the actual user. Investigators often have to contact operators of overseas servers to determine who is behind an account before the person behind the account shuts it down or destroys evidence."
What's being done?
Several lawyers and groups, such as Amnesty International Korea, are working to address the problem from both a legal and cultural position, trying to get at the root of the issue.
"This is not something that can be solved simply by strengthening the law, like an endless game of cat and mouse," Bang Suk-ho, head of the AI industry center at Seoul law firm Lin LLC, told Lee.
In a statement, OA Entertainment said Aug. 18 it had "recently identified ongoing misconduct targeting the artist [Jennie] across online communities and social media platforms," SCMP reported. The company said it was working with lawyers on civil and criminal action and promised "no compromise or leniency whatsoever."
Where can I learn more?
This is one example of how AI-related harm can spiral. AI content can also highlight concerns over the technology's social and infrastructure costs.
• At the Chicago Sun-Times, an AI-generated summer reading list embarrassed editors.
• At a global summit, leaders warned AI may become an engine of inequality.
• Across the tech sector, backlash is growing as AI strains jobs, water, and power.
• Lawmakers are being pushed to require revelations about data centers as AI's footprint grows.
• Utilities are signing unprecedented deals with the industry as AI data centers drive up demand.
Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.
