The Classrooms Hunted in a Chatroom
In 2024, South Korean police opened their first investigation into Telegram over sexually explicit deepfakes of women and girls circulating in chatrooms, some with thousands of participants. Reported deepfake sex-crime cases rose sharply that year, and most of the identified suspects were teenagers.
A first look inside Telegram
In late August 2024, South Korea's National Police Agency began investigating Telegram over its possible role in the spread of sexually explicit deepfakes, the first time the country's police had formally investigated the messaging platform, according to The Korea Herald. The move followed reporting and crowdsourced efforts that exposed chatrooms distributing AI-generated sexual images and videos of identifiable women and girls.
Woo Jong-soo, head of the National Office of Investigation, said the platform was hard to pursue. “Investigating Telegram is difficult, as they do not share account information and other forms of personal information with investigative officials or law enforcement agencies based in Korea and in other countries,” he said, according to The Korea Herald. He added: “It's not true that because Telegram didn't give us the material to investigate, we are unable to catch the actual perpetrators of the crimes.”
Telegram said it was cooperating. A spokesperson, Remi Vaughn, told NPR that the company “has been actively removing content reported from Korea that breached its terms of service and will continue to do so.”
Schoolgirls among the targets
Much of the abuse centered on women and girls the perpetrators knew. An overwhelming majority of deepfake victims in South Korea are women and teenage girls, NPR reported, with the manipulated images typically taken from victims' social-media accounts or captured in person. Some of the chatrooms, which surfaced in searches for terms like “mutual acquaintance room” or “humiliation room,” had thousands of participants. Deepfakes of female university, high-school and middle-school students were being shared, according to a widely cited analysis by the newspaper Hankyoreh reported by NBC News.
Chang Dahye, a research fellow at the Korean Institute of Criminology and Justice, told NPR that the abuse was rooted in contempt: “for most men consuming these contents, the goal is to belittle women in general.”
The numbers, and who is behind them
Reported cases climbed steeply. In the first seven months of 2024, 297 cases of crimes involving sexually explicit deepfakes were reported, up from 180 in all of 2023, NPR reported. A government media watchdog received nearly 6,500 requests to address sexually abusive deepfake videos between January and July 2024, roughly four times the volume over the same period a year earlier.
Many of those creating the images were themselves young. Of the 178 suspects police booked during that seven-month period, 74 percent were ages 10 to 19, up from 65 percent in 2021, according to NPR. In the days after the story broke, authorities logged another surge: between Aug. 26 and Aug. 29, they received 88 reports of deepfake pornography and identified 24 suspects, The Korea Herald reported.