“When we look at intimate image abuse, the vast majority of tools and weaponized use come from the open source space,” says Ajder. But they often start with well-intentioned developers, he says. “Someone creates something they think is interesting or cool and someone with bad intentions recognizes its malicious potential and weapons it.”
Some, like the repository that was deactivated in August, have purpose-built communities around them for explicit uses. The model has positioned itself as a tool for deeply adulterated pornography, Ajder claims, and is becoming a “funnel” for abuse, which mainly targets women.
Other videos uploaded to the porn-streaming site by an account credited to AI models downloaded from GitHub featured the faces of popular cheating targets, celebrities Emma Watson, Taylor Swift and Anya Taylor-Joy, as well as others featured lesser known but very real women. , superimposed in sexual situations.
The creators freely described the tools they used, including two that were scrubbed by GitHub but whose code survives in other existing repositories.
Offenders looking to deep-fake congregate online in many places, including in secret community forums on Discord and in plain sight on Reddit, exacerbating deep-fake prevention efforts. One Redditor offered their services with the archive repository’s software on September 29th. “Can someone do my cousin,” asked another.
Torrents from the main repository banned by GitHub in August are also available in other corners of the web, showing how difficult it is to police counterfeit open-source software across the board. Other deepfake pornographic tools, such as the app DeepNude, were similarly removed before new versions appeared.
“There are so many models, so many different forks in the models, so many different versions, it can be hard to track them all,” says Elizabeth Seger, director of digital policy at cross-party think tank Demos in the UK. “Once a model is open-sourced publicly available for download, there is no way to do a public rollback of it,” she adds.
One fake porn creator with 13 manipulated explicit videos of female celebrities credited one prominent GitHub repository marketed as an “NSFW” version of another project that encourages responsible use and explicitly asks users not to use it for nudity . “Learn all available Face Swap AI from GitHUB, don’t use online services,” says their profile on the tube website, cheekily.
GitHub had already disabled this NSFW version when WIRED identified the deeply fake videos. But other repositories branded as “unlocked” versions of the model were available on the platform on January 10, including one with 2,500 “stars”.
“It is technically true that once [a model is] out there it cannot be reversed. But we can still make it harder for people to gain access,” says Seger.
Left unchecked, she adds, the potential for harm from deep-fake “pornography” isn’t just psychological. Its effects include intimidation and manipulation of women, minorities and politicians, as seen with political forgeries affecting female politicians worldwide.
But it’s not too late to get the problem under control, and platforms like GitHub have options, Seger says, including intervention at the upload point. “If you put a model on GitHub and GitHub said no, and all hosting platforms said no, for a normal person it becomes harder to get that model.”
Curbing deeply adulterated pornography made with open-source models also relies on policymakers, technology companies, developers and, of course, creators of offensive content themselves.
At least 30 US states also have laws addressing deep-fake pornography, including bans, according to the nonprofit Public Citizen’s legislation tracker, although definitions and policies vary, and some laws only cover minors. Deepfake creators in the UK will also soon feel the force of the law after the government announced on January 7 that creating sexually explicit deepfakes, as well as sharing them, would be criminalised.