Deepfake scandal at high school calls lawmakers to action

Deepfake technology is not a new concept, but students in a New Jersey high school have become victims of their classmates using this AI technology. Several boys from the school had been using the images of their female classmates. One victim, Francesca, a 14-year-old student, reported that she and over 30 other teenagers appeared in these pictures, which were distributed around the school without their consent. However, as per an email from the school, the actual number of victims appears to be less than 30.

Francesca then reached out to various lawmakers and received responses from U.S. Representative Joe Morelle of New York, U.S. Representative Tom Kean Jr. of New Jersey, and New Jersey State Senator Jon Bramnick, among others. Thanks to her efforts, new legislative proposals have been introduced recently, with an emphasis on labeling and identifying AI-generated content and preventing the inclusion of underage material. Additionally, the girl has created a website to help those affected by deepfake technology.

I just realized that day [that] I need to speak out, because I really think this isn’t okay. This is such a new technology that people don’t really know about and don’t really know how to protect themselves against.

- Francesca

Joe Morelle drafted a bill in 2022 to criminalize the production of deepfakes without consent, but it lacked sufficient support and ultimately failed. Other initiatives have also been introduced to regulate deepfakes, such as the DEEPFAKES Accountability Act of 2019 by Representative Yvette Clarke, which requires all deepfakes to be watermarked by their authors.

Since then, the creation of deepfakes has become easier as artificial intelligence has developed significantly. With minimal image material, real-looking fakes are achievable. For instance, Lensa, a generative AI app, enables people to be undressed amongst others.

We’ve just given high school boys the mother of all nuclear weapons for them, which is to be able to create porn with [a single image] of whoever they want. And of course, they’re doing it.

- Hany Farid, Professor at the University of California, Berkeley

Joe Morelle's bill would allow criminal prosecution of creators and distributors of deepfake without consent, classifying it as a federal crime.