“Spotlight” is a newsletter about underreported cultural trends and news from around the world, emailed to subscribers every Monday and Wednesday. Sign up here.
Artificial Intelligence (AI) has broken into the mainstream consciousness at long last. Tools like ChatGPT are fascinating laypeople with their power and breadth of knowledge, propelling the valuation of its company OpenAI to almost $30 billion and spawning numerous memes of people’s eventual replacement by chatbots.
It’s about time too, as it makes it more likely for societies as a whole to have broader conversations about the vast potential as well as the tangible risks of AI tools becoming part of the mainstream. Nothing illustrates this more than a scandal that broke out last week that has taken the world of streaming video games by storm.
The story begins with a Twitch streamer called Atrioc. Twitch is a live-streaming platform that is primarily centered around video games and e-sports competitions. It boasts by some estimates about 140 million monthly users (content creators or streamers often have both free and paid subscription options for their followers), many of them teenagers and those in their 20s.
Atrioc is the alias of Brandon Ewing, a California-based streamer with almost 320,000 followers on the platform and an additional 115,000 on Twitter. The scandal began when, during an hourslong stream, Atrioc tabbed out for a split second from the main window on his computer. The video of the stream was later uploaded to the site, and keen-eyed viewers who combed through it noticed that one of the windows that were revealed when he tabbed out featured a website that sold deepfake pornography.
Deepfakes are essentially AI-generated photographs and videos that transpose someone’s likeness on an existing piece of media, creating a deeper form of fakery meant to trick viewers. You may have seen versions of this where Barack Obama or Tom Cruise appear to say things they didn’t say in real life.
Fakes of a similar sort have long existed online, but machine learning and AI that analyze and synthesize facial musculature and movements, vocal tonality and other human forms of expression can create far more convincing fakes. Numerous examples abound online of deepfakes of politicians and celebrities made to say inflammatory or false statements, and they have enormous disruptive potential for spreading disinformation as the technology develops.
But another popular use of AI deepfakes in the dark recesses of the internet is the use of the technology to create pornography by transposing the images of ordinary individuals or celebrities on existing pornographic media, creating videos and images of them performing sex acts.
Atrioc was apparently using a subscription service for the creation of deepfake pornography. Not only that, but it was revealed that the images he had purchased were deepfakes of other female streamers that he had known and collaborated with, including a streamer called Maya Higa and another whose alias is Pokimane.
Pokimane responded to the saga in a succinct tweet: “Stop sexualizing people without their consent.”
Higa, however, posted a longer statement that compared the deepfakes of her to a sexual assault she had endured in 2018, and which went to the heart of the nature of this new form of violation made possible by advances in AI.
“In 2018, I was inebriated at a party and I was used for a man’s sexual gratification without my consent,” she wrote. “Today, I have been used by hundreds of men for sexual gratification without my consent. The world calls my 2018 experience rape. The world is debating over the validity of my experience today. The debate over our experience as women in this is, not shockingly, amongst men. None of you should care or listen to what any male streamer’s ‘take’ is on how we feel.”
Higa said her primary interest has been in conservation work and raising money for a nonprofit animal sanctuary that she founded when she was 22 and had created no sexual content during her three years streaming on Twitch.
“Despite this, my face was stolen so men could make me into a sexual object to use for themselves,” she added. “If anyone doesn’t think it’s a big deal that MY NAME is in headlines where thousands of people are commenting on the sexualization of MY BODY against MY WILL, you are the problem. This situation makes me feel disgusting, vulnerable, nauseous, and violated — and all of these feelings are far too familiar to me. This is not your debate. Stop acting like it is.”
Atrioc, the streamer in question, published a written statement on Twitter apologizing for his actions: “First and foremost I want to specifically apologize to Maya and Pokimane. You were both in the screenshot that spread around the internet. Your names were dragged into it and you were sexualized against your will. You have both succeeded in this industry in spite of all the blatant unfounded sexist attempts to hurt or assassinate your character in a male dominated space. And now I’m another guy on that long list.”
He added that he would step away from streaming and fund the efforts of women working to take down pornography featuring them on the internet nonconsensually and excoriated supporters who tried to exonerate him on social media. In a tearful video on Twitch alongside his partner, he said that he was deeply embarrassed by his actions and that his actions were a one-off and did not represent a pattern of behavior. He said the category of behavior of creating and consuming deepfake porn was detestable. He described how his interest in AI had sparked an interest in deepfakes and how it had led to him clicking on ads for deepfake pornography that led to the incident.
Atrioc’s case is one of a recent number of cases involving sexual harassment allegations among high-profile streamers that has pierced an industry that is male-dominated and whose audiences can often turn toxic toward women. Twitch streamer Mizkif, who has 2 million followers on Twitch and nearly 650,000 on Twitter, briefly stopped streaming and apologized after allegations that he had downplayed the experience of a female streamer who had been sexually assaulted.
But Atrioc’s case itself offers up one facet of the challenges presented by AI’s inexorable entrance into the mainstream of public consciousness and pop culture. As we meme about chatbots replacing journalists and question the accuracy of the data they spew back in response to our questions, a symptom of the misinformation and disinformation plaguing today’s internet, it is worth contemplating the other aspects of this confounding technology — the invention and proliferation of new ways to harass, assault, violate and victimize. As always, women will bear the brunt of it.
Sign up to our mailing list to receive our stories in your inbox.