One in four women in the public sphere who experience online violence related to their professional activity has already faced attacks generated or amplified by artificial intelligence. These are the findings of a new global study by UN Women conducted across 119 countries.
This trend was discussed during the panel “AI-assisted gender-based violence: the chilling escalation of online abuse against women in the public sphere” at the International Journalism Festival in Perugia, attended by the team from Women in Media NGO.
What does this mean for women journalists, and how should we respond to this new wave of AI-enabled violence? We explain below.
GENDER BASED VIOLENCE: 2 TIMES MORE WOMEN IN 5 YEARS
‘The era of AI-assisted online violence is no longer looming. It has arrived. And it is reshaping the threat landscape for women who work in the public sphere around the world’, starts Julie Posetti.
She presents new research from a global study of women journalists, human rights defenders, and activists from 119 countries. It reveals the extent to which generative AI is being weaponised to produce abusive content – in a multitude of forms – at scale in a climate of rising authoritarianism.
Julie Posetti is not only the moderator of the panel, but also the director of Information Integrity Initiative and one of the authors of the research. Julie (PhD) is a multi award-winning internationally published Australian-British journalist and academic, who focuses on gender-based online violence, public interest media, and civic efforts to counter disinformation. And the research is produced for UN Women by the Information Integrity Initiative in partnership with TheNerve – a digital forensics lab established by the Nobel Peace Prize winning journalist, one of the panelists – Maria Ressa.
Maria Ressa herself experienced and continues to be subdued to an extremely large number of online violence campaigns. In 2016 at one point she was receiving 90 hate messages an hour. It was text-based abuse. New research shows how AI deep fakes and nudification apps exponentially increase these threats.
‘At first I thought people just don’t like what I’m doing. You think it’s about you, it’s not. These narrative information operations look for your vulnerabilities. Every time we published an investigation – they published a meme of my face (I have dry skin, I have eczema) with testicles on it. The platforms did not take it down. And then these memes evolved. We traced through Nerve a Russian scam network pushing out deep fakes of me’, Maria Ressa shares.
According to this new analysis, nearly one in four (24%) of the 70% of respondents who reported experiencing online violence in the course of their work identified abuse that was generated or amplified by AI tools, with writers or other public communicators, such as social media influencers, reporting the highest exposure to AI-assisted online violence at 30%.
Julie Posetti points to an important notion: AI amplified an already existent threat, not created a new one. It’s just the platforms ignored all previous attempts to hold them accountable.
‘We can’t treat these AI-related findings as isolated statistics. They exist amid broadening online violence against women in public life. They are also situated within a wider and deeply unsettling pattern – the vanishing boundary between online violence and offline harm’, she insists.
Four in ten (40.9%) women in the survey reported experiencing offline attacks, abuse or harassment that they linked to online violence. This includes physical assault, stalking, swatting and verbal harassment.
‘The data confirms what survivors have been telling us for years: digital violence is not “virtual” at all. In fact, it is often only the first act in a cycle of escalating harm. Women from Syria, Lebanon, Ukraine, who are experiencing the most horrific impact of this violence, often are undermined and forgotten, because it’s not ‘real world violence’. Well, I’m here to tell you that the data says; it is. And it continues to be a massive risk’, Julie adds.
Moreover, the number of women who report experiencing attacks is growing over the years. For women journalists, the trend is especially stark.
‘In a comparable 2020 survey, 20% of respondents reported experiencing offline attacks associated with online violence. But five years later, that figure has more than doubled to 42%. This dangerous trajectory should be a wake-up call for news organisations, governments and big tech companies alike’, the researcher points out.
Since the public launch of free, widely accessible generative AI tools such as ChatGPT at the end of 2022, the barriers to entry and cost of producing sexually explicit deepfake videos, gendered disinformation, and other forms of gender-based online violence have been significantly reduced. Meanwhile, the speed of distribution has intensified. The result is a digital landscape in which harmful, misogynistic content can be generated rapidly by anyone with a smart phone and access to a generative AI chatbot. Social media algorithms, meanwhile, are tuned to boost the reach of the hateful and abusive material, which then proliferates. And it can generate considerable personal, political and often financial gains for the perpetrators and facilitators, including technology companies.
NOT ONLY WOMEN, BUT MOSTLY WOMEN
A senior researcher with the Information Integrity Initiative and a PhD candidate at Columbia Journalism School Kaylee Williams shares the data on Grok and Twitter (X). Users of a mainstream platform used embedded Gen-AI bot to create non-consensual intimate imagery. In 11 days between December and January this year Grok generated 3 million of these sexualized images, 2300 of which depicted minors.
‘It mostly targeted women in politics, journalists and activists. Because every time something was shared about their work, someone could come and ask to create a nude image (often depicting abuse, rape and violence) of her right in the comments. But this technology also targets children’, she poses.
A UK MP who was campaigning for regulation of nudification apps had a deepfake created with the image of her being chloroformed and prepared for rape, which went viral, Julie Posetti adds.
‘You have the evolution of the manosphere in combination with these tools to not just systematically abuse women at scale, but these technologies also facilitate the rollback of the very hard-won rights to gender equality’, she says.
The previous study on gender based online violence for UNESCO from 2020 was conducted in 15 countries.
‘In more conservative countries we see a phenomenon of the shaming of men who are connected to the women targeted by online and offline attacks. And the creation of a risk profile of the women being targeted designed to get their parents to convince them that they should withdraw’, explains Julie Posetti.
In many ways we see these attacks as geopolitical, strategic silencing, which is designed to chill critical reporting, human rights work.
For example, Maria Ressa recalls, Leila De Lima, who was senator of the Philippines from 2016 to 2022, was targeted with fake pornographic photos and videos using her imagery. And then shamed in the House of Representatives for it and jailed unjustly for almost 7 years.
‘This is free speech used to stifle free speech and the attack on women, first sexist, now misogynistic, is designed to push women out of public life’, Maria Ressa assures.
WHAT CAN WE DO?
Recent research highlights AI both as a driver of disinformation and as a potential solution, powering synthetic content detection systems and counter-measures. But there’s limited evidence of how effective these detection tools are.
We need to think about the ways we stop it before it starts. Safety by design – regulation of platforms. Having the codes of conduct and the standards in place.
‘We’re working on creating tools and prevention models that can help find more information about the drivers of such perpetration. In order to change the narrative’, the Chief of the Ending Violence against Women Section at UN-Women in New York Kalliopi Mingeirou explains.
Researching and creating these tools takes time, but there is a terribly urgent need for regulation at a state and international level, Julie Posetti argues.
‘There’s a desperate need for stronger tools to identify, monitor, report and repel AI-assisted attacks. And legal and regulatory mechanisms must be established that require platforms and AI developers to prevent their technologies from being deployed to undermine women’s rights’, she says.
Demanding real regulation and using strategic legislation are two main instruments civic society should work with, Maria Rossi assures.
‘It all comes to power and money. Who is responsible? Tech companies that are making more money and gaining more power. And goes back to the business model. Surveillance capitalism which has commodified our humanity. So let’s move upstream and push for the laws that should protect us’, she proposes.
Kalliopi Mingeirou also points out how fueling the misogyny shapes younger men and boys’ attitudes and behaviours. And it’s not just violence against women and girls.
‘It’s very linked with violent extremism and extreme radicalization that’s compounded with racism and homophobia’, she points out.
One of the researchers in the audience, Sally Hammoud with PhD in AI, comms and media notes: but how can we hope for legislation when even the US president has been using and normalizing deep fakes.
Earlier panelists recalled that Elon Musk himself was one of those who asked Grok to nudify pictures of women on Twitter (X). So it’s really hard to fight against it, when people in power themselves abuse this harmful technology.
‘Tech companies only grow their power and have much more influence on legislation. So if it all comes to educating, civic society and journalism, there are countries that have different problems. I’m from Lebanon. And I had a conversation with the ministry on AI. They were just pushing for partnerships with tech companies without thinking of anything related to digital safety or education’, Sally Hammoud shares.
A senior researcher with the Information Integrity Initiative Kaylee Williams suggests that, unfortunately, we need to go back to the feminist fundamentals.
‘The importance of public awareness about consent, building systems that justify and protect women bodily autonomy. These are things I came up thinking we agreed upon and I’m finding now in my research that, perhaps now it’s backsliding, perhaps that was never true’, she shares.
During the session, the Ukrainian perspective was brought into the discussion by Liza Kuzmenko, head of the NGO Women in Media. She emphasized that in Ukraine, the women media workers most frequently targeted by online harassment are war correspondents, anti-corruption journalists, and writers covering LGBTQ+ issues and feminism. Against this backdrop, she posed the following question to the panelists: “If platforms and AI developers already know that their technologies are multiplying violence, at what point does their inaction become complicity?”
‘It’s important to understand that there cannot be a more dangerous environment for an online violence campaign than directed at a war correspondent in a war zone. And Ukraine is a prime example of that’, Julie Posetti starts addressing the question. ‘Your point whether or not these companies recognize they are complicit is best illustrated by the ways in which Meta (Facebook) has fought and continues to fight against an inquiry at the UN level’.
Facebook had a determining role, in the words of the UN investigators, Julie claims, in the Myanmar genocide. Also known as the Rohingya genocide it’s a series of ongoing persecutions and killings of the Muslim Rohingya people by the Tatmadaw (armed forces of Myanmar). The genocide has consisted of two phases to date: the first was a military crackdown that occurred from October 2016 to January 2017, and the second has been occurring since August 2017.
“There can be absolutely no doubt that they are aware that they are, in fact, facilitating and, arguably, complicit in the risks and the impact of these technologies on extremely vulnerable communities. The ways in which we can help one another include solidarity. Include creating safe physical spaces, where we can, trying not to use these platforms, organize for legal and regulatory reforms”, – Julie Posetti concludes.
According to the study by NGO Women in Media and UNESCO, When Artificial Intelligence Turns Hostile: Gendered Threats Against Ukrainian Women Journalists, 7% of Ukrainian women journalists have already experienced AI-generated online attacks, while another 16% have witnessed similar attacks targeting their colleagues.