This policy brief investigates how information disorder exacerbates online gender-based violence (GBV) by incentivising and amplifying the proliferation of harmful Generative AI (GenAI) content on social media platforms such as TikTok, X and Instagram.
The brief and its associated research paper found that participants in the study believed women experience increased online abuse through the use of GenAI tools, and are not confident that existing recourse to curb GBV online were sufficient. Participants had first-hand experience of witnessing non-consensual intimate imagery sharing on platforms, impacts on mental health, digital well-being and ability to equitably participate online. They expressed that GenAI content perpetuates cycles of violence against women, undermining efforts to curb domestic violence and high femicide rates in South Africa.
Lack of trust in the enforcement of community guidelines and existing moderation policies, participants called for transparent regulations that could detect and swiftly remove non-consensual GenAI content. Similarly, this brief argues for a multistakeholder approach to enforcing digital safety, making a case for the revision of the Cybercrimes Act of 2020 to explicitly address GenAI content, and the demonetisation of harmful content within engagement markets.
In doing so, it intends to mitigate the normalisation of violence against women, both online and offline. Without effective legal recourse and multistakeholder intervention, women will continue to be disproportionately affected by GenAI, forced to rely on self-censorship and self-protection measures that hinder their right to equal digital participation.
Summary of key policy recommendations
- Demonetising non-consensual sexual online content.
- Implementing robust content moderation policies.
- Developing transparent and enforceable regulations.
