GenAI & Child Online Safety: Shaping a Safer Digital Future Together

Gen AI and Child Online Safety

Tech Coalition and Safe Online: Funding innovative research on Gen AI

The Tech Coalition Safe Online Research Fund is going into its fourth year!

Ongoing collaboration between Safe Online and the Tech Coalition will center the Research Fund’s efforts on generative AI , ensuring we make meaningful progress in this critical area. While this year’s funding is at a smaller scale than past years – USD$220K – we are applying a more targeted approach for amplifying impact by honing in on one topic and applying the approaches and structures for industry-researcher engagement that we have jointly built over the past three years.

An Anticipatory Step Supporting Independent Research

The Tech Coalition and Safe Online identified a crucial need as GenAI is rapidly developing, to increase the sparse research landscape on the intersections of GenAI and online child sexual exploitation and abuse (CSEA).

The first project, announced at the Tech Coalition’s UK generative AI briefing, will be led by the University of Kent. The study will build on the work Kent has begun as part of the first cohort of the Research Fund with increased collaboration with tech industry. The focus will be on the proliferation of AI-generated child sexual abuse material (CSAM) and its potential impact on attitudes, norms, and behaviors among those who engage with CSAM. This research will also explore how the ecosystems of prevention and perpetration may respond to these emerging threats.

Youth Voices on AI: Shaping a Safer Digital Future

This project aims to directly engage children and youth in shaping AI development and regulation, particularly around online CSEA prevention. Led by Western Sydney University’s Young and Resilient Research Centre, the initiative will gather youth perspectives on AI, develop a “Youth Constitution for AI,” and inform policymakers about young people’s views on AI-related risks and safeguards.

The core focus of the project is aligning AI development with the values and safety concerns of young users. By understanding how youth perceive AI’s benefits and risks, particularly in relation to CSEA, the project will provide key insights for ethical AI development and governance. This initiative seeks to not only prevent online harms but also foster a generation of young people informed and engaged in shaping the future of AI.

GenAI misuse: Child-centered perspectives on risks and harms

This project, led by SaferNet Brasil, focuses on addressing the rising misuse of generative AI tools to create and share CSAM and self-generated explicit content by teenagers. 

By involving adolescents in semi-structured interviews and participatory workshops, the project aims to create a nuanced understanding of these emerging practices and inform the development of child-centered safety policies. The findings will be shared with tech companies, enabling them to improve safety strategies, provide risk mitigation tools, and address the misuse of AI for harassment and exploitation. The project will also use findings to raise awareness through advocacy, with a focus on preventing online exploitation and fostering child participation in these efforts.

Moving Forward: New Opportunities for Multi-Stakeholder Collaboration

This round of the Research Fund marks a critical development in understanding the benefits for children and young people of genAI technologies as well as the risks associated with AI-generated content, and these projects are kicking off now through the beginning of 2025. The goal is to foster collaboration among key stakeholders—ranging from child safety advocates to law enforcement and tech companies—ensuring that the potential dangers of genAI are met with informed, actionable responses.

See more stories from our family of grantees

Press Release: Survivors of childhood sexual violence seek urgent action to ‘redesign’ the internet to protect children

Press Release: July 17, 2024 [LONDON]: Today, survivors of childhood sexual violence will address the escalating threat to children’s lives posed by online abuse, and call for action to ‘redesign’ the internet and protect children ahead of the first ever global Ministerial on the issue in November 2024. This is a pivotal moment to redefine the digital world for children, ensuring that technology is used for their benefit and never for their harm.

Read More »

Learning, Networking And Visioning A Safe Digital Future:

On 30-31 May 2022, End Violence’s Safe Online initiative brought together over 40 grantees and partners from across the world alongside donors, and industry representatives from the Tech Coalition in a unique opportunity aligned around a common vision to end online child sexual exploitation and abuse (CSEA).

Read More »

Press Release: Survivors and allies demand urgent action to prevent digital harms and keep children safe online

Press Release

Survivors and allies demand urgent action to prevent digital harms and keep children safe online

World leaders urged to seize historic opportunity at the inaugural Global Ministerial on Ending Violence Against Children to make bold commitments to create a safe digital world for children and young people

November 6, 2024 [Bogota, Colombia]:

Today, survivors and allies came together to demand urgent action from world leaders to address the growing threat of online child abuse and ensure a safe digital future for children.

Safe Online, the Brave Movement, and WeProtect Global Alliance, along with 15+ partners, hosted the Satellite event ‘Safe Digital Futures for Children’. The event, held at the margins of the inaugural Global Ministerial Conference on Ending Violence Against Children, aimed to build momentum and urge governments to end all forms of violence against children, including online violence.

See more of our recent updates

Project VIC international

Our grantees Project VIC international Katalyst Countries involved:October 2017 – October 2019 Project VIC International will continue developing its KATALYST programme, which aims to boost countries’ technological and investigative capacity to combat online crimes against children. By working with local law enforcement, KATALYST will improve forensic and victim identification capabilities, and facilitate collaboration between

Read More »

UNICEF Philippines

Our grantees UNICEF Philippines “Project Safe” Countries involved:The Philippines UNICEF Philippines’s “Project Safe” will focus on strengthening systems so children will be protected from the risks and threats in online and offline platforms. It also aims to empower children to demonstrate personal skills and  protective behaviour in identifying and managing

Read More »
Stay in the loop.

Our purpose in detail

We are here to ensure every child and young person grows in to the digital world feeling safe, and is protected from harm.

We support, champion, and invest in innovative partners from the public, private, and third sectors working towards the same objective.

We believe in equipping guardians and young people with the skills to understand and see danger themselves once accessing digital experiences without supervision.

We'd love to have a chat

We're thrilled you're interested in donating to Safe Online - pop in the details below and we will get back to you to set up a discussion.