Tackling age assurance & live streaming of abuse to make the internet safer for children

Safe Online is investing USD $2 million across 10 innovative projects to seed and grow solutions that leverage existing and new technologies to tackle online child sexual exploitation and abuse (CSEA).

The new projects will focus not only on technology solutions, but also include research around the use of tech solutions for the two key areas – age assurance and live streaming of abuse in the online CSEA ecosystem.

These two themes were selected based on rising prioritisation in global discussions of online CSEA  across sectors; upcoming legislation and emerging policy focus globally that raises the urgency of proactively engaging with these challenges; as well as from extensive consultation with diverse experts across the online CSEA and related fields.

"Our understanding of how we approach age assurance and live streaming of abuse in the online CSEA ecosystem is fragmented and often reactive to current trends in digital harms. With these 10 new projects, Safe Online is promoting a unified and proactive approach that focuses on upcoming legislation, new and emerging technology and tackling rapidly evolving threats in the digital space."
- Marija Manojlovic, Director, Safe Online

The 10 organisations, awarded through a rigorous selection process supported by external technical experts, will build on existing tools and research in the space as well as existing evidence, frameworks and networks. These organisations were selected on the relevance of the proposed solution and potential for scale and impact, the soundness of the proposed technology or research, global/regional and national context (including similar projects and risk of duplication), legal basis for implementing the tool or policy relevance for the research, and the type of the organisation and geographic distribution of funds. 

Read more on the projects:

AVID: A global approach to tolerances, frequency, bias and resilience of Age Assurance. 

Lead research on appropriate levels of tolerance for age assurance systems in line with the development of international standards, including developing guidance on the frequency of re-authentication of users & research on bias, presentation attack and system-level attack.

https://www.accscheme.com/ 

Center for Democracy & Technology: CSEA & Social Media Policy

CDT will conduct research and lead multi-stakeholder workshops to address how child sexual exploitation and abuse material (CSEA) is identified and addressed in live-streamed content. Live streaming, which is increasingly important in the use of popular apps particularly among children, presents unique challenges for combatting CSEA content. Given the scale of user generated content on major social media and other online services, companies make use of a range of machine learning techniques to analyze user behavior and user generated video, audio, and text. Understanding the capabilities of such techniques, and how they may be deployed, is crucial for assessing the consequences for children’s safety, privacy, free expression, and other rights. It is also important to understand how these tools can be applied in non-English language settings, and the extent to which these tools can be effectively integrated into trust and safety systems in a way that protects and promotes children’s rights. CDT’s project will (1) assess proposed and existing technologies used to detect and address CSEA in live streamed content, and (2) facilitate dialogue and engagement between researchers, advocates, and policy-makers on how to address this problem.

http://www.cdt.org/ 

Pathfinder Labs: Jaeger

Jaeger is a cutting-edge solution to meet critical objectives to safeguard the vulnerable and bring offenders to justice. 

    • Identify offender profiles and groups.   
    • Capture volatile streaming data automatically. 
    • Alert investigators of ‘at-risk’ streaming sessions. 
    • Provide a unified platform reference. 
    • Facilitate collaborative operations and investigations.

https://pathfinderlabs.nz/

Plan International Australia: Safeguarding young people from the dark path of self-facilitated live streaming of CSEA

Plan International, with ECPAT, will conduct a participatory action research into the ‘value chain’ of live streaming of online child sexual exploitation and abuse in the Philippines, outline livestreaming risk factors, the steps that victims and perpetrators go through, and the perceived “benefits” and consequences of the actions.

http://www.plan.org.au/ 

Rigr AI Limited: Video Summarisation Tool (VST)

Rigr AI is crafting the “VST – Video Summarisation Tool,” tailored for Law Enforcement Agencies (LEA), Non-Governmental Organisations (NGOs), and private sectors. Its primary goal is to identify victims of Child Sexual Exploitation and Abuse (CSEA) content.

https://rigr.ai/ 

AVPA Limited: euCONSENT Phase 2

euCONSENT ASBL is a non-profit NGO that will put into live operation extensions to the eIDAS infrastructure required to deliver its vision for pan-European, open-system, secure and certified interoperable age verification and parental consent to access Information Society Services.

https://euconsent.eu/

T3K.AI: CounterACT - Countering Abuse through Coercion or Trafficking

T3K.AI will adapt their solution CORE, which is specialized on the AI-enabled detection of child sexual abuse material (CSAM) in pictures and videos, to CSAM detection in livestreams. The project will optimise the automatic screening of livestreams to empower Content Providing Companies in their fight against streamed abuse.

https://www.t3k.ai/ 

Yoti: Level Playing Field

Yoti is a digital identity company that makes it safer for people to prove who they are, providing a number of innovative solutions which span reusable Digital ID, age assurance, identity verification and esignatures. Yoti will aim to support data and content regulators in the online child safety area; to review a wider number of sites and achieve the objectives of their online safety regulations. This project will have a global reach and maintain the collaborative approach of Yoti in working with key partners including policy advisers, think tanks, researchers, humanitarian bodies and civil society.

https://www.yoti.com/

Universidad de los Andes (with Aulas en Paz and Te Protejo): AI alert system for the detection and deterrence of CSEA livestreaming in the Latin American context

This project aims to test and improve an Artificial Intelligence alert system for the detection of live streaming of abuses in Latin America. This initiative is based on a tool previously created by Universidad de los Andes through a project financed by Safe Online. By using information from conversations that take place between aggressors, the research team will analyse patterns and commonalities in these interactions in order to tackle this crime in a faster and more strategic way. Through a partnership established with hotlines and organisations that work on the prevention of CSEA in Colombia, Argentina, Chile, Mexico, Ecuador and Costa Rica, this alert system seeks to become a regional tool for the prevention of this crime.

WebIQ: OSAgE: Open Source Age Estimation

To help keep children safe online and enable authorities to enforce age legislations on online platforms, Web-IQ uses AI to create an age assurance tool, tailored to Safe Online’s mission. The tool aims to verify a user’s age by scanning open information available online, helping protect kids from online dangers of sexual abuse.

https://web-iq.com 

Image: © UNICEF:UNI316261

See more stories from our family of grantees

Hear from our grantee: Age Check Certification Scheme Why Age Assurance is key to empowering children in the digital age

Hosted in collaboration with the British Standards Institution (BSI), this groundbreaking summit united over 700 global stakeholders to affirm a bold declaration: “Age Assurance Can Be Done!” With a focus on safeguarding children online, the summit produced a landmark Communique on Age Assurance, outlining principles like prioritising individual rights and data minimisation. Safe Online grantee, Age Assurance Certification Scheme, explains why age assurance is critical, the role of the first-ever Communique on Age Assurance and the way forward from this landmark Summit.

Read More »
Stay in the loop.

Our purpose in detail

We are here to ensure every child and young person grows in to the digital world feeling safe, and is protected from harm.

We support, champion, and invest in innovative partners from the public, private, and third sectors working towards the same objective.

We believe in equipping guardians and young people with the skills to understand and see danger themselves once accessing digital experiences without supervision.

We'd love to have a chat

We're thrilled you're interested in donating to Safe Online - pop in the details below and we will get back to you to set up a discussion.