Join us

Category: Our blog

Posted on

Tackling age assurance & live streaming of abuse to make the internet safer for children

Tackling age assurance & live streaming of abuse to make the internet safer for children

Safe Online is investing USD $2 million across 10 innovative projects to seed and grow solutions that leverage existing and new technologies to tackle online child sexual exploitation and abuse (CSEA).

The new projects will focus not only on technology solutions, but also include research around the use of tech solutions for the two key areas – age assurance and live streaming of abuse in the online CSEA ecosystem.

These two themes were selected based on rising prioritisation in global discussions of online CSEA  across sectors; upcoming legislation and emerging policy focus globally that raises the urgency of proactively engaging with these challenges; as well as from extensive consultation with diverse experts across the online CSEA and related fields.

"Our understanding of how we approach age assurance and live streaming of abuse in the online CSEA ecosystem is fragmented and often reactive to current trends in digital harms. With these 10 new projects, Safe Online is promoting a unified and proactive approach that focuses on upcoming legislation, new and emerging technology and tackling rapidly evolving threats in the digital space."
- Marija Manojlovic, Director, Safe Online

The 10 organisations, awarded through a rigorous selection process supported by external technical experts, will build on existing tools and research in the space as well as existing evidence, frameworks and networks. These organisations were selected on the relevance of the proposed solution and potential for scale and impact, the soundness of the proposed technology or research, global/regional and national context (including similar projects and risk of duplication), legal basis for implementing the tool or policy relevance for the research, and the type of the organisation and geographic distribution of funds. 

Read more on the projects:

AVID: A global approach to tolerances, frequency, bias and resilience of Age Assurance. 

Lead research on appropriate levels of tolerance for age assurance systems in line with the development of international standards, including developing guidance on the frequency of re-authentication of users & research on bias, presentation attack and system-level attack.

https://www.accscheme.com/ 

Center for Democracy & Technology: CSEA & Social Media Policy

CDT will conduct research and lead multi-stakeholder workshops to address how child sexual exploitation and abuse material (CSEA) is identified and addressed in live-streamed content. Live streaming, which is increasingly important in the use of popular apps particularly among children, presents unique challenges for combatting CSEA content. Given the scale of user generated content on major social media and other online services, companies make use of a range of machine learning techniques to analyze user behavior and user generated video, audio, and text. Understanding the capabilities of such techniques, and how they may be deployed, is crucial for assessing the consequences for children’s safety, privacy, free expression, and other rights. It is also important to understand how these tools can be applied in non-English language settings, and the extent to which these tools can be effectively integrated into trust and safety systems in a way that protects and promotes children’s rights. CDT’s project will (1) assess proposed and existing technologies used to detect and address CSEA in live streamed content, and (2) facilitate dialogue and engagement between researchers, advocates, and policy-makers on how to address this problem.

http://www.cdt.org/ 

Pathfinder Labs: Jaeger

Jaeger is a cutting-edge solution to meet critical objectives to safeguard the vulnerable and bring offenders to justice. 

    • Identify offender profiles and groups.   
    • Capture volatile streaming data automatically. 
    • Alert investigators of ‘at-risk’ streaming sessions. 
    • Provide a unified platform reference. 
    • Facilitate collaborative operations and investigations.

https://pathfinderlabs.nz/

Plan International Australia: Safeguarding young people from the dark path of self-facilitated live streaming of CSEA

Plan International, with ECPAT, will conduct a participatory action research into the ‘value chain’ of live streaming of online child sexual exploitation and abuse in the Philippines, outline livestreaming risk factors, the steps that victims and perpetrators go through, and the perceived “benefits” and consequences of the actions.

http://www.plan.org.au/ 

Rigr AI Limited: Video Summarisation Tool (VST)

Rigr AI is crafting the “VST – Video Summarisation Tool,” tailored for Law Enforcement Agencies (LEA), Non-Governmental Organisations (NGOs), and private sectors. Its primary goal is to identify victims of Child Sexual Exploitation and Abuse (CSEA) content.

https://rigr.ai/ 

AVPA Limited: euCONSENT Phase 2

euCONSENT ASBL is a non-profit NGO that will put into live operation extensions to the eIDAS infrastructure required to deliver its vision for pan-European, open-system, secure and certified interoperable age verification and parental consent to access Information Society Services.

https://euconsent.eu/

T3K.AI: CounterACT - Countering Abuse through Coercion or Trafficking

T3K.AI will adapt their solution CORE, which is specialized on the AI-enabled detection of child sexual abuse material (CSAM) in pictures and videos, to CSAM detection in livestreams. The project will optimise the automatic screening of livestreams to empower Content Providing Companies in their fight against streamed abuse.

https://www.t3k.ai/ 

Yoti: Level Playing Field

Yoti is a digital identity company that makes it safer for people to prove who they are, providing a number of innovative solutions which span reusable Digital ID, age assurance, identity verification and esignatures. Yoti will aim to support data and content regulators in the online child safety area; to review a wider number of sites and achieve the objectives of their online safety regulations. This project will have a global reach and maintain the collaborative approach of Yoti in working with key partners including policy advisers, think tanks, researchers, humanitarian bodies and civil society.

https://www.yoti.com/

Universidad de los Andes (with Aulas en Paz and Te Protejo): AI alert system for the detection and deterrence of CSEA livestreaming in the Latin American context

This project aims to test and improve an Artificial Intelligence alert system for the detection of live streaming of abuses in Latin America. This initiative is based on a tool previously created by Universidad de los Andes through a project financed by Safe Online. By using information from conversations that take place between aggressors, the research team will analyse patterns and commonalities in these interactions in order to tackle this crime in a faster and more strategic way. Through a partnership established with hotlines and organisations that work on the prevention of CSEA in Colombia, Argentina, Chile, Mexico, Ecuador and Costa Rica, this alert system seeks to become a regional tool for the prevention of this crime.

WebIQ: OSAgE: Open Source Age Estimation

To help keep children safe online and enable authorities to enforce age legislations on online platforms, Web-IQ uses AI to create an age assurance tool, tailored to Safe Online’s mission. The tool aims to verify a user’s age by scanning open information available online, helping protect kids from online dangers of sexual abuse.

https://web-iq.com 

Image: © UNICEF:UNI316261

See more stories from our family of grantees
Posted on

Meet the new Safe Online visual identity!

Meet the new Safe Online brand!

We are thrilled to share a major milestone in the journey of Safe Online and our mission of shaping a safer digital world for all children and young people everywhere.

In line with the evolving technological landscape and the ever-changing needs of young people worldwide, we are establishing ourselves as a global funding vehicle and renewing our commitment to our core purpose; to protect and empower children and young people throughout the digital world.

Alongside our communications partner, Strategic Agenda, we have developed a powerful new brand that speaks directly to our purpose, mission, and vision to launch our evolution from the End Violence partnership.  

Why?

This crucial step allows us to wholeheartedly devote our resources, expertise, and unwavering passion to the cause of ensuring the online safety and empowerment of our most vulnerable technology users.

We believe that every child and young person deserves an online experience where they can benefit from all it has to offer without fear of harm, abuse or age-inappropriate content and experiences. The digital world is an integral part of their lives, and we owe it to them to ensure digital spaces meet the standards we set for the physical world.

Our new mandate and sole focus allows us to have a greater impact and broader reach than ever before, and strengthens our position as a network and advocate at the heart of online safety. 

As we embark on the new chapter of Safe Online, we renew our promise to create a safer digital future for children and young people. Our new identity is more than a brand: it's a commitment to the well-being of every child and young person who turns to the digital world to explore, create, and dream. We will ensure that their journey online is enriching, inspiring, and, above all, safe.
- Marija Manojlovic, Director, Safe Online
Safeonline's impact so far

What will change?

We will continue to invest in research, systems, and tools developed by innovative partners focused on online safety.

We will use the learnings from our investments to build an evidence base on what works to tackle abuse online, as well as to advocate to prioritise child online safety in business practices, policies and budgets of governments and industry alike. We are now able to dedicate more time, support, and resources to new and existing partners. 

Finally, a huge thank you to all who worked on, and with, the End Violence partnership over the past 7 years. The extraordinary commitment and professionalism of all during the duration of the partnership and the transition period has helped to build the exciting and ambitious path before us now. 

Posted on

Tech Coalition Safe Online Research Fund announces additional funding of US $500k to select existing grantees for research extension, product development & innovation

Tech Coalition Safe Online Research Fund announces additional funding of US $500k to select existing grantees for research extension, product development & innovation

The Tech Coalition Safe Online Research Fund is granting 4 awards to further accelerate the work of organisations from the first cohort of grantees. The additional funds will be for a duration of 12 months and will support projects to extend research to applications such as piloting solutions, technical collaboration and innovation efforts.   

The boost in funding is a move towards promoting more real-world application of independent research – and strengthening application of research to product and service development within the tech industry. Building on the existing research grants, the Fund aims to enhance the community of practice for online CSEA research to support capacity across the ecosystem in tackling digital harms against children.

Read more on the innovative projects that have been awarded the boost funding below:

Dublin City University: Understanding Online Child Sexual Exploitation and Abuse in the Philippines

The additional funding will help translate findings from a white paper developed in the first phase of the project to practical policy changes and recommendations to improve detection and reporting of online CSEA-related transactions on financial platforms – as well as refining a typology to support enhanced online CSEA-related payments detection. The project will also develop and pilot a public awareness campaign (online and offline) in the Philippines alongside key stakeholders to develop greater community understanding of the scope and harm of online CSEA to children, reduce associated stigma and encourage reporting – thus enabling better detection of the crime as well as deterring potential offenders at the level of online CSEA-affected communities.

Technological University Dublin: N-Light Discovering Child Grooming patterns

In the first phase, the N-Light project gathered and analysed data from the national child agency (ISPCC), national Hotline (Hotline.ie) and Dark web CSAM text posts. CSAM posts were identified and visual explorations were developed to allow investigation. The new funding will support the conversion of the static models and analyses into a deployed software tool, with a wider user base. This will allow agencies to analyse the most recent data to spot grooming techniques and correlations over relevant timelines between child victim reports and groomer activity discussions; extend into more sophisticated analyses including the ability to profile messages coming from child victims; deploy the tools to their partners and beyond; and adapt the tool such that other hotline/ child agency partners can take use the tool to analyse their own data.

University of Kent: Understanding and improving help seeking by people at risk of online child sexual exploitation and abuse

The findings from the first phase of the project demonstrate ways to further optimise existing ‘get help’ messaging to promote not only deterrence but importantly, help seeking. Furthermore, findings show alternative messages as well as reframed industry-typical messages that are perceived as more effective promoters of help seeking. These are findings from tightly controlled studies. To transform industry practice, these findings need to be implemented “in the wild” of the internet ecosystem. Through partnering with industry collaborators, real-world links can be established between message content and click-throughs to helplines and other prevention resources. The new funding will allow us greater collaboration with tech organisations to pilot alternative warning messages and to track their impact and through this to translate our existing work into industry-wide solutions promoting greater engagement with help resources. 

University of Middlesex: Invisible Risks

This funding would be used to further develop and expand the intervention prototype developed in the initial funding to enhance the wellbeing of content moderators addressing child sexual abuse and exploitation. The current research project helped develop skeleton versions of three online intervention modules: sleep hygiene, managing anxiety and staying positive. However, the evidence has identified a number of other areas that need development including, but not limited to, managing intrusive thoughts, maintaining positive relationships and maintaining a work/life balance. The additional funding will provide the opportunity to enhance and expand these modules even further, resulting in a highly effective online platform that can significantly support content moderators in their self-driven psychological wellbeing. 

Image: © UNICEF:UN0582709

See more stories from our family of grantees
Posted on

71 member states make first of its kind call to action to fight online violence against children

71 Un Member States Make First-of-its-kind Call to Action to Fight Online Violence Against Children

71 Member States of the United Nations have come together to make the very first call to action statement urging increased and fast action to remove known child sexual abuse materials online.

Online child sexual exploitation and abuse is growing significantly around the world and is increasingly involving younger children. In 2022, the US-based National Centre for Missing and Exploited Children (NCMEC) received 32 million reports (up from 29 million from 2021) of suspected child sexual exploitation.  

The first-ever call to action by the Member States directly recognises that governments and the private sector have a responsibility to take action to keep children safe from all forms of sexual exploitation and abuse, and points to the fact that law makers are rightfully concerned and are calling for greater urgency in our efforts to secure a safe digital environment for children. 

"Safe Online welcomes the ‘first of its kind’ call to action by over 70 Member States of the United Nations. The message from world leaders is clear – the digital space has to be rid of child sexual abuse material. We believe that the global call to action will signal the need for urgent concerted and decisive action by Governments and other stakeholders."
- Marija Manojlovic, Director, Safe Online

In the call to action, all the countries acknowledge the grave issue of child sexual exploitation and abuse material and how this puts children at risk of further sexual exploitation and abuse, including the risk of the widespread dissemination of this material. The statement recognises the stigma and long-lasting trauma caused to survivors, including revictimisation every time material is spread online.

It calls for:

    • urgent action by governments, internet service providers and other actors to protect children from online sexual exploitation and abuse
    • facilitation of dialogue between the different entities and sectors required for an effective response
    • the need for common data sources and knowledge of known child sexual abuse materials that makes detecting, reporting and removing materials easier
    • increasing public awareness of the serious nature of child sexual exploitation.

The call to action is informed by expert discussions hosted by the UN Office on Drugs and Crime (UNODC), supported by the UK government. These discussions brought together a wide range of experts and activists, including from child protection charities, the public sector, the private sector and the legal and financial sectors.

This growing global consensus on the issue of taking urgent steps to tackle online harms against children follows on the heels of other positive legislations to keep children safe from online harm. Recent legislative action in countries like the UK, US and Australia are creating a blueprint for online safety for children through new mandatory rules, robust safeguards and a strong enforcement capacity.

In 2021, G7 governments announced a set of ground-breaking commitments to combat online CSEA and highlighted End Violence as key partner to help make the internet safe for children. In a historic move, the UN Convention on the Rights of the Child (UNCRC) adopted General Comment No. 25, marking the first instance of including the digital rights of children in its framework.

Most recently, in the EU, the proposed new legislation presents an exceptional opportunity to set high standards to protect children online through policy action, which is key in ensuring a safe digital environment. This leadership of the EU will have a positive impact in other regions of the world. Safe Online is part of a growing movement of child protection organisations supporting the #ChildSafetyOnlineNow campaign to raise awareness and generate momentum around tackling online CSEA.

The Safe Online initiative has been drawing attention to the need for aligned and decisive action by governments and tech companies in order to drive change urgently needed for children. In fact, our policy call for a safe internet for children outlines three key steps for governments and industries.

Our work at the global, regional and national levels in over 80 countries has brought to light new knowledge and evidence on what works – especially through our Disrupting Harm project and the ground-breaking work that we do with the Tech Coalition Research Fund. Our partnerships with grantees on the ground demonstrate the need for strong systems by focusing on strong regulatory and policy frameworks.

Learn more about the call to action here.

LIST OF SIGNATORY COUNTRIES

Albania, Algeria, Andorra, Armenia, Argentina, Australia, Bosnia and Herzegovina, Brazil, Canada, Chile, Colombia, Costa Rica, Ecuador, Egypt, El Salvador, Ghana, Guatemala, Honduras, Iceland, India, Japan, Kenya, Liechtenstein, Mexico, Moldova, Morocco, New Zealand, Norway, Pakistan, Paraguay, Peru, Philippines, Singapore, South Africa, Sudan, Switzerland, Thailand, Türkiye, Ukraine, United Kingdom, United States of America, Uruguay, Vietnam, Yemen and the European Union and its 27 member states (Austria, Belgium, Bulgaria, Croatia, Republic of Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain and Sweden) 

See more of our recent updates
Posted on

Kenya launches groundbreaking training handbook to combat online child sexual exploitation and abuse

Kenya Launches Groundbreaking Training Handbook to Combat Online Child Sexual Exploitation and Abuse

Kenya's National Council on the Administration of Justice (NCAJ) has launched a training handbook on the investigation and prosecution of online child sexual exploitation and abuse (OCSEA) in Kenya.

The handbook was developed with technical support from UNICEF Kenya, a grantee of the Safe Online Initiative at the End Violence Partnership, and launched by the Hon. Lady Justice Martha K. Koome, Chief Justice and President of the Supreme Court of Kenya & Chairperson, National Council on the Administration of Justice.

Marija Manojlovic, Safe Online Director, spoke at the launch event and highlighted the need for ensuring effective justice for children. “There is an urgent need to streamline the justice process for children. When implemented well, for example via ‘one-stop’ centres, it can provide a child-friendly setting for child victims, and facilitate collaboration and coordination between support services”, she said. 

The handbook is a huge step forward in training officers in the justice system on handling OCSEA cases in the child’s best interests. “The training package is a major step in implementing the Children Act. It will help in enforcing laws such as the Sexual Offences Act, which are designed to protect children online,” said Dominic Stolarow, UNICEF Kenya’s representative at the launch event. 

Safe Online has been supporting key actors in Kenya to ensure:

    • frontline workers have the knowledge and skills to prevent and respond to online CSEA
    • improved laws and standards for quality services, including the Kenya Anti-Human Trafficking and Child Protection Unit that manages cases of online CSEA 
    • community outreach on online safety for children, caregivers and teachers.

Safe Online is proud to be a partner in Kenya’s journey towards a safe digital future for children.

Read the handbook here and view the work of Safe Online here. 

See more of our recent updates

Tech Coalition Safe Online Research Fund announces additional funding of US $500k to select existing grantees for research extension, product development & innovation

October 20, 2023

The Tech Coalition Safe Online Research Fund is granting 4 awards to further accelerate the work of organisations from the first cohort of grantees. The additional funds will be for a duration of 12 months and will support projects to extend research to applications such as piloting solutions, technical collaboration and innovation efforts.

Read More »
Posted on

Safe online invests $4 million to generate evidence on tackling online child sexual exploitation and abuse

Safe Online Invests $4 Million to Generate Evidence on Tackling Online Child Sexual Exploitation and Abuse

The Safe Online portfolio, which invests in cutting edge solutions and knowledge for the online safety of children, has grown manifold over the last six years and has so far reached $76 million in investments focusing on 95 projects with impact in over 80 countries globally. The current targeted funding round builds on learnings generated during previous rounds and offers a unique opportunity to use results reported by grantees to understand the impact of programmes and tools we are funding, as well as identify lessons from the ground on how the change happens. 

 

From solutions for the tech industry and policies for governments to strengthen online safety to resources and digital platforms that help children stay safe online, this funding round covers a range of diverse and critical projects.  

 

It will enhance the knowledge and evidence base to support the scale-up of tested approaches and solutions and also provide crucial insights to stakeholders in the larger online CSEA ecosystem.

Grantee projects have been selected from existing and previously funded Safe Online projects based on their potential for impact and scale. The additional $1.5M in the next quarter will be invested to evaluate these project interventions with a focus on relevance, effectiveness, scalability and sustainability. Safe Online was supported by the Evaluation Advisory Group in the robust selection process of the projects, who will also be guiding the evaluation design process and reviewing the deliverables throughout the life cycle of the projects. 

"As the only global fund dedicated to tackling online CSEA, Safe Online is a leading contributor of evidence on what works and what doesn't to address digital harms to children. Our selected cohort of grantees and interventions will help us go one step further in that regard and demonstrate what sustainable, scalable and impactful solutions look like in this field."
- Marija Manojlovic, Director, Safe Online
Meet the Grantees

5Rights Foundation 
5Right Foundation’s Child Online Safety Toolkit project aims to create a safer digital environment for children. By providing resources and training based on the toolkit, it seeks to empower countries and organisations to better protect young internet users. It intends to establish an internationally recognised framework for child online safety, fostering a unified and effective approach across different countries and jurisdictions. The toolkit serves as a resource for both civil societies and regulatory bodies worldwide, contributing to policy development and implementation, and the regulation of online child safety. 

 

Council of Europe
EndOCSEA@Europe Plus will build on the results of the previous project phase to strengthen actions to prevent and combat online child sexual exploitation and abuse (OCSEA) across the Council of Europe member states, with particular focus on Georgia, Montenegro and the Republic of Moldova. 

 

DeafKidz International
DeafKidz Defenders is an interactive digital platform designed to educate deaf children on how to stay safe online. This project will empower deaf children in Pakistan, South Africa and Zambia, and increase teachers’ and parents’ knowledge of safeguarding to better protect deaf children from abuse.

 

UNICEF EAPRO
UNICEF EAPRO’s project focuses on continuing momentum for work by being done by UNICEF East Asia and the ASEAN Pacific Regional Office for child online protection. It supports activities like tech industry working groups and a regional forum which will be held in Bangkok in November 2023 to support collaborative efforts that make children’s digital lives safer. Other activities include disseminating and applying detailed technical guidance for legislative reform and support to survivors of online child abuse.

 

UNICEF Ghana
UNICEF Ghana will implement the new project “Protecting children from online sexual exploitation in Ghana: Scale-up of interventions with the biggest potential for impact and evidence generation”. At least 10,000 children are expected to benefit from individual case management, national multi-sectoral and private sector engagement will be enhanced and the capacity of law enforcement, judiciary and prosecutors for effective investigations and successful prosecution of online CSEA will be strengthened. The project aims to scale interventions and generate evidence on the implementation of the Cyber Security Act and the National Child Online Protection Framework, criminal justice and victim support and empowerment, in collaboration with the Cyber Security Authority, the Ghana Police Service, the Ministry of Gender, Children and Social Protection and other partners. 

 

UNICEF Jordan
This programme supports UNICEF’s work with the Government of Jordan to prevent and respond to online child sexual exploitation and abuse. This includes programmes and policies that strengthen the capacity of the national protection system and raise awareness on keeping children safe online.

 

Tech Matters
Aselo, a contact center platform built by Tech Matters, is live in ten countries. One of the primary goals of Aselo is to increase the capacity of child helplines to meet the needs of much greater numbers of children and young people. The existing technology supported by Safe Online helps to identify repeat callers and texters in most communication channels, so that helpline counsellors can review past contact (and case) records. The new funding will enable Tech Matters to make a minimum set of enhancements to Aselo to significantly expand the capabilities of the helplines to address the issue of repeat callers and allow the helplines to handle a greater volume of actionable calls/texts, maximizing their impact and support for those in need.

 

Thorn
Support from Safe Online has enabled Thorn to accelerate the development of our ‘classifiers’ that use the latest artificial intelligence to identify child sexual abuse material (CSAM) at scale, helping to identify victims faster and stop the viral spread of CSAM. This new funding aims to enhance and scale the CSAM Classifier by 1) deploying the existing CSAM Classifier to leading forensic and victim identification technology platforms that will enable adoption by law enforcement and hotline analysts, and 2) identifying a scalable, consistent feedback mechanism for the CSAM Classifier.

See more of our recent updates

We'd love to stay in touch!

We will soon start releasing updates via our newsletter that covers everything Safe Online-related, from funding calls to upcoming events.

Register now
Follow our socials
Facebook Twitter Linkedin Instagram

Copyright Safe Online 2023 © 

All imagery is taken from the UNICEF image library. It is not available in the public domain.

  • info@end-violence.org
  • 633 Third Avenue, 23rd Floor, New York, NY 10017

Privacy Policy

Facebook Twitter Linkedin Instagram
Stay in the loop.

You'll recieve updates and stories from our grantees, events we attend and invites to join us virtually for webinars and discussions.

close menu icon

Our purpose in detail

We are here to ensure every child and young person grows in to the digital world feeling safe, and is protected from harm.

We support, champion, and invest in innovative partners from the public, private, and third sectors working towards the same objective.

We believe in equipping guardians and young people with the skills to understand and see danger themselves once accessing digital experiences without supervision.

We'd love to have a chat

We're thrilled you're interested in donating to Safe Online - pop in the details below and we will get back to you to set up a discussion.