INTERPOL will bring the most advanced technology to investigators of online CSEA through its new DevOps Group Project. The initiative will facilitate research and development by an expert group of investigators, non-governmental organisations, academia, and information technology companies, and extend solutions to specialised officers worldwide via INTERPOL’s secure channels. Headquartered in France, this project has a global reach.
Our grantees National Center for Missing and Exploited Children (NCMEC) CyberTipline Countries involved:United States of America NCMEC’s CyberTipline will enhance a critical survivor-serving tool through
Our grantees New Venture Fund Communication and Advocacy Support: Inspiring the End of Violence Against Girls and Boys Grant timeline:United Stated of America As a
Our grantees INTERPOL (Global) The Devops Group Project Countries involved:France INTERPOL will bring the most advanced technology to investigators of online CSEA through its new DevOps
Our grantees Thorn Bringing Light to the Dark Web Countries involved:United States of America The Fund is supporting Thorn’s Bringing Light to the Dark Web project, a
Our grantees Internet Watch Foundation Reporting Portal Project Countries involved:United Kingdom of Great Britain and Northern Ireland With support from the Fund, the Internet Watch Foundation is
The Fund is supporting Thorn’s Bringing Light to the Dark Web project, a global initiative that will use a new tool to help law enforcement identify children who are victims of online child sexual exploitation and abuse. Thorn helps gather the most important pieces of information related to child sexual abuse material online to support law enforcement investigations. By expanding the initiative’s reach, Thorn hopes to achieve the following goals:
Remove more children, faster, from abusive situations
Reduce the time it takes to identify new child sexual abuse material
Reduce the investigative time required to identify a child in abuse material
Foster collaboration among law enforcement worldwide
To date, Thorn has helped law enforcement in over 55 countries find over 800 children whose abuse material has appeared online.
Our grantees National Center for Missing and Exploited Children (NCMEC) CyberTipline Countries involved:United States of America NCMEC’s CyberTipline will enhance a critical survivor-serving tool through
Our grantees New Venture Fund Communication and Advocacy Support: Inspiring the End of Violence Against Girls and Boys Grant timeline:United Stated of America As a
Our grantees INTERPOL (Global) The Devops Group Project Countries involved:France INTERPOL will bring the most advanced technology to investigators of online CSEA through its new DevOps
Our grantees Thorn Bringing Light to the Dark Web Countries involved:United States of America The Fund is supporting Thorn’s Bringing Light to the Dark Web project, a
Our grantees Internet Watch Foundation Reporting Portal Project Countries involved:United Kingdom of Great Britain and Northern Ireland With support from the Fund, the Internet Watch Foundation is
Countries involved: United Kingdom of Great Britain and Northern Ireland
With support from the Fund, the Internet Watch Foundation is progressing its mission to eliminate child sexual abuse from the internet by working in 30 countries – the majority of which are in Africa – to implement web-based portals for citizens to report online child sexual abuse content and make the Internet a safer place for children and adults around the world.
The aim of the project is to support countries without anywhere to report images and videos of child sexual abuse. IWF provides a reporting button which feeds directly to their expert analysts in the UK, who assess the reports according to UK Law and work to remove the content as quickly as possible from the internet.
Our grantees National Center for Missing and Exploited Children (NCMEC) CyberTipline Countries involved:United States of America NCMEC’s CyberTipline will enhance a critical survivor-serving tool through
Our grantees New Venture Fund Communication and Advocacy Support: Inspiring the End of Violence Against Girls and Boys Grant timeline:United Stated of America As a
Our grantees INTERPOL (Global) The Devops Group Project Countries involved:France INTERPOL will bring the most advanced technology to investigators of online CSEA through its new DevOps
Our grantees Thorn Bringing Light to the Dark Web Countries involved:United States of America The Fund is supporting Thorn’s Bringing Light to the Dark Web project, a
Our grantees Internet Watch Foundation Reporting Portal Project Countries involved:United Kingdom of Great Britain and Northern Ireland With support from the Fund, the Internet Watch Foundation is
A year of progress, challenges and key milestones for Safe Online
2023 was a year of big strides on the digital landscape characterised by the rapid evolution of technologies such as Artificial Intelligence (AI) and immersive platforms. However, these technological innovations are also bringing forth new and evolving challenges, impacting some of our most vulnerable, particularly children and young people.
The challenges to children’s online safety have never been more significant, and demand an unprecedented response. Our work, now more critical than ever, has risen to meet these challenges. From strategic investments to critical research and advocacy, Safe Online has been at the frontline of the battle to tackle online child sexual exploitation and abuse.
This year also marks the beginning of Safe Online’s independent journey and our transition from the Global Partnership to End Violence Against Children. Our new role as a global funding vehicle enables us to renew our commitment to our core purpose; to protect and empower children and young people throughout the digital world.
We welcomed 10 new trailblazing grantees into the Tech Solutions fold with innovative projects and awarded 4 new projects as part of our Tech Coalition Safe Online Research Fund to advance the journey from research labs to the frontlines of tech.
Since 2016, Safe Online has been at the forefront of this global battle against online CSEA, investing over US $77 million in over 100 projects across 85+ countries. Through our investments, Safe Online is rewriting the future of online safety.
Disrupting Harm's Global Leap: Flagship research project on online CSEA expands to 11 new countries!
This year, 11 new countries: Armenia, Brazil, Colombia, Dominican Republic, Jordan, Montenegro, Morocco, North Macedonia, Pakistan, Serbia and Tunisia announced the implementation of Disrupting Harm in their respective countries. This second round of investment by Safe Online (totalling USD $7.5m) will expand Disrupting Harm to 4 more regions and enable robust research on online child sexual exploitation and abuse, conducted by Disrupting Harm partners ECPAT International, INTERPOL and UNICEF Innocenti with support from national expert organisations. By 2025, each participating country will be armed with evidence-informed action plans, tailored to fortify their defences against online child exploitation.
Key Moments and Strategic Opportunities
In 2023, Safe Online leveraged over 60 strategic opportunities including high-level events, convenings and meetings to elevate the profile of child online safety, placing it at the forefront of political and sectoral discussions and ensuring it remains a priority across various levels and sectors.
Highlights include:
Fostering collaboration to tackle online CSEA: In September, Tech Coalition and Safe Online organised a research fund convening in San Francisco, California bringing together 11 grantees and 25 representatives from the tech industry, creating a platform for sharing knowledge and fostering technical collaboration. The convening took place on the margins of the Stanford Trust & Safety Research Conference on 28-29 September 2023 where Safe Online organised a session with research grantees, sharing the lessons learned from the Research Fund and promoting the insights from research and data on online CSEA.
Safe Digital Futures for Children: Aligning Global Agendas Safe Online’s session, attended by key government partners from Australia, France, Ghana, and the United States of America, international organisations and technical experts, at the Internet Governance Forum in Kyoto, Japan sparked crucial conversations on fragmentation in the digital landscape and emphasised the necessity for evidence-based research and data to bridge gaps across various global digital agendas, including cybersecurity, child online safety, and tech-facilitated gender-based violence.
In partnership with the French Children Online Protection Lab and the WeProtect Global Alliance, Safe Online organised the Safe Digital Futures for Children: Data for Change event in November in Paris, France, in which over 50 technical experts participated. The event strengthened the conversation on the availability and quality of data related to online Child Sexual Exploitation and Abuse (CSEA) and brought together a community of experts to build on the existing work towards a more reliable, comparable, and comprehensive data ecosystem on online Child Sexual Exploitation and Abuse (CSEA).
Safe Online participated in the Paris Peace Forum in a session led by Ambassador Henri Verdier, Ambassador for Digital Affairs, France and attended by Jean-Noël Barrot, Minister for Digital Transitions and Telecommunications, France and other experts. Safe Online Executive Director, Marija Manojlovic advocated for fortified child-centric digital prevention and response mechanisms to online child sexual exploitation and abuse.
A Milestone Moment at the G7 Meet in Mito, Japan: Safe Online’s Call to Action
As the year drew to a close, Safe Online stepped onto the global stage again at the prestigious G7 Security and Interior Ministers’ Meet in Mito, Japan. Safe Online’s lead, Marija Manojlovic, urged the G7 nations to unite in their efforts against online child sexual exploitation and abuse (CSEA) by boosting international funding, harmonising legislations and standards and advancing safety technology.
Innovative Partnerships -Forging an Alliance for a Safer Digital Future
Safe Online this year formalised a groundbreaking partnership with the French Children Online Protection Lab. This collaboration will unite the expertise, resources and passions of both organisations to forge a more secure online world for children globally.
As part of the partnership, Safe Online and the Children Online Protection Lab will align our governance structures and support an exciting series of pilot projects where Safe Online will leverage its existing infrastructure to support and enhance the Lab’s experimentations.
Safe Online joins the Lanzarote Committee:
This year marks another significant milestone for Safe Online, as we proudly joined the esteemed Lanzarote Committee as an Observer. This collaboration opened a new chapter in our relentless pursuit to combat the growing threat of online CSEA and we remain strongly committed to continue working with key stakeholders to transform the capacity of national, regional and global systems to combat the escalating threat of online CSEA and the opportunity of closely collaborating with the Lanzarote Committee.
New Safe Online Brand!
And finally, this year, Safe Online launched our new brand, a significant milestone in our ongoing mission to create a safer digital world for children and young people.
We invite you to view our brand video and learn more about our renewed commitment: Safe Online’s Brand Video.
Press Release: July 17, 2024 [LONDON]: Today, survivors of childhood sexual violence will address the escalating threat to children’s lives posed by online abuse, and call for action to ‘redesign’ the internet and protect children ahead of the first ever global Ministerial on the issue in November 2024. This is a pivotal moment to redefine the digital world for children, ensuring that technology is used for their benefit and never for their harm.
Safe Online’s latest funding round is investing $4 million for generating crucial data and evidence to tackle online child sexual exploitation and abuse (CSEA). This initial investment will be followed by an additional $1.5 million for evaluations of the projects in the next quarter.
The objective of this Request for Expressions of Interest (EOI) is to develop a better understanding of the Children Online Protection Laboratory (COPL) community’s priorities, identify ideas and resources and co-construct next steps of the experimentation process.
Tech Industry and Researchers come together to tackle online child sexual exploitation and abuse
Promoting action from research
2024 Collaboration Announcement
We’re excited to announce that Safe Online and Tech Coalition’s partnership is entering its 4th year! With a generous USD $500,000 from the Tech Coalition, we’re set to enhance the impact of 13 research projects that play a crucial role in developing resources and applications for the tech industry. This effort is part of our ongoing commitment to combat online child sexual exploitation and abuse.
Safe Online’s collaboration with the Tech Coalition addresses a crucial gap in the online child safety ecosystem for research funding that is aligned with industry trends, policymaking, and practical applications. Over the next year Safe Online will aim to maximize the impact of the current investments and build on this pivotal partnership with the Tech Coalition.
The main aim of the Tech Coalition Safe Online Research Fund is to use the research to impact tech industry practices and steer technological developments. We do this not only through our research but also by organizing strategic events and engagements, to provide space and platform for the tech industry and researchers to share insights from the research projects. These events are key to translating these insights into products and practices that can significantly enhance the protection of children online. See an example of a recent convening below.
Why we came together – 2023 Research Fund Convening
More than 45 participants, a combination of researchers supported by the Tech Coalition Safe Online Research Fund and tech professionals, came together in San Francisco to discuss emerging findings from research and actions that the tech industry can take to protect children online.
What we covered
The day started with a public panel to share insights about this unique initiative and hear from collaborators who have made this work possible. Made up of industry members, independent experts and researchers, the panel was united in its call for more cross-sectoral collaboration, the need for greater inclusion of underrepresented geographies and groups, and the real impacts that the research projects from this fund are already having.
Marija Manojlovic, Executive Director, Safe Online made the exciting announcement that the Tech Coalition Safe Online Research Fund is granting 4 awards to further accelerate the work of organisations from the first cohort of grantees. The additional funds are for a duration of 12 months and will support projects to extend research to applications such as piloting solutions, technical collaboration and innovation efforts. Read morehere
We then dove straight into the research findings with a lightning round of presentations followed by a marketplace that gave tech professionals the chance to talk to researchers and understand what the findings mean for them.
In the afternoon we took a deep dive into the research themes with small groups unpacking the opportunities and challenges around grooming, user reporting, prevention and wellness and deterrence and offender behavior. We ended the day focusing on action – linking up researchers with tech professionals working in operations, policy and product to discuss how to continue to strengthen the links across sectors to better protect children online.
How the day went
Overwhelmingly, participants were excited about the new and strengthened connections with otherexperts working on the issue –both within the research and tech communities and across both sectors.
“Meeting the other researchers and hearing about their work was perhaps the most useful because it made me feel like part of a movement for change - and we can all do with feeling a bit of unity in this field”
- Amanda Third, Young & Resilient Research Centre, University of Western Australia
“Valuable connections were made for professional networking and information sharing - we solved mutual problems together!”
- Tech industry participant feedback from post-workshop survey
The research projects range from linguistic analysis that can track patterns of grooming, to understanding more about perpetrator behaviors, to increased evidence around parents’ awareness of online abuse. The Research Fund has a focus on promoting action, making it even more valuable to bring researchers together with tech professionals.
“I found it practical and action-oriented. I think there was a very fluid dialogue where we managed to put everyone's needs and work into perspective”
- Lina María Saldarriaga, Universidad de los Andes
There was a high level of consensus on the day about ensuring that research in this area is practical, actionable and is fast paced enough to make a change in the tech industry.
“Researchers could reflect on methods and think about the minimum viable product – a gold standard works in a vacuum but not real-world contexts. What matters is what’s making a difference.”
- Group discussion on policy
There is a lot of enthusiasm and energy to continue collaboration through a community of practice and individual connections that were made on the day.
Overall, we received incredible feedback from the attendees at the convening. Both research grantees and tech industry participants appreciated the chance to learn, exchange ideas, and have open and honest discussions. Participants highlighted the need for more such events bringing together researchers and tech partners to share learning, and to receive tailored research insights at regular intervals in order to grow and strengthen this unique new community of practice.
We want to build on the enthusiasm from the day and keep the momentum going. As a next step, the TC Safe Online Research Fund will keep working to set up effective ways to share findings from research, increase researchers’ knowledge of how different platforms work, promote collaboration and most importantly action.
The Tech Coalition Safe Online Research Fund is investing in knowledge and research to end online child sexual exploitation and abuse (CSEA). The Tech Coalition and Safe Online joined hands in 2020 to collectively work to support knowledge and research towards ending online harm. Going into its 4th year, the Fund is continuing its focus on innovative research that produces actionable insights to impact product and policy development, with a priority given to research that can help inform the technology industry’s approach to combating online child sexual exploitation and abuse.
A recent white paper by the World Economic Forum’s Global Coalition for Digital Safety highlights the critical need for assessing digital safety. It provides a roadmap to navigate the complexities of measuring digital safety in the context of ongoing technological advancements and evolving regulatory frameworks.
On 30-31 May 2022, End Violence’s Safe Online initiative brought together over 40 grantees and partners from across the world alongside donors, and industry representatives from the Tech Coalition in a unique opportunity aligned around a common vision to end online child sexual exploitation and abuse (CSEA).
Reflections: Safe Digital Futures for Children – Data for Change
Envisioning a digital world with the rights of children at the core
It has been a month since Safe Online together with the French Online Child Protection Laboratory and WeProtect Global Alliance met in Paris, France to host the second Safe Digital Futures for Children: Data for Change event.
Q: What does “Safe Digital Futures for Children and Young People” mean?
Natalie: When we came up with the name, we were trying to reflect on alignment of all digital harms against children and a proactive way of building the future, building many futures, actually. We were trying to include the diversity of children’s experiences in the various ways of imagining our future. This also goes back to our own Safe Online mission – and the fact that digital is not necessarily just online. So, for me, Safe Digital Futures was getting at all the different threads we were trying to pull together
Serena: For me, the name is about building the backbone and infrastructure of a safe digital world for all children and young people. Connectivity is growing and eventually all children will be connected or immersed in digital environments. We are heading towards a future where terms like offline and online will not make sense anymore – we will be immersed in the digital world, and we will not think about it as such. In fact, children of this generation have already made this transition.
The name, therefore, refers to the future digital world we need! A place designed with children’s safety in mind, especially the safety of the most vulnerable groups of children and young people. Today the digital world is designed by and for adults, and this is why we need to envision a future digital world that enables children to navigate with strong and age-appropriate safeguards to prevent risks and threats of violence.
Q: Why is data relevant for a safe digital future?
Serena: The design is the result, but how can we design a safe digital world without knowing how and where children and young people are at risk? The change that we want needs to be informed by comprehensive and quality data and evidence, including the perceptions and voices of children and young people from across the world. We do not only need better and reliable data, but also collaboration, capacity and sustained political support to ensure data is used effectively. This is why we named the initiative ‘data for change’!
The hidden nature of online child sexual exploitation and abuse
How can we make a difference for this and future generation of children without adequate resources?
Natalie: Echoing that, there is currently not only a fragmented data landscape but also fragmented investment in capacity and infrastructure. When we are speaking about data, we are referring to the whole ecosystem. So not just datasets, but also data resources, technical expertise, ethical principles, legal agreements, relationships, technology infrastructure, etc. We need this full picture of data to inform more dynamic systems and anticipatory frameworks so that we are not always reactive.
Rates of change are increasing for not only how quickly technology is evolving but also how rapidly it is affecting the nature of our daily lives. This is adding complexity to an already massive challenge in keeping children safe online. There is a huge opportunity and need to use data to frame this conversation more proactively and create the groundwork for a shared way forward so that this does not become an ongoing exercise in firefighting but rather in strategically shaping our digital futures.
Q: What was your experience with data coming into this initiative?
Natalie: This is an interesting question especially considering how everyone in the workshop had diverse expertise and experiences and came in thinking about data from different perspectives. It’s good to reflect on this while thinking about our own experience with data.
I have a background in data science, and I’ve worked a lot with data for social good including thinking about data governance, data privacy and data ethics. I see a lot of potential to find ways to put narratives and learnings from those topics within the context of and in service of child safety.
When we did the first data initiative in 2021 and again this year, I realised there is so much that exists in terms of data governance and data ethics work, and we have an opportunity to learn from other communities doing this work. But I also realised diving into this, that there’s so much that’s so unique about our field – for instance, how technology is intertwined, the sensitivity of much of the data, how the issues evolve so quickly. So, while we can learn from other communities and build on certain things there’s so much that is unique in our space and needs to be done very thoughtfully with people who understand this problem. If we are able to figure some of these things out in collaborative and dynamic ways, that can offer other folks a lot of potential to learn from our space too.
Serena: Every sector or professional, based on their experience and expertise, will see, think and talk about data in a different way and that remains one of the key barriers when we’re trying to align different data sets to create a more comprehensive and harmonised picture. For instance, law enforcement will have their own expectations from data as something that is going to lead them to identify the victim and bring the perpetrator to justice. Academics will focus more on sampling, methodology and quality. Professionals will want easy to use data for programming, policy and advocacy. This variety is a richness, but it’s also a challenge; because it’s about building bridges for people to talk to each other, understand different datasets, terminologies and working modalities and see how to better align efforts across sectors and levels.
Q: What was the biggest challenge and success in organising this workshop?
Natalie: The biggest challenge going into the workshop was striking the right balance in capacities to approach the data landscape effort that came out as a priority from the 2021 convening and was used to inform the conversations for this year’s workshop. The key question was do we bring in data experts and then try to complement the content expertise or do we bring in someone from our ecosystem and try to bolster or find some way to connect them to data expertise so that they’re not starting from scratch.
Our team is a great example of a mix of expertise and perspectives coming together and being able to look at the data ecosystem in collaborative ways with the goal of identifying concrete places to advance this work. We hope to take what we are learning in our team’s experience with these efforts so far to better build bridges and spaces for the wider ecosystem to do the same.
Serena: The biggest success was bringing the right people into the room and creating a safe space for them to share and learn from each other. We are grateful to all the participants who not only joined us but who were willing to be open and very engaged to build a better data ecosystem for child online safety.
Natalie: I agree, a success for me was the openness with which people approached the workshop and the fact that we heard from people that they came away with new ideas and connections. This group of people have invaluable expertise based on their work and perspectives and coming into the workshop some challenges to current ways of working were presented for people to sit with – of trying to make some shifts in looking at data not in terms of our individual organisational mandates and in silos but instead in more comprehensive and harmonised ways. By the end of the two days, we heard from many participants that they had changed their minds in some way in how they think about data. Some said, “Maybe I didn’t know everything that I thought I did” or “It’s important for me to see this differently.” This feedback and engagement of the community we brought together was hugely positive for me.
Q: What does the future with this initiative look like?
Natalie: In the longer term, one important focus is creating mechanisms for data governance and seeing that take shape – it is incredibly important in any data ecosystem, and it doesn’t exist for this field as a whole yet. We also want to see shifts in narratives and more inclusion as well. In fact, being intentional about inclusion of different actors that aren’t well represented is an immediate next step and then also over time, having a more inclusive ecosystem as a whole.
Serena: With data, we will be able to quantify, understand and contextualize risks and the extent to which children and young people experience the abuse in order to prevent and disrupt it before it happens. This is what ‘Data for change’ means! Without data this crime will remain invisible, and we would be investing in the dark. Data has the power to make this horrible crime against children visible and to guide us in designing safe digital futures for children and young people.
Since 2017, Safe Online has invested over USD 76 million across 100 projects in 85 countries. Of this, over USD 20 million has been invested in evidence and data generation.
In a huge step forward to tackle online child sexual exploitation and abuse (CSEA), Peru has passed a new law which makes it mandatory for Internet service providers to inform their users about parental filters.
On 30-31 May 2022, End Violence’s Safe Online initiative brought together over 40 grantees and partners from across the world alongside donors, and industry representatives from the Tech Coalition in a unique opportunity aligned around a common vision to end online child sexual exploitation and abuse (CSEA).
Elon Musk’s free and accessible AI system, Grok, has generated an estimated three million non-consensual nude images, triggering an urgent global response. A coalition of 107 leading child-protection and humanitarian organisations has united to confront what they describe as an unacceptable threat to human dignity and child safety.
12 new countries join flagship global initiative to combat online child sexual exploitation and abuse
Disrupting Harm is a large-scale research project that provides robust evidence across sectors, on children’s exposure to online sexual exploitation and abuse, and how countries can more effectively respond to this crime. Today, ECPAT International, INTERPOL, Safe Online and UNICEF Innocenti are announcing the implementation of Disrupting Harm in eleven new countries: Armenia, Brazil, Colombia, Dominican Republic, Jordan, Mexico, Montenegro, Morocco, North Macedonia, Pakistan, Serbia and Tunisia.
This second round of investment (totalling USD $7.5m) will enable robust research on online child sexual exploitation and abuse, conducted by Disrupting Harm partners with support from national expert organizations. By 2025, evidence-informed action plans will be produced for each country to strengthen their capacity to prevent and respond to online child sexual exploitation abuse.
In today’s increasingly digital world, the internet has become an integral part of children and young people’s lives, providing enormous opportunities for communication, education and entertainment. At the same time, it can also present a serious threat to their safety and well-being. Unfortunately, there is very little solid evidence available on the scale of these risks, which groups of children are more likely to be harmed, and where or how this happens.
In 2019, in response to the growing threat of online child sexual exploitation and abuse, Safe Online invested an initial USD $7m in the Disrupting Harm large-scale research project, implemented by ECPAT International, INTERPOL and UNICEF Innocenti. Disrupting Harm produces crucial evidence on children’s exposure to online sexual exploitation and abuse, and assesses how national protection systems are responding to this crime. Based on this research it proposes ways in which leaders and policy-makers can help strengthen online child sexual exploitation and abuse prevention and response approaches to more effectively address this issue.
Taking a holistic and cross-sectoral approach, Disrupting Harm conducts nationally representative household surveys with children and their caregivers; reviews existing national laws and policies; engages with national law enforcement agencies to understand their capacity to effectively respond to these crimes; documents the experiences of children and young people who have experienced online sexual exploitation and abuse; and studies the experiences of frontline welfare workers and justice professionals who are working with cases of online child sexual exploitation and abuse to understand their capacities and needs.
Disrupting Harm has so far been implemented in thirteen countries across South-East Asia and Eastern and Southern Africa.
Children who had experienced online sexual exploitation and abuse were between 40% and 400% more likely to self-harm and to have thoughts of suicide following their experience.
Specialized police units dedicated to online child sexual exploitation and abuse are crucial to improve the national (and global) response.
In response, a number of these countries have already initiated policy and legislative changes to strengthen prevention and response, for example:
In line with Disrupting Harm recommendations, the Government of Malaysia amended existing legislation to ban sexual extortion and livestreaming of child sexual abuse, making these criminal offences. This move better enables the effective prosecution of these crimes. These amendments represent a significant strengthening of the country’s legislative framework in this area and contribute to creating a safer online world for children in Malaysia.
In Indonesia, Disrupting Harm recommendations were used to help develop a National Roadmap on Child Online Protection and informed the revision of national standards and training modules for integrated child protection services. Findings from the program were also applied to the development of a web-based platform strengthening children’s awareness of violence, including online safety.
In Tanzania, insights and recommendations were integrated into education and training materials developed by the Ministry of Health.
Today, ECPAT International, INTERPOL, Safe Online and UNICEF Innocenti are announcing the implementation of the second phase of the Disrupting Harm program in twelve new countries. These are: Armenia, Brazil, Colombia, Dominican Republic, Jordan, Mexico, Montenegro, Morocco, North Macedonia, Pakistan, Serbia and Tunisia. The research will be implemented between 2023-2025 through close collaboration between a range of different actors including national UNICEF offices, civil society organisations, ECPAT network members, law enforcement and INTERPOL National Central Bureaus, independent expert organizations, and government institutions.
In an era where digital technology permeates every aspect of children’s lives, the digital threats impacting their safety have never been greater. The recent Digital Dialogue on Children’s Digital Safety, co-hosted by Safe Online, WHO, and UNICEF, brought together over 250 global stakeholders—governments, survivors and youth leaders, civil society, and experts—to transform commitments into tangible action.
On 30-31 May 2022, End Violence’s Safe Online initiative brought together over 40 grantees and partners from across the world alongside donors, and industry representatives from the Tech Coalition in a unique opportunity aligned around a common vision to end online child sexual exploitation and abuse (CSEA).
The objective of this Request for Expressions of Interest (EOI) is to develop a better understanding of the Children Online Protection Laboratory (COPL) community’s priorities, identify ideas and resources and co-construct next steps of the experimentation process.
Tackling age assurance & live streaming of abuse to make the internet safer for children
Safe Online is investing USD $2 million across 10 innovative projects to seed and grow solutions that leverage existing and new technologies to tackle online child sexual exploitation and abuse (CSEA).
The new projects will focus not only on technology solutions, but also include research around the use of tech solutions for the two key areas – age assurance and live streaming of abuse in the online CSEA ecosystem.
These two themes were selected based on rising prioritisation in global discussions of online CSEA across sectors; upcoming legislation and emerging policy focus globally that raises the urgency of proactively engaging with these challenges; as well as from extensive consultation with diverse experts across the online CSEA and related fields.
"Our understanding of how we approach age assurance and live streaming of abuse in the online CSEA ecosystem is fragmented and often reactive to current trends in digital harms. With these 10 new projects, Safe Online is promoting a unified and proactive approach that focuses on upcoming legislation, new and emerging technology and tackling rapidly evolving threats in the digital space."
- Marija Manojlovic, Director, Safe Online
The 10 organisations, awarded through a rigorous selection process supported by external technical experts, will build on existing tools and research in the space as well as existing evidence, frameworks and networks. These organisations were selected on the relevance of the proposed solution and potential for scale and impact, the soundness of the proposed technology or research, global/regional and national context (including similar projects and risk of duplication), legal basis for implementing the tool or policy relevance for the research, and the type of the organisation and geographic distribution of funds.
Read more on the projects:
AVID: A global approach to tolerances, frequency, bias and resilience of Age Assurance.
Lead research on appropriate levels of tolerance for age assurance systems in line with the development of international standards, including developing guidance on the frequency of re-authentication of users & research on bias, presentation attack and system-level attack.
Center for Democracy & Technology: CSEA & Social Media Policy
CDT will conduct research and lead multi-stakeholder workshops to address how child sexual exploitation and abuse material (CSEA) is identified and addressed in live-streamed content. Live streaming, which is increasingly important in the use of popular apps particularly among children, presents unique challenges for combatting CSEA content. Given the scale of user generated content on major social media and other online services, companies make use of a range of machine learning techniques to analyze user behavior and user generated video, audio, and text. Understanding the capabilities of such techniques, and how they may be deployed, is crucial for assessing the consequences for children’s safety, privacy, free expression, and other rights. It is also important to understand how these tools can be applied in non-English language settings, and the extent to which these tools can be effectively integrated into trust and safety systems in a way that protects and promotes children’s rights. CDT’s project will (1) assess proposed and existing technologies used to detect and address CSEA in live streamed content, and (2) facilitate dialogue and engagement between researchers, advocates, and policy-makers on how to address this problem.
Plan International Australia: Safeguarding young people from the dark path of self-facilitated live streaming of CSEA
Plan International, with ECPAT, will conduct a participatory action research into the ‘value chain’ of live streaming of online child sexual exploitation and abuse in the Philippines, outline livestreaming risk factors, the steps that victims and perpetrators go through, and the perceived “benefits” and consequences of the actions.
Rigr AI is crafting the “VST – Video Summarisation Tool,” tailored for Law Enforcement Agencies (LEA), Non-Governmental Organisations (NGOs), and private sectors. Its primary goal is to identify victims of Child Sexual Exploitation and Abuse (CSEA) content.
euCONSENT ASBL is a non-profit NGO that will put into live operation extensions to the eIDAS infrastructure required to deliver its vision for pan-European, open-system, secure and certified interoperable age verification and parental consent to access Information Society Services.
T3K.AI: CounterACT - Countering Abuse through Coercion or Trafficking
T3K.AIwill adapt their solution CORE, which is specialized on the AI-enabled detection of child sexual abuse material (CSAM) in pictures and videos, to CSAM detection in livestreams. The project will optimise the automatic screening of livestreams to empower Content Providing Companies in their fight against streamed abuse.
Yoti is a digital identity company that makes it safer for people to prove who they are, providing a number of innovative solutions which span reusable Digital ID, age assurance, identity verification and esignatures. Yoti will aim to support data and content regulators in the online child safety area; to review a wider number of sites and achieve the objectives of their online safety regulations. This project will have a global reach and maintain the collaborative approach of Yoti in working with key partners including policy advisers, think tanks, researchers, humanitarian bodies and civil society.
Universidad de los Andes (with Aulas en Paz and Te Protejo): AI alert system for the detection and deterrence of CSEA livestreaming in the Latin American context
This project aims to test and improve an Artificial Intelligence alert system for the detection of live streaming of abuses in Latin America. This initiative is based on a tool previously created by Universidad de los Andes through a project financed by Safe Online. By using information from conversations that take place between aggressors, the research team will analyse patterns and commonalities in these interactions in order to tackle this crime in a faster and more strategic way. Through a partnership established with hotlines and organisations that work on the prevention of CSEA in Colombia, Argentina, Chile, Mexico, Ecuador and Costa Rica, this alert system seeks to become a regional tool for the prevention of this crime.
Web-IQ: OSAgE: Open Source Age Estimation
To help keep children safe online and enable authorities to enforce age legislations on online platforms, Web-IQ uses AI to create an age assurance tool, tailored to Safe Online’s mission. The tool aims to verify a user’s age by scanning open information available online, helping protect kids from online dangers of sexual abuse.
May and Bay are exploring the vast online world – and learning to keep themselves safe from the threats to their safety. And by helping these two digital characters stay safe in the online world, children can now learn more about and gain the skills needed for navigating the risks in the digital world.
In a huge step forward to tackle online child sexual exploitation and abuse (CSEA), Peru has passed a new law which makes it mandatory for Internet service providers to inform their users about parental filters.
In a landmark step forward for children’s online safety, Malaysia has made sex extortion and livestreaming of child sexual abuse distinct offenses, to enable better prosecution of these crimes.
We are thrilled to share a major milestone in the journey of Safe Online and our mission of shaping a safer digital world for all children and young people everywhere.
In line with the evolving technological landscape and the ever-changing needs of young people worldwide, we are establishing ourselves as a global funding vehicle and renewing our commitment to our core purpose; to protect and empower children and young people throughout the digital world. Alongside our communications partner, Strategic Agenda, we have developed a powerful new brand that speaks directly to our purpose, mission, and vision to launch our evolution from the End Violence partnership.
Why?
This crucial step allows us to wholeheartedly devote our resources, expertise, and unwavering passion to the cause of ensuring the online safety and empowerment of our most vulnerable technology users.
We believe that every child and young person deserves an online experience where they can benefit from all it has to offer without fear of harm, abuse or age-inappropriate content and experiences. The digital world is an integral part of their lives, and we owe it to them to ensure digital spaces meet the standards we set for the physical world. Our new mandate and sole focus allows us to have a greater impact and broader reach than ever before, and strengthens our position as a network and advocate at the heart of online safety.
As we embark on the new chapter of Safe Online, we renew our promise to create a safer digital future for children and young people. Our new identity is more than a brand: it's a commitment to the well-being of every child and young person who turns to the digital world to explore, create, and dream. We will ensure that their journey online is enriching, inspiring, and, above all, safe.
- Marija Manojlovic, Director, Safe Online
What will change?
We will continue to invest in research, systems, and tools developed by innovative partners focused on online safety. We will use the learnings from our investments to build an evidence base on what works to tackle abuse online, as well as to advocate to prioritise child online safety in business practices, policies and budgets of governments and industry alike. We are now able to dedicate more time, support, and resources to new and existing partners.
Finally, a huge thank you to all who worked on, and with, the End Violence partnership over the past 7 years. The extraordinary commitment and professionalism of all during the duration of the partnership and the transition period has helped to build the exciting and ambitious path before us now.
Malaysia Takes Pivotal Steps Towards a Safer Internet for Children Following Data From Disrupting Harm
In a landmark step forward for children’s online safety, Malaysia has made sex extortion and livestreaming of child sexual abuse distinct offenses, to enable better prosecution of these crimes. The Parliament of Malaysia, the Dewan Rakyat, approved a Bill to amend several provisions to the Sexual Offences against Children (SOAC) Act 2017.
As per the Disrupting Harm Malaysia report, developed with active support of the Government of Malaysia, sexual extortion and livestreaming of child sexual abuse remain some of the biggest threats that children face in the country. Sexual extortion includes being blackmailed to engage in sexual activities, having sexual images shared without permission, or being coerced to engage in sexual activities through promises of money or gifts. In fact, data shows that in one year, an estimated 100,000 internet-using children aged 12-17 were subjected to clear instances of online sexual exploitation and abuse, including sex extortion.
Disrupting Harm represents the most comprehensive and large-scale research project ever undertaken on online child sexual exploitation and abuse at the national level. It has resulted in 13 country reports including Malaysia and a series of unique ‘data insights’. Implemented by UNICEF Innocenti, ECPAT International and INTERPOL and funded by the Safe Online initiative at End Violence, it provides comprehensive evidence concerning the risks children face online, how they interlink with other forms of violence and what can be done to prevent them.
The Malaysia report shares a series of unique insights specific to the country including the fact that children mainly experienced online child sexual exploitation and abuse (OCSEA) through the major social media providers including messaging platforms. More worryingly, Helplines and the police were almost never utilised to seek help by children who were subjected to OCSEA, who tended to confide in people within their interpersonal networks, particularly friends, caregivers or siblings.
Based on the findings, Disrupting Harm outlined a series of evidence-based recommendations spanning legislation, law enforcement, justice processes, social services and public awareness to support the implementation of a comprehensive and sustained response to OCSEA, or an actionable blueprint to protect children. These recommendations call on the Government of Malaysia and its partners to act, educate and invest.
The crucial amendments passed draw directly from the recommendations of the Disrupting Harm report and go a long way to further strengthen the country’s legislative framework. These include:
Make sexual extortion and livestreaming of child sexual abuse distinct offenses, to enable better prosecution of these crimes.
Stronger victim-protection in the justice system, including child-friendly mechanisms during hearings to avoid re-traumatisation of children.
Allow extraction of evidence at the first point of contact, such as police stations and hospitals, to prevent children having to testify multiple times.
Better mechanisms to provide compensation for victims; under the new law, courts have the discretion to decide on compensation instead of having to rely on prosecutors to submit such claims on behalf of the victim (which was rarely done as per the study).
“Our congratulations to the Government of Malaysia for taking urgent action to prioritise online safety of children by passing the amendments to the Sexual Offences Against Children Act,” said Marija Manojlovic, Director, Safe Online Initiative at the End Violence Partnership. “Malaysia is a critical example of the importance of data to drive policy action. As our flagship project Disrupting Harm shows, we need high quality data, and more importantly, political will to use the data and insights to create an actionable blueprint towards creating a safe online world for children”.
Malaysia is a critical example of the importance of data to drive policy action. As our flagship project Disrupting Harm shows, we need high quality data, and more importantly, political will to use the data and insights to create an actionable blueprint towards creating a safe online world for childre
- Marija Manojlovic, Director, Safe Online
The legislative changes in Malaysia are the latest in a series of progressive legislations that work towards creating a safer online world for our children.
In a huge step forward to tackle online child sexual exploitation and abuse (CSEA), Peru has passed a new law which makes it mandatory for Internet service providers to inform their users about parental filters.
In a landmark step forward for children’s online safety, Malaysia has made sex extortion and livestreaming of child sexual abuse distinct offenses, to enable better prosecution of these crimes.
Hosted in collaboration with the British Standards Institution (BSI), this groundbreaking summit united over 700 global stakeholders to affirm a bold declaration: “Age Assurance Can Be Done!” With a focus on safeguarding children online, the summit produced a landmark Communique on Age Assurance, outlining principles like prioritising individual rights and data minimisation. Safe Online grantee, Age Assurance Certification Scheme, explains why age assurance is critical, the role of the first-ever Communique on Age Assurance and the way forward from this landmark Summit.
We are here to ensure every child and young person grows in to the digital world feeling safe, and is protected from harm.
We support, champion, and invest in innovative partners from the public, private, and third sectors working towards the same objective.
We believe in equipping guardians and young people with the skills to understand and see danger themselves once accessing digital experiences without supervision.
We'd love to have a chat
We're thrilled you're interested in donating to Safe Online - pop in the details below and we will get back to you to set up a discussion.
