Mishcon de Reya page structure
Site header
Main menu
Main content section

Now & Next: Can you fight violent extremism online?

Posted on 28 October 2022

The internet has helped forge connections across the globe, fostering communities of like-minded people. But these benefits have also been a boon for extremist fringe groups.

As well as allowing dangerous rhetoric to leak into the ‘real’ world, smaller communities are morphing and intertwining with bigger movements, like the alt- and far-right. Can the internet fight back? 

Are Incels a Threat?
The threat from violent extremism online is growing.
Caitlin Huey-Burns
Homeland Security is warning that online extremist’s forums are encouraging copycat attacks.
But it is also changing.
Ross Frenett, Founder and CEO, Moonshot
We are going to see an acceleration of a fusion and a merger of different kinds of ideology.
The internet is spawning dangerous new sub-cultures in ever more fluid ways.
More and more male hatred against women is rising up through the hidden crevices of online forums and entering real life.
With fatal consequences.
At least ten people have been killed.
Dr Katlyn Regehr, Associate Professor, Digital Humanities, University College London
This is a pattern of behaviour that should be intercepted.
Authorities are struggling to keep up with this new face of violent extremism.
Dr Lewys Brace, Senior Lecturer in Computational Social Science, University of Exeter
Governments are not nimble enough to evolve at the speed at which the online ecosystems are evolving.
Can tech innovation help to prevent the next mass shooting?
NOW&NEXT
How the internet is changing violent extremism
Jake Davidson
Why do you think s**ual assault and all these things keep rising, women don’t need men no more.
This man Jake Davidson went on a violent rampage in 2021 in his home city of Plymouth in Britain.
The Police received several calls at 6.11 reporting a man firing a gun close to Biddick Drive.
Davidson murdered his mother before fatally shooting four others and then himself.  The 22 year old had links to an online community known as ‘Incels’.  Incels are mostly young men who define themselves as involuntarily celibate and often blame women for their own inability to form sexual relationships.
Jake Davidson
You talk about intelligence, smartness, you know, women don’t give a F*** about none of that unless it... you’ve got that.
Incels often share deeply misogynistic content promoting violence against women on platforms and forums such as Reddit, Discord, Twitch and others.  Davidson was active on multiple sub-Reddit forums.
Dr Kaitlyn Regehr, Associate Professor, Digital Humanities, University College London
It’s likely that this was an individual who was radicalised online.  Would this have happened without the internet?  That’s hard to day but if we think of two key indicators for involvement in the Incel community which is isolation and high dosages of online content, I have a huge concern that we will see increased membership in this community and increased potential for violence as a result of that.
In 2021 researchers began analysing almost 1.2 million posts linked to Incel content and sites.  Over an eighteen month period they found a 59% increase in terms and code words relating to mass murders.  Since 2014 at least nine perpetrators of fatal attacks, mostly in America are known to have consumed Incel related content and in Britain, Dr Kaitlyn Regehr now advises the Police on this threat.
Dr Kaitlyn Regehr, Associate Professor, Digital Humanities, University College London
From a law enforcement point of view we need to stop the lone wolf narrative.  We need to acknowledge that this is a pattern of behaviour that should be monitored and intercepted.
Incel ideology as part of a wider problem in the digital sphere.  It is well established that the internet is the primary incubator for dangerous and often bizarre new ideas and groups.  But these keep growing and evolving online in ever more fluid ways.
Coleman said he believed his children were going to grow into monsters and that he had to kill them after he was enlightened by QAnon and Illuminati conspiracy theories.  He says he was also having visions and signs that his wife had serpent DNA and passed that DNA on to his children.
Ross Frenett, Founder and CEO, Moonshot
What I am looking at here is the um, a threat bulletin that we share with law enforcement partners, local prevention activitists and others.
Ross Frenett is the CEO of Moonshot, a company which monitors violent extremism of all kinds online.
Ross Frenett, Founder and CEO, Moonshot
Al-Qaeda had really clear world view and set of ideas.  Many neo-Nazi movements have texts that they have been calling back to for generations.  These kinds of emerging social movements, ideologies don’t have that.
Moonshot has tracked how the internet has lowered the barriers to creating new violent sub-cultures and groups.
Ross Frenett, Founder and CEO, Moonshot
Violent extremism in a pre-internet age needed a certain critical mass and that bar has been significantly lowered so you don’t need that many individuals to feel like you are part of a movement.  For too long, online chatter has been dismissed somewhat by folks who just see it as something that takes place on the internet and doesn’t affect the real world and we have now seen without any doubt, that that just is not the case.
It’s not just the creation online of niche communities such as Incels which is a concern.  The bigger problem is how the internet enables these communities to morph and intertwine with larger extremist movements.  In particular, alt-right and far-right groups.
Dr Lewys Brace, Senior Lecturer in Computational Social Science, University of Exeter
Looking at the Incels relationship with the far-right.  We know there is some overlap in the ideologies, both talk about a sort of mythical golden age long gone in the past where things were different to what they are today.  All these kind of narrative structures appear in both.
Ross Frenett, Founder and CEO, Moonshot
I think that over the next ten years what we are going to see is an acceleration of the fusion and a merger of different kinds of ideologies in different kinds of movements.
The British Government’s counter-radicalisation programme, Prevent categorises these kinds of ideologies as mixed, unstable or unclear.  Between 2016 and 2021 referrals to prevent categorised in this way increased significantly.  By March 2021 this category accounted for over half of all annual referrals.  A sharp increase from 2016 when the vast majority of referrals were linked to radical Islam.
Dr Lewys Brace, Senior Lecturer in Computational Social Science, University of Exeter
The reason we are seeing more of this ideological overlap or cross-pollination if you will largely stems from the shift in ways in which we are using the internet.  So if you look at young people nowadays, quite often these ideas permeate into other communities so for example, a lot of extremist ideas are being shared on gaming platforms or quite often we hear now gamers referring to other people as being a chat which is a term that originated within Incel spaces.
Oliver Goodman, Project Manager, Moonshot
This is a really powerful tool and what’s key here really is the threat posed by harmful conspiracy theories.
Analysts monitoring new forms of violent extremism online point to another key factor.  The capacity to share false and conspiratorial information instantly and endlessly.
Oliver Goodman, Project Manager, Moonshot
Across the whole of the US after the Buffalo terror attack in which a white supremacist killed ten people at a supermarket in Buffalo, New York we saw over a 1000% increase in searches for the great replacement conspiracy theory.  A conspiracy theory that was referenced quite closely by the attacker in his manifesto.
The proliferation of extremism is also being encouraged by the normalisation of hateful content on mainstream platforms.  Every day in 2020 humans collectively spent a million years on social media.  Where it is not unusual for videos like this to be widely shared.
Andrew Tate
Fat, ugly, sack of s***.  Have you ever seen a woman try and do anything complicated?
Andrew Tate is former reality TV star infamous for his misogynistic views.  As of September 2022 hashtag Andrew Tate content in TikTok has been viewed more than 17 billion times and he amassed millions of followers before being suspended in August 2022 by almost all platforms.
Dr Kaitlyn Regehr, Associate Professor, Digital Humanities, University College London
This content is not necessarily that niche anymore but rather it is being saturated into youth culture more generally.  It might be a funny meme someone puts up on their Facebook feed that you may be new in high school, it’s through the higher dosages that people become indoctrinated.  The content really starts to permeate off screens and on to street.
For those in the business of tackling the threat from violent extremism online, keeping ahead of the enemy is a challenge.  One hope for turning the tide is designing technology to intercept individuals at scale before they succumb to online radicalisation.  In 2016 Moonshot designed something called the redirect method with Jigsaw, a part of Google.  Google searches for some violent content are met with links to curated material including links to psychological help and support.
Ross Frenett, Founder and CEO, Moonshot
We bought up the advertising space and tried to entice and encourage those who are trying to access extremist material to instead consume material that undermines those extremist movements.  At its core harnessing ad tech to counter terrorism is what the redirect method is all about.
There are some encouraging signs.  Moonshot has found that Americans consuming extremist or violent content online are at least 47% more like than average to take up offers of mental health services online.  But measures like this are a long way from reaching the required scale.  For now the fight against violent extremism in the digital world is buffering.
Ross Frenett, Founder and CEO, Moonshot
Extremists have always been about three steps ahead of those trying to stop them.  That shift needs to take place where we take this seriously as a national security threat because if it doesn’t then this problem is going to be one we are dealing with for a very, very long time.
NOW&NEXT
Tom Standage
Hello, I am Tom Standage, Deputy Editor at The Economist.  If you’d like to learn more about this topic, click on the link opposite and if you’d like to watch more of our Now & Next series, click on the other link.  Thanks for watching and don’t forget to subscribe.

How can we help you?
Help

How can we help you?

Subscribe: I'd like to keep in touch

If your enquiry is urgent please call +44 20 3321 7000

I'm a client

I'm looking for advice

Something else