Algorithmic Radicalization

The Engagement Machine

Every major social media platform runs on the same basic principle: keep users engaged as long as possible to show them more ads. The algorithms that determine what you see are optimized for one thing, maximizing the time you spend on the platform.

This creates a fundamental misalignment between what's good for the platform and what's good for users or society.

What engagement-based algorithms favor:

• Emotionally provocative content
• Outrage and anger
• Conflict and controversy
• In-group affirmation
• Out-group hostility
• Novel and extreme viewpoints

What they don't optimize for:

• Accuracy
• Nuance
• User satisfaction
• Social cohesion
• Mental health

A 2024 study published in PNAS Nexus found that Twitter's engagement-based algorithm amplifies emotionally charged, partisan, out-group hostile content compared to a reverse-chronological timeline. More importantly, users don't actually prefer this content: when asked what they want to see, they choose differently than what the algorithm serves them.

The algorithm isn't showing you what you want. It's showing you what will make you react.

The Rabbit Hole Effect

The "rabbit hole" describes how recommendation systems can lead users from mainstream content to increasingly extreme material through a chain of suggestions.

How It Works

1. User watches a video about fitness
2. Algorithm recommends "alpha male" lifestyle content
3. Recommendations shift toward men's rights content
4. Content becomes increasingly hostile toward women
5. User is now consuming incel or manosphere content

Each step in the chain is designed to keep the user engaged. More extreme content tends to be more emotionally provocative, which means more engagement, which means the algorithm promotes it.

The Research Is Mixed, But Concerning

Scientific research on the rabbit hole effect has produced conflicting results:

Studies finding algorithmic radicalization:

• A systematic review of 23 studies found 14 implicated YouTube's recommender system in facilitating problematic content pathways, 7 produced mixed results, and only 2 did not implicate the system
• A 2023 PNAS study found YouTube recommends ideologically congenial content to partisan users, with increasingly problematic recommendations deeper in the trail, especially for right-leaning users
• UC Davis research using "sock puppet" accounts found platforms do gradually recommend more biased content based on viewing history

Studies challenging the hypothesis:

• A 2025 University of Pennsylvania study found limited effects from recommendation algorithms on political views, with rabbit holes "not extremizing"
• A 2024 Northeastern study found that after YouTube changed its algorithm, off-site communities and subscriptions drove radicalization more than recommendations
• Some researchers argue deeply held views don't change easily from media exposure alone

The nuanced reality: The platforms and their algorithms rarely directly recommend extremist content. But they remain powerful tools for those who hold extremist beliefs, providing hosting, distribution, and community infrastructure. The algorithm may not push you to extremism, but it makes the journey easier once you start walking.

TikTok: Radicalization Accelerated

TikTok's algorithm is particularly efficient at personalization, and particularly concerning for radicalization.

A 2024 study in Social Science Computer Review audited TikTok's algorithm and found:

• A large portion of far-right content can be ascribed to platform recommendations
• The pathways through which users access this content are manifold
• The platform's ability to slot users into specific categories reinforces extreme ideas

Media Matters research found that after interacting solely with transphobic content, TikTok's "For You" page began populating accounts with hateful and far-right content. Their analysis of over 400 recommended videos showed transphobia acting as a "gateway prejudice" leading to broader far-right radicalization.

The speed is what's different. Researchers note that TikTok's rapid supply of short-form content allows exposure to extremist material in a fraction of the time it takes on YouTube.

Real-World Consequences

The connection between TikTok radicalization and violence isn't theoretical:

• Vienna, 2023: Austrian authorities thwarted a plot against an LGBTQ+ pride parade involving two teenagers and a 20-year-old who had been radicalized through jihadist content on TikTok. The youngest suspect, 14, had been exposed to videos by Islamist influencers glorifying jihad.
• Vienna, 2024: Several teenagers planning a terrorist attack at a Taylor Swift concert were arrested. The investigation revealed TikTok was one of the platforms used to disseminate extremist content.
• Singapore: Over the past decade, nine youths were arrested under the Internal Security Act for planning acts of terrorism, with cognitive radicalization occurring largely via TikTok.

A process that once took months or years now takes days or hours.

The Manosphere Pipeline

One of the most documented radicalization pathways involves young men and what's called the "manosphere", a collection of online communities promoting masculinity, misogyny, and opposition to feminism.

The Pipeline Structure

Research has documented migration patterns between these communities, with users moving from pick-up artist communities to Red Pill to MGTOW to incel forums.

The Algorithm's Role

The YouTube algorithm is designed to keep users on the site as long as possible. Divisive and edgy content is engaging. The next related video might contain fringe opinions. The video after that might be more extreme.

Experts describe it as a "slippery slope" that begins with algorithms pushing boys to increasingly harmful videos. Then "someone might engage you in a comment thread and tell you to join their Discord group, [where] the content gets darker and darker."

Scale of Exposure

A 2023 poll by Hope not Hate found that 80% of 16- and 17-year-old British boys had consumed content created by Andrew Tate, a prominent manosphere figure. This isn't a fringe phenomenon; it's mainstream exposure to extremist ideology.

Who Is Vulnerable

The targets are often young men who feel lost or isolated. They look to these communities for belonging and validation. Research shows their loneliness transforms into anger as they embrace a misogynist worldview. The echo chamber of the manosphere, where other men corroborate their grievances, amplifies their aggrievement.

Gaming spaces are particularly implicated. Long periods on gaming sites expose young men to incel content. Some streaming platforms, priding themselves on being immune to "cancel culture," allow streamers to disseminate misogynistic worldviews and conspiracy theories.

The Psychological Mechanism

How does algorithmic exposure actually change minds?

Filter Bubbles and Echo Chambers

Platforms learn user interests to modify their feeds, creating a "filter bubble", an algorithmically curated information environment. When users encounter only beliefs that magnify their existing thoughts, an "echo chamber" forms.

According to group polarization theory, echo chambers can push users and groups toward more extreme positions. Without exposure to opposing views, there's no friction to moderate beliefs.

Engagement ≠ Satisfaction

A crucial finding from recent research: the posts users engage with are not the ones they value upon reflection.

When researchers asked users what they wanted to see versus what they actually clicked on, the answers diverged. Engagement-based algorithms optimize for clicks, not satisfaction. Users react to outrage, but don't want a steady diet of it.

This means the algorithm is systematically showing people content that makes them feel worse.

Graduated Escalation

Radicalization rarely happens in one jump. The psychological mechanism involves graduated escalation:

1. User encounters content slightly outside their comfort zone
2. Content is engaging because it's novel or provocative
3. Engagement signals interest to the algorithm
4. Algorithm serves more similar (and slightly more extreme) content
5. User's baseline shifts; what was once extreme becomes normalized
6. Process repeats

Each step is small. The cumulative effect is significant.

The Polarization Evidence

A 2025 study in Science directly tested whether algorithmic content ranking affects political polarization:

Researchers created a browser extension that re-ranked participants' social media feeds using an LLM. Some users saw feeds with reduced exposure to posts expressing partisan hostility. Others saw feeds with increased exposure.

The results:

• Up-ranking hostile content increased political polarization
• Down-ranking hostile content decreased political polarization

This is causal evidence that algorithmic ranking affects political attitudes, not just reflects them.

The study's implications are stark: platforms could reduce polarization by changing their algorithms. They choose not to.

Platform Responses (And Their Limits)

Platforms have taken some action in response to radicalization concerns:

YouTube (2019): Adjusted algorithms to reduce recommendations of "borderline" content. Research shows mixed results: extremist content remains accessible, but subscription and off-site referrals now drive more traffic than recommendations.

Facebook/Meta: Created Oversight Board and content policies, but internal documents (the "Facebook Papers") showed the company was aware its algorithms amplified divisive content and chose not to fix it.

TikTok: Has policies against hate speech and extremism, but enforcement is inconsistent and the algorithm remains opaque.

Why Changes Are Limited

The fundamental problem: reducing polarizing content means reducing engagement, which means reducing ad revenue.

Research shows that ranking by user stated preferences (what people say they want) reduces angry, partisan content, but also potentially reduces engagement. Platforms face a choice between profit and public good, and they have consistently chosen profit.

Regulatory Responses

EU Digital Services Act (2023): Requires social media apps to disclose how their algorithms work and allows independent researchers to assess their impact. This is why we have more data on European platform behavior than American.

US Response: Limited. No comprehensive federal legislation addresses algorithmic amplification. Section 230 shields platforms from liability for user content and algorithmic decisions.

Research Access: The biggest barrier to understanding algorithmic radicalization is platform opacity. Researchers have limited access to recommendation systems, making independent audits difficult. The studies we have often use "sock puppet" accounts or browser extensions because platforms won't share data.

What You Can Do

For Yourself

• Disable algorithmic feeds: Switch to chronological timelines where available
• Use browser extensions: Tools like Unhook (YouTube) or News Feed Eradicator can limit recommendation exposure
• Diversify sources: Deliberately seek out sources from different perspectives
• Check your watch history: Periodically review and clear recommendation training data
• Be aware of emotional manipulation: If content makes you angry, ask why the algorithm showed it to you
• Use RSS feeds: Subscribe directly to sources instead of relying on algorithmic curation

For Young People in Your Life

• Maintain communication: Create space for discussing what they're seeing online
• Don't dismiss their content: Understand why it appeals before critiquing
• Media literacy education: Teach how algorithms work and why
• Provide alternatives: Offer communities that meet social needs without extremist content
• Watch for warning signs: Increased isolation, out-group hostility, obsession with online figures

For Society

• Support research access requirements for platforms
• Advocate for algorithmic transparency legislation
• Fund independent research on platform effects
• Support deradicalization resources (communities like r/IncelExit, streamers who challenge extremist content)

The Bottom Line

Social media algorithms are optimized for engagement, and engagement is driven by emotional provocation. The result is a systematic amplification of outrage, hostility, and extreme content, even when users don't want it and even when it harms them.

The research shows:

1. Engagement-based algorithms amplify divisive content
2. Users don't prefer this content when asked directly
3. Algorithmic changes can increase or decrease polarization
4. Radicalization pathways exist and have real-world consequences
5. Platforms know this and have chosen not to fundamentally change

The rabbit hole isn't an accident. It's the business model.

Understanding this is the first step. The platforms won't change unless forced to, by users demanding different products, by researchers exposing harms, and by regulators requiring transparency and accountability.

References

1. PNAS Nexus: Engagement, User Satisfaction, and the Amplification of Divisive Content
2. Science: Reranking partisan animosity in algorithmic social media feeds
3. PNAS: Auditing YouTube's recommendation system
4. Systematic review: YouTube recommendations and problematic content
5. Social Science Computer Review: How Algorithms Promote Self-Radicalization on TikTok
6. Media Matters: TikTok's algorithm and far-right rabbit holes
7. Northeastern: YouTube extremism research
8. Gender & Society: Radicalization pathways in the manosphere
9. CBC: How young men fall into online radicalization
10. The Soufan Center: Online Radicalization of Youth
11. Wikipedia: Algorithmic radicalization

Related Topics for Further Reading

This article is part of a series on algorithmic manipulation and social media harms. Related topics we cover:

• The Attention Economy: How Tech Hijacks Your Brain
• Social Media and Elections: How Algorithms Shape Votes
• Cambridge Analytica: The Playbook That Never Died
• Fake News Detection: How to Spot Misinformation
• Platform Alternatives: The Fediverse
• Digital Detox: Reclaim Your Attention

Coming Soon

We're developing additional coverage on these related topics:

• The Manosphere Deep Dive: From pick-up artists to incels, the complete pipeline
• Gaming and Radicalization: How streaming platforms and Discord servers radicalize young men
• Deradicalization: What works to bring people back from extremism
• Algorithmic Transparency: What platforms know and won't tell us
• Children and Algorithms: How recommendation systems affect developing minds
• Engagement Metrics: The numbers platforms optimize for and why it matters
• Alternative Ranking Systems: What would social media look like with different incentives?