The Attention Machine

← Symptoms

Xenophobia & Violence

The connection between online hate and real-world violence was documented before many people had heard of Facebook.

How to read this page. Tap any underlined word to see the precise academic term and a short definition. Expand any "Deeper" box for the evidence and contested points. The main text works on its own — you can skip both and still get the whole argument.

The timeline

Stormfront — the first major white supremacist website — was set up in 1995. The connection between online hate speech and real-world hate crimes was being studied by researchers by the early 2000s. In June 2004, the OSCE held a formal intergovernmental meeting specifically on racist and xenophobic online propaganda and its connection to hate crimes — twenty years before the EU Digital Services Act required platforms to assess this risk. By 2007, the Southern Poverty Law Center documented approximately 12,000 white supremacist videos openly available on YouTube and MySpace.

This is not history. These warnings were contemporaneous, specific, and well-evidenced — documented before the platforms that amplified the problem at scale had become dominant.

Myanmar: the documented case

Facebook launched in Myanmar in 2012 and within years became the dominant, nearly exclusive, information platform for most of the country's population. UN investigators and journalists documented Facebook's role in the amplification of anti-Rohingya hate speech from 2016 onward. Facebook itself acknowledged in 2018 that it had not done enough to prevent the platform being used for incitement to violence. In 2017, the Myanmar military conducted a campaign against the Rohingya people that the UN described as bearing "the hallmarks of genocide."

The causal pathway from Facebook amplification to atrocity is complex — existing ethnic tensions, military action, historical grievances all played roles. What is documented: Facebook was the primary information environment, it amplified content that dehumanised the Rohingya population, and internal documents later showed the company was aware its systems were being used for coordinated hate campaigns and did not act with adequate urgency.

The algorithm does not choose hate. It chooses engagement.

The mechanism is the same throughout this site. The algorithm does not have views on ethnicity or politics. It surfaces content that generates strong emotional responses — and hatred, fear, and tribal identity are among the most engaging emotions. The recommendation engine walks users incrementally toward more extreme content not because it supports extremism but because extremism outperforms moderation on engagement metrics.

Internal Facebook data found that the platform's recommendation features were responsible for 64% of extremist group joins. Users were not seeking these groups. They were recommended into them, incrementally, by a system optimising for time-on-platform.

What the research shows on online-offline violence connections

Research on the relationship between online hate speech and offline violence consistently finds correlations. A 2019 study found a significant positive relationship between Facebook posts with anti-refugee sentiment and attacks on refugees in Germany, at the local level. A 2021 study found that areas with higher social media penetration showed higher rates of coordinated ethnic violence in regions of conflict.

The honest framing: the causal pathway is real and documented. It is not deterministic — most people who consume hateful content do not commit violence. The direction is consistent. The scale of the amplification is what changes the base rates.

How we know — the natural experiment behind the anti-refugee finding +

The standard objection to "online hate correlates with offline attacks" is that towns with more hostility might simply have both more hateful posts and more attacks, with the internet just along for the ride. Müller and Schwarz (2019) addressed this with a natural experiment: they used local internet and Facebook outages — disruptions unrelated to local attitudes — as a lever. Where outages cut access, the relationship between anti-refugee sentiment and attacks on refugees weakened. Because the outages were essentially random with respect to local politics, this gets much closer to a causal claim than a simple correlation.

What it shows and its limits. It is strong evidence that the platform amplified existing sentiment into action in a specific country and period, not a universal constant. It does not say everyone exposed becomes violent — the claim is about shifted base rates across many communities, which is exactly how amplification at scale would look.

What you can do →
Elections & Democratic Harm →