The Attention Machine

← The Attention Machine

How It Works

The engagement loop. The recommendation engine. Why the algorithm has no incentive to care about your wellbeing.

How to read this page. Tap any underlined word to see the precise academic term and a short definition. Expand any "Deeper" box for the evidence and contested points. The main text works on its own — you can skip both and still get the whole argument.

The core mechanic

Every major social media platform is built around one decision made billions of times per day: what do you see next?

That decision is made by an algorithm. The algorithm does not decide based on what is true, what is important, or what would be good for you to know. It decides based on one signal: what kept people like you engaged in the past. A like, a share, a comment, a long pause while scrolling — every interaction is data. The algorithm learns from that data and uses it to predict what you will interact with next.

This sounds reasonable. In practice, it produces a systematic problem.

Engagement is not the same as wellbeing

A car crash stops traffic. A screaming match draws a crowd. Outrage, fear, and tribal conflict are engaging — not because people enjoy them, but because they are hard to look away from. The algorithm cannot distinguish between engagement that is good for you and engagement that is not. It does not try. That is not what it is optimising for.

Facebook's own internal research, leaked in 2021 by whistleblower Frances Haugen, showed the company knew its algorithms amplified what their researchers called "misinformation, toxicity, and violent content" disproportionately among reshared posts. Internal documents showed debate about whether to fix this. The company chose growth.

This is not an accusation of malice. It is a description of incentives. When engagement is the metric and outrage drives engagement, outrage is what the system surfaces — regardless of intent.

The feedback loop

The recommendation engine works through a continuous loop:

You interact with content — pause, like, comment, share, or simply watch for a few seconds. The system records what triggered that interaction. The algorithm updates its model of what you respond to. It serves more content predicted to trigger similar responses. The loop runs continuously, invisibly, across three billion users simultaneously.

Over time, the algorithm becomes highly accurate at predicting your responses — not because it understands you, but because it has accumulated enough data about your behaviour to model it. The result is a feed that feels personalised and relevant, because it has been calibrated to your specific psychological profile.

What the data showed

In 2018, Facebook changed its algorithm to boost "Meaningful Social Interactions" — content that generated more comments and reactions, intended to promote genuine connection. A peer-reviewed study published in 2025, using empirical data from both the United States and Italy, found the update instead increased ideological extremism and political polarisation in both countries. A commercial decision made in a product meeting had political consequences at national scale.

Internal Facebook data found that recommendation features were responsible for 64% of all extremist group joins. Not fringe sites. Facebook's own recommendation engine, walking ordinary users toward extreme content in incremental steps — each step individually small, the direction consistent.

Why individual willpower is the wrong frame

The common response to this is personal: use it less, take breaks, be more disciplined. This advice is not wrong, but it mistakes the scale of the problem.

The system optimising your feed was built by thousands of engineers over years, tested on billions of data points, and refined continuously based on what produces the most engagement. The idea that individual willpower is the primary defence against this — rather than structural change to the system itself — is itself a framing that benefits the platforms. It places the burden of response on users rather than on design.

Understanding the mechanism is the beginning. Structural change is the complete answer.

How we know — the 2018 algorithm change and what the evidence does and doesn't show +

The 2018 "Meaningful Social Interactions" (MSI) change re-weighted the News Feed to favour posts that drew comments and reactions, especially reshares between friends. Facebook framed it as promoting connection. The peer-reviewed analysis by Germano, Gómez and Sobbrio (2025) modelled how ranking on engagement signals propagates content, and tested the predictions against data from the United States and Italy; it found the change is consistent with increased polarisation and the wider spread of low-quality information. The Haugen disclosures later showed Facebook's own staff had flagged that MSI rewarded outrage and reshared misinformation.

What is contested. The direction of effect is well-supported; precise magnitudes and the share of real-world polarisation attributable to any single algorithm change are not settled, and depend on modelling assumptions. Platform effects interact with offline politics, media, and self-selection, which makes clean causal attribution hard. We state the mechanism and the documented internal knowledge; we do not claim the algorithm alone caused polarisation.

On the 64% figure. It comes from an internal Facebook presentation (c. 2016) reported via the Haugen-era disclosures: of users who joined extremist groups, 64% did so through Facebook's own recommendation tools. It is the company's own measurement of its own product, which is what gives it weight — but it is a single internal figure, not an independently replicated study.

Your Brain — why the machine works on almost everyone →
Advertising & Surveillance — what is being done with your attention →