← The Attention Machine

How It Works

The engagement loop. The recommendation engine. Why the algorithm has no incentive to care about your wellbeing.

How to read this page. Tap any underlined word to see the precise academic term and a short definition. Expand any "Deeper" box for the evidence and contested points. The main text works on its own — you can skip both and still get the whole argument.

The core mechanic

Every major social media platform is built around one decision made billions of times per day: what do you see next?

That decision is made by an algorithm. The algorithm does not decide based on what is true, what is important, or what would be good for you to know. It decides, overwhelmingly, on one kind of signal: what kept people like you engaged in the past. A like, a share, a comment, a long pause while scrolling — every interaction is data. The algorithm learns from that data and uses it to predict what you will interact with next.

This sounds reasonable. In practice, it produces a systematic problem.

Engagement is not the same as wellbeing

A car crash stops traffic. A screaming match draws a crowd. Outrage, fear, and tribal conflict are engaging — not because people enjoy them, but because they are hard to look away from. The algorithm cannot distinguish between engagement that is good for you and engagement that is not. It does not try. That is not what it is optimising for.

Facebook's own internal research, leaked in 2021 by whistleblower Frances Haugen, showed the company knew its algorithms amplified what their researchers called "misinformation, toxicity, and violent content" disproportionately among reshared posts. Internal documents showed debate about whether to fix this. Proposed fixes were resisted because they would have reduced engagement.

This is not an accusation of malice. It is a description of incentives. When engagement is the metric and outrage drives engagement, outrage is what the system surfaces — regardless of intent.

The feedback loop

The recommendation engine works through a continuous loop:

You interact with content — pause, like, comment, share, or simply watch for a few seconds. The system records what triggered that interaction. The algorithm updates its model of what you respond to. It serves more content predicted to trigger similar responses. The loop runs continuously, invisibly, for more than three billion people a day on Meta's apps alone. Facebook itself launched in 2004.

Over time, the algorithm becomes highly accurate at predicting your responses — not because it understands you, but because it has accumulated enough data about your behaviour to model it. The result is a feed that feels personalised and relevant, because it has been calibrated to your specific psychological profile.

The same loop pays the people who make the content. Creators — publishers, influencers, anyone posting for a living — earn from the advertising revenue the platform shares, from sponsorships priced on reach, and from the platform's own creator funds; each grows with engagement, so the ranking's preference for what provokes a reaction becomes their incentive too. The Facebook Files record the company's own finding: after the 2018 change rewarded comments and reshares, publishers and political parties told Facebook they had shifted to more divisive posts because that was what the ranking now paid for. No one instructed them; the revenue did.

What the data showed

In 2018, Facebook changed its algorithm to boost "Meaningful Social Interactions" — content that generated more comments and reactions, intended to promote genuine connection. A peer-reviewed study (Germano, Gómez and Sobbrio, published in the Journal of Public Economics in 2026), using survey data from both the United States and Italy, found the update instead contributed to increased ideological extremism and affective polarisation in both countries. On this evidence, a commercial decision made in a product meeting had political consequences — survey data, not an experiment, and how far any single ranking change moves polarisation is not settled (see below).

An internal Facebook presentation from 2016, reported by the Wall Street Journal in 2020, found that, among the German political groups it studied, 64% of all joins to extremist groups came through Facebook's own recommendation tools — one country, one year, the company's own measurement. Not fringe sites. The company's own engine, one suggested group at a time.

Why individual willpower is the wrong frame

The common response to this is personal: use it less, take breaks, be more disciplined. This advice is not wrong, but it mistakes the scale of the problem.

The system optimising your feed was built over years by large teams of highly skilled engineers, trained on vast records of what people do, and refined continuously — change after change tested on real users — towards what produces the most engagement. The idea that individual willpower is the primary defence against this — rather than structural change to the system itself — is itself a framing that benefits the platforms. It places the burden of response on users rather than on design.

Understanding the mechanism is the beginning. The rest is structural: a change to what the system is built to maximise, not to how disciplined its users are.

How we know — the 2018 algorithm change and what the evidence does and doesn't show +

The 2018 "Meaningful Social Interactions" (MSI) change re-weighted the News Feed to favour posts that drew comments and reactions, especially reshares between friends. Facebook framed it as promoting connection. The peer-reviewed analysis by Germano, Gómez and Sobbrio (2026; circulated as a working paper in 2025) modelled how ranking on engagement signals propagates content, and tested the predictions against data from the United States and Italy; the model predicts that weighting likes and shares more heavily raises engagement but also misinformation and polarisation, and the survey evidence indicates the MSI change contributed to increased ideological extremism and affective polarisation. The evidence is observational: survey data, not a controlled experiment. The Haugen disclosures later showed Facebook's own staff had flagged that MSI rewarded outrage and reshared misinformation.

What is contested. That engagement ranking amplifies divisive content is well-supported; how far that moves polarisation is not settled. The largest experiments on the question point the other way: in the 2020 US election studies run with Meta, three months of a chronological feed in place of the ranked one, or of a third less content from like-minded sources, did not measurably change polarisation or political attitudes — within the limits of three months and one election. Precise magnitudes, and the share of real-world polarisation attributable to any single algorithm change, depend on modelling assumptions. Platform effects interact with offline politics, media, and self-selection, which makes clean causal attribution hard. We state the mechanism and the documented internal knowledge; we do not claim the algorithm alone caused polarisation.

On the 64% figure. It comes from a 2016 internal Facebook presentation, reported by the Wall Street Journal in May 2020 — a year before the Haugen disclosures. The slide, from a study of political groups in Germany, said that 64% of all joins to extremist groups were due to Facebook's own recommendation tools ("Our recommendation systems grow the problem"). It is the company's own measurement of its own product, which is what gives it weight — but it is a single internal figure, from one country and one year, not an independently replicated study, and it says nothing about whether those users were already looking.

Your Brain — why the machine works on almost everyone →
Advertising & Surveillance — what is being done with your attention →
Search — the same ranking logic, applied to what you look for →
Yes, but… →