The Attention Machine

← The Attention Machine

AI & The Next Layer

The attention machine was already powerful. Generative AI makes its most dangerous applications dramatically cheaper and faster.

How to read this page. Tap any underlined word to see the precise academic term and a short definition. Expand any "Deeper" box for the evidence and contested points. The main text works on its own — you can skip both and still get the whole argument.

What has changed

The attention machine's manipulation capability was always limited by one thing: the cost and labour of producing personalised content at scale. Creating convincing fake personas, tailored disinformation, or emotionally targeted messaging required significant human effort. Generative AI removes that constraint.

Since 2022, tools exist that can produce convincing text, realistic images, cloned voices, and synthetic video at near-zero cost, at any scale, in seconds. When combined with the behavioural targeting infrastructure described on the Advertising page — which can identify specific psychological profiles and serve content to them precisely — the result is a system capable of producing personalised manipulation at a scale and speed that was not previously possible.

What is documented

Romania, November 2024. The first European Union presidential election was annulled following evidence of coordinated algorithmic amplification on TikTok. Romanian intelligence declassified documents showing that previously unknown ultranationalist candidate Călin Georgescu, who had been polling in single digits, received coordinated amplification and paid promotion on TikTok. He won the first round with 23% of the vote. Romania's Constitutional Court ordered the electoral process to be restarted entirely — the first presidential election annulment in EU history on grounds of digital interference. The role of AI in generating or amplifying content in this case is documented but contested in detail; the algorithmic amplification is confirmed.

The 2024 European Parliament elections were described by researchers as a "testing ground" for AI-powered disinformation. Instances of multimodal AI-generated content entering the electoral information environment were documented across multiple member states. Research published in 2025 (Farooq et al., Political Communication) documented citizens' concerns about and awareness of AI-generated content during the campaign.

Scale in 2024 US elections. The Centre for Emerging Technology and Security (CETaS) at the Alan Turing Institute found no conclusive evidence that AI-generated content materially altered the 2024 US election results. However, the same research found that AI-generated content did influence election discourse, amplified harmful narratives, and entrenched political polarisation. The absence of a decisive measurable effect in one election cycle is not evidence that the capability is not a structural concern.

What the infrastructure enables

The concern about AI and the attention machine is not primarily about individual incidents of fake content. It is about what the combination of cheap content generation and behavioural targeting infrastructure makes structurally possible:

Influence operations that previously required significant human labour can now be run at near-zero marginal cost. A disinformation campaign targeting specific psychological profiles across multiple countries — served to people when algorithms identify them as maximally susceptible — can be produced and deployed by actors who could not previously afford this kind of precision.

The EU AI Act, the DSA, and various national regulations are all attempting to address this. None are yet fully implemented or enforced. The capability is ahead of the governance, as it has been throughout the history of the attention machine.

AI companions

A separate and growing category: conversational AI products designed for ongoing personal relationships — AI companions, chatbots with persistent memory, products designed to simulate friendship or romance. The commercial incentive structure is identical to social media: optimise for engagement and return use. The evidence on effects — on loneliness, social development, emotional dependency — is thin. This is one of the genuinely open questions documented on the What We Don't Yet Know page.

How we know — Romania 2024, and what is documented versus contested +

The confirmed facts: Romania's Constitutional Court annulled the first round of the November 2024 presidential election; declassified intelligence described a coordinated TikTok amplification and paid-promotion operation around a candidate who had been polling in low single digits; he took 23% of the first-round vote. These are matters of public record and an official court ruling.

What is contested or unresolved. The precise role of generative AI in the Romanian operation — as opposed to coordinated human-run accounts and paid promotion — is not fully established. "Algorithmic amplification" (the recommender pushing the content) is confirmed; "AI-generated content" is a separate claim with thinner evidence in this specific case. We keep the two apart. Likewise, CETaS's finding on the 2024 US elections — no conclusive evidence of a decisive effect on the result — is a statement about one cycle and one kind of measurement, not a verdict that the capability is harmless.

State Actors & Information War →
What We Don't Yet Know →