← The Attention Machine

AI & The Next Layer

The attention machine was already powerful. Generative AI is expected to make its most dangerous applications dramatically cheaper and faster. This page sets out what has happened so far.

How to read this page. Tap any underlined word to see the precise academic term and a short definition. Expand any "Deeper" box for the evidence and contested points. The main text works on its own — you can skip both and still get the whole argument.

What has changed

The attention machine's manipulation capability was always limited by one thing: the cost and labour of producing personalised content at scale. Creating convincing fake personas, tailored disinformation, or emotionally targeted messaging required significant human effort. Generative AI is removing that constraint.

Since 2022, tools exist that can produce convincing text, realistic images, cloned voices, and synthetic video — and researchers expect them to drive down the cost of influence campaigns and make them easier to scale. When combined with the behavioural targeting infrastructure described on the Advertising page — which can identify specific psychological profiles and serve content to them precisely — the result is a system capable of producing personalised manipulation at a scale and speed that was not previously possible.

What is documented

Romania, November 2024. In an unprecedented step in the European Union, a presidential election was annulled following evidence of coordinated algorithmic amplification on TikTok. Romanian intelligence declassified documents stating that the little-known ultranationalist candidate Călin Georgescu, who had been polling behind at least five other candidates, received coordinated amplification and paid promotion on TikTok. He won the first round with 23% of the vote. Romania's Constitutional Court ordered the electoral process to be restarted entirely — an annulment without precedent in the EU, on grounds of digital interference. What the sources support differs by claim. The coordinated amplification rests on the declassified intelligence note, which the Court accepted and critics dispute; that influencers were paid to post content that ended up promoting him is also documented outside the note, by a news investigation that drew on the tax authority (below); the role of generative AI in the content is a further, thinner claim.

The 2024 European Parliament elections were regarded as a "testing ground" for AI-powered disinformation. The same research, published in 2025 (Farooq et al., The International Journal of Press/Politics), documented citizens' concerns about and awareness of AI-generated content during the campaign.

Scale in 2024 US elections. The Centre for Emerging Technology and Security (CETaS) at the Alan Turing Institute found no conclusive evidence that AI-generated content materially altered the 2024 US election results. However, the same research found that AI-generated content did influence election discourse, amplified harmful narratives, and entrenched political polarisation. The absence of a decisive measurable effect in one election cycle is not evidence that the capability is not a structural concern.

What the infrastructure enables

The concern about AI and the attention machine is not primarily about individual incidents of fake content. It is about what the combination of cheap content generation and behavioural targeting infrastructure makes structurally possible:

Influence operations that previously required significant human labour can now be run far more cheaply. A disinformation campaign targeting specific psychological profiles across multiple countries — served to people when algorithms identify them as maximally susceptible — can be produced and deployed by actors who could not previously afford this kind of precision.

The EU AI Act, the DSA, and various national regulations are all attempting to address this. The DSA is now being enforced — the Commission fined X €120 million under it in December 2025, for breaching its transparency rules — and the AI Act's transparency duties for AI-generated content have applied since 2 August 2026. How far either reaches influence operations of this kind is not yet settled. The capability is ahead of the governance, as it has been throughout the history of the attention machine.

AI companions

A separate category: conversational AI products designed for ongoing personal relationships — AI companions, chatbots with persistent memory, products designed to simulate friendship or romance. The commercial incentive is close to social media's — optimise for engagement and return use — even where the money comes from subscriptions rather than advertising. The evidence on effects — on loneliness, social development, emotional dependency — is thin. This is one of the genuinely open questions documented on the What We Don't Yet Know page.

How we know — Romania 2024, and what is documented versus contested +

The confirmed facts: Romania's Constitutional Court annulled the first round of the November 2024 presidential election; declassified intelligence described a coordinated TikTok amplification and paid-promotion operation around a candidate whom polls had placed behind at least five other candidates; he took 23% of the first-round vote. These are matters of public record and an official court ruling.

What is contested or unresolved. The precise role of generative AI in the Romanian operation — as opposed to coordinated human-run accounts and paid promotion — is not fully established. "Algorithmic amplification" (the recommender pushing the content) is what the declassified note asserts and the Court accepted; it has not been shown independently. "AI-generated content" is a separate claim, with thinner evidence still in this specific case. We keep the two apart. The declassified documents were themselves contested: a later investigation by the news site Snoop.ro, drawing on the tax authority, found that one influencer campaign which ended up promoting the candidate had been paid for by a rival pro-EU party, and the annulment was criticised by constitutional-law scholars and by the US administration, and defended by others as "an expression of militant democracy" (more on the Elections page). Likewise, CETaS's finding on the 2024 US elections — no conclusive evidence of a decisive effect on the result — is a statement about one cycle and one kind of measurement, not a verdict that the capability is harmless.

State Actors & Information War →
What We Don't Yet Know →
Yes, but… →