Intelligence Rising Insights Series - #1
Information, Trust and Social Cohesion
This article is part of a series of sector and topic-focused, in-depth looks at the participants’ actions, and resulting events and injects that emerged from the Intelligence Rising 2024 Game.
When the subject matter experts who designed Intelligence Rising sat down in 2023 to map AI’s broader social consequences, one question kept surfacing above the others. Often the question most discussed in AI discourse is what AI will be capable of in the future – but this is a question often addressed, and at this point, capabilities were already accelerating faster than it was reasonable to track. The question was this: what will people believe, and about whom?
The injects that followed were not about dramatic technological breakthroughs, but rather about the slow erosion of true communication, governance, and collective decision-making.
Fraud
Several injects examined the use of AI to generate convincing phishing messages, deepfake audio, and highly targeted fraud campaigns. At the time of design, these felt like plausible near-future threats, and their prominence has only risen since. Cybersecurity firms have documented an increase in AI-enhanced scams since the release of large language models, and these attacks are greater in sophistication as well as volume. The linguistic complexity of fraudulent approaches has increased measurably: longer messages, more accurate punctuation, more convincing social engineering. Darktrace, which has tracked this trend closely, noted the shift shortly after ChatGPT’s public release and has continued to flag AI-enabled scams as one of the most rapidly expanding threat vectors.
Another area of concern is voice cloning. AI systems are now capable of mimicking the voices of friends, family members, and colleagues with sufficient accuracy to deceive. One inject imagined a scenario in which a public figure was impersonated by AI fraudsters in a scam that cost the victim millions. Whilst the specific celebrity framing has not materialised verbatim, the underlying dynamic has. Vishing attacks exploiting voice cloning are well documented across financial services and consumer fraud. The response from institutions has itself been telling: many banks have already begun phasing out voice authentication entirely, recognising that the technology that once secured accounts has become a liability.
One inject described a German political figure condemning his government’s failure to protect citizens from AI-enabled criminality, citing a case of tens of thousands of bank customers defrauded through an AI-generated voice scam that defeated standard authentication. The specific event did not occur, but the political and institutional dynamic it captured, where opposition figures exploit public anxiety about AI-enabled crime, has become a recognisable feature of European political discourse.
Authenticity
The immediate fraud risks, significant though they are, are arguably the more tractable part of the problem. Institutions can adapt by removing voice verification, expanding public education, or by developing detection tools. The harder problem is what happens to the information environment when AI-generated content becomes indistinguishable from genuine content in everyday life.
The game’s designers anticipated this as a gradual fragmentation of the media environment in which audiences increasingly consume information through separate platforms and communities. In these instances, shared narratives become harder to maintain, and institutions find it progressively more difficult to establish authority or legitimacy. AI in this scenario acts primarily as an accelerant, lowering the cost of producing persuasive or misleading content, which amplifies existing polarisation, and erodes the baseline of common factual ground on which public discourse depends. The game’s most direct forecast on this theme scored the highest possible rating.
One inject explored a specific flashpoint in this dynamic: the collision between AI fact-checking systems and media organisations that felt those systems were operating with political bias. The scenario depicted a major media company threatening legal action over what it characterised as unfair moderation by AI-powered platforms. This inject is particularly interesting since reality has moved in a more extreme direction than the game anticipated. When political pressure mounted over perceived bias in AI content moderation, the response from at least one major platform was not to contest the accusation but to remove human fact-checkers entirely and replace structured moderation with a community-based system. The game imagined a battle over how AI would govern information, but the real world produced a different outcome: the retreat of institutional oversight altogether.
Trust
Whilst the erosion of trust in everyday communication certainly is a dangerous process, declining trust in the very institutions that democratic societies depend on could have far more serious consequences.
Several injects explored AI’s potential role in undermining electoral processes. One described an Electoral Commission issuing stark warnings about AI disinformation ahead of a contested presidential election, with concerns mounting that the integrity of the democratic process itself was at risk. The language of institutional alarm was more acute in the scenario than in reality, but it captured the present dynamic of AI-generated content circulating in ways that amplify doubt in election legitimacy. DHS has repeatedly flagged elevated domestic extremism risks in the context of contested election results, and disinformation, social media amplification, and public confidence in democratic outcomes has become a persistent feature of political life in several major democracies.
A related inject depicted Homeland Security on high alert over the prospect of civil unrest following a bitterly contested election, with calls for calm from both sides of the political divide and the Electoral Commission under pressure to review outstanding counts. The discussion note attached to this inject in the analysis references the storming of the Capitol as a precursor. The game did not predict that specific event (it had already occurred), but it did accurately model the ongoing fragility of institutional confidence around elections in a polarised information environment.
As public anxiety about AI’s role in these dynamics has grown, so has pressure for governance responses. One inject described protests spreading from Italy across major cities globally, with demonstrators demanding greater transparency about how AI was being used in critical systems. Real-world concerns demanding AI oversight have become increasingly visible. The specific trigger in the game was an AI-assisted cyber attack on Italian infrastructure; the real-world trigger has been a more generalised sense that the technology is advancing faster than the frameworks designed to govern it. The sentiment the inject captured, that citizens are entitled to know how AI is being deployed in systems that affect them, has moved from the margins of public debate toward its centre.
The Governance Gap
Behind all of these dynamics sits a more fundamental problem that the game identified early and that has not been resolved: the gap between the pace of AI development and the capacity of governance institutions to keep up with it.
One inject described a UK-hosted international effort to address AI cyber threats that was criticised for failing to tackle the most difficult questions. Similarly, the Bletchley Declaration, which emerged from the UK’s AI Safety Summit, drew precisely this kind of criticism.
This governance gap is the thread that runs through the entire cluster of injects. The fraud risks, whilst challenging, are manageable if institutions respond effectively. The authenticity problem is more difficult, but it is addressable through a combination of detection technology, media literacy, and regulatory pressure. The deeper challenge is what happens when these pressures compound simultaneously, faster than governance systems can adapt, in an environment where public trust in those systems is already declining. These are the challenges that the game is designed to surface, and what makes Intelligence Rising so valuable.
Why this matters
When participants in Intelligence Rising encountered these injects, the most valuable insight that emerged was how this erosion of trust interacts with governance, economic disruption, geopolitical rivalry. That interaction effect remains the most difficult thing to model analytically, and the most important thing to have rehearsed before it counts.
