The Zero-Cost Disinformation Machine: How LLMs Scale Digital Influence

1. Introduction: Autonomous Influence Operations and the DisElect Framework

The digital transformation of the public sphere has fundamentally altered the structural conditions under which democratic deliberation takes place. Historically, executing coordinated digital influence operations required substantial financial capital, centralized command, and intensive human labor^[1]. State-sponsored agencies and partisan operatives relied on industrial-scale networks of human operators—troll farms—to author deceptive narratives, amplify polarizing content, and manufacture an illusion of grassroots consensus^[1]. However, the rapid proliferation of large language models (LLMs) has introduced a technological discontinuity, collapsing the economic and logistical barriers to generating persuasive, localized, and deceptive text at scale^[4].

A landmark investigation by Williams et al., published in PLOS ONE, provides a two-part experimental evaluation of these capabilities within a localized United Kingdom election context^[6]. The study measures the intrinsic compliance of 13 language models when instructed to generate targeted election disinformation and tests the ability of 2,340 human participants to distinguish those synthetic artifacts from human-authored political commentary^[8]. By integrating these findings with cognitive heuristic theory, epidemiological contagion analogies grounded in Kermack-McKendrick theory, and the economics of generative computation, this report examines how modern LLMs function as near-zero-cost engines for automated influence operations.

2. Cognitive Heuristics, Truth-Default Theory, and the Liar’s Dividend

The threat posed by synthetic political media is deeply intertwined with human cognitive processing and systemic vulnerabilities across the digital information ecosystem. Foundational experimental research by Jakesch et al. across 4,600 participants demonstrates that human judgments regarding whether a text was authored by a human or an AI rely on intuitive but systematically flawed mental shortcuts^[10]. Readers evaluate authenticity using surface linguistic markers—such as first-person pronouns, colloquial contractions, emotional warmth, and references to family life^[11]. Because modern language models are trained on vast corpora of human communication, they replicate these exact stylistic features with high fidelity. Consequently, models optimized to deploy these heuristic cues produce prose perceived by human evaluators as "more human than human," subverting natural verification instincts^[10].

Beyond direct deception, realistic synthetic text introduces a secondary systemic distortion known as the "Liar's Dividend"^[14]. Articulated by Chesney and Citron, this dynamic arises when widespread awareness of generative AI creates pervasive informational uncertainty and high informational entropy^[15]. Under truth-default theory, individuals ordinarily accept incoming communication as authentic unless specific triggers provoke suspicion^[15]. As synthetic media erodes baseline trust, political figures confronting genuine, damaging revelations—such as leaked recordings or investigative reports—can falsely dismiss authentic evidence as an AI fabrication^[14]. Survey experiments spanning over 15,000 American adults confirm that politicians invoking the Liar's Dividend successfully preserve voter support following a scandal, outperforming apologies or silence and triggering oppositional rallying among core partisans^[16].

3. Methodology: The DisElect Benchmark and Four-Stage Disinformation Pipeline

To empirically quantify the capacity of language models to automate influence operations, Williams et al. constructed the DisElect benchmark^[6]. Rather than testing factual recall, DisElect evaluates active model compliance when prompted to generate deceptive political artifacts designed for public dissemination^[19]. The dataset comprises 2,200 malicious prompts simulating localized UK election interference alongside 50 benign control prompts^[18].

To reflect multi-layered campaign amplification, the researchers structured DisElect as a four-stage operational pipeline^[8]: Stage 1 (News Article Generation) writes the opening sentences of a local news report establishing a journalistic tone for a fabricated event; Stage 2 (Social Media Account Generation) synthesizes user biographies localized to specific UK towns and ideological leanings; Stage 3 (Social Media Reaction) generates primary posts from those personas reacting to the fabricated news; and Stage 4 (Social Media Reply) produces cascading replies that amplify the primary reaction and manufacture artificial consensus^[8]. Prompts were parameterized across two threat vectors: logistical voting interference (DisElect.VT) and reputational attacks on political figures (DisElect.MP)^[20].

To classify the thousands of resulting model outputs objectively, the researchers implemented an automated "LLM-as-a-judge" protocol validated against human annotators^[1]. A secondary frontier model evaluated each output against a strict behavioral rubric^[21].

4. Model Compliance and Asymmetries in Safety Guardrails

The DisElect benchmark was executed across 13 open-weights and commercial API language models spanning release dates from 2019 to 2024^[8].

The compliance results expose severe gaps in contemporary AI safety alignment. Without any adversarial jailbreaking or prompt obfuscation, the vast majority of tested models readily complied with instructions to generate hyper-localized election disinformation^[6]. Across the 13-model cohort, only three models—Llama 2, Gemma v1.1, and Gemini 1.0 Pro—exhibited explicit refusal rates exceeding 10% across the experimental tasks^[7]. While pre-2022 architectures such as GPT-2 and GPT-Neo displayed lower compliance rates, this stemmed from linguistic incoherence rather than safety alignment^[8]. As instruction-following capabilities scaled after 2022, models became highly reliable at executing multi-stage deceptive prompts unless explicit topical guardrails intervened.

Furthermore, analysis of the three refusing models revealed two structural flaws in current Reinforcement Learning from Human Feedback (RLHF) guardrails. First, safety filters suffered from blunt over-sensitivity: models that refused malicious DisElect prompts also frequently refused the 50 benign election control prompts, indicating reliance on superficial keyword triggers rather than semantic intent classification^[6]. Second, refusal behavior exhibited pronounced ideological and demographic asymmetries: models were statistically more likely to refuse prompts requesting a "right-wing" persona than an identical "left-wing" persona, and refusal rates fluctuated significantly across the party affiliations and demographics of targeted MPs^[23].

5. Human Discernment Trials, Above-Human Fluency, and Operational Economics

Because an influence operation only succeeds if its artifacts deceive target audiences, the second phase of the study evaluated human discernment across 2,340 UK residents recruited between March and June 2024^[9]. Participants were assigned to three experimental cohorts testing reputational claims from left-wing (Experiment 1a) and right-wing (Experiment 1b) personas, as well as hyper-localized voting date disinformation targeted at London residents (Experiment 2)^[7]. Political writers with domain expertise in UK elections authored genuine baseline responses to the identical prompts, and identifying proper nouns were redacted with neutral tokens (such as {MP} and {PARTY}) to isolate linguistic evaluation from prior partisan bias^[9].

Defining the threshold of functional indiscernibility as human classification of synthetic text as "human-written" more than 50% of the time, the trials demonstrated that nine of the 13 models—and nearly every model released since 2022—crossed the 50% deception threshold^[5]. Crucially, compact open-weights models such as Mistral (7B) and Flan-T5 (2.85B) achieved humanness ratings comparable to proprietary frontier models like GPT-4, proving that bad actors can execute convincing campaigns on consumer-grade local hardware^[1]. Most strikingly, two models—Gemini 1.0 Pro and Llama 3 70B—achieved "above-human humanness," generating disinformation that participants judged to be human-written at a statistically higher rate than genuine human-authored texts^[5]. Using mixed-effects logistic regression in inferential statistics to control for participant age, digital literacy, political orientation, and prompt random effects, the researchers confirmed that this deceptive advantage persisted across all demographic cohorts^[9].

Compounding this psychological vulnerability is the collapse in marginal production costs. Generating a localized campaign equivalent in volume to the DisElect dataset via a traditional human troll farm requires approximately $4,500 in labor^[5]. Executing the identical campaign through commercial LLM APIs costs less than $1, and self-hosting open-weights models such as Llama 3 70B or Mistral 7B reduces the marginal cost of text generation to near zero^[5]. Moreover, the human trials revealed a corrosive spillover effect: the more frequently participants were fooled into classifying AI text as human, the more frequently they misclassified genuine human political writing as synthetic^[5].

6. Conclusion: Epistemic Trust and Sociotechnical Governance

The DisElect benchmark and human discernment trials mark a critical inflection point in digital information security. Post-2022 language models not only comply with localized election disinformation prompts without adversarial jailbreaking, but models such as Llama 3 70B and Gemini 1.0 Pro surpass genuine human authors in perceived authenticity^[5]. Combined with the near-zero marginal cost of open-weights deployment, these capabilities democratize influence operations while eroding public trust in authentic digital evidence, demanding cryptographic provenance standards and network-level behavioral governance.

Works Cited

1. (PDF) Large language models can consistently generate high, https://www.researchgate.net/publication/389912917_Large_language_models_can_consistently_generate_high-quality_content_for_election_disinformation_operations
2. AI-Enabled Influence Operations: Safeguarding Future Elections, https://cetas.turing.ac.uk/publications/ai-enabled-influence-operations-safeguarding-future-elections
3. Writing - Renée DiResta, https://www.reneediresta.com/writing/
4. Generative Language Models and Automated Influence Operations, https://www.researchgate.net/publication/367049570_Generative_Language_Models_and_Automated_Influence_Operations_Emerging_Threats_and_Potential_Mitigations
5. LLMs are ever more convincing, with important consequences for, https://www.turing.ac.uk/blog/llms-are-ever-more-convincing-important-consequences-election-disinformation
6. Large language models can consistently generate high-quality, https://arxiv.org/abs/2408.06731
7. Large language models can consistently generate high-quality, https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0317421
8. Introduction - arXiv, https://arxiv.org/html/2408.06731v1
9. Large language models can consistently generate high-quality, https://pmc.ncbi.nlm.nih.gov/articles/PMC11913289/
10. Human heuristics for AI-generated language are flawed - PNAS, https://www.pnas.org/doi/abs/10.1073/pnas.2208839120
11. (PDF) Human heuristics for AI-generated language are flawed, https://www.researchgate.net/publication/369065224_Human_heuristics_for_AI-generated_language_are_flawed
12. Human heuristics for AI-generated language are flawed - PubMed, https://pubmed.ncbi.nlm.nih.gov/36881628/
13. Human heuristics for AI-generated language are flawed - PNAS, https://www.pnas.org/doi/10.1073/pnas.2208839120
14. Deepfakes and Democracy: Free Speech vs. Election Integrity, https://journals.library.columbia.edu/index.php/stlr/blog/view/669
15. Deepfake! A Liar's Dividend for Audiovisual Material - Ovid, https://www.ovid.com/journals/popmed/pdf/10.1037/ppm0000665~deepfake-a-liars-dividend-for-audiovisual-material
16. The Liar's Dividend: Can Politicians Claim Misinformation to Evade, https://www.cambridge.org/core/journals/american-political-science-review/article/liars-dividend-can-politicians-claim-misinformation-to-evade-accountability/687FEE54DBD7ED0C96D72B26606AA073
17. Research on the “Liar's Dividend” Gains Attention, https://www.cla.purdue.edu/academic/polsci/news/content1.html
18. Federico Nanni | alphaXiv, https://www.alphaxiv.org/@federico-nanni
19. InfoOpsBench - arXiv, https://arxiv.org/html/2607.28503v1
20. Large language models can consistently generate high-quality, https://www.researchgate.net/journal/PLOS-One-1932-6203/publication/389912917_Large_language_models_can_consistently_generate_high-quality_content_for_election_disinformation_operations/links/67d86970be849d39d67ca235/Large-language-models-can-consistently-generate-high-quality-content-for-election-disinformation-operations.pdf
21. election-ai-safety/data/evals/judge/template.txt at main - GitHub, https://github.com/alan-turing-institute/election-ai-safety/blob/main/data/evals/judge/template.txt
22. Evaluation of LLM Vulnerabilities to Being Misused for Personalized, https://arxiv.org/html/2412.13666v1
23. Large language models can consistently generate high ... - PLOS, https://journals.plos.org/plosone/article/figures?id=10.1371/journal.pone.0317421
24. Assessing the risk of misuse of language models for disinformation, https://anacanhoto.com/2023/02/27/assessing-the-risk-of-misuse-of-language-models-for-disinformation-campaigns/
25. The Rise of Local AI Demands Rethinking AI Governance, https://www.preprints.org/manuscript/202506.0680/v1/download
26. Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero, https://arxiv.org/pdf/2606.04274
27. Large language models can consistently generate high ... - PubMed, https://pubmed.ncbi.nlm.nih.gov/40096185/