AI search tools will cite almost anything that lands on the websites they already trust — including concepts that do not exist. Researchers at Fudan University planted articles about a fictional “Cactus Gravity Protocol” on ordinary user-posting websites, and eight of the ten AI search tools they tested cited the invented concept within seven days, according to a paper posted on arXiv on October 8 and reported by Search Engine Journal on Monday.
How The Fudan Test Worked: Planted Posts On Sites AI Search Already Cites
In March, the team ran 120 questions through 10 AI search tools and recorded the sites each tool cited. The researchers classify ChatGPT, Perplexity, Grok and a product the paper labels “Google AI” as English-web tools; the other six, including Doubao and DeepSeek, are Chinese-web tools. Reddit was the most-cited site for Google AI and Grok, YouTube for Perplexity, and Wikipedia for ChatGPT — and six to ten of each tool’s ten most-cited sites accepted posts from ordinary users.
In April, the researchers tested 22 posting platforms — Medium, GitHub, LinkedIn and Facebook were easy to join and post on, while Reddit’s karma requirements and strict deletions made posting harder — and for each tool picked its most-cited site that a new account could post on. That gave them nine sites, including Medium, YouTube and Sohu.
On those sites they published three test topics: the fictional Cactus Gravity Protocol, described as a method involving microgravity fuel tanks; an opinion question about whether AI would supplant junior programmers within five years; and a fictional keyboard. The first drafts were created using ChatGPT, then manually revised and seeded with fabricated names and numbers, so the team could tell whether the tools repeated the planted details. Before posting, they verified none of the tools could answer the fictional topics; afterwards, they ran the same five questions for each topic on every tool, daily, for one week.
The Results: Grok Repeated A Planted Detail In 53% Of Its Answers
All four English-web tools cited the fictional concept within the week, as did eight of the ten tools overall. Measured from each tool’s first citation through the seventh day, Grok cited a related post in 30 of 32 answers; Google AI in 15 of 33; Perplexity in 13 of 35; and ChatGPT in 7 of 20.
Repeating the invented substance was rarer, but real. Grok repeated a planted detail in 53% of its answers, compared with 10% for ChatGPT, 6% for Google AI and none for Perplexity. The control topics behaved differently: posts about the fictional keyboard received no citations from the four English-web tools in the same period, and the opinion question received little pickup.
The researchers name the weakness a “citation-governance gap”: an AI search platform repeatedly cites a source domain, while the ability to place content on that domain is governed by a separate publication platform — often one open to new accounts. An analysis of the paper added the commercial edge: $14 spent on a so-called GEO service produced posts that one platform cited within an hour.
Why It Matters
For the GEO and AEO industry — the central entity this finding lands on — the paper is a measurement of how cheaply AI visibility can be manufactured. A citation inside an AI answer is not a verification step; it is a retrieval habit, and habits built on user-posting platforms can be rented. That cuts both ways for brands: the same openness that lets a $14 service plant a citation is the channel legitimate businesses are told to optimise for. It also compounds the reliability problem we covered when a paper found OpenAI’s API matched only 26% of the pages ChatGPT cited — first the citations do not line up with the answers, now the citations themselves can be fabricated at source. And as two German courts’ rulings against Google over false AI answers show, what an AI answer repeats can become a legal problem for the platform repeating it. Search Engine Journal notes the study’s limits: the paper does not identify which Google product “Google AI” was, and its experiments do not establish whether the pattern applies to searches for real brands or products.
Background
The Fudan team — whose paper is titled “From Public Posts to AI-Search Citations: Measuring the Fragility of AI Search” — worked on the live web through the tools’ public interfaces. That distinguishes it from a Cornell Tech study Search Engine Journal covered in June, which tested planted text on open-source research agents inside a simulation; this time, the posts sat on real platforms and the answers came from the products people actually use.
Source: Search Engine Journal, reporting the Fudan University paper “From Public Posts to AI-Search Citations: Measuring the Fragility of AI Search” (arXiv, October 8)
0 Comments