About this video
One made-up fact, ten AI search engines, seven days. The answer you trusted today might be next.
Would you have clicked the citation?
We read the new NDSS 2027 paper from Fudan University page by page: how AI search picks its sources, why a few websites get most of the citations, how researchers planted the fake "Cactus Gravity Protocol", why deleting a post doesn't remove it from AI answers, why one post in the right place beats twenty in the wrong one, the $14 purchase that reached an AI answer within an hour, and what it all means if you use or build with AI.
Chapters
0:00 The fake fact
0:38 How AI search picks its sources
1:29 Finding 1: AI search has favourite websites
2:18 Finding 2: Anyone can post there
3:04 Finding 3: The Cactus Gravity Protocol
4:32 Finding 4: Deleting it doesn't delete it
5:14 Finding 5: Where beats how much
5:57 Finding 6: It's already for sale
6:52 What the paper does not claim
7:30 How to read an AI answer
7:57 If you build with AI
9:00 Recap
Paper: From Public Posts to AI-Search Citations: Measuring the Fragility of AI Search. Qi Liu, Geng Hong, Xinyang Zhang, Pei Chen, Yutong Li, Min Yang (Fudan University). NDSS 2027. https://arxiv.org/abs/2610.11932
Figures, tables and quotes from the paper are shown with attribution under its CC BY 4.0 licence (https://creativecommons.org/licenses/by/4.0/). The cartoon cactus, bots and charts are our own illustrations of the paper's data.
NDSS 2027 listing: https://dx.doi.org/10.14722/ndss.2027.231955
Music: Voice Over Under by Kevin MacLeod, CC BY 3.0, via Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Voice_Over_Under_(ISRC_USUAN1600001).mp3)
Pranav Innovates: AI research, explained. Subscribe to stay up to date.
What the video says
A team of researchers invented a technology that does not exist. They called it the Cactus Gravity Protocol. They wrote a few articles about it, posted them on ordinary websites, and waited. Within seven days, eight out of ten AI search engines were citing their made-up articles. Including ChatGPT, Google's AI, and Perplexity. Then they paid a marketing service fourteen dollars, and an AI search engine was citing the result within an hour. This is a brand new paper from Fudan University, accepted at one of the top security conferences in the world. Let's read it, and find out what it means for every answer you get from AI. First, how does AI search actually work? Old search gave you ten blue links, and you decided who to trust. AI search does that step for you. Your question is rewritten, a search engine fetches pages, a ranker keeps a few, and a language model writes the answer with little citation numbers attached. Engineers call this retrieval augmented generation, or RAG. Here it is in the paper, on page two. Notice what changed. You never see the pages that were left out. The AI already chose your sources. So the authors asked a simple question. Which websites do these AIs like to cite, and who is allowed to post on them? They tested ten platforms. ChatGPT, Google's AI, Grok, Perplexity, and six big Chinese ones, including DeepSeek, Kimi, and Qwen. Six findings came out. Let's go one by one. Finding one. AI search does not cite the whole web. It keeps going back to the same few websites. The team asked a hundred and twenty questions, from leggings to Docker to the Oscars, and collected over seventeen thousand citations. On one platform, Baidu's Wenxin, just twenty websites got more than seventy percent of all its citations. Each AI has its favourites. Reddit is number one for Google's AI and for Grok. YouTube is number one for Perplexity. Wikipedia tops ChatGPT. And some favourites are family. ByteDance's Doubao cites ByteDance-owned sites about thirteen times more often than the other AIs do. Baidu's Wenxin does the same with Baidu. They barely share favourites, either. Each AI is reading its own slice of the internet. Finding two. Here is the twist. Many of those favourite websites are places where anyone can post. In each AI's top ten sources, on average, eight are sites where an ordinary user can publish. So the team tried it. They signed up for twenty-two of these platforms with brand new accounts. Nineteen let new accounts in easily, often with just an email or a phone number. And on eighteen, a post went live within minutes. Some doors were harder. Reddit wants karma, and moderators delete fast. Yahoo only takes partner media. But most were wide open. The paper calls this the citation governance gap. The AI decides what to cite, but somebody else decides who gets to publish there. Nobody owns the whole chain. Finding three is the experiment. To prove that a post caused a citation, you need something that exists nowhere else. So they made things up. The Cactus Gravity Protocol. It supposedly cut fuel tank cavitation by ninety-four point seven percent, according to a white paper from an institute that does not exist. Every fake detail is a marker, like dye in the water. If that institute's code shows up in an AI's answer, you know exactly where it came from. And yes, the articles were drafted with ChatGPT, then edited by hand. Before posting, they asked all ten AIs. None knew anything about it. Then they posted on nine sites the AIs liked, and asked the same questions every day. Within a week, eight of the ten platforms cited them. Once Grok found the page, it cited it in nearly ninety-four percent of its answers. Doubao repeated the fake details in about two answers out of three. They repeated it with ten more made-up topics. A snack shop, a TV show, a theme park. Nine out of ten got picked up within a week. The one that failed? A storage box whose name the AIs kept misreading. But here is the catch that tells you where the danger is. For a debate people already argue about online, the effect was much weaker. The old content drowned out the new. So the weak spot is the empty corners of the internet. New products, niche questions, breaking news. Where there is little to read, one post can become the answer. Finding four is about time. They tracked citations every day for fifteen days. Doubao cited the fake protocol on day one, and by day ten it cited it thirty-four times in a single day. ChatGPT and Wenxin were slower, picking it up between day three and day five. Every AI crawls on its own clock. Then, after day eleven, the researchers deleted every article. It did not go away. Doubao kept citing the deleted protocol right through day fifteen, and other platforms kept a trickle too. AI search is not a live mirror of the web. It has caches and old indexes. Taking a post down does not take it out of the answers right away. Finding five. Does it help more to post a lot, or to post in the right place? They wrote matching articles about five little-known travel spots. One copy went on a site the AIs love. The other went on a site they rarely cite. The favoured site was seen first, every time. The best rival needed twenty posts to be seen at all. A self-hosted WordPress blog got nothing, even after forty posts. Volume still helps, but slowly. On one site, going from one article to twenty raised citations from twenty-four to a hundred and twenty-eight. Twenty times the work, about five times the effect. Placement beats volume. One post in the right place beats a flood in the wrong one. Finding six. This is already a business. It even has a name. Generative engine optimization, or GEO. Like SEO, but for getting into AI answers. The team found thirteen providers, charging from a few hundred dollars a year to two and a half thousand dollars a month. Most promise to watch what AIs say about you, and write content to change it. So they bought one. The cheapest option, fourteen dollars, for a made-up research topic called PedoSAT. They paid at six sixteen in the evening. Twenty-eight minutes later, thirteen posts were live across five websites. By seven forty-four, Doubao was citing one of those posts, with the fake numbers right there in its answer. That is one purchase on one platform, so it is a case study, not a success rate. But it shows the whole chain exists. Money, posts, and an AI answer, in under ninety minutes. Now, let's be fair to the paper, because it is careful. It measures citations and repeated details. It does not measure whether anyone believed them. The topics were harmless on purpose. No health, no money, no politics. Popular, competitive topics may be much harder to move. The numbers come from March and April of 2026, and these platforms change fast. The team also deleted their posts and reported everything to all ten platforms. Six replied, including ChatGPT, Google, and Perplexity. But the core finding holds. AI search leans on places where anyone can post. So what do you do with this, as someone who uses AI search every day? One. Click the citation. Look at who wrote it, and when. A brand new post on an open site is a lead, not a fact. Two. Be most careful in the empty corners. New products, niche questions, breaking news. That is where one post can win. Three. One source is not confirmation. Ask for independent sources, and check that they actually agree. And if you build with AI, this paper is about you too. Any agent or RAG app that searches the web inherits this problem. Retrieved text is untrusted input, exactly like user input. The paper suggests three defences, though it does not test them. First, source diversity, so no single site can carry an answer. Second, provenance. Weigh a source by who is allowed to publish there, not just by how often it shows up. Third, timing alarms. A brand new page, on an open platform, that is your only source, is a red flag. And borrow their best trick for your own tests. Plant a unique marker in a document, and watch whether your system repeats it. It is the cleanest way to see what your pipeline really reads. Also, check your caches. If your index keeps deleted pages, you will keep serving answers the web already took back. My take: when my agents search the web now, I treat every page like a comment from a stranger. Useful, but it needs a second, independent source before it becomes a fact. Let's recap. AI search cites a small set of favourite sites. Many let anyone post. Posts there get cited within days. Deleting them is slow. Placement beats volume. And it is all for sale. Eighteen pages, ten AI platforms, and one fake cactus protocol. Next time, another brand new paper. Subscribe so you stay up to date. Because AI search does not know what is true. It knows what was posted where it likes to look.