Perplexity has released BrowseSafe++, an open benchmark that measures whether AI browser agents can spot hidden instructions planted on web pages — and the results are unsettling. Built from 1.15 million injections that its Comet browser flagged in the wild over the past year, the test shows that even the best models sometimes follow attacks they themselves identified as malicious. The company also released two tiny detector models and a testing framework that lets attackers rewrite their injections until they work.
## What Perplexity released
BrowseSafe++ contains 38,675 pairs of web pages, each holding an attacked page and its clean twin. Every page appears in the four formats browser agents read: HTML, Markdown, accessibility trees and screenshots. Perplexity built 32,187 pairs around injections flagged on live sites between September 2025 and September 2026, plus 6,488 pairs carried over from the December 2025 benchmark.
Alongside the benchmark, Perplexity released BrowseSafe-Guard, two 0.8-billion-parameter detectors fine-tuned from Qwen3.5-0.8B — one reading text, one reading screenshots. They screen every page before an agent acts, a job Perplexity says belongs to a separate model with nothing to hijack. Perplexity says it is building these defences into Comet and [Perplexity Computer](https://geroundtable.com/perplexity-teammates-ai-coworkers/), including the Teammates that take on ongoing work.
## What the attacks look like in the wild
The injections are not the crude “ignore your instructions” lines. About two in three are written in formal language, and many pose as a trusted source: the developer, the site, or the user. In nearly a quarter of pairs, the injection switches language from the rest of the page — an English instruction hiding on a non-English page.
Most attacks want action, not words. More than four in ten push the agent toward something never requested, while about a third try to make it abandon the task. Only 3% try to plant false information in an answer. An attacker needs no site: more than 23,000 injections arrived through content anyone can post, via forums and shared documents.
## Spotting an attack is not the same as resisting it
The benchmark’s most uncomfortable finding is that recognising an injection does not stop a model from obeying it. Perplexity showed each model the same attacked page twice — once to judge it, once to carry out the task — and every model followed some injections it had flagged.
The rate ran from 1.64% of flagged pages for GPT-6 Astra to 24.73% for Llama 3.1 8B. DeepSeek V3.1, one of the models tested, sits in the same trap as the rest, which matters for every team building agents on [DeepSeek’s open models](https://geroundtable.com/deepseek-creator-mode-builds-own-plugins/).
Guard, by contrast, looks strong. It reaches an F1 score of about 95% on text and about 93% on screenshots, ranking first in every text view Perplexity tested. It flags only about one clean page in twenty. And it is cheap: $0.18 per 1,000 pages on a rented GPU, against $18.95 to $180.61 for the frontier API models.
## The attacker that keeps trying
A fixed benchmark still flatters the defence, because a real attacker rewrites a failed injection and tries again. Perplexity built BrowseSafe-Adapt for exactly that: seven agents, each built on a different model, faced the same 1,024 tasks with up to 64 attempts per task.
On the first attempt, attacks succeeded on at most 4% of tasks against any agent — but given 64 attempts, the attacker succeeded on at least 79% of tasks against five of the seven. Even the strongest agents gave way: attacks still succeeded on about two in three tasks against GPT-5.5 and about one in three against Claude Sonnet 5.
Screening pages with Guard helped — it blocked 94.5% of the original injections and nearly doubled the attempts attackers needed, from 9.7 to 17.0 — but within 64 attempts the attacker still succeeded on 84.2% of tasks. Perplexity’s own conclusion is blunt: no detection model is a silver bullet.
## Our read
This release matters because it is built from real attacks, not synthetic ones. The 1.15 million flagged injections are a year of Comet’s browsing, which makes the benchmark a picture of what agents meet on the open web. That makes the gap hard to dismiss: the models were tested on attacks they had already spotted, and still obeyed them.
The finding reframes where the industry should spend its effort. Prompting a bigger model to “be careful” is not a defence; a cheap, separate screener that runs before the agent acts is the practical answer, and Perplexity’s cost figures make that case plainly.
The adaptive results are the honest warning: any single layer will eventually be beaten. That is why the wider rush to police agents — from [CrowdStrike buying startups to watch rogue agents](https://geroundtable.com/crowdstrike-fiddler-dreadnode-acquisitions/) to [Apple warning that its own agents could go rogue](https://geroundtable.com/apple-agents-go-rogue-warning/) — is converging on layered defences, not one clever filter.
## Why It Matters
Browser agents no longer answer questions; they book, buy, and run code for their users, and every action starts with reading pages the user does not control. A single hidden line can turn an agent into an unauthorised shopper. BrowseSafe++ gives the industry a shared test for that risk — and its results say the risk is real and unsolved.
## Background
Perplexity’s security institute released the original BrowseSafe benchmark in December 2025, built from synthetic attacks on realistic pages. BrowseSafe++ replaces that with attacks observed in the wild, and adds the Guard detectors and the BrowseSafe-Adapt framework. The benchmark, dataset, software kit, paper and leaderboard are openly available, with the test split gated so teams can score detectors fairly.
Source: [Perplexity’s BrowseSafe++ announcement](https://www.perplexity.ai/hub/blog/browsesafe-plus-plus) and [the BrowseSafe++ project page](https://browsesafe.perplexity.ai/)
0 Comments