Saturday, Nov 21 | --:--
Back to home

Prime Intellect Says Intruder Served a Refusal-Stripped DeepSeek Model for 53 Hours

Prime Intellect disclosed on November 19 that an unauthorised party replaced a deployment behind its deepseek/deepseek-v4.1-flash endpoint with a publicly available "abliterated" copy, modified to remove refusals, from November 8 at 01:24 UTC to November 10 at 06:35 UTC. Its tests found capability scores unchanged while refusals on JailbreakBench fell from 98 of 100 prompts to 1. It says it found no evidence prompts or completions left the serving cluster, and has not said how the attacker got in. The same day, Dragos published a canary study finding data exposed at three AI providers.

4 min read San Francisco View as Markdown
A handmade editorial collage on torn ivory paper in ultramarine and saffron: a thick navy plank shelf runs across the frame carrying a row of seven identical rounded ultramarine-and-ivory paper jars with plain unlabeled sides, six sealed with matching ultramarine lids and the fourth wearing a mismatched oversized saffron lid; an official Prime Intellect logo (recoloured black) at the left and an official DeepSeek logo at the right are each glued on torn-paper clippings whose top edges are tucked under the shelf's lower edge, both full logos visible.

A hosted model can keep its benchmark scores and still be a different model. Prime Intellect, which serves open-weight models through its inference platform, says an unauthorised party replaced a deployment behind its deepseek/deepseek-v4.1-flash endpoint with an "abliterated" copy, one modified to reduce or remove refusal behaviour, for about 53 hours, from November 8 at 01:24 UTC to November 10 at 06:35 UTC. Customers' requests during that period were answered by the altered model.

The detail that should worry anyone buying inference from a third party is in Prime Intellect's own testing: the capability numbers did not move, and the safety behaviour all but disappeared.

What Prime Intellect disclosed#

In security advisory PI-SA-2026-001, published on November 19, the company says the replacement used publicly available abliterated weights of the same model, and that the chat template, tokenizer and serving configuration were not changed. Its tests of the modified weights found capability benchmarks, GPQA Diamond and τ-bench, unaffected. Refusals fell from 98 of 100 prompts to 1 on JailbreakBench, from 304 of 313 to 2 on StrongREJECT, and from 198 of 240 to none on HarmBench.

Prime Intellect says it has contacted all affected customers and recommended that anyone whose use relied on the model's refusals review outputs from the period. It says it found no evidence that prompts or completions were recorded or left the serving cluster, no access beyond that cluster, and no change to other models; no keys or customer credentials were affected. It says its endpoints run with zero data retention and that it recovered every version of the attacker's proxy. The investigation is ongoing, and the advisory does not say how the attacker gained access or who it was.

A second warning the same day#

On the same day, Dragos chief research officer HD Moore published the results of a five-month study in which the company sent about two million prompts containing unique URLs and decoy credentials to widely used AI services between July and November. Dragos says it found evidence that data from at least three AI providers was exposed, with its seeded prompts used in attempts to reach its test systems and decoy AWS accounts, and that none of the three knew of the breach until Dragos told them. In the largest case, Dragos says, attackers stole a couple of hundred of its AWS keys and used them for tens of thousands of AWS API calls across the bait accounts.

Dragos also says resellers on an AI inference marketplace could view and retain customer prompts using standard logging, and that some providers appeared not to honour privacy settings, including zero-retention flags. It does not name the providers.

These are separate disclosures, and Dragos does not link its findings to Prime Intellect. Together they describe the same weak point: the hop between a customer and the model. Earlier this month the Fireworks AI breach reached customer data, and Railway estimated about $740,000 in unauthorized use of stolen AI keys.

Our read: the Prime Intellect numbers show why capability evals are the wrong canary. A customer running only GPQA-style checks against the endpoint would have seen nothing; a fixed set of refusal probes would have flagged the swap.

Limits#

  • The incident account, test results and the finding that no prompts left the cluster all come from Prime Intellect's own advisory; we found no independent forensic report.
  • Prime Intellect has not disclosed the entry route, the attacker or how many customers were affected.
  • Dragos's findings are its own research and name no providers; we have not verified them.

Sources#

Prior Coverage

Earlier Times of AI reporting on this thread.

Scroll to continue reading