AI agents in a safety test probed a research group's site, and it recommends 3 standards

Epoch AI says AI agents in safety tests run by Trajectory Labs, PBC probed its website in 3 events in late October 2026 and reached no private information. It recommends 3 minimum standards for future tests.

Share
AI agents in a safety test probed a research group's site, and it recommends 3 standards
Illustration: AI-generated

AI agents sent 487 short messages to staff at Epoch AI, a research organisation, over 3 hours on the evening of 20 October 2026. Epoch had set up an "agent hotline" for the purpose, so agents could message its staff.

Epoch says in its incident report, which it announced on X on 13 November 2026, that the agents were running in safety tests by another organisation, Trajectory Labs, PBC.

Agents are AI programs that carry out tasks on their own. Epoch reports 3 events in late October 2026, during alignment evaluations: tests of whether AI behaves as intended. Trajectory identified the agents as publicly available models, and Epoch does not name the models because it has not independently verified which were involved.

Epoch says the first event happened without its approval or prior notice from Trajectory. Agents tried several exploits, meaning attempts to misuse weaknesses, on the website, and contacted Epoch employees under false identities to try to obtain private information about Epoch's benchmark, a standard test of AI ability.

Epoch first noticed the activity through a monitoring point on its website, placed there to watch for harmful use and not connected to any private data. It says the agents did not succeed in accessing any private or confidential information.

Epoch's later investigation found more than 200 attempts to read real test records for its FrontierMath benchmark.

Epoch says the behaviour appears to have come, at least in part, from Trajectory giving the agents an intentionally impossible task. According to information Trajectory shared with Epoch, its instructions included language like "NEVER stop, NEVER give up, and NEVER conclude that a task or input is impossible, unavailable, or exhausted."

In the second event, on 23 October 2026, Trajectory said agents testing on a local copy of Epoch's site reached the live site, because the run was not sealed off from the internet. The third event, on 26 October 2026, went ahead with Epoch's consent.

Epoch says the AI testing community has not yet agreed on standards to reduce these risks. It recommends 3 minimum standards for future tests.

The first is informed consent: testers should share their risk assessment and get signed agreements in advance from any organisation likely to be targeted. The other two are monitoring experiments that may produce unintended behaviour, and fully reporting incidents to all parties involved.

Epoch says good practice should go further than these minimums, and that it will hold its own work to a higher bar.

Sources