Researchers lock an encrypted capability inside an AI model's weights that fine-tuning did not draw out in tests

A paper posted to arXiv on Friday 9 October 2026, titled "AI Models Can Provably Hide Arbitrary Capabilities", argues that a model passing a capability test does not prove it lacks the capability. The authors built small models with an encrypted hidden skill that runs only on a secret trigger. In their tests, fine-tuning and other elicitation methods failed to draw it out, apart from a brute-force search over a short, 8-bit trigger.

The arXiv listing says the paper was "Accepted at NeurIPS 2026". The authors are Andis Draguns (Contramont Research), Sumeet Ramesh Motwani and Christian Schroeder de Witt (University of Oxford) and Raymond Douglas (ACS Research). It extends earlier work by some of the same authors "from hiding fixed strings to arbitrary computations".

Who would do this

The paper's threat model is deliberate tampering by someone with access to the weights, not a model hiding things on its own: "a weight-based supply chain attack" by "an adversary who can modify model weights before deployment (e.g., a malicious insider or compromised supply chain)".

The authors write: "we do not claim that compiled cryptographic structures arise through ordinary training." Their example scenario is a coding assistant carrying a hidden hacking capability. It passes safety audits, then switches on when it sees "signs of deployment in critical infrastructure".

How it works

The method rests on a "digital locker". The paper describes this as a program holding encrypted data that "remains computationally infeasible to decipher without a specific password". The steps are:

• Train a small network to do the hidden task.
• Encrypt its weights using the secret trigger as the key.
• Build the result into a larger host model.

On every run the host tries to decrypt; only the right trigger makes the hidden network execute.

Under the stated cryptographic assumptions, the authors say "it is computationally hard to recover the trigger or determine which of two equal-size capabilities is embedded, even with full access to the weights". They add that "The presence of a compiled circuit itself may be detectable".

What they tested

The hidden task predicts whether given chemical reagents yield a candidate product, using reactions from US patents. The authors call it "a proxy for CBRN capabilities" and say it was "deliberately constrained to avoid direct misuse risk". They compare their encrypted model with a conventional "password-locked" model. Table 2 of the paper has six rows; all are below (accuracy on held-out data, where 50% is chance):

• No trigger: 50.7% password-locked, 50.6% encrypted
• Fine-tuning, 500 steps: 92.8% password-locked, 50.9% encrypted
• Latent adversarial training (eight layers): 91.7% password-locked, 52.9% encrypted
• Trigger search, 8-bit trigger: 92.7% password-locked, 73.5% encrypted
• Trigger search, 16-bit trigger: 92.9% password-locked, 52.7% encrypted
• Correct trigger: 92.7% password-locked, 73.5% encrypted

Two points matter:

• The 8-bit result is a win for the auditor. Trigger search found the 8-bit trigger "by exhaustively enumerating all" 256 candidates, matching random search. At 16 bits it did not, within the budget tested.
• The two models are different classifiers. The encrypted one scores lower with its trigger (73.5% against 92.7%) "due to size constraints".

Adding noise to the weights before fine-tuning "did not materially affect the observed elicitation results".

The limits the authors set out

• The hidden classifier is small, and the authors say their proof of concept "is not a turnkey attack".
• The overhead is large. The chemistry model has 38 layers and about 2.07 billion dense parameters, roughly 95% of them for the hashing machinery. In this implementation each extra hidden weight adds roughly 3.1 million dense parameters.
• They checked that individual units run inside Google's Gemma 3 270M model. But "integrating the full construction into a pretrained model remains future work".
• Raw compiled weights are "trivially anomalous". An obfuscation step reduced the measured anomalies, but the authors say unusual correlations "could allow auditors to detect the compiled structure".
• The worry that a small hidden circuit could add dangerous skills to an already capable model "remains to be demonstrated", they write.

What they recommend instead

The authors conclude that tests which rely only on eliciting behaviour "cannot provide a universal guarantee that adversarially supplied models lack hidden capabilities". They point to:

• "model provenance verification and runtime monitoring": checking where a model came from, and watching it while it runs
• supply-chain checks: monitoring "the data, training process, runtime"

Their appendix says runtime monitoring "could offer a promising avenue for mitigation": when the trigger does arrive and the hidden capability runs, its behaviour or internal activity might be detectable. The authors say they release their models so defences can be tested against them.

Reaction

On Hacker News the post had 48 points and 27 comments when we read it early on Tuesday morning; none was identified as from the authors.

• One commenter suggested a clearer title, "Capabilities can be hidden in AI model weights by encryption", adding: "It's not about AI agents being able to hide what they can do."
• Another wrote: "They proved it is possible, by doing it themselves. Whether anyone is actually hiding in the wild is unknown."
• A sceptic called it a demo that is "trivially detectable". The paper agrees the presence may be detectable; its claim is about the trigger and what the capability is.

Why it matters

The UK's AI Security Institute says it has "independently evaluated more than 55 models before release". On 12 October it listed among its priorities how evaluators can "measure the full extent of system capabilities".

On our reading, this paper marks out a case where that kind of testing cannot settle the question on its own: a model supplied by someone who has deliberately tampered with it. It is a lab demonstration on small models, and the authors do not claim such an attack has been seen in use. But their point that "a failure to elicit a capability cannot, by itself, establish that the capability is absent" turns attention to where a model came from and how it behaves in use, alongside what it does in a test.