How Hidden A.I. Reasoning May No Longer Stay Hidden

New studies challenge a core assumption of generative A.I.: that what models “think” can be kept hidden

A pair of recent research papers is unsettling one of the A.I. industry’s more convenient assumptions — that the internal instructions and intermediate reasoning of large language models can be concealed simply by not showing them to users.

In one line of work, researchers reported that they were able to recover hidden “reasoning traces” from leading models made by OpenAI, Google and Anthropic by exploiting encrypted blocks returned by their application programming interfaces. In another, researchers described a way to reconstruct a model’s original prompt from its output alone, with unusually high accuracy and without access to the model’s internal weights.

Taken together, the findings suggest that some of the most closely guarded elements of modern A.I. systems — system prompts, hidden chains of thought and internal reasoning artifacts — may be more extractable than companies had hoped.

That matters for more than academic curiosity. If verified at scale, the methods could expose proprietary prompt designs, leak sensitive data embedded in reasoning logs, open new avenues for jailbreaks and prompt injection, and make it easier for rivals to imitate the behavior of frontier models.

Extracting what models were not supposed to reveal

The first paper, posted this month, focused on an increasingly important feature of so-called reasoning models: the hidden intermediate steps they generate before producing an answer.

Companies generally do not display those raw traces to users. But some providers do return encrypted reasoning artifacts through their APIs so that models can preserve continuity across turns. Documentation from OpenAI and Google has indicated that such artifacts may be passed back in later requests to help the system maintain its reasoning state.

The researchers said they found a weakness in that design. According to the paper, encrypted reasoning blocks generated by stronger models in a family could be replayed into weaker sibling models that shared the same cryptographic setup. Those weaker models, once coaxed with jailbreak-style prompts, could be induced to reproduce the hidden reasoning in plain text.

The authors said the technique worked against multiple model families before vendors were notified. They also claimed it could be used not only to recover internal traces, but to extract secrets that may have been embedded in public logs and to create a novel form of prompt-injection attack by smuggling instructions inside reasoning artifacts that other models would treat as trusted.

Researchers and developers who reviewed the work described the recovered text as rough, internal notes rather than polished explanations — the kind of shorthand that model makers have long argued should remain hidden, both because it can be misleading and because it may expose proprietary methods.

The paper’s authors said the attack was no longer reproducible after disclosure to the affected companies, suggesting that at least some mitigations have already been put in place.

Reverse-engineering prompts from answers

A second paper, published at the end of July, points to a different weakness: the possibility of inferring what a model was asked, simply by studying what it said in response.

That method, called “Previous-Token Prediction,” uses a separate model to work backward from output text and reconstruct the input prompt token by token. The researchers said the approach was black-box, meaning it did not require access to the target model’s training data, architecture or weights.

In experiments described in the paper, the method reportedly achieved near-perfect reconstruction on some test settings, raising concerns for companies that rely on carefully crafted system prompts as a source of product differentiation or safety control.

Prompt secrecy has become an informal layer of defense across the industry. Model providers often build hidden instructions that shape tone, behavior, refusals, tool use and policy compliance. Start-ups, in turn, frequently treat those prompts as valuable intellectual property. If outputs reliably reveal those inputs, that protective layer may be far thinner than advertised.

How well the method carries over from controlled research settings to top production systems remains an open question. But even partial success could be significant in a market where competitive advantages can hinge on small differences in prompting and orchestration.

A broader shift in A.I. security

The new papers arrive amid a wider 2026 push by researchers to probe whether reasoning traces can be hidden at all.

Earlier studies had already suggested that withholding raw chain-of-thought text may not prevent others from reconstructing useful approximations. Some researchers have shown that traces can be elicited indirectly through prompting or inferred from answers and summaries, then used to improve the performance of smaller “student” models trained to mimic larger ones.

That has made chain-of-thought a particularly sensitive subject in A.I. policy and product design. Companies have argued that keeping raw reasoning private can reduce abuse, protect user privacy and prevent competitors from harvesting a frontier model’s problem-solving process for distillation. Critics, however, have questioned whether those protections are technically robust or mostly aspirational.

The latest work adds force to the skeptics’ case. It suggests that hidden reasoning may still leak through the very mechanisms designed to preserve model performance, and that prompts themselves may be recoverable from the outputs they produce.

Why the stakes are unusually high now

The timing is significant. The competition around generative A.I. has shifted from simple chatbot performance to deeper questions of reasoning ability, agent behavior and model provenance. Companies are pouring billions into systems that can plan, use tools and sustain context over long tasks. Those capabilities increasingly depend on internal scaffolding that is not visible to end users.

If that scaffolding can be extracted, the consequences extend beyond embarrassment.

For one, there is the risk of intellectual-property leakage: system prompts, hidden policies and reasoning workflows could be copied or adapted by competitors. There is also a privacy concern. If sensitive information enters hidden traces and those traces can later be recovered from logs or replayed sessions, then data thought to be effectively concealed may not be.

And there is a geopolitical edge. One report accompanying the new research suggested that recovered reasoning patterns may offer clues about whether some Chinese models were trained, directly or indirectly, on outputs from top American systems. That is a serious claim, but also one that remains difficult to prove conclusively. Similar outputs can arise for many reasons, and independent replication will be essential before such evidence is treated as definitive.

Still, the allegation underscores why this category of research is drawing so much attention. Distillation — training one model to imitate another — has become a central issue in a global A.I. race shaped by export controls, talent competition and strategic concern over who can replicate frontier capabilities most quickly.

What remains uncertain

Both research efforts are new, and much of the work is still circulating as preprints rather than fully vetted journal publications. That leaves important questions unanswered.

It is not yet clear how broadly the reasoning-trace attack worked across providers and versions before patches were applied, or whether similar flaws remain elsewhere. Nor is it certain that prompt reconstruction will perform as strongly on the most advanced commercial deployments as it did in the authors’ experimental setups.

There is also a more basic uncertainty: even when hidden traces are extracted, how faithfully do they represent what a model “really” reasoned? A.I. companies have long maintained that raw chains of thought can be noisy, incomplete or strategically generated rather than literal windows into cognition.

Even so, the direction of the evidence is becoming harder for the industry to ignore. The emerging picture is that omission is not the same as secrecy. Internal prompts and reasoning traces may be invisible in ordinary use, but that does not mean they are inaccessible.

For A.I. companies building products around hidden instructions and concealed reasoning, the burden is shifting from assuming those internals are safe to proving that they are.

Sources

Further reading and reporting used to add context:

AI News

How Hidden A.I. Reasoning May No Longer Stay Hidden

New studies challenge a core assumption of generative A.I.: that what models “think” can be kept hidden