An evidence-led field guide
Neuralese names the numerical representations AI systems use before they produce words. The term began with messages between artificial agents. It now also points to a harder question: what happens when a model reasons without translating each step into language?
Definition
2017
Andreas, Dragan, and Klein used "Neuralese" for real-valued messages learned by agents that coordinate through a differentiable communication channel. The messages work, but humans cannot read them directly.
Andreas et al., ACL 2017Current usage
Recent writing uses the term more broadly for intermediate computation that stays in continuous hidden states. In the research literature, latent reasoning or continuous chain of thought is usually more precise.
Chen et al., 2025 surveyCalling it a language is a metaphor, not a measurement.
A necessary limit on the term
Mechanics
The model converts input tokens into vectors. Each layer transforms them using attention and learned weights.
Standard chain of thought decodes intermediate steps into tokens. Latent methods keep one or more steps as hidden states.
Coconut uses the last hidden state as a "continuous thought" and feeds it back as the next input embedding.
The model returns to language mode and produces an answer that a person can read.
"We feed it back to the LLM as the subsequent input embedding directly in the continuous space."
Coconut paper, Hao et al.What the evidence says
The 2017 method treats a vector and a sentence as equivalent when they induce the same belief about the world in a listener. The authors report little reward loss when agents communicate through the translation layer.
Coconut outperformed written chain of thought on selected logical planning tasks and used fewer thinking tokens. The authors describe hidden states that retain several possible next steps, which resembles breadth-first search.
Later experiments found Coconut sensitive to training choices. Reinforcement-learned latent reasoning still trailed language-space chain of thought on mathematical reasoning. Faster or broader reasoning remains a research claim, not an established property.
What we do not claim: that hidden states have a fixed dictionary, that every model uses the same code, or that current frontier systems run a deep private monologue. Those claims outrun the public evidence.
Interpretability and safety
OpenAI found that monitoring written chains of thought can reveal reward hacking that is harder to catch from actions and final answers alone. Its later monitorability work formalizes how well a monitor can predict properties of an agent's behavior.
Anthropic found that a model's written reasoning can omit factors that changed its answer. A chain of thought is evidence about computation, not a transcript of it.
If useful intermediate work never becomes language, text-based monitors lose access to it. Researchers would need activation probes, causal interventions, behavioral evaluations, or other controls. None currently yields a complete account.
Neuralese may remove a token bottleneck. It may also remove the part of reasoning we know how to inspect.
Research lineage
"Translating Neuralese" defines the original machine-learning problem: interpret a protocol that agents learned for themselves.
Coconut makes continuous hidden states part of the explicit inference procedure.
Surveys organize latent chain-of-thought methods. AI 2027 uses "neuralese" for recurrent internal computation in a speculative forecast.
Can latent steps improve capability without making model behavior harder to test and control?
FAQ
In AI research, Neuralese originally meant learned vector messages exchanged by artificial agents. The term is now also used, less formally, for reasoning that remains in a model's continuous hidden states instead of being decoded into words.
Not in the ordinary linguistic sense. It has no shared vocabulary, stable grammar, or model-independent dictionary. It is a metaphor for learned numerical representations whose meaning depends on a model, task, and receiver.
All neural language models compute with hidden numerical states. Calling those states thought or language is an interpretation. Some research systems explicitly reuse hidden states for several reasoning steps, but that does not prove that deployed models use a coherent secret language.
Continuous chain of thought keeps intermediate reasoning in latent vectors. Instead of decoding every step into a token, a method can pass a hidden state back into the model as the next input.
Coconut alternates between language mode and latent mode. In latent mode it takes the model's last hidden state, treats it as a continuous thought, and feeds it back as the next input embedding.
Sometimes, on selected tasks. Coconut reported gains on planning problems that require backtracking and used fewer thinking tokens. Later work found the gains sensitive to design choices and not yet competitive with language-space reasoning on harder mathematics.
The 2017 work translated agent messages by matching their effect on a listener's beliefs, not by assigning a word to every vector. Hidden-state interpretation remains an open research problem.
Readable reasoning can help monitors detect reward hacking and deception. Reasoning that stays latent removes that source of evidence. Written chains of thought are already imperfect, so neither readable nor latent reasoning should be treated as a complete audit trail.
Primary sources
Jacob Andreas, Anca Dragan, and Dan Klein · 2017
Shibo Hao et al. · 2024
Xinghao Chen et al. · 2025
Enes Özeren and Matthias Aßenmacher · 2025
Lanham et al., Anthropic · 2023
OpenAI · 2025
OpenAI · 2025
Daniel Kokotajlo et al. · 2025
Source standard: peer-reviewed papers, preprints, and first-party research pages lead. Forecasts are labeled as forecasts. This guide does not treat a model's self-description as evidence of its internal computation.