EPS Dashboard

Authors: Cedric Flamant, Udaya Ghai, Kanna Shimizu

Summary

The Bicameral Model connects two frozen pretrained language models through a tiny learned interface (~1% of total parameters) that lets them exchange information via hidden states rather than text. One model handles the main task while the other operates tools (calculator, logic solver, code interpreter), and both condition on each other's internal activations at every generation step. A learned "suppression gate" decides what gets communicated, trained only from task loss without any prescribed protocol. On arithmetic, coupling two 0.5B models with a calculator jumps accuracy from 36% to 96%.

Main takeaways:

Two frozen LMs can coordinate through continuous hidden-state channels instead of generating text back and forth
A trainable translation network plus suppression gate (~1% parameters) learns a communication protocol from task loss alone
On arithmetic (calculator backend), accuracy rises from 36% to 96% with two 0.5B models
On logic puzzles (Z3 solver backend), 1.7× improvement over baseline with two 0.6B models
The auxiliary model can generate problem-specific code from hidden-state signals without ever seeing the problem text

Relevance

Not directly related to my persona installation work, but loosely analogous to activation steering: this shows that models can be steered via internal state manipulation rather than prompt engineering. The suppression gate learning what to communicate from task loss alone is conceptually similar to finding which activations matter for behavioral changes.

Abstract

arXiv:2605.11167v1 Announce Type: new Abstract: Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretrained language models instead coordinate through a continuous, concurrent channel? The Bicameral Model couples two frozen language models through a trainable neural interface on their intermediate hidden states. At every generation step, both models run in lockstep: a primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code, with both conditioning on each other's activations through a translation network and a learned suppression gate ($\sim$1% of combined parameters). The gate learns a selective communication protocol from task loss alone, without a prescribed format. We demonstrate the mechanism across three tool backends. On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36% to 96%. On logic grid puzzles, coupling two 0.6B models with a Z3 solver achieves $1.7\times$ the unaugmented baseline on ZebraLogic. On mathematical reasoning, coupling with a Python sandbox enables the auxiliary to generate problem-specific code from hidden-state signals alone, without ever seeing the problem text.