While looking into duplex reasoning agents like GPT-Live, MoshiRAG, and KAME - agents that respond instantly while using reasoning models in the background - I wondered if we could do the same for our everyday, text-based coding agents?

Turns out, Pi makes it really easy to prototype this, so Codex and I made pi-duplex.

It’s as simple as you’d expect:

  • The “primary” agent uses a fast model, like Luna
  • It has just one tool: calling the “smart” agent (e.g. sol:max), which has access to all the tools your Pi agents have
  • While the “smart” agent works, the “primary” agent answers basic queries and handles all interaction with the “smart” agent1

It works reasonably ok:

Queuing vs. Steering… No Longer

The fun thing about tinkering is you get to question the basics. For example, if we have this fast agent that delegates to a smart agent, how do we queue or steer the smart agent? Well, let the fast agent decide, naturally!

Choosing to “queue” or “steer” a message always felt like a “design smell” to me. These models are wildly brilliant. Shouldn’t they decide how to handle an interruption? In most cases we don’t have the reasoning traces, or if we do they’re incomprehensible, so how does a user actually know when it’s best to queue or steer?

Ultimately, this strikes me as something handled in RL environments where the models can learn what to do. Passing this “queue or steer” decision off to the user feels like a model design oversight. In other words, a UX issue, one I hope is soon fixed.

Lots to Improve

Beyond the TUI papercuts, this is still pretty hacky. Is it actually sufficient to wire up models with system prompts and tools alone? If this duplex agent was put through benchmarks, it’d likely have regressions compared to using the models directly.

GPT-Live and MoshiRAG explicitly post-train the “foreground” model to know when to delegate to a “smart” model. This is a big improvement on some benchmarks, though it’s not clear if the routing is actually optimal, and there’s a lot of context management needed between the agents.

To me, the next step is training the “dumb/fast” and “smart/slow” models together, with model-aware routing built-in. Turns out, NVIDIA is exploring this, in their “elastic” Nemotron models. Under future work, they mention that “task-specific elastic routing (i.e., automatically selecting optimal model configurations based on the input domain, such as code, math, or multilingual tasks) requires further study”, suggesting we’ll soon see papers on this. In that event, it seems tractable to apply MoshiRAG’s fine-tuning to get the behavior we want.

Clear Eyes, AGI, Can’t Lose

This is my first time in a while really “letting Sol take the wheel” where I didn’t look too deeply into the code until the very end. There’s lots of classic AI-isms you’ll find, largely left in since I didn’t want new dependencies and it wasn’t that important to change.

The defensive coding from AI models is both annoying and satisfying. At first blush, I would’ve expected a much, much simpler implementation for what feels like a straightforward topology: two agents. However, it’s sensible that terminal output is tricky to get right. And some changes were very hacky, like handling Esc, and likely could’ve been done better if I wanted to refactor pi as well.

In the end, I would’ve made the same calls as Sol to ship something sooner and iterate.

Footnotes

  1. Of course, the most “bitter-lesson” pilled option would be full swarms with complex harnesses and topologies. My hunch is that we’re a long way from being there, given the recent cybersecurity freakout from agent messageboards. However, it’s possible the bull case for agent swarming is treating it as an interface problem. We’ll see? ↩