Patrick Collison recently noted that AI agents are still stuck in Terminal UIs:

I tend to agree, as do others. But why is this the case? Why haven’t we gotten past the MS-DOS era of AI Interfaces?

While the models are incredibly smart, they are so slow. If we want real “AI Intefaces”, we need models that are smart and fast. Low latency is critical for interactive applications!

I’m inspired by Krea’s Realtime Edit, where an image is generated while you draw or prompt, seemingly instantly:

Why can’t the rest of our “AI apps” feel this interactive? Perhaps they can, and soon!

Here’s a rough prototype of what “low-latency” AI Interfaces could look like in other domains. This is not faked nor sped up:

Remember that, like in the past, this is the dumbest and slowest the models will be. It’ll only get better from here.

Smart Enough; Then, Fast Enough

A few years ago, the bottleneck for AI was intelligence. GPT-3 simply wasn’t smart enough to complete compelling tasks. We’ve since hill-climbed that gap over 6 years, and with reasoning models we can accomplish incredible tasks at the cost of very, very long latency. Some models end up working for over a day… a fact that AI labs like to brag about!

Even if we can approximate faster feedback with Luna, it’s still much too slow for interactions like click, drag, etc. If we want snappy AI interfaces that feel like magic, we need AI models to complete responses within 100ms1.

That sounds hard, but I’ve learned not to doubt the number of talented, motivated researchers out there.

Can Autoregressive Transformers Get Us There?

Astute observers will question whether the demos above used the Mighty Autoregressive Transformer… and they’d be correct. The “real-time” models are, in fact, Diffusion models.

In my demo, I used a Diffusion Large Language Model (dLLM) from Inception Labs: Mercury 2. It was the best diffusion text model I could find that had an API, since I would’ve had to host DiffusionGemma.

The hesitancy around Diffusion models is that historically they haven’t been as smart as LLMs, and that they’ve been tricky to post-train. Fortunately, the outlook has changed a lot in the last year.

Kaiming He - Mr. ResNet himself - is back, publishing bangers about diffusion. Some of his work piques my curiosity, but it sounds like a big focus will be engineering work to train diffusion text models at scale. Nicholas Boffi’s group has also quite bit on diffusion text models and distillation.

Here’s a more comprehensive research recap for diffusion models by Sander Dieleman:

“Continuous diffusion language models”

How Will It Shake Out?

Taking it all in, I think the following is sensible to expect:

  • Smart + Real-time AI is not only tractable but inevitable. Once generally available, we’ll hit a new era of “AI Interfaces” that will make 2026 look like the ancient past.
  • I don’t know if Diffusion or Drifting or something else is the answer. But it is obvious that autoregressive models can’t be used for the AI Interfaces we want. However, they will be used for “long-horizon” tasks where intelligence is table-stakes, and certainly for labeling.
  • Outside of Deepmind, none of the AI labs are seriously focused on dLLMs. This means that a neolab can take the lead, assuming they are not acquired beforehand.

Strap-in, folks. It’s only just the beginning.

Footnotes

  1. In “How Much Faster Is Fast Enough”, direct and indirect taps have JND’s around 90-100ms. This seems like a nice proxy for “magical, instant” feedback driven by AI, but happy to look at other papers. ↩