Redesigning Human-AI Interaction: How Thinking Machines Lab's Interaction Models Enable True Real-Time Collaboration

Most AI systems today operate on a turn-based loop: you speak, the model listens, processes, and responds. This creates a narrow channel that limits how much of your intent and context the model can perceive. Thinking Machines Lab argues this is a fundamental bottleneck. Their solution, called interaction models, makes interactivity native to the AI itself—not an afterthought. In this Q&A, we explore how their architecture transforms real-time collaboration.

What is the fundamental flaw in current turn-based AI systems?

Turn-based AI models have no awareness of what you're doing while you are still speaking or typing. They cannot see you pause mid-sentence, notice your camera feed, or react to visual cues in real time. While the model is generating a response, it remains blind—perception freezes until the output is complete. This creates a narrow channel for human-AI collaboration, limiting how much of a person's knowledge, intent, and judgment can reach the model, and how much of the model's work can be understood. To work around these restrictions, developers build a harness of separate components (like voice-activity detection) to simulate responsiveness. But these components are less intelligent than the model itself and preclude capabilities like proactive visual reactions, speaking while listening, or responding to unspoken cues.

How do interaction models differ from traditional AI systems?

Traditional AI systems are built on a request-response paradigm: you send a query, the model processes it, and replies. Interaction models, introduced by Thinking Machines Lab, treat interactivity as a native property of the model. Instead of waiting for complete input, the model continuously ingests audio, video, and text streams in real time, generating outputs without a fixed turn boundary. This allows the AI to speak while listening, react to mid-sentence pauses, and incorporate visual feedback as it happens. The system never freezes perception—it's always on. This shift from discrete turns to continuous streams unlocks a deeper level of collaboration, where the model understands not just your words but the timing, tone, and visual context that inform them.

Why is the harness approach insufficient for real-time collaboration?

To simulate real-time behavior, current AI systems rely on a harness—a collection of separate components like voice-activity detection (VAD), speech-to-text, and interruption handling stitched together. This harness is made of components that are meaningfully less intelligent than the language model itself. For example, VAD predicts when a user has finished speaking, but it cannot understand semantics or context. This leads to brittle interactions: the system may cut you off, miss a subtle cue, or fail to react to something visual. Most importantly, the harness precludes capabilities that require true awareness, such as proactively reacting to visual events, speaking while the user types, or responding to non-verbal signals. The bitter lesson in machine learning suggests that hand-crafted systems will eventually be outpaced by scaling general capabilities—and for interactivity, that means embedding it within the model itself.

How does the multi-stream architecture of interaction models work?

The system uses two parallel components: an interaction model and a background model. The interaction model is always active, continuously ingesting audio, video, and text streams and producing real-time responses. It manages the conversational flow, providing immediate feedback and handling short-term dialogue. When a task requires sustained reasoning—like tool use, web search, or long-horizon planning—the interaction model delegates to the background model. It sends a rich context package containing the full conversation (not just a single query) to the background model. As the background model processes this deeper task, results stream back incrementally. The interaction model then interleaves these updates into the conversation at a moment that matches the user's current focus, creating a seamless blend of quick responses and thoughtful reasoning.

What role does the background model play in this architecture?

The background model handles the heavy lifting of deep reasoning that would slow down a real-time interaction. While the interaction model manages the continuous stream of perception and response, the background model can run computationally expensive operations—such as multi-step tool use, complex search, code generation, or planning—without blocking the conversation. It operates asynchronously, receiving a full context package from the interaction model and streaming back results as they become available. This division of labor allows the system to maintain real-time responsiveness while still performing sophisticated tasks. The interaction model seamlessly weaves the background model's outputs into the dialogue at an appropriate moment, so the user never feels a disconnect between quick chat and deep computation.

How does the interaction model handle continuous input without dropping context?

The interaction model is designed to be always on, maintaining a constant real-time exchange. It uses a micro-turn design that processes input and output in small, overlapping chunks rather than discrete full turns. This means the model can begin responding while still absorbing incoming data—for example, it might start a sentence while you're still speaking, or adjust its response based on a camera image that arrives mid-utterance. The model's perception never freezes; it continuously integrates audio frames, video frames, and text tokens. To avoid losing context, the interaction model builds a dynamic memory of the ongoing session, allowing it to refer back to earlier parts of the conversation or visual cues. This architecture enables capabilities like proactive visual reactions and speaking while listening, which are impossible in turn-based systems.

What is the 'bitter lesson' that Thinking Machines Lab applies to interaction design?

The bitter lesson, a concept in machine learning articulated by Rich Sutton, states that hand-crafted engineering solutions are eventually surpassed by approaches that leverage scale and computation. Thinking Machines Lab applies this lesson to interaction design: instead of building a hand-crafted harness of separate components to simulate real-time behavior, they argue that interactivity must be a native part of the model. As the model scales, it becomes not only smarter but also a better collaborator. By embedding interaction capabilities directly into the model's architecture, the system can improve continuously through scaling—learning to interpret subtle cues, manage overlapping streams, and react appropriately without brittle rules. This approach moves beyond the limitations of turn-based AI and creates a path toward AI that truly understands and responds to human behavior in real time.