Runway unveils Continuum — a project where a video model draws the interface

Runway closed its one-day AI Summit in San Francisco with a showing of Continuum — a new project from its research division Runway Labs. Co-founder and co-CEO Cristóbal Valenzuela devoted the closing talk to how generative video has been changing, and presented the project that Runway Labs calls a real-time video operating system. Runway promises to publish full recordings of the summit's talks later.

The company has not yet published technical details about Continuum: all that is known from the announcement is that it is Runway Labs' latest project and that it was shown in the closing talk. The summit itself took place at The Masonic on September 30 and was built around world models, real-time video generation, robotics, physical AI and regulation — with participation from representatives of NVIDIA, Google DeepMind, Wayve, Physical Intelligence, Adobe, Canva, Salesforce, Roblox, CoreWeave and the office of the Governor of California.

An interface without code: Solaris

Separately, Runway described Solaris — the first model in a family the company calls Interface World Models. This is an interactive real-time model that generates the interface itself frame by frame: each frame is synthesized in the course of interaction, so the image responds continuously to the user's actions. There is no intermediate representation in this scheme — no code of the kind design is usually translated into — and the entire frame becomes the interface.

Solaris is built on the Gen-4.5 video model, adapted for understanding interaction and responding in real time; the company describes it as a continuation of the GWM-1 line, its general world model. The model treats user input as a condition for the next frame — the same way as text or an image — and observes clicks, drags and other actions during generation. A language model works in parallel: it interprets requests, decides whether to change the current scene or move to a new one, and forms the prompts by which Solaris renders each state. Real time is assembled from three steps: autoregressive frame generation, distillation of multi-step denoising down to a few steps, and fine-tuning the fast model on its own outputs so that quality holds up across long sessions.

Runway lists three engineering constraints the project ran into. Interactivity is lost at roughly half a second of latency, while diffusion video models deliver a clip in seconds or minutes. Coherence has to be maintained throughout the session, and it is precisely text, layout and object identity that generative video has historically held poorly, while small errors accumulate. Generating every frame is more expensive than serving a page assembled once, although the same optimization that delivered real time has, according to the company, made the model orders of magnitude cheaper than ordinary video diffusion. The stated targets are real time, coherence over the session and image quality at 720p.

To justify the approach, Runway cites a measurement of the "cost of translation": multimodal language models, including Claude Fable 5, were asked to recreate a site's interface from a single screenshot across a set of 30 interfaces — from simple pages to ones dense with images and with natural photographs. Similarity was scored by SSIM and by DINOv3 features, comparing each region of the original with the most similar region of the reconstruction. All models lose information, reconstruction quality falls as visual complexity grows, and natural images suffer the most.

Among the scenarios the company describes for Solaris are a virtual showroom where the scene is the interface, assembling a salad by dragging ingredients onto a plate, and an interactive demonstration of burning with physically plausible reactions. A second purpose is an environment for training agents: by Runway's argument, text models cope poorly with computer tasks such as booking a hotel because they latch onto a specific layout, whereas Solaris makes it possible to train on interfaces that are constantly changing.

Why video models are moving into interfaces

Continuum reads as a continuation of the line Runway has been building lately: Gen-4.5, world models, real-time generation. The summit's framing is telling too — world models, robotics and physical AI rather than clip editing; that says as much about a shift in positioning as the new prototype does.

There is probably a simple calculation behind it. The market for clip generation has become crowded and competitive, while a video model capable of holding an interactive scene in real time opens a way into territories where Runway has fewer rivals: simulators and training environments for agents and robots, visualization, interfaces. If the trend continues, real-time generation will begin to compete with the familiar ways of drawing a frame, and the boundary between rendering and generation will become a practical question for graphics specialists.

For artists, modelers and developers, the immediate consequences look more modest than the demonstration: the company has not named access terms for Continuum, so for now there is no building it into a pipeline. The question of transparency also remains: code can be read and versions compared, generated pixels cannot. And the set of limitations Runway itself lists — speed, coherence, cost, the stability of text and layout — describes fairly precisely the list of problems that will have to be solved before generative video can claim the role of a working tool.

Will a video model that draws the interface in real time become a working tool — or remain a conference demo?