Surrey and NVIDIA teach AI video to follow the camera
A new training method from the University of Surrey and NVIDIA is closing one of the harder problems in real-time AI video: getting a generated scene to actually follow the camera command it was given.
A new training method from the University of Surrey and NVIDIA is closing one of the harder problems in real-time AI video: getting a generated scene to actually follow the camera command it was given.
Frame-by-frame video generators are usually trained by a slower "teacher" model that grades each frame after the fact, against a reconstruction of what should have happened. Small drift early on compounds, and the generated world stops tracking the input. The new method, called Context-Matched Distillation, instead has the teacher grade each frame against the model's own trial run, with controlled noise added so early wobble doesn't throw off later training. Built on NVIDIA's Cosmos-Predict2.5 model, it recorded the lowest camera-position error on both standard and complex test sets, and was preferred over six rival systems in up to 88% of blind comparisons.
The applications named in the paper are the tell: video games built on AI-generated worlds, virtual production sets where a director walks a camera through a generated environment, and simulated environments for training robots. All three depend on the same thing — a generated world has to obey the hand steering it, not just look convincing to someone watching it.
That is the real shift underway in generative video. Fidelity was the first problem to solve. Controllability, frame by frame, in real time, is the one that decides whether any of this is usable for production rather than a demo.
