UniWorld: A Unified World Model for Real-Time Visual World Simulation

Paper presentation

UniWorld in action.

Read the paper

02 / Visual worlds

Different scenes.
The same renderer.

Selected scenes with clear structure, detailed environments, and continuous motion.

Hover to preview. Select a scene to watch the complete clip.

01 / The idea

Geometry in.
A world unfolds.

A common visual interface for worlds that move.

UniWorld turns depth trajectories and a reference image into continuous video. Estimated depth, native depth, and on-demand simulator geometry all enter the same shared renderer.

Visual history and geometric history stay on one timeline, so each new window continues from what has already been generated.

UniWorld Figure 1: Any World, Depth Trajectory, Photorealistic Video — Figure 1 · From world geometry to visual observationsFull figure

4 domains

Driving, human activity,
simulation, and synthetic scenes

8 evaluations

Per sampling window
with the shared H3/VDN generator

120 seconds

Long-horizon continuation
evaluated in stress tests

1 timeline

Generated history and
rolling depth, kept in sync

03 / A shared interface

Shape the motion.
Keep the appearance.

Depth describes layout, occlusion, and geometric change. The reference image provides visual appearance.

Drag the divider to inspect the aligned depth control and generated video at the same instant.

The river reference image used for the depth-guided video — Appearance inputA single reference image
Persistent visual context
throughout the trajectory.

Depth trajectory

Generated RGB

From a simulator, on demand

One scene. Different possibilities.

Warm and cool references share exactly the same depth. A geometric control branch changes the future while retaining the committed first window.

Reference A

Warm appearanceShared geometry

Reference B

Cool appearanceSame depth, new appearance

Control branch

A different futureChanged geometry after frame 357

Shared first-window history / Branch depth changes from frame 357 (14.875 s). The camera revisits the scene at 20 s.

04 / Beyond a single window

Keep going.
Stay connected.

Fixed-size windows extend a visual trajectory while preserving committed latent history. Rolling depth brings the corresponding geometric state forward.

Complete 60- and 120-second sequences. Original playback speed.

60 seconds · continuous

01 / Long-horizon generation

The Mediterranean coast.

Sunlit rooftops and a sweeping coastline, in one continuous trajectory.

120 seconds · continuous

02 / Extended continuation

City lights at dusk.

An unbroken two-minute journey through a city at dusk.

60 seconds · continuous

03 / Another trajectory

Where ocean meets shore.

Follow the shoreline through another minute-long trajectory.

The paper evaluates 17 sixty-second validation/test sequences and 13 two-minute training-split stress tests. Latent history is preserved exactly; visual quality can still drift over long horizons.

05 / How it works

One renderer.
Many worlds.

Driving, human activity, simulation, and synthetic scenes share one packed DiT sequence with rolling depth state.

Read the method

Paper Figure 2: a unified renderer for driving, human activity, simulation, and synthetic worlds with rolling depth state and a packed DiT sequence — Figure 2 · A unified renderer for heterogeneous worldsFull figure

01

A common conditioning interface

Aligned depth defines geometric evolution. Reference RGB and optional text supply global appearance context.

02

Noise-aligned continuation

Generated history enters the next target prefix at the current flow noise level, and is restored at every sampling step.

03

Rolling geometric state

Retain the depth overlap, read new positions incrementally, and append only newly generated video latents.

UniWorld Figure 3: overlapping video windows and rolling depth positions — Figure 3 · Aligned visual and geometric stateFull figure

06 / In the paper

Measured.
Across worlds.

Quality, compute, and geometric state, evaluated separately. The tables below reproduce the reported manuscript values.

VBench quality comparison

100 matched sources · 5 s and 14 s, equally weighted · scores ×100, higher is better

Configuration	Quality score ↑	Background consistency ↑	Motion smoothness ↑	Imaging quality ↑
H3 50-step	79.87	92.94	98.47	62.28
LongVie2	78.54	92.86	97.83	58.64
LongVie AR2	78.62	92.19	97.84	60.24
H3/VDN DMD8 UniWorld generator	80.54	93.10	98.68	63.51

Table 1 in the manuscript. The full table also reports subject consistency, temporal flickering, dynamic degree, and aesthetic quality. These results evaluate the within-window generator.

Warmed DiT forward throughput

8 H100 GPUs per request · 8 evaluations per window · 12 hot requests per canvas

Canvas	30 s total DiT time	Full continuation DiT time	New frames / second ↑	RTF ↓
960 × 544	52.774 s	21.175 s	14.472	1.661
672 × 384	37.594 s	14.356 s	21.361	1.126

Tables S16–S17 in the paper. Full-continuation windows reach the 14 FPS throughput target at both resolutions. Timing includes Union and multi-GPU communication; depth preparation, VAE decoding, and export are measured separately.

A depth working state bounded by window size

Canonical active depth rows · independent of total trajectory duration

19.050 MiB

960 × 544 canvas

9.413 MiB

672 × 384 canvas

Bitwise agreement

Cached and incremental paths
for all three simulator conditions

Output duration	Rolling source positions	Direct-slicing positions
30 seconds	217	241
60 seconds	427	475
120 seconds	852	960

The state bound covers active canonical depth rows. Transient copies, conditioning features, activations, model weights, and source storage are accounted for separately in the appendix.

UniWorld Figure 4: reference, ground truth, and generated frames across four scene categories — Figure 4 · One model across four scene categoriesFull figure

UniWorld Figure 5: PSNR, LPIPS, and fixed-scale NMAE over time — Figure 5 · Continuous generation over 60–120 secondsFull figure

Explore the work

A closer look
at UniWorld.

A Unified World Model for
Real-Time Visual World Simulation

Main paper Supplementary

BibTeX

@misc{zhang2026uniworldunifiedworldmodel,
      title={UniWorld: A Unified World Model for Real-Time Visual World Simulation}, 
      author={Xin Zhang and Yabo Chen and Shuaiwei Wang and Zixuan Duan and Haibin Huang and Feng Xu and Xuelong Li},
      year={2026},
      eprint={2610.32381},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.32381}, 
}

UniWorld