Nvidia AI 3DGS: Lyra 2.0: Explorable Generative 3D Worlds

3D Gaussian Splatting (3DGS) is a cutting-edge 3D reconstruction and rendering technique that converts 2D images or video into highly detailed, photorealistic 3D scenes. Unlike traditional mesh-based methods, 3DGS uses millions of tiny 3D Gaussians (spheres/points) optimized via machine learning to represent scenes, enabling real-time, high-fidelity rendering for VR/AR and 3D modeling.

https://en.wikipedia.org/wiki/Gaussian_splatting

Lyra 2.0: Explorable Generative 3D Worlds

Method overview. (Left) Given an input image, Lyra 2.0 iteratively generates video segments guided by a user‑defined camera trajectory from an interactive 3D explorer and an optional text prompt, lifting each segment into 3D point clouds fed back for continued navigation. Generated video frames are finally reconstructed and exported as 3D Gaussians or meshes. (Right) At each step, history frames with maximal visibility of the target views are retrieved from the spatial memory. Their canonical coordinates are warped to establish dense 3D correspondences and injected into DiT via attention, together with compressed temporal history.