The traditional NeRF and its variations demand considerable computational resources, often surpassing the everyday availability in constrained settings. Moreover, client devices’ limited video memory capability imposes significant constraints on processing and rendering extensive assets concurrently in real-time. The considerable demand for resources poses a vital challenge in rendering expansive scenes in real-time, requiring rapid loading and processing of in depth datasets.

To tackle the challenges encountered within the real-time rendering of in depth scenes, researchers on the University of Science and Technology of China proposed a way called Cityon-Web. Taking inspiration from traditional graphics methods used for handling large-scale scenes, they partition the scene into manageable blocks and incorporate various Levels-of-Detail (LOD) to represent it. 

Radiance field baking techniques are employed to precompute and store rendering primitives into 3D atlas textures organized inside a sparse grid in each block, facilitating real-time rendering. Nevertheless, loading all atlas textures right into a single shader is unfeasible as a result of inherent limitations in shader resources. Consequently, the scene is represented as a hierarchy of segmented blocks, each rendered by a dedicated shader in the course of the rendering process.

Employing a “divide and conquer” strategy, they guarantee that every block has ample representation capability to reconstruct intricate details throughout the scene faithfully. Furthermore, to take care of high fidelity within the rendered output in the course of the training phase, they simulate mixing multiple shaders aligned with the rendering pipeline. 

These representations based on blocks and levels-of-detail (LOD) enable dynamic resource management, simplifying the real-time loading and unloading process based on the viewer’s position and field of view. This adaptable loading approach significantly reduces the bandwidth and memory requirements of rendering extensive scenes, resulting in smoother user experiences, especially on less powerful devices. 

The experiments conducted illustrate that City-on-Web achieves the rendering of photorealistic large-scale scenes at 32 frames per second (FPS) with a resolution of 1080p, utilizing an RTX 3060 GPU. It uses only 18% of the VRAM and 16% of the payload size in comparison with existing mesh-based methods.

The mixture of block partitioning and Levels-of-Detail (LOD) integration has notably decreased the payload on the net platform while enhancing resource management efficiency. This approach guarantees high-fidelity rendering quality by upholding consistency between the training process and the rendering phase.

Try the Paper and Project. All credit for this research goes to the researchers of this project. Also, don’t forget to affix our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, LinkedIn Group, and Email Newsletter, where we share the newest AI research news, cool AI projects, and more.

Should you like our work, you’ll love our newsletter..