-
-
- 
-
-Abstract: *Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.*
@Article{kerbl3Dgaussians,
- author = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
- title = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
- journal = {ACM Transactions on Graphics},
- number = {4},
- volume = {42},
- month = {July},
- year = {2023},
- url = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
+ @article{kheradmand20243d,
+ title={3D Gaussian Splatting as Markov Chain Monte Carlo},
+ author={Kheradmand, Shakiba and Rebain, Daniel and Sharma, Gopal and Sun, Weiwei and Tseng, Jeff and Isack, Hossam and Kar, Abhishek and Tagliasacchi, Andrea and Yi, Kwang Moo},
+ journal={arXiv preprint arXiv:2404.09591},
+ year={2024}
}
+ Novel view reconstructions for (right) our method + and (left) conventional 3D Gaussian Splatting with + random initializations. Our method, even with random initialization, + faithfully reconstructs the scene (e.g.. buildings at the back and + the ground texture) providing much higher quality renderings. +
++ While 3D Gaussian Splatting has recently become popular for neural + rendering, current methods rely on carefully engineered cloning and + splitting strategies for placing Gaussians, which can lead to + poor-quality renderings, and reliance on a good initialization. In + this work, we rethink the set of 3D Gaussians as a random sample + drawn from an underlying probability distribution describing the + physical representation of the scene---in other words, Markov Chain + Monte Carlo (MCMC) samples. Under this view, we show that the 3D + Gaussian updates can be converted as Stochastic Gradient Langevin + Dynamics (SGLD) update by simply introducing noise. We then rewrite + the densification and pruning strategies in 3D Gaussian Splatting as + simply a deterministic state transition of MCMC samples, removing + these heuristics from the framework. To do so, we revise the + `cloning' of Gaussians into a relocalization scheme that + approximately preserves sample probability. To encourage efficient + use of Gaussians, we introduce a regularizer that promotes the + removal of unused Gaussians. On various standard evaluation scenes, + we show that our method provides improved rendering quality, easy + control over the number of Gaussians, and robustness to + initialization. +
+| + '10' sequence from OMMO dataset + | +|||
| 3DGS-Random | +3DGS | +||
| + + | +|||
| Ours-Random | +Ours | +||
| + 'Stump' sequence from the MipNeRF360 dataset (pay attention to the + details between the leaves) + | +|||
| 3DGS-Random | +3DGS | +||
| + + | +|||
| Ours-Random | +Ours | +||