Skip to content

[NV] Add LiDAR sensor support for 3D Gaussian Splatting rasterization - #890

Merged
vince-brisebois merged 20 commits into
mainfrom
nv/07-lidar
Mar 14, 2026
Merged

[NV] Add LiDAR sensor support for 3D Gaussian Splatting rasterization#890
vince-brisebois merged 20 commits into
mainfrom
nv/07-lidar

Conversation

@rodlima78

Copy link
Copy Markdown
Collaborator

Summary

  • Lidar camera model: Introduces a spherical-coordinate lidar sensor with configurable elevation/azimuth beams, supporting both uniform and non-uniform beam distributions, plus rolling-shutter pose refinement.
  • Lidar tiling and intersection: Adds a dedicated tile-based intersection pipeline (IntersectTileLidar.cu) that partitions the spherical field of view into azimuth/elevation tiles and assigns Gaussians to
    tiles based on their projected 2D covariance ellipses.
  • Lidar rasterization: Extends the rendering pipeline (rendering.py, _wrapper.py) to support lidar sensors end-to-end — projection, tiling, alpha-compositing — reusing the existing 3DGS/3DGUT rasterization
    kernels with distance-based output.
  • Python reference implementation: Adds pure-PyTorch lidar equivalents (_torch_impl_lidar.py, _torch_lidars.py) for CPU testing and cross-validation against the CUDA kernels.
  • Bug fixes and robustness: Fixes singular matrix inversion in SH evaluation, corrects hit-distance clipping when global_z_order==false, adds a torch.tensor_hash fallback for PyTorch < 2.9, and removes
    premature culling of small Gaussians.
  • Comprehensive test coverage: Adds extensive tests for lidar projection, tiling, intersection, and rasterization, as well as a new 3DGUT distance-output rasterization test.

Key details

  • Lidar beam layout is defined by per-beam elevation/azimuth angles plus a beam-to-pixel mapping, allowing irregular scan patterns (e.g., Velodyne, Ouster, Hesai).
  • Tiling operates in spherical coordinates — azimuth bins form columns, elevation bins form rows — with special handling for azimuth wraparound across the 0°/360° boundary.
  • Rolling-shutter support iteratively refines Gaussian positions using per-column ego-motion transforms, controlled by rs_num_iters.
  • global_z_order=false switches rasterization from depth-based to hit-distance-based ordering, which is the natural sorting for lidar returns.
  • The TensorView::data() helper was added to simplify raw-pointer access in CUDA kernels.
  • NUM_CHANNELS must include 1 in the build to support single-channel (intensity-only) lidar rendering.
@rodlima78
rodlima78 force-pushed the nv/06-extra-signals branch from d4abcdd to 2cbc551 Compare March 14, 2026 02:37
@rodlima78
rodlima78 force-pushed the nv/06-extra-signals branch 2 times, most recently from 5fb18f9 to 2f16f71 Compare March 14, 2026 03:19
@vince-brisebois
vince-brisebois force-pushed the nv/06-extra-signals branch 2 times, most recently from 6d1148a to 026c361 Compare March 14, 2026 03:35
Base automatically changed from nv/06-extra-signals to main March 14, 2026 03:42
@vince-brisebois
vince-brisebois merged commit 6ac6e36 into main Mar 14, 2026
@vince-brisebois
vince-brisebois deleted the nv/07-lidar branch March 14, 2026 03:44
liruilong940607 added a commit that referenced this pull request Apr 30, 2026
  Fix empty API pages on docs.gsplat.studio/main

The API documentation pages (utils, rasterization, compression,
strategy) have been empty since the LiDAR PR (#890)
was merged. The pages render with section headers but no function or
class documentation.

Root cause: The docs CI pins torch==2.0.0, which is incompatible with
modern setuptools>=82 (removed pkg_resources).
When Sphinx tries to import gsplat, the import chain reaches _backend.py
which does import torch.utils.cpp_extension
at module level — this fails, and Sphinx silently skips all
autofunction/autoclass directives, deploying empty pages.

Fix: Unpin torch so the CI installs a version compatible with the
current setuptools.

  Also fixes three pre-existing Sphinx warnings in the docs:
  - Mismatched backticks in fully_fused_projection docstring
  - Unindented table body under a .. table:: directive in eval.rst
  - Duplicate here link targets in eval.rst (use anonymous refs)

Tested locally — all API pages now render correctly (utils: 16,
rasterization: 2, compression: 3, strategy: 9
  documented symbols).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants