Adam HaroonPerception Reliability · Trustworthy Data
VIRTUS-FPP
Virtual FPP sensors in NVIDIA Isaac Sim: physics-faithful synthetic scans as controllable ground truth for learned perception.
Abstract
Learned 3D perception is only as trustworthy as the data it is trained and benchmarked on, and for fringe projection profilometry (FPP), real labeled data is expensive, slow, and hard to control. VIRTUS-FPP models complete FPP sensor systems inside NVIDIA Isaac Sim to generate photorealistic synthetic scans with perfect, controllable ground truth.
Crucially, it models physics, not just pixels: the actual projective geometry, camera and projector intrinsics and extrinsics, and calibration procedures are reproduced, so the fringe data is physically faithful rather than merely photorealistic. The framework automates phase-shifted capture (an 18-step scan set in 1.4 seconds), calibration-target generation, and full reconstruction: phase wrapping, unwrapping, and triangulation to 3D.
Why it matters: trustworthy synthetic data is a controllable lab for perception reliability. It is the instrument behind our shortcut-diagnosis work and our comprehensive machine-learning benchmarking of FPP methods (SPIE Photonics West 2026).
