Reconstructing controllable Gaussian splats from monocular video is a challenging task due to its inherently insufficient constraints. Widely adopted approaches supervise complex interactions with additional masks and control signal annotations, limiting their real-world applications. In this paper, we propose an annotation guidance-free method, dubbed FreeGaussian, that mathematically derives dynamic Gaussian motion from optical flow and camera motion using novel dynamic Gaussian constraints. By establishing a connection between 2D flows and 3D Gaussian dynamic control, our method enables self-supervised optimization and continuity of dynamic Gaussian motions from flow priors. Furthermore, we introduce a 3D spherical vector controlling scheme, which represents the state with a 3D Gaussian trajectory, thereby eliminating the need for complex 1D control signal calculations and simplifying controllable Gaussian modeling. Quantitative and qualitative evaluations on extensive experiments demonstrate the state-of-the-art visual performance and control capability of our method. Project page:this https URL.
View on arXiv@article{chen2025_2410.22070,
title={ FreeGaussian: Annotation-free Controllable 3D Gaussian Splats with Flow Derivatives },
author={ Qizhi Chen and Delin Qu and Junli Liu and Yiwen Tang and Haoming Song and Dong Wang and Bin Zhao and Xuelong Li },
journal={arXiv preprint arXiv:2410.22070},
year={ 2025 }
}