This paper tackles the problem of novel view audio-visual synthesis along an arbitrary trajectory in an indoor scene, given the audio-video recordings from other known trajectories of the scene. Existing methods often overlook the effect of room geometry, particularly wall occlusions on sound propagation, making them less accurate in multi-room environments. In this work, we propose a new approach called Scene Occlusion-aware Acoustic Field (SOAF) for accurate sound generation. Our approach derives a global prior for the sound field using distance-aware parametric sound-propagation modeling and then transforms it based on the scene structure learned from the input video. We extract features from the local acoustic field centered at the receiver using a Fibonacci Sphere to generate binaural audio for novel views with a direction-aware attention mechanism. Extensive experiments on the real dataset RWAVS and the synthetic dataset SoundSpaces demonstrate that our method outperforms previous state-of-the-art techniques in audio generation.
View on arXiv@article{gao2025_2407.02264,
title={ SOAF: Scene Occlusion-aware Neural Acoustic Field },
author={ Huiyu Gao and Jiahao Ma and David Ahmedt-Aristizabal and Chuong Nguyen and Miaomiao Liu },
journal={arXiv preprint arXiv:2407.02264},
year={ 2025 }
}