Self-Tuning Spectral Clustering for Speaker Diarization

Spectral clustering has proven effective in grouping speech representations for speaker diarization tasks, although post-processing the affinity matrix remains difficult due to the need for careful tuning before constructing the Laplacian. In this study, we present a novel pruning algorithm to create a sparse affinity matrix called spectral clustering on p-neighborhood retained affinity matrix (SC-pNA). Our method improves on node-specific fixed neighbor selection by allowing a variable number of neighbors, eliminating the need for external tuning data as the pruning parameters are derived directly from the affinity matrix. SC-pNA does so by identifying two clusters in every row of the initial affinity matrix, and retains only the top p % similarity scores from the cluster containing larger similarities. Spectral clustering is performed subsequently, with the number of clusters determined as the maximum eigengap. Experimental results on the challenging DIHARD-III dataset highlight the superiority of SC-pNA, which is also computationally more efficient than existing auto-tuning approaches. Our implementations are available atthis https URL.

@article{raghav2025_2410.00023,
  title={ Self-Tuning Spectral Clustering for Speaker Diarization },
  author={ Nikhil Raghav and Avisek Gupta and Md Sahidullah and Swagatam Das },
  journal={arXiv preprint arXiv:2410.00023},
  year={ 2025 }
}