Accuracy of Bayesian Latent Variable Estimation with Redundant Dimension

Hierarchical learning models such as mixture models and Bayesian networks are widely employed for unsupervised learning tasks such as clustering analysis. They consist of two variables: observable and hidden variables, which represent the given data and their hidden generation process, respectively. It has been pointed out that the conventional statistical analysis is not applicable to these models because singularities exist in the parameter space. In recent years, a method based on algebraic geometry allows us to analyze accuracy of observable variable prediction on the Bayes estimation. However, analysis for the latent variable has not been studied well though one of the main issues in unsupervised learning is how precisely the latent variable is estimated. A previous study proposed a method for the latent variable when the range of a latent variable has redundancy compared with the model generating data. The present paper extends the method to another redundancy; there are redundant latent variables instead of the variable range. We formulate two types of the error function, and derive the asymptotic forms of both types. Moreover, calculation on the error functions is demonstrated in two-layered Bayesian networks.