Sparse principal component regression for generalized linear models

Principal component regression (PCR) is a widely-used two-stage procedure: we first perform principal component analysis (PCA) and next consider a regression model in which selected principal components are regarded as new explanatory variables. We should remark that PCA is based only on the explanatory variables, so the principal components are not selected using the information on the response variable. In this paper, we propose a one-stage procedure for PCR in the framework of generalized linear models. The basic loss function is based on a combination of the regression loss and PCA loss. The estimate of the regression parameter is obtained as the minimizer of the basic loss function with sparse penalty. The proposed method is called the sparse principal component regression for generalized linear models (SPCR-glm). SPCR-glm enables us to obtain sparse principal component loadings that are related to a response variable, because the two loss functions are simultaneously taken into consideration. A combination of loss functions may cause the identifiability problem on parameters, but it is overcome by virtue of sparse penalty. The sparse penalty plays two roles in this method. The parameter estimation procedure is proposed using various update algorithms with the coordinate descent algorithm. We apply SPCR-glm to two real datasets, Doctor visit data and mouse consomic strain data. SPCR-glm provides easier interpretable PC scores and clearer classification on PC plots than the usual PCA.