Sparse group lasso and high dimensional multinomial classification

We present a coordinate gradient descent algorithm for solving the sparse group lasso optimization problem with a broad class of convex loss functions. Convergence of the algorithm is established, and we use it to investigate the performance of the multinomial sparse group lasso classifier. On three different real data examples we find that multinomial group lasso clearly outperforms multinomial lasso in terms of achieved classification error rate and in terms of including fewer features for the classification. For the current implementation the time to compute the sparse group lasso solution is of the same order of magnitude as for the multinomial lasso algorithm as implemented in the R-package glmnet, and the implementation scales well with the problem size. One of the examples considered is a 50 class classification problem with 10k features, which amounts to estimating 500k parameters. The implementation is provided as an R package.