Kinetic Energy Plus Penalty Functions for Sparse Estimation

Motivated by iteratively reweighted $L_q$ methods, we propose and study a family of sparsity-inducing penalty functions. Since the penalty functions are related to the kinetic energy in special relativity, we call them \emph{kinetic energy plus} (KEP) functions. We construct the KEP function by using the concave conjugate of a $\chi^2$-distance function and present several novel insights into the KEP function with q=1. In particular, we derive a threshold function based on the KEP function, and prove its mathematical properties and asymptotic properties in sparsity modeling. Moreover, we show that a coordinate descent algorithm is especially appropriate for the KEP function. Additionally, we show that KEP functions can be transformed into proper priors, which can be expressed as a mixture of exponential power distributions with a generalized inverse Gaussian density. The prior is able to yield a strong consistent posterior. The theoretical and empirical analysis validates that the KEP function is effective and efficient in high-dimensional dada modeling.