Relativistic Penalties for Sparse Estimation

Motivated by iteratively reweighted $\ell_q$ methods, we propose and study a class of sparsity-inducing penalty functions. Since the penalty function corresponds to the kinetic energy in special relativity, we call it a relativistic penalty. We construct the penalty by using the concave conjugate of a $\chi^2$-distance function and present several novel insights into the relativistic penalty with $q=1$. In particular, we derive a threshold function based on the relativistic penalty and prove its mathematical properties in sparsity modeling. Moreover, we show that a coordinate descent algorithm is especially appropriate for the relativistic penalty. Finally, we show that relativistic penalties can be transformed into proper priors, which can be expressed as a mixture of exponential power distributions with a generalized inverse Gaussian density.