A Unified Network Architecture for Semi-Structured Deep Distributional Regression

We propose a unified network architecture for deep distributional regression in which entire distributions can be learned in a general framework of interpretable regression models and deep neural networks. Our approach combines advanced statistical models and deep neural networks within a unifying network, contrasting previous approaches that embed the neural network part as a predictor in an additive regression model. To avoid identifiability issues between different model parts, we construct an orthogonalization cell that projects the deep neural network part into the orthogonal complement of the statistical model predictor, facilitating both estimation and interpretability in high-dimensional settings. We identify appropriate default penalties that can also be understood as prior distribution assumptions in the Bayesian version of our network architecture. We consider several use-cases in experiments with synthetic data and real world applications to illustrate special merits of our approach.