Asymmetric predictability in causal discovery: an information theoretic approach

Causal investigations in observational studies pose a great challenge in scientific research where randomized trials or intervention-based studies are not feasible. Leveraging Shannon's seminal work on information theory, we develop a causal discovery framework of "predictive asymmetry" for bivariate $(X, Y)$. Predictive asymmetry is a central concept in information geometric causal inference; it enables assessment of whether $X$ is a stronger predictor of $Y$ or vice-versa. We propose a new metric called the Asymmetric Mutual Information ($AMI$) and establish its key statistical properties. The $AMI$ is not only able to detect complex non-linear association patterns in bivariate data, but also is able to detect and quantify predictive asymmetry. Our proposed methodology relies on scalable non-parametric density estimation using fast Fourier transformation. The resulting estimation method is manyfold faster than the classical bandwidth-based density estimation, while maintaining comparable mean integrated squared error rates. We investigate key asymptotic properties of the $AMI$ methodology; a new data-splitting technique is developed to make statistical inference on predictive asymmetry using the $AMI$. We illustrate the performance of the $AMI$ methodology through simulation studies as well as multiple real data examples.