Causal Transfer in Machine Learning

Methods of domain adaptation try to combine knowledge from several related domains (or tasks) to improve performance on a test domain. Inspired by causal methodology, we assume that the covariate shift assumption holds true for a subset of predictor variables: the conditional of the target variable given this subset of predictors is invariant over all tasks. We prove that in an adversarial setting using this subset for prediction is optimal if no examples from the test task are observed. For a specific scenario, in which tasks are drawn from a meta distribution, further optimality results are available. We introduce a practical method which allows for automatic inference of the above subset and provide corresponding code. We present results on synthetic data sets and a gene deletion data set.