Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows

We study approximation of probability measures supported on n-dimensional manifolds embedded in R^m by injective flows -- neural networks composed of invertible flow and one-layer injective components. When m <= 3n, we show that injective flows between R^n and R^m universally approximate measures supported on images of extendable embeddings, which are a proper subset of standard embeddings. In this regime topological obstructions preclude certain knotted manifolds as admissible targets. When m >= 3n + 1, we use an argument from algebraic topology known as the *clean trick* to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that optimality of an injective flow network can be established "in reverse," resolving a conjecture made in Brehmer et Cranmer 2020. Furthermore, the designed networks can be simple enough that they can be equipped with other properties, such as a novel projection result.