Learning Manifold Patch-Based Representations of Man-Made Shapes

Choosing the right shape representation for geometry is crucial for making 3D models compatible with existing applications. Focusing on piecewise-smooth man-made shapes, we propose a new representation that is usable in conventional CAD modeling pipelines and can also be learned by deep neural networks. We demonstrate the benefits of our representation by applying it to the task of sketch-based modeling. Given a raster image, our system infers a set of parametric surfaces that realize the input in 3D. To capture the piecewise smooth geometry of man-made shapes, we learn a special shape representation: a deformable parametric template composed of Coons patches. Naively training such a system, however, would suffer from non-manifold artifacts of the parametric shapes as well as from a lack of data. To address this, we introduce loss functions that bias the network to output non-self-intersecting shapes and implement them as part of a fully self-supervised system, automatically generating both shape templates and synthetic training data. To test the efficacy of our system, we develop a testbed for sketch-based modeling and show results on a gallery of synthetic and real artist sketches. As additional applications, we also demonstrate shape interpolation and provide comparison to related work.