ApproxNet: Content and Contention-Aware Video Analytics System for Embedded Clients

Videos take lot of time to transport over the network, hence running analytics on the live video at the embedded or mobile devices has become an important system driver. Considering such devices, e.g., surveillance cameras or AR/VR gadgets, are resource constrained, althrough there has been significant work in creating lightweight deep neural networks (DNNs) for such clients, none of these can adapt to changing runtime conditions, e.g. changes in resource availability on the device, the content characteristics, or requirements from the user. In this paper we introduce ApproxNet, a video analytics system for embedded or mobile clients (which we collectively refer to as ``sensor devices''). It enables novel dynamic approximation techniques to achieve desired inference latency and accuracy trade-off under changing runtime conditions. It achieves this by enabling two approximation knobs within a single DNN model, rather than creating and maintaining an ensemble of models (such as MCDNN [MobiSys-16]). We show that ApproxNet can adapt seamlessly at runtime to these changes, provide low and stable latency for the image and video frame classification problems, and show the improvement in accuracy and latency over ResNet [CVPR-16], MCDNN [MobiSys-16], and MobileNets [Google-17].