MODL: Multilearner Online Deep Learning

Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ``deep'' aspect than the ``fast'' aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available:this https URL

@article{valkanas2025_2405.18281,
  title={ MODL: Multilearner Online Deep Learning },
  author={ Antonios Valkanas and Boris N. Oreshkin and Mark Coates },
  journal={arXiv preprint arXiv:2405.18281},
  year={ 2025 }
}