v1v2 (latest)

Solvable Model for Inheriting the Regularization through Knowledge Distillation

Mathematical and Scientific Machine Learning (MSML), 2020

1 December 2020

Luca Saglietti

Lenka Zdeborová

ArXiv (abs)PDF HTML

Papers citing "Solvable Model for Inheriting the Regularization through Knowledge Distillation"

11 / 11 papers shown

The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model

Kaito Takanami

Takashi Takahashi

Ayaka Sakata

609

27 Jan 2025

CURing Large Models: Compression via CUR Decomposition

Sanghyeon Park

Soo-Mook Moon

424

08 Jan 2025

Features are fate: a theory of transfer learning in high-dimensional regression

Javan Tahir

Surya Ganguli

Grant M. Rotskoff

536

10 Oct 2024

Asymptotic Dynamics of Alternating Minimization for Bilinear Regression

Koki Okajima

Takashi Takahashi

347

07 Feb 2024

Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics

Yehonatan Avidan

Qianyi Li

H. Sompolinsky

396

08 Sep 2023

The Quest of Finding the Antidote to Sparse Double Descent

Victor Quétu

Marta Milovanović

382

31 Aug 2023

DSD$^2$: Can We Dodge Sparse Double Descent and Compress the Neural
Network Worry-Free?

DSD

^2

: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?AAAI Conference on Artificial Intelligence (AAAI), 2023

Victor Quétu

Enzo Tartaglione

416

02 Mar 2023

Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 TileMicro (MICRO), 2022

Zhe Wang

252

26 Sep 2022

An Analytical Theory of Curriculum Learning in Teacher-Student Networks

Luca Saglietti

Stefano Sarao Mannelli

Andrew M. Saxe

312

15 Jun 2021

Phase Transitions in Transfer Learning for High-Dimensional PerceptronsEntropy (Entropy), 2021

Oussama Dhifallah

Yue M. Lu

195

06 Jan 2021

A Concentration of Measure Framework to study convex problems and other implicit formulation problems in machine learning

Cosme Louart

333

19 Oct 2020