Born Again Neural Networks

12 May 2018

Papers citing "Born Again Neural Networks"

50 / 169 papers shown

Title
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 54 0 0 05 May 2025
CR-CTC: Consistency regularization on CTC for improved speech recognition Zengwei Yao Wei Kang Xiaoyu Yang Fangjun Kuang Liyong Guo Han Zhu Zengrui Jin Zhaoqing Li Long Lin Daniel Povey 53 0 0 17 Feb 2025
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model Kaito Takanami Takashi Takahashi Ayaka Sakata 35 0 0 27 Jan 2025
Knowledge Migration Framework for Smart Contract Vulnerability Detection Luqi Wang Wenbao Jiang 81 0 0 15 Dec 2024
GazeGen: Gaze-Driven User Interaction for Visual Content Generation He-Yen Hsieh Ziyun Li Sai Qian Zhang W. Ting Kao-Den Chang B. D. Salvo Chiao Liu H. T. Kung VGen 35 0 0 07 Nov 2024
Towards Model-Agnostic Dataset Condensation by Heterogeneous Models Jun-Yeong Moon Jung Uk Kim Gyeong-Moon Park DD 33 1 0 22 Sep 2024
Your Weak LLM is Secretly a Strong Teacher for Alignment Leitian Tao Yixuan Li 86 5 0 13 Sep 2024
Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation Feng Zhou Yanjie Zhou Longjie Wang Yun Peng David E. Carlson Liyun Tu 40 1 0 07 Aug 2024
Proximal Policy Distillation Giacomo Spigler OffRL 26 1 0 21 Jul 2024
Enhancing Weakly-Supervised Histopathology Image Segmentation with Knowledge Distillation on MIL-Based Pseudo-Labels Yinsheng He Xingyu Li Roger J. Zemp VLM 35 0 0 14 Jul 2024
Retraining with Predicted Hard Labels Provably Increases Model Accuracy Rudrajit Das Inderjit S Dhillon Alessandro Epasto Adel Javanmard Jieming Mao Vahab Mirrokni Sujay Sanghavi Peilin Zhong 48 1 0 17 Jun 2024
Robust Preference Optimization through Reward Model Distillation Adam Fisch Jacob Eisenstein Vicky Zayats Alekh Agarwal Ahmad Beirami Chirag Nagpal Peter Shaw Jonathan Berant 73 21 0 29 May 2024
Tailoring Vaccine Messaging with Common-Ground Opinions Rickard Stureborg Sanxing Chen Ruoyu Xie Aayushi Patel Christopher Li Chloe Qinyu Zhu Tingnan Hu Jun Yang Bhuwan Dhingra 35 0 0 17 May 2024
Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF Banghua Zhu Michael I. Jordan Jiantao Jiao 28 24 0 29 Jan 2024
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 32 0 0 12 Dec 2023
Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples Ziye Fang Xin Jiang Hao Tang Zechao Li 30 12 0 10 Nov 2023
Towards Comparable Knowledge Distillation in Semantic Image Segmentation Onno Niemann Christopher Vox Thorben Werner VLM 19 1 0 07 Sep 2023
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
Review helps learn better: Temporal Supervised Knowledge Distillation Dongwei Wang Zhi-Long Han Yanmei Wang Xi’ai Chen Baicheng Liu Yandong Tang 54 1 0 03 Jul 2023
Student-friendly Knowledge Distillation Mengyang Yuan Bo Lang Fengnan Quan 18 17 0 18 May 2023
Measuring and Mitigating Local Instability in Deep Neural Networks Arghya Datta Subhrangshu Nandi Jingcheng Xu Greg Ver Steeg He Xie Anoop Kumar Aram Galstyan 15 3 0 18 May 2023
CORSD: Class-Oriented Relational Self Distillation Muzhou Yu S. Tan Kailu Wu Runpei Dong Linfeng Zhang Kaisheng Ma 24 0 0 28 Apr 2023
LSFSL: Leveraging Shape Information in Few-shot Learning Deepan Padmanabhan Shruthi Gowda Elahe Arani Bahram Zonooz 23 6 0 13 Apr 2023
Self-Distillation for Gaussian Process Regression and Classification Kenneth Borup L. Andersen 11 2 0 05 Apr 2023
Improving Neural Topic Models with Wasserstein Knowledge Distillation Suman Adhya Debarshi Kumar Sanyal BDL 20 1 0 27 Mar 2023
Mixed-Type Wafer Classification For Low Memory Devices Using Knowledge Distillation Nitish Shukla Anurima Dey K. Srivatsan 25 1 0 24 Mar 2023
Decoupled Multimodal Distilling for Emotion Recognition Yong Li Yuan-Zheng Wang Zhen Cui 19 71 0 24 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 24 7 0 02 Mar 2023
Graph-based Knowledge Distillation: A survey and experimental evaluation Jing Liu Tongya Zheng Guanzheng Zhang Qinfen Hao 29 8 0 27 Feb 2023
Detecting software vulnerabilities using Language Models Marwan Omar 24 11 0 23 Feb 2023
Exploring Navigation Maps for Learning-Based Motion Prediction Julian Schmidt Julian Jordan Franz Gritschneder T. Monninger Klaus C. J. Dietmayer 25 3 0 13 Feb 2023
Rethinking Soft Label in Label Distribution Learning Perspective Seungbum Hong Jihun Yoon Bogyu Park Min-Kook Choi 31 0 0 31 Jan 2023
Supervision Complexity and its Role in Knowledge Distillation Hrayr Harutyunyan A. S. Rawat A. Menon Seungyeon Kim Surinder Kumar 22 12 0 28 Jan 2023
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models Sucheng Ren Fangyun Wei Zheng-Wei Zhang Han Hu 35 34 0 03 Jan 2023
Decentralized Learning with Multi-Headed Distillation A. Zhmoginov Mark Sandler Nolan Miller Gus Kristiansen Max Vladymyrov FedML 32 4 0 28 Nov 2022
Responsible Active Learning via Human-in-the-loop Peer Study Yu Cao Jingya Wang Baosheng Yu Dacheng Tao 15 0 0 24 Nov 2022
Accelerating Diffusion Sampling with Classifier-based Feature Distillation Wujie Sun Defang Chen Can Wang Deshi Ye Yan Feng Chun-Yen Chen 35 16 0 22 Nov 2022
AI-KD: Adversarial learning and Implicit regularization for self-Knowledge Distillation Hyungmin Kim Sungho Suh Sunghyun Baek Daehwan Kim Daun Jeong Hansang Cho Junmo Kim 17 5 0 20 Nov 2022
Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection Yu Hong Hang Dai Yong Ding 3DPC 25 70 0 14 Nov 2022
Teacher-Student Architecture for Knowledge Learning: A Survey Chengming Hu Xuan Li Dan Liu Xi Chen Ju Wang Xue Liu 20 35 0 28 Oct 2022
Respecting Transfer Gap in Knowledge Distillation Yulei Niu Long Chen Chan Zhou Hanwang Zhang 21 23 0 23 Oct 2022
Federated Learning with Privacy-Preserving Ensemble Attention Distillation Xuan Gong Liangchen Song Rishi Vedula Abhishek Sharma Meng Zheng ... Arun Innanje Terrence Chen Junsong Yuan David Doermann Ziyan Wu FedML 15 27 0 16 Oct 2022
Linkless Link Prediction via Relational Distillation Zhichun Guo William Shiao Shichang Zhang Yozen Liu Nitesh V. Chawla Neil Shah Tong Zhao 21 41 0 11 Oct 2022
Masked Autoencoders Enable Efficient Knowledge Distillers Yutong Bai Zeyu Wang Junfei Xiao Chen Wei Huiyu Wang Alan Yuille Yuyin Zhou Cihang Xie CLL 24 39 0 25 Aug 2022
FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification Yunqing Zhao Ngai-man Cheung BDL 21 12 0 23 Aug 2022
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao VLM CLL 29 41 0 22 Aug 2022
Effectiveness of Function Matching in Driving Scene Recognition Shingo Yashima 16 1 0 20 Aug 2022
Integrating Object-aware and Interaction-aware Knowledge for Weakly Supervised Scene Graph Generation Xingchen Li Long Chen Wenbo Ma Yi Yang Jun Xiao 13 26 0 03 Aug 2022
Utilizing Excess Resources in Training Neural Networks Amit Henig Raja Giryes 37 0 0 12 Jul 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 24 3 0 02 Jul 2022