The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

9 March 2018

Papers citing "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"

50 / 594 papers shown

Title
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel S. Pokutta VLM 52 10 0 23 Dec 2023
The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction Pratyusha Sharma Jordan T. Ash Dipendra Kumar Misra LRM 19 78 0 21 Dec 2023
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 31 0 0 19 Dec 2023
Cooperative Learning for Cost-Adaptive Inference Xingli Fang Richard M. Bradford Jung-Eun Kim 32 1 0 13 Dec 2023
MaxQ: Multi-Axis Query for N:M Sparsity Network Jingyang Xiang Siqi Li Junhao Chen Zhuangzhi Chen Tianxin Huang Linpeng Peng Yong-Jin Liu 16 0 0 12 Dec 2023
Deeper Understanding of Black-box Predictions via Generalized Influence Functions Hyeonsu Lyu Jonggyu Jang Sehyun Ryu H. Yang TDI AI4CE 18 5 0 09 Dec 2023
Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint Ael Quélennec Enzo Tartaglione Pavlo Mozharovskyi Van-Tam Nguyen 26 2 0 08 Dec 2023
Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy Bogdan Musat Razvan Andonie 13 0 0 08 Dec 2023
Pursing the Sparse Limitation of Spiking Deep Learning Structures Hao-Ran Cheng Jiahang Cao Erjia Xiao Mengshu Sun Le Yang Jize Zhang Xue Lin B. Kailkhura Kaidi Xu Renjing Xu 16 1 0 18 Nov 2023
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets A. D. Cunha Francesco d’Amore Emanuele Natale MLT 21 1 0 16 Nov 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 24 14 0 15 Nov 2023
One is More: Diverse Perspectives within a Single Network for Efficient DRL Yiqin Tan Ling Pan Longbo Huang OffRL 35 0 0 21 Oct 2023
How a student becomes a teacher: learning and forgetting through Spectral methods Lorenzo Giambagli L. Buffoni Lorenzo Chicchi Duccio Fanelli 19 7 0 19 Oct 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yu-xin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 37 40 0 13 Oct 2023
A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems Yuan-Heng Wang Hoshin V. Gupta AI4CE 32 6 0 12 Oct 2023
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu (Allen) Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 28 78 0 08 Oct 2023
Spectral alignment of stochastic gradient descent for high-dimensional classification tasks Gerard Ben Arous Reza Gheissari Jiaoyang Huang Aukosh Jagannath 27 14 0 04 Oct 2023
Elephant Neural Networks: Born to Be a Continual Learner Qingfeng Lan A. R. Mahmood CLL 42 9 0 02 Oct 2023
Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels Gonzalo Uribarri Federico Barone A. Ansuini Erik Fransén AI4TS 37 6 0 25 Sep 2023
Sparse Autoencoders Find Highly Interpretable Features in Language Models Hoagy Cunningham Aidan Ewart Logan Riggs R. Huben Lee Sharkey MILM 33 333 0 15 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 48 8 0 07 Sep 2023
Adaptive Consensus: A network pruning approach for decentralized optimization S. Shah A. Berahas Raghu Bollapragada 14 2 0 06 Sep 2023
T-SaS: Toward Shift-aware Dynamic Adaptation for Streaming Data Weijieying Ren Tianxiang Zhao Wei Qin Kunpeng Liu TTA AI4TS 19 5 0 05 Sep 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 33 0 0 29 Aug 2023
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition Samuel Horváth Stefanos Laskaridis Shashank Rajput Hongyi Wang BDL 32 4 0 28 Aug 2023
Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models S. N. Hari Matt Thomson 24 11 0 22 Aug 2023
Neural Networks at a Fraction with Pruned Quaternions Sahel Mohammad Iqbal Subhankar Mishra 28 4 0 13 Aug 2023
Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks Jue Chen Huan Yuan Jianchao Tan Bin Chen Chengru Song Di Zhang 25 3 0 09 Aug 2023
Accurate Retraining-free Pruning for Pretrained Encoder-based Language Models Seungcheol Park Ho-Jin Choi U. Kang VLM 32 5 0 07 Aug 2023
Image Synthesis under Limited Data: A Survey and Taxonomy Mengping Yang Zhe Wang 28 8 0 31 Jul 2023
f-Divergence Minimization for Sequence-Level Knowledge Distillation Yuqiao Wen Zichao Li Wenyu Du Lili Mou 30 53 0 27 Jul 2023
A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustness Jovon Craig Joshua Andle Theodore S. Nowak S. Y. Sekeh AAML 39 0 0 07 Jul 2023
Exploring the Lottery Ticket Hypothesis with Explainability Methods: Insights into Sparse Network Performance Shantanu Ghosh Kayhan Batmanghelich 30 0 0 07 Jul 2023
Distilled Pruning: Using Synthetic Data to Win the Lottery Luke McDermott Daniel Cummings SyDa DD 34 1 0 07 Jul 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 28 23 0 03 Jul 2023
Minimizing Energy Consumption of Deep Learning Models by Energy-Aware Training Dario Lazzaro Antonio Emanuele Cinà Maura Pintor Ambra Demontis Battista Biggio Fabio Roli Marcello Pelillo 27 6 0 01 Jul 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 36 2 0 21 Jun 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 62 355 0 20 Jun 2023
Magnificent Minified Models Richard E. Harang Hillary Sanders 14 0 0 16 Jun 2023
Systematic Architectural Design of Scale Transformed Attention Condenser DNNs via Multi-Scale Class Representational Response Similarity Analysis Andrew Hryniowski Alexander Wong 16 0 0 16 Jun 2023
Structural Restricted Boltzmann Machine for image denoising and classification Arkaitz Bidaurrazaga A. Pérez Roberto Santana AI4CE 20 0 0 16 Jun 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 28 2 0 08 Jun 2023
Biologically-Motivated Learning Model for Instructed Visual Processing R. Abel S. Ullman 20 0 0 04 Jun 2023
Incremental Randomized Smoothing Certification Shubham Ugare Tarun Suresh Debangshu Banerjee Gagandeep Singh Sasa Misailovic AAML 30 8 0 31 May 2023
Vision Transformers for Mobile Applications: A Short Survey Nahid Alam Steven Kolawole S. Sethi Nishant Bansali Karina Nguyen ViT 28 3 0 30 May 2023
On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models Bo-wen Li Alexandar J. Thomson Matthew M. Engelhard David Page David Page BDL AI4CE 19 0 0 27 May 2023
SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization Yingchun Wang Jingcai Guo Yi Liu Song Guo Weizhan Zhang Xiangyong Cao Qinghua Zheng AAML OODD 31 11 0 19 May 2023
Worst-Case VCG Redistribution Mechanism Design Based on the Lottery Ticket Hypothesis M. Guo 23 3 0 18 May 2023
Evaluation Metrics for DNNs Compression Abanoub Ghobrial S. Budgett Dieter Balemans Hamid Asgari Philippe Reiter Kerstin Eder 27 1 0 18 May 2023