No One Representation to Rule Them All: Overlapping Features of Training Methods

20 October 2021

Papers citing "No One Representation to Rule Them All: Overlapping Features of Training Methods"

47 / 47 papers shown

Title
VIBES -- Vision Backbone Efficient Selection Joris Guerin Shray Bansal Amirreza Shaban Paulo Mann Harshvardhan Gazula VLM 21 0 0 11 Oct 2024
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average Louis Fournier Adel Nabli Masih Aminbeidokhti M. Pedersoli Eugene Belilovsky Edouard Oyallon MoMe FedML 33 3 0 27 May 2024
Scaling (Down) CLIP: A Comprehensive Analysis of Data, Architecture, and Training Strategies Zichao Li Cihang Xie E. D. Cubuk CLIP 32 8 0 12 Apr 2024
Post-Hoc Reversal: Are We Selecting Models Prematurely? Rishabh Ranjan Saurabh Garg Mrigank Raman Carlos Guestrin Zachary Chase Lipton 27 0 0 11 Apr 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 32 1 0 01 Mar 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 102 92 0 22 Jan 2024
Learning to Compose SuperWeights for Neural Parameter Allocation Search Piotr Teterwak Soren Nelson Nikoli Dryden D. Bashkirova Kate Saenko Bryan A. Plummer 10 1 0 03 Dec 2023
Domain Aligned CLIP for Few-shot Classification Muhammad Waleed Gondal Jochen Gast Inigo Alonso Ruiz Richard Droste Tommaso Macri Suren Kumar Luitpold Staudigl VLM 11 11 0 15 Nov 2023
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model Karsten Roth Lukas Thede Almut Sophia Koepke Oriol Vinyals Olivier J. Hénaff Zeynep Akata AAML 17 11 0 26 Oct 2023
A Holistic Assessment of the Reliability of Machine Learning Systems Anthony Corso David Karamadian Romeo Valentin Mary Cooper Mykel J. Kochenderfer 18 6 0 20 Jul 2023
Tangent Model Composition for Ensembling and Continual Fine-tuning Tianlin Liu Stefano Soatto LRM MoMe CLL 8 15 0 16 Jul 2023
Exploring new ways: Enforcing representational dissimilarity to learn new features and reduce error consistency Tassilo Wald Constantin Ulrich Fabian Isensee David Zimmerer Gregor Koehler Michael Baumgartner Klaus H. Maier-Hein OOD 26 1 0 05 Jul 2023
Fisher-Weighted Merge of Contrastive Learning Models in Sequential Recommendation Jung Hyun Ryu Jaeheyoung Jeon Jewoong Cho Myung-joo Kang MoMe 11 1 0 05 Jul 2023
Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization Yimeng Chen Tianyang Hu Fengwei Zhou Zhenguo Li Zhiming Ma 12 11 0 05 Jun 2023
Accurate Knowledge Distillation with n-best Reranking Hendra Setiawan 21 2 0 20 May 2023
A Survey of Historical Learning: Learning Models with Learning History Xiang Li Ge Wu Lingfeng Yang Wenzhe Wang Renjie Song Jian Yang MU AI4TS 20 2 0 23 Mar 2023
Classification in Histopathology: A unique deep embeddings extractor for multiple classification tasks A. Nivaggioli Nicolas Pozin Rémy Peyret Stéphane Sockeel Marie Sockeel Nicolas Nerrienet Marceau Clavel Clara Simmat C. Miquel MedIm 11 0 0 09 Mar 2023
Your representations are in the network: composable and parallel adaptation for large scale models Yonatan Dukler Alessandro Achille Hao-Yu Yang Varsha Vivek L. Zancato Benjamin Bowman Avinash Ravichandran Charless C. Fowlkes A. Swaminathan Stefano Soatto 16 3 0 07 Mar 2023
To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning Ildus Sadrtdinov Dmitrii Pozdeev Dmitry Vetrov E. Lobacheva 16 4 0 06 Mar 2023
Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries Charlotte Loh Seung-Jun Han Shivchander Sudalairaj Rumen Dangovski Kai Xu F. Wenzel Marin Soljacic Akash Srivastava UQCV 18 1 0 04 Mar 2023
Pathologies of Predictive Diversity in Deep Ensembles Taiga Abe E. Kelly Buchanan Geoff Pleiss John P. Cunningham UQCV 17 13 0 01 Feb 2023
Towards Inference Efficient Deep Ensemble Learning Ziyue Li Kan Ren Yifan Yang Xinyang Jiang Yuqing Yang Dongsheng Li BDL 15 12 0 29 Jan 2023
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization Alexandre Ramé Kartik Ahuja Jianyu Zhang Matthieu Cord Léon Bottou David Lopez-Paz MoMe OODD 24 80 0 20 Dec 2022
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 17 13 0 14 Dec 2022
Accelerating Dataset Distillation via Model Augmentation Lei Zhang Jie M. Zhang Bowen Lei Subhabrata Mukherjee Xiang Pan Bo-Lu Zhao Caiwen Ding Y. Li Dongkuan Xu DD 10 62 0 12 Dec 2022
Weighted Ensemble Self-Supervised Learning Yangjun Ruan Saurabh Singh Warren Morningstar Alexander A. Alemi Sergey Ioffe Ian S. Fischer Joshua V. Dillon FedML 16 15 0 18 Nov 2022
Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation Cody Blakeney Jessica Zosa Forde Jonathan Frankle Ziliang Zong Matthew L. Leavitt VLM 17 4 0 01 Nov 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 16 24 0 19 Oct 2022
Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks Jaehoon Oh Jongwoo Ko Se-Young Yun 36 8 0 18 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 19 47 0 13 Oct 2022
Revisiting adapters with adversarial training Sylvestre-Alvise Rebuffi Francesco Croce Sven Gowal AAML 18 16 0 10 Oct 2022
Meta-Ensemble Parameter Learning Zhengcong Fei Shuman Tian Junshi Huang Xiaoming Wei Xiaolin K. Wei OOD 28 2 0 05 Oct 2022
Downstream Datasets Make Surprisingly Good Pretraining Corpora Kundan Krishna Saurabh Garg Jeffrey P. Bigham Zachary Chase Lipton 33 30 0 28 Sep 2022
Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP Thao Nguyen Gabriel Ilharco Mitchell Wortsman Sewoong Oh Ludwig Schmidt CLIP VLM 27 97 0 10 Aug 2022
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models Margaret Li Suchin Gururangan Tim Dettmers M. Lewis Tim Althoff Noah A. Smith Luke Zettlemoyer MoMe 26 142 0 05 Aug 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 188 128 0 19 May 2022
ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Training Yue Zhao Yantao Shen Yuanjun Xiong Shuo Yang Wei Xia Z. Tu Bernt Shiele Stefano Soatto BDL 25 6 0 12 May 2022
When does dough become a bagel? Analyzing the remaining mistakes on ImageNet Vijay Vasudevan Benjamin Caine Raphael Gontijo-Lopes Sara Fridovich-Keil Rebecca Roelofs VLM UQCV 25 57 0 09 May 2022
Language Models in the Loop: Incorporating Prompting into Weak Supervision Ryan Smith Jason Alan Fries Braden Hancock Stephen H. Bach 35 52 0 04 May 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 30 906 1 10 Mar 2022
Deconstructing Distributions: A Pointwise Framework of Learning Gal Kaplun Nikhil Ghosh Saurabh Garg Boaz Barak Preetum Nakkiran OOD 25 19 0 20 Feb 2022
Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty Jaehoon Oh Sungnyun Kim Namgyu Ho Jin-Hwa Kim Hwanjun Song Se-Young Yun 14 34 0 01 Feb 2022
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning Utku Evci Vincent Dumoulin Hugo Larochelle Michael C. Mozer 15 83 0 10 Jan 2022
Sparse MoEs meet Efficient Ensembles J. Allingham F. Wenzel Zelda E. Mariet Basil Mustafa J. Puigcerver ... Balaji Lakshminarayanan Jasper Snoek Dustin Tran Carlos Riquelme Ruiz Rodolphe Jenatton MoE 31 21 0 07 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 19 679 0 04 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 245 648 0 23 Mar 2020