Qualitatively characterizing neural network optimization problems

19 December 2014

Papers citing "Qualitatively characterizing neural network optimization problems"

50 / 111 papers shown

Title
Low-Loss Space in Neural Networks is Continuous and Fully Connected Yongding Tian Zaid Al-Ars Maksim Kitsak P. Hofstee 3DPC 31 0 0 05 May 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization Hao Mark Chen S. Hu Wayne Luk Timothy M. Hospedales Hongxiang Fan MoMe 72 0 0 16 Mar 2025
High-dimensional manifold of solutions in neural networks: insights from statistical physics Enrico M. Malatesta 56 4 0 20 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen Xinming Zhang Ninghui Li 122 1 0 28 Jan 2025
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 32 9 0 07 Jul 2024
Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function Kuo Gai Shihua Zhang FAtt 43 0 0 02 Jul 2024
A Multi-Level Framework for Accelerating Training Transformer Models Longwei Zou Han Zhang Yangdong Deng AI4CE 40 1 0 07 Apr 2024
Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications Lucas Böttcher Gregory R. Wheeler 32 0 0 05 Apr 2024
Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective Sunwoong Yang Hojin Kim Y. Hong K. Yee R. Maulik Namwoo Kang PINN AI4CE 31 17 0 05 Jan 2024
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 26 7 0 19 Dec 2023
Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion Yi Sun Xin Xu Jian Li Guanglei Xie Yifei Shi Qiang Fang CLL MoMe 34 1 0 12 Dec 2023
In Search of a Data Transformation That Accelerates Neural Field Training Junwon Seo Sangyoon Lee Kwang In Kim Jaeho Lee 49 3 0 28 Nov 2023
On-the-Fly Guidance Training for Medical Image Registration Yuelin Xin Yicheng Chen Shengxiang Ji Kun Han Xiaohui Xie OOD 35 1 0 29 Aug 2023
Addressing caveats of neural persistence with deep graph persistence Leander Girrbach Anders Christensen Ole Winther Zeynep Akata A. Sophia Koepke GNN 28 1 0 20 Jul 2023
Multiplicative update rules for accelerating deep learning training and increasing robustness Manos Kirtas Nikolaos Passalis Anastasios Tefas AAML OOD 36 2 0 14 Jul 2023
SING: A Plug-and-Play DNN Learning Technique Adrien Courtois Damien Scieur Jean-Michel Morel Pablo Arias Thomas Eboli 36 0 0 25 May 2023
Evolutionary Augmentation Policy Optimization for Self-supervised Learning Noah Barrett Zahra Sadeghi Stan Matwin 30 3 0 02 Mar 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari George Konidaris 79 7 0 29 Dec 2022
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off Shaoyi Huang Bowen Lei Dongkuan Xu Hongwu Peng Yue Sun Mimi Xie Caiwen Ding 29 19 0 30 Nov 2022
A survey of deep learning optimizers -- first and second order methods Rohan Kashyap ODL 37 6 0 28 Nov 2022
PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization Sanae Lotfi Marc Finzi Sanyam Kapoor Andres Potapczynski Micah Goldblum A. Wilson BDL MLT AI4CE 29 51 0 24 Nov 2022
Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models Mark Rofin Nikita Balagansky Daniil Gavrilov MoMe KELM 38 5 0 22 Nov 2022
Regression as Classification: Influence of Task Formulation on Neural Network Features Lawrence Stewart Francis R. Bach Quentin Berthet Jean-Philippe Vert 32 24 0 10 Nov 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 37 8 0 18 Oct 2022
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 35 24 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 316 0 11 Sep 2022
Exploring the trade off between human driving imitation and safety for traffic simulation Yann Koeberle S. Sabatini D. Tsishkou C. Sabourin 33 4 0 09 Aug 2022
Zero-shot Cross-lingual Transfer is Under-specified Optimization Shijie Wu Benjamin Van Durme Mark Dredze 30 6 0 12 Jul 2022
How many labelers do you have? A closer look at gold-standard labels Chen Cheng Hilal Asi John C. Duchi 13 6 0 24 Jun 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 79 27 0 17 Jun 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 244 45 0 24 May 2022
Overparameterization Improves StyleGAN Inversion Yohan Poirier-Ginter Alexandre Lessard Ryan Smith Jean-François Lalonde 46 4 0 12 May 2022
Federated Learning in Multi-Center Critical Care Research: A Systematic Case Study using the eICU Database Arash Mehrjou Ashkan Soleymani Annika Buchholz J. Hetzel Patrick Schwab Stefan Bauer OOD FedML 9 4 0 20 Apr 2022
FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks Aleksandar Doknic Torsten Moller 36 2 0 09 Apr 2022
Fusing finetuned models for better pretraining Leshem Choshen Elad Venezian Noam Slonim Yoav Katz FedML AI4CE MoMe 54 87 0 06 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 35 12 0 28 Mar 2022
Self-Ensemble Adversarial Training for Improved Robustness Hongjun Wang Yisen Wang OOD AAML 13 48 0 18 Mar 2022
Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attacks Siddhartha Datta N. Shadbolt AAML 32 6 0 07 Mar 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 24 58 0 01 Feb 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
TransMorph: Transformer for unsupervised medical image registration Junyu Chen Eric C. Frey Yufan He W. Paul Segars Ye Li Yong Du ViT MedIm 39 303 0 19 Nov 2021
Mode connectivity in the loss landscape of parameterized quantum circuits Kathleen E. Hamilton E. Lynn R. Pooser 27 3 0 09 Nov 2021
Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction Konstantin Schurholt Dimche Kostadinov Damian Borth SSL 35 14 0 28 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 64 691 0 04 Sep 2021
AdvRush: Searching for Adversarially Robust Neural Architectures J. Mok Byunggook Na Hyeokjun Choe Sungroh Yoon OOD AAML 22 44 0 03 Aug 2021
What can linear interpolation of neural network loss landscapes tell us? Tiffany J. Vlaar Jonathan Frankle MoMe 30 27 0 30 Jun 2021
Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation Agnieszka Słowik Léon Bottou FaML 45 19 0 17 Jun 2021
Extracting Global Dynamics of Loss Landscape in Deep Learning Models Mohammed Eslami Hamed Eramian Marcio Gameiro W. Kalies Konstantin Mischaikow 23 1 0 14 Jun 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 164 28 0 22 Apr 2021