Optimization Methods for Large-Scale Machine Learning

15 June 2016

Papers citing "Optimization Methods for Large-Scale Machine Learning"

50 / 1,407 papers shown

Title
Beyond adaptive gradient: Fast-Controlled Minibatch Algorithm for large-scale optimization Corrado Coppola Lorenzo Papa Irene Amerini L. Palagi ODL 84 0 0 24 Nov 2024
A Potential Game Perspective in Federated Learning Kang Liu Ziqi Wang Enrique Zuazua FedML 65 0 0 18 Nov 2024
Towards Accurate and Efficient Sub-8-Bit Integer Training Wenjin Guo Donglai Liu Weiying Xie Yunsong Li Xuefei Ning Zihan Meng Shulin Zeng Jie Lei Zhenman Fang Yu Wang MQ 36 1 0 17 Nov 2024
Convergence Rate Analysis of LION Yiming Dong Huan Li Zhouchen Lin 39 1 0 12 Nov 2024
Effectively Leveraging Momentum Terms in Stochastic Line Search Frameworks for Fast Optimization of Finite-Sum Problems Matteo Lapucci Davide Pucci ODL 32 0 0 11 Nov 2024
Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling Junyi Li Heng Huang 39 1 0 07 Nov 2024
Adaptive Consensus Gradients Aggregation for Scaled Distributed Training Yoni Choukroun Shlomi Azoulay P. Kisilev 39 0 0 06 Nov 2024
Forecasting Outside the Box: Application-Driven Optimal Pointwise Forecasts for Stochastic Optimization Tito Homem-de-Mello Juan Valencia Felipe Lagos Guido Lagos 31 1 0 05 Nov 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian Chengyue Huang Z. Kira 44 1 0 03 Nov 2024
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers Gavia Gray Aman Tiwari Shane Bergsma Joel Hestness 30 1 0 01 Nov 2024
Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors Massimo Bilancia Samuele Magro 38 0 0 29 Oct 2024
Neuro-symbolic Learning Yielding Logical Constraints Zenan Li Yunpeng Huang Zhaoyu Li Yuan Yao Jingwei Xu Taolue Chen Xiaoxing Ma Jian Lu NAI 58 5 0 28 Oct 2024
Fully Stochastic Primal-dual Gradient Algorithm for Non-convex Optimization on Random Graphs Chung-Yiu Yau Haoming Liu Hoi-To Wai 26 0 0 24 Oct 2024
Pipeline Gradient-based Model Training on Analog In-memory Accelerators Zhaoxian Wu Quan-Wu Xiao Tayfun Gokmen H. Tsai Kaoutar El Maghraoui Tianyi Chen 21 1 0 19 Oct 2024
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems Bingcong Li Liang Zhang Niao He 58 3 0 18 Oct 2024
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications Yue Huang Zhaoxian Wu Shiqian Ma Qing Ling 39 1 0 17 Oct 2024
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees Aleksandar Armacki Shuhua Yu Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 57 2 0 17 Oct 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
Stability and Sharper Risk Bounds with Convergence Rate $O(1/n^2)$ Bowei Zhu Shaojie Li Yong Liu 18 0 0 13 Oct 2024
Distribution-Aware Mean Estimation under User-level Local Differential Privacy Corentin Pla Hugo Richard Maxime Vono FedML 39 0 0 12 Oct 2024
Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering Joris Postmus Steven Abreu LLMSV 130 1 0 09 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 63 0 0 08 Oct 2024
Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information Emanuel Buttaci Giuseppe Carlo Calafiore FedML 29 0 0 07 Oct 2024
An Attention-Based Algorithm for Gravity Adaptation Zone Calibration Chen Yu 24 0 0 06 Oct 2024
Temporal Predictive Coding for Gradient Compression in Distributed Learning Adrian Edin Zheng Chen Michel Kieffer Mikael Johansson 32 1 0 03 Oct 2024
Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales Joakim Wallmark Maria Josefsson Marie Wiberg 15 1 0 02 Oct 2024
On the SAGA algorithm with decreasing step Luis Fredes Bernard Bercu Eméric Gbaguidi 29 1 0 02 Oct 2024
Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization Francisco Daunas I. Esnaola S. Perlaza H. Vincent Poor 38 2 0 02 Oct 2024
SetPINNs: Set-based Physics-informed Neural Networks Mayank Nagda Phil Ostheimer Thomas Specht Frank Rhein Fabian Jirasek Stephan Mandt Marius Kloft Sophie Fellenz 3DPC PINN 46 0 0 30 Sep 2024
Unifying back-propagation and forward-forward algorithms through model predictive control Lianhai Ren Qianxiao Li 36 1 0 29 Sep 2024
Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning Zhidong Gao Zhenxiao Zhang Yu Zhang Tongnian Wang Yanmin Gong Yuanxiong Guo 35 0 0 29 Sep 2024
Hierarchical Federated Learning with Multi-Timescale Gradient Correction Wenzhi Fang Dong-Jun Han Evan Chen Jianing Zhang Christopher G. Brinton 34 6 0 27 Sep 2024
Accelerating Multi-Block Constrained Optimization Through Learning to Optimize Ling Liang Cameron Austin Haizhao Yang 29 0 0 25 Sep 2024
Super Level Sets and Exponential Decay: A Synergistic Approach to Stable Neural Network Training J. Chaudhary Dipak Nidhi J. Heikkonen H. Merisaari R. Kanth 26 0 0 25 Sep 2024
Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networks Duong Thuy Anh Nguyen Su Wang Duong Tung Nguyen Angelia Nedich H. Vincent Poor 37 0 0 25 Sep 2024
FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch Sunny Gupta Mohit Jindal Pankhi Kashyap Pranav Jeevan Amit Sethi FedML 34 0 0 23 Sep 2024
DP $^2$ -FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimization Zhenxiao Zhang Yuanxiong Guo Yanmin Gong FedML 38 0 0 20 Sep 2024
JKO for Landau: a variational particle method for homogeneous Landau equation Yan Huang Li Wang 31 0 0 18 Sep 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 45 0 0 13 Sep 2024
Riemannian Federated Learning via Averaging Gradient Stream Zhenwei Huang Wen Huang Pratik Jawanpuria Bamdev Mishra FedML 35 1 0 11 Sep 2024
Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression Zhenxiao Zhang Zhidong Gao Yuanxiong Guo Yanmin Gong 29 0 0 06 Sep 2024
Robust Clustering on High-Dimensional Data with Stochastic Quantization Anton Kozyriev Vladimir Norkin MQ 24 3 0 03 Sep 2024
Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods Chanwoong Park Youngchae Cho Insoon Yang 42 1 0 02 Sep 2024
Hierarchical Learning and Computing over Space-Ground Integrated Networks Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Linling Kuang 31 2 0 26 Aug 2024
Predicting path-dependent processes by deep learning Xudong Zheng Yuecai Han 30 0 0 19 Aug 2024
Point Source Identification Using Singularity Enriched Neural Networks Tianhao Hu Bangti Jin Zhi Zhou 3DPC 32 0 0 17 Aug 2024
Enhancing Sharpness-Aware Minimization by Learning Perturbation Radius Xuehao Wang Weisen Jiang Shuai Fu Yu Zhang AAML 50 0 0 15 Aug 2024
Learning Decisions Offline from Censored Observations with ε-insensitive Operational Costs Minxia Chen Ke Fu Teng Huang Miao Bai OffRL 18 0 0 14 Aug 2024
Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks Md Ferdous Pervej Minseok Choi A. Molisch 33 0 0 12 Aug 2024
Incremental Gauss-Newton Descent for Machine Learning Mikalai Korbit Mario Zanon ODL 17 0 0 10 Aug 2024