Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout

14 October 2020

Henrik Kretzschmar

Papers citing "Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout"

50 / 136 papers shown

Title
Uniform Loss vs. Specialized Optimization: A Comparative Analysis in Multi-Task Learning Gabriel S. Gama Valdir Grassi Jr MoMe 42 0 0 15 May 2025
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging Wenju Sun Qingyong Li Yangli-ao Geng Boyang Li MoMe 34 0 0 11 May 2025
COPA: Comparing the Incomparable to Explore the Pareto Front Adrián Javaloy Antonio Vergari Isabel Valera 62 0 0 18 Mar 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization Hao Chen S. Hu Wayne Luk Timothy M. Hospedales Hongxiang Fan MoMe 72 0 0 16 Mar 2025
Continual Optimization with Symmetry Teleportation for Multi-Task Learning Zhipeng Zhou Ziqiao Meng Pengcheng Wu Peilin Zhao Chunyan Miao 47 0 0 06 Mar 2025
Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning Shijie Zhu Hui Zhao Tianshu Wu Pengjie Wang Hongbo Deng Jian Xu Bo Zheng 64 1 0 05 Mar 2025
Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalization Aviv Shamsian E. Shaar Aviv Navon Gal Chechik Ethan Fetaya MU 64 0 0 04 Mar 2025
Commute Your Domains: Trajectory Optimality Criterion for Multi-Domain Learning Alexey Rukhovich Alexander Podolskiy Irina Piontkovskaya 43 0 0 28 Jan 2025
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts Wenju Sun Qingyong Li Wen Wang Yangli-ao Geng Boyang Li 36 2 0 28 Jan 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond Weiyu Chen Xiaoyuan Zhang Baijiong Lin Xi Victoria Lin Han Zhao Qingfu Zhang James T. Kwok 75 2 0 19 Jan 2025
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning Hanwen Zhong Jiaxin Chen Yutong Zhang Di Huang Yunhong Wang MoE 42 0 0 12 Jan 2025
PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization Mingjing Xu Peizhong Ju Jia Liu Haibo Yang 62 2 0 14 Dec 2024
A Flexible Plug-and-Play Module for Generating Variable-Length Liyang He Yuren Zhang Rui Li Zhenya Huang Runze Wu Enhong Chen 75 0 0 12 Dec 2024
Preventing Conflicting Gradients in Neural Marked Temporal Point Processes T. Bosser Souhaib Ben Taieb AI4TS 69 0 0 11 Dec 2024
Diffusion-based Visual Anagram as Multi-task Learning Zhiyuan Xu Yinhe Chen Huan-ang Gao Weiyan Zhao Guiyu Zhang Hao Zhao DiffM 73 5 0 03 Dec 2024
Box for Mask and Mask for Box: weak losses for multi-task partially supervised learning Hoàng-Ân Lê P. Berg Minh Pham 69 0 0 26 Nov 2024
Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning Ziqing Fan Shengchao Hu Yuhang Zhou Li Shen Ya-Qin Zhang Yanfeng Wang Dacheng Tao OffRL 36 0 0 02 Nov 2024
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang V. Cevher Mingyi Hong MU 85 6 0 29 Oct 2024
Enabling Asymmetric Knowledge Transfer in Multi-Task Learning with Self-Auxiliaries Olivier Graffeuille Yun Sing Koh Joerg Wicker Moritz Lehmann 21 0 0 21 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 37 4 0 18 Oct 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 49 1 0 18 Oct 2024
CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models Zi Gong Hang Yu Cong Liao Bingchang Liu Chaoyu Chen Jianguo Li MoMe 21 4 0 09 Oct 2024
Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients Minh Duong Nguyen Khanh Le Khoi Do Nguyen H. Tran Duc Nguyen Chien Trinh Zhaohui Yang FedML 34 2 0 03 Oct 2024
LW2G: Learning Whether to Grow for Prompt-based Continual Learning Qian Feng Dawei Zhou Hanbin Zhao Chao Zhang Hui Qian VLM CLL 35 3 0 27 Sep 2024
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE Xun Zhu Ying Hu Fanbin Mo Miao Li Ji Wu 44 8 0 26 Sep 2024
Learning Representation for Multitask learning through Self Supervised Auxiliary learning Seokwon Shin Hyungrok Do Youngdoo Son SSL 26 1 0 25 Sep 2024
Task Weighting through Gradient Projection for Multitask Learning Christian Bohn Ido Freeman Hasan Tercan Tobias Meisen 22 1 0 03 Sep 2024
Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation Jiahao Xu Zikai Zhang Rui Hu 44 4 0 02 Sep 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Nicholas Andrews 55 1 0 26 Aug 2024
Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning Lukas Kirchdorfer Cathrin Elich Simon Kutsche Heiner Stuckenschmidt Lukas Schott Jan M. Kohler 33 5 0 15 Aug 2024
Gradient Harmonization in Unsupervised Domain Adaptation Fuxiang Huang Suqi Song Lei Zhang 22 7 0 01 Aug 2024
Efficient Pareto Manifold Learning with Low-Rank Structure Weiyu Chen James T. Kwok 30 6 0 30 Jul 2024
GRIDS: Grouped Multiple-Degradation Restoration with Image Degradation Similarity Shuo Cao Yihao Liu Wenlong Zhang Yu Qiao Chao Dong 36 4 0 17 Jul 2024
Towards stable training of parallel continual learning Li Yuepan Fan Lyu Yuyang Li Wei Feng Guangcan Liu Fanhua Shang 43 0 0 11 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 64 6 0 10 Jul 2024
DMTG: One-Shot Differentiable Multi-Task Grouping Yuan Gao Shuguo Jiang Moran Li Jin-Gang Yu Gui-Song Xia 50 1 0 06 Jul 2024
Jacobian Descent for Multi-Objective Optimization Pierre Quinton Valérian Rey 36 3 0 23 Jun 2024
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic Yuyan Zhou Liang Song Bingning Wang Weipeng Chen MoMe 30 15 0 17 Jun 2024
Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapters Yuhang Zhou Zihua Zhao Haolin Li Siyuan Du Jiangchao Yao Ya Zhang Yanfeng Wang MoMe MoE 40 3 0 14 Jun 2024
Enhancing Domain Adaptation through Prompt Gradient Alignment Hoang Phan Lam C. Tran Quyen Tran Trung Le 52 0 0 13 Jun 2024
Quantifying Task Priority for Multi-Task Optimization Wooseong Jeong Kuk-Jin Yoon 25 3 0 05 Jun 2024
HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning Shengchao Hu Ziqing Fan Li Shen Ya-Qin Zhang Yanfeng Wang Dacheng Tao OffRL 39 9 0 28 May 2024
Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost Yuan Gao Weizhong Zhang Wenhan Luo Lin Ma Jin-Gang Yu Gui-Song Xia Jiayi Ma 29 1 0 09 May 2024
GO4Align: Group Optimization for Multi-Task Alignment Jiayi Shen Cheems Wang Zehao Xiao N. V. Noord M. Worring 32 2 0 09 Apr 2024
Multi-Task Learning as enabler for General-Purpose AI-native RAN Hasan Farooq Julien Forgeat Shruti Bothe K. Čyras Md Moin 34 0 0 05 Apr 2024
Joint-Task Regularization for Partially Labeled Multi-Task Learning Kento Nishi Junsik Kim Wanhua Li Hanspeter Pfister 40 0 0 02 Apr 2024
FastCAR: Fast Classification And Regression Multi-Task Learning via Task Consolidation for Modelling a Continuous Property Variable of Object Classes Anoop Kini A. Jansche T. Bernthaler Gerhard Schneider 20 0 0 26 Mar 2024
Multi-Task Dense Prediction via Mixture of Low-Rank Experts Yuqi Yang Peng-Tao Jiang Qibin Hou Hao Zhang Jinwei Chen Bo-wen Li MoE 33 18 0 26 Mar 2024
DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Data Hanrong Ye Dan Xu DiffM 52 4 0 22 Mar 2024
Joint multi-task learning improves weakly-supervised biomarker prediction in computational pathology Omar S. M. El Nahhas Georg Wolflein M. Ligero Tim Lenz M. Treeck Firas Khader Daniel Truhn Jakob Nikolas Kather 24 1 0 06 Mar 2024