Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout

14 October 2020

Henrik Kretzschmar

Papers citing "Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout"

50 / 136 papers shown

Title
Smooth Tchebycheff Scalarization for Multi-Objective Optimization Xi Lin Xiao-Yan Zhang Zhiyuan Yang Fei Liu Zhenkun Wang Qingfu Zhang 35 16 0 29 Feb 2024
InterroGate: Learning to Share, Specialize, and Prune Representations for Multi-task Learning B. Bejnordi Gaurav Kumar Amelie Royer Christos Louizos Tijmen Blankevoort Mohsen Ghafoorian CVBM 39 0 0 26 Feb 2024
Fair Resource Allocation in Multi-Task Learning Hao Ban Kaiyi Ji 27 10 0 23 Feb 2024
Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space Zongru Wu Zhuosheng Zhang Pengzhou Cheng Gongshen Liu AAML 44 4 0 19 Feb 2024
Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement Dayou Mao Yuhao Chen Yifan Wu Maximilian Gilles Alexander Wong AAML 38 0 0 05 Feb 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 54 37 0 05 Feb 2024
Robust Multi-Task Learning with Excess Risks Yifei He Shiji Zhou Guojun Zhang Hyokun Yun Yi Tian Xu Belinda Zeng Trishul M. Chilimbi Han Zhao 53 7 0 03 Feb 2024
A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization Feiyang Ye Baijiong Lin Xiao-Qun Cao Yu Zhang Ivor Tsang 47 6 0 17 Jan 2024
Elastic Multi-Gradient Descent for Parallel Continual Learning Fan Lyu Wei Feng Yuepan Li Qing Sun Fanhua Shang Liang Wan Liang Wang 27 2 0 02 Jan 2024
FedHCA $^2$ : Towards Hetero-Client Federated Multi-Task Learning Yuxiang Lu Suizhi Huang Yuwen Yang Shalayiding Sirejiding Yue Ding Hongtao Lu FedML 44 3 0 22 Nov 2023
Examining Common Paradigms in Multi-Task Learning Cathrin Elich Lukas Kirchdorfer Jan M. Kohler Lukas Schott 29 0 0 08 Nov 2023
Scalarization for Multi-Task and Multi-Domain Learning at Scale Amelie Royer Tijmen Blankevoort B. Bejnordi 28 17 0 13 Oct 2023
Denoising Task Routing for Diffusion Models Byeongjun Park Sangmin Woo Hyojun Go Jin-Young Kim Changick Kim DiffM 14 18 0 11 Oct 2023
Factorized Tensor Networks for Multi-Task and Multi-Domain Learning Yash Garg Nebiyou Yismaw Rakib Hyder Ashley Prater-Bennette M. Salman Asif 26 2 0 09 Oct 2023
AdaMerging: Adaptive Model Merging for Multi-Task Learning Enneng Yang Zhenyi Wang Li Shen Shiwei Liu Guibing Guo Xingwei Wang Dacheng Tao MoMe 33 95 0 04 Oct 2023
Multi-task Learning with 3D-Aware Regularization Weihong Li Steven G. McDonagh A. Leonardis Hakan Bilen 26 3 0 02 Oct 2023
Multi-task View Synthesis with Neural Radiance Fields Shuhong Zheng Zhipeng Bao Martial Hebert Yu-Xiong Wang 21 4 0 29 Sep 2023
Multi-Task Cooperative Learning via Searching for Flat Minima Fuping Wu Le Zhang Yang Sun Yuanhan Mo Thomas Nichols Bartłomiej W. Papież 19 1 0 21 Sep 2023
Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspective Yuzheng Hu Ruicheng Xian Qilong Wu Qiuling Fan Lang Yin Han Zhao 27 39 0 27 Aug 2023
Dual-Balancing for Multi-Task Learning Baijiong Lin Weisen Jiang Feiyang Ye Yu Zhang Pengguang Chen Yingke Chen Shu Liu James T. Kwok CVBM 25 12 0 23 Aug 2023
OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes Tao Xie Kun Dai Si-Tong Lu K. Wang Zhiqiang Jiang Jin Gao Dedong Liu J. Xu Lijun Zhao Rui Li 50 9 0 23 Aug 2023
STEM: Unleashing the Power of Embeddings for Multi-task Recommendation Liangcai Su Junwei Pan Ximei Wang Xi Xiao Shijie Quan Xihua Chen Jie Jiang 17 15 0 16 Aug 2023
Deformable Mixer Transformer with Gating for Multi-Task Learning of Dense Prediction Yangyang Xu Yibo Yang Bernard Ghanemm L. Zhang Du Bo Dacheng Tao 13 1 0 10 Aug 2023
Improvable Gap Balancing for Multi-Task Learning Yanqi Dai Nanyi Fei Zhiwu Lu 40 4 0 28 Jul 2023
TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts Hanrong Ye Dan Xu MoE 36 26 0 28 Jul 2023
Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models Kecheng Zheng Wei Wu Ruili Feng Kai Zhu Jiawei Liu Deli Zhao Zhengjun Zha Wei Chen Yujun Shen VLM 19 8 0 27 Jul 2023
When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review Maxime Fontana Michael W. Spratling Miaojing Shi 44 6 0 25 Jul 2023
Gradient Sparsification For Masked Fine-Tuning of Transformers J. Ó. Neill Sourav Dutta 19 0 0 19 Jul 2023
Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation Yuwei Cao Liangwei Yang Chen Wang Zhiwei Liu Hao Peng Chenyu You Philip S. Yu 18 21 0 26 Jun 2023
InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding Hanrong Ye Dan Xu ViT 27 10 0 08 Jun 2023
Sample-Level Weighting for Multi-Task Learning with Auxiliary Tasks Emilie Grégoire M. H. Chaudhary Sam Verboven 24 1 0 07 Jun 2023
FAMO: Fast Adaptive Multitask Optimization B. Liu Yihao Feng Peter Stone Qian Liu 33 30 0 06 Jun 2023
Biologically-Motivated Learning Model for Instructed Visual Processing R. Abel S. Ullman 20 0 0 04 Jun 2023
Addressing Negative Transfer in Diffusion Models Hyojun Go Jinyoung Kim Yunsung Lee Seunghyun Lee Shinhyeok Oh Hyeongdon Moon Seungtaek Choi DiffM VLM 16 24 0 01 Jun 2023
Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs Yi Sun Xin Xu J. Li Xiaochang Hu Yifei Shi L. Zeng 19 2 0 31 May 2023
Independent Component Alignment for Multi-Task Learning Dmitry Senushkin Nikolay Patakin Arseny Kuznetsov Anton Konushin CVBM 32 41 0 30 May 2023
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms Peiyao Xiao Hao Ban Kaiyi Ji 24 18 0 28 May 2023
Efficient Computation Sharing for Multi-Task Visual Scene Understanding Sara Shoouri Mingyu Yang Zichen Fan Hun-Seok Kim MoE 26 3 0 16 Mar 2023
Efficient Diffusion Training via Min-SNR Weighting Strategy Tiankai Hang Shuyang Gu Chen Li Jianmin Bao Dong Chen Han Hu Xin Geng B. Guo 18 150 0 16 Mar 2023
RotoGBML: Towards Out-of-Distribution Generalization for Gradient-Based Meta-Learning Min Zhang Zifeng Zhuang Zhitao Wang Donglin Wang Wen-Bin Li 46 5 0 12 Mar 2023
Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learning Guangyuan Shi Qimai Li Wenlong Zhang Jiaxin Chen Xiao-Ming Wu 32 32 0 22 Feb 2023
Auxiliary Learning as an Asymmetric Bargaining Game Aviv Shamsian Aviv Navon Neta Glazer Kenji Kawaguchi Gal Chechik Ethan Fetaya 35 8 0 31 Jan 2023
ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning Junguang Jiang Baixu Chen Junwei Pan Ximei Wang Liu Dapeng Jie Jiang Mingsheng Long MoMe 26 19 0 30 Jan 2023
Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners Zitian Chen Yikang Shen Mingyu Ding Zhenfang Chen Hengshuang Zhao E. Learned-Miller Chuang Gan MoE 11 14 0 15 Dec 2022
Do Text-to-Text Multi-Task Learners Suffer from Task Conflict? David Mueller Nicholas Andrews Mark Dredze 31 6 0 13 Dec 2022
AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning Enneng Yang Junwei Pan Ximei Wang Haibin Yu Li Shen Xihua Chen Lei Xiao Jie Jiang G. Guo 38 43 0 28 Nov 2022
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 25 10 0 24 Nov 2022
M $^3$ ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design Hanxue Liang Zhiwen Fan Rishov Sarkar Ziyu Jiang Tianlong Chen Kai Zou Yu Cheng Cong Hao Zhangyang Wang MoE 31 81 0 26 Oct 2022
Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach H. Fernando Han Shen Miao Liu Subhajit Chaudhury K. Murugesan Tianyi Chen 23 8 0 23 Oct 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models Nikolaos Dimitriadis P. Frossard Franccois Fleuret 16 25 0 18 Oct 2022