Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2407.08056
Cited By
v1
v2 (latest)
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences
10 July 2024
Nikolaos Dimitriadis
Pascal Frossard
François Fleuret
MoE
Re-assign community
ArXiv (abs)
PDF
HTML
Github (7★)
Papers citing
"Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences"
50 / 71 papers shown
Towards Relaxed Multimodal Inputs for Gait-based Parkinson's Disease Assessment
Minlin Zeng
Z. Zhou
Yang Qiu
Martin J. McKeown
Zhiqi Shen
192
0
0
17 Oct 2025
One Rank at a Time: Cascading Error Dynamics in Sequential Learning
Mahtab Alizadeh Vandchali
Fangshuo
Liao
Anastasios Kyrillidis
221
3
0
28 May 2025
PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
Xiaoyuan Zhang
Weisen Jiang
Yuancheng Xu
Hao Chen
Ying-Cong Chen
421
13
0
06 May 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond
Weiyu Chen
Xiaoyuan Zhang
Xiaoyuan Zhang
Xi Lin
Han Zhao
Gang Qu
James T. Kwok
524
23
0
19 Jan 2025
MoIN: Mixture of Introvert Experts to Upcycle an LLM
Ajinkya Tejankar
K. Navaneet
Ujjawal Panchal
Kossar Pourahmadi
Hamed Pirsiavash
MoE
439
0
0
13 Oct 2024
LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch
Neural Information Processing Systems (NeurIPS), 2024
Xiaoyuan Zhang
Liang Zhao
Yingying Yu
Xi Lin
Yifan Chen
Han Zhao
Qingfu Zhang
AI4CE
360
9
0
04 Sep 2024
Efficient Pareto Manifold Learning with Low-Rank Structure
Weiyu Chen
James T. Kwok
248
11
0
30 Jul 2024
PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generation
Yinghua Yao
Yuangang Pan
Jing Li
Ivor Tsang
Xin Yao
397
14
0
05 Jul 2024
Mixture-of-Subspaces in Low-Rank Adaptation
Taiqiang Wu
Jiahao Wang
Zhe Zhao
Ngai Wong
681
54
0
16 Jun 2024
Localizing Task Information for Improved Model Merging and Compression
International Conference on Machine Learning (ICML), 2024
Ke Wang
Nikolaos Dimitriadis
Guillermo Ortiz-Jimenez
Franccois Fleuret
Pascal Frossard
MoMe
330
106
0
13 May 2024
HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
Chunlin Tian
Zhanying Shi
Zhijiang Guo
Li Li
Chengzhong Xu
256
122
0
30 Apr 2024
Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
Wenfeng Feng
Chuzhan Hao
Yuewei Zhang
Yu Han
Hao Wang
ALM
MoE
222
28
0
06 Mar 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters
Minyoung Huh
Brian Cheung
Jeremy Bernstein
Phillip Isola
Pulkit Agrawal
359
17
0
26 Feb 2024
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild
Ziyu Zhao
Yaoyao Yu
Guoyin Wang
Wangchunshu Zhou
Hongxia Yang
Kun Kuang
Leilei Gan
MoMe
325
64
0
15 Feb 2024
DoRA: Weight-Decomposed Low-Rank Adaptation
Shih-yang Liu
Chien-Yi Wang
Hongxu Yin
Pavlo Molchanov
Yu-Chiang Frank Wang
Kwang-Ting Cheng
Min-Hung Chen
899
777
0
14 Feb 2024
Panacea: Pareto Alignment via Preference Adaptation for LLMs
Yifan Zhong
Chengdong Ma
Xiaoyuan Zhang
Ziran Yang
Haojun Chen
Qingfu Zhang
Siyuan Qi
Yaodong Yang
472
61
0
03 Feb 2024
Dealing with Structure Constraints in Evolutionary Pareto Set Learning
IEEE Transactions on Evolutionary Computation (IEEE TEVC), 2023
Xi Lin
Xiao-Yan Zhang
Zhiyuan Yang
Qingfu Zhang
546
5
0
31 Oct 2023
VeRA: Vector-based Random Matrix Adaptation
International Conference on Learning Representations (ICLR), 2023
D. J. Kopiczko
Tijmen Blankevoort
Yuki Markus Asano
VLM
443
297
0
17 Oct 2023
Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation
Shin-Ying Yeh
Yu-Guan Hsieh
Zhidong Gao
Bernard B. W. Yang
Giyeong Oh
Yanmin Gong
296
128
0
26 Sep 2023
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
Chengsong Huang
Qian Liu
Bill Yuchen Lin
Tianyu Pang
Chao Du
Min Lin
MoMe
587
328
0
25 Jul 2023
FAMO: Fast Adaptive Multitask Optimization
Neural Information Processing Systems (NeurIPS), 2023
B. Liu
Yihao Feng
Peter Stone
Qian Liu
594
84
0
06 Jun 2023
TIES-Merging: Resolving Interference When Merging Models
Neural Information Processing Systems (NeurIPS), 2023
Prateek Yadav
Derek Tam
Leshem Choshen
Colin Raffel
Joey Tianyi Zhou
MoMe
461
624
0
02 Jun 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models
Neural Information Processing Systems (NeurIPS), 2023
Guillermo Ortiz-Jiménez
Alessandro Favero
P. Frossard
MoMe
677
201
0
22 May 2023
The geometry of hidden representations of large transformer models
Neural Information Processing Systems (NeurIPS), 2023
L. Valeriani
Diego Doimo
F. Cuturello
Alessandro Laio
A. Ansuini
Alberto Cazzaniga
MILM
427
100
0
01 Feb 2023
Editing Models with Task Arithmetic
International Conference on Learning Representations (ICLR), 2022
Gabriel Ilharco
Marco Tulio Ribeiro
Mitchell Wortsman
Suchin Gururangan
Ludwig Schmidt
Hannaneh Hajishirzi
Ali Farhadi
KELM
MoMe
MU
1.5K
873
0
08 Dec 2022
Improving Pareto Front Learning via Multi-Sample Hypernetworks
AAAI Conference on Artificial Intelligence (AAAI), 2022
L. P. Hoang
Dung D. Le
Tuan A. Tran
Thang Tran Ngoc
894
31
0
02 Dec 2022
Neural networks trained with SGD learn distributions of increasing complexity
International Conference on Machine Learning (ICML), 2022
Maria Refinetti
Alessandro Ingrosso
Sebastian Goldt
UQCV
397
58
0
21 Nov 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models
International Conference on Machine Learning (ICML), 2022
Nikolaos Dimitriadis
P. Frossard
Franccois Fleuret
352
33
0
18 Oct 2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation
Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2022
Mojtaba Valipour
Mehdi Rezagholizadeh
I. Kobyzev
A. Ghodsi
540
268
0
14 Oct 2022
A Generalist Agent
Scott E. Reed
Konrad Zolna
Emilio Parisotto
Sergio Gomez Colmenarejo
Alexander Novikov
...
Yutian Chen
R. Hadsell
Oriol Vinyals
Mahyar Bordbar
Nando de Freitas
LM&Ro
LLMAG
AI4CE
629
1,023
0
12 May 2022
Auto-Lambda: Disentangling Dynamic Task Relationships
Shikun Liu
Stephen James
Andrew J. Davison
Edward Johns
338
100
0
07 Feb 2022
Multi-Task Learning as a Bargaining Game
International Conference on Machine Learning (ICML), 2022
Aviv Navon
Aviv Shamsian
Idan Achituve
Haggai Maron
Kenji Kawaguchi
Gal Chechik
Ethan Fetaya
402
231
0
02 Feb 2022
Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning
Xiaoyuan Zhang
Feiyang Ye
Yu Zhang
Ivor W. Tsang
404
126
0
20 Nov 2021
Conflict-Averse Gradient Descent for Multi-task Learning
Bo Liu
Xingchao Liu
Xiaojie Jin
Peter Stone
Qiang Liu
515
477
0
26 Oct 2021
FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning
International Conference on Learning Representations (ICLR), 2021
Nam Hyeon-Woo
Moon Ye-Bin
Tae-Hyun Oh
FedML
431
172
0
13 Aug 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Elad Ben-Zaken
Shauli Ravfogel
Yoav Goldberg
1.2K
1,675
0
18 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
Neural Information Processing Systems (NeurIPS), 2021
Rabeeh Karimi Mahabadi
James Henderson
Sebastian Ruder
MoE
621
619
0
08 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Brian Lester
Rami Al-Rfou
Noah Constant
VPVLM
1.6K
5,294
0
18 Apr 2021
Scalable Pareto Front Approximation for Deep Multi-Objective Learning
Industrial Conference on Data Mining (IDM), 2021
Michael Ruchte
Josif Grabocka
274
71
0
24 Mar 2021
RotoGrad: Gradient Homogenization in Multitask Learning
International Conference on Learning Representations (ICLR), 2021
Adrián Javaloy
Isabel Valera
528
112
0
03 Mar 2021
Curriculum Learning: A Survey
International Journal of Computer Vision (IJCV), 2021
Petru Soviany
Radu Tudor Ionescu
Paolo Rota
Andrii Zadaianchuk
ODL
703
521
0
25 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Xiang Lisa Li
Abigail Z. Jacobs
849
5,574
0
01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
Annual Meeting of the Association for Computational Linguistics (ACL), 2020
Armen Aghajanyan
Luke Zettlemoyer
Sonal Gupta
389
806
1
22 Dec 2020
Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout
Zhao Chen
Jiquan Ngiam
Yanping Huang
Thang Luong
Henrik Kretzschmar
Yuning Chai
Dragomir Anguelov
279
278
0
14 Oct 2020
Controllable Pareto Multi-Task Learning
Xi Lin
Zhiyuan Yang
Qingfu Zhang
Sam Kwong
MoE
324
80
0
13 Oct 2020
Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models
Zirui Wang
Yulia Tsvetkov
Orhan Firat
Yuan Cao
414
238
0
12 Oct 2020
Learning the Pareto Front with Hypernetworks
Aviv Navon
Aviv Shamsian
Gal Chechik
Ethan Fetaya
536
183
0
08 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization
International Conference on Learning Representations (ICLR), 2020
Pierre Foret
Ariel Kleiner
H. Mobahi
Behnam Neyshabur
AAML
961
1,815
0
03 Oct 2020
Multi-Task Learning with Deep Neural Networks: A Survey
M. Crawshaw
CVBM
525
760
0
10 Sep 2020
Efficient Continuous Pareto Exploration in Multi-Task Learning
Pingchuan Ma
Tao Du
Wojciech Matusik
445
107
0
29 Jun 2020
1
2
Next
Page 1 of 2
