Meta-Learning to Improve Pre-Training

Meta-Learning to Improve Pre-Training

2 November 2021

Jonathan Lorraine

Simon Kornblith

Matthew B. A. McDermott

Papers citing "Meta-Learning to Improve Pre-Training"

12 / 12 papers shown

Title
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling David Grangier Simin Fan Skyler Seto Pierre Ablin 36 3 0 30 Sep 2024
JacNet: Learning Functions with Structured Jacobians Jonathan Lorraine Safwan Hossain 40 6 0 23 Aug 2024
Task Discovery: Finding the Tasks that Neural Networks Generalize on Andrei Atanov Andrei Filatov Teresa Yeo Ajay Sohmshetty Amir Zamir OOD 38 10 0 01 Dec 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 32 49 0 25 Oct 2022
Task Selection for AutoML System Evaluation Jon Lorraine Nihesh Anderson Chansoo Lee Quentin de Laroussilhe Mehadi Hassen 41 4 0 26 Aug 2022
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen-li Ma Zixuan Liu Xue Liu 84 35 0 24 Jul 2022
Betty: An Automatic Differentiation Library for Multilevel Optimization Sang Keun Choe W. Neiswanger P. Xie Eric P. Xing AI4CE 23 29 0 05 Jul 2022
Cluster Head Detection for Hierarchical UAV Swarm With Graph Self-supervised Learning Zhiyu Mou J. Liu Xiang Yun F. Gao Qi-hui Wu 23 2 0 08 Mar 2022
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 248 656 0 23 Mar 2020
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 172 639 0 19 Sep 2019
Probabilistic Model-Agnostic Meta-Learning Chelsea Finn Kelvin Xu Sergey Levine BDL 165 666 0 07 Jun 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 281 11,681 0 09 Mar 2017