Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 108 papers shown

Title
Heterogeneous Data Game: Characterizing the Model Competition Across Multiple Data Sources Renzhe Xu K. Wang Bo Li 29 0 0 12 May 2025
Query-based Knowledge Transfer for Heterogeneous Learning Environments Norah Alballa Wenxuan Zhang Ziquan Liu A. Abdelmoniem Mohamed Elhoseiny Marco Canini 24 0 0 12 Apr 2025
Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters Ruiqi Zhu Endong Sun Guanhe Huang Oya Celiktutan CLL OnRL 59 0 0 24 Mar 2025
Statistical Deficiency for Task Inclusion Estimation Loïc Fosse Frédéric Béchet Benoit Favre Géraldine Damnati Gwénolé Lecorvé Maxime Darrin Philippe Formont Pablo Piantanida 103 0 0 07 Mar 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 114 0 0 03 Mar 2025
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation Zhiyuan Liu Yanchen Luo Han Huang Enzhi Zhang Sihang Li Junfeng Fang Yaorui Shi X. Wang Kenji Kawaguchi Tat-Seng Chua 100 3 0 18 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 115 3 0 06 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 89 1 0 06 Feb 2025
Understanding the Capabilities and Limitations of Weak-to-Strong Generalization Wei Yao Wenkai Yang Z. Wang Yankai Lin Yong Liu ELM 95 1 0 03 Feb 2025
Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Scheduling Nannan Li Kevin J. Shih Bryan A. Plummer DiffM 54 0 0 08 Jan 2025
Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models Benjamin Laufer Jon M. Kleinberg Hoda Heidari 55 8 0 03 Jan 2025
Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Aodi Li Liansheng Zhuang Xiao Long Minghong Yao Shafei Wang 142 0 0 18 Dec 2024
Meta Curvature-Aware Minimization for Domain Generalization Z. Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 148 1 0 16 Dec 2024
LAGUNA: LAnguage Guided UNsupervised Adaptation with structured spaces Anxhelo Diko Antonino Furnari Luigi Cinque G. Farinella 88 0 0 23 Nov 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 31 1 0 07 Nov 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 90 1 0 11 Oct 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 29 0 0 01 Oct 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024
Macroeconomic Forecasting with Large Language Models Andrea Carriero Davide Pettenuzzo Shubhranshu Shekhar 38 4 0 01 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 50 13 0 24 Jun 2024
Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs Rui Yang Ruomeng Ding Yong Lin Huan Zhang Tong Zhang 26 42 0 14 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 69 3 0 05 Jun 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 34 2 0 22 May 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View Chenming Shang Hengyuan Zhang Hao Wen Yujiu Yang 33 5 0 13 Apr 2024
Mixture of Low-rank Experts for Transferable AI-Generated Image Detection Zihan Liu Hanyi Wang Yaoyu Kang Shilin Wang MoE 36 12 0 07 Apr 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 74 3 0 18 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 37 1 0 01 Mar 2024
Embedding Compression for Teacher-to-Student Knowledge Transfer Yiwei Ding Alexander Lerch 19 1 0 09 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 32 4 0 31 Jan 2024
LanDA: Language-Guided Multi-Source Domain Adaptation Zhenbin Wang Lei Zhang Lituan Wang Minjuan Zhu 23 10 0 25 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 29 246 0 19 Jan 2024
HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization Guanglin Zhou Zhongyi Han Shiming Chen Biwei Huang Liming Zhu Tongliang Liu Lina Yao Kun Zhang 32 3 0 18 Jan 2024
Preserving Silent Features for Domain Generalization Chujie Zhao Tianren Zhang Feng Chen 18 0 0 06 Jan 2024
LaViP:Language-Grounded Visual Prompts Nilakshan Kunananthaseelan Jing Zhang Mehrtash Harandi VLM 8 0 0 18 Dec 2023
Open Domain Generalization with a Single Network by Regularization Exploiting Pre-trained Features Inseop Chung Kiyoon Yoo Nojun Kwak VLM 14 0 0 08 Dec 2023
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 43 1 0 30 Nov 2023
Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks Jiarong Xu Renhong Huang Xin Jiang Yuxuan Cao Carl Yang Chunping Wang Yang Yang AI4CE 31 14 0 02 Nov 2023
Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies Kieran Wood Samuel Kessler Stephen J. Roberts S. Zohren AI4TS 21 2 0 16 Oct 2023
Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning Lapo Frati Neil Traft Jeff Clune Nick Cheney CLL 11 0 0 12 Oct 2023
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection Jiaying Wu Xinyu Chen Haobin Yang Qi Zhao Yuhui Shi AAML 19 11 0 28 Sep 2023
Test-Time Training for Speech Sri Harsha Dumpala Chandramouli Shama Sastry Sageev Oore 35 1 0 19 Sep 2023
LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters Ching Chang Wei-Yao Wang Wenjie Peng Tien-Fu Chen AI4TS 30 45 0 16 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 64 33 0 15 Aug 2023
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study Luis P. Silvestrin Shujian Yu Mark Hoogendoorn OOD 24 1 0 17 Jul 2023
Advances and Challenges in Meta-Learning: A Technical Review Anna Vettoruzzo Mohamed-Rafik Bouguelia Joaquin Vanschoren Thorsteinn Rögnvaldsson K. Santosh OffRL 19 70 0 10 Jul 2023
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn OOD 14 5 0 19 Jun 2023
Drag-guided diffusion models for vehicle image generation Nikos Arechiga Frank Permenter Binyang Song Chenyang Yuan DiffM 29 12 0 16 Jun 2023
LRVS-Fashion: Extending Visual Search with Referring Instructions Simon Lepage Jérémie Mary David Picard 18 1 0 05 Jun 2023
Identifying Visible Tissue in Intraoperative Ultrasound Images during Brain Surgery: A Method and Application Alistair Weld L. Dixon Giulio Anichini Michael Dyck Alex Ranne Sophie Camp Stamatia Giannarou 16 2 0 01 Jun 2023
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection Rui Shao Tianxing Wu Liqiang Nie Ziwei Liu 16 11 0 01 Jun 2023