Initializing Models with Larger Ones

Initializing Models with Larger Ones

30 November 2023

Kirill Vishniakov

Papers citing "Initializing Models with Larger Ones"

16 / 16 papers shown

Title
A Model Zoo of Vision Transformers Damian Falk Léo Meynent Florence Pfammatter Konstantin Schurholt Damian Borth 32 0 0 14 Apr 2025
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization Siyuan Li L. Zhang Zedong Wang Juanxi Tian Cheng Tan ... Chang Yu Qingsong Xie Haonan Lu Haoqian Wang Zhen Lei 46 0 0 01 Apr 2025
CoRe^2: Collect, Reflect and Refine to Generate Better and Faster Shitong Shao Zikai Zhou Dian Xie Yuetong Fang Tian Ye Lichen Bai Zeke Xie DiffM VLM 68 0 0 12 Mar 2025
Towards Superior Quantization Accuracy: A Layer-sensitive Approach Feng Zhang Yanbin Liu Weihua Li Jie Lv Xiaodan Wang Q. Bai MQ 42 0 0 09 Mar 2025
FedBaF: Federated Learning Aggregation Biased by a Foundation Model Jong-Ik Park Srinivasa Pranav J. M. F. Moura Carlee Joe-Wong AI4CE 68 2 0 24 Oct 2024
JPEG Inspired Deep Learning Ahmed H. Salamah Kaixiang Zheng Yiwen Liu E. Yang 27 0 0 09 Oct 2024
FINE: Factorizing Knowledge for Initialization of Variable-sized Diffusion Models Yucheng Xie Fu Feng Ruixiao Shi Jing Wang Xin Geng AI4CE 27 2 0 28 Sep 2024
MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks Xingkui Zhu Yiran Guan Dingkang Liang Yuchao Chen Yuliang Liu Xiang Bai MoE 23 5 0 07 Jun 2024
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models Shiyu Xia Wenxuan Zhu Xu Yang Xin Geng 21 1 0 25 Apr 2024
Adapting LLaMA Decoder to Vision Transformer Jiahao Wang Wenqi Shao Mengzhao Chen Chengyue Wu Yong Liu Taiqiang Wu Kaipeng Zhang Songyang Zhang Kai-xiang Chen Ping Luo MLLM 22 4 0 10 Apr 2024
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong Chenxin An Weizhu Chen Jiawei Han Pengcheng He 21 8 0 17 Oct 2023
Understanding the Covariance Structure of Convolutional Filters Asher Trockman Devin Willmott J. Zico Kolter 41 11 0 07 Oct 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,554 0 04 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 5,723 0 29 Apr 2021
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 261 10,106 0 16 Nov 2016