ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2302.09419
  4. Cited By
A Comprehensive Survey on Pretrained Foundation Models: A History from
  BERT to ChatGPT
v1v2v3 (latest)

A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT

18 February 2023
Ce Zhou
Qian Li
Chen Li
Jun Yu
Yixin Liu
Guan Wang
Kaichao Zhang
Cheng Ji
Qi Yan
Lifang He
Hao Peng
Jianxin Li
Hongzhi Zhang
Ziwei Liu
P. Xie
Caiming Xiong
Jian Pei
Philip S. Yu
U. Chicago
    AI4CE
ArXiv (abs)PDFHTML

Papers citing "A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT"

50 / 293 papers shown
Title
Partially Trained Graph Convolutional Networks Resist Oversmoothing
Partially Trained Graph Convolutional Networks Resist Oversmoothing
Dimitrios Kelesis
Dimitris Fotakis
George Giannakopoulos
SSL
69
0
0
17 Oct 2024
FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs
  with Adaptive Compression
FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression
Zhenheng Tang
Xueze Kang
Yiming Yin
Xinglin Pan
Yuxin Wang
...
Shaohuai Shi
Amelie Chi Zhou
Bo Li
Bingsheng He
Xiaowen Chu
AI4CE
109
8
0
16 Oct 2024
Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations
Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations
M. Germán-Morales
A. J. Rivera-Rivas
M. J. del Jesus Díaz
C. J. Carmona
AI4TSAI4CE
279
0
0
15 Oct 2024
Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale
  Models with Structured Pruning in Resource-Limited Clients
Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clients
Yan Li
Mingyi Li
Xiao Zhang
Guangwei Xu
Feng Chen
Yuan Yuan
Yifei Zou
Mengying Zhao
Jianbo Lu
Dongxiao Yu
60
0
0
11 Oct 2024
InstructBioMol: Advancing Biomolecule Understanding and Design Following
  Human Instructions
InstructBioMol: Advancing Biomolecule Understanding and Design Following Human Instructions
Xiang Zhuang
Keyan Ding
Tianwen Lyu
Yinuo Jiang
Xiaotong Li
...
Ming Qin
Kehua Feng
Jike Wang
Qiang Zhang
Huajun Chen
75
5
0
10 Oct 2024
Exploring Foundation Models in Remote Sensing Image Change Detection: A
  Comprehensive Survey
Exploring Foundation Models in Remote Sensing Image Change Detection: A Comprehensive Survey
Zihan Yu
Tianxiao Li
Yuxin Zhu
Rongze Pan
80
1
0
10 Oct 2024
NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired
  People
NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People
Jun Yu
Yifan Zhang
Badrinadh Aila
V. Namboodiri
106
1
0
08 Oct 2024
On the Robustness of Machine Learning Models in Predicting Thermodynamic
  Properties: a Case of Searching for New Quasicrystal Approximants
On the Robustness of Machine Learning Models in Predicting Thermodynamic Properties: a Case of Searching for New Quasicrystal Approximants
Fedor S. Avilov
R. Eremin
S. Budennyy
Innokentiy S. Humonen
AAML
87
0
0
02 Oct 2024
Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving
Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving
Zhenghao Peng
Wenjie Luo
Yiren Lu
Tianyi Shen
Cole Gulino
Ari Seff
Justin Fu
62
9
0
26 Sep 2024
Optical Lens Attack on Deep Learning Based Monocular Depth Estimation
Optical Lens Attack on Deep Learning Based Monocular Depth Estimation
Ce Zhou
Qiben Yan
Daniel Kent
Guangjing Wang
Ziqi Zhang
Hayder Radha
AAML
71
0
0
25 Sep 2024
Surveying the MLLM Landscape: A Meta-Review of Current Surveys
Surveying the MLLM Landscape: A Meta-Review of Current Surveys
Ming Li
Keyu Chen
Ziqian Bi
Ming Liu
Benji Peng
...
Jinlang Wang
Sen Zhang
X. Pan
Jiawei Xu
Pohsun Feng
OffRL
118
2
0
17 Sep 2024
COMFORT: A Continual Fine-Tuning Framework for Foundation Models
  Targeted at Consumer Healthcare
COMFORT: A Continual Fine-Tuning Framework for Foundation Models Targeted at Consumer Healthcare
Chia-Hao Li
N. Jha
92
1
0
14 Sep 2024
LLM-POTUS Score: A Framework of Analyzing Presidential Debates with
  Large Language Models
LLM-POTUS Score: A Framework of Analyzing Presidential Debates with Large Language Models
Zhengliang Liu
Yiwei Li
Oleksandra Zolotarevych
Rongwei Yang
Tianming Liu
ELM
22
1
0
12 Sep 2024
Exploring Straightforward Conversational Red-Teaming
Exploring Straightforward Conversational Red-Teaming
George Kour
Naama Zwerdling
Marcel Zalmanovici
Ateret Anaby-Tavor
Ora Nova Fandina
E. Farchi
AAML
417
1
0
07 Sep 2024
A Comprehensive Survey on Evidential Deep Learning and Its Applications
A Comprehensive Survey on Evidential Deep Learning and Its Applications
Junyu Gao
Mengyuan Chen
Liangyu Xiang
Changsheng Xu
EDLBDLUQCV
129
6
0
07 Sep 2024
An overview of domain-specific foundation model: key technologies, applications and challenges
An overview of domain-specific foundation model: key technologies, applications and challenges
Haolong Chen
Hanzhi Chen
Zijian Zhao
Kaifeng Han
Guangxu Zhu
Yichen Zhao
Ying Du
Wei Xu
Qingjiang Shi
ALMVLM
111
5
0
06 Sep 2024
A Comparative Study of Pre-training and Self-training
A Comparative Study of Pre-training and Self-training
Yiheng Wang
Jiayu Lin
Zuoquan Lin
SSL
112
0
0
04 Sep 2024
Tracing Privacy Leakage of Language Models to Training Data via Adjusted
  Influence Functions
Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence Functions
Jinxin Liu
Zao Yang
51
1
0
20 Aug 2024
FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated
  Knowledge Injection
FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection
Jiaqi Wang
Xiaochen Wang
Lingjuan Lyu
Jinghui Chen
Fenglong Ma
112
6
0
17 Aug 2024
FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models
FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models
Xiaochen Wang
Jiaqi Wang
Houping Xiao
Jianfei Chen
Fenglong Ma
MedIm
185
8
0
17 Aug 2024
Large Investment Model
Large Investment Model
Jian Guo
H. Shum
AIFin
133
0
0
12 Aug 2024
Enhancing Representation Learning of EEG Data with Masked Autoencoders
Enhancing Representation Learning of EEG Data with Masked Autoencoders
Yifei Zhou
Sitong Liu
81
0
0
09 Aug 2024
Depth Any Canopy: Leveraging Depth Foundation Models for Canopy Height
  Estimation
Depth Any Canopy: Leveraging Depth Foundation Models for Canopy Height Estimation
Daniele Rege Cambrin
Isaac Corley
Paolo Garza
68
3
0
08 Aug 2024
Defining and Evaluating Decision and Composite Risk in Language Models
  Applied to Natural Language Inference
Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference
Ke Shen
Mayank Kejriwal
74
0
0
04 Aug 2024
Efficient Training of Large Language Models on Distributed
  Infrastructures: A Survey
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
Jiangfei Duan
Shuo Zhang
Zerui Wang
Lijuan Jiang
Wenwen Qu
...
Dahua Lin
Yonggang Wen
Xin Jin
Tianwei Zhang
Peng Sun
154
13
0
29 Jul 2024
Leveraging Foundation Models for Zero-Shot IoT Sensing
Leveraging Foundation Models for Zero-Shot IoT Sensing
Dinghao Xue
Xiaoran Fan
Tao Chen
Guohao Lan
Qun Song
AI4CE
74
2
0
29 Jul 2024
Speech-Guided Sequential Planning for Autonomous Navigation using Large
  Language Model Meta AI 3 (Llama3)
Speech-Guided Sequential Planning for Autonomous Navigation using Large Language Model Meta AI 3 (Llama3)
Alkesh K. Srivastava
Philip Dames
LLMAGLM&Ro
120
1
0
13 Jul 2024
Federated Learning and AI Regulation in the European Union: Who is
  Responsible? -- An Interdisciplinary Analysis
Federated Learning and AI Regulation in the European Union: Who is Responsible? -- An Interdisciplinary Analysis
Herbert Woisetschläger
Simon Mertel
Christoph Krönke
R. Mayer
Hans-Arno Jacobsen
FedML
72
2
0
11 Jul 2024
Synthetic Data Aided Federated Learning Using Foundation Models
Synthetic Data Aided Federated Learning Using Foundation Models
Fatima Abacha
Sin G. Teo
Lucas C. Cordeiro
Mustafa A. Mustafa
FedML
61
2
0
06 Jul 2024
On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboards
On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboards
Zhimin Zhao
A. A. Bangash
F. Côgo
Bram Adams
Ahmed E. Hassan
200
1
0
04 Jul 2024
For a semiotic AI: Bridging computer vision and visual semiotics for
  computational observation of large scale facial image archives
For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives
Lia Morra
A. Santangelo
Pietro Basci
Luca Piano
Fabio Garcea
Fabrizio Lamberti
Massimo Leone
71
1
0
03 Jul 2024
LegalTurk Optimized BERT for Multi-Label Text Classification and NER
LegalTurk Optimized BERT for Multi-Label Text Classification and NER
Farnaz Zeidi
Mehmet Fatih Amasyali
Çiğdem Erol
VLM
58
2
0
30 Jun 2024
Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with
  3D Semantic Maps
Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps
Dicong Qiu
Wenzong Ma
Zhenfu Pan
Hui Xiong
Junwei Liang
LM&Ro
97
8
0
26 Jun 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness
Large Batch Analysis for Adagrad Under Anisotropic Smoothness
Yuxing Liu
Boyao Wang
Tong Zhang
62
6
0
21 Jun 2024
Younger: The First Dataset for Artificial Intelligence-Generated Neural
  Network Architecture
Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture
Zhengxin Yang
Wanling Gao
Luzhou Peng
Yunyou Huang
Fei Tang
Jianfeng Zhan
71
1
0
20 Jun 2024
InternalInspector $I^2$: Robust Confidence Estimation in LLMs through
  Internal States
InternalInspector I2I^2I2: Robust Confidence Estimation in LLMs through Internal States
Mohammad Beigi
Ying Shen
Runing Yang
Zihao Lin
Qifan Wang
Ankith Mohan
Jianfeng He
Ming Jin
Chang-Tien Lu
Lifu Huang
HILM
78
10
0
17 Jun 2024
A Complete Survey on LLM-based AI Chatbots
A Complete Survey on LLM-based AI Chatbots
Sumit Kumar Dam
Choong Seon Hong
Yu Qiao
Chaoning Zhang
104
62
0
17 Jun 2024
Save It All: Enabling Full Parameter Tuning for Federated Large Language
  Models via Cycle Block Gradient Descent
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent
Lin Wang
Zhichao Wang
Xiaoying Tang
75
1
0
17 Jun 2024
Multiple Sources are Better Than One: Incorporating External Knowledge
  in Low-Resource Glossing
Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing
Changbing Yang
Garrett Nicolai
Miikka Silfverberg
70
1
0
16 Jun 2024
GraphFM: A Comprehensive Benchmark for Graph Foundation Model
GraphFM: A Comprehensive Benchmark for Graph Foundation Model
Yuhao Xu
Xinqi Liu
Keyu Duan
Yi Fang
Yu-Neng Chuang
Daochen Zha
Qiaoyu Tan
AI4CE
50
1
0
12 Jun 2024
Reading Miscue Detection in Primary School through Automatic Speech
  Recognition
Reading Miscue Detection in Primary School through Automatic Speech Recognition
Lingyun Gao
Cristian Tejedor-García
H. Strik
C. Cucchiarini
75
1
0
11 Jun 2024
Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based
  Interactions
Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions
Cheng Tan
Dongxin Lyu
Siyuan Li
Zhangyang Gao
Jingxuan Wei
Siqi Ma
Zicheng Liu
Stan Z. Li
LLMAG
81
13
0
09 Jun 2024
Generalist Multimodal AI: A Review of Architectures, Challenges and
  Opportunities
Generalist Multimodal AI: A Review of Architectures, Challenges and Opportunities
Sai Munikoti
Ian Stewart
Sameera Horawalavithana
Henry Kvinge
Tegan H. Emerson
Sandra E Thompson
Karl Pazdernik
102
2
0
08 Jun 2024
Large Generative Graph Models
Large Generative Graph Models
Yu Wang
Ryan Rossi
Namyong Park
Huiyuan Chen
Nesreen K. Ahmed
Puja Trivedi
Franck Dernoncourt
Danai Koutra
Hanyu Wang
AI4CE
102
4
0
07 Jun 2024
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language
  Models
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models
Tao Fan
Guoqiang Ma
Yan Kang
Hanlin Gu
Yuanfeng Song
Lixin Fan
Kai Chen
Qiang Yang
106
12
0
04 Jun 2024
MiniGPT-Reverse-Designing: Predicting Image Adjustments Utilizing MiniGPT-4
MiniGPT-Reverse-Designing: Predicting Image Adjustments Utilizing MiniGPT-4
Vahid Azizi
Fatemeh Koochaki
VLM
106
0
0
03 Jun 2024
LIDAO: Towards Limited Interventions for Debiasing (Large) Language
  Models
LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models
Tianci Liu
Haoyu Wang
Shiyang Wang
Yu Cheng
Jing Gao
ALM
75
1
0
01 Jun 2024
Large Language Models are Zero-Shot Next Location Predictors
Large Language Models are Zero-Shot Next Location Predictors
Ciro Beneduce
Bruno Lepri
Massimiliano Luca
106
9
0
31 May 2024
Optimizing Foundation Model Inference on a Many-tiny-core Open-source
  RISC-V Platform
Optimizing Foundation Model Inference on a Many-tiny-core Open-source RISC-V Platform
Viviane Potocnik
Luca Colagrande
Tim Fischer
L. Bertaccini
Daniele Jahier Pagliari
Luca Bompani
Luca Benini
87
3
0
29 May 2024
PLUG: Revisiting Amodal Segmentation with Foundation Model and
  Hierarchical Focus
PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus
Zhaochen Liu
Limeng Qiao
Xiangxiang Chu
Tingting Jiang
69
2
0
25 May 2024
Previous
123456
Next