Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior

1 June 2023

Papers citing "Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior"

50 / 53 papers shown

Title
Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models Congcong Zhu Xiaoyan Xu Jiayue Han Jingrun Chen OOD AI4CE 31 0 0 16 May 2025
Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning Yuanzhao Zhang William Gilpin AI4TS 12 0 0 16 May 2025
TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series Xiaolei Qin Di Wang Jingyang Zhang Fengxiang Wang Xin Su Bo Du Liangpei Zhang AI4TS 24 0 0 13 May 2025
CodePDE: An Inference Framework for LLM-driven PDE Solver Generation Shanda Li Tanya Marwah Junhong Shen W. Sun Andrej Risteski Yiming Yang Ameet Talwalkar AI4CE 39 0 0 13 May 2025
A Physics-preserved Transfer Learning Method for Differential Equations Hao-Ran Yang Chuan-Xian Ren 143 0 0 02 May 2025
Improving Routing in Sparse Mixture of Experts with Graph of Tokens Tam Minh Nguyen Ngoc N. Tran Khai Nguyen Richard G. Baraniuk MoE 66 0 0 01 May 2025
Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence Moein Darman P. Hassanzadeh Laure Zanna A. Chattopadhyay 27 0 0 21 Apr 2025
Learning Dual-Arm Coordination for Grasping Large Flat Objects Yongliang Wang H. Kasaei 32 0 0 04 Apr 2025
Paving the way for scientific foundation models: enhancing generalization and robustness in PDEs with constraint-aware pre-training A. Totounferoush Serge Kotchourko Michael W. Mahoney Shri Kiran Srinivasan AI4CE 34 1 0 24 Mar 2025
Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimization Philipp Wagner Tobias Nagel Philipp Leube Marco F. Huber 58 0 0 20 Mar 2025
Machine learning for modelling unstructured grid data in computational physics: a review Sibo Cheng Marc Bocquet Weiping Ding Tobias S. Finn Rui Fu ... Yong Zeng Mingrui Zhang Hao Zhou Kewei Zhu Rossella Arcucci PINN AI4CE 114 0 0 13 Feb 2025
Towards Foundational Models for Dynamical System Reconstruction: Hierarchical Meta-Learning via Mixture of Experts Roussel Desmond Nzoyem David A.W. Barton Tom Deakin 72 1 0 07 Feb 2025
Neural equilibria for long-term prediction of nonlinear conservation laws Jose Antonio Lara Benitez Junyi Guo Kareem Hegazy Ivan Dokmanić Michael W. Mahoney Maarten V. de Hoop 38 0 0 12 Jan 2025
Data-Efficient Inference of Neural Fluid Fields via SciML Foundation Model Yuqiu Liu Jingxuan Xu Mauricio Soroco Yunchao Wei Wuyang Chen AI4CE 84 2 0 18 Dec 2024
The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning Ruben Ohana Michael McCabe Lucas Meyer Rudy Morel Fruzsina J. Agocs ... François Rozet Liam Parker M. Cranmer S. Ho Shirley Ho PINN AI4CE 72 7 1 30 Nov 2024
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts R. Teo Tan M. Nguyen MoE 33 3 0 18 Oct 2024
Model Balancing Helps Low-data Training and Fine-tuning Zihang Liu Yihan Hu Tianyu Pang Yefan Zhou Pu Ren Yaoqing Yang 36 2 0 16 Oct 2024
MaD-Scientist: AI-based Scientist solving Convection-Diffusion-Reaction Equations Using Massive PINN-Based Prior Data Mingu Kang Dongseok Lee Woojin Cho Jaehyeon Park Kookjin Lee Anthony Gruber Youngjoon Hong Noseong Park DiffM AI4CE 34 0 0 09 Oct 2024
Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs Louis Serrano Armand K. Koupai Thomas X. Wang Pierre Erbacher Patrick Gallinari AI4CE 35 3 0 04 Oct 2024
Scalable and Consistent Graph Neural Networks for Distributed Mesh-based Data-driven Modeling Shivam Barwey Riccardo Balin Bethany Lusch Saumil Patel Ramesh Balakrishnan Pinaki Pal R. Maulik V. Vishwanath GNN AI4CE 28 1 0 02 Oct 2024
Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study Hao Liu Zecheng Zhang Wenjing Liao Hayden Schaeffer 25 1 0 01 Oct 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024
Provable In-Context Learning of Linear Systems and Linear Elliptic PDEs with Transformers Frank Cole Yulong Lu Riley OÑeill Tianhao Zhang 45 2 0 18 Sep 2024
LeMON: Learning to Learn Multi-Operator Networks Jingmin Sun Zecheng Zhang Hayden Schaeffer 38 6 0 28 Aug 2024
Self-supervised Pretraining for Partial Differential Equations Varun Madhavan Amal S Sebastian Bharath Ramsundar Venkatasubramanian Viswanathan AI4CE 33 0 0 03 Jul 2024
DeltaPhi: Learning Physical Trajectory Residual for PDE Solving Xihang Yue Linchao Zhu Yi Yang AI4CE 34 1 0 14 Jun 2024
Strategies for Pretraining Neural Operators Anthony Y. Zhou Cooper Lorsung AmirPouya Hemmasian Amir Barati Farimani AI4CE 39 5 0 12 Jun 2024
Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers Zhou Hang Yuezhou Ma Haixu Wu Haowen Wang Mingsheng Long AI4CE 36 9 0 27 May 2024
FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model Zezheng Song Jiaxin Yuan Haizhao Yang AI4CE 40 17 0 23 Apr 2024
MODNO: Multi Operator Learning With Distributed Neural Operators Zecheng Zhang 40 6 0 03 Apr 2024
Masked Autoencoders are PDE Learners Anthony Y. Zhou A. Farimani AI4CE 38 6 0 26 Mar 2024
Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs Md Ashiqur Rahman Robert Joseph George Mogab Elleithy Daniel Leibovici Zong-Yi Li ... Julius Berner Raymond A. Yeh Jean Kossaifi Kamyar Azizzadenesheli A. Anandkumar AI4CE 54 20 0 19 Mar 2024
Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs S. C. Mouli Danielle C. Maddix S. Alizadeh Gaurav Gupta Andrew Stuart Michael W. Mahoney Yuyang Wang UQCV AI4CE 40 2 0 15 Mar 2024
PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models S. Mishra-Sharma Yiding Song Jesse Thaler 31 7 0 13 Mar 2024
UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptation Junhong Shen Tanya Marwah Ameet Talwalkar AI4CE 44 4 0 11 Mar 2024
Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models Philip Harris Michael Kagan J. Krupa B. Maier Nathaniel Woodward 38 13 0 11 Mar 2024
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training Zhongkai Hao Chang Su Songming Liu Julius Berner Chengyang Ying Hang Su A. Anandkumar Jian Song Jun Zhu AI4TS AI4CE 24 22 0 06 Mar 2024
Building Flexible Machine Learning Models for Scientific Computing at Scale Tianyu Chen Haoyi Zhou Ying Li Hao Wang Chonghan Gao Shanghang Zhang Jianxin Li AI4CE 32 0 0 25 Feb 2024
Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning Wuyang Chen Jialin Song Pu Ren Shashank Subramanian Dmitriy Morozov Michael W. Mahoney AI4CE 52 9 0 24 Feb 2024
A novel Fourier neural operator framework for classification of multi-sized images: Application to three dimensional digital porous media Ali Kashefi T. Mukerji AI4CE 38 4 0 18 Feb 2024
PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws Liu Yang Stanley J. Osher AI4CE 48 18 0 14 Jan 2024
Generating synthetic data for neural operators Erisa Hasani Rachel A. Ward AI4CE 47 8 0 04 Jan 2024
Operator learning for hyperbolic partial differential equations Christopher Wang Alex Townsend 34 2 0 29 Dec 2023
Multiple Physics Pretraining for Physical Surrogate Models Michael McCabe Bruno Régaldo-Saint Blancard Liam Parker Ruben Ohana M. Cranmer ... Francois Lanusse Mariel Pettee Tiberiu Teşileanu Kyunghyun Cho Shirley Ho PINN AI4CE 34 52 0 04 Oct 2023
Neural Operators for Accelerating Scientific Simulations and Design Kamyar Azzizadenesheli Nikola B. Kovachki Zong-Yi Li Miguel Liu-Schiaffini Jean Kossaifi Anima Anandkumar AI4CE 35 122 0 27 Sep 2023
Fine-Tune Language Models as Multi-Modal Differential Equation Solvers Liu Yang Siting Liu Stanley J. Osher 21 0 0 09 Aug 2023
Artificial General Intelligence for Medical Imaging Xiang Li Lu Zhang Zihao Wu Zheng Liu Lin Zhao ... Pingkuan Yan Quanzheng Li Wei Liu Tianming Liu Dinggang Shen LM&MA AI4CE 19 40 0 08 Jun 2023
CONFIDE: Contextual Finite Differences Modelling of PDEs Ori Linial Orly Avner Dotan Di Castro 41 0 0 28 Mar 2023
Fourier Neural Operator with Learned Deformations for PDEs on General Geometries Zong-Yi Li Daniel Zhengyu Huang Burigede Liu Anima Anandkumar AI4CE 119 251 0 11 Jul 2022
Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains Hengjie Wang R. Planas Aparna Chandramowlishwaran Ramin Bostanabad AI4CE 50 61 0 22 Apr 2021