Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2002.06716
Cited By
v1
v2 (latest)
Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data
Nature Communications (Nat Commun), 2020
17 February 2020
Charles H. Martin
Tongsu Peng
Peng
Michael W. Mahoney
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data"
50 / 77 papers shown
Using Synthetic Data to estimate the True Error is theoretically and practically doable
Hai Hoang Thanh
Duy-Tung Nguyen
Hung The Tran
Khoat Than
146
0
0
02 Nov 2025
Early-stopping for Transformer model training
Jing He
Hua Jiang
Cheng Li
Siqian Xin
Shuzhen Yang
173
0
0
17 Oct 2025
Learning Model Representations Using Publicly Available Model Hubs
Damian Falk
Konstantin Schürholt
Konstantinos Tzevelekakis
Léo Meynent
Damian Borth
3DH
278
2
0
02 Oct 2025
Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
Fabrizio Boncoraglio
Vittorio Erba
Emanuele Troiani
Florent Krzakala
Lenka Zdeborová
Lenka Zdeborová
210
0
0
29 Sep 2025
Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
Leonardo Defilippis
Yizhou Xu
Julius Girardin
Emanuele Troiani
Vittorio Erba
Lenka Zdeborová
Bruno Loureiro
Florent Krzakala
182
7
0
29 Sep 2025
Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning
Zedong Wang
Siyuan Li
Dan Xu
208
3
0
28 Jul 2025
RADIANT: Retrieval AugmenteD entIty-context AligNmenT -- Introducing RAG-ability and Entity-Context Divergence
Vipula Rawte
Rajarshi Roy
Gurpreet Singh
Danush Khanna
Yaswanth Narsupalli
...
Aadi Krishna Vikram
Vinija Jain
Vasu Sharma
Amit P. Sheth
Amitava Das
RALM
232
1
0
28 Jun 2025
Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation
Hao Guan
D. Bates
Li Zhou
OOD
274
5
0
20 Jun 2025
AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
Di He
Ajay Jaiswal
Songjun Tu
Li Shen
Ganzhao Yuan
Shiwei Liu
L. Yin
435
3
0
17 Jun 2025
Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
Yuanzhe Hu
Kinshuk Goel
Vlad Killiakov
Yaoqing Yang
512
5
0
06 Jun 2025
Models of Heavy-Tailed Mechanistic Universality
Liam Hodgkinson
Zhichao Wang
Michael W. Mahoney
302
7
0
04 Jun 2025
LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
Zihang Liu
Tianyu Pang
Oleg Balabanov
Chaoqun Yang
Tianjin Huang
L. Yin
Yaoqing Yang
Shiwei Liu
LRM
323
8
0
01 Jun 2025
Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling
Hojun Son
Asma Almutairi
Arpan Kusari
419
0
0
24 May 2025
A Model Zoo on Phase Transitions in Neural Networks
Konstantin Schurholt
Léo Meynent
Yefan Zhou
Haiquan Lu
Yaoqing Yang
Damian Borth
450
3
0
25 Apr 2025
The Impact of Model Zoo Size and Composition on Weight Space Learning
Damian Falk
Konstantin Schurholt
Damian Borth
447
1
0
14 Apr 2025
A Model Zoo of Vision Transformers
Damian Falk
Léo Meynent
Florence Pfammatter
Konstantin Schurholt
Damian Borth
558
3
0
14 Apr 2025
Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performance
Nirvan Patil
Malhar Abhay Inamdar
Agnivo Gosai
Guruprasad Pathak
Anish Joshi
Aryan Sagavekar
Anish Joshirao
Raj Abhijit Dandekar
Rajat Dandekar
Sreedath Panat
460
1
0
07 Apr 2025
An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainability
Frontiers in Artificial Intelligence (Front. Artif. Intell.), 2025
David Herrera-Poyatos
Carlos Peláez-González
Cristina Zuheros
Andrés Herrera-Poyatos
Virilo Tejedor
F. Herrera
Rosana Montes
348
26
0
06 Apr 2025
ZeroLM: Data-Free Transformer Architecture Search for Language Models
Zhen-Song Chen
Hong-Wei Ding
Xian-Jia Wang
Witold Pedrycz
401
1
0
24 Mar 2025
Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction
Léo Meynent
Ivan Melev
Konstantin Schurholt
Göran Kauermann
Damian Borth
449
6
0
21 Mar 2025
Can We Optimize Deep RL Policy Weights as Trajectory Modeling?
Hongyao Tang
OffRL
450
0
0
06 Mar 2025
Using Pre-trained LLMs for Multivariate Time Series Forecasting
Malcolm Wolff
Shenghao Yang
Kari Torkkola
Michael W. Mahoney
AI4TS
AIFin
292
6
0
10 Jan 2025
LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics
IEEE Computer Graphics and Applications (IEEE CG&A), 2024
Tiankai Xie
Jiaqing Chen
Yaoqing Yang
Caleb Geniesse
Ge Shi
...
J. Cava
Michael W. Mahoney
Talita Perciano
Gunther H. Weber
Ross Maciejewski
353
2
0
17 Dec 2024
Visualizing Loss Functions as Topological Landscape Profiles
Caleb Geniesse
Jiaqing Chen
Tiankai Xie
Ge Shi
Yaoqing Yang
Dmitriy Morozov
Talita Perciano
Michael W. Mahoney
Ross Maciejewski
Gunther H. Weber
279
2
0
19 Nov 2024
Evaluating Loss Landscapes from a Topology Perspective
Tiankai Xie
Caleb Geniesse
Jiaqing Chen
Yaoqing Yang
Dmitriy Morozov
Michael W. Mahoney
Ross Maciejewski
Gunther H. Weber
311
3
0
14 Nov 2024
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities
International Conference on Artificial Intelligence and Statistics (AISTATS), 2024
Yatin Dandi
Luca Pesce
Hugo Cui
Florent Krzakala
Yue M. Lu
Bruno Loureiro
MLT
389
12
0
24 Oct 2024
Data Diversity as Implicit Regularization: How Does Diversity Shape the Weight Space of Deep Neural Networks?
Yang Ba
M. Mancenido
Rong Pan
341
0
0
18 Oct 2024
Model Balancing Helps Low-data Training and Fine-tuning
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Zihang Liu
Yihan Hu
Tianyu Pang
Yefan Zhou
Pu Ren
Yaoqing Yang
259
10
0
16 Oct 2024
AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
Neural Information Processing Systems (NeurIPS), 2024
Haiquan Lu
Yefan Zhou
Shiwei Liu
Zhangyang Wang
Michael W. Mahoney
Yaoqing Yang
185
33
0
14 Oct 2024
AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Peijun Qing
Chongyang Gao
Yefan Zhou
Xingjian Diao
Yaoqing Yang
Soroush Vosoughi
MoMe
MoE
331
20
0
14 Oct 2024
Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning
Siyuan Li
Juanxi Tian
Zedong Wang
Luyuan Zhang
Zicheng Liu
Weiyang Jin
Yang Liu
Baigui Sun
Stan Z. Li
270
2
0
08 Oct 2024
Mitigating Memorization In Language Models
Mansi Sakarvadia
Aswathy Ajith
Arham Khan
Nathaniel Hudson
Caleb Geniesse
Kyle Chard
Yaoqing Yang
Ian Foster
Michael W. Mahoney
KELM
MU
468
11
0
03 Oct 2024
MD tree: a model-diagnostic tree grown on loss landscape
Yefan Zhou
Jianlong Chen
Qinxue Cao
Konstantin Schürholt
Yaoqing Yang
395
2
0
24 Jun 2024
Towards Scalable and Versatile Weight Space Learning
International Conference on Machine Learning (ICML), 2024
Konstantin Schurholt
Michael W. Mahoney
Damian Borth
343
38
0
14 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models
Jack Merullo
Carsten Eickhoff
Ellie Pavlick
677
40
0
13 Jun 2024
From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
Vignesh Kothapalli
Tianyu Pang
Shenyang Deng
Zongmin Liu
Yaoqing Yang
462
4
0
07 Jun 2024
Transferable and Principled Efficiency for Open-Vocabulary Segmentation
Jingxuan Xu
Wuyang Chen
Yao-Min Zhao
Yunchao Wei
VLM
353
2
0
11 Apr 2024
Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance
Chiraag Kaushik
Ran Liu
Chi-Heng Lin
Amrit Khera
Matthew Y Jin
Wenrui Ma
Vidya Muthukumar
Eva L. Dyer
313
4
0
18 Feb 2024
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
Neural Information Processing Systems (NeurIPS), 2023
Yefan Zhou
Tianyu Pang
Keqin Liu
Charles H. Martin
Michael W. Mahoney
Yaoqing Yang
487
18
0
01 Dec 2023
Can we infer the presence of Differential Privacy in Deep Learning models' weights? Towards more secure Deep Learning
Daniel Jiménez-López
Daniel
Nuria Rodríguez Barroso
Nuria
M. V. Luzón
M. Victoria
Francisco Herrera
Francisco
AAML
217
0
0
20 Nov 2023
A PAC-Bayesian Perspective on the Interpolating Information Criterion
Liam Hodgkinson
Christopher van der Heide
Roberto Salomone
Fred Roosta
Michael W. Mahoney
334
3
0
13 Nov 2023
Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities
BigData Congress [Services Society] (BSS), 2023
Leman Akoglu
Jaemin Yoo
209
3
0
28 Aug 2023
To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks
Rajan Sahu
Shivam Chadha
N. Nagaraj
A. Mathur
Snehanshu Saha
295
1
0
19 Aug 2023
Fast Unsupervised Deep Outlier Model Selection with Hypernetworks
Knowledge Discovery and Data Mining (KDD), 2023
Xueying Ding
Yue Zhao
Leman Akoglu
OODD
314
11
0
20 Jul 2023
The Interpolating Information Criterion for Overparameterized Models
Liam Hodgkinson
Christopher van der Heide
Roberto Salomone
Fred Roosta
Michael W. Mahoney
317
11
0
15 Jul 2023
Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices
IEEE Access (IEEE Access), 2023
M. Ferrag
Mthandazo Ndhlovu
Norbert Tihanyi
Lucas C. Cordeiro
Merouane Debbah
Thierry Lestable
Narinderjit Singh Thandi
229
191
0
25 Jun 2023
Revealing Model Biases: Assessing Deep Neural Networks via Recovered Sample Analysis
M. Mehmanchi
Mahbod Nouri
Mohammad Sabokrou
AAML
307
1
0
10 Jun 2023
A Three-regime Model of Network Pruning
International Conference on Machine Learning (ICML), 2023
Yefan Zhou
Yaoqing Yang
Arin Chang
Michael W. Mahoney
272
15
0
28 May 2023
Heavy-Tailed Regularization of Weight Matrices in Deep Neural Networks
International Conference on Artificial Neural Networks (ICANN), 2023
Xuanzhe Xiao
Zengyi Li
Chuanlong Xie
Fengwei Zhou
375
7
0
06 Apr 2023
Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting
Yitzchak Shmalo
Jonathan Jenkins
Oleksii Krupchytskyi
236
5
0
15 Mar 2023
1
2
Next
Page 1 of 2
