A Position Paper on the Automatic Generation of Machine Learning Leaderboards

23 May 2025

Papers citing "A Position Paper on the Automatic Generation of Machine Learning Leaderboards"

25 / 25 papers shown

Title
Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboards Furkan Şahinuç Thy Thy Tran Yulia Grishina Yufang Hou Bei Chen Iryna Gurevych 62 7 0 19 Sep 2024
Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets Satanu Ghosh Neal R. Brodnik Carolina Frey Collin Holgate Tresa M. Pollock Samantha Daly Samuel Carton 43 2 0 08 Jun 2024
Effective Context Selection in LLM-based Leaderboard Generation: An Empirical Study Salomon Kabongo Jennifer D'Souza Sören Auer 47 4 0 06 Jun 2024
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 126 420 0 28 Nov 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 206 11,636 0 18 Jul 2023
DMDD: A Large-Scale Dataset for Dataset Mentions Detection Huitong Pan Qi Zhang Eduard Constantin Dragut Cornelia Caragea Longin Jan Latecki 28 11 0 19 May 2023
ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graph Salomon Kabongo KABENAMUALU Jennifer D'Souza Sören Auer 74 19 0 10 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 631 13,788 0 15 Mar 2023
Galactica: A Large Language Model for Science Ross Taylor Marcin Kardas Guillem Cucurull Thomas Scialom Anthony Hartshorn Elvis Saravia Andrew Poulton Viktor Kerkez Robert Stojnic ELM ReLM 88 754 0 16 Nov 2022
Automated Mining of Leaderboards for Empirical AI Research Salomon Kabongo KABENAMUALU Jennifer D'Souza Sören Auer 60 30 0 31 Aug 2021
A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation Sebastin Santy Prasanta Bhattacharya LLMAG 50 3 0 11 Jun 2021
TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics Yufang Hou Charles Jochim Martin Gleize Francesca Bonin Debasis Ganguly 48 46 0 25 Jan 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 484 2,051 0 28 Jul 2020
SciREX: A Challenge Dataset for Document-Level Information Extraction Sarthak Jain Madeleine van Zuylen Hannaneh Hajishirzi Iz Beltagy 37 163 0 01 May 2020
AxCell: Automatic Extraction of Results from Machine Learning Papers Marcin Kardas Piotr Czapla Pontus Stenetorp Sebastian Ruder Sebastian Riedel Ross Taylor Robert Stojnic 20 75 0 29 Apr 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 141 10,720 0 29 Oct 2019
Span-based Joint Entity and Relation Extraction with Transformer Pre-training Markus Eberts A. Ulges LRM ViT 184 382 0 17 Sep 2019
Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction Yufang Hou Charles Jochim Martin Gleize Francesca Bonin Debasis Ganguly 23 92 0 21 Jun 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 183 8,386 0 19 Jun 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 152 1,553 0 08 May 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 81 2,948 0 26 Mar 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 961 93,936 0 11 Oct 2018
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction Yi Luan Luheng He Mari Ostendorf Hannaneh Hajishirzi 91 674 0 29 Aug 2018
SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications Isabelle Augenstein Mrinal Das Sebastian Riedel Lakshmi Vikraman Andrew McCallum 45 337 0 10 Apr 2017
You Only Look Once: Unified, Real-Time Object Detection Joseph Redmon S. Divvala Ross B. Girshick Ali Farhadi ObjD 568 36,643 0 08 Jun 2015