On "Scientific Debt" in NLP: A Case for More Rigour in Language Model
Pre-Training Research

On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research

5 June 2023

Made Nindyatama Nityasya

Haryo Akbarianto Wibowo

Alham Fikri Aji

Genta Indra Winata

Radityo Eko Prasojo

Papers citing "On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research"

13 / 13 papers shown

Title
On the Rigour of Scientific Writing: Criteria, Analysis, and Insights Joseph James Chenghao Xiao Yucheng Li Chenghua Lin 20 1 0 07 Oct 2024
Systematic Task Exploration with LLMs: A Study in Citation Text Generation Furkan Şahinuç Ilia Kuznetsov Yufang Hou Iryna Gurevych 27 3 0 04 Jul 2024
Sequential Compositional Generalization in Multimodal Models Semih Yagcioglu Osman Batur .Ince Aykut Erdem Erkut Erdem Desmond Elliott Deniz Yuret 34 1 0 18 Apr 2024
A Controlled Reevaluation of Coreference Resolution Models Ian Porada Xiyuan Zou Jackie Chi Kit Cheung 28 1 0 31 Mar 2024
Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models Chenyang Lyu Minghao Wu Alham Fikri Aji ELM 38 13 0 21 Feb 2024
Revisiting Topic-Guided Language Models Carolina Zheng Keyon Vafa David M. Blei BDL 17 1 0 04 Dec 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 239 2,232 0 22 Mar 2023
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 237 588 0 14 Jul 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 239 642 0 21 Apr 2021
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? Tal Linzen 216 188 0 03 May 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,815 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018