Evaluating Distributional Distortion in Neural Language Modeling

24 March 2022

Papers citing "Evaluating Distributional Distortion in Neural Language Modeling"

16 / 16 papers shown

Title
LLM as a Broken Telephone: Iterative Generation Distorts Information Amr Mohamed Mingmeng Geng Michalis Vazirgiannis Guokan Shang 69 1 0 27 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 115 3 0 06 Feb 2025
Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory Aymane El Firdoussi M. Seddik Soufiane Hayou Réda Alami Ahmed Alzubaidi Hakim Hacid 28 1 0 11 Oct 2024
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Reinforcement Yunzhen Feng Elvis Dohmatob Pu Yang Francois Charton Julia Kempe 53 17 0 11 Jun 2024
ModelShield: Adaptive and Robust Watermark against Model Extraction Attack Kaiyi Pang Tao Qi Chuhan Wu Minhao Bai Minghu Jiang Yongfeng Huang AAML WaLM 70 2 0 03 May 2024
Predict the Next Word: Humans exhibit uncertainty in this task and language models _____ Evgenia Ilia Wilker Aziz 29 2 0 27 Feb 2024
A Tale of Tails: Model Collapse as a Change of Scaling Laws Elvis Dohmatob Yunzhen Feng Pu Yang Francois Charton Julia Kempe 21 62 0 10 Feb 2024
On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translation Anssi Moisio Mathias Creutz M. Kurimo CoGe 12 1 0 14 Nov 2023
EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling Siyu Ren Zhiyong Wu Kenny Q. Zhu 26 3 0 07 Oct 2023
Tailoring Language Generation Models under Total Variation Distance Haozhe Ji Pei Ke Zhipeng Hu Rongsheng Zhang Minlie Huang 28 18 0 26 Feb 2023
Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning Xinyi Wang Wanrong Zhu Michael Stephen Saxon Mark Steyvers William Yang Wang BDL 43 91 0 27 Jan 2023
Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification Zi Lin J. Liu Jingbo Shang UQLM 15 5 0 26 Jan 2023
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 37 27 0 20 Sep 2022
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN R. Thomas McCoy P. Smolensky Tal Linzen Jianfeng Gao Asli Celikyilmaz SyDa 19 119 0 18 Nov 2021
On Language Models for Creoles Heather Lent Emanuele Bugliarello Miryam de Lhoneux Chen Qiu Anders Søgaard 32 20 0 13 Sep 2021
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 146 370 0 09 May 2020