Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models

19 March 2022

Papers citing "Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models"

22 / 22 papers shown

Title
Can Language Models Induce Grammatical Knowledge from Indirect Evidence? Miyu Oba Yohei Oseki Akiyo Fukatsu Akari Haga Hiroki Ouchi Taro Watanabe Saku Sugawara 24 1 0 08 Oct 2024
Measuring Cross-lingual Transfer in Bytes Leandro Rodrigues de Souza Thales Sales Almeida R.A. Lotufo Rodrigo Nogueira CLL 19 3 0 12 Apr 2024
Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification Anirudh S. Sundar Chao-Han Huck Yang David M. Chan Shalini Ghosh Venkatesh Ravichandran P. S. Nidadavolu MoMe 28 8 0 22 Dec 2023
Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance? Ahmed Alajrami Katerina Margatina Nikolaos Aletras AAML 11 1 0 26 Oct 2023
Pre-training with Synthetic Data Helps Offline Reinforcement Learning Zecheng Wang Che Wang Zixuan Dong Keith Ross OffRL 22 5 0 01 Oct 2023
Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Ability Jiacheng Ye Xijia Tao Lingpeng Kong LRM 22 22 0 11 Jun 2023
Second Language Acquisition of Neural Language Models Miyu Oba Tatsuki Kuribayashi Hiroki Ouchi Taro Watanabe 13 3 0 05 Jun 2023
To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis Fuzhao Xue Yao Fu Wangchunshu Zhou Zangwei Zheng Yang You 79 74 0 22 May 2023
Gloss-Free End-to-End Sign Language Translation Kezhou Lin Xiaohan Wang Linchao Zhu Ke Sun Bang Zhang Yezhou Yang SLR 14 17 0 22 May 2023
Injecting structural hints: Using language models to study inductive biases in language learning Isabel Papadimitriou Dan Jurafsky 14 12 0 25 Apr 2023
Does Deep Learning Learn to Abstract? A Systematic Probing Framework Shengnan An Zeqi Lin B. Chen Qiang Fu Nanning Zheng Jian-Guang Lou 23 4 0 23 Feb 2023
A Survey of Deep Learning for Mathematical Reasoning Pan Lu Liang Qiu Wenhao Yu Sean Welleck Kai-Wei Chang ReLM LRM 32 137 0 20 Dec 2022
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning S. Takagi OffRL 8 7 0 17 Nov 2022
Intermediate Fine-Tuning Using Imperfect Synthetic Speech for Improving Electrolaryngeal Speech Recognition Lester Phillip Violeta D. Ma Wen-Chin Huang T. Toda 11 7 0 02 Nov 2022
Robustness of Demonstration-based Learning Under Limited Data Scenario Hongxin Zhang Yanzhe Zhang Ruiyi Zhang Diyi Yang 27 13 0 19 Oct 2022
MonoByte: A Pool of Monolingual Byte-level Language Models Hugo Queiroz Abonizio Leandro Rodrigues de Souza R. Lotufo Rodrigo Nogueira 23 1 0 22 Sep 2022
Insights into Pre-training via Simpler Synthetic Tasks Yuhuai Wu Felix Li Percy Liang AIMat 20 20 0 21 Jun 2022
DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering Guan-Ting Lin Yung-Sung Chuang Ho-Lam Chung Shu-Wen Yang Hsuan-Jui Chen Shuyan Dong Shang-Wen Li Abdel-rahman Mohamed Hung-yi Lee Lin-Shan Lee 20 21 0 09 Mar 2022
Can Wikipedia Help Offline Reinforcement Learning? Machel Reid Yutaro Yamada S. Gu 3DV RALM OffRL 124 95 0 28 Jan 2022
UnNatural Language Inference Koustuv Sinha Prasanna Parthasarathi Joelle Pineau Adina Williams 203 80 0 30 Dec 2020
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 876 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018