Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities

27 April 2024

Papers citing "Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities"

39 / 39 papers shown

Title
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code Kazuki Fujii Yukito Tajima Sakae Mizuki Hinari Shimada Taihei Shiotani ... Kakeru Hattori Youmi Ma Hiroya Takamura Rio Yokota Naoaki Okazaki SyDa 45 0 0 05 May 2025
Towards Robust Dialogue Breakdown Detection: Addressing Disruptors in Large Language Models with Self-Guided Reasoning Abdellah Ghassel Xianzhi Li Xiaodan Zhu 43 0 0 26 Apr 2025
Application of Contrastive Learning on ECG Data: Evaluating Performance in Japanese and Classification with Around 100 Labels Junichiro Takahashi JingChuan Guan Masataka Sato Kaito Baba Kazuto Haruguchi Daichi Nagashima Satoshi Kodera Norihiko Takeda 23 0 0 12 Apr 2025
Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources Zihao Li Shaoxiong Ji Hengyu Luo Jörg Tiedemann CLL 69 0 0 05 Apr 2025
On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation Jirui Qi Raquel Fernández Arianna Bisazza RALM 56 0 0 01 Apr 2025
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models Youmi Ma Sakae Mizuki Kazuki Fujii Taishi Nakamura Masanari Ohi ... Takumi Okamoto Shigeki Ishida Rio Yokota Hiroya Takamura Naoaki Okazaki ALM 52 0 0 31 Mar 2025
Analyzing the Safety of Japanese Large Language Models in Stereotype-Triggering Prompts Akito Nakanishi Yukie Sano Geng Liu Francesco Pierri 55 0 0 03 Mar 2025
Causal Tree Extraction from Medical Case Reports: A Novel Task for Experts-like Text Comprehension Sakiko Yahata Zhen Wan Fei Cheng Sadao Kurohashi Hisahiko Sato Ryozo Nagai CML 54 0 0 03 Mar 2025
R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning Minggui He Yilun Liu Shimin Tao Yuanchang Luo Hongyong Zeng ... Daimeng Wei Weibin Meng Hao Yang Boxing Chen Osamu Yoshie LRM 61 2 0 27 Feb 2025
Do Multilingual LLMs Think In English? Lisa Schut Y. Gal Sebastian Farquhar 40 3 0 24 Feb 2025
MergePrint: Merge-Resistant Fingerprints for Robust Black-box Ownership Verification of Large Language Models Shojiro Yamabe Tsubasa Takahashi Futa Waseda Koki Wataoka MoMe 81 1 0 21 Feb 2025
Efficient Continual Pre-training of LLMs for Low-resource Languages Arijit Nag Soumen Chakrabarti Animesh Mukherjee Niloy Ganguly 72 0 0 13 Dec 2024
Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model Divyanshu Aggarwal Sankarshan Damle Navin Goyal Satya Lokam Sunayana Sitaram CLL 18 0 0 21 Oct 2024
What's New in My Data? Novelty Exploration via Contrastive Generation Masaru Isonuma Ivan Titov 26 0 0 18 Oct 2024
Optimizing Low-Resource Language Model Training: Comprehensive Analysis of Multi-Epoch, Multi-Lingual, and Two-Stage Approaches Kosuke Akimoto M. Oyamada 26 0 0 16 Oct 2024
JMedBench: A Benchmark for Evaluating Japanese Biomedical Large Language Models Junfeng Jiang Jiahao Huang Akiko Aizawa LM&MA 21 4 0 20 Sep 2024
Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources Issey Sukeda ELM 32 1 0 18 Sep 2024
Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough Konstantin Dobler Gerard de Melo 37 1 0 28 Aug 2024
Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation Vivek Iyer Bhavitvya Malik Pavel Stepachev Pinzhen Chen Barry Haddow Alexandra Birch ALM 29 3 0 23 Aug 2024
Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? Chengzhi Zhong Fei Cheng Qianying Liu Junfeng Jiang Zhen Wan Chenhui Chu Yugo Murawaki Sadao Kurohashi LRM 34 11 0 20 Aug 2024
Enhancing Large Language Model-based Speech Recognition by Contextualization for Rare and Ambiguous Words Kento Nozawa Takashi Masuko Toru Taniguchi 38 1 0 15 Aug 2024
Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset Kentaro Ozeki Risako Ando Takanobu Morishita Hirohiko Abe K. Mineshima Mitsuhiro Okada LRM 14 2 0 08 Aug 2024
Robust ASR Error Correction with Conservative Data Filtering Takuma Udagawa Masayuki Suzuki Masayasu Muraoka Gakuto Kurata 48 0 0 18 Jul 2024
Bilingual Adaptation of Monolingual Foundation Models Gurpreet Gosal Yishi Xu Gokul Ramakrishnan Rituraj Joshi Avraham Sheinin ... Rahul Pal Parvez Mullah Soundar Doraiswamy Mohamed El Karim Chami Preslav Nakov CLL 21 3 0 13 Jul 2024
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages Yinquan Lu Wenhao Zhu Lei Li Yu Qiao Fei Yuan 42 24 0 08 Jul 2024
Large Vocabulary Size Improves Large Language Models Sho Takase Ryokan Ri Shun Kiyono Takuya Kato 32 3 0 24 Jun 2024
Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models? Yuu Jinnai 47 1 0 24 Jun 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages Fabian David Schmidt Philipp Borchert Ivan Vulić Goran Glavaš 42 5 0 18 Jun 2024
Analyzing Social Biases in Japanese Large Language Models Hitomi Yanaka Namgi Han Ryoma Kumon Jie Lu Masashi Takeshita Ryo Sekizawa Taisei Kato Hiromi Arai 40 3 0 04 Jun 2024
InstructionCP: A fast approach to transfer Large Language Models into target language Kuang-Ming Chen Hung-yi Lee CLL 36 2 0 30 May 2024
Evaluating the External and Parametric Knowledge Fusion of Large Language Models Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu ... Yasheng Wang Lifeng Shang Qun Liu Yong-jin Liu Ruiming Tang KELM 33 4 0 29 May 2024
The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights Wenhao Zhu Shujian Huang Fei Yuan Cheng Chen Jiajun Chen Alexandra Birch LRM 44 5 0 02 May 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 37 62 0 25 Apr 2024
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation Ikuya Yamada Ryokan Ri KELM 8 0 0 18 Feb 2024
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference Atsuki Yamaguchi Aline Villavicencio Nikolaos Aletras 19 7 0 16 Feb 2024
Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models Tomoki Sugimoto Yasumasa Onoe Hitomi Yanaka 38 4 0 19 Jun 2023
Language Models are Multilingual Chain-of-Thought Reasoners Freda Shi Mirac Suzgun Markus Freitag Xuezhi Wang Suraj Srivats ... Yi Tay Sebastian Ruder Denny Zhou Dipanjan Das Jason W. Wei ReLM LRM 170 324 0 06 Oct 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 134 116 0 24 May 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,986 0 31 Dec 2020