An Emulator for Fine-Tuning Large Language Models using Small Language Models

19 October 2023

Christopher D. Manning

ALM

ArXiv PDF HTML

Papers citing "An Emulator for Fine-Tuning Large Language Models using Small Language Models"

47 / 47 papers shown

Title
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Janet Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Y. Zhang ALM 51 0 0 24 Apr 2025
Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions Chaoyue Niu Yucheng Ding Junhui Lu Zhengxiang Huang Hang Zeng Yutong Dai Xuezhen Tu Chengfei Lv Fan Wu Guihai Chen 27 0 0 17 Apr 2025
A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models Zhouhang Xie Junda Wu Yiran Shen Yu Xia Xintong Li ... Sachin Kumar Bodhisattwa Prasad Majumder Jingbo Shang Prithviraj Ammanabrolu Julian McAuley 31 0 0 09 Apr 2025
Bridging the Gap Between Preference Alignment and Machine Unlearning Xiaohua Feng Yuyuan Li Huwei Ji Jiaming Zhang L. Zhang Tianyu Du Chaochao Chen MU 38 0 0 09 Apr 2025
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs Weixiang Zhao Yulin Hu Yang Deng Jiahe Guo Xingyu Sui ... An Zhang Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu 49 1 0 28 Feb 2025
Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMs Zhaowei Zhang Fengshuo Bai Qizhi Chen Chengdong Ma Mingzhi Wang Haoran Sun Zilong Zheng Yaodong Yang 47 3 0 26 Feb 2025
Can ChatGPT Learn to Count Letters? Javier Conde Gonzalo Martínez Pedro Reviriego Zhen Gao Shanshan Liu Fabrizio Lombardi 31 1 0 23 Feb 2025
Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models Daiki Chijiwa Taku Hasegawa Kyosuke Nishida Kuniko Saito Susumu Takeuchi 34 0 0 18 Feb 2025
ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantation Weilong Dong Xinwei Wu Renren Jin Shaoyang Xu Deyi Xiong 47 6 0 31 Dec 2024
A Theoretical Perspective for Speculative Decoding Algorithm Ming Yin Minshuo Chen Kaixuan Huang Mengdi Wang 27 0 0 30 Oct 2024
Transferable Post-training via Inverse Value Learning Xinyu Lu Xueru Wen Y. Lu Bowen Yu Hongyu Lin Haiyang Yu Le Sun Xianpei Han Yongbin Li 15 1 0 28 Oct 2024
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs A. S. Rawat Veeranjaneyulu Sadhanala Afshin Rostamizadeh Ayan Chakrabarti Wittawat Jitkrittum ... Rakesh Shivanna Sashank J. Reddi A. Menon Rohan Anil Sanjiv Kumar 13 2 0 24 Oct 2024
Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model Wenhong Zhu Zhiwei He Xiaofeng Wang Pengfei Liu Rui Wang OSLM 45 3 0 24 Oct 2024
Cross-model Control: Improving Multiple Large Language Models in One-time Training Jiayi Wu Hao-Lun Sun Hengyi Cai Lixin Su S. Wang Dawei Yin Xiang Li Ming Gao MU 24 0 0 23 Oct 2024
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment Qin Liu Fei Wang Chaowei Xiao Muhao Chen 33 0 0 18 Oct 2024
GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment Yuancheng Xu Udari Madhushani Sehwag Alec Koppel Sicheng Zhu Bang An Furong Huang Sumitra Ganesh 40 5 0 10 Oct 2024
Erasing Conceptual Knowledge from Language Models Rohit Gandikota Sheridan Feucht Samuel Marks David Bau KELM ELM MU 40 5 0 03 Oct 2024
Inference-Time Language Model Alignment via Integrated Value Guidance Zhixuan Liu Zhanhui Zhou Yuanfu Wang Chao Yang Yu Qiao 24 7 0 26 Sep 2024
CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration Jiahui Gao Renjie Pi Tianyang Han Han Wu Lanqing Hong Lingpeng Kong Xin Jiang Zhenguo Li 34 5 0 17 Sep 2024
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts Tingchen Fu Yupeng Hou Julian McAuley Rui Yan 22 3 0 09 Aug 2024
Orchestrating LLMs with Different Personalizations Jin Peng Zhou Katie Z Luo Jingwen Gu Jason Yuan Kilian Q. Weinberger Wen Sun 38 2 0 04 Jul 2024
Decoding with Limited Teacher Supervision Requires Understanding When to Trust the Teacher Hyunjong Ok Jegwang Ryu Jaeho Lee 29 0 0 26 Jun 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 26 2 0 24 Jun 2024
Adversaries Can Misuse Combinations of Safe Models Erik Jones Anca Dragan Jacob Steinhardt 37 6 0 20 Jun 2024
Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding Kaiyan Zhang Jianyu Wang Ning Ding Biqing Qi Ermo Hua Xingtai Lv Bowen Zhou 28 7 0 18 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 33 5 0 17 Jun 2024
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference Jiabao Ji Yujian Liu Yang Zhang Gaowen Liu Ramana Rao Kompella Sijia Liu Shiyu Chang KELM MU 23 21 0 12 Jun 2024
Proofread: Fixes All Errors with One Tap Renjie Liu Yanxiang Zhang Yun Zhu Haicheng Sun Yuanbo Zhang Michael Xuelin Huang Shanqing Cai Lei Meng Shumin Zhai ALM 18 2 0 06 Jun 2024
Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models Zhanhui Zhou Zhixuan Liu Jie Liu Zhichen Dong Chao Yang Yu Qiao ALM 36 20 0 29 May 2024
RE-Adapt: Reverse Engineered Adaptation of Large Language Models William Fleshman Benjamin Van Durme VLM 27 3 0 23 May 2024
Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast Chufan Shi Cheng Yang Xinyu Zhu Jiahao Wang Taiqiang Wu Siheng Li Deng Cai Yujiu Yang Yu Meng MoE 32 9 0 23 May 2024
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? D. Zhu Pinzhen Chen Miaoran Zhang Barry Haddow Xiaoyu Shen Dietrich Klakow 30 9 0 22 Apr 2024
Offset Unlearning for Large Language Models James Y. Huang Wenxuan Zhou Fei Wang Fred Morstatter Sheng Zhang Hoifung Poon Muhao Chen MU 22 13 0 17 Apr 2024
Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment Yuu Jinnai Tetsuro Morimura Kaito Ariu Kenshi Abe 57 3 0 01 Apr 2024
Stealing Part of a Production Language Model Nicholas Carlini Daniel Paleka Krishnamurthy Dvijotham Thomas Steinke Jonathan Hayase ... Arthur Conmy Itay Yona Eric Wallace David Rolnick Florian Tramèr MLAU AAML 20 71 0 11 Mar 2024
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang Jianyu Wang Ermo Hua Biqing Qi Ning Ding Bowen Zhou SyDa 17 20 0 05 Mar 2024
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models Chao Qian Jie M. Zhang Wei Yao Dongrui Liu Zhen-fei Yin Yu Qiao Yong Liu Jing Shao LLMSV LRM 37 13 0 29 Feb 2024
Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment Yiju Guo Ganqu Cui Lifan Yuan Ning Ding Jiexin Wang ... Ruobing Xie Jie Zhou Yankai Lin Zhiyuan Liu Maosong Sun 26 17 0 29 Feb 2024
Emulated Disalignment: Safety Alignment for Large Language Models May Backfire! Zhanhui Zhou Jie Liu Zhichen Dong Jiaheng Liu Chao Yang Wanli Ouyang Yu Qiao 10 16 0 19 Feb 2024
Decoding-time Realignment of Language Models Tianlin Liu Shangmin Guo Leonardo Bianco Daniele Calandriello Quentin Berthet Felipe Llinares-López Jessica Hoffmann Lucas Dixon Michal Valko Mathieu Blondel AI4CE 45 11 0 05 Feb 2024
Weak-to-Strong Jailbreaking on Large Language Models Xuandong Zhao Xianjun Yang Tianyu Pang Chao Du Lei Li Yu-Xiang Wang William Yang Wang 19 49 0 30 Jan 2024
Tuning Language Models by Proxy Alisa Liu Xiaochuang Han Yizhong Wang Yulia Tsvetkov Yejin Choi Noah A. Smith ALM 23 44 0 16 Jan 2024
Large Language Models for Social Networks: Applications, Challenges, and Solutions Jingying Zeng Richard Huang Waleed Malik Langxuan Yin Bojan Babic Danny Shacham Xiao Yan Jaewon Yang Qi He 9 3 0 04 Jan 2024
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 197 2,232 0 22 Mar 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 393 2,216 0 03 Sep 2019