Supervising strong learners by amplifying weak experts

19 October 2018

Papers citing "Supervising strong learners by amplifying weak experts"

36 / 36 papers shown

Title
An alignment safety case sketch based on debate Marie Davidsen Buhl Jacob Pfau Benjamin Hilton Geoffrey Irving 38 0 0 06 May 2025
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents Christian Schroeder de Witt AAML AI4CE 150 1 0 04 May 2025
Scaling Laws For Scalable Oversight Joshua Engels David D. Baek Subhash Kantamneni Max Tegmark ELM 75 0 0 25 Apr 2025
Redefining Superalignment: From Weak-to-Strong Alignment to Human-AI Co-Alignment to Sustainable Symbiotic Society Feifei Zhao Y. Wang Enmeng Lu Dongcheng Zhao Bing Han ... Chao Liu Yaodong Yang Yi Zeng Boyuan Chen Jinyu Fan 83 0 0 24 Apr 2025
aiXamine: Simplified LLM Safety and Security Fatih Deniz Dorde Popovic Yazan Boshmaf Euisuh Jeong M. Ahmad Sanjay Chawla Issa M. Khalil ELM 80 0 0 21 Apr 2025
System 2 Reasoning Capabilities Are Nigh Scott C. Lowe VLM LRM 46 0 0 04 Oct 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 59 14 0 24 Jun 2024
Learning Task Decomposition to Assist Humans in Competitive Programming Jiaxin Wen Ruiqi Zhong Pei Ke Zhihong Shao Hongning Wang Minlie Huang ReLM 37 8 0 07 Jun 2024
Detecting Mode Collapse in Language Models via Narration Sil Hamilton 12 9 0 06 Feb 2024
Scalable AI Safety via Doubly-Efficient Debate Jonah Brown-Cohen Geoffrey Irving Georgios Piliouras 32 15 0 23 Nov 2023
Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints Chaoqi Wang Yibo Jiang Yuguang Yang Han Liu Yuxin Chen 36 82 0 28 Sep 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 76 232 0 12 Aug 2023
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Hanze Dong Wei Xiong Deepanshu Goyal Yihan Zhang Winnie Chow Rui Pan Shizhe Diao Jipeng Zhang Kashun Shum Tong Zhang ALM 18 404 0 13 Apr 2023
A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors Shangding Gu Alap Kshirsagar Yali Du Guang Chen Jan Peters Alois C. Knoll 34 14 0 25 Feb 2023
Discovering Language Model Behaviors with Model-Written Evaluations Ethan Perez Sam Ringer Kamilė Lukošiūtė Karina Nguyen Edwin Chen ... Danny Hernandez Deep Ganguli Evan Hubinger Nicholas Schiefer Jared Kaplan ALM 22 364 0 19 Dec 2022
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 79 1,477 0 15 Dec 2022
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 70 327 0 07 Dec 2022
Measuring Progress on Scalable Oversight for Large Language Models Sam Bowman Jeeyoon Hyun Ethan Perez Edwin Chen Craig Pettit ... Tristan Hume Yuntao Bai Zac Hatfield-Dodds Benjamin Mann Jared Kaplan ALM ELM 28 122 0 04 Nov 2022
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals Rohin Shah Vikrant Varma Ramana Kumar Mary Phuong Victoria Krakovna J. Uesato Zachary Kenton 34 68 0 04 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 502 0 28 Sep 2022
The Alignment Problem from a Deep Learning Perspective Richard Ngo Lawrence Chan Sören Mindermann 56 183 0 30 Aug 2022
Self-critiquing models for assisting human evaluators William Saunders Catherine Yeh Jeff Wu Steven Bills Ouyang Long Jonathan Ward Jan Leike ALM ELM 29 280 0 12 Jun 2022
TALM: Tool Augmented Language Models Aaron T Parisi Yao-Min Zhao Noah Fiedel KELM RALM LLMAG 29 144 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Safe Deep RL in 3D Environments using Human Feedback Matthew Rahtz Vikrant Varma Ramana Kumar Zachary Kenton Shane Legg Jan Leike 29 4 0 20 Jan 2022
WebGPT: Browser-assisted question-answering with human feedback Reiichiro Nakano Jacob Hilton S. Balaji Jeff Wu Ouyang Long ... Gretchen Krueger Kevin Button Matthew Knight B. Chess John Schulman ALM RALM 81 1,196 0 17 Dec 2021
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 35 294 0 22 Sep 2021
Question Decomposition with Dependency Graphs Matan Hasson Jonathan Berant GNN 22 9 0 17 Apr 2021
An overview of 11 proposals for building safe advanced AI Evan Hubinger AAML 22 23 0 04 Dec 2020
Avoiding Tampering Incentives in Deep RL via Decoupled Approval J. Uesato Ramana Kumar Victoria Krakovna Tom Everitt Richard Ngo Shane Legg 26 14 0 17 Nov 2020
Learning to summarize from human feedback Nisan Stiennon Long Ouyang Jeff Wu Daniel M. Ziegler Ryan J. Lowe Chelsea Voss Alec Radford Dario Amodei Paul Christiano ALM 19 1,984 0 02 Sep 2020
AI safety: state of the field through quantitative lens Mislav Juric A. Sandic Mario Brčič 23 24 0 12 Feb 2020
Risks from Learned Optimization in Advanced Machine Learning Systems Evan Hubinger Chris van Merwijk Vladimir Mikulik Joar Skalse Scott Garrabrant 31 146 0 05 Jun 2019
Embedded Agency A. Demski Scott Garrabrant AIFin 19 34 0 25 Feb 2019
Scalable agent alignment via reward modeling: a research direction Jan Leike David M. Krueger Tom Everitt Miljan Martic Vishal Maini Shane Legg 34 395 0 19 Nov 2018
AI safety via debate G. Irving Paul Christiano Dario Amodei 204 200 0 02 May 2018