v1v2v3v4 (latest)

Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

27 March 2020

Joelle Pineau

Philippe Vincent-Lamarre

Papers citing "Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)"

50 / 179 papers shown

Title
Artificial intelligence in government: Concepts, standards, and a unified framework Vince J. Straub Deborah Morgan Jonathan Bright Helen Z. Margetts AI4TS 113 42 0 31 Oct 2022
A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges Mario Alfonso Prado-Romero Bardh Prenkaj Giovanni Stilo F. Giannotti CML 167 36 0 21 Oct 2022
Navigating the challenges in creating complex data systems: a development philosophy Sören Dittmer M. Roberts J. Gilbey A. Biguri Guang Yang J. Preller James H. F. Rudd J. Aston Carola-Bibiane Schönlieb 33 7 0 21 Oct 2022
Reproducibility of the Methods in Medical Imaging with Deep Learning A. Simkó A. Garpebring J. Jonsson T. Nyholm Tommy Löfstedt OOD 68 10 0 20 Oct 2022
DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning Siqi Xu Lin Liu Zhong Liu CML MedIm 107 10 0 10 Oct 2022
Replicable Bandits Hossein Esfandiari Alkis Kalavasis Amin Karbasi Andreas Krause Vahab Mirrokni Grigoris Velegkas 105 17 0 04 Oct 2022
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements Leandro von Werra Lewis Tunstall A. Thakur A. Luccioni Tristan Thrush ... Julien Chaumond Margaret Mitchell Alexander M. Rush Thomas Wolf Douwe Kiela ELM 134 29 0 30 Sep 2022
Reproducibility in machine learning for medical imaging O. Colliot Elina Thibeau-Sutre Ninon Burgos OOD 85 8 0 12 Sep 2022
When Bioprocess Engineering Meets Machine Learning: A Survey from the Perspective of Automated Bioprocess Development Nghia Duong-Trung Stefan Born Jong Woo Kim M. Schermeyer Katharina Paulick ... Thorben Werner Randolf Scholz Lars Schmidt-Thieme Peter Neubauer Ernesto Martinez 116 23 0 02 Sep 2022
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage Kurt Shuster Jing Xu M. Komeili Da Ju Eric Michael Smith ... Naman Goyal Arthur Szlam Y-Lan Boureau Melanie Kambadur Jason Weston LM&Ro KELM 217 249 0 05 Aug 2022
Innovations in Neural Data-to-text Generation: A Survey Mandar Sharma Ajay K. Gogineni Naren Ramakrishnan 127 10 0 25 Jul 2022
Leakage and the Reproducibility Crisis in ML-based Science Sayash Kapoor Arvind Narayanan 86 185 0 14 Jul 2022
Open High-Resolution Satellite Imagery: The WorldStrat Dataset -- With Application to Super-Resolution Julien Cornebise Ivan Orsolic F. Kalaitzis 102 60 0 13 Jul 2022
Long-term Reproducibility for Neural Architecture Search David Towers M. Forshaw Amir Atapour-Abarghouei A. Mcgough 112 1 0 11 Jul 2022
The "Collections as ML Data" Checklist for Machine Learning & Cultural Heritage Benjamin Charles Germain Lee VLM 70 7 0 06 Jul 2022
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers Julio A. Placed Jared Strader Henry Carrillo Nikolay Atanasov Vadim Indelman Luca Carlone J. A. Castellanos 206 212 0 01 Jul 2022
Benchopt: Reproducible, efficient and collaborative optimization benchmarks Thomas Moreau Mathurin Massias Alexandre Gramfort Pierre Ablin Pierre-Antoine Bannier Benjamin Charlier ... Binh Duc Nguyen A. Rakotomamonjy Zaccharie Ramzi Joseph Salmon Samuel Vaiter 189 40 0 27 Jun 2022
The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality Rex Chen Fei Fang Norman M. Sadeh 116 12 0 23 Jun 2022
Towards Better User Studies in Computer Graphics and Vision Zoya Bylinskii L. Herman Aaron Hertzmann Stefanie Hutka Yile Zhang 105 14 0 23 Jun 2022
DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation Zhu Sun Hui Fang Jie Yang Xinghua Qu Hongyang Liu Di Yu Yew-Soon Ong Jie M. Zhang 123 42 0 22 Jun 2022
The Fallacy of AI Functionality Inioluwa Deborah Raji Indra Elizabeth Kumar Aaron Horowitz Andrew D. Selbst 127 213 0 20 Jun 2022
Modeling the Machine Learning Multiverse Samuel J. Bell Onno P. Kampman Jesse Dodge Neil D. Lawrence 122 19 0 13 Jun 2022
SoK: The Impact of Unlabelled Data in Cyberthreat Detection Giovanni Apruzzese Pavel Laskov A.T. Tastemirova 103 33 0 18 May 2022
ConfLab: A Data Collection Concept, Dataset, and Benchmark for Machine Analysis of Free-Standing Social Interactions in the Wild Chirag Raman Jose Vargas-Quiros Stephanie Tan Ashraful Islam Ekin Gedik Hayley Hung 99 9 0 10 May 2022
Data Governance in the Age of Large-Scale Data-Driven Language Technology Yacine Jernite Huu Nguyen Stella Biderman A. Rogers Maraim Masoud ... Jorg Frohberg Aaron Gokaslan Peter Henderson Rishi Bommasani Margaret Mitchell 110 54 0 04 May 2022
Prescriptive and Descriptive Approaches to Machine-Learning Transparency David Adkins B. Alsallakh Adeel Cheema Narine Kokhlikyan Emily McReynolds Pushkar Mishra Chavez Procope Jeremy Sawruk Erin Wang Polina Zvyagina 44 13 0 27 Apr 2022
A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms Maxime Alvarez Jean-Charles Verdier D'Jeff K. Nkashama Marc Frappier Pierre Martin Tardif F. Kabanza 97 17 0 21 Apr 2022
Sources of Irreproducibility in Machine Learning: A Review Odd Erik Gundersen Kevin Coakley Christine R. Kirkpatrick Yolanda Gil SyDa 172 37 0 15 Apr 2022
Data-Driven Evaluation of Training Action Space for Reinforcement Learning Rajat Ghosh Debojyoti Dutta 76 0 0 08 Apr 2022
The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning Jessica Hullman Sayash Kapoor Priyanka Nanayakkara Andrew Gelman Arvind Narayanan 219 40 0 12 Mar 2022
Reliable validation of Reinforcement Learning Benchmarks Matthias Muller-Brockhausen Aske Plaat Mike Preuss OffRL 85 1 0 02 Mar 2022
Deep Learning Reproducibility and Explainable AI (XAI) Anastasia-Maria Leventi-Peetz T. Östreich 108 9 0 23 Feb 2022
Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations G. Shamir Dong Lin HAI OffRL 79 7 0 14 Feb 2022
Reproducibility in Optimization: Theoretical Framework and Limits Kwangjun Ahn Prateek Jain Ziwei Ji Satyen Kale Praneeth Netrapalli G. Shamir 89 24 0 09 Feb 2022
Towards a consistent interpretation of AIOps models Yingzhe Lyu Gopi Krishnan Rajbahadur Dayi Lin Boyuan Chen Zhen Ming Z. Jiang AI4CE 116 23 0 04 Feb 2022
Towards Training Reproducible Deep Learning Models Boyuan Chen Mingzhi Wen Yong Shi Dayi Lin Gopi Krishnan Rajbahadur Zhen Ming Z. Jiang SyDa 93 42 0 04 Feb 2022
Pearl: Parallel Evolutionary and Reinforcement Learning Library Rohan Tangri Danilo P. Mandic A. Constantinides 84 3 0 24 Jan 2022
Reproducibility in Learning R. Impagliazzo Rex Lei T. Pitassi Jessica Sorrell 131 51 0 20 Jan 2022
Disentanglement by Cyclic Reconstruction David Bertoin Emmanuel Rachelson OOD 91 4 0 24 Dec 2021
A Synthetic Prediction Market for Estimating Confidence in Published Work Sarah Rajtmajer Christopher Griffin Jian Wu Robert Fraleigh Laxmaan Balaji ... A. Menon Sai Ajay Modukuri Rajal Nivargi Xin Wei C. Lee Giles OffRL 48 10 0 23 Dec 2021
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 160 28 0 16 Dec 2021
Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach Setareh Ariafar Justin Gilmer Zachary Nado Jasper Snoek Rodolphe Jenatton George E. Dahl 78 2 0 15 Dec 2021
Software Engineering for Responsible AI: An Empirical Study and Operationalised Patterns Qinghua Lu Liming Zhu Xiwei Xu Jon Whittle David M. Douglas Conrad Sanderson 78 39 0 18 Nov 2021
Off-policy Reinforcement Learning with Optimistic Exploration and Distribution Correction A. Ahmad Shuo Cheng D. Saraswat Aly El Gamal Wenjie Wang Gurmukh Johal OffRL OnRL 95 1 0 22 Oct 2021
CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability Martin Mundt Steven Braun Quentin Delfosse Kristian Kersting 136 37 0 07 Oct 2021
Trustworthy AI: From Principles to Practices Yue Liu Peng Qi Bo Liu Shuai Di Jingen Liu Jiquan Pei Jinfeng Yi Bowen Zhou 279 423 0 04 Oct 2021
Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms Albert Jiménez Sanfiz Mohamed Akrout OOD AAML 104 9 0 30 Aug 2021
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Aaron Courville Marc G. Bellemare OffRL 346 724 0 30 Aug 2021
Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study Tianlong Chen Kaixiong Zhou Keyu Duan Wenqing Zheng Peihao Wang Helen Zhou Zhangyang Wang AAML GNN 97 69 0 24 Aug 2021
Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards Angelina McMillan-Major Salomey Osei Juan Diego Rodriguez Pawan Sasanka Ammanamanchi Sebastian Gehrmann Yacine Jernite 129 51 0 16 Aug 2021