Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2007.03488
Cited By
v1
v2 (latest)
Benchmarking in Optimization: Best Practice and Open Issues
7 July 2020
Thomas Bartz-Beielstein
Carola Doerr
Daan van den Berg
Jakob Bossek
Sowmya Chandrasekaran
T. Eftimov
A. Fischbach
P. Kerschke
William La Cava
Manuel López-Ibánez
K. Malan
J. Moore
B. Naujoks
Patryk Orzechowski
Vanessa Volz
Markus Wagner
T. Weise
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Benchmarking in Optimization: Best Practice and Open Issues"
50 / 53 papers shown
PORTAL: Controllable Landscape Generator for Continuous Optimization-Part I: Framework
D. Yazdani
Mai Peng
Delaram Yazdani
Shima F. Yazdi
M. Omidvar
Yuan Sun
Trung Thanh Nguyen
Changhe Li
Xiaodong Li
102
1
0
29 Nov 2025
From Performance to Understanding: A Vision for Explainable Automated Algorithm Design
Niki van Stein
Anna V. Kononova
Thomas Bäck
170
0
0
20 Nov 2025
Benchmarking that Matters: Rethinking Benchmarking for Practical Impact
Anna V. Kononova
Niki van Stein
Olaf Mersmann
Thomas Bäck
Thomas Bartz-Beielstein
...
Quentin Renau
Jeroen Rook
Lennart Schäpermeier
Diederick Vermetten
D. Zaharie
120
0
0
15 Nov 2025
Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons
Junbo Jacob Lian
90
0
0
10 Sep 2025
Benchmarking MOEAs for solving continuous multi-objective RL problems
Carlos Hernández
Roberto Santana
271
0
0
19 May 2025
A Standardized Benchmark Set of Clustering Problem Instances for Comparing Black-Box Optimizers
Foundations of Genetic Algorithms (FOGA), 2025
Diederick Vermetten
Catalin-Viorel Dinu
Marcus Gallagher
218
0
0
14 May 2025
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows
Wenhao Li
Bo Jin
Mingyi Hong
Changhong Lu
Xiangfeng Wang
502
0
0
07 May 2025
BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics
Niki van Stein
Anna V. Kononova
Haoran Yin
Thomas Bäck
331
9
0
28 Apr 2025
Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation
Maria Eriksson
Erasmo Purificato
Arman Noroozian
Joao Vinagre
Guillaume Chaslot
Emilia Gomez
David Fernandez-Llorca
ELM
832
44
0
10 Feb 2025
Evolving Hard Maximum Cut Instances for Quantum Approximate Optimization Algorithms
Annual Conference on Genetic and Evolutionary Computation (GECCO), 2025
Shuaiqun Pan
Yash J. Patel
Aneta Neumann
Frank Neumann
Thomas Bäck
Hao Wang
521
0
0
30 Jan 2025
MO-IOHinspector: Anytime Benchmarking of Multi-Objective Algorithms using IOHprofiler
Diederick Vermetten
Jeroen Rook
Oliver L. Preuß
Jacob De Nobel
Carola Doerr
Manuel López-Ibánez
Heike Trautmann
Thomas Bäck
305
4
0
10 Dec 2024
BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
Neural Information Processing Systems (NeurIPS), 2024
Anka Reuel
Amelia F. Hardy
Chandler Smith
Max Lamparth
Malcolm Hardy
Mykel J. Kochenderfer
ELM
620
88
0
20 Nov 2024
Deconstructing What Makes a Good Optimizer for Language Models
Rosie Zhao
Depen Morwani
David Brandfonbrener
Nikhil Vyas
Sham Kakade
524
42
0
10 Jul 2024
Generalizability of experimental studies
Federico Matteucci
Vadim Arzamasov
Jose Cribeiro-Ramallo
Marco Heyden
Konstantin Ntounas
Klemens Bohm
441
0
0
25 Jun 2024
A Novel Ranking Scheme for the Performance Analysis of Stochastic Optimization Algorithms using the Principles of Severity
Sowmya Chandrasekaran
Thomas Bartz-Beielstein
247
0
0
31 May 2024
Impact of Training Instance Selection on Automated Algorithm Selection Models for Numerical Black-box Optimization
Konstantin Dietrich
Diederick Vermetten
Carola Doerr
P. Kerschke
253
6
0
11 Apr 2024
Clustering in Dynamic Environments: A Framework for Benchmark Dataset Generation With Heterogeneous Changes
D. Yazdani
Juergen Branke
M. S. Khorshidi
M. Omidvar
Xiaodong Li
Amir H. Gandomi
Xin Yao
209
4
0
24 Feb 2024
Benchmarking multi-component signal processing methods in the time-frequency plane
J. M. Miramont
Rémi Bardenet
P. Chainais
François Auger
186
4
0
13 Feb 2024
Impact of spatial transformations on landscape features of CEC2022 basic benchmark problems
Haoran Yin
Diederick Vermetten
Furong Ye
Thomas H. W. Bäck
Anna V. Kononova
239
2
0
12 Feb 2024
Explainable Benchmarking for Iterative Optimization Heuristics
Niki van Stein
Diederick Vermetten
Anna V. Kononova
Thomas Bäck
419
24
0
31 Jan 2024
A Generalized and Configurable Benchmark Generator for Continuous Unconstrained Numerical Optimization
D. Yazdani
M. Omidvar
Delaram Yazdani
Kalyanmoy Deb
255
5
0
12 Dec 2023
Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym
IEEE Transactions on Evolutionary Computation (IEEE TEVC), 2023
Elena Raponi
Nathanaël Carraz Rakotonirina
Jérémy Rapin
Carola Doerr
O. Teytaud
648
10
0
29 Sep 2023
Challenges of ELA-guided Function Evolution using Genetic Programming
International Joint Conference on Computational Intelligence (IJCCI), 2023
Fu Xing Long
Diederick Vermetten
Anna V. Kononova
Roman Kalkreuth
Kaifeng Yang
Thomas Bäck
Niki van Stein
157
8
0
24 May 2023
PyTorch Hyperparameter Tuning - A Tutorial for spotPython
Thomas Bartz-Beielstein
BDL
GP
256
5
0
19 May 2023
When to be Discrete: Analyzing Algorithm Performance on Discretized Continuous Problems
Annual Conference on Genetic and Evolutionary Computation (GECCO), 2023
André Thomaser
Jacob De Nobel
Diederick Vermetten
Furong Ye
Thomas Bäck
Anna V. Kononova
155
10
0
25 Apr 2023
Using Affine Combinations of BBOB Problems for Performance Assessment
Annual Conference on Genetic and Evolutionary Computation (GECCO), 2023
Diederick Vermetten
Furong Ye
Carola Doerr
308
13
0
08 Mar 2023
Using Automated Algorithm Configuration for Parameter Control
Foundations of Genetic Algorithms (FOGA), 2023
D. Chen
M. Buzdalov
Carola Doerr
Nguyen Dang
341
8
0
23 Feb 2023
Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games
Applied Soft Computing (Appl. Soft Comput.), 2023
Vanessa Volz
B. Naujoks
P. Kerschke
Tea Tušar
246
6
0
16 Feb 2023
Characterization of Constrained Continuous Multiobjective Optimization Problems: A Performance Space Perspective
IEEE Transactions on Evolutionary Computation (IEEE TEVC), 2023
Aljoša Vodopija
Tea Tušar
B. Filipič
136
8
0
04 Feb 2023
On the Algebraic Properties of Flame Graphs
G. Tornetta
95
0
0
21 Jan 2023
A Local Optima Network View of Real Function Fitness Landscapes
Marco Tomassini
193
5
0
20 Dec 2022
BBOB Instance Analysis: Landscape Properties and Algorithm Performance across Problem Instances
F. Long
Diederick Vermetten
Bas van Stein
Anna V. Kononova
193
29
0
29 Nov 2022
OPTION: OPTImization Algorithm Benchmarking ONtology
IEEE Transactions on Evolutionary Computation (TEVC), 2021
Ana Kostovska
Diederick Vermetten
Carola Doerr
Jannis Brugger
P. Panov
T. Eftimov
201
15
0
21 Nov 2022
Benchopt: Reproducible, efficient and collaborative optimization benchmarks
Neural Information Processing Systems (NeurIPS), 2022
Thomas Moreau
Mathurin Massias
Alexandre Gramfort
Pierre Ablin
Pierre-Antoine Bannier Benjamin Charlier
...
Binh Duc Nguyen
A. Rakotomamonjy
Zaccharie Ramzi
Joseph Salmon
Samuel Vaiter
401
49
0
27 Jun 2022
A Survey on Learnable Evolutionary Algorithms for Scalable Multiobjective Optimization
IEEE Transactions on Evolutionary Computation (TEVC), 2022
Songbai Liu
Qiuzhen Lin
Jianqiang Li
Kay Chen Tan
425
94
0
23 Jun 2022
A Framework for Generating Informative Benchmark Instances
International Conference on Principles and Practice of Constraint Programming (CP), 2022
Nguyen Dang
Ozgur Akgun
Joan Espasa
Ian Miguel
Peter William Nightingale
204
9
0
29 May 2022
Analyzing the Impact of Undersampling on the Benchmarking and Configuration of Evolutionary Algorithms
Annual Conference on Genetic and Evolutionary Computation (GECCO), 2022
Diederick Vermetten
Hongya Wang
Manuel López-Ibánez
Carola Doerr
Thomas Bäck
260
12
0
20 Apr 2022
The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants
Annual Conference on Genetic and Evolutionary Computation (GECCO), 2022
Ana Kostovska
Diederick Vermetten
Jannis Brugger
Carola Doerr
Peter Korošec
T. Eftimov
234
15
0
15 Apr 2022
An Instance Space Analysis of Constrained Multi-Objective Optimization Problems
IEEE Transactions on Evolutionary Computation (TEVC), 2022
Hanan Alsouly
M. Kirley
Mario Andrés Muñoz
130
11
0
02 Mar 2022
IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics
Jacob De Nobel
Furong Ye
Diederick Vermetten
Hao Wang
Carola Doerr
Thomas Bäck
313
57
0
07 Nov 2021
HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO
Katharina Eggensperger
Philip Muller
Neeratyoy Mallik
Matthias Feurer
René Sass
Aaron Klein
Noor H. Awad
Marius Lindauer
Katharina Eggensperger
493
130
0
14 Sep 2021
A Continuous Optimisation Benchmark Suite from Neural Network Regression
K. Malan
C. Cleghorn
ODL
127
2
0
12 Sep 2021
Characterization of Constrained Continuous Multiobjective Optimization Problems: A Feature Space Perspective
Information Sciences (Inf. Sci.), 2021
Aljoša Vodopija
Tea Tušar
B. Filipič
207
22
0
09 Sep 2021
Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT
Thomas Bartz-Beielstein
Frederik Rehbach
Amrita Sen
Martin Zaefferer
299
0
0
30 May 2021
Generating Instances with Performance Differences for More Than Just Two Algorithms
Jakob Bossek
Markus Wagner
227
5
0
29 Apr 2021
Towards Standardising Reinforcement Learning Approaches for Production Scheduling Problems
Procedia CIRP (PC), 2021
Alexandru Rinciog
Anne Meyer
OffRL
179
14
0
16 Apr 2021
On the Assessment of Benchmark Suites for Algorithm Comparison
D. I. Mattos
Lucas Ruud
J. Bosch
Helena Holmström Olsson
142
1
0
15 Apr 2021
Towards Large Scale Automated Algorithm Design by Integrating Modular Benchmarking Frameworks
Amine Aziz-Alaoui
Carola Doerr
Johann Dréo
151
13
0
12 Feb 2021
Reproducibility in Evolutionary Computation
ACM Transactions on Evolutionary Learning and Optimization (TELO), 2021
Manuel López-Ibánez
Juergen Branke
L. Paquete
284
38
0
05 Feb 2021
Cognitive Capabilities for the CAAI in Cyber-Physical Production Systems
The International Journal of Advanced Manufacturing Technology (IJAMT), 2020
Jan Strohschein
A. Fischbach
Andreas Bunte
Heide Faeskorn-Woyke
N. Moriz
Thomas Bartz-Beielstein
174
3
0
03 Dec 2020
1
2
Next
Page 1 of 2
