Seminar: From Nature-Inspired Design to AI-Generated Metaheuristics

Abstract: 

For decades, researchers have been looking at paradigms gleaned from nature as inspiration for problem solving approaches, for example in the domain of optimization. There are many classes of such algorithms, including evolutionary algorithms, particle swarms, differential evolution, ant colony optimization, and the number of proposed variants of them is quite large.

This makes it hard to keep track of the variants and their respective strengths, and even more so it creates a difficult situation for non-experts who are interested in selecting the best algorithm for their real-world application problem. In this presentation, I will introduce and discuss ideas to automatically optimize the optimization heuristic. This task can be approached as an algorithm configuration problem, for which I will present some examples illustrating that this task can be handled by direct global optimization algorithms – in other words, by “automatically optimizing the optimization algorithm”.

I will give an example how a combinatorial design space of thousands of configuration variants of evolution strategies in a so-called modular Covariance Matrix Adaptation Evolution Strategy framework can be searched, and how the results can be analyzed using data mining. This approach provides an opportunity for discovering the unexplored areas of the optimization algorithm design space. As a next step into the direction of automated algorithm generation, I will briefly show a novel approach for using a large language model in an iterative loop to automatically generate metaheuristic algorithms for optimization, and how this approach which we call LLaMEA (Large Language Model Evolutionary Algorithm) can generate novel metaheuristics that perform very well on a standard set of benchmark functions.

Biography: