A possibility to optimize a black box system using algebraic surrogate models which can be identified using a symbolic regression approach.

Performing an optimization is a really interesting task. In our each day life, we could be excited about the most effective approach to get to work within the shortest period of time, or perhaps in the most effective particle size of our ground coffee to attain a really tasty cup of coffee ☕. Industries are also excited about optimizing things, similar to supply chains, carbon emissions, or waste accumulation.

There are is numerous possibilities how arrange an optimization, depending on how the actual situation looks. Let me divide these situations in two parts for this text:

On the one hand we might need knowledge in regards to the physics, chemistry or biologics that drive the system under study. With this, we could arrange algebraic equations that accurately describe what we observe (first-principles). These situations allow the usage of off-the-shelf solvers, similar to GLPK, BARON, ANTIGONE, SBB, or others, since we have now closed-form expressions and might calculate their derivatives.

However, we would not likely have an idea of how our system looks or behaves. One approach to get some information out of it will be to perform experiments, meaning define some inputs and observe what happens within the output. To optimize such a system, we could use heuristics, like particle swarm optimization, apply a genetic algorithm, or use powerful techniques like Bayesian optimization.

We could dive deeply into literature and plenty of discussion now. But allow us to keep it easy here. Allow us to focus only the second case, where we would not have a pleasant and accurate mathematical closed-form description of our system, or we don’t have time to give you one because we’re busy drinking coffee ☕. Allow us to also assume we have now some past observations, but we cannot sample recent data from our system because of whatever reason.

Such a situation might arise if you end up working with very expensive material, similar to pharmaceuticals. You may have produced some batches of drug product prior to now, but you can’t produce one other batch only for the sake…