ENH: Enforce `stopping_maxfun` option via optimagic

[timmens]

What would you like to enhance and why? Is it related to an issue/problem?

Some algorithms support the stopping_maxfun: int option, which stops the optimization if that number of function evaluations took place. Unfortunately, many algorithms do not support it.

Describe the solution you'd like

For algorithms that do not support the option, we handle it via optimagic.

Implementation ideas

The implementation would be algorithm / library specific. Some thoughts to consider:

• How to handle parallelism and batch evaluations?
• Does the library support callbacks / early stopping?
• Do we need to raise / catch some custom exception?

[janosg]

I think we should implement this in an algorithm independent way for all algorithms that do not support the option. And when we do it, we should also implement stopping_maxtime (which is supported by very few optimizers but can be useful.

The only way to do this in an algorithm independent way is an exception in InternalOptimizationProblem.fun (and related methods) that then gets caught in Algorithm.solve_internal_problem (yes, the one without underscore). The hard part is to preserve as much informatio as possible that can be used to construct an InternalOptimizeResult.

A rough sketch of the solution could be:

• Keep track of the best function value and corresponding parameters during the optimization in a similar way as we do history collection but even if history collection is disabled.
• Use this exception to abort an optimization and attach information to the exception.
• Catch the exception in Algorithm.solve_internal_problem, extract the information from the exception and construct an InternalOptimizeResult from it.