Learning rate suggestion is not robust enough

[Nidish96]

The simple numerical gradient based selection approach employed for the suggested learning rate is not robust enough. Consider the following example:

[NaleRaphael]

That's the weakness of searching with gradient since differential equation is sensitive to high frequency changes, i.e., the vibration parts in your graph. But I'm afraid it's not a fixable problem.

It's a hard decision to make since there is still not a single valid algorithm to determine the optimal learning rate for now, and we also had a discussion about this problem before and we preferred leaving it to users to decide how to deal with it, either implementing their own selection algorithm or picking one manually from the lr-loss curve.

So, if you want to stick with the default learning rate selection algorithm supported in LRFinder, try increasing the value of smooth_f when calling range_test(). Or you can implement your own one with LRFinder.history according to those suggested approaches mentioned in this comment.

As a side note: the main idea of a learning rate selection/searching algorithm is to "find a learning rate that can make loss descent faster". This idea is still applicable according to AdaLRS (2506.13274), but you might still want to investigate whether it can be applied on the problem you are working on.