IPOP and BIPOP restart mechanisms

[beny-bee]

Hi everyone!

I've taken a break from the termination criteria and have been implementing restart mechanisms for CMA-ES. The plan was to do IPOP, but adding BIPOP was quite simple so I suggest adding it too.

IPOP: is an extension of the standard CMA-ES that features an increasing population size strategy, where at each termination of the optimizer the population is increased by a constant factor. This is based on the initial work found in PR #373.

BIPOP: is pretty straightforward after setting up IPOP. BIPOP uses two interlaced strategies: one with an increase in population size with a constant factor (identical to IPOP) and another with a smaller, varying population size. I've seen some places that this strategy works better on certain functions.

Additionally, there are two things to consider:

• Termination Criteria and API Adjustments: Depending on the termination criteria, we might need to introduce additional parameters for finer control. Thus, I know there are more parameters we can add on IPOP and BIPOP, but i would suggest waiting until the termination criteria are done to add all.
• Testing: I have copy pasted some of the tests from the other CMA-ES, but feel free to suggest more.

[zoq]

Should this be configurable?

[beny-bee]

Yeah, most likely. Plus there should be more parameters to control the termination criteria of the underlying CMA (from the other PR) which I will need to add. But since I still need to review the correctness of the termination criteria I don't want to add/delete stuff now.

[zoq]

Okay, makes sense.

[rcurtin]

Don't forget to add documentation to doc/. Without documentation nobody will ever know about these features.

[beny-bee]

Completely forgot about the documentation, added it now.

[rcurtin]

Thanks for adding documentation, just a couple comments about design and style. I hope to get the automatic style checker job online sometime, but until then if you don't mind going through and making sure everything matches I would appreciate it (and that would make it easier to fix that CI job whenever it does come online)

[rcurtin]

(This applies to BIPOPCMAES too.) For the other CMAES variants, like ActiveCMAES, we reflect the constructor parameters of CMAES instead of asking users to pass a CMAES object. That's also the strategy done for SGD variants. Do you think we could do the same here? So this would be more like

IPOPCMAES<SelectionPolicyType, TransformationPolicyType>(lambda, transformationPolicy, batchSize, maxIterations, tolerance, selectionPolicy, stepSize, populationFactor, maxRestarts)

or similar.

[beny-bee]

Both IPOP and BIPOP are restart mechanisms for the underlying CMAES. They are originally defined as extensions for the original "vanilla" CMAES, however, the restarts could be done for any variant. My intent was to have a CMAES object so the user could do the restart mechanism on whatever CMAES they wanted ("vanilla" CMAES or ActiveCMAES), which is why the user would input a CMAES object.

Nevertheless, I could just initialize whichever CMAES the user wants inside with the constructor you suggest. Or just ignore the different CMAES variants and just use the "vanilla" CMAES always.

What do you suggest?

[rcurtin]

The way you described it there, does it make more sense to simply modify CMAES to add another template parameter that represents the restart mechanism? That could simplify the code significantly.

From a user perspective, regardless of what choice you make internally, I do think it's important to make the constructor "feel" the same as the other CMAES variants, whether that is done by having IPOP-CMAES and BIPOP-CMAES be typedefs of CMAES with certain template parameters, or by holding a CMAES object internally. So, e.g., a constructor taking each of the same arguments as the CMAES constructor (or however many of those arguments are applicable).

Hopefully that makes sense, let me know if I can clarify anything I wrote. 👍

[beny-bee]

The way you described it there, does it make more sense to simply modify CMAES to add another template parameter that represents the restart mechanism? That could simplify the code significantly.

In this case, you mean adding something like template typename SelectionPolicyType = FullSelection, typename TransformationPolicyType = EmptyTransformation<>, typename RestartPolicyType = NoRestart> instead of having what I did right now?

Then I would need to modify the CMAES_impl.hpp Optimize to have something along the lines of restartPolicy.shouldRestart() and restartPolicy.restart()? Where the restart policy could either be BIPOP or IPOP or new policies.

If that's the case, I see what you mean, and it might indeed be cleaner.

But currently, I'm using the Optimize from the CMAES, and when it terminates, that elicits a restart. I would have to change quite a bit of code so that instead of returning when there is a stop condition, it loops until the restarts are done.

I'm fine doing it either way, but I might need to think about how to do it.

[zoq]

This could be handled by the callback feature, that said, I don't want to make the code too complex. Instead should we just create an independent IPOP implementation, that does not reuse CMAES? BIPOP could be a flag to the IPOP implementation. What do you think?

[beny-bee]

BIPOP could be a flag to the IPOP implementation.

I can definitely reduce both into one, with a flag to choose.

As for the code, I'm also fine doing either way, I feel like creating an instance of CMAES inside the IPOP and then use Optimize might be the best though. This way the user can control all the parameters in the same fashion, while also not duplicating code, and any improvement we do to the CMAES will be reflected in the IPOP (and BIPOP).

[beny-bee]

@zoq @rcurtin I refactored it into one code only (POP-CMA-ES) where with a flag it launches BIPOP or IPOP. I also tried to make it as similar to the other CMAES optimizers. However, now instead of passing it an CMAES object, I just initializes the CMAES class inside and use it.

Let me know if you prefer this version, and if you do, i'll write out the documentation for this and should be pretty much ready.

[rcurtin]

I like it, thanks for taking the effort to do that. Two quick notes:

• Feel free to use underscores in the class names, like POP_CMAES; this can make the class name a little more decipherable. :)
• If you want to keep the BIPOP and IPOP versions as separate optimizers, you could make the useBIPOP parameter into a template parameter and use a using statement to define the IPOPCMAES and BIPOPCMAES classes. Up to you whether you want to do that---it really hinges on whether you want to provide to the user a single class POPCMAES or two classes. (I guess two classes would be more in line with how ensmallen is structured, with one optimizer per class, but either way is fine with me.)

Thanks again for the effort; at least from my side I have no other major comments. 👍

[zoq]

All the comments make sense to me.

[beny-bee]

Done the suggestions @rcurtin!

Going to mark this no longer as draft

[rcurtin]

One more documentation bit I realized; can you add BIPOP_CMAES and IPOP_CMAES to the list of optimizers for separable functions in function_types.md? Thanks 👍

[beny-bee]

Just did that, also added ActiveCMAES which wasn't added.

Also, had to increase a bit the margin of the tests as in 2 environments it was failing

[zoq]

Looks reasonable.

[conradsnicta]

@rcurtin I've applied all your suggestions, as @beny-bee appears MIA. His GH history shows no substantial activity since September, presumably when GSOC'24 ended.

Suggest we merge this PR as is, and update HISTORY.md separately.

[coatless]

@conradsnicta let's wait until 2.22 launches

[beny-bee]

Thank you for the suggestions, @rcurtin, and for handling the suggestions, @conradsnicta. I apologize for the delay in my response; I've been quite busy recently.

I'm fine with all the style changes, as well as the RNG one. Additionally, I tested them locally and everything works as before.

I updated the HISTORY.md accordingly.

Thanks again for your support!

[rcurtin]

I pushed one more changes for the "CMA-ES" name changes to the documentation. Otherwise, I think this is ready for merge so I'll merge once there is a green build.