Adding AGEMOEA and IGD

doc/function_types.md

@@ -884,13 +884,98 @@ arma::cube bestFront = optimizer.ParetoFront();
 
 </details>
 
+### Performance Indicators
+
+Performance indicators in multiobjective optimization provide essential metrics 
+for evaluating solution quality, such as convergence to the Pareto front and solution 
+diversity.
+
+The ensmallen library offers three such indicators, aiding in the assessment and comparison 
+of different optimization methods:
+
+#### Epsilon
+
+Epsilon metric is a performance metric used in multi-objective optimization which measures 
+the smallest factor by which a set of solution objectives must be scaled to dominate a 
+reference set of solutions. Specifically, given a set of Pareto-optimal solutions, the 
+epsilon indicator finds the minimum value ϵ such that each solution in the set is at 
+least as good as every solution in the reference set when the objectives are scaled by ϵ.
+
+<details open>
+<summary>Click to collapse/expand example code.
+</summary>
+
+```c++
+arma::cube referenceFront(2, 1, 3);
+double tol = 1e-10;
+referenceFront.slice(0) = arma::vec{0.01010101, 0.89949622};
+referenceFront.slice(1) = arma::vec{0.02020202, 0.85786619};
+referenceFront.slice(2) = arma::vec{0.03030303, 0.82592234};
+arma::cube front = referenceFront * 1.1;
+// eps is approximately 1.1
+double eps = Epsilon::Evaluate(front, referenceFront);
+```
+</details>
+
+#### IGD
+
+Inverse Generational Distance (IGD) is a performance metric used in multi-objective optimization 
+to evaluate the quality of a set of solutions relative to a reference set, typically representing 
+the true Pareto front. IGD measures the average distance from each point in the reference set to 
+the closest point in the obtained solution set.
+
+<details open>
+<summary>Click to collapse/expand example code.
+</summary>
+
+```c++
+arma::cube referenceFront(2, 1, 3);
+double tol = 1e-10;
+referenceFront.slice(0) = arma::vec{0.01010101, 0.89949622};
+referenceFront.slice(1) = arma::vec{0.02020202, 0.85786619};
+referenceFront.slice(2) = arma::vec{0.03030303, 0.82592234};
+arma::cube front = referenceFront * 1.1;
+// The third parameter is the power constant in the distance formula.
+// IGD is approximately 0.05329
+double igd = IGD::Evaluate(front, referenceFront, 1);
+```
+</details>
+
+#### IGD Plus
+
+IGD Plus (IGD+) is a variant of the Inverse Generational Distance (IGD) metric used in multi-objective 
+optimization. It refines the traditional IGD metric by incorporating a preference for Pareto-dominance 
+in the distance calculation. This modification helps IGD+ better reflect both the convergence to the 
+Pareto front and the diversity of the solution set.
+
+<details open>
+<summary>Click to collapse/expand example code.
+</summary>
+
+```c++
+arma::cube referenceFront(2, 1, 3);
+double tol = 1e-10;
+referenceFront.slice(0) = arma::vec{0.01010101, 0.89949622};
+referenceFront.slice(1) = arma::vec{0.02020202, 0.85786619};
+referenceFront.slice(2) = arma::vec{0.03030303, 0.82592234};
+arma::cube front = referenceFront * 1.1;
+// IGDPlus is approximately 0.05329
+double igdPlus = IGDPlus::Evaluate(front, referenceFront);
+```
+</details>
+
 *Note*: all multi-objective function optimizers have both the function `Optimize()` to find the
 best front, and also the function `ParetoFront()` to return all sets of solutions that are on the
 front.
 
 The following optimizers can be used with multi-objective functions:
 - [NSGA2](#nsga2)
 - [MOEA/D-DE](#moead)
+- [AGEMOEA](#agemoea)
+
+#### See also:
+* [Performance Assessment of Multiobjective Optimizers: An Analysis and Review](https://sop.tik.ee.ethz.ch/publicationListFiles/ztlf2003a.pdf)
+* [Modified Distance Calculation in Generational Distance and Inverted Generational Distance](https://link.springer.com/chapter/10.1007/978-3-319-15892-1_8)
 
 ## Constrained functions
 

doc/optimizers.md

@@ -501,6 +501,97 @@ optimizer.Optimize(f, coordinates);
  * [Adam: A Method for Stochastic Optimization](http://arxiv.org/abs/1412.6980) (see section 7)
  * [Differentiable separable functions](#differentiable-separable-functions)
 
+## AGEMOEA
+
+*An optimizer for arbitrary multi-objective functions.*
+
+Adaptive Geometry Estimation based Multi-Objective Evolutionary Algorithm (AGE-MOEA) is an optimization framework based on NSGA-II yet differs from it in replacing the crowding distance of NSGA-II by a survival score, for which calculations need the diversity and proximity of non-dominated sets. To simplify the computation of the survival score, in each generation, the geometry of the initial non-dominated subset is estimated by AGE-MOEA afterwards, this estimation which gets more accurate as the algorithm matures, is used as the geometry of the Pareto set.
+
+#### Constructors
+
+ * `AGEMOEA()`
+ * `AGEMOEA(`_`populationSize, maxGenerations, crossoverProb, distributionIndex, epsilon, eta, lowerBound, upperBound`_`)`
+
+#### Attributes
+
+| **type** | **name** | **description** | **default** |
+|----------|----------|-----------------|-------------|
+| `size_t` | **`populationSize`** | The number of candidates in the population. This should be at least 4 in size and a multiple of 4. | `100` |
+| `size_t` | **`maxGenerations`** | The maximum number of generations allowed for AGEMOEA. | `2000` |
+| `double` | **`crossoverProb`** | Probability that a crossover will occur. | `0.6` |
+| `double` | **`distributionIndex`** | The crowding degree of the mutation. | `20` |
+| `double` | **`epsilon`** | The value used internally to evaluate approximate equality in crowding distance based sorting. | `1e-6` |
+| `double` | **`eta`** | The distance parameters of the crossover distribution. | `20` |
+| `double`, `arma::vec` | **`lowerBound`** | Lower bound of the coordinates on the coordinates of the whole population during the search process. | `0` |
+| `double`, `arma::vec` | **`upperBound`** | Lower bound of the coordinates on the coordinates of the whole population during the search process. | `1` |
+
+Note that the parameters `lowerBound` and `upperBound` are overloaded. Data types of `double` or `arma::mat` may be used. If they are initialized as single values of `double`, then the same value of the bound applies to all the axes, resulting in an initialization following a uniform distribution in a hypercube. If they are initialized as matrices of `arma::mat`, then the value of `lowerBound[i]` applies to axis `[i]`; similarly, for values in `upperBound`. This results in an initialization following a uniform distribution in a hyperrectangle within the specified bounds.
+
+Attributes of the optimizer may also be changed via the member methods
+`PopulationSize()`, `MaxGenerations()`, `CrossoverRate()`, `DistributionIndex()`, `Eta()`, `Epsilon()`, `LowerBound()` and `UpperBound()`.
+
+#### Examples
+
+<details open>
+<summary>Click to collapse/expand example code.
+</summary>
+
+```c++
+SchafferFunctionN1<arma::mat> SCH;
+arma::vec lowerBound("-1000");
+arma::vec upperBound("1000");
+AGEMOEA opt(50, 1000, 0.6, 20, 1e-6, 20, lowerBound, upperBound);
+
+typedef decltype(SCH.objectiveA) ObjectiveTypeA;
+typedef decltype(SCH.objectiveB) ObjectiveTypeB;
+
+arma::mat coords = SCH.GetInitialPoint();
+std::tuple<ObjectiveTypeA, ObjectiveTypeB> objectives = SCH.GetObjectives();
+
+// obj will contain the minimum sum of objectiveA and objectiveB found on the best front.
+double obj = opt.Optimize(objectives, coords);
+// Now obtain the best front.
+arma::cube bestFront = opt.ParetoFront();
+```
+
+</details>
+
+<details open>
+<summary>Click to collapse/expand example code.
+</summary>
+
+```c++
+ZDT3<> ZDT_THREE(300);
+const double lowerBound = 0;
+const double upperBound = 1;
+
+AGEMOEA opt(50, 500, 0.8, 20, 1e-6, 20, lowerBound, upperBound);
+typedef decltype(ZDT_THREE.objectiveF1) ObjectiveTypeA;
+typedef decltype(ZDT_THREE.objectiveF2) ObjectiveTypeB;
+bool success = true;
+arma::mat coords = ZDT_THREE.GetInitialPoint();
+std::tuple<ObjectiveTypeA, ObjectiveTypeB> objectives = ZDT_THREE.GetObjectives();
+opt.Optimize(objectives, coords);
+const arma::cube bestFront = opt.ParetoFront();
+
+NSGA2 opt2(50, 5000, 0.5, 0.5, 1e-3, 1e-6, lowerBound, upperBound);
+// obj2 will contain the minimum sum of objectiveA and objectiveB found on the best front.
+double obj2 = opt2.Optimize(objectives, coords);
+
+arma::cube NSGAFront = opt2.ParetoFront();
+// Get the IGD score for NSGA front using AGEMOEA as reference.
+double igd = IGD::Evaluate(NSGAFront, bestFront, 1);
+std::cout << igd << std::endl;
+```
+
+</details>
+
+#### See also:
+
+ * [An adaptive evolutionary algorithm based on non-euclidean geometry for many-objective optimization](https://doi.org/10.1145/3321707.3321839)
+ * [Multi-Objective Optimization in Wikipedia](https://en.wikipedia.org/wiki/Multi-objective_optimization)
+ * [Performance Indicators](#performance-indicators)
+
 ## AMSBound
 
 *An optimizer for [differentiable separable functions](#differentiable-separable-functions).*
@@ -2091,6 +2182,7 @@ arma::cube bestFront = opt.ParetoFront();
 * [MOEA/D-DE Algorithm](https://ieeexplore.ieee.org/document/4633340)
 * [Multi-objective Functions in Wikipedia](https://en.wikipedia.org/wiki/Test_functions_for_optimization#Test_functions_for_multi-objective_optimization)
 * [Multi-objective functions](#multi-objective-functions)
+* [Performance Indicators](#performance-indicators)
 
 ## NSGA2
 
@@ -2157,7 +2249,8 @@ arma::cube bestFront = opt.ParetoFront();
 
  * [NSGA-II Algorithm](https://www.iitk.ac.in/kangal/Deb_NSGA-II.pdf)
  * [Multi-objective Functions in Wikipedia](https://en.wikipedia.org/wiki/Test_functions_for_optimization#Test_functions_for_multi-objective_optimization)
-  * [Multi-objective functions](#multi-objective-functions)
+ * [Multi-objective functions](#multi-objective-functions)
+ * [Performance Indicators](#performance-indicators)
 
 ## OptimisticAdam
 

include/ensmallen.hpp

@@ -66,6 +66,7 @@
 #include "ensmallen_bits/utility/any.hpp"
 #include "ensmallen_bits/utility/arma_traits.hpp"
 #include "ensmallen_bits/utility/indicators/epsilon.hpp"
+#include "ensmallen_bits/utility/indicators/igd.hpp"
 #include "ensmallen_bits/utility/indicators/igd_plus.hpp"
 
 // Contains traits, must be placed before report callback.
@@ -111,6 +112,7 @@
 #include "ensmallen_bits/katyusha/katyusha.hpp"
 #include "ensmallen_bits/lbfgs/lbfgs.hpp"
 #include "ensmallen_bits/lookahead/lookahead.hpp"
+#include "ensmallen_bits/agemoea/agemoea.hpp"
 #include "ensmallen_bits/moead/moead.hpp"
 #include "ensmallen_bits/nsga2/nsga2.hpp"
 #include "ensmallen_bits/padam/padam.hpp"