Quality indicators
- Author:
Antonio J. Nebro
- Version:
1.0
- Date:
2022-12-12
Quality indicators are entities that typically measure the convergence and/or the diversity Pareto front approximations, and most of
them require a reference Pareto front to be applied. These indicators are located in package org.uma.jmetal.qualityindicator
(sub-project jmetal-core
), and all of them implement the QualityIndicator
abstract class:
public abstract class QualityIndicator {
protected double[][] referenceFront ;
protected QualityIndicator(double[][] referenceFront) {
Check.notNull(referenceFront);
this.referenceFront = referenceFront;
}
public abstract double compute(double[][] front) ;
public abstract boolean isTheLowerTheIndicatorValueTheBetter();
...
}
It is worth noticing than both the fronts are represented as matrices, which will contain objectives values, and the code implementing the indicators does not use any jMetal dependence, and it is assumed that the fronts are normalized before computing the indicators. All the quality indicators have to implement the isTheLowerTheIndicatorValueTheBetter()
method, which is required to compare the indicator values of two or more fronts.
The list of available quality indicators is the following:
EP
: Additive epsilonER
: Error ratioSPREAD
: Spread (two objectives)GSPREAD
: Generalized spread (more than two objectives)GD
: Generational distanceIGD
: Inverted generational distanceIGD+
: Inverted generational distance plusSC
: Set coverageHV
: HypervolumeNHV
: Normalized hypervolume
Example of use
The following code snippet shows the typical way of computing a quality indicator (we use the additive epsilon as an example):
double[][] normalizedReferenceFront = NormalizeUtils.normalize(referenceFront);
double[][] normalizedFront =
NormalizeUtils.normalize(
front,
NormalizeUtils.getMinValuesOfTheColumnsOfAMatrix(referenceFront),
NormalizeUtils.getMaxValuesOfTheColumnsOfAMatrix(referenceFront));
Epsilon epsilon = new Epsilon(normalizedReferenceFront) ;
double indicatorValue = epsilon.compute(normalizedFront) ;
A typical scenario involves finding a Pareto front approximation and compute a quality indicator but the reference front is stored in a file. In this case, an utility function is readVectors()
:
double[][]referenceFront = VectorUtils.readVectors(referenceParetoFront, ",")
In this code, we assume the file containing the front has a CSV format.
Computing quality indicators from the command line
If you need to compute the value of a quality indicator of a front of solutions from the command line you can use the
CommandLineQualityIndicatorTool
. We assume the following:
The current working directory is the root of jMetal.
The obtained front is stored in a file calle
FUN.csv
.The solved problem is ZDT1, whose reference front is in
resources/referenceFrontsCSV/ZDT1.csv
.The instruction
mvn package
(ormvn package -DskipTests=True
) has been executed.
The list of available quality indicators can be obtained by calling the class without parameters:
% java -cp jmetal-core/target/jmetal-core-6.0-jar-with-dependencies.jar org.uma.jmetal.utilities.CommandLineQualityIndicatorTool
INFO: Loggers configured with null [org.uma.jmetal.util.JMetalLogger configureLoggers]
INFO: Parameters: indicatorName referenceFrontFile frontFile
Where indicatorValue can be one of these:
EP - Additive epsilon
GD - Generational distance
IGD - Inverted generational distance
IGD+ - Inverted generational distance plus
HV - Hypervolume
NHV - Normalized hypervolume
ER - Error ratio
SP - Spread (two objectives)
GSPREAD - Generalized Spread (more than two objectives)
SC - Set coverage
[ org.uma.jmetal.utilities.CommandLineQualityIndicatorTool printAvailableIndicators]
Exception in thread "main" org.uma.jmetal.util.errorchecking.JMetalException: Invalid arguments
We can compute a single indicator, as the inverted generational distance:
% java -cp jmetal-core/target/jmetal-core-6.0-jar-with-dependencies.jar org.uma.jmetal.utilities.CommandLineQualityIndicatorTool IGD FUN.csv resources/referenceFrontsCSV/ZDT1.csv
INFO: Loggers configured with null [org.uma.jmetal.util.JMetalLogger configureLoggers]
INFO: The fronts are normalized before computing the indicators [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: 2.2536457240826101E-4 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
Or all the quality indicators if we use ALL
as indicator:
% java -cp jmetal-core/target/jmetal-core-6.0-jar-with-dependencies.jar org.uma.jmetal.utilities.CommandLineQualityIndicatorTool ALL FUN.csv resources/referenceFrontsCSV/ZDT1.csv
INFO: Loggers configured with null [org.uma.jmetal.util.JMetalLogger configureLoggers]
INFO: The fronts are normalized before computing the indicators [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: EP: 7.098510362065962E-4 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: HV: 0.6665198715008906 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: NHV: -0.009962058513842686 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: GD: 1.933259493087464E-4 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: IGD: 2.2536457240826101E-4 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: IGD+: 2.3576944009587076E-5 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: SP: 0.27852463257758625 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: GSPREAD: 0.2780440568993888 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: SC(refFront, front): 0.0 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]
INFO: SC(front, refFront): 0.9090909090909091 [org.uma.jmetal.utilities.CommandLineQualityIndicatorTool calculateAndPrintIndicators]