org.moeaframework
Class Analyzer

java.lang.Object
  org.moeaframework.Analyzer

public class Analyzer
extends Object

Performs basic end-of-run analysis. For example, the following demonstrates its typical use. First construct and configure the analyzer:

   Analyzer analyzer = new Analyzer()
       .withProblem("DTLZ2_2")
       .includeGenerationalDistance()
       .includeInvertedGenerationalDistance()
       .includeAdditiveEpsilonIndicator()
       .includeContribution()
       .showAggregate()
       .showStatisticalSignificance();
 

The problem must always be specified. Next, add the data to be analyzed:

   Executor executor = new Executor().withProblem("DTLZ2_2");
   add("NSGAII", executor.withAlgorithm("NSGAII").run());
   add("eMOEA", executor.withAlgorithm("eMOEA").run());
 

Lastly, print the results of the analysis:

   analyzer.printAnalysis();
 

The output produced is compatible with the YAML format, and thus can be postprocessed easily with any YAML parser.

Constructor Summary

Analyzer()
Constructs a new analyzer initialized with default settings.

Method Summary

Analyzer	add(String name, NondominatedPopulation result) Adds a new sample with the specified name.
Analyzer	addAll(String name, Collection<NondominatedPopulation> results) Adds the collection of new samples with the specified name.
Analyzer	clear() Clears all data stored in this analyzer.
NondominatedPopulation	getReferenceSet() Returns the reference set used by this analyzer.
Analyzer	includeAdditiveEpsilonIndicator() Enables the evaluation of the additive ε-indicator metric.
Analyzer	includeAllMetrics() Enables the evaluation of all metrics.
Analyzer	includeContribution() Enables the evaluation of the contribution metric.
Analyzer	includeGenerationalDistance() Enables the evaluation of the generational distance metric.
Analyzer	includeHypervolume() Enables the evaluation of the hypervolume metric.
Analyzer	includeInvertedGenerationalDistance() Enables the evaluation of the inverted generational distance metric.
Analyzer	includeMaximumParetoFrontError() Enables the evaluation of the maximum Pareto front error metric.
Analyzer	includeR1() Enables the evaluation of the R1 indicator.
Analyzer	includeR2() Enables the evaluation of the R2 indicator.
Analyzer	includeR3() Enables the evaluation of the R3 indicator.
Analyzer	includeSpacing() Enables the evaluation of the spacing metric.
Analyzer	loadAs(String name, File resultFile) Loads the samples stored in a result file using ResultFileReader.
Analyzer	loadData(File directory, String prefix, String suffix) Loads data into this analyzer, which was previously saved using saveData(File, String, String) with matching arguments.
Analyzer	printAnalysis() Prints the analysis of all data recorded in this analyzer to standard output.
Analyzer	printAnalysis(PrintStream ps) Prints the analysis of all data recorded in this analyzer.
Analyzer	saveAnalysis(File file) Saves the analysis of all data recorded in this analyzer to the specified file.
Analyzer	saveAs(String name, File resultFile) Saves the samples to a result file using ResultFileWriter.
Analyzer	saveData(File directory, String prefix, String suffix) Saves all data stored in this analyzer, which can subsequently be read using loadData(File, String, String) with matching arguments.
Analyzer	saveReferenceSet(File file) Saves the reference set to the specified file.
Analyzer	showAggregate() Enables the output of the metric value of the aggregate approximation set, produced by merging all individual seeds.
Analyzer	showAll() Enables the output of all analysis results.
Analyzer	showIndividualValues() Enables the output of individual metric values for each seed.
Analyzer	showStatistic(org.apache.commons.math3.stat.descriptive.UnivariateStatistic statistic) Specifies the UnivariateStatistics calculated during the analysis.
Analyzer	showStatisticalSignificance() Enables the output of statistical significance tests.
Analyzer	usingProblemFactory(ProblemFactory problemFactory) Sets the problem factory used by this builder.
Analyzer	withEpsilon(double... epsilon) Sets the ε values used by this builder, specifying the archive returned by newArchive().
Analyzer	withProblem(String problemName) Sets the problem used by this builder.
Analyzer	withProblemClass(Class<?> problemClass, Object... problemArguments) Sets the problem used by this builder.
Analyzer	withProblemClass(String problemClassName, Object... problemArguments) Sets the problem used by this builder.
Analyzer	withReferenceSet(File referenceSetFile) Sets the file containing the reference set to be used by this builder.
Analyzer	withSameProblemAs(org.moeaframework.ProblemBuilder builder) Configures this builder to use the same problem, reference set, ε values and ProblemFactory as the specified instance.
Analyzer	withSignifianceLevel(double significanceLevel) Sets the level of significance used when testing the statistical significance of observed differences in the medians.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Analyzer

public Analyzer()

Constructs a new analyzer initialized with default settings.

Method Detail

withSameProblemAs

public Analyzer withSameProblemAs(org.moeaframework.ProblemBuilder builder)

Configures this builder to use the same problem, reference set, ε values and ProblemFactory as the specified instance.

Parameters:

builder - the instance whose problem, reference set, ε values and ProblemFactory are to be copied

Returns:

a reference to this builder

usingProblemFactory

public Analyzer usingProblemFactory(ProblemFactory problemFactory)

Sets the problem factory used by this builder.

Parameters:

problemFactory - the problem factory

Returns:

a reference to this builder

withProblem

public Analyzer withProblem(String problemName)

Sets the problem used by this builder.

Parameters:

problemName - the problem name

Returns:

a reference to this builder

withProblemClass

public Analyzer withProblemClass(Class<?> problemClass,
                                 Object... problemArguments)

Sets the problem used by this builder. Instances of the problem are created using the Class.newInstance() method. Ensure the problem class satisfies all the conditions of newInstance(), otherwise exceptions may be thrown when attempting to create an instance of the problem.

Parameters:

problemClass - the problem class

problemArguments - the arguments passed to the problem constructor

Returns:

a reference to this builder

withProblemClass

public Analyzer withProblemClass(String problemClassName,
                                 Object... problemArguments)
                          throws ClassNotFoundException

Sets the problem used by this builder. Instances of the problem are created using the Class.newInstance() method. Ensure the problem class satisfies all the conditions of newInstance(), otherwise exceptions may be thrown when attempting to create an instance of the problem.

Parameters:

problemClassName - the problem class name

problemArguments - the arguments passed to the problem constructor

Returns:

a reference to this builder

Throws:

ClassNotFoundException - if the specified problem class name could not be found

withEpsilon

public Analyzer withEpsilon(double... epsilon)

Sets the ε values used by this builder, specifying the archive returned by newArchive().

Parameters:

epsilon - the ε values

Returns:

a reference to this builder

withReferenceSet

public Analyzer withReferenceSet(File referenceSetFile)

Sets the file containing the reference set to be used by this builder. If not specified, the reference set should be aggregated from all individual approximation sets.

Parameters:

referenceSetFile - the reference set file

Returns:

a reference to this builder

includeHypervolume

public Analyzer includeHypervolume()

Enables the evaluation of the hypervolume metric.

Returns:

a reference to this analyzer

includeGenerationalDistance

public Analyzer includeGenerationalDistance()

Enables the evaluation of the generational distance metric.

Returns:

a reference to this analyzer

includeInvertedGenerationalDistance

public Analyzer includeInvertedGenerationalDistance()

Enables the evaluation of the inverted generational distance metric.

Returns:

a reference to this analyzer

includeAdditiveEpsilonIndicator

public Analyzer includeAdditiveEpsilonIndicator()

Enables the evaluation of the additive ε-indicator metric.

Returns:

a reference to this analyzer

includeMaximumParetoFrontError

public Analyzer includeMaximumParetoFrontError()

Enables the evaluation of the maximum Pareto front error metric.

Returns:

a reference to this analyzer

includeSpacing

public Analyzer includeSpacing()

Enables the evaluation of the spacing metric.

Returns:

a reference to this analyzer

includeContribution

public Analyzer includeContribution()

Enables the evaluation of the contribution metric.

Returns:

a reference to this analyzer

includeR1

public Analyzer includeR1()

Enables the evaluation of the R1 indicator.

Returns:

a reference to this analyzer

includeR2

public Analyzer includeR2()

Enables the evaluation of the R2 indicator.

Returns:

a reference to this analyzer

includeR3

public Analyzer includeR3()

Enables the evaluation of the R3 indicator.

Returns:

a reference to this analyzer

includeAllMetrics

public Analyzer includeAllMetrics()

Enables the evaluation of all metrics.

Returns:

a reference to this analyzer

showAll

public Analyzer showAll()

Enables the output of all analysis results.

Returns:

a reference to this analyzer

showIndividualValues

public Analyzer showIndividualValues()

Enables the output of individual metric values for each seed.

Returns:

a reference to this analyzer

showAggregate

public Analyzer showAggregate()

Enables the output of the metric value of the aggregate approximation set, produced by merging all individual seeds.

Returns:

a reference to this analyzer

showStatisticalSignificance

public Analyzer showStatisticalSignificance()

Enables the output of statistical significance tests. If enabled, it is necessary to record multiple seeds for each entry.

Returns:

a reference to this analyzer

showStatistic

public Analyzer showStatistic(org.apache.commons.math3.stat.descriptive.UnivariateStatistic statistic)

Specifies the UnivariateStatistics calculated during the analysis. If none are specified by the user, then Min, Median and Max are used.

Parameters:

statistic - the statistic to calculate

Returns:

a reference to this analyzer

withSignifianceLevel

public Analyzer withSignifianceLevel(double significanceLevel)

Sets the level of significance used when testing the statistical significance of observed differences in the medians. Commonly used levels of significance are 0.05 and 0.01.

Parameters:

significanceLevel - the level of significance

Returns:

a reference to this analyzer

addAll

public Analyzer addAll(String name,
                       Collection<NondominatedPopulation> results)

Adds the collection of new samples with the specified name.

Parameters:

name - the name of these samples

results - the approximation sets

Returns:

a reference to this analyzer

add

public Analyzer add(String name,
                    NondominatedPopulation result)

Adds a new sample with the specified name. If multiple samples are added using the same name, each sample is treated as an individual seed. Analyses can be performed on both the individual seeds and aggregates of the seeds.

Parameters:

name - the name of this sample

result - the approximation set

Returns:

a reference to this analyzer

saveData

public Analyzer saveData(File directory,
                         String prefix,
                         String suffix)
                  throws IOException

Saves all data stored in this analyzer, which can subsequently be read using loadData(File, String, String) with matching arguments.

Parameters:

directory - the directory in which the data is stored

prefix - the prefix for filenames

suffix - the suffix (extension) for filenames

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

loadData

public Analyzer loadData(File directory,
                         String prefix,
                         String suffix)
                  throws IOException

Loads data into this analyzer, which was previously saved using saveData(File, String, String) with matching arguments.

Parameters:

directory - the directory in which the data is stored

prefix - the prefix for filenames

suffix - the suffix (extension) for filenames

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

loadAs

public Analyzer loadAs(String name,
                       File resultFile)
                throws IOException

Loads the samples stored in a result file using ResultFileReader.

Parameters:

name - the name of the samples

resultFile - the result file to load

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

saveAs

public Analyzer saveAs(String name,
                       File resultFile)
                throws IOException

Saves the samples to a result file using ResultFileWriter. If name is null, the reference set is saved. Otherwise, the approximation sets for the named entries are saved.

Parameters:

name - the name of the samples

resultFile - the result file to which the data is saved

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

saveAnalysis

public Analyzer saveAnalysis(File file)
                      throws IOException

Saves the analysis of all data recorded in this analyzer to the specified file.

Parameters:

file - the file to which the analysis is saved

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

printAnalysis

public Analyzer printAnalysis()
                       throws IOException

Prints the analysis of all data recorded in this analyzer to standard output.

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

saveReferenceSet

public Analyzer saveReferenceSet(File file)
                          throws IOException

Saves the reference set to the specified file.

Parameters:

file - the file to which the reference set is saved

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

See Also:

getReferenceSet()

getReferenceSet

public NondominatedPopulation getReferenceSet()

Returns the reference set used by this analyzer. The reference set is generated as follows:

1. If withReferenceSet(File) has been set, the contents of the reference set file are returned;
2. If the problem factory provides a reference set via the ProblemFactory.getReferenceSet(String) method, this reference set is returned;
3. Otherwise, the reference set is aggregated from all individual approximation sets.

Returns:

the reference set used by this analyzer

Throws:

IllegalArgumentException - if the reference set could not be loaded

printAnalysis

public Analyzer printAnalysis(PrintStream ps)
                       throws IOException

Prints the analysis of all data recorded in this analyzer.

Parameters:

ps - the stream to which the analysis is written

Returns:

a reference to this analyzer

Throws:

IOException - if an I/O error occurred

clear

public Analyzer clear()

Clears all data stored in this analyzer.

Returns:

a reference to this analyzer