Matrix benchmarking in Planner

Thanks to a pull request by Lukas, Planner 5.5.0.Beta1 will support Matrix benchmarking.

Matrix benchmarking is benchmarking a combination of value sets. For example: benchmark 4 planningEntityTabuSize values (5, 7, 11 and 13) combined with 3 minimalAcceptedSelection values (500, 1000 and 2000), resulting in 12 solver configurations.

To reduce the verbosity of such a benchmark configuration, you can use a Freemarker template for the benchmark configuration:

<plannerBenchmark>
...

<inheritedSolverBenchmark>
...
</inheritedSolverBenchmark>

<#list [5, 7, 11, 13] as planningEntityTabuSize>
<#list [500, 1000, 2000] as minimalAcceptedSelection>
<solverBenchmark>
<name>entityTabu ${planningEntityTabuSize} acceptedSelection ${minimalAcceptedSelection}</name>
<solver>
<localSearch>
<selector>
<selector>
<moveFactoryClass>...CloudComputerChangeMoveFactory</moveFactoryClass>
</selector>
<selector>
<moveFactoryClass>...CloudProcessSwapMoveFactory</moveFactoryClass>
</selector>
</selector>
<acceptor>
<planningEntityTabuSize>${planningEntityTabuSize}</planningEntityTabuSize>
</acceptor>
<forager>
<minimalAcceptedSelection>${minimalAcceptedSelection}</minimalAcceptedSelection>
</forager>
</localSearch>
</solver>
</solverBenchmark>
</#list>
</#list>
</plannerBenchmark>

Notice how easy it is to add 7 additional minimalAcceptedSelection values, resulting in 300 solving configurations instead.

Author

Comments are closed.