Thanks to a pull request by Lukas, Planner 5.5.0.Beta1 will support Matrix benchmarking.
Matrix benchmarking is benchmarking a combination of value sets. For example: benchmark 4 planningEntityTabuSize
values (5
, 7
, 11
and 13
) combined with 3 minimalAcceptedSelection
values (500
, 1000
and 2000
), resulting in 12 solver configurations.
To reduce the verbosity of such a benchmark configuration, you can use a Freemarker template for the benchmark configuration:
<plannerBenchmark>
...
<inheritedSolverBenchmark>
...
</inheritedSolverBenchmark>
<#list [5, 7, 11, 13] as planningEntityTabuSize>
<#list [500, 1000, 2000] as minimalAcceptedSelection>
<solverBenchmark>
<name>entityTabu ${planningEntityTabuSize} acceptedSelection ${minimalAcceptedSelection}</name>
<solver>
<localSearch>
<selector>
<selector>
<moveFactoryClass>...CloudComputerChangeMoveFactory</moveFactoryClass>
</selector>
<selector>
<moveFactoryClass>...CloudProcessSwapMoveFactory</moveFactoryClass>
</selector>
</selector>
<acceptor>
<planningEntityTabuSize>${planningEntityTabuSize}</planningEntityTabuSize>
</acceptor>
<forager>
<minimalAcceptedSelection>${minimalAcceptedSelection}</minimalAcceptedSelection>
</forager>
</localSearch>
</solver>
</solverBenchmark>
</#list>
</#list>
</plannerBenchmark>
Notice how easy it is to add 7 additional minimalAcceptedSelection values, resulting in 300 solving configurations instead.