KIE Community

Thanks to a pull request by Lukas, Planner 5.5.0.Beta1 will support Matrix benchmarking.

Matrix benchmarking is benchmarking a combination of value sets. For example: benchmark 4 planningEntityTabuSize values (5, 7, 11 and 13) combined with 3 minimalAcceptedSelection values (500, 1000 and 2000), resulting in 12 solver configurations.

To reduce the verbosity of such a benchmark configuration, you can use a Freemarker template for the benchmark configuration:

<plannerBenchmark>
  ...

  <inheritedSolverBenchmark>
    ...
  </inheritedSolverBenchmark>

<#list [5, 7, 11, 13] as planningEntityTabuSize>
<#list [500, 1000, 2000] as minimalAcceptedSelection>
  <solverBenchmark>
    <name>entityTabu ${planningEntityTabuSize} acceptedSelection ${minimalAcceptedSelection}</name>
    <solver>
      <localSearch>
        <selector>
          <selector>
            <moveFactoryClass>...CloudComputerChangeMoveFactory</moveFactoryClass>
          </selector>
          <selector>
            <moveFactoryClass>...CloudProcessSwapMoveFactory</moveFactoryClass>
          </selector>
        </selector>
        <acceptor>
          <planningEntityTabuSize>${planningEntityTabuSize}</planningEntityTabuSize>
        </acceptor>
        <forager>
          <minimalAcceptedSelection>${minimalAcceptedSelection}</minimalAcceptedSelection>
        </forager>
      </localSearch>
    </solver>
  </solverBenchmark>
</#list>
</#list>
</plannerBenchmark>

Notice how easy it is to add 7 additional minimalAcceptedSelection values, resulting in 300 solving configurations instead.