[GENERAL SUPPORT]: Does Ax normalise scores

[HOLL95]

Question

When performing multiobjective optimisation on metrics with different orders of magnitude, does Ax normalise the objective values when computing the hypervolume? If not, is there a recommendation for objectives of different magnitudes?

Please provide any relevant code snippet if applicable.

Code of Conduct

• [x] I agree to follow this Ax's Code of Conduct

[mpolson64]

Hi @HOLL95 -- sorry to get back to you later than I intended. Ax does not normalize the objective values, but this shouldn't matter and you will be completely fine using objectives of different magnitude; maximizing hypervolume expected improvement (and all multiobjective optimization in Ax) is effectively scale free.

That being said, one thing that is almost always worth doing in real world applications of MOO is setting a "reference point" by providing an outcome constraint on each of your objectives, directing Ax to avoid pursuing any solutions which will provide outcomes worse than some floor in a given dimension.

For example: say I'm optimizing hyperparameters for an ML model for inference speed and accuracy. By placing a constraint on both of these outcomes I can prevent Ax from suggesting parameters which might provide a horribly slow but very accurate model or a lightning fast model with awful accuracy. You provide these constraints by passing them into Client.configure_optimization as follows:

client = Client()

client.configure_experiment(...)
client.configure_optimization(
    objective="acc, speed",
    outcome_constraints=["acc >= 0.8", "speed >= 1000"],
)

Let me know if you have any further questions about MOO or Ax more generally.

[HOLL95]

Thanks for the response! Would you mind providing a reference as to why? My (very surface level) assumption was that the hypervolume would be dominated by larger objectives, and proposing a point that improves the objective "speed" by 0.1% would expand the hypervolume more than an improvement of 90% in the objective "accuracy".

Cheers