Add support for sigma=0 in normally distributed hyperparameters

[nchristensen]

Moving this to separate pull request per #280

This pull request does mainly two things:

1. It loosens the type restriction to allow floating type values for mu. For instance mu=4.5 might be chosen if one desires four and five to be chosen with equal probability. This required changing default_value to be mu rounded to the nearest integer when not otherwise specified.
2. It allows (or rather handles more correctly) sigma=0 in NormalIntegerHyperparameter objects. This means the distribution is a Dirac delta function and the only value returned from the distribution is the mean (mu). If the mu is not an integer then an error is raised during instantiation.

[codecov[bot]]

Codecov Report

Base: 67.97% // Head: 67.97% // No change to project coverage :thumbsup:

Coverage data is based on head (88051eb) compared to base (88051eb). Patch has no changes to coverable lines.

:exclamation: Current head 88051eb differs from pull request most recent head a8b1d6d. Consider uploading reports for the commit a8b1d6d to get more accurate results

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #285   +/-   ##
=======================================
  Coverage   67.97%   67.97%           
=======================================
  Files          25       25           
  Lines        1786     1786           
=======================================
  Hits         1214     1214           
  Misses        572      572           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[eddiebergman]

Hi, this PR is out of date and will entirely be so once #346 is merged. As a result, I will close this but thank you for the PR!

However I was still wondering why there's a need for a dirac delta function, given this is equivalent to a Constant?

[nchristensen]

The idea was to autotune with ytopt using certain fixed values that might change later on. For example, autotuning the multiplication of a 32 x 32 matrix and a 32 x 1000 matrix at first and then later autotuning the multiplication of a 32 x 32 and 32 x 5000 matrix (but using the data from the first case to inform initial model).

I think I tried using Constant, but it either ConfigSpace or ytopt didn't like it when I changed the value of the Constant between executions. I ended up taking a different approach (essentially, only tuning the largest matrix dimensions).

[eddiebergman]

Thanks for the use case! Would it ha e helped if you could easily copy and modify a Configuration space between executions? A lot of ConfigSpace relies on hyperparameter's attributes remaining fixed but there's no real API for mutating spaces or hyperparameter's which I guess might have helped here

[nchristensen]

Yeah, that likely would have been helpful.