feat: Adding several acquisition functions

[miguelgondu]

Adding several acquisition functions

Currently we have Thompson Sampling and Probability of Improvement as acquisition functions. It would be ideal to include

• [x] Expected Improvement (EI).
• [ ] Upper Confidence Bound (UCB).
• [ ] Log-Probability of Improvement (logPI).
• [ ] Log-Expected Improvement (logEI).

Describe Preferred Solution

These acquisition functions could naturally fit inside the decision making module. GPJax already has the core ingredients for e.g. acquisition function maximization using L-BFGS, and BO loops using utility-driven decision making.

Additional Context

Here are some relevant references we could use for implementing these acquisition functions:

• Taking the human out of the loop: a review of BO, by Shahriari et al., in which we can find formulas for EI and UCB.
• Unexpected Improvements to Expected Improvement for Bayesian Optimization, by Ament et al. The authors describe a numerically estable way of computing logEI.

If the feature request is approved, would you be willing to submit a PR?

Yes, I am glad to contribute implementations of e.g. UCB and logEI.

[daniel-dodd]

Thanks @miguelgondu - this would make an excellent contribution!

[github-actions[bot]]

There has been no recent activity on this issue. To keep our issues log clean, we remove old and inactive issues. Please update to the latest version of GPJax and check if that resolves the issue. Let us know if that works for you by leaving a comment. This issue is now marked as stale and will be closed if no further activity occurs. If you believe that this is incorrect, please comment. Thank you!

[github-actions[bot]]

There has been no activity on this PR for some time. Therefore, we will be automatically closing the PR if no new activity occurs within the next seven days. Thank you for your contributions.