GPJax icon indicating copy to clipboard operation
GPJax copied to clipboard

feat: Variational expectations.

Open daniel-dodd opened this issue 3 years ago • 2 comments

Feature Request

GPJax's SVGP computes variational expectations $\int \text{d} q(f(x_i)) \log( p(y_i|f(x_i))$ via Gauss-Hermite quadrature.

This is superfluous in cases e.g., of a Gaussian likelihood when the integrand is known is closed form. It would be beneficial to override this automatically.

Additionally, it would nice to offer alternatives such as Monte Carlo.

daniel-dodd avatar Aug 20 '22 14:08 daniel-dodd

Questions:

  1. When the integrand is known in closed form, where should we put it?
  2. Could we define the integration strategy when initialising StochasticVI?

daniel-dodd avatar Aug 20 '22 14:08 daniel-dodd

@thomaspinder Resolving the Gaussian case for the analytical variational expectation, would allow us to test the collapsed bound agrees with Natural gradients after one gradient update with fixed hyper-parameters (#90). Currently, both bounds almost agree, but differ due to the quadrature approximation.

daniel-dodd avatar Aug 20 '22 14:08 daniel-dodd

Closing in lieu of #270

thomaspinder avatar May 18 '23 20:05 thomaspinder