feat: Variational expectations.
Feature Request
GPJax's SVGP computes variational expectations $\int \text{d} q(f(x_i)) \log( p(y_i|f(x_i))$ via Gauss-Hermite quadrature.
This is superfluous in cases e.g., of a Gaussian likelihood when the integrand is known is closed form. It would be beneficial to override this automatically.
Additionally, it would nice to offer alternatives such as Monte Carlo.
Questions:
- When the integrand is known in closed form, where should we put it?
- Could we define the integration strategy when initialising
StochasticVI?
@thomaspinder Resolving the Gaussian case for the analytical variational expectation, would allow us to test the collapsed bound agrees with Natural gradients after one gradient update with fixed hyper-parameters (#90). Currently, both bounds almost agree, but differ due to the quadrature approximation.
Closing in lieu of #270