Thang Bui

Enhancement: rewriting the mutual information as in BALD [Houlsby et al 2011]

1

[In discussion with Jose Miguel Hernandez-Lobato @jmhernandezlobato and Daniel Hernandez-Lobato @danielhernandezlobato] The current exploration objective used in the paper is a sum of expected reductions in entropy of the parameters...

Approximate Power-EP for Deep GPs with intermediate variables for density models/latent variable models

2

Related to #7, need to figure out a good initialisation scheme

save/load models

1

Is saving/loading just the hyperparameters/parameters by calling get_hypers()/update_hypers() enough? also create an example for doing so

Investigating ways to parameterise the variational approximations in the uncollapsed variational bound or approximate Power-EP energy, in particular q(u) = N(u; 0, LL^T) or q(u) = N_{natural}(u; \theta_1; Kuu^{-1} +...

It seems that SDGPR fit is just a flat line for 3, 4 or less training points (in 1D), whereas SGPR fit is fine.

Quadrature/Monte Carlo

2

An alternative to the currently implemented Gaussian projection and the linearisation technique in #5 is a quadrature/Monte Carlo/particle based approach. Need code to implement this and comparison to moment matching...

Make sure that the code passes pep8 standard

Frigola et al's VI for GPSSMs

1

Also make sure that the results for a mean-field version of this is the same with (approx.) Power-EP solutions when alpha -> 0.

Make sure that the gradients are correct and the variance becomes smaller when the batch size gets larger. For each model, write an example use case.

What is the common way to handle invalid updates? how to automatically control damping? is using different damping factors for updates of different variables, a sensible scheme?