stan-dev
Handle frequency weights with loo
In some cases models are specified ("compressed") with frequency weights to speed up the fitting, however, this doesn't work well with loo as the Pareto k's indicates that all observations are heavily influencing the posterior, which of course it true on the aggregated level, but may not be true in the disaggregated level. Some kind of adjustment (I suppose disaggregation of the log-likelihood is a part of it) would be needed for such a case.
Hi Staffan, If you have a weighted likelihood e.g. target + = frequency_weight[i]* likelihood [i], then I think you can still treat it as a vanilia exchangebable model--the only difference is to extract this weighted product as the pointwise likelihood and feed into loo. If some frequency_weight[i]≈0, then I would expect a small k hat[i] accordingly.
On Fri, Feb 19, 2021 at 9:45 AM Staffan Betnér [email protected] wrote:
In some cases models are specified ("compressed") with frequency weights to speed up the fitting, however, this doesn't work well with loo as the Pareto k's indicates that all observations are heavily influencing the posterior, which of course it true on the aggregated level, but may not be true in the disaggregated level. Some kind of adjustment (I suppose disaggregation of the log-likelihood is a part of it) would be needed for such a case.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/loo/issues/168, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFQE7254LNWVMFFADYAUI43S7Z2QZANCNFSM4X4PTWQA .
