Questions about r2_loo()

[jmgirard]

1. How concerned should I be about Pareto k warnings from r2_loo()?
2. Is there a way to tweak the LOO settings within {performance} to address this or would I have to do the LOO separately (e.g., through brms::loo()) and then plug the results into some equation?
3. Is there a way to apply LOO adjustment to the marginal R^2 in a mixed model?

[strengejacke]

1. Not 100% sure, I would point you to help('pareto-k-diagnostic') .
2. We could add a ... argument that is passed down to loo::E_loo(). I must check, if e.g. threshold would resolve the warning.
3. I would need to look deeper into the code. After quick look at r2_loo(), I would say it is possible to do this (not yet, but could be implemented). However, the approach I have in mind conflicts with #153. So I don't think this can be addressed before #153 is resolved.

[jmgirard]

Thanks as always.

1. I'll dig into the literature to try to determine how concerned to be about Pareto k diagnostics.
2. Note that I re-ran the {brms} model with save_pars = save_pars(all = TRUE) and then brms::loo(fit, moment_match = TRUE) resolved the issue, shrinking all the k values to below 0.7 (and therefore "ok" instead of "bad"). Not sure if loo::E_loo() has a similar argument, but worth looking into.
3. Thank you! I really like the marginal/conditional R^2 distinction and would love to adjust the former with LOO.