parameters icon indicating copy to clipboard operation
parameters copied to clipboard

`select_parameters()` outdated with respect to projpred

Open fweber144 opened this issue 4 years ago • 4 comments

It seems like select_parameters.stanreg() is outdated with respect to projpred:

  1. Line https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L25 uses element beta which doesn't exist in newer projpred versions anymore. I don't know when beta was removed exactly, but it doesn't exist in the current CRAN version 2.0.2 (from October 2020) anymore. To access the projected parameter draws in projpred, the as.matrix.projection() method can be used.
  2. Line https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L24 uses argument nv which doesn't exist in newer projpred versions anymore (at least in the current CRAN version 2.0.2 from October 2020, it doesn't exist anymore). This argument was replaced by a new argument called nterms.

Apart from that, I noticed that parameters doesn't make use of the projected parameter draws: Lines https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L25-L32 show that only the names of the predictors from the selected submodel are used, not their projected draws. I don't know if this was done on purpose or if it was even necessary to make projpred-based selections fit into the parameters workflow, but in principle, it would be preferable to use the projected parameter draws instead of refitting the model with only the selected predictors. Refitting the model with only the selected predictors throws away uncertainty inherent to the reference model, thereby leading to improper post-selection inference. Using the projected parameter draws after selection is even one of the key steps of the projpred workflow, I would say.

fweber144 avatar Mar 31 '22 10:03 fweber144

@DominiqueMakowski Currently, select_parameters() doesn't work anymore. I'm, not sure how it worked or how it is supposed to work now. Can you look at this?

strengejacke avatar Apr 02 '22 16:04 strengejacke

@fweber144 thanks for letting us know! Since you seem to have a pretty good understanding of it, would you be interested maybe in maybe opening a PR to make some fixes to select_parameters()?

DominiqueMakowski avatar Apr 03 '22 00:04 DominiqueMakowski

Sorry, I currently don't have time for this. But I hope my comment above helps in fixing this.

fweber144 avatar Apr 03 '22 19:04 fweber144

I moved the files for stanreg and brmsfit objects into the WIP folder for now...

strengejacke avatar Apr 04 '22 06:04 strengejacke