Skip to content

Conversation

fweber144
Copy link

Following upon stan-dev/projpred#311, this PR shows how to adapt the "body fat" case study code so that it can be used with the most recent CRAN version of projpred (v2.1.1). Note that I did not run the code from bodyfat_bootstrap.R, bodyfat_kfoldcv.R, and bodyfat_kfoldcv2.R because they are probably going to take very long. For the same reason, I also did not run the second cv_varsel() call in bodyfat.Rmd. I also only looked at the "body fat" case study. Other case studies probably have to be adapted, too.

I also noticed uses of nloo in bodyfat_kfoldcv.R and bodyfat_kfoldcv2.R which were probably not doing what they were supposed to. I now switched to projpred's natively supported K-fold CV because I guess that's what those lines were supposed to do.

See the commit messages for details, but feel free to ask if you need more information.

fweber144 added 11 commits May 10, 2022 16:41
experimental in projpred (see stan-dev/projpred#94). Setting `nloo = n` should not change results compared to omitting argument `nloo`, but I guess it's better not to point to an experimental feature where not needed.
projpred v2.1.0 back to the matrix that `fitrhs_cvvs[["pct_solution_terms_cv"]]` delivered before.
```
Warning message:
Continuous limits supplied to discrete scale.
Did you mean `limits = factor(...)` or `scale_*_continuous()`?
```
…s not

need to be set after a call to `set.seed()`, but I add it here so that the `cv_varsel()` command can be re-run easily without having to re-run all the code from the previous `set.seed()` call on).
introduces a "TODO" with respect to `nloo`.
introduces a "TODO" with respect to `nloo`.
implemented in projpred. This resolves the `nloo` "TODO" added earlier.
implemented in projpred. This resolves the `nloo` "TODO" added earlier.
@fweber144
Copy link
Author

PR stan-dev/projpred#471 implements what was missing for lines https://github.com/fweber144/modelselection/blob/9e65616c6bf9e234747b2e97a3f602cb5c264c92/bodyfat_kfoldcv.R#L29-L32. If I have the time, I can try to make use of the new performances() function in those lines.

Changes not related to projpred v2.8.0:

1) Object `df` (files `bodyfat_bootstrap.R`, `bodyfat_kfoldcv.R`, and
`bodyfat_kfoldcv2.R`) is modified for consistency with file `bodyfat.Rmd`.

2) Objects consisting of a model formula are coerced to `formula`s because
otherwise, `projpred:::get_refmodel.stanreg()` throws an error.
@fweber144
Copy link
Author

If I have the time, I can try to make use of the new performances() function in those lines.

Done now in commit 1f010f1 (with projpred 2.8.0 which has just been released on CRAN). See the commit message of that commit for some more details.

@fweber144 fweber144 changed the title Update "body fat" to projpred v2.1.1 Update "body fat" to projpred v2.8.0 Dec 15, 2023
@avehtari
Copy link
Owner

avehtari commented Sep 7, 2024

Uh, I never received any notification about this. I'll check this in next couple weeks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants