remove dataset and datasets (#854)

* remove dataset and datasets

The dataset functionality is now housed in MixedModelsDatasets. By fully separating out this functionality, we set ourselves up for easier and lighter weight upgrades to which and how we store datasets.

* NEWS

Author: palday

NEWS.md

@@ -8,6 +8,7 @@ MixedModels v5.0.0 Release Notes
 - Internal code around the default optimizer has been restructured. In particular, the NLopt backend has been moved to a submodule, which will make it easier to move it to an extension if we promote another backend to the default. [#853]
 - Internal code around optimization in profiling has been restructuring so that fitting done during calls to `profile` respect the `backend` and `optimizer` settings. [#853]
 - The `prfit!` convenience function has been removed. [#853]
+- The `dataset` and `datasets` functions have been removed. They are now housed in `MixedModelsDatasets`.[#854]
 
 MixedModels v4.38.0 Release Notes
 ==============================

benchmark/benchmarks.jl

@@ -1,5 +1,5 @@
 using BenchmarkTools, MixedModels
-using MixedModels: dataset
+using MixedModelsDatasets: dataset
 
 const SUITE = BenchmarkGroup()
 

docs/Project.toml

@@ -9,6 +9,7 @@ ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
 FreqTables = "da1fdf0e-e0ff-5433-a45f-9bb5ff651cb1"
 Gadfly = "c91e804a-d5a3-530f-b6f0-dfbca275c004"
 MixedModels = "ff71e718-51f3-5ec2-a782-8ffcbfa3c316"
+MixedModelsDatasets = "7e9fb7ac-9f67-43bf-b2c8-96ba0796cbb6"
 ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 StatsAPI = "82ae8749-77ed-4fe6-ae5f-f523153014b0"

docs/src/GaussHermite.md

@@ -73,7 +73,7 @@ The definition of `MixedModels.GHnorm` is similar to the `gausshermitenorm` func
 GHnorm
 ```
 ```@example Main
-using MixedModels
+using MixedModels, MixedModelsDatasets
 GHnorm(3)
 ```
 
@@ -100,7 +100,7 @@ Several covariates were recorded including the woman's age (centered at the mean
 The version of the data used here is that used in review of multilevel modeling software conducted by the Center for Multilevel Modelling, currently at University of Bristol (http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-rev.html).
 These data are available as the `:contra` dataset.
 ```@example Main
-contra = DataFrame(MixedModels.dataset(:contra))
+contra = DataFrame(MixedModelsDatasets.dataset(:contra))
 describe(contra)
 ```
 

docs/src/bootstrap.md

@@ -27,12 +27,12 @@ package for [`R`](https://www.r-project.org) is fit by
 ```@example Main
 using DataFrames
 using Gadfly          # plotting package
-using MixedModels
+using MixedModels, MixedModelsDatasets
 using Random
 ```
 
 ```@example Main
-dyestuff = MixedModels.dataset(:dyestuff)
+dyestuff = MixedModelsDatasets.dataset(:dyestuff)
 m1 = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)
 ```
 
@@ -88,7 +88,7 @@ However, it is not as straightforward to detect singularity in vector-valued ran
 
 For example, if we bootstrap a model fit to the `sleepstudy` data
 ```@example Main
-sleepstudy = MixedModels.dataset(:sleepstudy)
+sleepstudy = MixedModelsDatasets.dataset(:sleepstudy)
 contrasts = Dict(:subj => Grouping())
 m2 = let f = @formula reaction ~ 1+days+(1+days|subj)
     fit(MixedModel, f, sleepstudy; contrasts)

docs/src/constructors.md

@@ -14,8 +14,8 @@ using DisplayAs
 ```
 
 ```@example Main
-using DataFrames, MixedModels, StatsModels
-dyestuff = MixedModels.dataset(:dyestuff)
+using DataFrames, MixedModels, MixedModelsDatasets, StatsModels
+dyestuff = MixedModelsDatasets.dataset(:dyestuff)
 ```
 
 ```@example Main
@@ -71,7 +71,7 @@ Notice that both are equivalent.
 
 ```@example Main
 using BenchmarkTools
-dyestuff2 = MixedModels.dataset(:dyestuff2)
+dyestuff2 = MixedModelsDatasets.dataset(:dyestuff2)
 @benchmark fit(MixedModel, $fm, $dyestuff2)
 ```
 
@@ -110,7 +110,7 @@ It corresponds to a shift in the intercept for each level of the grouping factor
 The *sleepstudy* data are observations of reaction time, `reaction`, on several subjects, `subj`, after 0 to 9 days of sleep deprivation, `days`.
 A model with random intercepts and random slopes for each subject, allowing for within-subject correlation of the slope and intercept, is fit as
 ```@example Main
-sleepstudy = MixedModels.dataset(:sleepstudy)
+sleepstudy = MixedModelsDatasets.dataset(:sleepstudy)
 fm2 = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy)
 DisplayAs.Text(ans) # hide
 ```
@@ -120,14 +120,14 @@ DisplayAs.Text(ans) # hide
 A model for the *Penicillin* data incorporates random effects for the plate, and for the sample.
 As every sample is used on every plate these two factors are *crossed*.
 ```@example Main
-penicillin = MixedModels.dataset(:penicillin)
+penicillin = MixedModelsDatasets.dataset(:penicillin)
 fm3 = fit(MixedModel, @formula(diameter ~ 1 + (1|plate) + (1|sample)), penicillin)
 DisplayAs.Text(ans) # hide
 ```
 
 In contrast, the `cask` grouping factor is *nested* within the `batch` grouping factor in the *Pastes* data.
 ```@example Main
-pastes = DataFrame(MixedModels.dataset(:pastes))
+pastes = DataFrame(MixedModelsDatasets.dataset(:pastes))
 describe(pastes)
 ```
 This can be expressed using the solidus (the "`/`" character) to separate grouping factors, read "`cask` nested within `batch`":
@@ -147,7 +147,7 @@ In observational studies it is common to encounter *partially crossed* grouping
 For example, the *InstEval* data are course evaluations by students, `s`, of instructors, `d`.
 Additional covariates include the academic department, `dept`, in which the course was given and `service`, whether or not it was a service course.
 ```@example Main
-insteval = MixedModels.dataset(:insteval)
+insteval = MixedModelsDatasets.dataset(:insteval)
 fm5 = fit(MixedModel, @formula(y ~ 1 + service * dept + (1|s) + (1|d)), insteval)
 DisplayAs.Text(ans) # hide
 ```
@@ -222,7 +222,7 @@ To create a GLMM representation, the distribution family for the response, and p
 You can either use `fit(MixedModel, ...)` or `glmm(...)` to fit the model. For instance:
 
 ```@example Main
-verbagg = MixedModels.dataset(:verbagg)
+verbagg = MixedModelsDatasets.dataset(:verbagg)
 verbaggform = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));
 gm1 = fit(MixedModel, verbaggform, verbagg, Bernoulli())
 DisplayAs.Text(ans) # hide
@@ -258,7 +258,7 @@ The optional argument `nAGQ=k` causes evaluation of the deviance function to use
 adaptive Gauss-Hermite quadrature rule.
 This method only applies to models with a single, simple, scalar random-effects term, such as
 ```@example Main
-contraception = MixedModels.dataset(:contra)
+contraception = MixedModelsDatasets.dataset(:contra)
 contraform = @formula(use ~ 1 + age + abs2(age) + livch + urban + (1|dist));
 bernoulli = Bernoulli()
 deviances = Dict{Symbol,Float64}()
@@ -511,7 +511,7 @@ sum(leverage(fm2))
 
 When a model converges to a singular covariance, such as
 ```@example Main
-fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModels.dataset(:dyestuff2))
+fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModelsDatasets.dataset(:dyestuff2))
 DisplayAs.Text(ans) # hide
 ```
 the effective degrees of freedom is the lower bound.
@@ -522,7 +522,7 @@ sum(leverage(fm3))
 Models for which the estimates of the variances of the random effects are large relative to the residual variance have effective degrees of freedom close to the upper bound.
 ```@example Main
 fm4 = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),
-    MixedModels.dataset(:penicillin))
+    MixedModelsDatasets.dataset(:penicillin))
 DisplayAs.Text(ans) # hide
 ```
 ```@example Main
@@ -532,7 +532,7 @@ sum(leverage(fm4))
 Also, a model fit by the REML criterion generally has larger estimates of the variance components and hence a larger effective degrees of freedom.
 ```@example Main
 fm4r = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),
-    MixedModels.dataset(:penicillin), REML=true)
+    MixedModelsDatasets.dataset(:penicillin), REML=true)
 DisplayAs.Text(ans) # hide
 ```
 ```@example Main

docs/src/derivatives.md

@@ -15,9 +15,9 @@ ForwardDiff.hessian(::LinearMixedModel{T}, ::Vector{T}) where {T}
 ### Exact zero at optimum for trivial models
 
 ```@example Derivatives
-using MixedModels, ForwardDiff
+using MixedModels, MixedModelsDatasets, ForwardDiff
 using DisplayAs # hide
-fm1 = lmm(@formula(yield ~ 1 + (1|batch)), MixedModels.dataset(:dyestuff2))
+fm1 = lmm(@formula(yield ~ 1 + (1|batch)), MixedModelsDatasets.dataset(:dyestuff2))
 DisplayAs.Text(ans) # hide
 ```
 
@@ -32,7 +32,7 @@ ForwardDiff.hessian(fm1)
 ### Approximate zero at optimum for non trivial models
 
 ```@example Derivatives
-fm2 = lmm(@formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))
+fm2 = lmm(@formula(reaction ~ 1 + days + (1+days|subj)), MixedModelsDatasets.dataset(:sleepstudy))
 DisplayAs.Text(ans) # hide
 ```
 

docs/src/index.md

@@ -16,8 +16,8 @@ You can fit a model using a `lmer`-style model formula using `@formula` and a da
 Here is a short example of how to fit a linear mixed-effects modeling using the `dyestuff` dataset:
 
 ```@example Main
-using DataFrames, MixedModels           # load packages
-dyestuff = MixedModels.dataset(:dyestuff);              # load dataset
+using DataFrames, MixedModels, MixedModelsDatasets  # load packages
+dyestuff = MixedModelsDatasets.dataset(:dyestuff);   # load dataset
 
 lmod = lmm(@formula(yield ~ 1 + (1|batch)), dyestuff)   # fit the model!
 DisplayAs.Text(ans) # hide
@@ -28,7 +28,7 @@ A quick example of generalized linear model using the `verbagg` dataset:
 
 ```@example Main
 using DataFrames, MixedModels               # load packages
-verbagg = MixedModels.dataset(:verbagg);    # load dataset
+verbagg = MixedModelsDatasets.dataset(:verbagg);    # load dataset
 
 frm = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));
 bernmod = glmm(frm, verbagg, Bernoulli())   # fit the model!

docs/src/mime.md

@@ -12,7 +12,7 @@ Packages like `IJulia` and `Documenter` can often detect the presence of these d
 
 
 ```@example Main
-using MixedModels
+using MixedModels, MixedModelsDatasets
 form = @formula(rt_trunc ~ 1 + spkr * prec * load +
                           (1 + load | item) +
                           (1 + spkr + prec + load | subj))
@@ -21,7 +21,7 @@ contr = Dict(:spkr => EffectsCoding(),
              :load => EffectsCoding(),
              :item => Grouping(),
              :subj => Grouping())
-kbm = fit(MixedModel, form, MixedModels.dataset(:kb07); contrasts=contr)
+kbm = fit(MixedModel, form, MixedModelsDatasets.dataset(:kb07); contrasts=contr)
 ```
 
 Note that the display here is more succinct than the standard REPL display:
@@ -52,8 +52,8 @@ kbm.optsum
 ```
 
 ```@example Main
-m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModels.dataset(:sleepstudy))
-m1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))
+m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModelsDatasets.dataset(:sleepstudy))
+m1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModelsDatasets.dataset(:sleepstudy))
 MixedModels.likelihoodratiotest(m0,m1)
 ```
 

docs/src/optimization.md

@@ -101,10 +101,10 @@ For the simple example
 using DisplayAs
 ```
 ```@example Main
-using BenchmarkTools, DataFrames, MixedModels
+using BenchmarkTools, DataFrames, MixedModels, MixedModelsDatasets
 ```
 ```@example Main
-dyestuff = MixedModels.dataset(:dyestuff)
+dyestuff = MixedModelsDatasets.dataset(:dyestuff)
 fm1 = fit(MixedModel, @formula(yield ~ 1 + (1|batch)), dyestuff)
 DisplayAs.Text(ans) # hide
 ```
@@ -133,7 +133,7 @@ Furthermore, there is only one block in $\Lambda_\theta$.
 
 For a vector-valued random-effects term, as in
 ```@example Main
-sleepstudy = MixedModels.dataset(:sleepstudy)
+sleepstudy = MixedModelsDatasets.dataset(:sleepstudy)
 fm2 = fit(MixedModel, @formula(reaction ~ 1+days+(1+days|subj)), sleepstudy)
 DisplayAs.Text(ans) # hide
 ```
@@ -168,7 +168,7 @@ MixedModels.getθ(t31)
 Random-effects terms with distinct grouping factors generate distinct elements of the `reterms` field of the `LinearMixedModel` object.
 Multiple `ReMat` objects are sorted by decreasing numbers of random effects.
 ```@example Main
-penicillin = MixedModels.dataset(:penicillin)
+penicillin = MixedModelsDatasets.dataset(:penicillin)
 fm4 = fit(MixedModel,
     @formula(diameter ~ 1 + (1|sample) + (1|plate)),
     penicillin)
@@ -314,7 +314,7 @@ Poisson
 A `GeneralizedLinearMixedModel` object is generated from a formula, data frame and distribution family.
 
 ```@example Main
-verbagg = MixedModels.dataset(:verbagg)
+verbagg = MixedModelsDatasets.dataset(:verbagg)
 const vaform = @formula(r2 ~ 1 + anger + gender + btype + situ + (1|subj) + (1|item));
 mdl = GeneralizedLinearMixedModel(vaform, verbagg, Bernoulli());
 typeof(mdl)