@@ -23,12 +23,12 @@ a two-step procedure:
2323In practice, neither the marginal posterior nor the conditional posterior
2424are available in closed form and so they must be approximated.
2525The marginal posterior can be written as $p(\phi \mid y) \propto p(y \mid \phi) p(\phi)$,
26- where $p(y \mid \phi) = \int p(y \mid \phi, \theta) p(\theta) d\theta$ $
27- is called marginal likelihood. The Laplace method approximates
26+ where $p(y \mid \phi) = \int p(y \mid \phi, \theta) p(\theta) d\theta$ $
27+ is called marginal likelihood. The Laplace method approximates
2828$p(y \mid \phi, \theta) p(\theta)$ with a normal distribution and the
2929resulting Gaussian integral can be evaluated analytically to obtain an
30- approximation to the log marginal likelihood
31- $\log \hat p(y \mid \phi) \approx \log p(y \mid \phi)$.
30+ approximation to the log marginal likelihood
31+ $\log \hat p(y \mid \phi) \approx \log p(y \mid \phi)$.
3232
3333Combining this marginal likelihood with the prior in the ` model `
3434block, we can then sample from the marginal posterior $p(\phi \mid y)$
@@ -40,21 +40,21 @@ depends on the case.
4040
4141To obtain posterior draws for $\theta$, we sample from the normal
4242approximation to $p(\theta \mid y, \phi)$ in ` generated quantities ` .
43- The process of iteratively sampling from $p(\phi \mid y)$ (say, with MCMC) and
43+ The process of iteratively sampling from $p(\phi \mid y)$ (say, with MCMC) and
4444then $p(\theta \mid y, \phi)$ produces samples from the joint posterior
4545$p(\theta, \phi \mid y)$.
4646
4747The Laplace approximation is especially useful if $p(\theta)$ is
4848multivariate normal and $p(y \mid \phi, \theta)$ is
4949log-concave. Stan's embedded Laplace approximation is restricted to
50- have multivariate normal prior $p(\theta)$ and ... likelihood
50+ have multivariate normal prior $p(\theta)$ and ... likelihood
5151$p(y \mid \phi, \theta)$.
5252
5353
5454## Specifying the likelihood function
5555
56- The first step to use the embedded Laplace approximation is to write down a
57- function in the ` functions ` block which returns the log joint likelihood
56+ The first step to use the embedded Laplace approximation is to write down a
57+ function in the ` functions ` block which returns the log joint likelihood
5858` \log p(y \mid \theta, \phi) ` . There are a few constraints on this function:
5959
6060* The function return type must be ` real `
@@ -63,7 +63,7 @@ function in the `functions` block which returns the log joint likelihood
6363have type ` vector ` .
6464
6565* The operations in the function must support higher-order automatic
66- differentation (AD). Most functions in Stan support higher-order AD.
66+ differentiation (AD). Most functions in Stan support higher-order AD.
6767The exceptions are functions with specialized calls for reverse-mode AD, and
6868these are higher-order functions (algebraic solvers, differential equation
6969solvers, and integrators) and the suite of hidden Markov model (HMM) functions.
@@ -74,7 +74,7 @@ real ll_function(vector theta, ...)
7474```
7575There is no type restrictions for the variadic arguments ` ... ` and each
7676argument can be passed as data or parameter. As always, users should use
77- parameter arguments only when nescessary in order to speed up differentiation.
77+ parameter arguments only when necessary in order to speed up differentiation.
7878In general, we recommend marking data only arguments with the keyword ` data ` ,
7979for example,
8080```
@@ -98,11 +98,11 @@ is implicitly defined as the collection of all non-data arguments passed to
9898## Approximating the log marginal likelihood $\log p(y \mid \phi)$
9999
100100In the ` model ` block, we increment ` target ` with ` laplace_marginal ` , a function
101- that approximates the log marginal likelihood $\log p(y \mid \phi)$.
101+ that approximates the log marginal likelihood $\log p(y \mid \phi)$.
102102This function takes in the
103103user-specified likelihood and prior covariance functions, as well as their arguments.
104104These arguments must be passed as tuples, which can be generated on the fly
105- using parenthesis.
105+ using parenthesis.
106106We also need to pass an argument $\theta_0$ which serves as an initial guess for
107107the optimization problem that underlies the Laplace approximation,
108108$$
@@ -113,11 +113,11 @@ passed to `ll_function`.
113113
114114The signature of the function is:
115115```
116- target += laplace_marginal(function ll_function, tupple (...), vector theta_0,
117- function K_function, tupple (...));
116+ real laplace_marginal(function ll_function, tuple (...), vector theta_0,
117+ function K_function, tuple (...));
118118```
119- The tuple ` (...) ` after ` ll_function ` contains the arguments that get passed
120- to ` ll_function ` * excluding $\theta$* . Likewise, the tuple ` (...) ` after
119+ The ` tuple (...)` after ` ll_function ` contains the arguments that get passed
120+ to ` ll_function ` * excluding $\theta$* . Likewise, the ` tuple (...)` after
121121` ll_function ` contains the arguments that get passed to ` K_function ` .
122122
123123It also possible to specify control parameters, which can help improve the
@@ -159,8 +159,8 @@ maximum number of steps in the linesearch is reached. By default,
159159With these arguments at hand, we can call ` laplace_marginal_tol ` with the
160160following signature:
161161```
162- target += laplace_margina_tol(function ll_function, tupple (...), vector theta_0,
163- function K_function, tupple (...),
162+ target += laplace_margina_tol(function ll_function, tuple (...), vector theta_0,
163+ function K_function, tuple (...),
164164 real tol, int max_steps, int hessian_block_size,
165165 int solver, int max_steps_linesearch);
166166```
@@ -172,22 +172,22 @@ approximation of $p(\theta \mid \phi, y)$ using `laplace_latent_rng`.
172172The signature for ` laplace_latent_rng ` follows closely
173173the signature for ` laplace_marginal ` :
174174```
175- vector theta =
176- laplace_latent_rng(function ll_function, tupple (...), vector theta_0,
177- function K_function, tupple (...));
175+ vector theta =
176+ laplace_latent_rng(function ll_function, tuple (...), vector theta_0,
177+ function K_function, tuple (...));
178178```
179179Once again, it is possible to specify control parameters:
180180```
181- vector theta =
182- laplace_latent_tol_rng(function ll_function, tupple (...), vector theta_0,
183- function K_function, tupple (...),
181+ vector theta =
182+ laplace_latent_tol_rng(function ll_function, tuple (...), vector theta_0,
183+ function K_function, tuple (...),
184184 real tol, int max_steps, int hessian_block_size,
185185 int solver, int max_steps_linesearch);
186186```
187187
188188## Built-in Laplace marginal likelihood functions
189189
190- Stan supports certain built-in Laplace marginal likelihood functions.
190+ Stan supports certain built-in Laplace marginal likelihood functions.
191191This selection is currently
192192narrow and expected to grow. The built-in functions exist for the user's
193193convenience but are not more computationally efficient than specifying log
@@ -196,7 +196,7 @@ likelihoods in the `functions` block.
196196### Poisson with log link
197197
198198Given count data, with each observed count $y_i$ associated with a group
199- $g(i)$ and a corresponding latent variable $\theta_ {g(i)}$, and Poisson model,
199+ $g(i)$ and a corresponding latent variable $\theta_ {g(i)}$, and Poisson model,
200200the likelihood is
201201$$
202202p(y \mid \theta, \phi) = \prod_i\text{Poisson} (y_i \mid \exp(\theta_{g(i)})).
@@ -211,18 +211,18 @@ The signatures for the embedded Laplace approximation function with a Poisson
211211likelihood are
212212```
213213real laplace_marginal_poisson_log_lpmf(array[] int y | array[] int y_index,
214- vector theta0, function K_function, (...));
214+ vector theta0, function K_function, tuple (...));
215215
216216real laplace_marginal_tol_poisson_log_lpmf(array[] int y | array[] int y_index,
217- vector theta0, function K_function, (...),
217+ vector theta0, function K_function, tuple (...),
218218 real tol, int max_steps, int hessian_block_size,
219219 int solver, int max_steps_linesearch);
220220
221221vector laplace_latent_poisson_log_rng(array[] int y, array[] int y_index,
222- vector theta0, function K_function, (...));
222+ vector theta0, function K_function, tuple (...));
223223
224224vector laplace_latent_tol_poisson_log_rng(array[] int y, array[] int y_index,
225- vector theta0, function K_function, (...),
225+ vector theta0, function K_function, tuple (...),
226226 real tol, int max_steps, int hessian_block_size,
227227 int solver, int max_steps_linesearch);
228228```
@@ -237,37 +237,37 @@ The signatures for this function are:
237237```
238238real laplace_marginal_poisson2_log_lpmf(array[] int y | array[] int y_index,
239239 vector x, vector theta0,
240- function K_function, (...));
240+ function K_function, tuple (...));
241241
242242real laplace_marginal_tol_poisson2_log_lpmf(array[] int y | array[] int y_index,
243243 vector x, vector theta0,
244- function K_function, (...),
244+ function K_function, tuple (...),
245245 real tol, int max_steps, int hessian_block_size,
246246 int solver, int max_steps_linesearch);
247247
248248vector laplace_latent_poisson2_log_rng(array[] int y, array[] int y_index,
249- vector x, vector theta0,
250- function K_function, (...));
249+ vector x, vector theta0,
250+ function K_function, tuple (...));
251251
252252vector laplace_latent_tol_poisson2_log_rng(array[] int y, array[] int y_index,
253253 vector x, vector theta0,
254- function K_function, (...),
254+ function K_function, tuple (...),
255255 real tol, int max_steps, int hessian_block_size,
256256 int solver, int max_steps_linesearch);
257257```
258258
259259
260260### Negative Binomial with log link
261261
262- The negative Bionomial distribution generalizes the Poisson distribution by
262+ The negative Binomial distribution generalizes the Poisson distribution by
263263introducing the dispersion parameter $\eta$. The corresponding likelihood is then
264264$$
265265p(y \mid \theta, \phi) = \prod_i\text{NegBinomial2} (y_i \mid \exp(\theta_{g(i)}), \eta).
266266$$
267- Here we use the alternative paramererization implemented in Stan, meaning that
267+ Here we use the alternative parameterization implemented in Stan, meaning that
268268$$
269- \mathbb E(y_i) = \exp (\theta_{g(i)}), \\
270- \text{Var}(y_i) = \mathbb E(y_i) + \frac{(\mathbb E(y_i))^2}{\eta}.
269+ \mathbb E(y_i) = \exp (\theta_{g(i)}), \\
270+ \text{Var}(y_i) = \mathbb E(y_i) + \frac{(\mathbb E(y_i))^2}{\eta}.
271271$$
272272The arguments for the likelihood function are:
273273
@@ -279,23 +279,23 @@ group the $i^\text{th}$ observation belongs to.
279279The function signatures for the embedded Laplace approximation with a negative
280280Binomial likelihood are
281281```
282- real laplace_marginal_neg_binomial_2_log_lpmf(array[] int y |
283- array[] int y_index, real eta, vector theta0,
284- function K_function, (...));
282+ real laplace_marginal_neg_binomial_2_log_lpmf(array[] int y |
283+ array[] int y_index, real eta, vector theta0,
284+ function K_function, tuple (...));
285285
286- real laplace_marginal_tol_neg_binomial_2_log_lpmf(array[] int y |
287- array[] int y_index, real eta, vector theta0,
288- function K_function, (...),
286+ real laplace_marginal_tol_neg_binomial_2_log_lpmf(array[] int y |
287+ array[] int y_index, real eta, vector theta0,
288+ function K_function, tuple (...),
289289 real tol, int max_steps, int hessian_block_size,
290290 int solver, int max_steps_linesearch);
291291
292- vector laplace_latent_neg_binomial_2_log_rng(array[] int y,
293- array[] int y_index, real eta, vector theta0,
294- function K_function, (...));
292+ vector laplace_latent_neg_binomial_2_log_rng(array[] int y,
293+ array[] int y_index, real eta, vector theta0,
294+ function K_function, tuple (...));
295295
296- vector laplace_latent_tol_neg_binomial_2_log_rng(array[] int y,
297- array[] int y_index, real eta, vector theta0,
298- function K_function, (...),
296+ vector laplace_latent_tol_neg_binomial_2_log_rng(array[] int y,
297+ array[] int y_index, real eta, vector theta0,
298+ function K_function, tuple (...),
299299 real tol, int max_steps, int hessian_block_size,
300300 int solver, int max_steps_linesearch);
301301```
@@ -314,23 +314,23 @@ group the $i^\text{th}$ observation belongs to.
314314
315315The function signatures for the embedded Laplace approximation with a Bernoulli likelihood are
316316```
317- real laplace_marginal_bernoulli_logit_lpmf(array[] int y |
318- array[] int y_index, real eta, vector theta0,
319- function K_function, (...));
317+ real laplace_marginal_bernoulli_logit_lpmf(array[] int y |
318+ array[] int y_index, real eta, vector theta0,
319+ function K_function, tuple (...));
320320
321- real laplace_marginal_tol_bernoulli_logit_lpmf(array[] int y |
322- array[] int y_index, real eta, vector theta0,
323- function K_function, (...),
321+ real laplace_marginal_tol_bernoulli_logit_lpmf(array[] int y |
322+ array[] int y_index, real eta, vector theta0,
323+ function K_function, tuple (...),
324324 real tol, int max_steps, int hessian_block_size,
325325 int solver, int max_steps_linesearch);
326326
327- vector laplace_latent_bernoulli_logit_rng(array[] int y,
328- array[] int y_index, real eta, vector theta0,
329- function K_function, (...));
327+ vector laplace_latent_bernoulli_logit_rng(array[] int y,
328+ array[] int y_index, real eta, vector theta0,
329+ function K_function, tuple (...));
330330
331- vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
332- array[] int y_index, real eta, vector theta0,
333- function K_function, (...),
331+ vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
332+ array[] int y_index, real eta, vector theta0,
333+ function K_function, tuple (...),
334334 real tol, int max_steps, int hessian_block_size,
335335 int solver, int max_steps_linesearch);
336336```
@@ -374,7 +374,7 @@ vector laplace_latent_tol_bernoulli_logit_rng(array[] int y,
374374<!-- and admits nearly the same arguments as `laplace_marginal`. A key difference -->
375375<!-- is that -->
376376<!-- ``` -->
377- <!-- vector laplace_latent_rng(function ll_function, tupple (...), vector theta_0, -->
378- <!-- function K_function, tupple (...)); -->
377+ <!-- vector laplace_latent_rng(function ll_function, tuple (...), vector theta_0, -->
378+ <!-- function K_function, tuple (...)); -->
379379<!-- ``` -->
380380
0 commit comments