Deploy preview for PR 663

Author: penelopeysm

pr-previews/663/developers/compiler/design-overview/index.html

@@ -813,7 +813,7 @@ <h2 class="anchored" data-anchor-id="the-model">The model</h2>
 <!-- Very outdated
 A `model::Model` is a callable struct that one can sample from by calling
 
-```epejxqt
+```raktqau
 #| eval: false
 (model::Model)([rng, varinfo, sampler, context])
 ```

pr-previews/663/developers/models/varinfo-overview/index.html

@@ -810,7 +810,7 @@ <h2 class="anchored" data-anchor-id="the-outputs-of-a-model">The outputs of a mo
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>(xplus1 = 0.3052728587999989,)</code></pre>
+<pre><code>(xplus1 = 0.6272477661284759,)</code></pre>
 </div>
 </div>
 <p>and the latent variables using <code>rand()</code>:</p>
@@ -824,7 +824,7 @@ <h2 class="anchored" data-anchor-id="the-outputs-of-a-model">The outputs of a mo
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
 <pre><code>Dict{VarName, Any} with 1 entry:
-  x =&gt; -1.15781</code></pre>
+  x =&gt; -1.11997</code></pre>
 </div>
 </div>
 <div class="callout callout-style-default callout-note callout-titled">
@@ -856,14 +856,14 @@ <h2 class="anchored" data-anchor-id="passing-latent-values-into-a-model">Passing
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>-1.5892030481364472</code></pre>
+<pre><code>-1.5461046679563166</code></pre>
 </div>
 </div>
 <p>One can check this against the expected log prior:</p>
 <div id="10" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb9"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="fu">logpdf</span>(<span class="fu">Normal</span>(), latents[<span class="pp">@varname</span>(x)])</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>-1.5892030481364472</code></pre>
+<pre><code>-1.5461046679563166</code></pre>
 </div>
 </div>
 <p>Likewise, you can evaluate the return value of the model given the latent variables:</p>
@@ -876,7 +876,7 @@ <h2 class="anchored" data-anchor-id="passing-latent-values-into-a-model">Passing
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>(xplus1 = -0.15781217382766743,)</code></pre>
+<pre><code>(xplus1 = -0.11996976276294524,)</code></pre>
 </div>
 </div>
 </section>
@@ -885,7 +885,7 @@ <h2 class="anchored" data-anchor-id="varinfo">VarInfo</h2>
 <p>The above functions are convenient, but for many ‘serious’ applications they might not be flexible enough. For example, if you wanted to obtain the return value <em>and</em> the log joint, you would have to execute the model twice: once with <code>returned</code> and once with <code>logjoint</code>.</p>
 <p>If you want to avoid this duplicate work, you need to use a lower-level interface, which is <code>DynamicPPL.evaluate!!</code>. At its core, <code>evaluate!!</code> takes a model and a VarInfo object, and returns a tuple of the return value and the new VarInfo. So, before we even get to <code>evaluate!!</code>, we need to understand what a VarInfo is.</p>
 <p>A VarInfo is a container that tracks the state of model execution, as well as any outputs related to its latent variables, such as log probabilities. DynamicPPL’s source code contains many different kinds of VarInfos, each with different trade-offs. The details of these are somewhat arcane and unfortunately cannot be fully abstracted away, mainly due to performance considerations.</p>
-<p>For the vast majority of users, it suffices to know that you can generate one of them for a model with the constructor <code>VarInfo([rng, ]model)</code>. Note that this construction executes the model once (in the process, sampling new parameter values from the prior).</p>
+<p>For the vast majority of users, it suffices to know that you can generate one of them for a model with the constructor <code>VarInfo([rng, ]model)</code>. Note that this construction executes the model once (sampling new parameter values from the prior in the process).</p>
 <div id="14" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb13"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a>v <span class="op">=</span> <span class="fu">VarInfo</span>(model)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-stdout">
@@ -895,21 +895,21 @@ <h2 class="anchored" data-anchor-id="varinfo">VarInfo</h2>
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [0.45236330277074077], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-1.0212548120514993), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-2.1165282065100177))))</code></pre>
+<pre><code>VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [-0.12582201327469894], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-0.926854122716922), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-3.17849814926632))))</code></pre>
 </div>
 </div>
 <p>(Don’t worry about the printout of the VarInfo object: we won’t need to understand its internal structure.) We can index into a VarInfo:</p>
 <div id="16" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb15"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a>v[<span class="pp">@varname</span>(x)]</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>0.45236330277074077</code></pre>
+<pre><code>-0.12582201327469894</code></pre>
 </div>
 </div>
 <p>To access the values of log-probabilities, DynamicPPL provides the <code>getlogprior</code>, <code>getloglikelihood</code>, and <code>getlogjoint</code> functions:</p>
 <div id="18" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb17"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a>DynamicPPL.<span class="fu">getlogprior</span>(v)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>-1.0212548120514993</code></pre>
+<pre><code>-0.926854122716922</code></pre>
 </div>
 </div>
 <p>What about the return value? Well, the VarInfo does not store this directly: recall that <code>evaluate!!</code> gives us back the return value separately from the updated VarInfo. So, let’s try calling it to see what happens. The default behaviour of <code>evaluate!!</code> is to use the parameter values stored in the VarInfo during model execution. That is, when it sees <code>x ~ Normal()</code>, it will use the value of <code>x</code> stored in <code>v</code>. We will see later how to change this behaviour.</p>
@@ -922,7 +922,7 @@ <h2 class="anchored" data-anchor-id="varinfo">VarInfo</h2>
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>((xplus1 = 1.4523633027707408,), VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [0.45236330277074077], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-1.0212548120514993), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-2.1165282065100177)))))</code></pre>
+<pre><code>((xplus1 = 0.874177986725301,), VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [-0.12582201327469894], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-0.926854122716922), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-3.17849814926632)))))</code></pre>
 </div>
 </div>
 <p>So here in a single call we have obtained both the return value and an updated VarInfo <code>vout</code>, from which we can again extract log probabilities and variable values. We can see from this that the value of <code>vout[@varname(x)]</code> is the same as <code>v[@varname(x)]</code>:</p>
@@ -949,7 +949,7 @@ <h2 class="anchored" data-anchor-id="specifying-parameter-values">Specifying par
 </div>
 </div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>((xplus1 = 1.0072388050558645,), VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [0.0072388050558645925], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-0.9189647333539912), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-2.904487123242262)))))</code></pre>
+<pre><code>((xplus1 = -0.3598335031273716,), VarInfo{@NamedTuple{x::DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}}, DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}}((x = DynamicPPL.Metadata{Dict{VarName{:x, typeof(identity)}, Int64}, Vector{Normal{Float64}}, Vector{VarName{:x, typeof(identity)}}, Vector{Float64}}(Dict(x =&gt; 1), [x], UnitRange{Int64}[1:1], [-1.3598335031273716], Normal{Float64}[Distributions.Normal{Float64}(μ=0.0, σ=1.0)], Bool[0]),), DynamicPPL.AccumulatorTuple{3, @NamedTuple{LogPrior::LogPriorAccumulator{Float64}, LogJacobian::LogJacobianAccumulator{Float64}, LogLikelihood::LogLikelihoodAccumulator{Float64}}}((LogPrior = DynamicPPL.LogPriorAccumulator(-1.8435121113185025), LogJacobian = DynamicPPL.LogJacobianAccumulator(0.0), LogLikelihood = DynamicPPL.LogLikelihoodAccumulator(-6.563179117573247)))))</code></pre>
 </div>
 </div>
 <p>This updates <code>v_new</code> with the new values that were sampled, and also means that log probabilities are computed using these new values.</p>
@@ -1021,15 +1021,15 @@ <h2 class="anchored" data-anchor-id="parameters-in-the-form-of-vectors">Paramete
 <span id="cb29-5"><a href="#cb29-5" aria-hidden="true" tabindex="-1"></a>    <span class="fu">median</span>(<span class="pp">@be</span> DynamicPPL.<span class="fu">evaluate!!</span>(model, v_new))</span>
 <span id="cb29-6"><a href="#cb29-6" aria-hidden="true" tabindex="-1"></a><span class="cf">end</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>368.062 ns (7 allocs: 288 bytes)</code></pre>
+<pre><code>336.794 ns (7 allocs: 288 bytes)</code></pre>
 </div>
 </div>
 <div id="32" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb31"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb31-1"><a href="#cb31-1" aria-hidden="true" tabindex="-1"></a><span class="fu">with_logger</span>(<span class="fu">ConsoleLogger</span>(<span class="cn">stderr</span>, <span class="bu">Logging</span>.Warn)) <span class="cf">do</span></span>
 <span id="cb31-2"><a href="#cb31-2" aria-hidden="true" tabindex="-1"></a>    <span class="fu">median</span>(<span class="pp">@be</span> DynamicPPL.<span class="fu">init!!</span>(model, v_new, <span class="fu">InitFromParams</span>(<span class="fu">Dict</span>(<span class="pp">@varname</span>(x) <span class="op">=&gt;</span> <span class="fl">3.0</span>))))</span>
 <span id="cb31-3"><a href="#cb31-3" aria-hidden="true" tabindex="-1"></a><span class="cf">end</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>491.917 ns (12 allocs: 624 bytes)</code></pre>
+<pre><code>441.864 ns (12 allocs: 624 bytes)</code></pre>
 </div>
 </div>
 <p>When evaluating models in tight loops, as is often the case in inference algorithms, this overhead can be quite unwanted. DynamicPPL provides a rather dangerous, but powerful, way to get around this, which is the <code>DynamicPPL.unflatten</code> function. <code>unflatten</code> allows you to directly modify the internal storage of a VarInfo, without having to go through <code>init!!</code> and model evaluation. Its input is a vector of parameters.</p>
@@ -1250,15 +1250,15 @@ <h2 class="anchored" data-anchor-id="advanced-typed-and-untyped-varinfo">Advance
 <span id="cb55-2"><a href="#cb55-2" aria-hidden="true" tabindex="-1"></a>    <span class="fu">median</span>(<span class="pp">@be</span> DynamicPPL.<span class="fu">evaluate!!</span>(model, v_filled_untyped))</span>
 <span id="cb55-3"><a href="#cb55-3" aria-hidden="true" tabindex="-1"></a><span class="cf">end</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>2.079 μs (32 allocs: 1.328 KiB)</code></pre>
+<pre><code>2.091 μs (32 allocs: 1.328 KiB)</code></pre>
 </div>
 </div>
 <div id="58" class="cell" data-execution_count="1">
 <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb57"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb57-1"><a href="#cb57-1" aria-hidden="true" tabindex="-1"></a><span class="fu">with_logger</span>(<span class="fu">ConsoleLogger</span>(<span class="cn">stderr</span>, <span class="bu">Logging</span>.Warn)) <span class="cf">do</span></span>
 <span id="cb57-2"><a href="#cb57-2" aria-hidden="true" tabindex="-1"></a>    <span class="fu">median</span>(<span class="pp">@be</span> DynamicPPL.<span class="fu">evaluate!!</span>(model, v_filled_typed))</span>
 <span id="cb57-3"><a href="#cb57-3" aria-hidden="true" tabindex="-1"></a><span class="cf">end</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
 <div class="cell-output cell-output-display" data-execution_count="1">
-<pre><code>365.741 ns (7 allocs: 288 bytes)</code></pre>
+<pre><code>335.930 ns (7 allocs: 288 bytes)</code></pre>
 </div>
 </div>