3. Calculate $$q^*=\prod_{k=1}^K f(y|X^{*(k)},\theta^*).$$ 4. Generate a $\theta^{\#}\sim u(\theta^{\#}|\theta^*)$. Generate independent $X^{\#(1)},...,X^{\#(K)}$ from $p(X|\theta^{\#})$. Compute $q^{\#}=\prod_{k=1}^K f(y|X^{\#(k)},\theta^{\#})$. 5. Generate a uniform random variable $\omega\sim U(0,1)$, and calculate the acceptance probability $$\begin{aligned}
& \alpha = \min\biggl[1,\underbrace{\frac{q^{\#}\cdot p(X^{\#(1)}|\theta^{\#})\cdots p(X^{\#(K)}|\theta^{\#})}{q^* \cdot p(X^{*(1)}|\theta^{*})\cdots p(X^{*(K)|\theta^{*}})  }}_{\text{ratio of likelihoods}}\nonumber\\
&\times  \underbrace{\frac{v(X^{*(1)}|\theta^{*})\cdots v(X^{*(K)}|\theta^{*})u(\theta^*|\theta^{\#})}{v(X^{\#(1)}|\theta^{\#})\cdots v(X^{\#(K)}|\theta^{\#})u(\theta^{\#}|\theta^{*})}}_{\text{ratio of proposals}} \times \underbrace{\frac{\pi(\theta^{\#})}{\pi(\theta^*)}}_{\text{ratio of priors}} \biggr]\nonumber\\
&= \min\biggl[1,\frac{q^{\#}}{q^* }
\times  \frac{u(\theta^*|\theta^{\#})}{u(\theta^{\#}|\theta^{*})} \times \frac{\pi(\theta^{\#})}{\pi(\theta^*)} \biggr]. \label{eq:generalized-acceptance-prob}\end{aligned}$$ If $\omega>\alpha$, set $\theta_{j+1}:=\theta_{j}$ otherwise set $\theta_{j+1}:=\theta^{\#}$, $\theta^*:=\theta^{\#}$ and $q^*:=q^{\#}$. Increase $j$ by 1 and go to step 6. 6. Repeat steps 4–5 as long as $j\leq R$ for $R$ “large”.

\[alg:generalized-lele-algorithm\]

The notation $:=$ means “assign the value on the right hand side to the left hand side”. For a large enough number of iterations $R$ this algorithm produces a chain having $\pi(\theta,X|y^{(K)})$ as its stationary distribution, with $X=(X^{(1)},...,X^{(K)})$. In order to obtain draws from the desired marginal distribution $\pi(\theta|y^{(K)})$ it is sufficient to discard the $\{X^{(1)},...,X^{(K)}\}_{j=1,...,R}$ obtained from the algorithm output $\{\theta,X^{(1)},...,X^{(K)}\}_{j=1,...,R}$ (after some appropriate burnin period). For $K\rightarrow\infty$ the sample mean of the $\{\theta\}_j$ is the MLE of $\theta$ and $K$ times the covariance matrix of the draws returns the covariance of the MLE, the inverse of the Fisher information based on the original 