\nonumber \\
& = p(s, a) \cdot \lim_{c\rightarrow + \infty} {\operatorname{\mathbb{E}}}_{T \sim \tilde{p}(T|Q_{k,i}, \tau)}  \exp\big(-c [Q_{k, i+1}(s,a) - r - \gamma {\operatorname{\mathbb{E}}}_{s'' \sim T(\cdot|s, a)} \max_{a'}Q_{k,i+1}(s'', a')]^2\big) \nonumber \\
& \geq p(s, a) \cdot \lim_{c\rightarrow + \infty} \exp\big(- c {\operatorname{\mathbb{E}}}_{T \sim \tilde{p}(T|Q_{k,i}, \tau)}[Q_{k, i+1}(s,a) - r - \gamma {\operatorname{\mathbb{E}}}_{s'' \sim T(\cdot|s, a)} \max_{a'}Q_{k,i+1}(s'', a')]^2  \big)
\nonumber \\
& = p(s, a) \cdot \mathbbm{1}_{{\operatorname{\mathbb{E}}}_{T \sim \tilde{p}(T|Q_{k,i}, \tau)}[Q_{k, i+1}(s,a) - r - \gamma {\operatorname{\mathbb{E}}}_{s'' \sim T(\cdot|s, a)} \max_{a'}Q_{k,i+1}(s'', a')]^2=0}. \label{eqn:ind-hold}\end{aligned}$$ where we apply a limit representation of the indicator function in the third equation. The fourth equation is due to the bounded convergence theorem. The inequality is Jensen’s inequality. The last equation replaces the limit with an indicator function.

A sufficient condition for is to maximize the lower-bound of the posterior distribution in by ensuring the indicator function in to hold. We can replace with the following update $$\label{eqn:Q-k}
Q_{k, i+1} \leftarrow \operatorname*{argmin}_{Q_{k, i+1}} {\operatorname{\mathbb{E}}}_{T \sim \tilde{p}(T|Q_{k,i}, \tau)} \big[ Q_{k, i+1}(s, a) -\big( r + \gamma \cdot \mathbb{E}_{s'' \sim  T(\cdot|s, a)} \max_{a'} Q_{k, i+1}(s'', a') \big) \big]^2.$$ However, is not tractable because the expectation in is taken with respect to the posterior $\tilde{p}(T|Q_{k, i}, \tau)$ of the transition $T$. To overcome this 