internally.[^8] We seek the optimal control law $$\begin{alignedat}{2}u_{k}^{*}(\tilde{x}_{k},\Gamma_{k}) & = & \underset{u}{\arg\min}\quad & {\mathbb{E}\left[\sum_{j=0}^{N-1}g(\tilde{x}_{k+j},u_{k+j},\Gamma_{k+j})\right]}\\
 &  & \textrm{subject to}\quad & \begin{aligned}\tilde{x}_{i+1} & =\tilde{A}\tilde{x}_{i}+\tilde{B}u_{i}\quad\forall i\in\mathbb{Z}^{+}\\
0 & \leq u\leq u_{\max}
\end{aligned}
\end{alignedat}$$ where $u_{k+j}$ is an individual control action and $\Gamma_{k+j}\in[0,1]$ is an occupancy measurement or prediction. This is standard except that the stage cost adjusts the discomfort weigh using occupancy, i.e. $$\begin{aligned}g(\tilde{x},u,\Gamma) & =\tilde{x}^{\top}\Gamma Q\tilde{x}+r\left|u\right|\\
 & =\Gamma\beta\left(x_{\textrm{zone}}-\tau\right)^{2}+r\left|u\right|,
\end{aligned}$$ where

-   $\tilde{x}$ is the augmented system state vector and $x_{\textrm{zone}}$ is the zone air temperature being controlled,

-   $u$ is the heat input to the zone,

-   $\Gamma$ is the observed or predicted occupancy,

-   $\tau$ is the comfortable setpoint temperature (constant),

-   $Q$ is a matrix that extracts $\beta(x_{\textrm{zone}}-\tau)^{2}$ from $\mbox{\ensuremath{\tilde{x}}}^{\top}Q\tilde{x}$, and

-   $\beta$ and $r$ are the discomfort and energy cost gains.[^9]

The many ($2^{N}$) possible occupancy state trajectories, along with the constraints on $u$, make it difficult to find a closed-form solution using exact dynamic programming. (Recall from Figure \[fig:markovChain\] that each occupancy state has two possible outgoing transitions.) If we instead condition all occupancy predictions solely on the present observation, we obtain the approximation $$\begin{aligned}u_{k}^{*}(\tilde{x}_{k},\Gamma_{k}) & \approx\underset{0\leq u\leq u_{\max}}{\arg\min}\Biggl\{ g(\tilde{x}_{k},u_{k},\Gamma_{k})\\
 & +\sum_{j=1}^{N-1}g\left(\tilde{x}_{k+j},u_{k+j},\mathbb{E}\left[\Gamma_{k+j}\mid\Gamma_{k}\right]\right)\Biggr\},
\end{aligned}$$ where ${\mathbb{E}\left[\Gamma_{k+j}\mid\Gamma_{k}\right]}$ comes from Equation 