The input is a stream of asynchronous pulses from pyroelectric infrared (PIR) or similar sensors that indicate whether at least one person is in the space. We have chosen the Mitsubishi Electric Research Lab (MERL) motion detector data set [@merldata], which consists of a series of one-second pulses from various motion sensors located throughout hallways and conference rooms in MERL.[^5] The meetings in the Belady conference room show a good balance between repetition and variety to showcase the benefits of on-line learning.

Markov Chain Formulation\[sub:Markov-chain-formulation\]
--------------------------------------------------------

![image](markovChain)

The occupancy model is as a periodic Markov chain updated at every observation. The occupancy at time $k$ is either $\gamma_{k}=$1 (occupied) or $\gamma_{k}=0$ (vacant). The current occupancy state and the time of day determine the probability of future occupancy. We wish to estimate the probabilities $$\begin{aligned}p_{k} & ={\mathbb{P}\left(\gamma_{k+1}=1\mid\gamma_{k}=1\right)},\\
q_{k} & ={\mathbb{P}\left(\gamma_{k+1}=1\mid\gamma_{k}=0\right)}.
\end{aligned}$$ The transition probabilities of this two-state time-varying Markov chain (Figure \[fig:markovChain\]a) are periodic; we have chosen a period $M=24$ hours, so $p_{24}\equiv p_{0}$ and $q_{24}\equiv q_{0}$. To better visualize the periodicity, we unroll the Markov chain into $2M$ states (Figure \[fig:markovChain\]b), where each hour has a $1_{k}$ and $0_{k}$ state. Although $k$ in general grows without bound, its range is 