that the $i^{\text{th}}$ time frame results in an energy outage, and we have $e(i)=0$; and hence, the battery is in state $w(i)=0$. In the subsequent $(i+1)^{\text{th}}$ time frame, if the harvested energy is greater than the demanded energy, i.e., $u(i+1)\geq p(i+1)$, the battery enters state $w(i+1)=1$, and stores the excess energy. However, if $u(i+1)<p(i+1)$, another energy outage occurs, and the battery remains in state 0, i.e., $w(i+1)=0$. Similarly, given that $w(i+1)=1$, the transition from state 1 to state 2 occurs when $u(i+1)+u(i+2)-p(i+1)\geq p(i+2)$. On the other hand, when $u(i+1)+u(i+2)-p(i+1)<p(i+2)$, the battery goes from state 1 to state 0. Similarly, given that $w(i+2)=2$, the battery transitions to state 3, i.e., $w(i+3)=3$, if $u(i+1)+u(i+2)+u(i+3)-p(i+1)-p(i+2)\geq p(i+3)$, and $w(i+3)=0$ otherwise. Generalizing the above observations and considering again an infinite-size battery, we can state that given $w(i+m-1)=m-1$, we have $w(i+m)=m$ if $\sum_{j=i+1}^{i+m}u(j)\geq\sum_{j=i+1}^{i+m}p(j)$, and $w(i+m)=0$ otherwise.

Notice that the battery in state $m-1$ for $m\in\{1,2,\cdots\}$ will go to either state $m$ or state 0. Given that the battery is in state $m-1$ in the $(i+m-1)^{\text{th}}$ time interval, we can express the state transition probability from state $m-1$ to state $m$ for $m\in\{1,2,\cdots\}$, denoted by $q_{m}(i+m)$, as $$\begin{aligned}
&q_{m}(i+m)\triangleq\Pr\{w(i+m)=m|w(i+m-1)=m-1\}\label{eq:state_transition_probability_2_i}\\
&=\Pr\{w(i+m)=m|w(i+m-1)=m-1,w(i+m-2)=m-2,\cdots,w(i)=0\}\label{eq:state_transition_probability_2_f}\\
&=\frac{\Pr\{w(i+m)=m,\cdots,w(i)=0\}}{\Pr\{w(i+m-1)=m-1,\cdots,w(i)=0\}}\label{eq:state_transition_probability_2_e}\\
&=\frac{\Pr\left\{U(i+m,i)\geq P(i+m,i),\cdots,U(i+1,i)\geq P(i+1,i),e(i)=0\right\}}{\Pr\left\{U(i+m-1,i)\geq P(i+m-1,i),\cdots,U(i+1,i)\geq P(i+1,i),e(i)=0\right\}},\label{eq:state_transition_probability_2}\end{aligned}$$ where 