(Right) If the graph is not fully connected, some of the transitions may not be available (e.g. state 4 to state 1 in our case).
the online algorithm might not be able to join the optimal path in one step, and thus may have to forgo additional rewards.