up to a window-length of $n$. This seems to be a much more realistic model, although the choice of $n$ is totally unclear, and is often dictated by memory and computational limitations in practice—since the size of the joint distribution grows exponentially with $n$. What is more, it is somewhat difficult to extract “topics” from this model, despite some preliminary attempts [@wallach2006topic; @wang2007topical].

We propose to model a document as the realization of a HMM, in which the topics are hidden states emitting words, and the states are evolving according to a Markov chain, hence the name *hidden topic Markov model* (HTMM). For a set of documents, this means we are working with a *collection* of HMMs. Similar to other topic modeling works, we assume that the topic matrix is shared among all documents, meaning all the given HMMs share the same emission probability. For the bag-of-words model, each document has a specific topic distribution ${\boldsymbol{p}}_d$, whereas for our model, each document has its own *topic transition probability* ${\boldsymbol{\varTheta}}_d$; as per our previous discussion, the row-sum and column-sum of ${\boldsymbol{\varTheta}}_d$ are the same, which are also the topic probability for the specific document. The difference is the Markovian assumption on 