use the co-occurrence statistics for the entire corpus: Denote the co-occurrence statistics for the $d$-th document as ${\boldsymbol{\varOmega}}_d$, then $\operatorname{E}{\boldsymbol{\varOmega}}_d = {\boldsymbol{M}}{\boldsymbol{\varTheta}}_d{\boldsymbol{M}}^{{\!\top\!}}$; consequently $${\boldsymbol{\varOmega}}= \frac{1}{\sum_{d=1}^{D}L_d}\sum_{d=1}^{D}L_d{\boldsymbol{\varOmega}}_d,$$ which is an unbiased estimator for $${\boldsymbol{M}}{\boldsymbol{\varTheta}}{\boldsymbol{M}}^{{\!\top\!}}= \frac{1}{\sum_{d=1}^{D}L_d}\sum_{d=1}^{D}L_d{\boldsymbol{M}}{\boldsymbol{\varTheta}}_d{\boldsymbol{M}}^{{\!\top\!}},$$ where $L_d$ is the length of the $d$-th document and ${\boldsymbol{\varTheta}}$ is conceptually a weighted average of all the topic-transition matrices. Then we may apply Algorithm \[alg:hmm\_id\] to learn the topic matrix. We illustrate the performance of our HTMM by comparing it to three popular bag-of-words topic modeling approaches: pLSA [@hofmann2001unsupervised], LDA [@blei2003latent], and FastAnchor [@arora2013practical], which guarantees identifiability if every topic has a characteristic *anchor word*. Our HTMM model guarantees identifiability if the topic matrix is *sufficiently scattered*, which is a more relaxed condition than the anchor word one. On the Reuters21578 data set obtained at [@reuters21578], we use the raw document to construct the word co-occurrence statistics, as well as bag-of-words representations for each document for the baseline algorithms. We use the version in which the stop-words have been removed, which makes the HTMM model more plausible since any syntactic dependencies have been removed, leaving only semantic dependencies. The vocabulary size of Reuters21578 is around $200,000$, making any method relying on triple-occurrences impossible to implement, 