<p>There are many related answers to this on the broader question of how <a href="http://probabilistic-programming.org" rel="noreferrer">probabilistic programming</a> benefits from <a href="https://arxiv.org/abs/1701.03757" rel="noreferrer">deep probabilistic programming</a> systems.</p>

<p>I can give one pointed answer for Latent Dirichlet Allocation (LDA) in TensorFlow. A key benefit is from recognizing that LDA is just a model. Given this model, and a dataset represented as a document-by-term matrix (e.g., via <a href="https://www.tensorflow.org/api_docs/python/tf/SparseTensor" rel="noreferrer">tf.SparseTensor</a>), TensorFlow lets you not only perform scalable inference but very flexible inference. Specific ops to use in TF depends on the specific algorithm. You can write a Gibbs sampler or coordinate ascent variational inference algorithm—both highly efficient for LDA (usable with manual <code>tf.assign</code> ops on trainable variables). CAVI is computationally and memory-efficient, <a href="https://arxiv.org/abs/1206.7051" rel="noreferrer">scaling to millions of documents</a> and reifiable with efficient data pipelines such as <a href="https://www.tensorflow.org/api_docs/python/tf/data" rel="noreferrer">tf.data</a>.</p>

<p>With TensorFlow, you can also use generic methods such as black box variational inference, which are extremely versatile and do not require manual <code>tf.assign</code> ops. Once you've written it to work well on your problem, you can extend LDA in many ways such as with nonconjugate priors, hierarchical priors, and deep network parameterizations (possible with <a href="https://www.tensorflow.org/api_docs/python/tf/layers" rel="noreferrer">tf.layers</a>). Generic methods require tools such as TensorFlow optimizers and TensorFlow's automatic differentiation for gradient-based optimization. These are not available in Python unless you expoit tracing tools such as <a href="https://github.com/HIPS/autograd/" rel="noreferrer">autograd</a>.</p>
Advantages to implement LDA(latent dirichlet allocation) with tensorflow
<p>I wanted to implement LDA with tensorflow as a practice, and I think the tensorflow version may have the advantages below:</p>

<ul>
<li>Fast. If I can use the built-in ops to express the sampling process.</li>
<li>Easy to parallelize. Many ops have been implemented with optimizations for parallelization, so this lda should be easy to run on gpus or distributed clusters.</li>
<li>Shorter and cleaner code. Like many other models, especially NNs, building such models with tensorflow involves less code.</li>
</ul>

<p>While after I inspected some python implementations of lda(for example, <a href="https://github.com/ariddell/lda/" rel="nofollow">https://github.com/ariddell/lda/</a>), I have no idea what ops of tensorflow can be used, what kind of graph should be built and what optimizer should I choose. Because the process of the gibbs sampling looks like all about element-wise updating of the doc-topics, the topic-words matrices and the topic counting table. So what can tensorflow do to simplify and optimze this process?</p>

<p>And can I treat the likelihood of the generated doc to the real input doc as the optimization target and utilize a gradient boost optimizer to minimize the negative of the likelihood, thus get alpha, beta and doc-topics distributions? Because if this is tractable, tensorflow definitely can be used here.</p>