We consider the problem of topic modeling in a weakly semi-supervised setting. In this scenario, we assume that the user knows a priori a subset of the topics she wants the model to learn and is able to provide a few exemplar documents for those topics. In addition, while each document may typically consist of multiple topics, we do not assume that the user will identify all its topics exhaustively.
      
 Recent state-of-the-art topic models such as NVDM, referred to herein as Neural Topic Models (NTMs), fall under the variational autoencoder framework. We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective. After analyzing the effect of informative priors, we propose a simple modification of the NVDM model using a logit-normal posterior that we show achieves better alignment to user-desired topics versus other NTM models. Topic models are probabilistic models of data that assume an abstract set of topics underlies the data generating process BID0 . These abstract topics are often not only useful as feature representations for downstream tasks, but also for exploring and analyzing a corpus. Topic models are used to explore natural scenes in images BID2 BID9 , genetics data BID12 , and numerous text corpora BID11 BID10 .While latent Dirichlet allocation (LDA) serves as the classical benchmark for topic models, recent state-of-the-art topic models such as NVDM BID7 fall under the variational autoencoder (VAE) framework BID5 , which we refer to as Neural Topic Models (NTMs). NTMs leverage the flexibility of deep learning to fit an approximate posterior using variational inference. This posterior can then be used to efficiently predict the topics contained in a document. NTMs have been shown to model documents well, as well as associate a set of meaningful top words with each topic BID8 .Often , the top words associated with each extracted topic only approximately match the user's intuition. Therefore , the user may want to guide the model towards learning topics that better align with natural semantics by providing example documents. To our knowledge , supervision has been explored in more classical LDA models, but has not been explored yet in the NTM literature.Labeling the existence of topics in each document across a corpus is prohibitively expensive. Hence, we focus on a weak form of supervision. Specifically, we assume a user may identify the existence of a single topic in a document. Furthermore, if a user does not specify the existence of a topic, it does not mean the topic does not appear in the document.The main contribution of our work is an NTM with the ability to leverage minimal user supervision to better align topics to desired semantics. In this work, we proposed supervising Neural Topic Models with weak supervision via informative priors and explored a variety of model posteriors. A careful analysis of their KL divergences and decoding mechanisms led us to an NTM with logit-normal posterior which best aligned extracted topics to desired user semantics.
We propose supervising VAE-style topic models by intelligently adjusting the prior on a per document basis. We find a logit-normal posterior provides the best performance. A flexible method of weakly supervising a topic model to achieve better alignment with user intuition.