This paper describes a simple method to achieve logical constraints on words for topic models based on a recently developed topic modeling framework with Dirichlet forest priors (LDA-DF). Logical constraints mean logical expressions of pairwise constraints, Must-links and Cannot-Links, used in the literature of constrained clustering. Our method can not only cover the original constraints of the existing work, but also allow us easily to add new customized constraints. We discuss the validity of our method by defining its asymptotic behaviors. We verify the effectiveness of our method with comparative studies on a synthetic corpus and interactive topic analysis on a real corpus.
Introduction Topic models such as Latent Dirichlet Allocation or LDA (Blei et al., 2003) are widely used to capture hidden topics in a corpus. When we have domain knowledge of a target corpus, incorporating the knowledge into topic models would be useful in a practical sense. Thus there have been many studies of semi-supervised extensions of topic models (Andrzejewski et al., 2007; Toutanova and Johnson, 2008; ), although topic models are often regarded as unsupervised learning. Recently, ) developed a novel topic modeling framework, LDA with Dirichlet Forest priors (LDA-DF), which achieves two links Must-Link (ML) and Cannot-Link (CL) in the constrained clustering literature (Basu et al., 2008) . For given words A and B, ML(A, B) and CL (A, B) are soft constraints that A and B must appear in the same topic, and that A and B cannot appear in the same topic, respectively. Let us consider topic analysis of a corpus with movie reviews for illustrative purposes. We know that two words 'jackie' (means Jackie Chan) and 'kung-fu' should appear in the same topic, while 'dicaprio' (means Leonardo DiCaprio) and 'kung-fu' should not appear in the same topic. In this case, we can add constraints ML('jackie', 'kung-fu') and CL ('dicaprio', 'kung-fu') to smoothly conduct analysis. However, what if there is a word 'bruce' (means Bruce Lee) in the corpus, and we want to distinguish between 'jackie' and 'bruce'? Our full knowledge among 'kung-fu', 'jackie', and 'bruce' should be (ML('kung-fu', 'jackie') ∨ ML('kung-fu', 'bruce')) ∧ CL('bruce', 'jackie'), although the original framework does not allow a disjunction (∨) of links. In this paper, we address such logical expressions of links on LDA-DF framework. Combination between a probabilistic model and logical knowledge expressions such as Markov Logic Network (MLN) is recently getting a lot of attention (Riedel and Meza-Ruiz, 2008; Yu et al., 2008; Meza-Ruiz and Riedel, 2009; Yoshikawa et al., 2009; Poon and Domingos, 2009) , and our work can be regarded as on this research line. At least, to our knowledge, our method is the first one that can directly incorporate logical knowledge into a prior for topic models without MLN. This means the complexity of the inference in our method is essentially the same as in the original LDA-DF, despite that our method can broaden knowledge expressions. LDA with Dirichlet Forest Priors We briefly review LDA-DF. Let w := w 1 . . . w n be a corpus consisting of D documents, where n is the total number of words in the documents. Let d i and z i be the document that includes the i-th word w i and the hidden topic that is assigned to w i , respectively. Let T be the number of topics. As in LDA, we assume a probabilistic language model that generates a corpus as a mixture of hidden topics and infer two parameters: a documenttopic probability θ that represents a mixture rate of topics in each document, and a topic-word probability ϕ that represents an occurrence rate of words in each topic. The model is defined as θ d i ∼ Dirichlet(α), z i |θ d i ∼ Multinomial(θ d i ), q ∼ DirichletForest(β, η), ϕ z i ∼ DirichletTree(q), w i |z i , ϕ z i ∼ Multinomial(ϕ z i ), where α and (β, η) are hyper parameters for θ and ϕ, respectively. The only difference between LDA and LDA-DF is that ϕ is chosen not from the Dirichlet distribution, but from the Dirichlet tree distribution (Dennis III, 1991) , which is a generalization of the Dirichlet distribution. The Dirichlet forest distribution assigns one tree to each topic from a set of Dirichlet trees, into which we encode domain knowledge. The trees assigned to topics z are denoted as q. In the framework, ML (A, B) is achieved by the Dirichlet tree in Fig. 1(a) , which equalizes the occurrence probabilities of A and B in a topic when η is large. This tree generates probabilities with Dirichlet(2β, β) and redistributes the probability for "2β" with Dirichlet(ηβ, ηβ). In the case of CLs, we use the following algorithm. For examples, the algorithm creates the two trees in Fig. 1 (b) for the constraint CL(A, B) ∧ CL(A, C). The constraint is achieved when η is large, since words in each topic are chosen from the distribution of either the left tree that zeros the occurrence probability of A, or the right tree that zeros those of B and C. Inference of ϕ and θ is achieved by alternately sampling topic z i for each word w i and Dirichlet tree q z for each topic z. Since the Dirichlet tree distribution is conjugate to the multinomial distribution, the sampling equation of z i is easily derived like LDA as follows: p(z i = z | z −i , q, w) ∝ (n (d i ) −i,z + α) Iz(↑i) ∏ s γ (Cz(s↓i)) z + n (Cz(s↓i)) −i ∑ Cz(s) k ( γ (k) z + n (k) −i,z ) , where n (d) −i,z represents the number of words (ex- cluding w i ) assigning topic z in document d. n (k) −i,z represents the number of words (excluding w i ) assigning topic z in the subtree rooted at node k in tree q z . I z (↑ i) and C z (s ↓ i) represents the set of internal nodes and the immediate child of node s, respectively, on the path from the root to leaf w i in tree q z . C z (s) represents the set of children of node s in tree q z . γ (k) z represents a weight of the edge to node k in tree q z . Additionally, we define ∑ S s := ∑ s∈S . Sampling of tree q z is achieved by sequentially sampling subtree q (r) z corresponding to the r-th connected component by using the following equation: p(q (r) z = q ′ | z, q −z , q (−r) z , w) ∝ |M r,q ′ |× I (q ′ ) z,r ∏ s   Γ ( ∑ Cz(s) k γ (k) z ) ∏ Cz(s) k Γ ( γ (k) z + n (k) z ) Γ ( ∑ Cz(s) k (γ (k) z + n (k) z ) ) ∏ Cz(s) k Γ ( γ (k) z )   , where I (q ′ ) z,r represents the set of internal nodes in the subtree q ′ corresponding to the r-th connected component for tree q z . |M r,q ′ | represents the size of the maximal independent set corresponding to the subtree q ′ for r-th connected component. After sufficiently sampling z i and q z , we can infer posterior probabilitiesφ andθ using the last sampled z and q, in a similar manner to the standard LDA as follows. θ (d) z = n (d) z + α ∑ T z ′ =1 ( n (d) z ′ + α ) ϕ (w) z = Iz(↑w) ∏ s γ (Cz(s↓w)) z + n (Cz(s↓w)) z ∑ Cz(s) k ( γ (k) z + n (k) z ) Logical Constraints on Words In this section, we address logical expressions of two links using disjunctions (∨) and negations (¬), as well as conjunctions (∧), e.g., ¬ML(A, B) ∨ ML(A, C). We denote it as (∧,∨,¬)-expressions. Since each negation can be removed in a preprocessing stage, we focus only on (∧,∨)-expressions. Interpretation of negations is discussed in Sec. 3.4. (∧,∨)-expressions of Links We propose a simple method that simultaneously achieves conjunctions and disjunctions of links, where the existing method can only treat conjunctions of links. The key observation is that any Dirichlet trees constructed by MLs and CLs are essentially based only on two primitives. One is Ep(A, B) that equalizes the occurrence probabilities of A and B in a topic as in Fig. 1(a) , and the other is Np(A) that zeros the occurrence probability of A in a topic as in the left tree of Fig. 1(b) . The right tree of Fig. 1(b) is created by Np(B) ∧ Np(C). Thus, we can substitute ML and CL with Ep and Np as follows: ML(A, B) = Ep(A, B) CL(A, B) = Np(A) ∨ Np(B) Using this substitution, we can compile a (∧, ∨)expression of links to the corresponding Dirichlet trees with the following algorithm. 1. Substitute all links (ML and CL) with the corresponding primitives (Ep and Np). 2. Calculate the minimum DNF of the primitives. 3. Construct Dirichlet trees corresponding to the (monotone) monomials of the DNF. Let us consider three words A = 'kung-fu', B = 'jackie', and C = 'bruce' in Sec. 1. We want to constrain them with (ML(A, B) ∨ ML(A, C)) ∧ CL (B, C) . In this case, the algorithm calculates the minimum DNF of primitives as (ML(A, B) ∨ ML(A, C)) ∧ CL(B, C) = (Ep(A, B) ∨ Ep(A, C)) ∧ (Np(B) ∨ N p(C)) = (Ep(A, B) ∧ Np(B)) ∨ (Ep(A, B) ∧ Np(C)) ∨ (Ep(A, C) ∧ Np(B)) ∨ (Ep(A, C) ∧ Np(C)) and constructs four Dirichlet trees corresponding to the four monomials Ep(A, B) ∧ Np(B), Ep(A, B) ∧ Np(C), Ep(A, C) ∧ Np(B), and Ep(A, C) ∧ Np(C) in the last equation. Considering only (∧)-expressions of links, our method is equivalent to the existing method in the original framework in terms of an asymptotic behavior of Dirichlet trees. We define asymptotic behavior as Asymptotic Topic Family (ATF) as follows. Definition 1 (Asymptotic Topic Family). For any (∧, ∨)-expression f of primitives and any set W of words, we define the asymptotic topic family of f with respect to W as a family f * calculated by the following rules: Given (∧, ∨)-expressions f 1 and f 2 of primitives and words A, B ∈ W, (i) (f 1 ∨ f 2 ) * := f * 1 ∪ f * 2 (ii) (f 1 ∧ f 2 ) * := f * 1 ∩ f * 2 (iii) Ep * (A, B) := {∅, {A, B}} ⊗ 2 W−{A,B} , (iv) Np * (A) := 2 W−{A} Here, notation ⊗ is defined as X ⊗ Y := {x ∪ y | x ∈ X, y ∈ Y } for given two sets X and Y . ATF expresses all combinations of words that can occur in a topic when η is large. In the above example, the ATF of its expression with respect to W = {A, B, C} is calculated as ((ML(A, B) ∨ ML(A, C)) ∧ CL(B, C)) * = (Ep(A, B) ∨ Ep(A, C)) ∧ (Np(B) ∨ Np(C)) * = ( {∅, {A, B}} ⊗ 2 W−{A,B} ∪{∅, {A, C}} ⊗ 2 W−{A,C} ) ∩ ( 2 W−{B} ∪ 2 W−{C} ) = {∅, {B}, {C}, {A, B}, {A, C}}. As we expected, the ATF of the last equation indicates such a constraint that either A and B or A and C must appear in the same topic, and B and C cannot appear in the same topic. Note that the part of {B} satisfies ML(A, C) ∧ CL(B, C). If you want to remove {B} and {C}, you can use exclusive disjunctions. For the sake of simplicity, we omit descriptions about W when its instance is arbitrary or obvious from now on. The next theorem gives the guarantee of asymptotic equivalency between our method and the existing method. Let MIS(G) be the set of maximal independent sets of graph G. We define (x) ) is equivalent to the union of the power sets of every max- L := {{w, w ′ } | w, w ′ ∈ W, w ̸ = w ′ }. imal independent set S ∈ MIS(G) of a graph G := (W, ℓ), that is, ∪ X∈X (∩ x∈X Np * (x) ) = ∪ S∈MIS(G) 2 S . Proof. For any (∧)-expressions of links characterized by ℓ ⊆ L, we denote f ℓ and G ℓ as the corresponding minimum DNF and graph, respectively. We define U ℓ := ∪ S∈MIS(G ℓ ) 2 S . When |ℓ| = 1, f * ℓ = U ℓ is trivial. Assuming f * ℓ = U ℓ when |ℓ| > 1, for any set ℓ ′ := ℓ ∪ {{A, B}} with an additional link characterized by {A, B} ∈ L, we obtain f * ℓ ′ = ((Np(A) ∨ Np(B)) ∧ f ℓ ) * = (2 W−{A} ∪ 2 W−{B} ) ∩ U ℓ = ∪ S∈MIS(G ℓ ) ( (2 W−{A} ∩ 2 S ) ∪(2 W−{B} ∩ 2 S ) ) = ∪ S∈MIS(G ℓ ) (2 S−{A} ∪ 2 S−{B} ) = ∪ S∈MIS(G ℓ ′ ) 2 S = U ℓ ′ This proves the theorem by induction. In the last line of the above deformation, we used ∪ S∈MIS(G) 2 S = ∪ S∈IS(G) 2 S and MIS(G ℓ ′ ) ⊆ ∪ S∈MIS(G ℓ ) ((S − {A}) ∪ (S − {B})) ⊆ IS(G ℓ ′ ), where IS(G) represents the set of all independent sets on graph G. In the above theorem, ∪ X∈X (∩ x∈X Np * (x) ) represents asymptotic behaviors of our method, while ∪ S∈MIS(G) 2 S represents those of the existing method. By using a similar argument to the proof, we can prove the elements of the two sets are completely the same, i.e., ∩ x∈X Np * (x) = {2 S | S ∈ MIS(G)}. This interestingly means that for any logical expression characterized by CLs, calculating its minimum DNF is the same as calculating the maximal independent sets of the corresponding graph, or the maximal cliques of its complement graph. Shrinking Dirichlet Forests Focusing on asymptotic behaviors, we can reduce the number of Dirichlet trees, which means the performance improvement of Gibbs sampling for Dirichlet trees. This is achieved just by minimizing DNF on asymptotic equivalence relation defined as follows. Definition 3 (Asymptotic Equivalence Relation). Given two (∧, ∨)-expressions f 1 , f 2 , we say that f 1 is asymptotically equivalent to f 2 , if and only if f * 1 = f * 2 . We denote the relation as notation ≍, that is, f 1 ≍ f 2 ⇔ f * 1 = f * 2 . The next proposition gives an intuitive understanding of why asymptotic equivalence relation can shrink Dirichlet forests. Proposition 4. For any two words A, B ∈ W, (a) Ep(A, B) ∨ (Np(A) ∧ Np(B)) ≍ Ep(A, B) (b) Ep(A, B) ∧ Np(A) ≍ Np(A) ∧ Np(B) Proof. We prove (a) only. We conduct an experiment to clarify how many trees can be reduced by asymptotic equivalency. In the experiment, we prepare conjunctions of random links of MLs and CLs when |W| = 10, and compare the average numbers of Dirichlet trees compiled by minimum DNF (M-DNF) and asymptotic minimum DNF (AM-DNF) in 100 trials. The experimental result shown in Tab. 1 indicates that asymptotic equivalency effectively reduces the number of Dirichlet trees especially when the number of links is large. Customizing New Links Two primitives Ep and Np allow us to easily customize new links without changing the algorithm. Let us consider Imply-Link (A, B) or IL(A, B) , which is a constraint that B must appear if A appears in a topic (informally, A → B). In this case, the setting IL(A, B) = Ep(A, B) ∨ Np(A) is acceptable, since the ATF of IL(A, B) IL(A, B) is effective when B has multiple meanings as mentioned later in Sec. 4. with respect to W = {A, B} is {∅, {A, B}, {B}}. Informally regarding IL(A, B) as A → B and  ML(A, B) as A ⇔ B, ML(A, B) seems to be the same meaning of IL(A, B) ∧ IL(B, A) . However, this anticipation is wrong on the normal equivalency, i.e., ML(A, B) ̸ = IL(A, B) ∧ IL(B, A) . The asymptotic equivalency can fulfill the anticipation with the next proposition. This simultaneously suggests that our definition is semantically valid. IL(B, A) ≍ ML(A, B) Proof. From Proposition 4, Ep(A, B) = ML(A, B) Further, we can construct XIL(X 1 , · · · , X n , Y ) as an extended version of IL (A, B) , which allows us to use multiple conditions like Horn clauses. This informally means ∧ n i=1 X i → Y as an extension of A → B. In this case, we set Proposition 5. For any two words A, B ∈ W, IL(A, B) ∧ IL(A, B) ∧ IL(B, A) = (Ep(A, B) ∨ Np(A)) ∧ (Ep(B, A) ∨ Np(B)) = Ep(A, B) ∨ (Ep(A, B) ∧ Np(A)) ∨ (Ep(A, B) ∧ Np(B)) ∨ (Np(A) ∧ Np(B)) ≍ Ep(A, B) ∨ (Np(A) ∧ Np(B)) ≍ XIL(X 1 , · · · , X n , Y ) = n ∧ i=1 Ep(X i , Y )∨ n ∨ i=1 Np(X i ). When we want to isolate unnecessary words (i.e., stop words), we can use Isolate-Link (ISL) defined as ISL(X 1 , · · · , X n ) = n ∧ i=1 Np(X i ). This is easier than considering CLs between highfrequency words and unnecessary words as described in ). Negation of Links There are two types of interpretation for negation of links. One is strong negation, which regards ¬ML (A, B) as "A and B must not appear in the same topic", and the other is weak negation, which regards it as "A and B need not appear in the same topic". We set ¬ML(A, B) ≍ CL(A, B) for strong negation, while we just remove ¬ML(A, B) for weak negation. We consider the strong negation in this study. According to Def. 1, the ATF of the negation ¬f of primitive f seems to be defined as (¬f ) * := 2 W − f * . However, this definition is not fit in strong negation, since ¬ML(A, B) ̸ ≍ CL(A, B) on the definition. Thus we define it to be fit in strong negation as follows. Definition 6 (ATF of strong negation of links). Given a link L with arguments X 1 , · · · , X n , letting f L be the primitives of L, we define the ATF of the negation of L as (¬L(X 1 , · · · , X n )) * := (2 W − f * L (X 1 , · · · , X n )) ∪ 2 W−{X 1 ,··· ,Xn} . Note that the definition is used not for primitives but for links. Actually, the similar definition for primitives is not fit in strong negation, and so we must remove all negations in a preprocessing stage. The next proposition gives the way to remove the negation of each link treated in this study. We define no constraint condition as ϵ for the result of ISL. Proposition 7. For any words A, B, X 1 , · · · , X n , Y ∈ W, (a) ¬ML(A, B) ≍ CL(A, B) (b) ¬CL(A, B) ≍ ML(A, B) (c) ¬IL(A, B) ≍ Np(B) (d) ¬XIL(X 1 , · · · , X n , Y ) ≍ ∧ n−1 i=1 Ep(X i , X n ) ∧ Np(Y ) (e) ¬ISL(X 1 , · · · , X n ) ≍ ϵ Proof. We prove (a) only. (¬ML (A, B) ) * = (2 W − Ep * (A, B) (CL(A, B) ) * ) ∪ 2 W−{A,B} = (2 {A,B} − {∅, {A, B}}) ⊗ 2 W−{A,B} ∪ 2 W−{A,B} = {∅, {A}, {B}} ⊗ 2 W−{A,B} = 2 W−{A} ∪ 2 W−{B} = Np * (A) ∪ Np * (B) = Comparison on a Synthetic Corpus We experiment using a synthetic corpus {ABAB, ACAC} × 2 with vocabulary W = {A, B, C} to clarify the property of our method in the same way as in the existing work . We set topic size as T = 2. The goal of this experiment is to obtain two topics: a topic where A and B frequently occur and a topic where A and C frequently occur. We abbreviate the grouping type as AB|AC. In preliminary experiments, LDA yielded almost four grouping types: AB|AC, AB|C, AC|B, and A|BC. Thus, we naively classify a grouping type of each result into the four types. Concretely speaking, for any two topic-word probabilitiesφ andφ ′ , we calculate the average of Euclidian distances between each vector component ofφ and the corresponding one ofφ ′ , ignoring the difference of topic labels, and regard them as the same type if the average is less than 0.1. Fig. 2 shows the occurrence rates of grouping types on 1,000 results after 1,000 iterations by LDA-DF with six constraints (1) no constraint, better. The results of (1-4) can be achieved even by the existing method, and those of (5-6) can be achieved only by our method. Roughly speaking, the figure shows that our method is clearly better than the existing method, since our method can obtain almost 100% as the rate of AB|AC, which is the best of all results, while the existing methods can only obtain about 60%, which is the best of the results of (1-4). The result of (1) is the same result as LDA, because of no constraints. In the result, the rate of AB|AC is only about 50%, since each of AB|C, AC|B, and A|BC remains at a high 15%. As we expected, the result of (2) shows that ML(A, B) cannot remove AB|C although it can remove AC|B and A|BC, while the result of (3) shows that CL(B, C) cannot remove AB|C and AC|B although it can remove A|BC. The result of (4) indicates that ML(A, B) ∧ CL(B, C) is the best of knowledge expressions in the existing method. Note that ML(A, B) ∧ ML(A, C) implies ML(B, C) by transitive law and is inconsistent with all of the four types. The result (80%) of (5) IL (B, A) is interestingly better than that (60%) of (4), despite that (5) has less primitives than (4). The reason is that (5) allows A to appear with C, while (4) does not. In the result of (6) ML (A, B)∨ML(A, C) , the constraint achieves almost 100%, which is the best of knowledge expressions in our method. Of course, the constraint of (ML(A, B) ∨ ML(A, C)) ∧ CL(B, C) can also achieve almost 100%. Interactive Topic Analysis We demonstrate advantages of our method via interactive topic analysis on a real corpus, which consists of stemmed, down-cased 1,000 (positive) movie reviews used in (Pang and Lee, 2004) . In this experiment, the parameters are set as α = 1, β = 0.01, η = 1000, and T = 20. We first ran LDA-DF with 1,000 iterations without any constraints and noticed that most topics have stop words (e.g., 'have' and 'not') and corpus-specific, unnecessary words (e.g., 'film', 'movie'), as in the first block in Tab. 2. To remove them, we added ISL('film', 'movie', 'have', 'not', 'n't') to the constraint of LDA-DF, which is compiled to one Dirichlet tree. After the second run of LDA-DF with the isolate-link, we specified most topics such as Comedy, Disney, and Family, since cumbersome words are isolated, and so we noticed that two topics about Star Wars and Star Trek are merged, as in the second block. Each topic label is determined by looking carefully at highfrequency words in the topic. To split the merged two topics, we added CL ('jedi', ' trek') to the constraint, which is compiled to two Dirichlet trees. However, after the third run of LDA-DF, we noticed that there is no topic only about Star Trek, since 'star' appears only in the Star Wars topic, as in the third block. Note that the topic including 'trek' had other topics such as a topic about comedy film Big Lebowski. We finally added ML('star', 'jedi') ∨ ML ('star', ' trek') to the constraint, which is compiled to four Dirichlet trees, to split the two topics considering polysemy of 'star'. After the fourth run of LDA-DF, we appropriately obtained two topics about Star Wars and Star Trek as in the fourth block. Note that our solution is not ad-hoc, and we can easily apply it to similar problems. Conclusions We proposed a simple method to achieve topic models with logical constraints on words. Our method compiles a given constraint to the prior of LDA-DF, which is a recently developed semisupervised extension of LDA with Dirichlet forest priors. As well as covering the constraints in the original LDA-DF, our method allows us to construct new customized constraints without changing the algorithm. We proved that our method is asymptotically the same as the existing method for any constraints with conjunctive expressions, and showed that asymptotic equivalency can shrink a constructed Dirichlet forest. In the comparative Table 2 : Characteristic topics obtained in the experiment on the real corpus. Four blocks in the table corresponds to the results of the four constraints ϵ, ISL(· · · ), CL('jedi', 'trek') ∧ ISL(· · · ), and (ML('jedi', 'trek') ∨ ML('star', 'trek')) ∧ CL('jedi', 'trek') ∧ ISL(· · · ), respectively. Topic High frequency words in each topic ? have give night film turn performance ? not life have own first only family tell ? movie have n't get good not see ? have black scene tom death die joe ? film have n't not make out well see Isolated have film movie not good make n't ? star war trek planet effect special Comedy comedy funny laugh school hilarious Disney disney voice mulan animated song Family life love family mother woman father Isolated have film movie not make good n't StarWars star war lucas effect jedi special ? science world trek fiction lebowski Comedy funny comedy laugh get hilarious Disney disney truman voice toy show Family family father mother boy child son Isolated have film movie not make good n't StarWars star war toy jedi menace phantom StarTrek alien effect star science special trek Comedy comedy funny laugh hilarious joke Disney disney voice animated mulan Family life love family man story child study on a synthetic corpus, we clarified the property of our method, and in the interactive topic analysis on a movie review corpus, we demonstrated its effectiveness. In the future, we intend to address detail comparative studies on real corpora and consider a simple method integrating negations into a whole, although we removed them in a preprocessing stage in this study.
1 2 3 3.1 3.2 3.3 3.4 4 5 6
Introduction LDA with Dirichlet Forest Priors Logical Constraints on Words (∧,∨)-expressions of Links Shrinking Dirichlet Forests Customizing New Links Negation of Links Comparison on a Synthetic Corpus Interactive Topic Analysis Conclusions
This work
Logical constraints on words for topic modeling
Conjunctions (), disjunctions (), negations ()
W ant to split into fire dance
ML dance dance CL fire sexy
ancient bikini Must-Link(dance, sexy))
Algorithm to generate logically constrained distributions on LDA-DF
We can not apply the existing algorithm
This constraint cannot be
mapped to a graph
Logical constraints on words for topic modeling
Conjunctions (), disjunctions (), negations ()
W ant to split into fire dance
ML dance dance CL fire sexy
ancient bikini Must-Link(dance, sexy))
Algorithm to generate logically constrained distributions on LDA-DF
We can not apply the existing algorithm
This constraint cannot be
mapped to a graph