# embedding_metadata

key: chroma:document
string_value:
contextual-bandit problems,” Journal of Machine Learning Research , vol. 13, pp. 2069–
2106, 2012.
[48] O. Chapelle and L. Li, “An Empirical Evaluation of Thompson Sampling,” in Advances in
Neural Information Processing Systems 24 , J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett,
F. Pereira, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2011, pp. 2249–2257.
[49] P. Auer, “Using conﬁdence bounds for exploitation-exploration trade-oﬀs,” in Journal of


key: chroma:document
string_value:
https://doi.or g/10.1007/s10844-021-00682-5
Arelationaltsetlinmachinewithapplications
tonaturallanguageunderstanding
RupsaSaha1·Ole-ChristoﬀerGranmo1·VladimirI.Zadorozhny1,2·
MortenGoodwin1
Received:19March2021/Revised:11October2021/Accepted:11October2021/
©TheAuthor(s)2021
Abstract
Tsetlin machines (TMs) are a pattern recognition approach that uses finite state machines
for learning and propositional logic to represent patterns. In addition to being natively


key: chroma:document
string_value:
interpretable, they have provided competitive accuracy for various tasks. In this paper, we
increase the computing power of TMs by proposing a first-order logic-based framework
with Herbrand semantics. The resulting TM is relational and can take advantage of logical
structures appearing in natural language, to learn rules that represent how actions and con-
sequences are related in the real world. The outcome is a logic program of Horn clauses,


key: chroma:document
string_value:
bringing in a structured view of unstructured data. In closed-domain question-answering,
the first-order representation produces 10 ×more compact KBs, along with an increase
in answering accuracy from 94.83% to 99.48%. The approach is further robust towards
erroneous, missing, and superfluous information, distilling the aspects of a text that are
important for real-world understanding
Keywords Tsetlin machine ·Natural language processing ·Logic programming ·


key: chroma:document
string_value:
Knowledge representation ·Question answering
1 Introduction
Using Artificial Intelligences (AI) to answer natural language questions has long been an
active research area, considered as an essential aspect in machines ultimately achieving
human-level world understanding. Large-scale structured knowledge bases (KBs), such as
Freebase (Bollacker et al., 2008 ), have been a driving force behind successes in this field.


key: chroma:document
string_value:
The KBs encompass massive ever-growing amounts of information, which enable easier
handling of Open-Domain Question-Answering (QA) (Prager, 2006 ) by organizing a large
/envelopebackRupsa Saha
rupsa.saha@uia.no
1Centre for AI Research, Department of IKT, University of Agder, Agder, Norway
2School of Computing and Information, University of Pittsburgh, Pittsburgh, PA, USAPublished online: 1 January 2022Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
variety of answers in a structured format. The difficulty arises in successfully interpret-
ing natural language by artificially intelligent agents, both to build the KBs from naturallanguage text resources and to interpret the questions asked.
Generalization beyond the information stored in a KB further complicates the QA prob-


key: chroma:document
string_value:
lem. Human-level world understanding requires abstracting from specific examples to buildmore general concepts and rules. When the information stored in the KB is error-freeand consistent, generalization becomes a standard inductive reasoning problem. How-ever, abstracting world-knowledge entails dealing with uncertainty, vagueness, exceptions,errors, and conflicting information. This is particularly the case when relying on Artifi-cial Intelligences (AI) approaches to extract and structure


key: chroma:document
string_value:
This paper addresses the above QA challenges by proposing a Relational Tsetlin machine
(TM) that builds non-recursive first-order Horn clauses from specific examples, distilling
general concepts and rules.
Tsetlin Machines (Granmo, 2018) are a pattern recognition approach to constructing
human-understandable patterns from given data, founded on propositional logic. While theidea of Tsetlin automaton (TA) (Tsetlin, 1961) have been around since 1960s, using them


key: chroma:document
string_value:
in pattern recognition is relatively new. TMs have successfully addressed several machinelearning tasks, including natural language understanding (Yadav et al., 2021a, b;B h a t -
tarai et al., 2021;S a h ae ta l . ,2020; Berge et al., 2019), image analysis (Granmo et al.,
2019), classification (Abeyrathna et al., 2021), regression (Abeyrathna et al., 2019), and
speech understanding (Lei et al., 2021). The propositional clauses constructed by a Tsetlin


key: chroma:document
string_value:
machine (TM) have high discriminative power and constitute a global description of thetask learnt (Blakely & Granmo, 2020;S a h ae ta l . ,2020). Apart from maintaining accuracy
comparable to state-of-the-art machine learning techniques, the method also has provided asmaller memory footprint and faster inference than more traditional neural network-basedmodels (Wheeldon et al., 2020; Lei et al., 2020; Abeyrathna et al., 2021; Lei et al., 2021;


key: chroma:document
string_value:
Phoulady et al., 2019). Furthermore, Shafik et al. ( 2020) shows that TMs can be fault-
tolerant, able to mask stuck-at faults. However, although TMs can express any propositionalformula by using disjunctive normal form, first-order logic is required to obtain the com-puting power equivalent to a universal Turing machine. In this paper, we take the first stepstowards increasing the computing power of TM by introducing a first order TM framework


key: chroma:document
string_value:
with Herbrand semantics, referred to as the Relational TM. Accordingly, we will in the
following denote the original approach as Propositional TM .
Closed-DomainQuestion-Answering: As proof-of-concept, we apply our proposed Rela-


key: chroma:document
string_value:
tional TM to so-called Closed-Domain QA. Closed-Domain QA assumes a text (single ormultiple sentences) followed by a question which refers to some aspect of the precedingtext. Accordingly, the amount of information that must be navigated is less than for openquestion-answering. Yet, answering closed-domain questions poses a significant naturallanguage understanding challenge.
Consider the following example of information, taken from Rajpurkar et al. ( 2016): “The


key: chroma:document
string_value:
Black Death is thought to have originated in the arid plains of Central Asia, where it thentravelled along the Silk Road, reaching Crimea by 1343. From there, it was most likelycarried by Oriental rat fleas living on the black rats that were regular passengers on merchantships.”One can then have questions such as “Where did the black death originate?” or “Howdid the black death make it to the Mediterranean and Europe?”. These questions can be122 Journal of Intelligent Information Systems (2022)


key: chroma:document
string_value:
answered completely with just the information provided, hence it is an example of closed-
domain question answering. However, mapping the question to the answer requires not onlynatural language processing, but also a fair bit of language understanding.
Here is a much simpler example: “Bob went to the garden. Sue went to the cafe. Bob


key: chroma:document
string_value:
walked to the office.” This information forms the basis for questions like “Where is Bob?” or“Where is Sue?”. Taking it a step further, given the previous information and questions, onecan envision a model that learns to answer similar questions based on similar information,even though the model has never seen the specifics of the information before (i.e., the namesand the locations).
With QA being such an essential area of Natural Language Understanding, there has been


key: chroma:document
string_value:
a lot of different approaches proposed. Common methods to QA include the following:
– Linguistic techniques, such as tokenization, POS tagging and parsing that transform
questions into a precise query that merely extracts the respective response from a
structured database;
– Statistical techniques such as Support Vector Machines, Bayesian Classifiers, and
maximum entropy models, trained on large amount of data, specially for open QA;


key: chroma:document
string_value:
– Pattern matching using surface text patterns with templates for response generation.
Many methods use a hybrid approach encompassing more than one of these approaches for
increased accuracy. Most QA systems suffer from a lack of generality, and are tuned forperformance in restricted use cases. Lack of available explainabilty also hinders researchers’quest to identify pain points and possible major improvements (Soares & Parreiras, 2020;
Pundge et al., 2016).


key: chroma:document
string_value:
Pundge et al., 2016).
PaperContributions: Our main contributions in this paper are as follows:
– We introduce a Relational TM, as opposed to a propositional one, founded on non-
recursive Horn clauses and capable of processing relations, variables and constants.
– We propose an accompanying relational framework for efficient representation and
processing of the QA problem.
– We provide empirical evidence uncovering that the Relational TM produces at least one


key: chroma:document
string_value:
order of magnitude more compact KBs than the Propositional TM. At the same time,answering accuracy increases from 94.83% to 99.48% because of more general rules.
– We provide a model-theoretical interpretation for the proposed framework.
Overall, our Relational TM unifies knowledge representation, learning, and reasoning in a
single framework.
Paper Organization: The paper is organized as follows. In Section 2, we present related


key: chroma:document
string_value:
work on Question Answering. Section 3focuses on the background of the Propositional TM
and the details of the new Relational TM. In Sections 4and5, we describe how we employ
Relational TM in QA and related experiments.
2 Backgroundandrelatedwork
The problem of QA is related to numerous aspects of Knowledge Engineering and DataManagement.123 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Knowledge engineering deals with constructing and maintaining knowledge bases to
store knowledge of the real world in various domains. Automated reasoning techniques


key: chroma:document
string_value:
use this knowledge to solve problems in domains that ordinarily require human logicalreasoning. Therefore, the two key issues in knowledge engineering are how to constructand maintain knowledge bases, and how to derive new knowledge from existing knowledgeeffectively and efficiently. Automated reasoning is concerned with the building ofcomputing systems that automate this process. Although the overall goal is to automatedifferent forms of reasoning, the term has largely been identified with


key: chroma:document
string_value:
the term has largely been identified with valid deductivereasoning as conducted in logical systems. This is done by combining known (yet possi-bly incomplete) information with background knowledge and making inferences regardingunknown or uncertain information.


key: chroma:document
string_value:
knowledge base itself, inference engine, explanation subsystem and user interface. Theknowledge model has to represent the relations between multiple components in a symbolic,machine understandable form, and the inference engine has to manipulate those symbolsto be capable of reasoning. The “way to reason” can range from earlier versions that weresimple rule-based systems to more complex and recent approaches based on machine learn-ing, especially on Deep Learning. Recently, transformer based


key: chroma:document
string_value:
on Deep Learning. Recently, transformer based systems have significantlyimproved question-answering performance in general, even more so when they can be sup-plied with a very large amount of training data (Lu et al., 2020; Lewis et al., 2019;W o l f


key: chroma:document
string_value:
et al., 2019; Jung & Kim, 2020;S r i n i v a s a ne ta l . , 2021). Typically, rule-based systems suf-
fered from lack of generality, and the need for human experts to create rules in the firstplace. On the other hand most machine learning based approaches have the disadvantage ofnot being able to justify decisions taken by them in a human understandable form Ludwig(2010) and Cyras et al. (2020).
While databases have long been a mechanism of choice for storing information, they


key: chroma:document
string_value:
only had inbuilt capability to identify relations between various components, and did nothave the ability to support reasoning based on such relations. Efforts to combine formallogic programming with relational databases led to the advent of deductive databases. Infact, the field of QA is said to have arisen from the initial goal of performing deduc-tive reasoning on a set of given facts (Green, 1969). In deductive databases, the semantics


key: chroma:document
string_value:
of the information are represented in terms of mathematical logic. Queries to deductivedatabases also follow the same logical formulation (Gallaire et al., 1989). One such exam-
ple is ConceptBase (Jarke et al., 1995), which used the Prolog-inspired language O-Telos
for logical knowledge representation and querying using deductive object-oriented databaseframework.
With the rise of the internet, there came a need for unification of information on the web.


key: chroma:document
string_value:
The Semantic Web (SW) proposed by W3C is one of the approaches that bridges the gapbetween the Knowledge Representation and the Web Technology communities. However,reasoning and consistency checking is still not very well developed, despite the underlyingformalism that accompanies the semantic web. One way of introducing reasoning is viadescriptive logic. It involves concepts (unary predicates) and roles (binary predicates) andthe idea is that implicitly captured knowledge can be inferred from


key: chroma:document
string_value:
captured knowledge can be inferred from the given descriptionsof concepts and roles (Dong et al., 2003; Turhan, 2011).


key: chroma:document
string_value:
One of the major learning exercises is carried out by the NELL mechanism proposed
by Mitchell et al. ( 2018), which aims to learn many semantic categories from primarily
unlabeled data. At present, NELL uses simple frame-based knowledge representation, aug-mented by the PRA reasoning system. The reasoning system performs tractable, but limited124 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
types of reasoning based on restricted Horn clauses. NELL’s capabilities is already lim-
ited in part by its lack of more powerful reasoning components; for example, it currentlylacks methods for representing and reasoning about time and space. Hence, core ArtificialIntelligence (AI) problems of representation and tractable reasoning are also core researchproblems for never-ending learning agents.
While other approaches such as neural networks are considered to provide attribute-based


key: chroma:document
string_value:
learning, Inductive Logic Programming (ILP) is an attempt to overcome their limitations bymoving the learning away from the attributes themselves and more towards the level of first-order predicate logic. ILP builds upon the theoretical framework of logic programming andlooks to construct a predicate logic given background knowledge, positive examples andnegative examples. One of the main advantages of ILP over attribute-based learning is ILP’sgenerality of representation for background


key: chroma:document
string_value:
ILP’sgenerality of representation for background knowledge. This enables the user to provide, ina more natural way, domain-specific background knowledge to be used in learning. The useof background knowledge enables the user both to develop a suitable problem representationand to introduce problem-specific constraints into the learning process. Over the years, ILPhas evolved from depending on hand-crafted background knowledge only, to employingdifferent technologies in order to learn the


key: chroma:document
string_value:
technologies in order to learn the background knowledge as part of the process. Incontrast to typical machine learning, which uses feature vectors, ILP requires the knowledgeto be in terms of facts and rules governing those facts. Predicates can either be supplied ordeducted, and one of the advantages of this method is that newer information can be addedeasily, while previously learnt information can be maintained as required (Cropper et al.,2020). Probabilistic inductive logic programming is


key: chroma:document
string_value:
Probabilistic inductive logic programming is an extension of ILP, where logic rules,as learnt from the data, are further enhanced by learning probabilities associated with suchrules (Bratko & Muggleton, 1995; Nickles & Mileo, 2014; De Raedt & Kersting, 2008;


key: chroma:document
string_value:
Kramer et al., 2001).
To sum up, none of the above approaches can be efficiently and systematically applied to
the QA problem, especially in uncertain and noisy environments. In this paper we proposea novel approach to tackle this problem. Our approach is based on relational representationof QA, and using a novel Relational TM technique for answering questions. We elaborateon the proposed method in the next two sections.
3 Buildingarelationaltsetlinmachine
3.1 Tsetlinmachinefoundation


key: chroma:document
string_value:
3.1 Tsetlinmachinefoundation
A Tsetlin Automaton (Tsetlin Automaton (TA)) is a deterministic automaton that learns theoptimal action among the set of actions offered by an environment. It performs the actionassociated with its current state, which triggers a reward or penalty based on the groundtruth. The state is updated accordingly, so that the Tsetlin automaton (TA) progressivelyshifts focus towards the optimal action (Tsetlin, 1961). A 2-action Tsetlin automaton (TA),


key: chroma:document
string_value:
as shown in Fig. 1a, has 2N states, such that, a final state between 1 and Nindicates Action
1, while final state between N+1a n d2 Nindicates Action 2. A reward reinforces a par-
ticular action by changing state away from the central state, i.e. moving “deeper” towardsAction 1 or 2, as the case maybe. The opposite happens in the case of a penalty, where theresultant state after an iteration is closer to the central states ( N/N+1). A TM consists


key: chroma:document
string_value:
of a collection of such TAs, which together create complex propositional formulas usingconjunctive clauses.125 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
3.1.1 Classiﬁcation
A TM takes a vector X=(x1,...,x f)of propositional features as input, to be classified
into one of two classes, y=0o ry =1. Together with their negated counterparts, ¯xk=
¬xk=1−xk, the features form a literal set L={x1,...,x f,¯x1,...,¯xf}. We refer to
this “regular” TM as a Propositional TM, due to the input it works with and the output it
produces.
A TM pattern is formulated as a conjunctive clause Cj, formed by ANDing a subset
Lj⊆Lof the literal set:
Cj(X)=⋀
lk∈Ljlk=∏


key: chroma:document
string_value:
Lj⊆Lof the literal set:
Cj(X)=⋀
lk∈Ljlk=∏
lk∈Ljlk.( 1)
E.g., the clause Cj(X)=x1∧x2=x1x2consists of the literals Lj={x1,x2}and outputs
1i f fx 1=x2=1.
Each clause has access to 2 ×fTsetlin automaton (TAs), one per literal k. The bidirec-
tional action of each TA decides which of the literals form a part of a clause. The TA (TA)states from 1 to Nmap to the exclude action, which means that the corresponding literal


key: chroma:document
string_value:
Fig.1 Clause Formation in a Tsetlin Machine126 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
is excluded from the clause. For states from N+1t o2N, the decision becomes include,
i.e., the literal is included instead. The states of all the TAs in all of the clauses are jointly
stored in the matrix A:A=(aj,k)ϵ{1,...,2N }m×2fwithjandkreferring to the clause
and the literal respectively. Thus II
j={k|aj,k>N , 1≤k≤2f}encompasses the literal
indices contained in II
j. Figure 1b shows the structure of the clauses and the voting scheme
diagrammatically.


key: chroma:document
string_value:
diagrammatically.
The number of clauses employed is a user set parameter m.H a l fo ft h em clauses are
assigned positive polarity (C+
j). The other half is assigned negative polarity ( C−
j). The
clause outputs, in turn, are combined into a classification decision through summation:
v=∑m/2
j=1C+
j(X)−∑m/2
j=1C−
j(X).( 2)
In effect, the positive clauses vote for y=1a n dt h en e g a t i v ef o ry =0. Classification is


key: chroma:document
string_value:
performed based on a majority vote, using the unit step function: ˆy=u(v)=1ifv≥
0else0. The classifier ˆy=u(x1¯x2+¯x1x2−x1x2−¯x1¯x2), for instance, captures the XOR-
relation.
3.1.2 Learning
Alg. 1 encompasses the entire learning procedure. We observe that learning is performed bya team of 2f Tsetlin automaton (TAs) per clause, one Tsetlin automaton (TA) per literal l
k


key: chroma:document
string_value:
k
(Alg. 1, Step 2). Each Tsetlin automaton (TA) has two actions – Include or Exclude – anddecides whether to include its designated literal l
kin its clause.
TM learn on-line, processing one training example (X,y) at a time (Step 7). The Tsetlin
automaton (TAs) first produce a new configuration of clauses (Step 8), C+
1,...,C−
n/2,
followed by calculating a voting sum v(Step 9).
Feedback are then handed out stochastically to each Tsetlin automaton (TAs) team. The


key: chroma:document
string_value:
difference ϵbetween the clipped voting sum vcand a user-set voting target Tdecides the
probability of each Tsetlin automaton (TAs) team receiving feedback (Steps 12-20). Notethat the voting sum is clipped to normalize the feedback probability. The voting target fory=1i sTand for y=0i ti s−T. Observe that for any input X, the probability of reinforc-


key: chroma:document
string_value:
ing a clause gradually drops to zero as the voting sum approaches the user-set target. Thisensures that clauses distribute themselves across the frequent patterns, rather than missingsome and over-concentrating on others.
Clauses receive two types of feedback. Type I feedback produces frequent patterns, while
Type II feedback increases the discrimination power of the patterns.
Type I feedback is given stochastically to clauses with positive polarity when y=1


key: chroma:document
string_value:
and to clauses with negative polarity when y=0. Each clause, in turn, reinforces its Tsetlin
automaton (TAs) based on: (1) its output C
j(X); (2) the action of the Tsetlin automaton (TA)
– Include or Exclude; and (3) the value of the literal lkassigned to the Tsetlin automaton
(TA). Two rules govern Type I feedback:
– Include is rewarded and Exclude is penalized with probabilitys−1
swhenever Cj(X)=
1andlk=1. This reinforcement is strong (triggered with high probability) and makes


key: chroma:document
string_value:
the clause remember and refine the pattern it recognizes in X.1
– Include is penalized and Exclude is rewarded with probability1
swhenever Cj(X)=
0orlk=0. This reinforcement is weak (triggered with low probability) and coarsens
infrequent patterns, making them frequent.
1Note that the probabilitys−1
sis replaced by 1 when boosting true positives.127 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Above, the user-configurable parameter scontrols pattern frequency, i.e., a higher s
produces less frequent patterns.
Type II feedback is given stochastically to clauses with positive polarity when y=0
and to clauses with negative polarity when y=1. It penalizes Exclude whenever Cj(X)=
1andlk=0. Thus, this feedback produces literals for discriminating between y=0a n d
y=1, by making the clause evaluate to 0 when facing its competing class. Further details


key: chroma:document
string_value:
3.2 Relationaltsetlinmachine
In this section, we introduce the Relational TM, which is a major contribution of this paper.
It is designed to take advantage of logical structures appearing in natural language andprocess them in a way that leads to a compact, logic-based representation, which can ulti-mately reduce the gap between structured and unstructured data. While the PropositionalTM operates on propositional input variables X=(x
1...,x f), building propositional con-


key: chroma:document
string_value:
1...,x f), building propositional con-
junctive clauses, the Relational TM processes relations, variables and constants, buildingHorn clauses. Based on Fig. 2and Alg. 2, we here describe how the Relational TM builds


key: chroma:document
string_value:
upon the original TM in three steps. First, we establish an approach for dealing with rela-tions and constants. This is done by mapping the relations to propositional inputs, allowingthe use of a standard TM. We then introduce Horn clauses with variables, showing howthis representation detaches the TM from the constants, allowing for a more compact rep-resentation compared to only using propositional clauses. We finally introduce a novelconvolution scheme that effectively manages multiple


key: chroma:document
string_value:
scheme that effectively manages multiple possible mappings from constants tovariables. While the mechanism of convolution remains the same as in the original (Granmoet al., 2019), what we wish to attain from using it in a Relational TM context is completely


key: chroma:document
string_value:
different, as explained in the following.
3.2.1 Model-theoreticalinterpretation
The concept of Relational TM can be grounded in the model-theoretical interpretation of alogic program without functional symbols and with a finite Herbrand model (Lloyd, 1984;
Kowalski, 2014). The ability to represent learning in the form of Horn clauses is extremely
useful due to the fact that Horn clauses are both simple, as well as powerful enough todescribe any logical formula (Kowalski, 2014).


key: chroma:document
string_value:
N e x t ,w ed e f i n et h eHerbrand model of a logic program. A Herbrand Base (HB) is the set
of all possible ground atoms, i.e., atomic formulas without variables, obtained using predi-cate names and constants in a logic program P.A Herbrand Interpretation i sas u b s e t Iof
the Herbrand Base (I⊆HB). To introduce the Least Herbrand Model we define the imme-
diate consequence operator TP:P(HB)→P(HB),w h i c hf o ra nHerbrand Interpretation


key: chroma:document
string_value:
Iproduces the interpretation that immediately follows from Iby the rules (Horn clauses)
in the program P:
TP(I) ={A
0∈HB|A0←A1,...,A n
∈ground(P) ∧{A1,...,A n}⊆I}∪I.
The least fixed point lfp(TP) of the immediate consequence operator with respect to
subset-inclusion is the Least Herbrand Model (LHM) of the program P.LHM identifies the
semantics of the program P:i ti st h e Herbrand Interpretation that contains those and only
those atoms that follow from the program:
∀A∈HB:P|=A⇔A∈LHM.


key: chroma:document
string_value:
∀A∈HB:P|=A⇔A∈LHM.
As an example, consider the following program P:
p(a) .q(c).
q(X)←p(X) .
Its Herbrand base is
HB={p(a),p(c),q(a),q(c)},129 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Fig.2 Relational TM processing steps130 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
and its Least Herbrand Model is:
LHM =lfp(TP)={p(a),q(a),q(c)},
which is the set of atoms that follow from the program P.
3.2.2 Learningproblem
LetA={a1,a2,...,a q}be a finite set of constants and let R={r1,r2,...,r p}be a
finite set of relations of arity wu≥1,u∈{1,2,...,p }, which forms the alphabet /Sigma1.T h e
Herbrand base
HB ={r1(a1,a2,...,a w1),r1(a2,a1,...,a w1),
...,r p(a1,a2,...,a wp),rp(a2,a1,...,a wp),...} (3)


key: chroma:document
string_value:
is then also finite, consisting of all the ground atoms that can be expressed using AandR.
We also have a logic program P, with program rules expressed as Horn clauses without
recursion. Each Horn clause has the form:
B0←B1,B2,···,Bd.( 4)
Here,Bl,l∈{0,...,d },is an atom ru(Z1,Z2,...,Z wu)with variables Z1,Z2,...,
Zwu, or its negation ¬ru(Z1,Z2,...,Z wu). The arity of ruis denoted by wu.
Now, letXbe a subset of the LHM of P,X⊆lfp(TP),a n dl e t Ybe the subset of the


key: chroma:document
string_value:
LHM that follows from Xdue to the Horn clauses in P. Further assume that atoms are
randomly removed and added to XandYto produce a possibly noisy observation (˜X,˜Y),
i.e.,˜Xand˜Yare not necessarily subsets of lfp (TP).The learning problem is to predict the
atoms inYfrom the atoms in ˜Xby learning from a sequence of noisy observations (˜X,˜Y),
thus uncovering the underlying program P.
3.2.3 Relationaltsetlinmachinewithconstants


key: chroma:document
string_value:
3.2.3 Relationaltsetlinmachinewithconstants
We base our Relational TM on mapping the learning problem to a Propositional TM pattern
recognition problem. We consider Horn clauses without variables first. In brief, we mapevery atom in HBto a propositional input x
k, obtaining the propositional input vector X=
(x1,...,x o)(cf. Section 3.1). That is, consider the w-arity relation ru∈R,w h i c ht a k e s
wsymbols from Aas input. This relation can thus take qwunique input combinations. As


key: chroma:document
string_value:
an example, with the constants A={a1,a2}and the binary relations R={r1,r2},w e
get 8 propositional inputs: x1,1
1≡r1(a1,a1);x1,2
1≡r1(a1,a2);x2,1
1≡r1(a2,a1);x2,2
1≡
r1(a2,a2);x1,1
2≡r2(a1,a1);x1,2
2≡r2(a1,a2);x2,1
2≡r2(a2,a1);a n dx2,2
2≡r2(a2,a2).
Correspondingly, we perform the same mapping to get the propositional output vector Y.
Finally, obtaining an input (˜X,˜Y), we set the propositional input xkto true iff its corre-


key: chroma:document
string_value:
sponding atom is in ˜X, otherwise it is set to false. Similarly, we set the propositional output
variable ymto true iff its corresponding atom is in ˜Y, otherwise it is set to false.
Clearly, after this mapping, we get a Propositional TM pattern recognition problem that
can be solved as described in Section 3.1for a single propositional output ym.T h i si s
illustrated as Step 1 in Fig. 2.
3.2.4 DetachingtherelationalTMfromconstants


key: chroma:document
string_value:
3.2.4 DetachingtherelationalTMfromconstants
The TM can potentially deal with thousands of propositional inputs. However, we now detachour Relational TM from the constants, introducing Horn clauses with variables. Our intent131 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
is to provide a more compact representation of the program and to allow generalization
beyond the data. Additionally, the detachment enables faster learning even with less data.
LetZ={Z1,Z2,...,Z z}bezvariables representing the constants appearing in an
observation (˜X,˜Y). Here, zis the largest number of unique constants involved in any
particular observation (˜X,˜Y), each requiring its own variable.
Seeking Horn clauses with variables instead of constants, we now only need to consider


key: chroma:document
string_value:
atoms over variable configurations (instead of over constant configurations). Again, wemap the atoms to propositional inputs to construct a propositional TM learning problem.That is, each propositional input x
krepresents a particular atom with a specific variable
configuration: xk≡ru(Zα1,Zα2,...,Z αwu), with wubeing the arity of ru. Accordingly,
the number of constants in Ano longer affects the number of propositional inputs xkneeded


key: chroma:document
string_value:
to represent the problem. Instead, this is governed by the number of variables in Z(and,
again, the number of relations in R). That is, the number of propositional inputs is bounded
byO(zw), with wbeing the largest arity of the relations in R.
To detach the Relational TM from constants, we first replace the constants in ˜Ywith vari-
ables, from left to right. Accordingly, the corresponding constants in ˜Xis also replaced with


key: chroma:document
string_value:
the same variables (Step 2, Fig. 2). Finally, the constants now remaining in ˜Xis arbitrarily
replaced with additional variables (Step 3, Fig. 2).
3.2.5 Relationaltsetlinmachineconvolutionovervariableassignmentpermutations


key: chroma:document
string_value:
Since there may be multiple ways of assigning variables to constants, the above approachmay produce redundant rules. One may end up with equivalent rules whose only differ-ence is syntactic, i.e., the same rules are expressed using different variable symbols. Thisis illustrated in Step 3 of Fig. 2, where variables can be assigned to constants in two ways.


key: chroma:document
string_value:
To avoid redundant rules, the Relational TM produces all possible permutations of variableassignments. To process the different permutations, we finally perform a convolution overthe permutations in Step 4, employing a TM convolution operator (Granmo et al., 2019).
The target value of the convolution is the truth value of the consequent (Step 5).
3.2.6 Walk-throughofalgorithmwithexamplelearningproblem
Figure 2contains an example of the process of detaching a Relational TM from constants.


key: chroma:document
string_value:
We use the parent-grandparent relationship as an example, employing the following Hornclause.
grandparent(Z
1,Z2)←parent(Z 1,Z3),parent(Z 3,Z2).
We replace the constants in each training example with variables, before learning theclauses. Thus the Relational TM never “sees” the constants, just the generic variables.
Assume the first training example isIn put :parent(Bob, Mary) =1;
Targe to u t pu t :child(Mary, Bob) =1.
Then Mary is replaced with Z
1and Bob with Z2in the target output:


key: chroma:document
string_value:
1and Bob with Z2in the target output:
In put :parent(Bob, Mary) =1;
Targe to u t pu t :child(Z 1,Z 2)=1.
We perform the same exchange for the input, getting:In put :parent(Z
2,Z 1)=1;
Targe to u t pu t :child(Z 1,Z 2)=1.
Here, “parent(Z 2,Z1)” is treated as an input feature by the Relational TM. That is,
“parent(Z 2,Z1)” is seen as a single propositional variable that is either 0 or 1, and the name132 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
of the variable is simply the string “parent( Z2,Z1)”. The constants may be changing from
example to example, so next time it may be Mary and Ann. However, they all end up as Z1
orZ2after being replaced by the variables. After some time, the Relational TM would then
learn the following clause:
child(Z 1,Z 2)←parent(Z 2,Z1).
This is because the feature “parent( Z2,Z1)” predicts “child( Z1,Z2)” perfectly. Other
candidate features like “parent( Z1,Z1)” or “parent( Z2,Z3)” are poor predictors of


key: chroma:document
string_value:
“child(Z 1,Z2)” and will be excluded by the TM. Here, Z3is a free variable representing
some other constant, different from Z1andZ2.
Then the next training example comes along:
In put :parent(Bob, Mary) =1;
Targe to u t pu t :child(Jane, Bob) =0.
Again, we start with replacing the constants in the target output with variables:In put :parent(Bob, Mary) =1;
Targe to u t pu t :child(Z
1,Z 2)=0
which is then completed for the input:In put :parent(Z
2,M a r y ) =1;


key: chroma:document
string_value:
2,M a r y ) =1;
Targe to u t pu t :child(Z 1,Z 2)=0.
The constant Mary was not in the target output, so we introduce a free variable Z3for
representing Mary:
In put :parent(Z 2,Z 3)=1;
Targe t o u t pu t :child(Z 1,Z 2)=0.
The currently learnt clause was:child(Z
1,Z 2)←parent(Z 2,Z1).
The feature “parent(Z 2,Z1)” is not present in the input in the second training example,
only “parent(Z 2,Z3)”. Assuming a closed world, we thus have
parent(Z 2,Z1)=0.


key: chroma:document
string_value:
parent(Z 2,Z1)=0.
Accordingly, the learnt clause correctly outputs 0.For some inputs, there can be many different ways variables can be assigned to constants
(for the output, variables are always assigned in a fixed order, from Z
1toZz). Returning to
our grandparent example in Fig. 2,i fw eh a v e :
In put :parent(Bob,Mary) ;parent(Mary,Peter) ; parent(Bob,Jane)
Targe to u t pu t :grandparent(Bob,Peter) .
replacing Bob with Z1and Peter with Z2,w eg e t :


key: chroma:document
string_value:
replacing Bob with Z1and Peter with Z2,w eg e t :
In put :parent(Z 1,Mary) ;parent(Mary,Z 2);
parent(Z 1,Jan e)
Targe t o u t pu t :grandparent(Z 1,Z2).
Above, both Mary and Jane are candidates for being Z3. One way to handle this ambigu-
ity is to try both, and pursue those that make the clause evaluate correctly, which is exactlyhow the TM convolution operator works (Granmo et al., 2019).
Note that above, there is an implicit existential quantifier over Z
3.T h a ti s ,∀Z 1,Z2


key: chroma:document
string_value:
3.T h a ti s ,∀Z 1,Z2
(∃Z 3(parent(Z 1,Z3)∧parent(Z 3,Z2)→grandparent(Z 1,Z2)).
A practical view of how the TM learns in such a scenario is shown in Fig. 3. Continuing


key: chroma:document
string_value:
in the same vein as the previous examples, the Input Text in the figure is a set of state-ments, each followed by a question. The text is reduced to a set of relations, which act as thefeatures for the TM to learn from. The figure illustrates how the TM learns relevant infor-mation (while disregarding the irrelevant), in order to successfully answer a new question133 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Fig.3 The Relational TM in operation134 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
(Test Document). The input text is converted into a features vector which indicates the pres-
ence or absence of relations ( R1,R2) in the text, where R1andR2are respectively MoveTo
(in the statements) and WhereIs (in the question) relations. For further simplification and
compactness of representation, instead of using person and location names, those specificdetails are replaced by (P
1,P2)a n d( L1,L2), respectively. In each sample, the person name


key: chroma:document
string_value:
that occurs first is termed P1throughout, the second unique name is termed P2, and so on,
and similarly for the locations. As seen in the figure, the TM reduces the feature-set rep-resentation of the input into a set of logical conditions or clauses, all of which togetherdescribe scenarios in which the answer is L
2(or Location 2).
Remark 1 We now return to the implicit existential and universal quantifiers of the Horn


key: chroma:document
string_value:
clauses, exemplified in: ∀Z1,Z2(∃Z 3(parent (Z1,Z3)∧parent (Z3,Z2)→ grandparent
(Z1,Z2)). A main goal of the procedure in Fig. 2is to correctly deal with the quantifiers
“for all” and “exists”. “For all” maps directly to the TM architecture because the TM isoperating with conjunctive clauses and the goal is to make these evaluate to 1 (True) when-ever the learning target is 1. “For all” quantifiers are taken care of in Step 3 of the relationallearning procedure.


key: chroma:document
string_value:
Remark 2 “Exists” is more difficult because it means that we are looking for a specific value
for the variables in the scope of the quantifier that makes the expression evaluate to 1. This
is handled in the Steps 4-6 in Fig. 2, by evaluating all alternative values (all permutations


key: chroma:document
string_value:
with multiple variables involved). Some values make the expression evaluate to 0 (False)and some make the expression become 1. If none makes the expression 1, the output ofthe clause is 0. Otherwise, the output is 1. Remarkably, this is exactly the behavior of theTM convolution operator defined in Granmo et al. ( 2019), so we have an existing learning


key: chroma:document
string_value:
procedure in place to deal with the “Exists” quantifier. (If there exists a patch in the imagethat makes the clause evaluate to 1, the clause evaluates to 1).
4 QAinarelationalTMframework
In this section, we describe the general pipeline to reduce natural language text into amachine-understandable relational representation to facilitate question answering. Fig. 4


key: chroma:document
string_value:
shows the pipeline diagrammatically, with a small example. Throughout this section, wemake use of two toy examples in order to illustrate the steps. One of them is derivedfrom a standard question answering dataset (Weston et al., 2015). The other is a simple
handcrafted dataset, inspired by Kowalski ( 1979), that refers to parent-child relationships


key: chroma:document
string_value:
among a group of people. Both datasets consist of instances, where each instance is a setof two or more statements, followed by a query. The expected output for each instanceis the answer to the query based on the statements. We would like to clarify here, thatwe have chosen to highlight the details of our method with toy examples in this work,due to ease of explainability. The ease arises primarily from the fact that the relationsthat form a part of these two sets are well-understood, simple


key: chroma:document
string_value:
of these two sets are well-understood, simple to describe and have well-defined interactions between them. Further expansions on more advanced datasets, such asSQUAD/SQUADv2.0 (which involves question-answering from longer context texts), aswell as on other real-world data is currently work in progress and beyond the scope of thisparticular paper.135 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Fig.4 General Pipeline
4.1 Relationextraction
As a first step, we need to extract the relation(s) present in the text. A relation here is a


key: chroma:document
string_value:
connection between two (or more) elements of a text. As discussed before, relations occurin natural language, and reducing a text to its constituent relations makes it more structuredwhile ignoring superfluous linguistic elements, leading to easier understanding. We assumethat our text consists of simple sentences, that is, each sentence contains only one relation.The relation found in the query is either equal to, or linguistically related to the relationsfound in the statements preceding the


key: chroma:document
string_value:
Table 1shows examples of Relation Extraction on our two datasets. In Example:Movement,
each statement has the relation “MoveTo”, while the query is related to “Location”. The“Location” can be thought of as a result of the “MoveTo” relations. Example:Parentage has“Parent” relations as the information and “Grandparent” as the query.
4.2 Entityextraction


key: chroma:document
string_value:
Once the relations have been identified, the next step is to identify the elements of the text(or the entities) that are participating in the respective relations. Doing so allows us to furtherenrich the representation with the addition of restrictions (often real-world ones), whichallow the Relational TM to learn rules that best represent actions and their consequences in aconcise, logical form. Since the datasets we are using here consist only of simple sentences,each relation is limited to


key: chroma:document
string_value:
of simple sentences,each relation is limited to having a maximum of two entities (the relations are unary or binary).


key: chroma:document
string_value:
Table1 Relation extraction
Sentence Relation
Mary went to the ofﬁce. MoveTo
John moved to the hall MoveTo
Where is Mary? Location
Example:MovementSentence Relation
Bob is a parent of Mary. Parent
Bob is a parent of Jane. Parent
Mary is a parent of Peter Parent
Is Bob a grandparent of Peter? Grandparent
Example:Parentage136 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
T
able2 Entity extractionSentence Relation Entities
Mary went to the ofﬁce. MoveTo Mary, ofﬁce
John moved to the hall MoveTo John, hall
Where is Mary? Location Mary, ?
Example:Movement
Sentence Relation Entities
Bob is a parent of Mary. Parent Bob, Mary
Bob is a parent of Jane. Parent Bob, Jane
Mary is a parent of Peter Parent Mary, Peter
Is Bob a grandparent of Peter? Grandparent Bob, Peter
Example:Parentage
In this step, the more external world knowledge that can be combined with the extracted


key: chroma:document
string_value:
entities, the richer the resultant representation. In Table 2, Example:Movement, we could
add the knowledge that “MoveTo” relation always involves a “Person” and a “Location”. Or
in Example:Parentage, “Parent” is always between a “Person” and a “Person”. This could,for example, prevent questions like “Jean-Joseph Pasteur was the father of Louis Pasteur.Louis Pasteur is the father of microbiology. Who is the grandfather of microbiology?”


key: chroma:document
string_value:
Note that, as per Fig. 4, it is only possible to start answering the query after both Rela-
tion Extraction and Entity Extraction have been performed, and not before. Knowledgeof the Relation also allows us to narrow down possible entities for answering the querysuccessfully.
4.3 Entitygeneralization


key: chroma:document
string_value:
One of the drawbacks of the relational representation is that there is a huge increase in thenumber of possible relations as more and more examples are processed. One way to reducethe spread is to reduce individual entities from their specific identities to a more gener-alised identity. Let us consider two instances : “Mary went to the office. John moved to thehall. Where is Mary?” and “Sarah moved to the garage. James went to the kitchen. Whereis Sarah?”. Without generalization, we end up with


key: chroma:document
string_value:
Sarah?”. Without generalization, we end up with six different relations : MoveTo(Mary,Office), MoveTo(John, Hall), Location(Mary), MoveTo(Sarah, Garage), MoveTo(James,Kitchen), Location(Sarah). However, to answer either of the two queries, we only need therelations pertaining to the query itself. Taking advantage of that, we can generalize bothinstances to just 3 relations: MoveTo(Person


key: chroma:document
string_value:
1, Location 1), MoveTo(Person 2, Location 2)a n d
Location(Person 1).
In order to prioritise, the entities present in the query relation are the first to be general-
ized. All occurrences of those entities in the relations preceding the query are also replacedby suitable placeholders (Table 3).
The entities present in the other relations are then replaced by what can be considered as
free variables (since they do not play a role in answering the query) (Table 4).


key: chroma:document
string_value:
In the next section we explain how this relational framework is utilized for question
answering using TM.
We have not gone into the details of the relation extraction and entity extraction processes


key: chroma:document
string_value:
in this work. Depending upon the initial data, there can be different strategies for the same.In a related work using TM on natural text data, the researchers show that a TM basedclassification mechanism successfully identifies various semantic relation the text. Such asystem could be a possible way to identify and extract various relations from the text. Onthe other hand, there are also existing methods that can be used for relation identification.137 Journal of Intelligent Information Systems


key: chroma:document
string_value:
Table3 Entity generalization : Part 1
Sentence Relation Entities Reduced Relation
Mary went to the ofﬁce. MoveTo Mary, ofﬁce Source MoveTo (X,ofﬁce)
John moved to the hall. MoveTo John, hall Source MoveTo (John, hall)
Where is Mary? Location Mary, ? Target Location(X,? )
Example:Movement
Sentence Relation Entities Reduced Relation
Bob is a parent of Mary. Parent Bob, Mary Source Parent (X,Mary)
Bob is a parent of Jane. Parent Bob, Jane Source Parent (X,Jane)


key: chroma:document
string_value:
Mary is a parent of Peter Parent Mary, Peter Source Parent (Mary, Y)
Is Bob a grandparent
of Peter?Grand-parentBob, Peter Target Grandparent (X, Y )
Example:Parentage
For example, in one of our ongoing work, we use data derived from the PULSAR system,
where raw natural language text on various human rights violations from across the worldhas already been processed by experts into a database-based ecosystem (Park et al., 2020).


key: chroma:document
string_value:
The relational aspect of a database system allows for access to relations and entities thathave already been extracted with a certain level of confidence, and leaves our system tofocus on enhancing existing logical constructs or extracting newer ones based on the giveninformation. A similar approach would be to use available knowledge-bases, e.g. NELL orFreebase, to use extracted relations as an initial ’seed’ set to infer or extract newer ones.
4.4 ComputationalcomplexityinrelationTM


key: chroma:document
string_value:
4.4 ComputationalcomplexityinrelationTM
One of the primary differences between the relational framework proposed in this paperversus the existing TM framework lies in Relation Extraction and Entity Generalization.The reason for these steps is that they allow us more flexibility in terms of what the TMlearns, while keeping the general learning mechanism unchanged.
Extracting relations allows the TM to focus only on the major operations expressed via


key: chroma:document
string_value:
language, without getting caught up in multiple superfluous expressions of the same thing.It also enables to bridge the gap between structured and unstructured data. Using relationshelps the resultant clauses be closer to real-world phenomena, since they model actions andconsequences, rather than the interactions between words.
Entity Generalization allows the clauses to be succinct and precise, adding another layer


key: chroma:document
string_value:
of abstraction away from specific literals, much like Relation Extraction. It also gives the
Table4 Entity Generalization : Part 2
Sentence Relation Entities Reduced Relation
Mary went to the ofﬁce. MoveTo Mary, ofﬁce Source MoveTo (X, A )
John moved to the hall. MoveTo John, hall Source MoveTo (Y , B)
Where is Mary? Location Mary, ? Target Location(X,? )
Example:Movement
Sentence Relation Entities Reduced Relation
Bob is a parent of Mary. Parent Bob, Mary Source Parent (X, Z )


key: chroma:document
string_value:
Bob is a parent of Jane. Parent Bob, Jane Source Parent (X,W )
Mary is a parent of Peter Parent Mary, Peter Source Parent (Z, Y)
Is Bob a grandparent
of Peter?Grand-parentBob, Peter Target Grandparent (X, Y )
Example:Parentage138 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
added benefit of making the learning of the TM more generalized. In fact, due to this pro-
cess, the learning reflects ‘concepts’ gleaned from the training examples, rather than theexamples themselves.
To evaluate computational complexity, we employ the three costs α,β,a n dγ,w h e r e
in terms of computational cost, αis cost to perform the conjunction of two bits, βis cost
of computing the summation of two integers, and γis cost to update the state of a single


key: chroma:document
string_value:
automaton. In a Propositional TM, the worst case scenario would be the one in which all theclauses are updated. Therefore, a TM with mclauses and an input vector of ofeatures, needs
to perform (2o+1)×mnumber of Tsetlin automaton (TA) updates for a single training
sample. For a total of dtraining samples, the total cost of updating is d×γ×(2o+1)×m.
Once weight updates have been successfully performed, the next step is to calculate


key: chroma:document
string_value:
the clause outputs. Here the worst case is represented by all the clauses including all thecorresponding literals, giving us a cost of α×2o×m(for a single sample).
The last step involves calculating the difference in votes from the clause outputs, thus
incurring a per-sample cost of β×(m−1).
Taken together, the total cost function for a Propositional TM can be expressed as:
f( d)=d×[(γ×(2o+1)×m)+(α×2o×m)+(β×(m−1))] (5)


key: chroma:document
string_value:
f( d)=d×[(γ×(2o+1)×m)+(α×2o×m)+(β×(m−1))] (5)
Expanding this calculation to the Relation TM scenario, we need to account for the extra


key: chroma:document
string_value:
operations being performed, as detailed earlier: Relation Extraction and Entity Generaliza-tion. The number of features per sample is restricted by the number of possible relations,both in the entirety of the training data, as well as only in the context of a single sample.For example, in the experiments involving “MoveTo” relations, we have restricted our datato have 3 statements, followed by a question (elaborated further in the next section). Eachstatement gives rise to a single “MoveTo”


key: chroma:document
string_value:
Eachstatement gives rise to a single “MoveTo” relation, which has 2 entities (a location and aperson).


key: chroma:document
string_value:
When using the textual constants (i.e., without Entity Generalization), the maximum num-
ber of possible features thus becomes equal to the number of possible combination between
the unique entities in the relations. Thus if each sample contains rRelations,
 and a Rela-
tionRinvolves edifferent entities (E 1,E2,...,E e), and cardinality of sets E1,E2,...,E ebe
represented as |E1|,|E2|,...,|Ee|, the number of features in Eq. 5can be re-written as
o=[ |E1|×|E2|×...×|Ee|] ×r.


key: chroma:document
string_value:
o=[ |E1|×|E2|×...×|Ee|] ×r.
As discussed earlier in Section 4.3as well as shown in the previous paragraph, this
results in a large o, since it depends on the number of unique textual elements in each of the
entity sets. Using Entity Generalization, the number of features no longer depends on thecardinality of set E
n,1≤n≤ein the context of the whole dataset. Instead, it only depends
on the context of the single sample. Thus, if each sample contains rRelations, and a Relation


key: chroma:document
string_value:
Rinvolves edifferent entities (E 1,E2,...,E e), and maximum possible cardinality of sets
E1,E2,...,E eare|E1|=|E2|=... =|En|=r , the number of features simply become :
o=r(e+1).
In most scenarios, this number is much lower than the one obtained without the use of
Entity Generalization.
A little further modification is required when using the convolutional approach. In cal-
culating f( d) , the measure of oremains the same as we just showed with/without Entity


key: chroma:document
string_value:
Generalization. However the second term in the Eq. 5, which refers to the calculation of
clause outputs ( d×α×2o×m), changes due to the difference in the mechanism of cal-


key: chroma:document
string_value:
culating outputs for convolutional and non-convolutional approaches. In the convolutionalapproach, with Entity Generalization, we need to consider free and bound variables in thefeature representation. Bound variables are the ones which are linked by appearance in the139 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
tion, can be referred to as the free variables. Each possible permutation of the free variablesin different positions are used by the convolutional approach to determine the best genericrule that describes the scenario. In certain scenarios, it may be possible to have certain per-mutations among the bound variables as well, without violating the restrictions added bythe relations. One such scenario is detailed with an example in Section 5.2, where a bound


key: chroma:document
string_value:
“Person” entity can be permuted to allow any other “Person” entity, as long as the order of“MoveTo” relations are not violated. However, it is difficult to get a generic measure forthe same, which would work irrespective of the nature of the relation (or their restrictions).Therefore, for the purpose of this calculation, we only take into account the permutationsafforded to us by the free variable. Using the same notation as before, if each sample con-tainsrRelations, and a Relation Rinvolves


key: chroma:document
string_value:
isR×e.O ft h e s e ,i f vis the number of free variables, then they can be arranged in v!differ-
ent ways. Assuming vis constant for all samples, the worst case f( d) from Eq. 5can thus
be rewritten as:
f( d)=d×[(γ×(2o+1)×m)+(v!×α×2o×m)+(β×(m−1))] (6)
5 Experimentalstudy
To further illustrate how the TM based logic modelling works practically, we employ exam-ples from a standard question answering dataset (Weston et al., 2015), which defines a set


key: chroma:document
string_value:
of prerequisite tasks as part of QA. For the scope of this work, we limit ourselves to thefirst subtask as defined in the dataset, viz. a question that can be answered by the precedingcontext, and the context contains a single supporting fact.
To start with, there is a set of statements, followed by a question, as discussed previously.


key: chroma:document
string_value:
For this particular subtask, the answer to the question is obtained by a single statement outof the set of statements provided (hence the term, single supporting fact). Choosing sampleswhere the context contained a maximum of 6 sentences, this resulted in a set of 6000 uniquesentences.
Input:William moved to the office. Susan went to the garden. William walked to the
pantry. Where is William?
Output: pantry
We assume the following knowledge to help us construct the task:


key: chroma:document
string_value:
1. All statements only contain information pertaining to relation MoveTo2. All questions only relate to information pertaining to relation CurrentlyAt3. Relation MoveTo involves 2 entities, such that MoveTo(a,b) :a∈{Persons },
b∈{Locations }
4. Relation CurrentlyAt involves 2 entities, such that CurrentlyAt(a,b) :a∈
{Persons },b∈{Locations }
5. MoveTo is a time-bound relation, it’s effect is superseded by a similar action performed


key: chroma:document
string_value:
at a later time. In this scenario, ‘later time’ is indicated by position in text.
5.1 Withoutentitygeneralization
The size of the set of statements from which the model has to identify the correct answerinfluences the complexity of the task. For the purpose of this experiment, the data is capped140 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
to have a maximum of three statements per input, and over all has five possible locations.
This means that the task for the TM model is reduced to classifying the input into one offive possible classes.
To prepare the data for the TM, the first step involves reducing the input to relation-entity


key: chroma:document
string_value:
bindings. These bindings form the basis of our feature set, which is used to train the TM.We encode sentence order into the features as well, since MoveTo is time-sensitive (asmentioned in assumed world knowledge previously). Consider the following input example:
Input => MoveT o (S1,William ,Office), MoveT o (S2,Susan,Garden ), MoveT o(S 3,
William,Pantry) ,Q(William) .
Since the TM requires binary features, each input is converted to a vector, where each


key: chroma:document
string_value:
element represents the presence (or absence) of the relationship instances. Secondly, the listof possible answers is obtained from the data, which is the list of class labels. Continuingour example, possible answers to the question could be:
Possible Answers : [Office, Pantry, Garden, Foyer, Kitchen]
Once training is complete, we can use the inherent interpretability of the TM to get


key: chroma:document
string_value:
an idea about how the model learns to discriminate the information given to it. The setof all clauses arrived at by the TM at the end of training represents a global view of thelearning, i.e. what the model has learnt in general. The global view can also be thought ofas a description of the task itself, as understood by the machine. We also have access to alocal snapshot, which is particular to each input instance. The local snapshot involves onlythose clauses that help in arriving at the


key: chroma:document
string_value:
the TM model depends on two sets of clauses for each class, a positive set and a negative set.The positive set represents information favour of the class, while the negative set representsthe opposite. The sum of the votes given by these two sets thus represent the final class themodel decides on. As seen in the example, all the classes other than “Pantry” receive morenegative votes than positive, making it the winning class. The clauses themselves allow us


key: chroma:document
string_value:
Table 5 Local Snapshot of Clauses for example “William moved to the office. Susan went to the garden.
William walked to the pantry. Where is William?” The winning vote is indicated in bold
Class Clause +/- VotesTotal
Votes
for Class
OfﬁceQ(William) AND MoveTo(S1,William, Ofﬁce) + 12-35Q(William) AND MoveTo(S1,William, Ofﬁce)
AND MoveTo(S3,William, Pantry)- 47
PantryQ(William) AND MoveTo(S1,William, Ofﬁce)
AND MoveTo(S3,William, Pantry)+ 64+49
Q(William) AND MoveTo(S1,William, Ofﬁce) - 15


key: chroma:document
string_value:
Q(William) AND MoveTo(S1,William, Ofﬁce) - 15
GardenMoveTo(S2,Susan, Garden) + 12-36Q(William) AND MoveTo(S1,William, Ofﬁce)AND MoveTo(S3,William, Pantry)- 48
Foyer- + 0-106Q(William) AND MoveTo(S1,William, Ofﬁce)AND MoveTo(S3,William, Pantry)AND MoveTo(S2,Susan, Garden)
- 106
Kitchen- + 0-113Q(William) AND MoveTo(S1,William, Ofﬁce)AND MoveTo(S3,William, Pantry)AND MoveTo(S2,Susan, Garden)
- 113141 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
tion that (a) the question contains “William”, and (b) the relationship MoveTo(S1,William,Office) is available. This clause votes in support of the class, i.e. this is an evidence thatthe answer to the question “Where is William?” may be “Office”. However, another clauseencapsulates the previous two pieces of information, as well as something more : (c) the rela-tionship MoveTo(S3,William, Pantry) is available. Not only does this clause vote against theclass “Office”, it also ends up with a


key: chroma:document
string_value:
against theclass “Office”, it also ends up with a larger share of votes than the clause voting positively,thus pushing “Office” out of contention.


key: chroma:document
string_value:
The accuracy obtained over 100 epochs for this experiment was 94.83%, with a F-score
of 94.80.
5.1.1 Allowingnegativeliteralsinclauses


key: chroma:document
string_value:
5.1.1 Allowingnegativeliteralsinclauses
Above results were obtained by only allowing positive literals in the clauses. The descriptivepower of the TM goes up if negative literals are also added. However, the drawback to thatis, while the TM is empowered to make more precise rules (and by extension, decisions),the overall complexity of the clauses increase, making them less readable. Using the aboveexample again, if we do allow negative literals the positive evidence for Class Office lookslike:


key: chroma:document
string_value:
Q(William) AND Not(Q(Susan)) AND MoveT o(S 1,William,Office) AND
Not(MoveTo(S 3,William,Garden)) AND Not(MoveTo(S 3,William,Foyer))
AND Not(MoveT o(S 3,William,Kitchen)) AND
Not(MoveTo(S 3,Susan ,Garden ))AN DNot(MoveTo(S 3,Susan,Office))
AND Not(MoveT o(S 3,Susan,Pantry)) AND Not(MoveTo(S 3,Susan, Foyer))
AND Not(MoveT o(S 3,Susan,Kitchen)) .
At this point, we can see that the use of constants lead to a large number of repetitive


key: chroma:document
string_value:
information in terms of the rules learnt by the TM. Generalizing the constants as per theirentity type can prevent this.
5.2 Entitygeneralization


key: chroma:document
string_value:
Given a set of sentences and a following question, the first step remains the same as in theprevious subsection, i.e. reducing the input to relation-entity bindings. In the second step,we carry out a grouping by entity type, in order to generalize the information. Once theconstants have been replaced by general placeholders, we continue as previously, collectinga list of possible outputs (to be used as class labels), and further, training a TM based modelwith binary feature vectors.


key: chroma:document
string_value:
As before, the data is capped to have a maximum of three statements per input.
Continuing with the same example as above,
Input:William moved to the office. Susan went to the garden. William walked to the
pantry. Where is William?
Output: pantry
1. Reducing to relation-entity bindings:Input=> MoveT o(S 1,William,Office) ,MoveTo(S 2,Susan,Garden) ,
MoveTo(S 3,William,Pantry) ,Q(William) .
2. Generalizing bindings:Modified=> MoveT o(S 1,Per 1,L oc 1),MoveTo(S 2,Per 2,L oc 2),


key: chroma:document
string_value:
MoveTo(S 3,Per 1,L oc 3),Q(Per 1)
3. Possible Answers : [Loc1, Loc2, Loc3]142 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
T
able6 Clause snapshot for ”William moved to the office. Susan went to the garden. William walked to the
pantry. Where is William?” after generalization. The winning vote is indicated in bold
Class Clause +/- VotesTotal
Votes
for Class
Loc1Q(Per1) AND MoveTo(S1,Per1, Loc1) + 3-44Q(Per1) AND MoveTo(S1,Per1, Loc1)
AND MoveTo(S3,Per1, Loc3)- 47
Loc2MoveTo(S1,Per1, Loc1) + 2-88Q(Per1) AND MoveTo(S3,Per1, Loc3) - 90
Loc3Q(Per1) AND MoveTo(S1,Per1, Loc1)
AND MoveTo(S3,Per1, Loc3)+ 51+51
- - 0


key: chroma:document
string_value:
AND MoveTo(S3,Per1, Loc3)+ 51+51
- - 0
The simplifying effect of generalization is seen right away: even though there are 5 pos-
sible location in the whole dataset, for any particular instance there are always maximum of
three possibilities, since there are maximum three statements per instance.
As seen from the local snapshot (Table 6), the clauses formed are much more compact


key: chroma:document
string_value:
and easily understandable. The generalization also releases the TM model from the restric-tion of having had to seen definite constants before in order to make a decision. The modelcan process “Rory moved to the conservatory. Rory went to the cinema. Cecil walked to theschool. Where is Rory?”, without needing to have encountered constants “Rory”, “Cecil”,“school”, “cinema” and “conservatory”.


key: chroma:document
string_value:
Accuracy for this experiment over 100 epochs was 99.48% , with a F-score of 92.53.A logic based understanding of the relation “Move” could typically be expressed as :→ MoveTo(S 1,William,office) +MoveTo(S 2,Susan ,garden ) +
MoveTo(S 3,William,pantry)
→MoveTo(S 1,P
1,office) +MoveTo(S 2,P 2,garden ) +MoveTo(S 3,P 1,pan try)
→MoveTo(S 1,P 1,L1)+MoveTo(S 2,P 2,L2)+MoveTo(S 3,P 1,L3)
→MoveTo(S 1,P 1,L1)+∗+MoveTo(Sn,P 1,Ln)
=⇒Location(P 1,Ln).


key: chroma:document
string_value:
=⇒Location(P 1,Ln).
From the above two subsections, we can see that with more and more generalization, the
learning encapsulated in the TM model can approach what could possibly be a human-levelunderstanding of the world.
Moreover, as shown in Table 7, compared to the Propositional TM, not only does the
Relational TM give a more compact and easily understandable explanation, it also has the
Table7 Representative Winning Clauses for “William moved to the office


key: chroma:document
string_value:
Propositional TM Relational TM Relational TM with entity
generalization
Q(William) AND Q(Where) AND
S1William AND S1 Office AND
S1moved AND S3 William
AND S3 Pantry AND S3 walked
AND NOT(S3 Garden) AND
NOT(S3 Susan)Q(William) ANDMoveTo (S1,William,
Office) AND MoveTo(S3,William, Pantry)Q(Per1) AND
MoveTo(S1,Per1,
Loc1) ANDMoveTo(S3,Per1, Loc3)
Susan went to the garden. William walked to the pantry. Where is William?” with Propositional, Relational


key: chroma:document
string_value:
and Relational TM with Entity Generalization143 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
added advantage of allowing reasoning to be built on top of obtained explanations (as dis-
cussed above). How different entities interact with each other in a given environment, andwhat is the result of those interactions, is something that is very important to be able tosuccessfully model the world around us, rather than just the presence (or absence) of cer-tain words in a textual description of that environment. This can only be captured by theRelational TM, and not by the propositional one.


key: chroma:document
string_value:
5.3 Variablepermutationandconvolution
As described in Section 3.2.5, we can produce all possible permutations of the available
variables in each sample (after Entity Generalization) as long as the relation constraints arenot violated. Doing this gives us more information per sample:
Input:William moved to the office. Susan went to the garden. William walked to the
pantry. Where is William?
Output: pantry


key: chroma:document
string_value:
pantry. Where is William?
Output: pantry
1. Reducing to relation-entity bindings:Input => MoveT o(S 1,William,Office) ,MoveTo(S 2,Susan ,Garden ) ,
MoveTo(S 3,William,Pantry) ,Q(William) .
2. Generalizing bindings:Modified=> MoveT o(S 1,Per 1,L oc 1), MoveTo(S 2,Per 2,L oc 2),
MoveTo(S 3,Per 1,L oc 3),Q(Per 1)
3. Performing Permutations:Permuted=> MoveT o(S 1,Per 2,L oc 1), MoveTo(S 2,Per 1,L oc 2),
MoveTo(S 3,Per 2,L oc 3),Q(Per 2)
4. Possible Answers : [Loc1, Loc2, Loc3]


key: chroma:document
string_value:
4. Possible Answers : [Loc1, Loc2, Loc3]
This has two primary benefits. Firstly, in a scenario where the given data does not encom-


key: chroma:document
string_value:
pass all possible structural differences in which a particular information maybe represented,using the permutations allows the TM to view a closer-to-complete representation fromwhich to build it’s learning (and hence, explanations). Moreover, since the TM can learndifferent structural permutations from a single sample, it ultimately requires fewer clausesto learn effectively. In our experiments, permutations using Relational TM Convolutionallowed for up to 1.5 times less clauses than using a


key: chroma:document
string_value:
As detailed in Section 4.4, convolutional and non-convolutional approaches have differ-
ent computational complexity. Hence, the convolutional approach makes sense only whenthe reduction in complexity from fewer clauses balance the increase due to processing theconvolutional window itself.
5.4 ComparisonwithsomeotherQAapproaches


key: chroma:document
string_value:
5.4 ComparisonwithsomeotherQAapproaches
We use the following related approaches as a guide for judging the performance of theproposed method on the task (Table 8). All the approaches were trained (where applicable)
and tested on the same set as described earlier for our proposed approach, i.e. collection
Table8 Comparing accuracies with other QA approaches
Method Relational TM RTM+ Entity
Gen.Mem Net Cross- encoders BART BERT+ Bi-
Encoder


key: chroma:document
string_value:
Encoder
Accuracy 94.8 99.5 98.2 99.6 98.5 84.5144 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
of 6000 samples from the single supporting fact dataset. Implementations of the same have
been suitably modified from the ParlAI platform (Miller et al., 2017).
– MemNet (Weston et al., 2014) integrates basic inference with long-term memory
component that is used for prediction of output (or answer).
– Cross-encoders (Wolf et al., 2019) perform full (cross) self-attention over a given input
and label candidate (with pre-trained representations). This is a neural network basedmodel.


key: chroma:document
string_value:
– BART (Lewis et al., 2019) adds noise to the text and trains by learning a model to
reconstruct the original. This is a Tranformer-based neural machine translation model.
– BERT-based Bi-encoder (Lu et al., 2020) which uses two separate representations of
the document and query based on a BERT-like model. We use the pre-trained largelanguage model as available and do not train it specifically on our data.


key: chroma:document
string_value:
While detailed comparisons in terms of memory usage and training times with these modelsare beyond the scope of this work, other researchers have favourably compared the perfor-mance of Tsetlin Machine based systems to similar models (Yadav et al., 2021b; Bhattarai
et al., 2021). As an example, while the MemNet takes an average of 220 seconds to train on
the current dataset, the RTM takes 218 seconds (both averaged over 100 separate runs).
5.5 Noisetolerance


key: chroma:document
string_value:
5.5 Noisetolerance
To verify our claims of noise tolerance as shown by the TM based architecture, the aboveexperiments were repeated, but with increasing amount of noise artificially introduced intothe training data. The results are shown in Table 9. We observe that with 1%, 2%, 5% and
10% of noise, the testing accuracy fell only by approximately 1.1 percentage points eachtime when entity generalization was used.


key: chroma:document
string_value:
Along with the obvious strengths of having a transparent explainable system, the noise
tolerance is one of the more compelling pros of the TM based solution. Typical methodsare more isomorphic to the text provided, along with its imperfections, and not capable ofgeneralizing like the TM. Comparing with accuracies obtained from a few other standardclassification techniques, viz. CNN-LSTM, Naive Bayes, and Random Forests, the drop ison average 3-5%.
5.6 Hornclauserepresentation


key: chroma:document
string_value:
5.6 Hornclauserepresentation
The example elaborated in the previous section can be formulated as the following Hornclause representation:
1.Person(Susan).
2.Person(William) .
3.Location(Office) .
4.Location(Garden) .
Table9 Average accuracy on test with increase in error in training data
ErrorRate 0 %1 %2 %5 %1 0%
Accuracy 99.48 98.79 98.24 97.02 95.08145 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
5.Location(Pantry) .
6.CurrentlyAt(Susan, Pantry).
7.CurrentlyAt(William,Pantry) .
8.MoveT o(Susan,Garden) ←Person(Susan),Location(Garden), not
CurrentlyAt(Susan, Garden)
9.MoveTo(William,Office) ←Person(William), Location(Off ice),
not CurrentlyAt(William, Off ice).
After generalization, we substitute ground rules 8 and 9 with the following rule:
10.MoveTo(P,L) ←Person(P),Location(L),not CurrentlyAt(P,L).
The above set of Horn clauses define the immediate consequences operator whose LFP


key: chroma:document
string_value:
represents the Herbrand interpretation of our QA framework.
6 Conclusion
Making interpretable logical decisions in question answering system is an area of active


key: chroma:document
string_value:
research. In this work, we propose a novel relational logic based TM framework to approachQA tasks systematically. Our proposed method takes advantage of noise tolerance showedby TMs to work in uncertain or ambiguous contexts. We reduce the context-question-answertuples to a set of logical arguments, which is used by the TM to determine rules that mimicreal-world actions and consequences.
The resulting TM is relational (as opposed to the previously propositional TM) and can


key: chroma:document
string_value:
take work on logical structures that occur in natural language in order to encode rules rep-resenting actions and effects in the form of Horn clauses. We show initial results using theRelational TM on artificial datasets of closed-domain question answering, and those resultsare extremely promising. The use of first-order representations, as described in this paper,allows KBs to be up to 10 times smaller, while at the same time showing an answeringaccuracy increase of almost 5% to 99.48%.


key: chroma:document
string_value:
inter-dependencies, and analyzing how well the TM can learn the inherent logical structuregoverning such dependencies. We also intend to introduce recursive Horn clauses to makethe computing power of the Relational TM equivalent to a universal Turing machine. More-over, we wish to experiment with this framework on real-world natural language datasets,rather than on toy ones. A prominent example is exploring large corpus of documents relatedto human rights violation and using them to assess


key: chroma:document
string_value:
human rights violation and using them to assess risks of social instability. We expect thatthe resultant logic structures will be large and complicated, however, once obtained, can beused to effectively translate to and fro between the machine world and the real world.


key: chroma:document
string_value:
Data A vailability The toy tasks dataset analysed during the current study are available publicly in the
BABI repository (http://www.thespermwhale.com/jaseweston/babi/tasks 1-20 v1-2.tar.gz ). The data refer-
ring to parent-child relationships, generated for analysis during the current study are available from the
corresponding author on reasonable request.
Declarations


key: chroma:document
string_value:
Declarations
C o n f l i c to fI n t e r e s t The Authors declare that there is no conflict of interest.146 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Open
Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give


key: chroma:document
string_value:
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence,and indicate if changes were made. The images or other third party material in this article are included in thearticle’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material isnot included in the article’s Creative Commons licence and your intended use is not permitted by statutory


key: chroma:document
string_value:
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .
References
Abeyrathna, K. D., Granmo, O. C., & Goodwin, M. (2021). extending the tsetlin machine with Integer-
Weighted clauses for increased interpretability. IEEE Access,9 .
Abeyrathna, K. D., Granmo, O. C., Zhang, X., Jiao, L., & Goodwin, M. (2019). The regression tsetlin


key: chroma:document
string_value:
machine - a novel approach to interpretable Non-Linear regression. Philosophical Transactions of the
Royal Society A, 378.
Berge, G. T., Granmo, O. C., Tveit, T. O., Goodwin, M., Jiao, L., & Matheussen, B.V . (2019). Using the
Tsetlin Machine to learn human-interpretable rules for high-accuracy text categorization with medical
applications. IEEE Access, 7, 115134–115146.
Bhattarai, B., Jiao, L., & Granmo, O.C. (2021). Measuring the Novelty of Natural Language Text Using the


key: chroma:document
string_value:
Conjunctive Clauses of a Tsetlin Machine Text Classifier. In 13Th international conference on agents
and artificial intelligence (ICAART 2021) . INSTICC.
Blakely, C. D., & Granmo, O. C. (2020). Closed-Form Expressions for Global and Local Interpretation of
Tsetlin Machines with Applications to Explaining High-Dimensional Data. arXiv: 2007.13885.
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: a collaboratively created graph


key: chroma:document
string_value:
database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international
conference on Management of data (pp. 1247–1250).
Bratko, I., & Muggleton, S. (1995). Applications of inductive logic programming. Communications of the
ACM ,38(11), 65–70.
Cropper, A., Duman ˇci´c, S., & Muggleton, S.H. (2020). Turning 30: New ideas in inductive logic program-
ming. arXiv:2002.11002.


key: chroma:document
string_value:
ming. arXiv:2002.11002.
Cyras, K., Badrinath, R., Mohalik, S. K., Mujumdar, A., Nikou, A., Previti, A., Sundararajan, V ., & Feljan,
A. V . (2020). Machine reasoning explainability. arXiv: 2009.00418.
De Raedt, L., & Kersting, K. (2008). Probabilistic inductive logic programming. In Probabilistic inductive
logic programming (pp. 1–27). Springer.
Dong, J. S., Sun, J., & Wang, H. (2003). Checking and reasoning about semantic web through alloy. In


key: chroma:document
string_value:
International symposium of formal methods europe (pp. 796–813). Springer.
Gallaire, H., Minker, J., & Nicolas, J. M. (1989). Logic and databases: a deductive approach. In Readings in
artificial intelligence and databases (pp. 231–247). Elsevier.
Granmo, O. C. (2018). The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern
Recognition with Propositional Logic. arXiv: 1804.01508.


key: chroma:document
string_value:
Granmo, O. C., Glimsdal, S., Jiao, L., Goodwin, M., Omlin, C. W., & Berge, G.T. (2019). The Convolutional
Tsetlin Machine. arXiv:1905.09688.
Green, C. (1969). Theorem proving by resolution as a basis for question-answering systems. Machine
Intelligence, 4, 183–205.
Jarke, M., Gallersd ¨orfer, R., Jeusfeld, M. A., Staudt, M., & Eherer, S. (1995). Conceptbase—a deductive
object base for meta data management. Journal of Intelligent Information Systems ,4(2), 167–192.


key: chroma:document
string_value:
Jung, H., & Kim, W. (2020). Automated conversion from natural language query to sparql query. Journal of
Intelligent Information Systems , 1–20.
Kowalski, R. (1979). Algorithm= logic+ control. Communications of the ACM, 22(7), 424–436.
Kowalski, R. (2014). Logic programming. In J. H. Siekmann (Ed.) Computational logic, handbook of
the history of logic , (V ol. 9 pp. 523–569). North-holland. https://doi.org/10.1016/B978-0-444-51624-
4.50012-5 .


key: chroma:document
string_value:
4.50012-5 .
Kramer, S., Lavra ˇc, N., & Flach, P. (2001). Propositionalization approaches to relational data mining.
Relational data mining, 262–291.
Lei, J., Rahman, T., Shafik, R., Wheeldon, A., Yakovlev, A., Granmo, O. C., Kawsar, F., & Mathur, A. (2021).
Low-Power Audio Keyword Spotting using Tsetlin Machines. arXiv: 2101.11336.147 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Lei, J., Wheeldon, A., Shafik, R., Yakovlev, A., & Granmo, O.C. (2020). From arithmetic to logic based AI:
a comparative analysis of neural networks and tsetlin machine. In 27Th IEEE international conference
on electronics circuits and systems (ICECS2020). IEEE.
Lewis, M., Liu, Y ., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V ., & Zettlemoyer, L.
(2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation,


key: chroma:document
string_value:
and comprehension. arXiv:1910.13461.
Lloyd, J. (1984). F oundations of logic programming . New York: Springer.
Lu, W., Jiao, J., & Zhang, R. (2020). Twinbert: Distilling knowledge to twin-structured compressed bert mod-
els for large-scale retrieval. In Proceedings of the 29th ACM International Conference on Information &
Knowledge Management (pp. 2645–2652).
Ludwig, S. A. (2010). Comparison of a deductive database with a semantic web reasoning engine.
Knowledge-Based Systems ,23(6), 634–642.


key: chroma:document
string_value:
Knowledge-Based Systems ,23(6), 634–642.
Miller, A. H., Feng, W., Fisch, A., Lu, J., Batra, D., Bordes, A., Parikh, D., & Weston, J. (2017). Parlai: A
dialog research software platform. arXiv:1705.06476.
Mitchell, T., Cohen, W., Hruschka, E., Talukdar, P., Yang, B., Betteridge, J., Carlson, A., Dalvi, B., Gardner,
M., Kisiel, B., & et al. (2018). Never-ending learning. Communications of the ACM, 61(5), 103–115.


key: chroma:document
string_value:
Nickles, M., & Mileo, A. (2014). Probabilistic inductive logic programming based on answer set program-
ming. arXiv:1405.0720.
Park, B., Greene, K., & Colaresi, M. (2020). How to teach machines to read human rights reports and identify
judgments at scale. Journal of Human Rights ,19(1), 99–116.
Phoulady, A., Granmo, O. C., Gorji, S. R., & Phoulady, H.A. (2019). The Weighted Tsetlin Machine:
Compressed Representations with Weighted Clauses. arXiv:1911.12607.


key: chroma:document
string_value:
Prager, J. M. (2006). Open-domain question-answering. F ound Trends Inf Retr ,1(2), 91–231.
Pundge, A. M., Khillare, S., & Mahender, C. N. (2016). Question answering system, approaches and
techniques: a review. International Journal of Computer Applications ,141(3), 0975–8887.
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine
comprehension of text. arXiv: 1606.05250.


key: chroma:document
string_value:
comprehension of text. arXiv: 1606.05250.
Saha, R., Granmo, O. C., & Goodwin, M. (2020). Mining Interpretable Rules for Sentiment and Seman-
tic Relation Analysis using Tsetlin Machines. In Lecture Notes in Computer Science: Proceedings of
the 40th International Conference on Innovative T echniques and Applications of Artificial Intelligence(SGAI-2020) . Springer.
Shafik, R., Wheeldon, A., & Yakovlev, A. (2020). Explainability and Dependability Analysis of Learning


key: chroma:document
string_value:
Automata based AI Hardware. In IEEE 26Th international symposium on on-line testing and robust
system design (IOLTS). IEEE.
Soares, M. A. C., & Parreiras, F. S. (2020). A literature review on question answering techniques, paradigms
and systems. Journal of King Saud University-Computer and Information Sciences ,32(6), 635–646.
Srinivasan, V ., Santhanam, S., & Shaikh, S. (2021). Using reinforcement learning with external rewards for


key: chroma:document
string_value:
open-domain natural language generation. Journal of Intelligent Information Systems ,56(1), 189–206.
Tsetlin, M. L. (1961). On behaviour of finite automata in random medium. Avtom I T elemekhanika ,22(10),
1345–1354.
Turhan, A. Y . (2011). Description logic reasoning for semantic web ontologies. In Proceedings of the
International Conference on W eb Intelligence, Mining and Semantics (pp. 1–5).


key: chroma:document
string_value:
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merri ¨enboer, B., Joulin, A., & Mikolov, T. (2015).
Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv: 1502.05698.
Weston, J., Chopra, S., & Bordes, A. (2014). Memory networks. arXiv:1410.3916 .
Wheeldon, 
A., Shafik, R., Yakovlev, A., Edwards, J., Haddadi, I., & Granmo, O.C. (2020). Tsetlin machine:
a new paradigm for pervasive AI. In SCONA W orkshop at design, automation and test in europe (DATE
2020).


key: chroma:document
string_value:
2020).
Wolf, T., Sanh, V ., Chaumond, J., & Delangue, C. (2019). Transfertransfo: A transfer learning approach for
neural network based conversational agents. arXiv:1901.08149.
Yadav, R. K., Jiao, L., Granmo, O. C., & Goodwin, M. (2021). Human-Level interpretable learning for
Aspect-Based sentiment analysis. In The thirty-fifth AAAI conference on artificial intelligence (AAAI-
21). AAAI.
Yadav, R. K., Jiao, L., Granmo, O. C., & Goodwin, M. (2021). Interpretable classification of word sense dis-


key: chroma:document
string_value:
ambiguation using tsetlin machine. In 13Th international conference on agents and artificial intelligence
(ICAART 2021). INSTICC.
Publisher’snote Springer Nature remains neutral with regard to jurisdictional claims in published maps
and institutional affiliations.148 Journal of Intelligent Information Systems (2022) 59:121–148


key: chroma:document
string_value:
Coalesced Multi-Output Tsetlin Machines with Clause Sharing∗
Sondre Glimsdal†and Ole-Christoﬀer Granmo‡
August 18, 2021
Abstract
Using ﬁnite-state machines to learn patterns, Tsetlin machines (TMs) have obtained
competitive accuracy and learning speed across several benchmarks, with frugal memory-
and energy footprint. A TM represents patterns as conjunctive clauses in propositional
logic (AND-rules), each clause voting for or against a particular output. While eﬃcient for


key: chroma:document
string_value:
single-output problems, one needs a separate TM per output for multi-output problems.
Employing multiple TMs hinders pattern reuse because each TM then operates in a silo.
In this paper, we introduce clause sharing, merging multiple TMs into a single one. Each
clause is related to each output by using a weight. A positive weight makes the clause vote
for output 1, while a negative weight makes the clause vote for output 0. The clauses thus


key: chroma:document
string_value:
coalesce to produce multiple outputs. The resulting coalesced Tsetlin machine (CoTM)
simultaneously learns both the weights and the composition of each clause by employing
interacting Stochastic Searching on the Line (SSL) and Tsetlin automata (TAs) teams.
Our empirical results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST show that CoTM
obtains signiﬁcantly higher accuracy than TM on 50- to 1K-clause conﬁgurations, indicating


key: chroma:document
string_value:
an ability to repurpose clauses. E.g., accuracy goes from 71 .99% to 89.66% on Fashion-
MNIST when employing 50 clauses per class (22 Kb memory). While TM and CoTM
accuracy is similar when using more than 1K clauses per class, CoTM reaches peak accuracy
3ˆfaster on MNIST with 8K clauses. We further investigate robustness towards imbalanced
training data. Our evaluations on imbalanced versions of IMDb- and CIFAR10 data show


key: chroma:document
string_value:
that CoTM is robust towards high degrees of class imbalance. Being able to share clauses,
we believe CoTM will enable new TM application domains that involve multiple outputs,
such as learning language models and auto-encoding.
1 Introduction
TMs [19] have recently demonstrated competitive accuracy-, memory footprint-, energy-, and
learning speed on several benchmarks, spanning tabular data [2, 40], images [20, 37], regres-


key: chroma:document
string_value:
sion [4], natural language [6, 45, 46, 9, 44], and speech [27]. By not relying on minimizing
output error, TMs are less prone to overﬁtting. Instead, they use frequent pattern mining
and resource allocation principles to extract common patterns in the data. Unlike the inter-
twined nature of pattern representation in neural networks, a TM decomposes problems into
self-contained patterns. These are expressed using conjunctive clauses in propositional logic.


key: chroma:document
string_value:
That is, each pattern is an AND-rule, such as: ifinput x satisﬁes conditionAand not
conditionBthen outputy“1. The clause outputs, in turn, are combined into a classiﬁcation
decision through a majority vote, akin to logistic regression, however, with binary weights and
a unit step output function. Being based on the human-interpretable disjunctive normal form
[39], like Karnaugh maps [23], a TM can map an exponential number of input feature value
combinations to an appropriate output [19].


key: chroma:document
string_value:
combinations to an appropriate output [19].
∗Source code for this paper can be found at https://github.com/cair/PyCoalescedTsetlinMachineCUDA.
†Author’s status: Lead Researcher at Forsta. E-mail: sondre.glimsdal@forsta.com
‡Author’s status: Professor . The author can be contacted at: Centre for Artiﬁcial Intelligence Research
(https://cair.uia.no), University of Agder, Grimstad, Norway. E-mail: ole.granmo@uia.no
1arXiv:2108.07594v1  [cs.AI]  17 Aug 2021


key: chroma:document
string_value:
………+-+-
∑⌵⌵⌵⌵0 1 0 … 1 0 1∑
Output
InputThreshold FunctionSummationConjunctiveClausesFigure 1: The Tsetlin Machine inference structure, introducing clause polarity, a summation
operator collecting “votes”, and a threshold function arbitrating the ﬁnal output [19].
Recent progress on TMs. Recent research reports several distinct TM properties. The
TM can be used in convolution [20], providing competitive performance on MNIST, Fashion-


key: chroma:document
string_value:
MNIST, and Kuzushiji-MNIST, in comparison with CNNs, K-Nearest Neighbor, Support Vec-
tor Machines, Random Forests [11], XGBoost [13], BinaryConnect [15], Logistic Circuits [28]
and ResNet [21]. The TM has also achieved promising results in text classiﬁcation [6, 44], word
sense disambiguation [46], novelty detection [9, 8], fake news detection [7], semantic relation
analysis [35], and aspect-based sentiment analysis [45] using the conjunctive clauses to capture


key: chroma:document
string_value:
textual patterns. Recently, regression TMs [4] compared favorably with Regression Trees [12],
Random Forest Regression [11], and Support Vector Regression [16].
The above TM approaches have further been enhanced by various techniques. By introduc-
ing real-valued clause weights, it turns out that the number of clauses can be reduced by up to
50ˆwithout loss of accuracy [34]. Also, the logical inference structure of TMs makes it possible


key: chroma:document
string_value:
to index the clauses on the features that falsify them, increasing inference- and learning speed
by up to an order of magnitude [17]. Multi-granular clauses simplify the hyper-parameter search
by eliminating the pattern speciﬁcity parameter [18]. In [2], SSL automata [33] learn integer
clause weights, performing on-par or better than Random Forests [11], XGBoost [13], Neural
Additive Models [5], StructureBoost [29], and Explainable Boosting Machines [32]. Closed


key: chroma:document
string_value:
form formulas for both local and global TM interpretation, akin to SHAP [30], were proposed
by Blakely et al. [10].
Computationally, TMs are natively parallel [1] and hardware near [41, 40, 42], allowing
energy usage to be traded oﬀ against accuracy by making inference deterministic [3]. Ad-
ditionally, Shaﬁk et al. show that TMs can be fault-tolerant, completely masking stuck-at
faults [36].
Recent theoretical work proves convergence to the correct operator for “identity” and “not”.


key: chroma:document
string_value:
It is further shown that arbitrarily rare patterns can be recognized using a quasi-stationary
Markov chain-based analysis. The work ﬁnally proves that when two patterns are incompatible,
the most accurate pattern is selected [47]. Convergence for the “XOR” operator has also
recently been proven by Jiao et al. [22].
Paper Contributions. Figure 1 depicts the original multi-output TM architecture from [19].


key: chroma:document
string_value:
A TM represents patterns as a collection of conjunctive clauses in propositional logic. Each
clause vote for or against a particular output, arbitrated by a majority vote. As further depicted
in the ﬁgure, there is one separate TM per output, each maintaining its own set of clauses.
This hinders reuse of patterns among the outputs, because each TM operates in a silo.
In this paper, we introduce a new TM architecture that employs a shared pool of clauses,


key: chroma:document
string_value:
illustrated in Figure 2. Each clause in the pool is related to each output by using a weight.
A positive weight makes the clause vote for output 1, while a negative weight makes it vote
2


key: chroma:document
string_value:
……+-
∑⌵⌵0 1 0 … 1 0 1∑
Output
InputThreshold FunctionSummationConjunctiveClauses+-Figure 2: The Coalesced Tsetlin Machine (CoTM) inference structure, introducing clause shar-
ing.
for output 0. The magnitude of the weight decides the impact of the vote. The clauses thus
coalesce to produce multiple outputs. The resulting CoTM simultaneously learns both the
weights and the composition of each clause. The weights are learnt using one SSL automaton


key: chroma:document
string_value:
per clause-output pair. Simultaneously, a team of TAs conﬁgures each clause to maximise
output prediction accuracy.
Paper Organization. In Section 2, we introduce the basics of TM inference and learning,
providing the basis for CoTM. Then, in Section 3, we describe the CoTM in detail, including
the architecture for coalescing clauses and learning weights. Our empirical results are presented
in Section 4, where we evaluate the CoTM on 2D Noisy XOR, MNIST, Fashion-MNIST, K-


key: chroma:document
string_value:
MNIST, CIFAR10, and IMDb comparing with various baselines. We conclude the paper in
Section 5 and discuss ideas for further work.
2 Tsetlin Machine Basics
In this section, we provide an introduction to the principles of TM inference and learning,
which we formalize in Section 3 when presenting the CoTM.
2.1 Inference
Figure 1 depicts a multi-output TM architecture. As seen, the architecture uses a simple
pattern matching scheme to decide upon the output:


key: chroma:document
string_value:
•One TM is associated with each output, assembling a set of patterns. Each pattern, in
turn, is an AND-rule, called a conjunctive clause .
•The AND-operator ( ^in the ﬁgure) binds together propositional inputs. These are either
False or True (0 or 1 in the ﬁgure).
•For each TM, half of the clauses can vote for output value True. We refer to these as
positive clauses. The other half can vote for output value False. We call these negative
clauses.


key: chroma:document
string_value:
clauses.
•To decide upon a particular output, we count the positive clauses that match the input.
Matching negative clauses, on the other hand, decrease the match count.
•A majority vote decides the ﬁnal output, indicated by the threshold functions in the
ﬁgure. If there are more matching negative clauses than positive clauses, the output is
False. Otherwise the output is True.
3


key: chroma:document
string_value:
As an example, consider the task of determining the sentiment of a collection of movie
reviews. To this end, we employ a Set of Words (SoW) representation. That is, each word
in the vocabulary is a propositional input, representing the presence or absence of the word
in a particular review. Consider, for instance, the review “The movie was good, and I had
popcorn” . Here, the words “the”, “movie”, “was”, “good”, “and”, “I”, “had”, “popcorn” are


key: chroma:document
string_value:
present. These thus take the input value True. Any other words are absent, taking the input
value False.
Based on the SoW, the task is to decide upon the nature of each review, whether it is
Positive orNegative . The task thus has two outputs and we consider each output to either be
False or True.1
Clause TM: Positive TM: Negative
#1 ”good” AND ”movie” Ñ+1 ”bad”Ñ+1
#2 NOT ”bad”Ñ+1 NOT ”good” Ñ+1
#3 ”horrible” AND ”popcorn” Ñ-1 ”good”Ñ-1
#4 ”bad”Ñ-1 ”high value”Ñ-1


key: chroma:document
string_value:
#4 ”bad”Ñ-1 ”high value”Ñ-1
Table 1: Eight example clauses for classifying IMDb movie reviews.
Table 1 contains two example TMs for the above task, one for output Positive and one for
output Negative . The ﬁrst column speciﬁes the clause number; the second column contains
the clauses for the TM assigned to output Positive ; and the third column covers the Negative
output TM. Clause #1 and Clause #2 can vote for output True (’ `1’-votes), while Clause


key: chroma:document
string_value:
#3 and Clause #4 can vote for output False (’ ´1’-votes). For instance, Clause #1 for TM
Positive says that if the input review contains the words “good” and “movie”, the clause votes
for output True. Notice that Clause #2 consists of a negated input. That is, if the input review
does not contain the word “bad”, this also counts as a vote towards Positive output True. Such
negation is useful because one can strengthen the belief in Positive from the absence of negative
sentiment words.


key: chroma:document
string_value:
sentiment words.
Inference proceeds as follows. Again, consider the example review: “The movie was good,
and I had popcorn” . To classify this review, we ﬁrst identify clauses that are matching the
text. For TM Positive , Clause #1 matches because both ”good” and ”movie” are in the text.
Clause #2 also matches the text, since ”bad” is absent. Therefore, the score for TM Positive
becomes Clause #1 + Clause #2 “1`1“2. For TM Negative only Clause #3 is active,


key: chroma:document
string_value:
providing a score of ´1. From a propositional perspective, if a TM obtains a non-negative
score it outputs True. Otherwise, it outputs False. Accordingly, the output of TM Positive
becomes True, while the output of TM Negative becomes False.
Maximally
Memorized
Maximally
ForgottenMemorized
ForgottenN+12N
1N
Figure 3: Sticky memory with depth Nfor the clause: “good” AND “movie”.
1Note that for this particular task, a movie review cannot be both Positive andNegative at the same time.


key: chroma:document
string_value:
Accordingly, we could have modelled the problem as a multi-class problem instead of a multi-output problem [19].
4


key: chroma:document
string_value:
2.2 Learning
Sticky Memory. Memorizing input is the basis of TM learning, such as remembering the
essence of: “The movie was good, and I had popcorn” . To this end, each clause has its own
memory where it stores its AND-pattern. In the IMDb case, the memory contains the truth
value of words from the SoW. TM Positive Clause #1 stores True for “movie” and “good”,
for instance. However, instead of simply storing the truth values as is, the memory simulates


key: chroma:document
string_value:
memorization and forgetting. I.e., every time a clause observes a particular truth value, it
remembers it longer. Oppositely, without observations, it eventually forgets the value. Ac-
cordingly, each memory entry does not contain a truth value but an integer in the range from 1
to 2N. The resulting memory is illustrated in Figure 3. Note that the user sets Nto a speciﬁc
value to control maximal memorization and forgetting. From N`1 to 2N, the word’s truth


key: chroma:document
string_value:
value is part of the clause’s AND-pattern. Integer N`1 means in memory but easily forgotten.
Integer 2Nmeans maximally memorized. A truth value is not part of the pattern in the span
1 toN. Integer 1 means maximally forgotten, while integer Nmeans almost memorized. The
clause in the ﬁgure will, for instance, remember “good” for a longer time than “movie” because
“good” is more deeply stored. It is close to memorizing “had”, which is close to N`1.2
Maximally
Memorized
Maximally


key: chroma:document
string_value:
Maximally
Memorized
Maximally
ForgottenMemorized
ForgottenN+12N
1N
Figure 4: The memorize (x,s) operator triggered by input “The movie was good, and I had
popcorn.”
Maximally
Memorized
Maximally
ForgottenMemorized
ForgottenN+12N
1N
Figure 5: The forget (s) operator triggered by input “Was good, and I had popcorn.”
Memory Updating. Now, let xbe one particular input to the TM. We then have three
operators for updating the memory of each clause.


key: chroma:document
string_value:
operators for updating the memory of each clause.
1.memorize(x ,s)strengthens the memory of x. However, we do not perform a plain
copy. Instead, each truth value in xincreases the corresponding integer in the memory.
2Note that each memory entry can be seen as a Tsetlin Automaton [38], hence the name Tsetlin Machine.
5


key: chroma:document
string_value:
Maximally
Memorized
Maximally
ForgottenMemorized
ForgottenN+12N
1NFigure 6: The invalidate (x) operator applied to input “The movie was bad, and I had pop-
corn.”
As exempliﬁed in Figure 4, “good” in the input increases the integer of “good”. Further,
“bad” is missing in the input, which increases the integer of not“bad”. Simultaneously,
the truth values that conﬂict with the input have their integers decreased. However,
this decrease is randomized, happening with probability1


key: chroma:document
string_value:
s. For instance, not“good” is
conﬂicting with the input. Hence, its integer is decreased randomly with probability1
s.
The intuition is that memorization must be stronger than forgetting so that patterns can
be retained over time. The parameter sis set by the user so that the user can control
how quickly truth values are forgotten. In eﬀect, increasing smakes the patterns ﬁner,
while decreasing smakes them coarser.


key: chroma:document
string_value:
while decreasing smakes them coarser.
2.forget(s)performs pure forgetting, as described above. Figure 5 provides an example.
As seen, all the truth values have their integer randomly decreased, with probability1
s.
Performing this operator several times eventually makes all the truth values maximally
forgotten.
3.invalidate(x) changes the clause so that it eventually rejects the input x. This operator
increases the integer of all False truth values, illustrated in Figure 6 for input “The movie


key: chroma:document
string_value:
was bad, and I had popcorn.” The word “bad”, for instance, appears in the input. Hence,
not“bad” gets its integer increased.
Learning Single Clauses. Four diﬀerent prediction outcomes guide the learning of each
TM clause:
•True Positive. The clause correctly votes for its assigned output. For instance, we have
a True Positive outcome when Clause #1 for TM Positive matches a review of positive
sentiment. In this case, the clause performs memorize( x,s). This feedback makes the


key: chroma:document
string_value:
clause remember and reﬁne the pattern it recognizes in x.
•False Negative. The clause fails to vote for its assigned output. If Clause #1 for TM
Positive does not match a review of positive sentiment, that would be a False Nega-
tive outcome. In this case, the clause performs forget( s). This reinforcement coarsens
infrequent patterns, making them frequent.
•False Positive. The clause incorrectly votes for its assigned output. If Clause #1 for


key: chroma:document
string_value:
TMPositive matches a review of negative sentiment, we have a False Positive. The clause
then performs invalidate( x). This feedback makes the clause more discriminative.
•True Negative . The clause correctly refrains from voting for its assigned output. That
would be the case if Clause #1 for TM Positive does not match a review of negative
sentiment. This outcome does not trigger any memory updates.
6


key: chroma:document
string_value:
Learning Multiple Clauses. The clauses must learn to coordinate, taking diﬀerent roles
in providing correct output. The TM achieves this by introducing a voting margin that we
callt. We use this parameter to specify how many votes we want the winning output to win
by. That is, we want the winning output to win by tvotes, but not more. The voting margin
makes sure that a suﬃcient number of clauses support each output. Simultaneously, we do not


key: chroma:document
string_value:
want the winning output to win by much more than the margin either. That would mean that
we use more clauses than necessary. So, the voting margin also ensures prudent usage of the
available clauses. Learning of multiple clauses is coordinated as follows:
1. Obtain next training example. The training example consists of the input truth values x
as well the correct output truth value y.
2. Evaluate each clause on input x.


key: chroma:document
string_value:
2. Evaluate each clause on input x.
3. Calculate a voting sum for the clauses that evaluate to True (clauses matching x):
(a) Add up the votes in favour of y, i.e., the correct truth value for the output.
(b) Subtract the votes in favour of noty, i.e., the incorrect truth value for the output.
(c) We call the summation outcome v.
(d) Setvtotif larger than tand to´tif smaller than ´t.
4. Go through each clause and update it if Randpqďt´v
2t, where Randpqdraws a random


key: chroma:document
string_value:
2t, where Randpqdraws a random
value uniformly from the interval r0,1s:
(a) Perform memorize (x,s) if the clause matches xand belongs to output truth value y.
(True Positive)
(b) Perform forget (s) if the clause does not match xand belongs to output truth
valuey. (False Negative)
(c) Perform invalidate (x) if the clause matches xand belongs to output truth value noty.
(False Positive)
Observe that if we are far from achieving the voting margin for a particular example, we update


key: chroma:document
string_value:
clauses more aggressively. E.g., if the voting sum is ´tor smaller, we update all the clauses. If
the voting sum is zero, we randomly update each clause with probability 0 .5. If we are close to
t, we calm down the updating. When reaching or surpassing t, we update none of the clauses.
In this way, clauses individually and gradually assign themselves to training examples that
have not yet reached the voting margin. As a result, inference accuracy increases over time as


key: chroma:document
string_value:
the clauses specialize on diﬀerent subsets of the training examples.
3 Coalesced Tsetlin Machine
We here introduce the CoTM, formulated in terms of propositional- and linear algebra. While
we cover all the key structures and formulas here, the full details are found in Appendix A.
3.1 Structure
A CoTM can be deﬁned as a quadruple:
tX,Y,C,Wu. (1)
We start with introducing each element of the quadruple, with reference to Figure 7.


key: chroma:document
string_value:
•Theinput space of the CoTM is denoted X. The input space consists of vectors xof
opropositional inputs: x“ rx1,...,x os PX,X“ t0,1uo. In the ﬁgure, we have two
propositional inputs x“rx1,x2sPX,X“t0,1u2.
7


key: chroma:document
string_value:
Summation
ClausesClause W eightingMargin
t=5 -3 +4Thresholding  
 Output
Input8 2 1 7
2 7 8 1
7 1 7 2
1 7 2 8Clause 1
Clause 2
Clause 3
Clause 4Clause Memory
N=4Action Mapping+4
Clause 1
Clause 2
Clause 3
Clause 44 -3 4
3 -5 5
-4 5 4
-3 -4 -4Figure 7: The Coalesced Tsetlin Machine structure with inference example.
•Theoutput space is denoted Y. The output space contains vectors yofmpropositional
outputs: y“ ry1,y2,...,yms PY,Y“ t0,1um. In the ﬁgure, we have three outputs
y“ry1,y2,y3sPY,Y“t0,1u3.


key: chroma:document
string_value:
y“ry1,y2,y3sPY,Y“t0,1u3.
•Thememory space C“ t1,2,..., 2Nunˆ2ois the space of memory matrices C. A
memory matrix Cformalizes the sticky memory from Section 2. Each of the nrows
represents the pattern memory of a single clause. The columns, in turn, represent the o
inputs in xand their negations. Together, the inputs and their negations are referred to
as literals. In the ﬁgure, we have four literals: x1,␣x1,x2, and␣x2. With four clauses,


key: chroma:document
string_value:
four literals, and memory depth N“4, we get the memory space C“t1,2,..., 8u4ˆ4.
•Theweight space W“ t...,´2,´1,1,2,...umˆnconsists of weight matrices W. A
weight matrix Wrelates each clause in Cto an output in y. Consider the output and
clause of a speciﬁc entry in the weight matrix. A positive weight assigns the clause to
output value 1. A negative weight assigns it to output value 0. The magnitude of the
weight decides the impact of the assignment. With the three outputs and four clauses


key: chroma:document
string_value:
in the ﬁgure, we get the weight space WPW“ t...,´2,´1,1,2,...u3ˆ4. As seen,
Clause #1 is assigned output values y1“1,y2“0, andy3“1.
Observe how the pattern matrix Ccombined with the weight matrix Wconﬁgure a complete
CoTM.
8


key: chroma:document
string_value:
3.2 Output Prediction
Based on the above quadruple, we can predict the output yfrom input xusing propositional-
and linear algebra as shown in Eqn. 2:
ˆy“UpW¨AndpImplypGpCq,xqqqT. (2)
Above, ˆyis the multi-output prediction. As speciﬁed, it is calculated from the input xusing
the following operators:
•Gp¨qmaps the memory matrix Celement-wise to Exclude and Include actions, respec-
tively 0 and 1. Each entry is mapped using the function:
gpcq“#
0,if1ďcďN
1,ifN`1ďcď2N.(3)


key: chroma:document
string_value:
gpcq“#
0,if1ďcďN
1,ifN`1ďcď2N.(3)
•Imply is an element-wise logical imply operator (ñ), which implements the Exclude and
Include actions. It takes a matrix of Exclude/Include actions raj,ksas input, with j
referring to a clause and kreferring to a literal. If aj,kis 0 (Exclude), then aj,kñxkis
always 1. Accordingly, the xkvalue does not impact the result. Conversely, the value of
aj,kñxkis decided solely by the xkvalue ifajkis 1 (Include).


key: chroma:document
string_value:
•And is a row-wise AND operator. The operator takes a matrix of truth values as input.
For each row, it ANDs together all the truth values of that row.
•Uis an element-wise unit step thresholding operator. Utakes a vector as input and
applies the unit step function vě0 on each vector entry.
These operators are fully speciﬁed in Appendix A, while Figure 7 provides a prediction example.
In the ﬁgure, we trace the input x“r0,1sthrough each operator. The clause memory in


key: chroma:document
string_value:
the ﬁgure is of depth N“4. So, for example, Clause #1 includes x1and␣x2because their
respective memory matrix entries are 8 and 7 (larger than or equal to 5). It excludes ␣x1and
x2, having memory entries 2 and 1 (equal to or smaller than 4). Notice how the memory and
weight matrices implement XOR for y1, AND fory2, and OR for y3. Hence, input x“r0,1s
provides output y“r1,0,1s.
3.3 Updating of Memory Matrix


key: chroma:document
string_value:
3.3 Updating of Memory Matrix
The memory matrix is updated based on a training example px,yqusing three kinds of feedback
matrices:FIa
i,FIb
i, andFII
i. Further, which clauses are eligible for feedback are compiled in
the matrices RIandRII. The clauses are randomly selected according to the voting margin
of each output values yiinyas described in Section 2. In brief, the memory matrix CT`1for
time stepT`1 is calculated from the memory matrix CTof time step T:
C˚
T`1“CT`mÿ
i“1´
QII
i˝FII
i`QI


key: chroma:document
string_value:
C˚
T`1“CT`mÿ
i“1´
QII
i˝FII
i`QI
i˝FIa
i´QI
i˝FIb
i¯
. (4)
As seen, each output yiof output vector yis considered one at a time. Both FIa
iandFII
i
increments memory entries, while FIb
idecrements them. The matrix QI
imaps thei-rows of
matrixRIfrom the clause level to the literal level. Similarly, QII
imaps thei-rows of matrix
RII. These single out which clauses to update using the Hadamard product ˝. The ﬁnal step
is to clip the entries in Cto make sure that they stay within 1 and 2 N:


key: chroma:document
string_value:
•Type Ia Feedback. This feedback matrix is denoted FIa
iand concerns output yiof
output vector y. It operates on clauses that both match the input xand that are assigned
to output value yi(by a negative clause-output weight for yi“0 and a positive clause-
output weight for yi“1).FIa
ionly aﬀects literals that are True and tunes the clauses
to represent the current input xmore ﬁnely, reinforcing Include actions.
•Type Ib Feedback. This feedback matrix is denoted FIb
iand again concerns output yi.


key: chroma:document
string_value:
iand again concerns output yi.
It operates on all the clauses that are assigned to output value yiby the clause-output
weights. For those clauses that does not match the input x, it operates on all of the
literals. For clauses that match the input, it only aﬀects False literals. In eﬀect, the
matrix coarsens the clauses by making them forget literals, reinforcing Exclude actions.
•Type II Feedback. This feedback matrix is denoted FII
iand concerns output yi. It


key: chroma:document
string_value:
iand concerns output yi. It
operates on clauses that both match the input xand that are assigned to the negated
output value, notyi. Only literals that are False are aﬀected. This matrix increases the
discrimination power of the matching clauses by introducing literals that invalidate the
matching.
Together,FIa
iandFIb
iimplement forget andmemorize from Section 2. FII
iimplements
invalidate . See Appendix A for a formal deﬁnition of the feedback matrices.
+1 +2 ...True Positive Clause Output


key: chroma:document
string_value:
+1 +2 ...True Positive Clause Output
False Positive Clause Output-2 -1 ...
Figure 8: Type Ia Feedback changes the weights in one-step increments away from 0. Type II
Feedback changes the weights in one-step decrements towards 0.
3.4 Updating of Weight Matrix
As shown in Figure 8, the updating of weights is quite straightforward. Weights are only
updated in the case of Type Ia or Type II feedback. Type Ia Feedback changes the weights in


key: chroma:document
string_value:
one-step increments away from 0. Type II Feedback changes the weights in one-step decrements
towards 0. Type Ib Feedback leaves the weights unchanged. The following equation captures
the updating of the weight matrix from time step TtoT`1:
WT`1“WT`pRI`RIIq˝¨
˚˚˚˝»
———–c1y1c2y1¨¨¨cny1
c1y2c2y2¨¨¨cny2
............
c1ymc2ym¨¨¨cnymﬁ
ﬃﬃﬃﬂ´»
———–c1¯y1c2¯y1¨¨¨cn¯y1
c1¯y2c2¯y2¨¨¨cn¯y2
............
c1¯ymc2¯ym¨¨¨cn¯ymﬁ
ﬃﬃﬃﬂ˛
‹‹‹‚.(6)


key: chroma:document
string_value:
............
c1¯ymc2¯ym¨¨¨cn¯ymﬁ
ﬃﬃﬃﬂ˛
‹‹‹‚.(6)
The matrices RIandRIIare non-overlapping, singling out which clause rows jare eligible for
Type I and Type II Feedback. For this purpose, we again use the Hadamard product ˝. Let
cj,jP t1,...,nurefer to the output of the clause in row jinC. Together with the output
valueyithe clause output value controls the direction of the weight update.
4 Empirical Results
In this section, we evaluate the CoTM on six diﬀerent datasets.
10


key: chroma:document
string_value:
2D Noisy XOR. The 2D Noisy XOR dataset contains 4 ˆ4 binary images, 2500 training
examples and 10 000 test examples. The image bits have been set randomly, except for the
2ˆ2 patch in the upper right corner, which reveals the class of the image. A diagonal line
is associated with class 1, while a horizontal or vertical line is associated with class 0. Thus
the dataset models a 2D version of the XOR-relation. Furthermore, the dataset contains a


key: chroma:document
string_value:
large number of random non-informative features to measure susceptibility towards the curse
of dimensionality. To examine robustness towards noise we have further randomly inverted
40% of the outputs in the training data.
IMDB. The IMDb dataset contains 50 000 highly polar movie reviews for binary sentiment
classiﬁcation [31].
CIFAR10. The CIFAR-10 dataset consists of 60 000 32 ˆ32 colour images. There are 10
classes with 6 000 images per class [24].


key: chroma:document
string_value:
classes with 6 000 images per class [24].
MNIST. The MNIST dataset has been used extensively to benchmark machine learning
algorithms, consisting of 28 ˆ28 grey scale images of handwritten digits [26].
Kuzushiji-MNIST. This dataset contains 28 ˆ28 grayscale images of Kuzushiji charac-
ters, cursive Japanese. Kuzushiji-MNIST is more challenging than MNIST because there are
multiple distinct ways to write some of the characters [14].


key: chroma:document
string_value:
Fashion-MNIST. This dataset contains 28 ˆ28 grayscale images of articles from the Za-
lando catalogue, such as t-shirts, sandals, and pullovers [43]. This dataset is quite challenging,
with a human accuracy of 83 .50%.
The latter three datasets contain 60 000 training examples and 10 000 test examples. We
binarize these datasets using an adaptive Gaussian thresholding procedure with window size
11 and threshold value 2. Accordingly, the CoTM operates on images with 1 bit per pixel. All


key: chroma:document
string_value:
experiments are repeated 10 times and we report average results for the last 25 epochs, unless
otherwise noted. We ran the CoTM on a NDVIDIA Tesla V100 GPU, calculating the clause
outputs and updates in parallel.
#Clauses per class 50 100 250 500 1K 2K 4K 8K
Voting Margin 625 1250 3125 6.25K 12.5K 25K 5K 10K
Speciﬁcity 10.0 10.0 10.0 10.0 10.0 10.0 5.0 5.0
Weighted, Ac. (%) 97.86 98.43 98.82 98.98 99.14 99.22 99.28 99.33
Coalesced, Ac. (%) 98.33 98.80 99.03 99.14 99.18 99.21 99.26 99.29


key: chroma:document
string_value:
Training Time (s) 21.6 21.0 21.1 21.3 20.9 24.1 42.6 86.7
Testing Time (s) 1.7 1.6 1.6 1.6 1.5 1.6 4.0 6.5
Model Size (kB) 22 44 110 220 439 879 1758 3516
Table 2: Weighted Convolutional TM and Convolutional CoTM MNIST mean test accuracy
and execution time per epoch, for an increasing number of clauses.
MNIST Results. We consider scalability ﬁrst, investigating how the number of clauses
aﬀects performance. Table 2 reports performance on unaugmented MNIST, for various number


key: chroma:document
string_value:
of clauses per class. We see that CoTM outperforms TM accuracy-wise for 50 to 1000 clauses.
Indeed, CoTM operates at a similar level as TM with half the number of clauses, indicating
an ability to repurpose clauses towards multiple classes. For 2000 to 8000 clauses, CoTM
performs competitively. As further seen in the table, going from 50 clauses to 8000 clauses
(aˆ160 increase) increases testing time 3 .8 times. Similarly, training time increases 4 times.


key: chroma:document
string_value:
Finally, model size increases proportionally with the number of clauses. Employing 50 clauses
per class gives a model of size 22 kB, providing a test accuracy of 98 .26. As the number of
clauses increases, so does test accuracy, with 8000 clauses per class giving a test accuracy of
99.29.
Fashion-MNIST and Kuzushiji-MNIST Results. We observe similar behaviour for
Fashion-MNIST in Table 3 and Kuzushiji-MNIST in Table 4. However, the accuracy diﬀerence
11


key: chroma:document
string_value:
between the weighted TM and CoTM is even larger for these datasets.
#Clauses per class 50 100 250 500 1K 2K 4K 8K
Voting Margin 625 1250 3125 6.25K 12.5K 2.5K 5K 10K
Speciﬁcity 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0
Weighted, Ac. (%) 82.33 83.73 88.25 88.79 89.42 89.89 90.65 91.18
Coalesced, Ac. (%) 86.79 87.4 88.0 89.20 89.83 90.00 90.71 91.18
Training Time (s) 25.1 24.1 24.3 23.3 23.7 25.3 45.6 81.9
Testing Time (s) 1.6 1.6 1.6 1.5 1.7 1.6 4.1 6.4


key: chroma:document
string_value:
Testing Time (s) 1.6 1.6 1.6 1.5 1.7 1.6 4.1 6.4
Model Size (kB) 22 44 110 220 439 879 1758 3516
Table 3: Weighted Convolutional TM and Convolutional CoTM Fashion-MNIST mean test
accuracy and execution time per epoch, for an increasing number of clauses.
#Clauses per class 50 100 250 500 1K 2K 4K 8K
Voting Margin 625 1250 3125 6.25K 1.25K 2.5K 5K 10K
Speciﬁcity 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0
Weighted, Ac. (%) 71.99 89.19 92.75 93.86 94.89 95.40 95.85 96.08


key: chroma:document
string_value:
Coalesced, Ac. (%) 89.66 91.92 93.71 94.65 95.05 95.77 96.17 96.33
Training Time (s) 24.2 23.4 23.4 23.7 23.7 25.6 47.3 85.1
Testing Time (s) 1.7 1.6 1.6 1.6 1.6 1.6 4.1 6.4
Model Size (kB) 22 44 110 220 439 879 1758 3516
Table 4: Weighted Convolutional TM and Convolutional CoTM Kuzushiji-MNIST mean test
accuracy and execution time per epoch, for an increasing number of clauses.
Learning Speed. Figure 9 plots MNIST test accuracy per epoch for CoTM and weighted


key: chroma:document
string_value:
TM. As seen, CoTM accuracy climbs much faster, surpassing 99 .3 in epoch 19. The weighted
TM achieves this after 57 epochs, three times slower.
Baselines. Table 5 contains a comparison between the CoTM with 64 000 clauses and other
baselines. Here, CoTM clearly outperforms traditional machine learning techniques, like SVM,
Random Forest, and Gradient Boosting. Also, it performs competitively with widely used deep
learning architectures.


key: chroma:document
string_value:
learning architectures.
Class Imbalance Robustness. We here investigate whether the clause sharing of the CoTM
provides any robustness towards imbalanced training data. Table 6 contains robustness results
for the IMDb dataset. We evaluate robustness by removing a progressively larger fraction
of the positive sentiment training examples. The TM test accuracy drops signiﬁcantly with
increasing class imbalance. However, CoTM test accuracy is almost unaﬀected, indicating high
robustness.


key: chroma:document
string_value:
robustness.
The ﬁnal robustness evaluation is done with highly imbalanced CIFAR10 training data. We
make the data imbalanced by ordering the classes. We then use 0 .5rof the training data for the
class of rank rin the ordering. Table 7 contains the class-wise F1-scores for TM and CoTM.
We report the mean F1 score over epochs 90 ´100, averaged over 10 independent trials. As
seen, the F1 scores of TM are more severely aﬀected by class imbalance than for CoTM.
5 Conclusions and Further Work


key: chroma:document
string_value:
5 Conclusions and Further Work
In this paper, we proposed a new TM architecture where clauses are shared among multiple
outputs. We achieved this by merging multiple TMs into a single one, and then relating each
clause to each output by weighting. A positive weight makes the clause vote for output 1,
12


key: chroma:document
string_value:
051015202530354045505560657075808590951009595.59696.59797.59898.59999.5100
EpochAccuracy (%)
Weighted TM
Coalesced TM
Figure 9: Single-run test accuracy per epoch for CoTM and TM on MNIST.
while a negative weight makes it vote for output 0. By means of a weight matrix, the clauses
coalesce to produce multiple outputs.
Our empirical evaluations support the following main conclusions:
•CoTM seems to make signiﬁcantly better use of few clauses, both for MNIST, Fashion-


key: chroma:document
string_value:
MNIST, and Kuzushiji-MNIST. The diﬀerence is the largest for conﬁgurations with 50
clauses per class, leading to high accuracy with frugal memory footprint and faster learn-
ing and inference.
•Peak accuracy is reached faster, for instance 3 times faster on MNIST with 8 000 clauses
per class.
•CoTM is highly robust towards imbalanced training data, indicating an ability to repur-
pose clauses from one class to another. We believe the repurposing helps classify more
data sparse classes.


key: chroma:document
string_value:
data sparse classes.
In our further work, we intend to use the CoTM to build self-supervised rule-based language
models. Language models are usually based on predicting individual words or tokens from large
vocabularies, requiring a large number of outputs. Further, words may have similar meanings,
and we will investigate whether clause reuse can capture semantic relationships among words.
We also see opportunities for using the CoTM as an auto-encoder, supporting applications such


key: chroma:document
string_value:
as turning low-resolution images into high-resolution ones.
References
[1] K. D. Abeyrathna, B. Bhattarai, M. Goodwin, S. Gorji, O.-C. Granmo, L. Jiao, R. Saha,
and R. K. Yadav. Massively Parallel and Asynchronous Tsetlin Machine Architecture
Supporting Almost Constant-Time Scaling. In The Thirty-eighth International Conference
on Machine Learning (ICML 2021) . ICML, 2021.
[2] K. D. Abeyrathna, O.-C. Granmo, and M. Goodwin. Extending the Tsetlin Machine With


key: chroma:document
string_value:
Model 2D N-XOR MNIST K-MNIST F-MNIST
4-Nearest Neighbour [14, 43] 61.62 97.14 91.56 85.40
SVM [14] 94.63 98.57 92.82 89.7
Random Forest [28] 70.73 97.3 - 81.6
Gradient Boosting Classiﬁer [43] 87.15 96.9 - 88.0
Simple CNN [14, 28] 91.05 99.06 95.12 90.7
BinaryConnect [15] - 98.99 - -
FPGA-accelerated BNN [25] - 98.70 - -
Logistic Circuit (binary) [28] - 97.4 - 87.6
Logistic Circuit (real-valued) [28] - 99.4 - 91.3
PreActResNet-18 [14] - 99.56 97.82 92.0


key: chroma:document
string_value:
PreActResNet-18 [14] - 99.56 97.82 92.0
ResNet18 + VGG Ensemble [14] - 99.60 98.90 -
TM [20] 99.12 98.57 92.03 90.09
CoTM (Mean) 99.99˘0.099.31˘0.096.65˘0.01 91.83˘0.01
CoTM (95 %ile) 100.0 99.34 96.70 91.93
CoTM (Peak) 100.0 99.36 96.71 92.0
Table 5: Comparison with other machine learning techniques – test accuracy in percent.
Fraction Removed 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
∆ TM 0.0 1.0 4.3 7.9 10.3 17.2 24.8 27.3 29.5 30.5
∆ CoTM 0.0 0.5 0.2 0.0 0.2 0.4 0.4 0.8 1.4 2.8


key: chroma:document
string_value:
∆ CoTM 0.0 0.5 0.2 0.0 0.2 0.4 0.4 0.8 1.4 2.8
Table 6: Reduction in IMDb test accuracy after removing a fraction of the training examples
with positive sentiment.
[3] K. D. Abeyrathna, O.-C. Granmo, R. Shaﬁk, A. Yakovlev, A. Wheeldon, J. Lei, and
M. Goodwin. A Novel Multi-Step Finite-State Automaton for Arbitrarily Deterministic
Tsetlin Machine Learning. In Proceedings of the 40th International Conference on In-


key: chroma:document
string_value:
novative Techniques and Applications of Artiﬁcial Intelligence (SGAI), Cambridge, UK .
Springer International Publishing, 2020.
[4] K. D. Abeyrathna, O.-C. Granmo, X. Zhang, L. Jiao, and M. Goodwin. The Regression
Tsetlin Machine - A Novel Approach to Interpretable Non-Linear Regression. Philosophical
Transactions of the Royal Society A , 378, 2020.
[5] R. Agarwal, N. Frosst, X. Zhang, R. Caruana, and G. E. Hinton. Neural additive models:


key: chroma:document
string_value:
Interpretable machine learning with neural nets. arXiv preprint arXiv:2004.13912 , 2020.
[6] G. T. Berge, O.-C. Granmo, T. O. Tveit, M. Goodwin, L. Jiao, and B. V. Matheussen.
Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text
Categorization with Medical Applications. IEEE Access , 7:115134–115146, 2019.
[7] B. Bhattarai, O.-C. Granmo, and L. Jiao. Explainable Tsetlin Machine framework for


key: chroma:document
string_value:
fake news detection with credibility score assessment. arXiv preprint arXiv:2105.09114 ,
2021.
[8] B. Bhattarai, O.-C. Granmo, and L. Jiao. Word-level Human Interpretable Scoring Mech-
anism for Novel Text Detection Using Tsetlin Machines. arXiv preprint arXiv:2105.04708 ,
2021.
[9] B. Bhattarai, L. Jiao, and O.-C. Granmo. Measuring the Novelty of Natural Language Text
Using the Conjunctive Clauses of a Tsetlin Machine Text Classiﬁer. In 13th International


key: chroma:document
string_value:
Class airplane automobile bird cat deer dog frog horse ship truck
#examples 5000 2500 1250 625 315 157 80 40 20 10
CoTM : F1 0.47 0.50 0.36 0.36 0.26 0.15 0.02 0.04 0.00 0.00
TM : F1 0.39 0.42 0.25 0.20 0.10 0.00 0.00 0.00 0.00 0.00
Table 7: The class-wise F1 scores for a highly imbalanced version of the CIFAR10 dataset.
[10] C. D. Blakely and O.-C. Granmo. Closed-Form Expressions for Global and Local Interpre-


key: chroma:document
string_value:
tation of Tsetlin Machines. In 34th International Conference on Industrial, Engineering
and Other Applications of Applied Intelligent Systems (IEA/AIE 2021) . Springer, 2021.
[11] L. Breiman. ”random forests”. Machine Learning , 45(1):5–32, 2001.
[12] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. ”Classiﬁcation and Regression
Trees” . Wadsworth and Brooks, Monterey, CA, 1984.
[13] T. Chen and C. Guestrin. ”xgboost: A scalable tree boosting system”. In ”Proceedings


key: chroma:document
string_value:
of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining” , pages 785–794, 2016.
[14] T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha. Deep
Learning for Classical Japanese Literature. arXiv:1812.01718 , Dec 2018.
[15] M. Courbariaux, Y. Bengio, and J.-P. David. Binaryconnect: Training deep neural net-
works with binary weights during propagations. In Advances in neural information pro-
cessing systems , pages 3123–3131, 2015.


key: chroma:document
string_value:
cessing systems , pages 3123–3131, 2015.
[16] H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik. ”support vector
regression machines”. In M. C. Mozer, M. Jordan, and T. Petsche, editors, Advances in
Neural Information Processing Systems , volume 9. MIT Press, 1997.
[17] S. Gorji, O. C. Granmo, S. Glimsdal, J. Edwards, and M. Goodwin. Increasing the In-
ference and Learning Speed of Tsetlin Machines with Clause Indexing. In International


key: chroma:document
string_value:
Conference on Industrial, Engineering and Other Applications of Applied Intelligent Sys-
tems (IEA/AIE), kitakyushu, Japan . Springer, 2020.
[18] S. R. Gorji, O.-C. Granmo, A. Phoulady, and M. Goodwin. A Tsetlin Machine with
Multigranular Clauses. In Proceedings of the Thirty-ninth International Conference on
Innovative Techniques and Applications of Artiﬁcial Intelligence (SGAI), Cambridge, UK ,
volume 11927. Springer International Publishing, 2019.


key: chroma:document
string_value:
[19] O.-C. Granmo. The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to
Optimal Pattern Recognition with Propositional Logic. arXiv:1804.01508 , Apr 2018.
[20] O.-C. Granmo, S. Glimsdal, L. Jiao, M. Goodwin, C. W. Omlin, and G. T. Berge. The
Convolutional Tsetlin Machine. arXiv preprint arXiv:1905.09688 , 2019.
[21] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In


key: chroma:document
string_value:
Proceedings of the IEEE conference on computer vision and pattern recognition , pages
770–778, 2016.
[22] L. Jiao, X. Zhang, O.-C. Granmo, and K. D. Abeyrathna. On the Convergence of Tsetlin
Machines for the XOR Operator. arXiv preprint arXiv:2101.02547 , 2021.
[23] M. Karnaugh. The map method for synthesis of combinational logic circuits. Transactions
of the American Institute of Electrical Engineers, Part I: Communication and Electronics ,
72(5):593–599, 1953.
15


key: chroma:document
string_value:
[24] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report,
2009.
[25] C. Lammie, W. Xiang, and M. Rahimi Azghadi. Accelerating Deterministic and Stochastic
Binarized Neural Networks on FPGAs Using OpenCL. arXiv:1905.06105 , May 2019.
[26] Y. LeCun, L. Bottou, Y. Bengio, P. Haﬀner, et al. Gradient-based learning applied to
document recognition. Proceedings of the IEEE , 86(11):2278–2324, 1998.


key: chroma:document
string_value:
[27] J. Lei, T. Rahman, R. Shaﬁk, A. Wheeldon, A. Yakovlev, O.-C. Granmo, F. Kawsar, and
A. Mathur. Low-Power Audio Keyword Spotting Using Tsetlin Machines. Journal of Low
Power Electronics and Applications , 11, 2021.
[28] Y. Liang and G. Van den Broeck. Learning logistic circuits. In Proceedings of the 33rd
Conference on Artiﬁcial Intelligence (AAAI) , jan 2019.
[29] B. Lucena. ”structureboost: Eﬃcient gradient boosting for structured categorical vari-


key: chroma:document
string_value:
ables”. arXiv preprint arXiv:2007.04446 , 2020.
[30] S. M. Lundberg and S.-I. Lee. A uniﬁed approach to interpreting model predictions.
In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and
R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Cur-
ran Associates, Inc., 2017.
[31] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word


key: chroma:document
string_value:
vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association
for Computational Linguistics: Human Language Technologies , pages 142–150, Portland,
Oregon, USA, June 2011. Association for Computational Linguistics.
[32] H. Nori, S. Jenkins, P. Koch, and R. Caruana. ”interpretml: A uniﬁed framework for
machine learning interpretability”. arXiv preprint arXiv:1909.09223 , 2019.


key: chroma:document
string_value:
[33] B. J. Oommen. Stochastic searching on the line and its applications to parameter learning
in nonlinear optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B
(Cybernetics) , 27(4):733–739, 1997.
[34] A. Phoulady, O.-C. Granmo, S. R. Gorji, and H. A. Phoulady. The Weighted Tsetlin
Machine: Compressed Representations with Clause Weighting. In Proceedings of the Ninth
International Workshop on Statistical Relational AI (StarAI), New York, USA , 2020.


key: chroma:document
string_value:
[35] R. Saha, O.-C. Granmo, and M. Goodwin. Mining Interpretable Rules for Sentiment
and Semantic Relation Analysis using Tsetlin Machines. In Proceedings of the 40th Inter-
national Conference on Innovative Techniques and Applications of Artiﬁcial Intelligence
(SGAI), Cambridge, UK . Springer International Publishing, 2020.
[36] R. Shaﬁk, A. Wheeldon, and A. Yakovlev. Explainability and Dependability Analysis
of Learning Automata based AI Hardware. In IEEE 26th International Symposium on


key: chroma:document
string_value:
On-Line Testing and Robust System Design (IOLTS), Naples, Italy . IEEE, 2020.
[37] J. Sharma, R. Yadav, O.-C. Granmo, and L. Jiao. Human Interpretable AI: Enhancing
Tsetlin Machine Stochasticity with Drop Clause. arXiv preprint arXiv:2105.14506 , 2021.
[38] M. L. Tsetlin. On behaviour of ﬁnite automata in random medium. Avtomat. i Telemekh ,
22(10):1345–1354, 1961.
[39] L. G. Valiant. A Theory of the Learnable. Communications of the ACM , 27(11):1134–1142,
1984.
16


key: chroma:document
string_value:
[40] A. Wheeldon, R. Shaﬁk, T. Rahman, J. Lei, A. Yakovlev, and O.-C. Granmo. Learning
Automata based Energy-eﬃcient AI Hardware Design for IoT. Philosophical Transactions
of the Royal Society A , 2020.
[41] A. Wheeldon, R. Shaﬁk, A. Yakovlev, J. Edwards, I. Haddadi, and O.-C. Granmo. Tsetlin
Machine: A New Paradigm for Pervasive AI. In Proceedings of the SCONA Workshop at
Design, Automation and Test in Europe (DATE) , 2020.


key: chroma:document
string_value:
[42] A. Wheeldon, A. Yakovlev, R. Shaﬁk, and J. Morris. Low-Latency Asynchronous Logic
Design for Inference at the Edge. In 2021 Design, Automation and Test in Europe Con-
ference (DATE21) , pages 370–373, 2021.
[43] H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking
machine learning algorithms. arXiv:1708.07747 , 2017.
[44] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Distributed Word Representation


key: chroma:document
string_value:
in Tsetlin Machine. arXiv preprint arXiv:2104.06901 , 2021.
[45] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Human-Level Interpretable Learn-
ing for Aspect-Based Sentiment Analysis. In The Thirty-Fifth AAAI Conference on Arti-
ﬁcial Intelligence (AAAI-21) . AAAI, 2021.
[46] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Interpretability in Word Sense
Disambiguation using Tsetlin Machine. In 13th International Conference on Agents and


key: chroma:document
string_value:
Artiﬁcial Intelligence (ICAART), Vienna, Austria . INSTICC, 2021.
[47] X. Zhang, L. Jiao, O.-C. Granmo, and M. Goodwin. On the Convergence of Tsetlin Ma-
chines for the IDENTITY- and NOT Operators. IEEE Transactions on Pattern Analysis
and Machine Intelligence , 2021.
A Appendix
A.1 Structure and Prediction
A CoTM can be deﬁned as a quadruple:
tX,Y,C,Wu. (7)
We start with deﬁning each element of the quadruple.
A.1.1 Input Space


key: chroma:document
string_value:
A.1.1 Input Space
The input space of the CoTM is denoted X. The input space consists of vectors xofo
propositional inputs: x“rx1,...,x osPX,X“t0,1uo.
A.1.2 Output Space
The output space is denoted Y. The output space contains vectors yofmpropositional
outputs: y“ry1,y2,...,ymsPY,Y“t0,1um.
17


key: chroma:document
string_value:
A.1.3 Memory Matrix
We deﬁne Cas a space of matrices Cformed from nˆ2ointeger entries, CPC“t1,2,..., 2Nunˆ2o:
C“x1x2¨¨¨xo ¯x1 ¯x2¨¨¨ ¯xo»
——–ﬁ
ﬃﬃﬂc1,1c1,2¨¨¨c1,oc1,o`1c1,o`2¨¨¨c1,2oclause 1
c2,1c2,2¨¨¨c2,oc2,o`1c2,o`2¨¨¨c2,2oclause 2
........................
cn,1cn,2¨¨¨cn,ocn,o`1cn,o`2¨¨¨cn,2oclausen(8)
“ rcj,ksPt1,2,..., 2Nunˆ2o. (9)
This is the pattern memory of the CoTM, with each row specifying a clause. The nmatrix


key: chroma:document
string_value:
rows represent nclauses while the 2 ocolumns speciﬁes each pattern. That is, each column
refers to an input xkor its negation ¯ xk(we use ¯xkas shorthand notation for ␣xk). The entries
in the matrix take values from t1,2,..., 2Nu, specifying the composition of the nclauses as
well as the conﬁdence of the composition (how strongly the pattern is memorized).
A.1.4 Pattern Construction
ACmatrix provides npatterns as follows. The ﬁrst step is to map each entry in Cto one of


key: chroma:document
string_value:
two actions aPt0,1u, obtaining the matrix APt0,1unˆ2obelow. The function Gp¨qperforms
this mapping by selecting action 0 for entries in Cwith values from 1 to Nand by selecting
action 1 for entries with values from N`1 to 2N:
gpcq “#
0,if1ďcďN
1,ifN`1ďcď2N.(10)
A“GpCq “»
———–gpc1,1qgpc1,2q ¨¨¨gpc1,2oq
gpc2,1qgpc2,2q ¨¨¨gpc2,2oq
............
gpcn,1qgpcn,2q ¨¨¨gpcn,2oqﬁ
ﬃﬃﬃﬂ(11)
“»
———–a1,1a1,2¨¨¨a1,2o
a2,1a2,2¨¨¨a2,2o
............
an,1an,2¨¨¨an,2oﬁ
ﬃﬃﬃﬂ“raj,ksPt0,1unˆ2o. (12)


key: chroma:document
string_value:
an,1an,2¨¨¨an,2oﬁ
ﬃﬃﬃﬂ“raj,ksPt0,1unˆ2o. (12)
Values close to NandN`1 signify low-conﬁdence actions while values close to 1 and 2 N
signify high-conﬁdence actions (deeply memorized).
After the actions A“GpCqhave been decided, the next step is to formulate the patterns.
Each pattern is formed by ANDing selected entries xkin the input vector x, or their negations
¯xk.3The pattern
x1^x2^¯x3, (13)
for instance, ANDs the values x1,x2, and ¯x3. It evaluates to 1 if and only if x1“1,x2“1,
andx3“0.


key: chroma:document
string_value:
andx3“0.
The actions from GpCqdecide which inputs to AND, and whether to negate them. For
1ďkďo, if actionaj,kis 1 then input xktakes part in the AND of pattern j. Foro`1ďkď2o,
¯xk´otakes part in the AND instead. If aj,kis 0, on the other hand, the corresponding input
does not play a role in the evaluation of the jth clause and can be ignored.
The imply (ñ) operator captures the above dynamics. That is, if ajkis 0, thenaj,kñxk


key: chroma:document
string_value:
is always 1. Accordingly, the xkvalue does not impact the result. Conversely, the value of
3Such an expression is referred to as a conjunctive clause in propositional logic, with xkand ¯xkbeing called
literals .
18


key: chroma:document
string_value:
aj,kñxkis decided solely by the xkvalue ifajkis 1. We thus deﬁne an Imply operator to
calculate the eﬀect of each input on each clause:
ImplyrA,xs“»
———–a1,1ñx1¨¨¨a1,oñxoa1,o`1ñ¯x1¨¨¨a1,2oñ¯xo
a2,1ñx1¨¨¨a2,oñxoa2,o`1ñ¯x1¨¨¨a2,2oñ¯xo
..................
an,1ñx1¨¨¨an,oñxoan,o`1ñ¯x1¨¨¨an,2oñ¯xoﬁ
ﬃﬃﬃﬂ. (14)
Finally, we deﬁne the operator And for evaluating the clauses:
AndrZs“»
———–Ź2o
k“1z1,kŹ2o
k“1z2,k
...Ź2o
k“1zn,kﬁ
ﬃﬃﬃﬂ. (15)


key: chroma:document
string_value:
k“1z1,kŹ2o
k“1z2,k
...Ź2o
k“1zn,kﬁ
ﬃﬃﬃﬂ. (15)
Accordingly, the value of each clause can be calculated from the input vector xand the pattern
memoryC, providing a pattern value column vector c:
c“»
———–c1
c2
...
cnﬁ
ﬃﬃﬃﬂ“AndpImplypGpCq,xqq. (16)
As seen, we employ the npatterns from GpCqwhen processing the input vector x.
A.1.5 Output Prediction
For predicting the output vector, the CoTM uses mˆninteger weights. These capture how


key: chroma:document
string_value:
each of the nclauses relates to each of the moutputs. The weights are organized as a matrix:
W“»
———–w1,1w1,2¨¨¨w1,n
w2,1w2,2¨¨¨w2,n
............
wm,1wm,2¨¨¨wm,nﬁ
ﬃﬃﬃﬂ“rwi,jsPt...,´2,´1,1,2,...umˆn. (17)
We have one weight per output iper clausej. In the following, we will refer to row iofWas
wi:
wi““
wi,1wi,2¨¨¨wi,n‰
. (18)
The weights decide the eﬀect a clause value cj“1 from chas on predicting the output vector.


key: chroma:document
string_value:
If the weight wi,jis positive and cj“1, thejth clause casts wi,jvotes in favour of output
yi“1. If the weight is negative and cj“1, clausejcasts´wi,jvotes in favour of output
yi“0 instead. If cj“0, on the other hand, no votes are cast from that pattern. The overall
calculation can be formulated as a product between the matrix Wand the column vector c,
obtaining a column vector vof votes:
v“Wc. (19)
Finally, the output vector prediction is decided by a majority vote using a unit step operator U:


key: chroma:document
string_value:
Upvq“»
———–v1ě0
v2ě0
...
vmě0ﬁ
ﬃﬃﬃﬂ“ruisPt0,1umˆ1. (20)
Combining the diﬀerent steps, we predict the output yfrom input xas follows:
ˆy“UpW¨AndpImplypGpCq,xqqqT. (21)
19


key: chroma:document
string_value:
A.2 Learning Patterns and Weights
The memory matrix Cand the weight matrix Ware learnt online from a set Sof training
examplespx,yqPS. The goal is to maximize prediction accuracy Ppˆyi“yiqper entry in y.
We here consider the procedure for updating CandWfrom a single training example px,yq.
To learn from a stream of training examples, this procedure is repeated for every example. We
ﬁrst extend the TM clause coordination mechanism to stimulate production of diverse clauses


key: chroma:document
string_value:
for multiple outputs. This mechanism activates clause rows in Cfor updating based on the
vote sums vand the example outputs y. We then go through the details of how we update each
activated clause in Cbased onpx,yq. Finally, we consider the updating of the corresponding
weights inW, learning the relationship between each clause in Cand each output in y.
A.2.1 Resource Allocation
To ensure that the nclause rows of Care diverse and robust, the CoTM employs a voting


key: chroma:document
string_value:
margintfor each entry yiiny. The voting margin is a user-conﬁgurable constant that puts
a max on how many clauses from Ccoalesce to predict a particular output yiwhen facing a
training example px,yq. While the original TM considers a single output y, we here present a
scheme for handling a vector yof outputs.
For CoTM, when an entry yiin the training output yis 0, the summation margin is ´t.
Conversely, when the entry is 1 the summation margin is t. The resulting nsummation margins


key: chroma:document
string_value:
foryare organized in the column vector q:
q“yt´¯yt“»
———–y1t´¯y1t
y2t´¯y2t
...
ymt´¯ymtﬁ
ﬃﬃﬃﬂ“rqisPt´t,tumˆ1. (22)
After the summation margins qfor the training example px,yqhave been set, the next step
is to decide which clause rows in Cto update. This is done by comparing each vote sum vi
fromv(Eqn. 19) with the corresponding margin yit´¯yitinq. By clipping the vote sum to fall
between´tandt, we calculate an update probability per output yiiny:abs´
qi´clippvi,´t,tq
2t¯
.


key: chroma:document
string_value:
qi´clippvi,´t,tq
2t¯
.
The update probability is 0 .0 when the summation margin tis met or surpassed, and gradually
increases to 1 .0 towards the maximum distance 2 t. The update probabilities are organized in
the column vector d:
d“abspq´clippv,´t,tqq¨1
2t“abs¨
˚˚˚˚˝»
————–q1´clippv1,´t,tq
2tq2´clippv2,´t,tq
2t...
qm´clippvm,´t,tq
2tﬁ
ﬃﬃﬃﬃﬂ˛
‹‹‹‹‚“rdisPtxPR|0ďxď1umˆ1.
(23)
From the update probabilities in d, we next randomly select which clause rows in Cto update.


key: chroma:document
string_value:
In brief, many clauses are updated when the vote sum viis far from its margin yit´¯yit, and
comes to a complete standstill when the margin is met or surpassed. Accordingly, we attain a
natural balancing of pattern representation resources.
A.2.2 Type I Feedback – Suppressing False Negative Output
There are two kinds of updates, Type I and Type II. We consider the nature of these next.
Type I Feedback reinforces the output entries yifrom the training example px,yq, considering


key: chroma:document
string_value:
one output entry yiat a time. Each output entry yiis either 0 or 1. As seen from Eqn. 21,
outputyi“1 can be reinforced by identifying the clause rows jinCthat has a positive weight
wi,jě0, and then making those evaluate to 1. Output yi“0, on the other hand, can be
reinforced by making clauses with negative weight wi,jă0 evaluate to 1. In other words,
Type I Feedback is given when yidpwi,jě0q, wheredis the XNOR-operator.
20


key: chroma:document
string_value:
The selection of clauses to update is done by sampling a random value 0 or 1 per output-
clause pairpi,jq, withireferring to an entry in yandjreferring to a row in C. The probability
of updating from a pair pi,jqis decided by the update probability diind. We draw mˆn
random values πi,juniformly in the range r0,1sto decide the updating. The updating performed
per pair is the same as for the original TM, except that the CoTM combines multiple outputs yi.


key: chroma:document
string_value:
We organize the selection criterion yidpwi,jě0qin onemˆnmatrix and the random
sampling in another. The two matrices are combined by element-wise multiplication using the
Hadamard product ˝:
RI“»
———–y1dpw1,1ě0qy1dpw1,2ě0q ¨¨¨y1dpw1,ně0q
y2dpw2,1ě0qy1dpw2,2ě0q ¨¨¨y2dpw2,ně0q
............
ymdpwm,1ě0qy1dpwm,2ě0q ¨¨¨ymdpwm,ně0qﬁ
ﬃﬃﬃﬂ˝ (24)
»
———–π1,1ďd1π1,2ďd1¨¨¨π1,nďd1
π2,1ďd2π2,2ďd2¨¨¨π2,nďd2
............
πm,1ďdmπm,2ďdm¨¨¨πm,nďdmﬁ
ﬃﬃﬃﬂ(25)
“ rrI
i,jsPt0,1umˆn. (26)


key: chroma:document
string_value:
ﬃﬃﬃﬂ(25)
“ rrI
i,jsPt0,1umˆn. (26)
In other words, matrix RIpicks out which rows in Cto update, per output yi. Like for the
original TM, the actual updates subdivides into Type Ia and Type Ib feedback, which we
consider next.
Type Ia Feedback – Reﬁning Clauses. Type Ia Feedback is given to clause rows jin
Cthat evaluate to 1. Reinforcement is given per variable xkor its negation ¯ xk, by updating
matrixC. In brief, action a“1 is reinforced by increasing cj,kwhenevercj“1andxk“1


key: chroma:document
string_value:
for 1ěkěoandcj“1and ¯xk´o“1 foro`1ěkě2o. This reinforcement is strong and
makes the clause remember and reﬁne the pattern it recognizes in x.4Which entries cj,kinC
to increase in this manner are organized in the matrix FIa
i, per output yi:
FIa
i““
c¨¨¨c‰
˝»
———–x1¨¨¨xo¯x1¨¨¨ ¯xo
x1¨¨¨xo¯x1¨¨¨ ¯xo
..................
x1¨¨¨xo¯x1¨¨¨ ¯xoﬁ
ﬃﬃﬃﬂ(27)
“»
———–c1c1¨¨¨c1
c2c2¨¨¨c2
............
cncn¨¨¨cnﬁ
ﬃﬃﬃﬂ˝»
———–x1¨¨¨xo¯x1¨¨¨ ¯xo
x1¨¨¨xo¯x1¨¨¨ ¯xo
..................
x1¨¨¨xo¯x1¨¨¨ ¯xoﬁ
ﬃﬃﬃﬂ. (28)


key: chroma:document
string_value:
..................
x1¨¨¨xo¯x1¨¨¨ ¯xoﬁ
ﬃﬃﬃﬂ. (28)
Type Ib Feedback – Coarsening Clauses. Type Ib Feedback reinforces actions aj,k“0.
That is,aj,k“0 in matrix Ais reinforced by reducing cj,kin matrixC. The reinforcement
happens randomly with probability1
swhenevercj“0orxk“0 for 1ěkěoand whenever
cj“0or¯xk´o“0 foro`1ěkě2o. This reinforcement is weak (triggered with low
probability) and coarsens infrequent patterns, making them frequent. The user-conﬁgurable


key: chroma:document
string_value:
parameterscontrols pattern frequency, i.e., a higher sproduces less frequent patterns. Which
entriescj,kinCto reduce in this manner are organized in the matrix FIb
i, per output entry yi.
We ﬁrst randomly identify which entries in Care candidates for updating. We draw nˆ2o
random values πi,juniformly in the range r0,1sto decide this updating, organized in the nˆ2o
4Note that we here apply boosting of true positives. The standard setup is to apply the update with
probabilitys´1
s.
21


key: chroma:document
string_value:
matrixBi:
Bi“»
———–π1,1ď1
sπ1,2ď1
s¨¨¨π1,2oď1
s
π2,1ď1
sπ2,2ď1
s¨¨¨π2,2oď1
s............
πn,1ď1
sπn,2ď1
s¨¨¨πn,2oď1
sﬁ
ﬃﬃﬃﬂ“rbi
j,ksPt0,1unˆ2o. (29)
In this manner, which entries in Cto decrease is organized in the mˆ2omatrixFIb
ifor the
particular output yi:
FIb
i“Bi˝»
———–¯x1_¯c1¨¨¨ ¯xo_¯c1x1_¯c1¨¨¨xo_¯c1
¯x1_¯c2¨¨¨ ¯xo_¯c2x1_¯c2¨¨¨xo_¯c2
..................
¯x1_¯cn¨¨¨ ¯xo_¯cnx1_¯cn¨¨¨xo_¯cnﬁ
ﬃﬃﬃﬂ. (30)
A.2.3 Type II Feedback – Increasing Discrimination Power


key: chroma:document
string_value:
Type II Feedback is given to clause rows jinCthat outputs false positives. That is, when yi“1
the aﬄicted clauses are the ones with negative weight wi,jă0, considering each output-clause
pair (j,i). Conversely, when yi“0 clauses with positive weight wi,jě0 are targeted. More
concisely stated, Type II Feedback is given when yi‘pwi,jě0q, where‘is the XOR-operator.
In addition to this selection criteria, rows jinCare selected randomly for updating per pair


key: chroma:document
string_value:
pi,jqdecided by the update probability diind. Uniquely for Type II Feedback, dis further
multiplied by a scalar e, which usually is set to 1 .0. However, for one-hot-encoded multi-class
problems,eis set to 1.0{pm´1q. The two selection criteria are combined through element-wise
multiplication using the Hadamard product, and organized in the mˆnmatrixRII:
RII“»
———–y1‘pw1,1ě0qy1‘pw1,2ě0q ¨¨¨y1‘pw1,ně0q
y2‘pw2,1ě0qy1‘pw2,2ě0q ¨¨¨y2‘pw2,ně0q
............
ym‘pwm,1ě0qy1‘pwm,2ě0q ¨¨¨ym‘pwm,ně0qﬁ


key: chroma:document
string_value:
ym‘pwm,1ě0qy1‘pwm,2ě0q ¨¨¨ym‘pwm,ně0qﬁ
ﬃﬃﬃﬂ˝ (31)
»
———–π1,1ďd1e π 1,2ďd1e¨¨¨π1,nďd1e
π2,1ďd2e π 2,2ďd2e¨¨¨π2,nďd2e
............
πm,1ďdme π m,2ďdme¨¨¨πm,nďdmeﬁ
ﬃﬃﬃﬂ(32)
“ rrII
i,jsPt0,1umˆn. (33)
Like Type Ia Feedback, we only update clause rows jinCthat evaluates to 1, i.e., cj“1.
Finally, only Exclude actions for literals of value 0 are given feedback, reinforcing Include .
That is, state cj,kincreases when ¯ xk¯aj,k“1, for 1ěkěo. Foro`1ěkě2o, statecj,k


key: chroma:document
string_value:
increases when xk´o¯aj,k“1. These criteria are ANDed in the FII
imatrix through element-wise
multiplication:
FII
i““
c¨¨¨c‰
˝»
———–¯x1¯a1,1¨¨¨ ¯xo¯a1,ox1¯a1,o`1¨¨¨xo¯a1,2o
¯x1¯a2,1¨¨¨ ¯xo¯a2,ox1¯a2,o`1¨¨¨xo¯a2,2o
..................
¯x1¯an,1¨¨¨ ¯xo¯an,ox1¯an,o`1¨¨¨xo¯an,2oﬁ
ﬃﬃﬃﬂ(34)
“»
———–c1c1¨¨¨c1
c2c2¨¨¨c2
............
cncn¨¨¨cnﬁ
ﬃﬃﬃﬂ˝»
———–¯x1¯a1,1¨¨¨ ¯xo¯a1,ox1¯a1,o`1¨¨¨xo¯a1,2o
¯x1¯a2,1¨¨¨ ¯xo¯a2,ox1¯a2,o`1¨¨¨xo¯a2,2o
..................
¯x1¯an,1¨¨¨ ¯xo¯an,ox1¯an,o`1¨¨¨xo¯an,2oﬁ
ﬃﬃﬃﬂ(35)
22


key: chroma:document
string_value:
A.2.4 Updating the Memory Matrix
The memory matrix Cis updated based the training example px,yqusing the Type I and Type
II feedback matrices FIa
i,FIa
i, andFII
i. This updating is done per output yiiny, simply by
summation:
C˚
T`1“CT`mÿ
i“1´
QII
i˝FII
i`QI
i˝FIa
i´QI
i˝FIb
i¯
. (36)
Above,QI
imapsRI
idown to the literals:
QI
i“»
—–rI
i...
rI
iﬁ
ﬃﬂT
“»
———–rI
i,1rI
i,1¨¨¨rI
i,1
rI
i,2rI
i,2¨¨¨rI
i,2............
rI
i,nrI
i,n¨¨¨rI
i,nﬁ
ﬃﬃﬃﬂ. (37)
Correspondingly, QII
ideals withRII
i:
QII
i“»


key: chroma:document
string_value:
Correspondingly, QII
ideals withRII
i:
QII
i“»
—–rII
i...
rII
iﬁ
ﬃﬂT
“»
———–rII
i,1rII
i,1¨¨¨rII
i,1
rII
i,2rII
i,2¨¨¨rII
i,2............
rII
i,nrII
i,n¨¨¨rII
i,nﬁ
ﬃﬃﬃﬂ. (38)
The ﬁnal step is to clip the state values in Cto make sure that they stay within 1 and 2 N:
Ct`1“clip`
C˚
t`1,1,2N˘
. (39)
A.2.5 Learning Weights
The perhaps most novel part of CoTM is the fact that the clauses are not initially related to


key: chroma:document
string_value:
any particular output yiiny. By learning how to relate each clause to each output, the CoTM
opens up for clause sharing. That is, the weight matrix Wis initialized randomly with values
fromt´1,1u, thus randomly assigning each clause to a truth value for yi. The original TM,
on the other hand, operates with a ﬁxed weight matrix Wwhere each clause is preassigned to
a particular output. That is, the clause rows are ﬁrst partitioned into mpartitions of equal


key: chroma:document
string_value:
size. Half of the clause rows inside a partition is then assigned the weight ´1 and the other
half is assigned the weight `1. For other outputs the weight is set to 0, thus, blocking sharing
of clauses between the outputs.
The updating of weights is quite straightforward. Type Ia Feedback changes the weight
in one-step increments away from 0, while Type II Feedback changes the weight in one-step
decrements towards 0. Type Ib Feedback leaves the weights unchanged. This updating of the


key: chroma:document
string_value:
weight matrix WfromWttoWt`1is captured by the following equation:
Wt`1“Wt`pRI`RIIq˝¨
˚˚˚˝»
———–c1y1c2y1¨¨¨cny1
c1y2c2y2¨¨¨cny2
............
c1ymc2ym¨¨¨cnymﬁ
ﬃﬃﬃﬂ´»
———–c1¯y1c2¯y1¨¨¨cn¯y1
c1¯y2c2¯y2¨¨¨cn¯y2
............
c1¯ymc2¯ym¨¨¨cn¯ymﬁ
ﬃﬃﬃﬂ˛
‹‹‹‚.(40)
Note that the 1-entries of RIandRIIare non-overlapping, singling out which clause rows j
are eligible for Type Ia and Type II Feedback. The value of yicontrols the direction of the
weight update.
23


key: chroma:document
string_value:
Chapter 1Your First Tsetlin MachineIn this chapter, you build your ﬁrst Tsetlin machine. Using toy data onvehicles and real-life data on breast cancer, you learn how Tsetlin machinesunify three powerful strategies for pattern recognition:1.First, you explore how to use simple if-then rules to recognize regular-ities in data – so-called frequent patterns. Frequent patterns captureknowledge like: “most cars have four wheels”.2.Next, you discover how to distinguish between di↵erent types of


key: chroma:document
string_value:
how to distinguish between di↵erent types of ob-jects. For instance, both cars and planes may have four wheels. How-ever, planes have wings, while cars do not. You will ﬁnd out howTsetlin machines learn patterns that are both frequent and able todi↵erentiate, demonstrated on the vehicle- and breast cancer data.3.Finally, you uncover how a Tsetlin machine dissects the data to con-struct multiple rules, making them interact to predict breast cancer.At the end of this chapter, you know how a


key: chroma:document
string_value:
cancer.At the end of this chapter, you know how a standard Tsetlin machine works.You have also learned how to implement one.The rest of the book introduces several variants of the above mecha-nisms, like regression and convolution. After mastering Chapter 1, thosevariations will be easy to pick up because they all build on the standardTsetlin machine.6


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo
ObservationIf-then Rules Section 1.2Boolean Features Section 1.1Rule Coordination Section 1.5-1.6Plane
Rule Learning Section 1.3-1.4Output


key: chroma:document
string_value:
Figure 1.1: Overview of Tsetlin machine and chapter structure.Figure1.1shows the overall structure of a Tsetlin machine, which youexplore in Chapter 1. First, in Section1.1,y o ul e a r nh o wt op r e p a r et h einput to a Tsetlin machine using so-calledBoolean features.T h e n y o udiscover how to describe patterns with if-then rules in Section1.2. Usingthe if-then rules, Section1.3explains how you uncover frequent patterns indata, while Section1.4covers how to make the if-then rules


key: chroma:document
string_value:
Section1.4covers how to make the if-then rules discriminative.Finally, you study the complete Tsetlin machine algorithm in Section1.5and Section1.6, investigating how the Tsetlin machine combines multipleif-then rules.1.1 Boolean FeaturesA Tsetlin machine observes objects (or phenomena), which it recognizesbased on the objects’ features. Each feature is an object property that eitherisTrueorFalse,m a k i n gi tBoolean.C o n s i d e r ,f o re x a m p l e ,ad a t a b a s e o nvehicles of di↵erent


key: chroma:document
string_value:
m p l e ,ad a t a b a s e o nvehicles of di↵erent types, like cars, boats, trains, and planes. Vehicles ofthe same kind will naturally have some properties in common. The cars7


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmodepicted in Figure1.2,f o ri n s t a n c e ,a l lh a v ef o u rw h e e l s ,l a c kw i n g s ,a n dtransport people. At the same time, they are di↵erent in various ways. Oneis a sedan, the others are a pickup, an SUV, and a van. They also appearin di↵erent colors. All these properties are examples of Boolean features.For instance, a vehicle is either yellow or not yellow.Table1.1lists six example vehicles and their ﬁve Boolean


key: chroma:document
string_value:
six example vehicles and their ﬁve Boolean features:FourWheels,Transports People,Wings,Yellow,a n dBlue.T h e r e i s o n e c o l u m nper feature, each entry taking the valueTrueorFalse,d e n o t e d‘•’a n d‘·’,Tsetlin Machine OriginsThe Tsetlin machine is named after Michael Lvovitch Tsetlin (22 Septem-ber 1924 – 30 May 1966), a Soviet mathematician and physicist. Tsetlininvented a learning algorithm that optimizes its behavior online, embeddedin a random environment. Tsetlin’s algorithm is


key: chroma:document
string_value:
a random environment. Tsetlin’s algorithm is particularly fascinatingbecause it keeps track of previous experience only using a single integer.Further, it is adaptive, capable of “changing its mind”. Based on this al-gorithm, Tsetlin introduced algorithm collectives for solving complex prob-lems. The Tsetlin machine builds upon the work of Tsetlin, dealing withadvanced pattern recognition tasks.Machine Learning BasicsMachine learning deals with how computers can improve from experienceand what


key: chroma:document
string_value:
how computers can improve from experienceand what constitutes the fundamental laws of learning. A typical machinelearning system consists of:•Atasksuch as classiﬁcation or regression, for instance, photo clas-siﬁcation or breast cancer recurrence prediction.•Training datawith individual cases, which is the experience.•Analgorithmthat learnssupervisedfrom labelled training data,unsupervisedfrom unlabelled data, or fromreinforcementsuchas penalties and rewards.•Testing datafor evaluating the


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo#Four Wheels Transports People Wings Yellow BlueCar1.•• · · ••2.•• · • ·•3.•• · • ·•4.•• • · •·5.•· • • ··6.·• • · •·Table 1.1: A table of three cars and three planes, with ﬁve Boolean features.There is one column per feature, each entry taking the valueTrue(•)o rFalse(·). The ﬁnal column decides type of vehicle,Car(•)o rPlane(·).respectively. The last column contains type of vehicle:Car(•)o rPlane(·).Vehicle #1, for instance, is


key: chroma:document
string_value:
rPlane(·).Vehicle #1, for instance, is of typeCar,h a sFour Wheels,Transports People,does not haveWings,i sBlue,b u tn o tYellow.S o m eo ft h ev e h i c l ef e a t u r e sare essential for recognition, such asFour Wheels. Others are peripheral,Tsetlin Machines vs Black BoxesRecent research has brought increasingly accurate learning algorithms andpowerful computation platforms. However, the accuracy gains come withescalating computation costs, and models are getting too complicated forhumans to


key: chroma:document
string_value:
models are getting too complicated forhumans to comprehend. Mounting computation costs make AI an assetfor the few and impact the environment. Simultaneously, the obscurity ofAI-driven decision-making raises ethical concerns. We are risking unfair,erroneous, and, in high-stakes domains, fatal decisions. Tsetlin machinesaddress the following key challenges:•They are universal function approximators, like neural networks.•They are rule-based, like decision trees.•They are summation-based, like


key: chroma:document
string_value:
decision trees.•They are summation-based, like Naive Bayes classiﬁer and logisticregression.•They are hardware-near, with low energy- and memory footprint.As such, the Tsetlin machine is a general-purpose, interpretable, and low-energy machine learning approach.9


key: chroma:document
string_value:
Figure 1.2: Four di↵erent vehicles - van, pickup, sedan, and SUV.PropertyValuesDescriptionMenopauselt40, ge40, premenoPre- or postmenopausal statusat time of diagnosisInv-nodes0-2, 3-5, 6-8Axillary lymph nodes containingvisible metastatic breast cancerDeg-malig1, 2, 3Degree of tumor malignancyRecurrenceyes, noPatient recurrence statusTable 1.2: Four properties and their values from the breast cancer


key: chroma:document
string_value:
and their values from the breast cancer data.#MenopauseInv-nodesDeg-maligRecurrence1.ge403-53yes2.lt400-23no3.ge406-83yes4.ge400-22no5.premeno0-23yes6.premeno0-21noTable 1.3: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.such asBlue. Later, you discover how a Tsetlin machine distinguishesessential features from more peripheral ones.10


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoMenopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 3Recurrence1.·• ··•··· ••2.•· ·•···· •·3.·• ···•·· ••4.·• ·•···• ··5.·· ••···· ••6.·· ••··•· ··Table 1.4: A Booleanized version of Table1.3with one Boolean feature pervalue ofMenopause,Inv-nodes,Deg-malig,a n dRecurrence.E a c h c o l u m nis a Boolean feature, each entry taking the value True (•)o rF a l s e(·).Breast Cancer Recurrence.In your ﬁrst task in this book,


key: chroma:document
string_value:
Cancer Recurrence.In your ﬁrst task in this book, you predictbreast cancer recurrence using a real-life dataset. The dataset containsinformation on breast cancer patients ﬁve years after surgery.1You willfocus on six patients and four (out of nine) multi-valued properties, whichis enough data to lay out how a Tsetlin machine operates.The four properties are:Menopause,Inv-nodes,Deg-malig,a n dRecur-rence.Y o u ﬁ n d a d e s c r i p t i o n o f t h e s e p r o p e r t i e s a n d t h e v a l u e


key: chroma:document
string_value:
h e s e p r o p e r t i e s a n d t h e v a l u e s t h e y t a k e i nTable1.2.F o r i n s t a n c e ,Menopausetakes the valueslt40,ge40,o rpremeno,describing menopausal status at the time of diagnosis.Table1.3contains the six selected patients, our objects of study. Theﬁrst object, marked #1, is a patient withRecurrence.Y o uo b t a i nt h ec l a s sfrom the last column, whereRecurrenceisyesfor patient #1. Moreover,this patient hasMenopausetypege40,Inv-nodesvalue3-5,a n dDeg-maligvalue3.B y b


key: chroma:document
string_value:
n dDeg-maligvalue3.B y b e i n g m u l t i - v a l u e d , t h e p a t i e n t p r o p e r t i e s a r e n o t y e t r e a d yfor use by the Tsetlin machine and need to be transformed by Booleanizingthe data.Data Booleanization.The ﬁrst step of building a Tsetlin machine isto Booleanize your data by making suitable Boolean features. The breastcancer data demonstrates the starting point. You start by inspecting the1This breast cancer dataset was obtained from the University Medical Centre,


key: chroma:document
string_value:
was obtained from the University Medical Centre, In-stitute of Oncology, Ljubljana, Slovenia. Thanks go to M. Zwitter and M. Soklic forproviding the data, found athttps://archive.ics.uci.edu/ml/datasets/Breast+Cancer.11


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmovalues in Table1.2.F o re a c hp r o p e r t y ,y o uc r e a t eo n eB o o l e a nf e a t u r ep e rvalue.Menopause,f o ri n s t a n c e ,g i v e st h eB o o l e a nf e a t u r e slt40,ge40,a n dpremeno. After Booleanization, you obtain Table1.4,w h i c he n c o m p a s s e snine Boolean features. You get nine features because the three originalproperties each takes three values. The table provides the same informationon each


key: chroma:document
string_value:
The table provides the same informationon each patient as the information in Table1.3.T h e d i ↵ e r e n c e i s t h a tTable1.4only contains values that are eitherTrueorFalse. Notice howtheRecurrencecases all have aDeg-maligvalue of3.Y o ue x p l o i tt h i sa n dother data regularities later in this chapter.Advantages of Boolean FeaturesUsing Boolean features has several advantages:•Boolean features ﬁt well with a computer because you can storethem individually as bits.Truebecomes1,a n


key: chroma:document
string_value:
storethem individually as bits.Truebecomes1,a n dFalsebecomes0.Hardware-near bitwise operators then dramatically increase infer-ence speed. Further, the resulting small energy- and memory foot-print is ideal for edge computing and Internet of Things (IoT).•You can use Boolean algebra – the language of computers – to buildpatterns. If you know a programming language, you have probablyused Boolean algebra. You write Boolean expressions to control theﬂow of a program, for example, with if-then


key: chroma:document
string_value:
theﬂow of a program, for example, with if-then constructs.•Boolean algebra is the same as propositional logic, which models log-ical reasoning withTrueandFalsestatements. Such logical reason-ing is easy for humans to understand, making the Tsetlin machinetransparent.I n c o m p a r i s o n , s o - c a l l e dblack boxmachine learningmodels are too complicated for human comprehension.


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo1.2 Creating Patterns with AND and NOTThe Tsetlin machine solves pattern recognition problems by buildingif-thenrulesfrom object observations. Each rule has the formifconditionthenclass.Theconditionpart is a placeholder for a Boolean expression that describesap a t t e r ni nt h ed a t a ,t ob el e a r n tb yt h eT s e t l i nm a c h i n e . L e tu sn o wreturn to our vehicle data from Table1.1. As you can see from the table,the


key: chroma:document
string_value:
from Table1.1. As you can see from the table,the conditionFour WheelsandTransports Peoplecharacterises the three cars (object #1, #2, and #3) and one of the planes(object #4).Notice how a Tsetlin machine uses theand-operator to combine severalfeatures. Usingandmeans that all of the features must beTruefor theoverall condition to beTrue.I n t h i s i n s t a n c e , t h e r u l ematchesthe prop-erties of the observed object. On the other hand, if one or more of thefeatures areFalse,t h eo v e r


key: chroma:document
string_value:
one or more of thefeatures areFalse,t h eo v e r a l lc o n d i t i o nb e c o m e sFalsebecause the rule nolonger matches the object’s properties.Theclasspart is a placeholder for a type of object, such asCar.T h i s i sthe type of object that the rule predicts when the rule’s condition matchesthe object’s properties. For example, the ruleifFour WheelsandTransports PeoplethenCarpredictsCarwhen it sees an object withFour WheelsthatTransportsPeople.Negation.The Tsetlin machine also uses


key: chroma:document
string_value:
Tsetlin machine also uses thenot-operator to tell whichfeatures a classdoes nothave. One says thatnotnegatesthe feature. Forinstance, a car does not haveWings.Y o u c a n i n c l u d e t h i s f a c t i n t h e r u l efor recognizing cars by usingnot:ifFour WheelsandTransports Peopleand notWingsthenCar.The negated featurenotWingsmakes the rule able to distinguish betweenclassCarandPlane.C o r r e s p o n d i n g l y ,c l a s sPlanegets its own rule:ifWingsthenPlane.13


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoLiterals.Taken together, features and negated features are calledliterals,as customary in Boolean algebra. The three literals of ourCarexamplerule are:Four Wheels,Transports People,a n dnotWings. Using the wordliteralis convenient because Tsetlin machines treat features and negatedfeatures the same way.Remark.Note that a Tsetlin machine only uses the above kind ofand-expressions. Using pureand-expressions makes it easier to


key: chroma:document
string_value:
Using pureand-expressions makes it easier to understand therules that the Tsetlin machine builds, increasing transparency.1.3 Learning Frequent Patterns withRecognize and Erase FeedbackAT s e t l i nm a c h i n el e a r n sb yo b s e r v i n ge x a m p l e so fo b j e c t so fd i ↵ e r e n tt y p e s .It also gets to know the type of each object. It sees a plane, then a car, andso on. However, the Tsetlin machine perceives the object’s features, notthe object itself. For example, it observes


key: chroma:document
string_value:
notthe object itself. For example, it observes aBluevehicle of typeCarthatTransports People,h a sFour Wheelsand does not haveWings.T h et a s ko fthe Tsetlin machine is then to learn to recognize and distinguish betweenthe di↵erent types of objects. Learning from such labeled examples goesunder the namesupervised learning.AT s e t l i nm a c h i n el e a r n sb ym e m o r i z i n gf e a t u r e s . T h el e a r n i n gi si n -spired by how humans remember. If you observe something multiple


key: chroma:document
string_value:
remember. If you observe something multiple times,you remember it better. However, without observations, forgetting sets inafter some time. Also, general properties tend to stick while ﬁne detailsdisappear. You remember that your neighbor’s car is blue and has fourwheels but may forget the kind of tire. After some time, you may forgetthe color as well. It appears as if everything memorized is in some state offorgetting.Rule MemoryIn traditional computer memory, everything sticks.


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoIn MemoryNot in MemoryFigure 1.3: Traditional memory for the rule:ifFour WheelsandTransports PeopleandBluethenCar.MaximallyMemorized


key: chroma:document
string_value:
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.4: Example of Tsetlin machine memory for the rule:ifFour WheelsandTransports PeopleandBluethenCar.ifFour WheelsandTransports PeopleandBluethenCarin the traditional way. The literalsFour Wheels,Transports People,a n dBluestay in memory until you delete them. Notice that the example rulefor predictingCarincludes the literalBluein its condition. If aCarofanother color appears, the rule will not recognize the object asCar.Y o


key: chroma:document
string_value:
the rule will not recognize the object asCar.Y o uwill discover how Tsetlin machines ﬁx too specialized rules below.A Tsetlin machine simulates forgetting and memorization. Figure1.4shows the Tsetlin machine memory of our example rule. The y-axis rangesfrom 1 to 10 and measures how deeply the memory stores each literal:•Values 1 to 5 stand forForgotten.V a l u e 1 m e a n s m a x i m a l l y f o r g o t t e nwhile value 5 means almost memorized. Literals in this rangedo nottake part in the


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo•Values 6 to 10 meanMemorized.V a l u e 6 s t a n d s f o r l i g h t l y r e t a i n e d ,and value 10 represents maximally memorized. Literals in this rangetake part in the condition of the rule.In Figure1.4,Four WheelsandTransports Peopleare deeply memorized,whileBlueis lightly memorized. The other literals are forgotten to varyingdegrees, measured by their position in memory (from 1 to 5).Remark.Observe how the negated and


key: chroma:document
string_value:
(from 1 to 5).Remark.Observe how the negated and non-negated versions of each fea-ture co-exist in memory. Both versions are candidates for being memo-rized, however, memorizing both gives a contradiction. A vehicle cannotboth haveFour WheelsandnothaveFour Wheelsat the same time. Thatis, ifFour WheelsisTruethennotFour WheelsbecomesFalse,a n dv i c eversa. Accordingly, the conditionFour Wheelsand notFour WheelsisalwaysFalse. As explored below, Tsetlin machines avoid contradictions byforgetting


key: chroma:document
string_value:
machines avoid contradictions byforgetting literal combinations that evaluate toFalse.Learning of a Single RuleThe key to understanding how a complete Tsetlin machine learns is to studyhow a single if-then rule learns by itself. By learning independently, eachrule becomes more self-contained and simpler to understand. As a sidebeneﬁt, independent learning also accommodates parallel processing. Laterin this chapter, you will see how multiple rules coordinate indirectly withoutbeing aware of each


key: chroma:document
string_value:
coordinate indirectly withoutbeing aware of each other, providing a simple and ecient algorithm fordissecting the complete dataset.Rule Initialization.Ar u l es t a r t su pw i t ha l lt h el i t e r a l si nm e m o r yposition 5. That is, the literals are about to beMemorizedbut currentlyForgotten. Figure1.5shows the initial memory for your example rule. Notethat you can initialize the rule any way you like. The initialization doesnot a↵ect the end result of learning because the Tsetlin


key: chroma:document
string_value:
the end result of learning because the Tsetlin machine is self-correcting. This is in contrast to so-called neural network learning, whichis more sensitive to initialisation.Algorithm – Learning Steps for Single Rule.Let us ﬁrst look atwhat happens when a rule faces an object of its own class. That would16


key: chroma:document
string_value:
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.5: Initialization of example rule for predicting classCar.be the case when our example rule for recognizingCarobserves a vehiclelabeled asCar.L e a r n i n gt h e np r o c e e d si nt h r e es t e p s :1.Rule Evaluation.Evaluate the condition part of the rule by assess-ing the object’s literals.2.Recognize Feedback.Perform this step if the condition part isTrue.a)Memorize the literals that areTruefor the object. You


key: chroma:document
string_value:
the literals that areTruefor the object. You memorizeal i t e r a lb yi n c r e m e n t i n gi t sp o s i t i o ni nm e m o r yu n l e s sa l r e a d yMaximally Memorizedin position 10.b)Forget theFalseliterals by pushing them towards beingMaxi-mally Forgotten.Y o u f o r g e t a l i t e r a l b y d e c r e m e n t i n g i t s p o s i t i o nin memory unless alreadyMaximally Forgottenin position 1.The name of this kind of feedback is Recognize Feedback because itmakes the condition part of the


key: chroma:document
string_value:
because itmakes the condition part of the rule mimic the observed object.3.Erase Feedback.Perform this step if the condition part isFalse.Then forget all of the literals, independently of their truth values.Again, you forget literals by pushing them towards beingMaximally17


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoForgotten. The name of this type of feedback is Erase Feedback be-cause it erases the condition part of the rule.2Randomization.Because coincidences happen and events sometimes oc-cur by chance, learning should be ﬂexible. A simple way to achieve suchﬂexibility is randomization. Then no coincidence or chance event can makethe learning get stuck. The Tsetlin machine, therefore, randomizes theincrements and decrements. Before


key: chroma:document
string_value:
randomizes theincrements and decrements. Before performing an increment, you draw arandom ﬂoating-point value between 0.0a n d1.0. Skip the increment if thevalue is above 0.5. The value 0.5g e t st h en a m eMemorize Value.Y o ua l s orandomize the decrements. Again, you draw a random ﬂoating-point valuebefore decrementing. A value above 0.5l e a v e so u tt h ed e c r e m e n t . T h i ss e c -ond 0.5v a l u ei sc o i n e dForget Value. Note that the randomized updatingfurther diversiﬁes the


key: chroma:document
string_value:
the randomized updatingfurther diversiﬁes the rules and boosts exploration.Memorization and Forgetting Speed.The Memorize Value decideshow quickly the rule memorizes literals. Conversely, the Forget Value de-cides how quickly the rule forgets in the absence of observations. If youincrease the Memorize Value and reduce the Forget Value, the rule willremember literals longer. Remembering literals longer allows the rule toretain more details. In the extreme, a Memorize Value of 1.0a n daF o r g e


key: chroma:document
string_value:
extreme, a Memorize Value of 1.0a n daF o r g e tValue of 0.0m a k et h er u l em e m o r i z eevery Trueliteral of one single object.Conversely, Memorize Value 0.0a n dF o r g e tV a l u e1.0r e n d e rt h er u l ei n c a -pable of memorizing anything at all. It is usual to keep the relationship:Forget Value + Memorize Value = 1.0. The reason is that you then canobtain one from the other, and therefore you only need to specify one ofthem. Alternatively, you can ﬁx the Memorize Value to


key: chroma:document
string_value:
Alternatively, you can ﬁx the Memorize Value to 1.0.ExamplesUpdating of Memory – Example 1.The ﬁrst example traverses thedata from Table1.1, using Memorize Value 0.1a n dF o r g e tV a l u e0.9. Con-sider a rule for predictingCar. After initialization, the rule ﬁrst faces vehi-cle #1 in the table. The vehicle is of typeCar,h e n c ee i t h e rR e c o g n i z eo r2In the research literature, Recognize and Erase Feedback are typically called TypeIa and Type Ib Feedback. In this book, I use the


key: chroma:document
string_value:
and Type Ib Feedback. In this book, I use the former names to make it easier toremember the roles of the feedback.18


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoMaximallyMemorized
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Memorize Value 0.1Forget Value 0.9Skipped Memorize


key: chroma:document
string_value:
Figure 1.6: Memory updating for newly initialized rule when facing aBluecar thatTransports Peopleand doesnothaveWings.Erase Feedback applies. It has ﬁveTrueliterals:Four Wheels,TransportsPeople,notWings,notYellow, andBlue.T h e r e m a i n i n g l i t e r a l s a r eFalse.Learning proceeds as follows:•All the literals of the rule startForgotten, as shown in Figure1.5.S o ,the rule’s condition is empty. An empty condition does not specifyany literal requirements so it is alwaysTrue. Accordingly,


key: chroma:document
string_value:
requirements so it is alwaysTrue. Accordingly, you proceedwith Recognize Feedback.•The solid arrows in Figure1.6visualize how Recognize Feedback in-crements theTrueliterals randomly. Because of the small Mem-orize Value of 0.1, only a few increments will happen, hence theshort arrows. Recall how each increment is skipped whenever a ran-dom ﬂoating-point value between 0.0a n d1.0s u r p a s s e st h eM e m o r i z eValue. For the sake of the example, execute the increments


key: chroma:document
string_value:
the sake of the example, execute the increments forFourWheels,Transports People,a n dBlue.T h e nnotWingsandnotYellowstay in place, signiﬁed by the greyed-out arrows in the ﬁgure.•Conversely, the dotted arrows in the ﬁgure visualize the random decre-ments for theFalseliterals. These arrows are visualized as beinglonger because of the higher Forget Value of 0.9. Again, you drawa random ﬂoating-point value per pinpointed decrement. However,19


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoMaximallyMemorized
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Forget Value 0.9Skipped Forget


key: chroma:document
string_value:
Figure 1.7: Memory updating for rule ‘ifFour WheelsandTransports PeopleandBluethenCar’w h e nf a c i n gaYellowcar.now a value equal to or below 0.9e x e c u t e st h ed e c r e m e n t . F o rt h i sparticular example, assume that all of the decrements happen.The updated rule then becomes:ifFour WheelsandTransports PeopleandBluethenCar.Notice how the rule now is more similar to the object it observed.Updating of Memory – Example 2.The updated rule then faces ve-hicle #2 in Table1.1–aYellowcar.


key: chroma:document
string_value:
then faces ve-hicle #2 in Table1.1–aYellowcar. Figure1.7shows how to update therule. This time, the rule’s condition does not match the car’s literals be-cause of the mismatching color. Accordingly, you use Erase Feedback. Asvisualized, Erase Feedback randomly decrements the memory position of allof the literals. Execute these decrements, except forFour WheelsandnotWingsto simulate the randomization (indicated by the greyed out arrows).The new rule becomes:ifFour WheelsthenCar.20


key: chroma:document
string_value:
Figure 1.8: Memory updating for rule ‘ifFour WheelsthenCar’w h e nfacing aYellowcar.Updating of Memory – Example 3.The rule now faces vehicle #3,which also happens to beYellow.T h i s t i m e t h e r u l e ’ s c o n d i t i o n m a t c h e s t h evehicle’s features becauseBlueis no longer included. Figure1.8depicts howthe resulting Recognize Feedback acts on the literals. First, it incrementstheTrueliteralsFour Wheels,Transports People,notWings,Yellow,a n dnotBlue. Again, the increments are


key: chroma:document
string_value:
n dnotBlue. Again, the increments are random, based on the Memorize Value.To simulate randomization, execute only the increments forFour Wheels,Transports People,a n dYellow.T h e ﬁ g u r e a l s o s h o w s t h e d e c r e m e n t i n gof theFalseliterals:notFour Wheels,notTransports People,Wings,notYellow,a n dBlue.T o s i m u l a t e r a n d o m i z a t i o n w i t h F o r g e t V a l u e 0.9,perform all of these decrements. The resulting rule after observing vehicle#3 becomes:ifFour


key: chroma:document
string_value:
rule after observing vehicle#3 becomes:ifFour WheelsandTransports PeoplethenCar.Figure1.9shows the memory of the updated rule. Observe how the rulenow memorizesFour WheelsandTransports Peoplemore strongly, whilegetting ready to memorizenotWings. Also observe how the color literalsYellow,notYellow,Blue,a n dnotBlueare gradually forgotten. Varyingfrom car to car, the frequency of each color is too low to be memorized withMemorize Value 0.1a n dF o r g e tV a l u e0.9. Finally, notice how the


key: chroma:document
string_value:
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.9: Memory for rule ‘ifFour WheelsandTransports PeoplethenCar’a f t e ru p d a t i n g .Menopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 31.·• ··•··· •3.·• ···•·· •5.·· ••···· •Table 1.5: ThreeRecurrencepatients with nine Boolean features. Eachentry takes the valueTrue(•)o rFalse(·).Menopause Inv-nodes Deg-malig#¬lt40¬ge40¬premeno¬0-2¬3-5¬6-8¬1¬2¬31.•· ••·•••·3.•· •••·••·5.•• ··••••·Table 1.6:


key: chroma:document
string_value:
••·•••·3.•· •••·••·5.•• ··••••·Table 1.6: ThreeRecurrencepatients with nine negated Boolean features,signiﬁed with ‘¬’. Each entry takes the valueTrue(•)o rFalse(·).notFour Wheels,notTransports People,a n dWingsare quickly forgotten,not representing any car observed thus far.High-Frequency Breast Cancer Pattern.Let us return to the prog-nosis of breast cancer recurrence, examining the e↵ects of Memorize Value22


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo0.2a n dF o r g e tV a l u e0.8. Consider a rule for predictingRecurrence.T h erule learns its condition by traversing theRecurrencepatients in the data.There is no need for you to consider theNon-Recurrencepatients at thispoint because they do not inﬂuence Recognize and Erase Feedback. Table1.5compiles the features of theRecurrencepatients for ease of reference.The Tsetlin machine also uses the negated features to make rules.


key: chroma:document
string_value:
also uses the negated features to make rules. Table1.6contains the truth values of these. You get the complete set of literalsthat the Tsetlin machine uses by combining the two tables.With Memorize Value 0.2 and Forget Value 0.8,Trueliterals that matchall threeRecurrencepatients increment 3⇥0.2=0.6t i m e so na v e r a g e .They never decrement because everyRecurrencepatient matches. In con-trast, consider the literals that areFalsefor one of the patients, such asMenopause ge40. On average,


key: chroma:document
string_value:
the patients, such asMenopause ge40. On average, this literal increments 2⇥0.2=0.4t i m e s ,while it decrements 1⇥0.8=0.8t i m e s . S o ,t h el i t e r a ld e c r e m e n t sm o r ethan it increments, moving towards beingMaximally Forgotten.T h e o n l ypossible rule then becomes:ifDeg-malig 3and notMenopause lt40and notDeg-malig 1and notDeg-malig 2thenRecurrence.Note that sinceDeg-malig 3impliesnotDeg-malig 1andnotDeg-malig 2,the rule simpliﬁes to:ifDeg-malig 3and notMenopause


key: chroma:document
string_value:
rule simpliﬁes to:ifDeg-malig 3and notMenopause lt40thenRecurrence.Mid-Frequency Breast Cancer Pattern.You capture less frequentpatterns by diminishing forgetting and boosting memorization. For in-stance, let us see how Memorize Value 0.5a n dF o r g e tV a l u e0.5a ↵ e c tl e a r n -ing ofMenopause ge40.O n a v e r a g e , t h i s l i t e r a l n o w i n c r e m e n t s 2⇥0.5=1.0times and decrements 1⇥0.5=0.5 times. Accordingly, the new Memorizeand Forget Values encourage rules that match two


key: chroma:document
string_value:
Forget Values encourage rules that match two out of treeRecurrencepatients. Then you, for instance, could get the ruleifDeg-malig 3andMenopause ge40and notMenopause lt40and notMenopause premenoand notDeg-malig 1and notDeg-malig 2then23


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoRecurrence,which simpliﬁes toifDeg-malig 3andMenopause ge40thenRecurrence.You could also get another rule than the one above when other regularitiesoccur two out of three times. However, which one you get does not mat-ter. You will later use multiple rules that together capture many di↵erentpatterns in the data.1.4 Increasing Discrimination Power withReject FeedbackFrequent patterns that describe what is typical can be powerful


key: chroma:document
string_value:
that describe what is typical can be powerful in their ownright. They express how the world appears, for instance, that the worldcontains vehicles that have four wheels and transport people. However,sometimes you want to classify objects into predeﬁned classes such asCarandPlane.T h e n i t i s c r u c i a l t o m a k e s u r e t h a t t h e f r e q u e n t p a t t e r n sdistinguish between them. For instance, the condition of ruleifFour WheelsandTransports PeoplethenCarcharacterizes both cars


key: chroma:document
string_value:
PeoplethenCarcharacterizes both cars and planes, so it cannot see the di↵erence betweenthe two.You make a rule more discriminative with the following third and ﬁnalfeedback type.Algorithm – Increasing Discrimination Power.Ar u l ei n c r e a s e si t sdiscrimination power when it faces an object of a class di↵erent from itsown. That would be the case when our example rule for recognizingCarobserves a vehicle labeledPlane.L e a r n i n gt h e ns k i p sR e c o g n i z ea n dE r a s eFeedback,


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoMaximallyMemorized
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Memorize Value 1.0


key: chroma:document
string_value:
Figure 1.10: Memory updating for rule ‘ifFour WheelsandTransports PeoplethenCar’w h e nf a c i n gaBlueplane withFour WheelsthatTransports People.4.Reject Feedback.Check if the condition part of the rule isTrueby assessing the object’s literals. If the condition part isTrue,t h e nmemorize allForgottenliterals that areFalsefor the object. Again,you memorize a literal by incrementing its position in memory. How-ever, this time there is no randomization – the increment is alwaysperformed. In


key: chroma:document
string_value:
– the increment is alwaysperformed. In e↵ect, the memorization pushes the literals that arein memory position 1 to 5, and at the same time areFalse,t o w a r d sbeingMemorized.The above learning step has the name Reject Feedback because it makesthe rule reject the observed object by memorizing theFalseliterals.Example with Vehicle Data.Figure1.10shows how our example rulefor predictingCarupdates when observing vehicle #4 in Table1.1–aPlanewithFour WheelsthatTransports People,i


key: chroma:document
string_value:
WheelsthatTransports People,i sBlue(notYellow),and hasWings.L o o k a t h o w a l l t h e l i t e r a l s t h a t a r eFalseand on theForgottenside all move towards beingMemorized.T h e s e a r e :notFourWheels,notTransports People,notWings,Yellow,a n dnotBlue.I nparticular,notWingsgoes fromForgottentoMemorizedby switching frommemory position 5 to memory position 6. The other literals are allTrue,25


key: chroma:document
string_value:
MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.11: Memory for rule ‘ifFour WheelsandTransports PeopleandnotWingsthenCar’a f t e ru p d a t i n g .Menopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 32.•· ·•···· •4.·• ·•···• ·6.·· ••··•· ·Table 1.7: ThreeNon-Recurrencepatients with nine Boolean features. Eachentry takes the valueTrue(•)o rFalse(·).hence Reject Feedback leaves them alone. Figure1.11depicts the updatedmemory. The rule then becomesifFour


key: chroma:document
string_value:
the updatedmemory. The rule then becomesifFour WheelsandTransports Peopleand notWingsthenCar.At this point, the condition isFalsefor the observed plane because thecondition now contains the literalnotWings.T h en e x tt i m et h er u l ef a c e sap l a n e ,i tn ol o n g e rn e e d st ou p d a t eb e c a u s ei t sc o n d i t i o nw i l lt h e nb eFalse.I tc a nn o wd i s t i n g u i s hb e t w e e nCarandPlane.Example with Breast Cancer Data.Consider a rule that votes forRecurrenceif the


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoMenopause Inv-nodes Deg-malig#¬lt40¬ge40¬premeno¬0-2¬3-5¬6-8¬1¬2¬32.·• •·••••·4.•· •·•••·•6.•• ··••·••Table 1.8: ThreeNon-Recurrencepatients with nine negated Boolean fea-tures, signiﬁed with ‘¬’. Each entry takes the valueTrue(•)o rFalse(·).ifDeg-malig 3thenRecurrence.Now, take a look at the literals of theNon-Recurrencepatients in Table1.7and Table1.8.Deg-malig 3isTruefor patient #2, despite being aNon-Recurrencepatient. When


key: chroma:document
string_value:
#2, despite being aNon-Recurrencepatient. When the above rule faces patient #2, it thus votesfor the wrong class, i.e.,Recurrence.R e j e c tF e e d b a c kc o r r e c t st h i se r r o rb ymemorizing theForgottenliterals that areFalse.F o r p a t i e n t # 2 ,t h e s e a r e :notMenopause lt40,Menopause ge40,Menopause premeno,notInv-nodes 0-2,Inv-nodes 3-5,Inv-nodes 6-8,Deg-malig 1,Deg-malig 2,a n dnotDeg-malig 3.Facing patient #2 several times, Reject Feedback eventually pushes one


key: chroma:document
string_value:
times, Reject Feedback eventually pushes one oftheForgotten Falseliterals to memory position 6, making itMemorized.From all of the literals listed above, Recognize and Erase Feedback makesure that the rule eventually memorizes literalnotMenopause lt40.T h eresulting rule becomesifDeg-malign 3and notMenopause lt40thenRecurrence.From this point, Reject Feedback no longer applies, and Recognize andErase Feedback operate alone.27


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo
ObservationRule MatchingVotes
FeaturesMajority VoteOutputPlane


key: chroma:document
string_value:
Figure 1.12: A majority vote among the rules decides the output. Here,Planeobtains more votes thanCar(1 vs. 0 votes) and is thus output bythe Tsetlin machine.1.5 Putting the Pieces Together – HowSeveral Rules CoordinateTsetlin machines use multiple rules that interact. You will now explore howthis interaction takes place by dissecting the toy data on vehicles and thereal-life data on breast cancer.Classiﬁcation Procedure.The Tsetlin machine classiﬁes input by vot-ing. That is, a single rule


key: chroma:document
string_value:
input by vot-ing. That is, a single rule does not get the ﬁnal say on the class by itself.Instead, it casts a vote for its class. To classify, you simply count how manyrules vote for each class. The Tsetlin machine then outputs the class withthe most votes.In other words, a majority vote among the rules decidesthe output.28


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo#Menop.Inv-nodesDeg-maligRecur.R1R2R3v1.ge403-53yes••·+22.lt400-23no··•13.ge406-83yes••·+24.ge400-22no··•15.premeno0-23yes•••+16.premeno0-21no··•1Table 1.9: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.C o l u m n sR1,R2,a n dR3list the truthvalue of each rule while columnvcontains the resulting vote sums.Demonstration of Vehicle Classiﬁcation.Figure1.12illustrates theclassiﬁcation


key: chroma:document
string_value:
theclassiﬁcation of vehicle #4 in Table1.1–aBlue PlanethatTransportsPeopleand hasFour WheelsandWings.T h e d e p i c t e d T s e t l i n m a c h i n eemploys two rules:R1ifFour WheelsandTransports Peopleand notWingsthenCar.R2ifWingsthenPlane.Rule R1 does not match the vehicle because it speciﬁesnotWings. Hence,Cargets no votes. Rule R2, on the other hand, matches. So,Planegets avote and wins the majority vote. The Tsetlin machine then outputsPlane.Demonstration of Breast Cancer Recurrence


key: chroma:document
string_value:
of Breast Cancer Recurrence Prediction.Recallthe breast cancer dataset from Table1.4. Here are three rules that demon-strate the di↵erent ways rules can cooperate:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3and notMenopause lt40thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.Notice that rules R1 and R2 for predictingRecurrenceare duplicates. Byvoting together, R1 and R2 can win majority votes that otherwise wouldbe tied, demonstrated below.29


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoColumnsR1,R2,a n dR3in Table1.9contain the truth value of eachrule’s condition, per patient from Table1.4. Again, ‘•’m e a n sTrueand ‘·’meansFalse.F o ry o u rc o n v e n i e n c e ,c o l u m nvcontains the outcome of themajority vote. Each value in the column is simply the sum of matchingrules that votes forRecurrenceminus the sum of matching rules that voteforNon-Recurrence. Note that to ﬁt all the information in one table,


key: chroma:document
string_value:
Note that to ﬁt all the information in one table, thetable only listsTrueliterals. All non-listed literals areFalse.Breast Cancer Classiﬁcation Example 1.Now consider patient #1in Table1.9.R u l e sR 1a n dR 2m a t c ht h i sp a t i e n t ,w h i l er u l eR 3d o e sn o t .This means that classRecurrencegets two votes for patient #1. The classNon-Recurrencegets zero votes. Accordingly, the vote sum becomes +2andRecurrencewins the majority vote. This is the class that the Tsetlinmachine predicts


key: chroma:document
string_value:
is the class that the Tsetlinmachine predicts for patient #1. The prediction matches the patient’s labelin columnRecur.Breast Cancer Classiﬁcation Example 2.Sometimes a rule is not suf-ﬁciently precise by itself. Then the majority vote arbitrates disagreement.Patient #5 matches all three rules. While the correct class isRecurrence,rule R3 still votes forNon-Recurrence.T h e r e a s o n i s t h a t p a t i e n t # 5 h a sTrue Inv-nodes 0-2.B e c a u s eMenopause lt40isFalse(not listed)


key: chroma:document
string_value:
e c a u s eMenopause lt40isFalse(not listed) andDeg-malig 3isTrue, rules R1 and R2 matches as well. As a result, the votingsum becomes +1.Recurrencethen wins the majority vote.Remark.Note how the last example uses detailed rules with several lit-erals. At the same time, the output is decided by summation. One cansay that the Tsetlin machine integrates rule-based and summation-baseddecision making. Sometimes it is useful to describe objects in detail withrules, and sometimes it is useful to add


key: chroma:document
string_value:
withrules, and sometimes it is useful to add up evidence from multiple rulesusing summation. For comparison, so-calleddecision treesare rule-based,while so-calledlogistic regressionis summation-based.1.6 Learning to CoordinateStarting Up.The Tsetlin machine uses multiple rules. You start up byassembling how many rules you like. Then you assign one class to each ofthem. In the breast cancer example, you used twoRecurrencerules and one30


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoNon-Recurrencerule, for instance. The Tsetlin machine then initialises eachrule by placing all the literals in memory position 5, i.e., barelyForgotten.From there on, each rule learns its condition from observing ob jects one byone, together with each object’s class. You already know how a single rulelearns. You will now discover how several rules coordinate the learning,ﬁlling di↵erent roles in the classiﬁcation of objects.Vote


key: chroma:document
string_value:
roles in the classiﬁcation of objects.Vote Margin.The Tsetlin machine coordinates learning of multiple ruleswith aVote Margin.A V o t e M a r g i n i s a n i n t e g e r n u m b e r t h a t y o u u s e t ocreate a margin between the winning and the losing class. If you set theVote Margin to 2, you tell the Tsetlin machine that the winning class musthaveexactlytwo more votes than the losing class. It must then attempt tofulﬁll this requirement for every object it observes during learning. In


key: chroma:document
string_value:
for every object it observes during learning. In thismanner, the Tsetlin machine creates complementary rules. In the following,you can use a Vote Margin of 2 for illustration purposes.Complete Learning Algorithm.The Tsetlin machine learns comple-mentary rules as follows:1.Observe a new object and its class. The observation consists of theobject’s literals (see Section1.1).2.Evaluate each rule’s condition using the truth values of the literals(see Section1.2).3.Calculate the vote sum (see


key: chroma:document
string_value:
Section1.2).3.Calculate the vote sum (see Section1.5):a)Identify the rules whose condition isTrue. Use these for voting.b)Add up the votes in favour of the object’s class.c)Subtract the votes in favour of the other class.d)Refer to the summation outcome asv.e)Setvto the Vote Margin2if larger than2and to2if smallerthan2.4.Go through each rule and give it feedback ifRand()2v4, drawnrandomly per rule:31


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo#Menop.Inv-nodesDeg-maligRecur.R1R2R3v1.ge403-53yes••·+22.lt400-23no·••+03.ge406-83yes••·+24.ge400-22no··•15.premeno0-23yes•••+16.premeno0-21no··•1Table 1.10: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.a)Give the rule Recognize or Erase Feedback if the rule belongs tothe object’s class (see Section1.3).b)Give the rule Reject Feedback if it belongs to another class


key: chroma:document
string_value:
Reject Feedback if it belongs to another class (seeSection1.4).5.Goto 1.Above,Rand() provides a random ﬂoating-point value between 0.0a n d1.0. Observe how the random updating of rules according toRand()2v4is crucial for coordination. If you are far from achieving the Vote Margin forap a r t i c u l a ro b j e c t ,y o uu p d a t et h er u l e sm o r ea g g r e s s i v e l y . I nt h ee x t r e m e ,if the vote sum is2o rs m a l l e r ,y o uu p d a t ea l lo ft h er u l e s . T h i si sb e c


key: chroma:document
string_value:
p d a t ea l lo ft h er u l e s . T h i si sb e c a u s e2v4then becomes2(2)4,w h i c hi se q u a lt o1.0.If the vote sum is zero, you randomly update each rule with probability0.5. This happens because2v4then becomes204,w h i c hi se q u a lt o0.5.If you are close to 2, updating calms down. Indeed, if the voting sum is2 or larger, you update none of the rules. This is because2v4becomes224,which is equal to 0.0.Because of the above random updating of the rules, they individuallyand


key: chroma:document
string_value:
updating of the rules, they individuallyand gradually assign themselves to classify the di↵erent kinds of objectsthey face. When doing so, they prioritize objects that are further awayfrom the Vote Margin. In this manner, you achieve a resource allocatione↵ect.32


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoPrioritization Example.Consider a situation where you have the fol-lowing three rules:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.Table1.10shows how each rule votes per patient. For convenience, col-umnvagain includes the resulting vote sums. Observe how the vote sumsof patients #1, #3, #4, #5, and #6 give the correct classiﬁcation. How-ever, patients #4,


key: chroma:document
string_value:
the correct classiﬁcation. How-ever, patients #4, #5, and #6 do not reach the Vote Margin of 2. As aresult, every time these patients appear you update each rule randomly ifRand()214, that is, one fourth of the time. Accordingly, patients #4,#5, and #6 are prioritized over patients #1 and #3. The vote sum ofpatient #2 gives the wrong classiﬁcation and is even further away fromclosing the margin. Every time this patient appears, you update each rulewhenRand() is less than204,w h i c hi sh a l


key: chroma:document
string_value:
rulewhenRand() is less than204,w h i c hi sh a l fo ft h et i m e . S o ,p a t i e n t# 2i sprioritized over all the other patients.Coordination Example.Complementary dynamics drive the coordina-tion of rules. First of all, only rules that vote for the wrong class are a↵ectedby Reject Feedback, shielding the rules that cast their votes correctly. Forinstance, rule R2 votes for the wrong class when facing patient #2, how-ever, rule R1 does not. Accordingly, only rule R2 receives the feedback.


key: chroma:document
string_value:
Accordingly, only rule R2 receives the feedback. Inbrief, Reject Feedback singles out exactly those rules that can reduce thevote sum deviance.Further, a Tsetlin machine memorizes precise and frequent rules quickerand more deeply. A rule memorizes the fastest when it matches all theobjects of its class and never casts a wrong vote. Rule R1 is such an idealrule. In e↵ect, rule R1 receives the maximum amount of Recognize Feedbackand never experiences Erase or Reject Feedback. One can say that the


key: chroma:document
string_value:
Erase or Reject Feedback. One can say that the moreuseful a rule is, the quicker and more deeply it is memorized. So, rule R1ends up deeply memorizingDeg-malign 3andnotMenopause lt40.33


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er GranmoOn the other side of the spectrum, infrequently matching rules thatvote on the wrong class forget quicker and memorize slower. It is Recognizeand Erase Feedback that ensure forgetting of rare patterns. Recognize andErase Feedback also slow down memorization when the matching frequencydrops. Simultaneously, when voting on the wrong class, Reject FeedbackintroducesFalseliterals into the rule’s condition, disrupting learning. RuleR2


key: chroma:document
string_value:
the rule’s condition, disrupting learning. RuleR2 learns, for instance, slower than R1 because the Reject Feedback pushestheFalseliterals of patient #5 upwards in the rule’s memory, interruptingthe memorization. Slower learning leads to more lightly memorized literals.This is helpful because a rule with lightly memorized literals can aptly shiftto more powerful patterns when found.Finally, observe that patient #5 will relentlessly trigger Recognize andErase Feedback because the voting sum never


key: chroma:document
string_value:
andErase Feedback because the voting sum never becomes +2. However, thisfeedback only makes rule R1 more robust and helps rule R2 become moreprecise.Accordingly, the Tsetlin machine quickly settles in the goal state:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3and notMenopause lt40thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.1.7 SummaryHere are the key points from this chapter:•The Tsetlin machine recognizes objects by coordinating multiple if-then rules.•Each rule


key: chroma:document
string_value:
by coordinating multiple if-then rules.•Each rule belongs to a class and learns by itself to recognize objectsof that class.•The condition part of the rule consists ofand-a n dnot-operators.These form a Boolean expression that describes a pattern. The pat-tern is frequent and unique for the rule’s class.34


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo•Am a j o r i t yv o t ea m o n gt h er u l e sd e c i d e st h eo u t p u t . T h eT s e t l i nm a -chine thus uniﬁes rule-based and summation-based decision making.•There are three kinds of feedback that help produce rules that are bothfrequent and unique. Recognize and Erase Feedback produce frequentpatterns, while Reject Feedback makes the pattern more unique forthe class.•A Vote Margin helps coordinate multiple rules. In


key: chroma:document
string_value:
Vote Margin helps coordinate multiple rules. In brief, each ruleacquires a suitable role by itself, supporting the other rules.•Tsetlin machines thus unify three powerful strategies for learning pat-terns:1.Frequent pattern mining withRecognizeand Erase Feedback.2.Pattern discrimination withReject Feedback.3.Data dissection by means of theVote Margin.•Being based on pureand-rules in Boolean algebra, Tsetlin machinerules are transparent and easy to comprehend by humans. This is incontrast to


key: chroma:document
string_value:
to comprehend by humans. This is incontrast to so-called black box machine learning models.•And-rules in Boolean algebra support hardware-near bitwise opera-tors that dramatically increase inference speed. Further, the resultingsmall energy- and memory footprint is ideal for edge computing andInternet of Things (IoT).1.8 Exercises1.What is the structure of a Tsetlin machine rule?2.How are data features prepared as input to a Tsetlin machine?3.What are the three learning steps of a Tsetlin


key: chroma:document
string_value:
are the three learning steps of a Tsetlin machine?4.When and why areFalseandForgottenliterals memorized?5.What part of Tsetlin machine learning coordinates the learning ofmultiple rules and how is the coordination done?35


key: chroma:document
string_value:
An Introduction to Tsetlin Machines (7/11/22) Ole-Christo↵er Granmo6.When is Recognize Feedback triggered and how is this feedback re-lated to the Tsetlin machine learning steps?7.When is Reject Feedback triggered and how is this feedback relatedto the Tsetlin machine learning steps?8.What happens if we set the Forget Value closer to 0.0?9.Implement learning of a single rule with Recognize and Erase Feed-back, using the dataset in Table1.1.10.Implement learning of a single rule combining


key: chroma:document
string_value:
learning of a single rule combining Recognize, Erase, andReject Feedback, using the dataset in Table1.4.11.Implement learning of three rules using Vote Margin 1 ﬁrst and thenVote Margin 2. Investigate any di↵erences in the rules produced. Usethe dataset in Table1.4.


key: source
string_value:
source_documents\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf


key: chroma:document
string_value:
Received November 26, 2020, accepted December 18, 2020, date of publication January 6, 2021, date of current version January 14, 2021.
Digital Object Identifier 10.1 109/ACCESS.2021.3049569
Extending the Tsetlin Machine With
Integer-Weighted Clauses for Increased
Interpretability
K. DARSHANA ABEYRATHNA
 , OLE-CHRISTOFFER GRANMO
 ,
AND MORTEN GOODWIN
Centre for Artiﬁcial Intelligence Research, University of Agder, 4879 Grimstad, Norway


key: chroma:document
string_value:
Corresponding author: K. Darshana Abeyrathna (darshana.abeyrathna@uia.no)
ABSTRACT Building models that are both interpretable and accurate is an unresolved challenge for
many pattern recognition problems. In general, rule-based and linear models lack accuracy, while deep
learning interpretability is based on rough approximations of the underlying inference. However, recently, the
rule-based Tsetlin Machines (TMs) have obtained competitive performance in terms of accuracy, memory


key: chroma:document
string_value:
footprint, and inference speed on diverse benchmarks (image classiﬁcation, regression, natural language
understanding, and game-playing). TMs construct rules using human-interpretable conjunctive clauses in
propositional logic. These, in turn, are combined linearly to solve complex pattern recognition tasks. This
paper addresses the accuracy-interpretability challenge in machine learning by introducing a TM with integer


key: chroma:document
string_value:
weighted clauses – the Integer Weighted TM (IWTM). The intent is to increase TM interpretability by
reducing the number of clauses required for competitive performance. The IWTM achieves this by weighting
the clauses so that a single clause can replace multiple duplicates. Since each TM clause is formed adaptively
by a Tsetlin Automata (TA) team, identifying effective weights becomes a challenging online learning


key: chroma:document
string_value:
problem. We solve this problem by extending each team of TA with another kind of automaton: the stochastic
searching on the line (SSL) automaton. We evaluate the performance of the new scheme empirically using
ﬁve datasets, along with a study of interpretability. On average, IWTM uses 6.5 times fewer literals than
the vanilla TM and 120 times fewer literals than a TM with real-valued weights. Furthermore, in terms of


key: chroma:document
string_value:
average memory usage and F1-Score, IWTM outperforms simple Multi-Layered Artiﬁcial Neural Networks,
Decision Trees, Support Vector Machines, K-Nearest Neighbor, Random Forest, Gradient Boosted Trees
(XGBoost), Explainable Boosting Machines (EBMs), as well as the standard and real-value weighted TMs.
IWTM ﬁnally outperforms Neural Additive Models on Fraud Detection and StructureBoost on CA-58 in
terms of Area Under Curve, while performing competitively on COMPAS.


key: chroma:document
string_value:
INDEX TERMS Tsetlin machine, integer-weighted Tsetlin machine, interpretable AI, interpretable machine
learning, XAI, rule-based learning, decision support system.
I. INTRODUCTION
Interpretable Machine Learning refers to machine learning
models that obtain transparency by providing the reasons
behind their output. Linear Regression, Logistic Regression,
Decision Trees, and Decision Rules are traditional inter-
pretable machine learning approaches. However, as discussed


key: chroma:document
string_value:
in [1], the degree of interpretability of these algorithms vary.
More importantly, such methods struggle with obtaining high
The associate editor coordinating the review of this manuscript and
approving it for publication was Pengcheng Liu
 .accuracy for complex problems, especially in comparison to
deep learning. On the other hand, deep learning inference
cannot easily be interpreted [2] and is thus less suitable for
high-stakes domains such as credit-scoring [3], [4], medicine


key: chroma:document
string_value:
[5], [6], bioinformatics [7], [8], churn prediction [9], [10]
healthcare, and criminal justice [11]. Therefore, developing
machine learning algorithms capable of achieving a better
trade-off between interpretability and accuracy is of signiﬁ-
cant importance and continues to be an active area of research.
One of the ways to tackle the above-stated research prob-
lem is explaining the deep learning inference. Different


key: chroma:document
string_value:
VOLUME 9, 2021This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/8233


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
approaches have been proposed for local interpretability, i.e.,
explaining individual predictions [12]. However, they fail to
provide clear explanations of model behavior globally [13].
Recently, Agarwal et al. [11] proposed a novel deep learning
approach that belongs to the family of Neural Additive Mod-
els (NAMs). Even though NAMs are inherently interpretable,


key: chroma:document
string_value:
they are still surpassed by regular deep learning algorithms
when it comes to accuracy [11].
Interpretable linear and rule-based methods can sometimes
offer a better trade-off between interpretability and accuracy.
Learning propositional formulae to represent data patterns
has a long history, with association rule learning [14] being
one well-known approach, which has been used to predict
sequential events [15]. Other examples include the work of


key: chroma:document
string_value:
Feldman on the hardness of learning formulae in Disjunctive
Normal Form (DNF) [16] and Probably Approximately Cor-
rect (PAC) learning, which has provided fundamental insight
into machine learning as well as a framework for learning for-
mulae in DNF [17]. Approximate Bayesian approaches have
recently been introduced to provide more robust learning of
rules [18], [19]. Furthermore, hybrid Logistic Circuits have
had success in image classiﬁcation [20]. Logical operators in


key: chroma:document
string_value:
one layer of the logistic circuit are wired to logical operators
in the next layer, and the whole system can be represented
as a logistic regression function. This approach uses local
search to build a Bayesian model that captures the logical
expression, and learns to classify by employing stochastic
gradient descent. Yet, in general, rule-based machine learning
scales poorly and is prone to noise. Indeed, for data-rich
problems, in particular those involving natural language and


key: chroma:document
string_value:
sensory inputs, rule-based machine learning is inferior to
deep learning.
Tsetlin Machines (TMs) are entirely based on logical oper-
ators and summation, founded on TA-based bandit learning
[21]–[27]. Despite being rule-based, TMs have obtained com-
petitive performance in terms of accuracy, memory footprint,
and inference speed on diverse benchmarks, including image
classiﬁcation, regression, natural language understanding,
and game-playing. Employing a team of TA [28], a TM learns


key: chroma:document
string_value:
a linear combination of conjunctive clauses in propositional
logic, producing decision rules similar to the branches in
a decision tree (e.g., ifXsatisﬁes condition A and not
condition B then Y=1) [23].
A. RECENT PROGRESS ON TMS
Recent research on TMs reports several distinct TM prop-
erties. The TM performs competitively on several classic
datasets, such as Iris, Digits, Noisy XOR, and MNIST,
compared to Support Vector Machines (SVMs), Decision


key: chroma:document
string_value:
Trees (DTs), Random Forest (RF), Naive Bayes Classiﬁer,
Logistic Regression, and simple Artiﬁcial Neural Networks
(ANNs) [21]. The TM can further be used in convolu-
tion, providing competitive performance on MNIST, Fashion-
MNIST, and Kuzushiji-MNIST, in comparison with CNNs,
K-Nearest Neighbour (KNN), SVMs, RF, Gradient Boost-
ing, BinaryConnect, Logistic Circuits and ResNet [29]. TheTM has also achieved promising results in text classiﬁ-


key: chroma:document
string_value:
cation by using the conjunctive clauses to capture textual
patterns [23]. Further, hyper-parameter search can be sim-
pliﬁed with multi-granular clauses, eliminating the pattern
speciﬁcity parameter [25]. By indexing the clauses on the
features that falsify them, up to an order of magnitude faster
inference and learning has been reported [26]. Furthermore,
TM hardware has demonstrated up to three orders of magni-
tude reduced energy usage and faster learning, compared to


key: chroma:document
string_value:
neural networks alike [27]. While TMs are binary through-
out, binarization schemes open up for continuous input [30].
Finally, the Regression Tsetlin Machine addresses continuous
output problems, obtaining on par or better accuracy on pre-
dicting dengue incidences, stock price, real estate value and
aerofoil noise, in comparison to Regression Trees, RF, and
Support Vector Regression [24].
B. PAPER CONTRIBUTIONS
Although TMs are capable of achieving competitive perfor-


key: chroma:document
string_value:
mance levels, they often require a large number of clauses
to do so, which impedes interpretability. To overcome this
accuracy-interpretability challenge in TMs, we propose the
IWTM, encompassing the following contributions.
•We extend each clause with the Stochastic Searching
on the Line (SSL) automaton [31]. This automaton is
to learn an effective weight for its clause by interacting
with the corresponding TA team. As a result, the set
of clauses can be rendered signiﬁcantly more compact,


key: chroma:document
string_value:
without sacriﬁcing accuracy.
•Through the above scheme, we allow the TM to identify
which clauses are inaccurate. These clauses are given
smaller weights so that they must team up to obtain high
accuracy as a team. Furthermore, the clauses that are
sufﬁciently accurate are assigned larger weights so that
they can operate more independently.
•Empirically, we evaluate the IWTM using the eight data
sets: Bankruptcy, Balance Scale, Breast Cancer, Liver


key: chroma:document
string_value:
Disorders, Heart Disease, Fraud Detection, COMPAS,
and CA-58. The results show that IWTM on average
uses 6.5 times fewer literals than the vanilla TM, and
120.0 times fewer literals than a TM with real-valued
weights [22]. Furthermore, performance is competitive
with recent state-of-the-art machine learning models.
C. PAPER ORGANIZATION
In Section II, we present the IWTM and describe how each
team of TA, composing the clauses, is extended with the


key: chroma:document
string_value:
SSL. We further discuss the adaptive learning procedure
which simultaneously update both clauses and weights. Then
in Section III, we evaluate the classiﬁcation accuracy of
IWTM empirically using ﬁve datasets, including a study
of rule extraction for Bankruptcy prediction in detail and
compare against several ANNs, DTs, SVMs, KNN, RF, Gra-
dient Boosted Trees (XGBoost), EBMs (the current state-of-
the-art of Generalized Additive Models (GAMs) [32], [33])
8234 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
FIGURE 1. Transition graph of a two-action Tsetlin Automaton.
and competing TMs. Further, we contrast the performance
of IWTM against reported results on recent state-of-the-art
machine learning models, namely NAMs [11] and Structure-
Boost [34]. Finally, the paper is concluded in Section IV.
II. INTEGER-WEIGHTED TSETLIN MACHINE
In this section, we introduce the integer weighting scheme


key: chroma:document
string_value:
for TMs. First, we cover the basics of TA, which determine
the composition of the TM clauses. Then we introduce the
basic TM structure, before we present how integer weights
are assigned to the TM clauses. We cover how the individual
clauses are trained to learn sub-patterns and how the weight
values are updated using SSL. We conclude the section by
analysing the computational complexity of the IWTM.
A. TSETLIN AUTOMATA
In the TM, a collective of two-action TA [28] (reviewed in


key: chroma:document
string_value:
[35]) is used for bandit-based learning. FIGURE 1 shows a
two-action TA with 2N states. As illustrated, a TA decides its
next action from its present state. States from 1 to Ntrigger
Action 1, while states from N+1 to 2N trigger Action 2.
The TA iteratively interacts with an environment. At each
iteration, the environment produces a reward or a penalty in
response to the action performed by the TA, according to
an unknown probability distribution. Reward feedback rein-


key: chroma:document
string_value:
forces the action performed and penalty feedback weakens it.
In order to reinforce an action, the TA changes state towards
one of the ‘‘deeper’’ states, direction depending on the current
state. Conversely, an action is weakened by changing state
towards the center states (N /N+1). Hence, penalty feedback
eventually forces the TA to change action, shifting its state
from NtoN+1 or vice versa. In this manner, with a sufﬁcient
number of states, a TA converges to perform the action with


key: chroma:document
string_value:
the highest probability of receiving a reward – the optimal
action – with probability arbitrarily close to unity, as long as
the reward probability is greater than 0.5 [28].
B. TM STRUCTURE
The goal of a basic TM is to categorize input feature vec-
torsXinto one of two classes, y∈ {0, 1}. As shown in
FIGURE 2,Xconsists of opropositional variables, xk∈
{0,1}o. Further, a TM also incorporates the negation ¬xkof


key: chroma:document
string_value:
the variables to capture more sophisticated patterns. Togetherthese are referred to as literals: L=[x1,x2,..., xo,
¬x1,¬x2,...,¬xo]=[l1,l2,..., l2o].
1) CLAUSE CONSTRUCTION
At the core of a TM one ﬁnds a set of mconjunctive clauses.
The conjunctive clauses are to capture the sub-patterns asso-
ciated with each output y. All of the clauses in the TM
receive identical inputs, which is the vector of literals L.
We formulate a TM clause as follows:
cj=⋀
k∈Ijlk. (1)


key: chroma:document
string_value:
cj=⋀
k∈Ijlk. (1)
Notice that each clause, indexed by j, includes distinct
literals. The indexes of the included literals are contained in
the set Ij⊆{1,..., 2o}. For the special case of Ij=∅, i.e.,
an empty clause, we have:
cj={
1during learning
0otherwise.(2)
That is, during learning, empty clauses output 1 and during
classiﬁcation they output 0.
It is the two-action TAs that assign literals to clauses. Each
clause is equipped with 2 ×oTAs, one per literal k, as shown


key: chroma:document
string_value:
inClause-1 of FIGURE 2. The TA states from 1 to Nmap
to the exclude action, which means that the corresponding
literal is excluded from the clause. For states from N+1 to
2N, the decision becomes include, i.e., the literal is included
instead. The states of all the TAs in all of the clauses are
jointly stored in the matrix A:A=(aj,k)∈{1,..., 2N}m×2o,
with jreferring to the clause and kto the literal. Hence, the
literal indexes contained in the set Ijcan be expressed as
Ij={k|aj,k>N,1≤k≤2o}.


key: chroma:document
string_value:
Ij={k|aj,k>N,1≤k≤2o}.
2) CLAUSE OUTPUT
The output of the clauses can be produced as soon as the deci-
sions of the TAs are known. Since the clauses are conjunctive,
they evaluate to 0 if any of the literals included are of value 0.
For a given input X, let the set I1
Xcontain the indexes of the
literals of value 1. Then the output of clause jcan be expressed
as:
cj={
1 if Ij⊆I1
X,
0 otherwise.(3)
In the following, we let the vector Cdenote the complete


key: chroma:document
string_value:
set of clause outputs C=(cj)∈{0, 1}m, as deﬁned above.
3) CLASSIFICATION IN TM
The TM classiﬁes data into two classes, which means that
sub-patterns associated with both classes must be identiﬁed.
This is done by dividing clauses into two groups. Clauses with
odd index are assigned positive polarity (c+
j), and they are
to capture sub-patterns of output y=1. Clauses with even
index, on the other hand, are assigned negative polarity (c−
j)
and they seek the sub-patterns of output y=0.


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
FIGURE 2. The Integer Weighted Tsetlin Machine structure.
Once a clause recognizes a sub-pattern, it outputs 1, casting
a vote according to its polarity. The ﬁnal output of the TM
is found by summing up the clause outputs, subtracting the
votes of the clauses with negative polarity from the votes of
the clauses with positive polarity. With vbeing the difference
in clause output, v=∑
jc+
j−∑
jc−


key: chroma:document
string_value:
in clause output, v=∑
jc+
j−∑
jc−
j, the output of the TM
is decided as follows:
ˆy={
1 if v≥0
0 if v<0.(4)
C. INCORPORATING INTEGER WEIGHTS INTO THE TM
In contrast to the weighting scheme proposed by Phoulady et
al.[22], which employs real-valued weights that require mul-
tiplication and an additional hyperparameter, our scheme is
parameter-free and uses increment and decrement operations
to update the weights.
1) CLASSIFICATION IN IWTM


key: chroma:document
string_value:
to update the weights.
1) CLASSIFICATION IN IWTM
The weights decide the impact of each clause during classiﬁ-
cation, replacing Eq. 4 with:
ˆy={
1 if∑
jw+
jc+
j−∑
jw−
jc−
j≥0
0 otherwise.(5)Above, w+
jis the weight of the jthclause with positive polar-
ity, while w−
jis the weight of the jthclause with negative
polarity.
D. LEARNING PROCEDURE
In this sub-section, we ﬁrst discuss how individual clauses are
trained to learn sub-patterns. Then the procedure of updating


key: chroma:document
string_value:
weights is explained in detail.
1) CLAUSE LEARNING
A TM learns online, processing one training example (X ,y)
at a time. Within each clause, a local team of TAs decide the
clause output by selecting which literals are included in the
clause. Jointly, the TA teams thus decide the overall output
of the TM, mediated through the clauses. This hierarchical
structure is used to update the state of each TA, with the
purpose of maximizing output accuracy. We achieve this with


key: chroma:document
string_value:
two kinds of reinforcement: Type I and Type II feedback.
Type I and Type II feedback control how the individual
TAs either receive a reward, a penalty, or inaction feedback,
depending on the context of their actions. In the following
we focus on clauses with positive polarity. For clauses with
negative polarity, Type I feedback replaces Type II, and vice
versa.
8236 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
Type I feedback: Type I feedback consists of two sub-
feedback schemes: Type Ia and Type Ib. Type Ia feedback
reinforces include actions of TAs whose corresponding literal
value is 1, however, only when the clause output also is 1.
Type Ib feedback combats over-ﬁtting by reinforcing exclude
actions of TAs when the corresponding literal is 0 or when


key: chroma:document
string_value:
the clause output is 0. Consequently, both Type Ia and Type
Ib feedback gradually force clauses to output 1.
Type I feedback is given to clauses with positive polarity
when y=1. This stimulates suppression of false negative
output. To diversify the clauses, they are targeted for Type I
feedback stochastically as follows:
p+
j=

1 with probabilityT−max(−T,min(T,v))
2T,
0 otherwise.
(6)
Here, p+
jis the decision whether to target clause jwith posi-


key: chroma:document
string_value:
tive polarity for feedback. The user set target Tfor the clause
output sum vdecides how many clauses should be involved
in learning a particular sub-pattern. Higher Tincreases the
robustness of learning by allocating more clauses to learn
each sub-pattern. The decisions for the complete set of posi-
tive clauses are organized in the vector P+=(p+
j)∈{0, 1}m
2.
Similarly, decisions for the complete set of negative clauses
can be found in P−=(p−
j)∈{0, 1}m
2.


key: chroma:document
string_value:
can be found in P−=(p−
j)∈{0, 1}m
2.
If a clause is eligible to receive feedback per Eq. 6, the
individual TAs of the clause are singled out stochastically
using a user-set parameter s(s≥1). The decision whether the
kthTA of the jthclause of positive polarity is to receive Type
Ia feedback, r+
j,k, and Type Ib feedback, q+
j,k, are stochastically
made as follows:
r+
j,k=

1 with probabilitys−1
s,
0 otherwise.(7)
q+
j,k=

1 with probability1
s,
0 otherwise.(8)


key: chroma:document
string_value:
j,k=

1 with probability1
s,
0 otherwise.(8)
The above decisions are respectively stored in the two
matrices R+andQ+, i.e., R+=(r+
j,k)∈ {0, 1}m×2o
andQ+=(q+
j,k)∈ {0, 1}m×2o. Using the complete set
of conditions, TA indexes selected for Type Ia are IIa=
{(j,k)|lk=1∧c+
j=1∧p+
j=1∧r+
j,k=
1}.Similarly TA indexes selected for Type Ib are IIb= {
(j,k)|(lk=0∨c+
j=0)∧p+
j,y=1∧q+
j,k=1}
.
Once the indexes of the TAs are identiﬁed, the states of


key: chroma:document
string_value:
those TAs are updated. Available updating options are ⊕and
⊖, where⊕adds 1 and⊖subtracts 1 from the current state.
The processing of the training example ends with the state
matrix A+being updated as follows: A+←(
A+⊕IIa)
⊖IIb.
Type II feedback: Type II feedback is given to clauses
with positive polarity for target output y=0. Clauses to
receive Type II feedback are again selected stochastically.
The decision for the jthclause of positive polarity is madeas follows:
p+
j=



key: chroma:document
string_value:
p+
j=

1 with probabilityT+max(−T,min(T,v))
2T,
0 otherwise.
(9)
The idea behind Type II feedback is to change the out-
put of the affected clauses from 1 to 0. This is achieved
simply by including a literal of value 0 in the clause. TAs
selected for Type II can accordingly be found in the index
set:III={(j, k)|lk=0∧c+
j=1∧p+
j=1}.To obtain
the intended effect, these TAs are reinforced to include their
literals in the clause by increasing their corresponding states:
A+←A+⊕III.


key: chroma:document
string_value:
A+←A+⊕III.
When training has been completed, the ﬁnal decisions
of the TAs are recorded, and the resulting clauses can be
deployed for operation.
2) WEIGHT LEARNING
The learning of weights is based on increasing the weight of
clauses that receive Type Ia feedback (due to true positive
output) and decreasing the weight of clauses that receive
Type II feedback (due to false positive output). The overall
rationale is to determine which clauses are inaccurate and


key: chroma:document
string_value:
thus must team up to obtain high accuracy as a team (low
weight clauses), and which clauses are sufﬁciently accurate
to operate more independently (high weight clauses).
The weight updating procedure is summarized in Algo-
rithm 1and in the ﬂowchart in FIGURE 3. Here, wj(n) is the
weight of clause jat the nthtraining round (ignoring polarity
to simplify notation). The ﬁrst step of a training round is to
calculate the clause output as per Eq. 3. The weight of a clause


key: chroma:document
string_value:
is only updated if the clause output cj(n) is 1 and the clause
has been selected for feedback (p j=1). Then the polarity of
the clause and the class label ydecide the type of feedback
given. That is, like a regular TM, positive polarity clauses
receive Type Ia feedback if the clause output is a true positive
and Type II feedback if the clause output is a false positive.
For clauses with negative polarity, the feedback types switch
roles.
When clauses receive Type Ia or Type II feedback, their


key: chroma:document
string_value:
weights are updated accordingly. We use the stochastic
searching on the line (SSL) automaton to learn appropri-
ate weights. SSL is an optimization scheme for unknown
stochastic environments pioneered by Oommen [31]. The
goal is to ﬁnd an unknown location λ∗within a search interval
[0,1]. In order to ﬁnd λ∗, the only available information for
the Learning Mechanism (LM) is the possibly faulty feedback
from its attached environment (E ).
In SSL, the search space λis discretized into Npoints,


key: chroma:document
string_value:
{0,1/N,2/N,..., (N−1)/N,1}, with Nbeing the dis-
cretization resolution. During the search, the LM has a loca-
tionλ∈ {0, 1/N, 2/N, ..., (N−1)/N, 1}, and can freely
move to the left or to the right from its current location. The
environment Eprovides two types of feedback: E=1 is
the environment suggestion to increase the value of λby one
VOLUME 9, 2021 8237


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
FIGURE 3. The complete learning process of the IWTM in a flowchart.
step, and E=0 is the environment suggestion to decrease
the value ofλby one step. The next location of λ,λ(n+1)
can thus be expressed as follows:
λ(n+1)={
λ(n)+1/N,ifE(n)=1,
λ(n)−1/N,ifE(n)=0.(10)
λ(n+1)={
λ(n), ifλ(n)=1 and E(n)=1,
λ(n), ifλ(n)=0 and E(n)=0.(11)
Asymptotically, the learning mechanics is able to ﬁnd a value


key: chroma:document
string_value:
arbitrarily close to λ∗when N→∞ andn→∞.
In our case, the search space of clause weights is [0, ∞],
so we use resolution N=1, with no upper bound for λ.
Accordingly, we operate with integer weights. As seen in
FIGURE 3, if the clause output is a true positive, we simply
increase the weight by 1. Conversely, if the clause output is a
false positive, we decrease the weight by 1.By following the above procedure, the goal is to make
low precision clauses team up by giving them low weights,


key: chroma:document
string_value:
so that they together can reach the summation target T.
By teaming up, precision increases due to the result-
ing ensemble effect. Clauses with high precision, how-
ever, gets a higher weight, allowing them to operate more
independently.
The above weighting scheme has several advantages. First
of all, increment and decrement operations on integers are
computationally less costly than multiplication based updates
of real-valued weights. Additionally, a clause with an inte-


key: chroma:document
string_value:
ger weight can be seen as multiple copies of the same
clause, making it more interpretable than real-valued weight-
ing, as studied in the next section. Additionally, clauses
can be turned completely off by setting their weights to
0 if they do not contribute positively to the classiﬁcation
task.
8238 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
Algorithm 1 The Complete IWTM Learning Process
1:Input: Training data (X, y),m,T,s
2:Initialize: Random initialization of TAs
3:Begin: nthtraining round
4:forj=1,..., mdo if pj=1⊿Eq. (6) and (9)
5: if(y=1andjis odd) or(y=0andjis even) then
6: ifcj=1then ⊿Eq. (3)
7: wj(n+1)←wj(n)+1⊿Eq. (10-11)
8: forfeature k=1,..., 2odo
9: iflk=1then
10: Type Ia Feedback
11: else:
12: Type Ib Feedback


key: chroma:document
string_value:
11: else:
12: Type Ib Feedback
13: end if
14: end for
15: else:
16: wj(n+1)←wj(n)⊿Eq. (10-11)
17: Type Ib Feedback
18: end if
19: else: (y=1andjis even) or(y=0andjis odd)
20: ifcj=1then ⊿Eq. (3)
21: ifwj(n)>0then
22: wj(n+1)←wj(n)−1⊿Eq. (10-11)
23: end if
24: forfeature k=1,..., 2odo
25: iflk=0then
26: Type II Feedback
27: else:
28: Inaction
29: end if
30: end for
31: else:
32: wj(n+1)←wj(n)⊿Eq. (10-11)
33: Inaction
34: end if
35: end if
36:end for
E. COMPUTATIONAL COMPLEXITY OF THE IWTM


key: chroma:document
string_value:
E. COMPUTATIONAL COMPLEXITY OF THE IWTM
To evaluate computational complexity, we introduce the three
constantsα,β, andγ, whereαrepresents the computational
cost to perform the conjunction of two bits, βis the com-
putational cost of computing the summation of two integers,
andγis the computational cost to update the state of a single
automaton (TA or SSL) in IWTM.
We here consider worst-case computational costs for train-
ing and testing, assuming all the TAs in all of the clauses


key: chroma:document
string_value:
are operative and updated. In a TM with mclauses and
when the input vector consists of ofeatures, the TM per-
forms 2o×mnumber of TA updates for a single training
sample. In the IWTM, this becomes (2o +1)×mupdates
due to the weights. Hence, we compute the computational
cost of updating TA states during the IWTM training as
γ×(2o+1)×m. The cost is simply dtimes higher whenthere are dnumber of training samples in the dataset, i.e.,
d×γ×(2o+1)×m.


key: chroma:document
string_value:
d×γ×(2o+1)×m.
The other two TM operations are to compute the output of
clauses and to sum up the outputs to get the vote difference.
Assuming all the TAs in all of the clauses have decided to
include their corresponding literals in the clause, the com-
putational cost for obtaining the clause outputs becomes
α×2o×m. Once the clause outputs are ready, the vote
difference is calculated. This requires a computational cost of
β×(m−1). We only encounter the above stated computational


key: chroma:document
string_value:
requirements during the testing phase. However, both these
components have to be multiplied with the number of samples
to obtain the training cost, i.e., d[α×2o×m+β×(m−1)].
Accordingly, we can formulate the computational com-
plexity as a function of dwhich exhibits how the com-
putational complexity of the IWTM varies with dduring
the IWTM training. Combining the costs of updating TAs,
computing clause outputs, and calculating the vote difference,
we get a linear function f(d):


key: chroma:document
string_value:
we get a linear function f(d):
f(d)=d[γ×(2o+1)×m+α×2o×m+β×(m−1)].
Then, using the Big O notation [36], the computational com-
plexity of IWTM training becomes O(d), which means that
the complexity of the IWTM increases linearly with the
number of training samples d. Similarly, complexity grows
linearly with the number of clauses mand with the number of
inputs o.
III. EMPIRICAL EVALUATION
In this section, we empirically evaluate the impact of integer


key: chroma:document
string_value:
weighting on the TM using ﬁve real-world datasets. Three
of these datasets are from the health sector: Breast Cancer
dataset, Liver Disorder dataset, and Heart Disease dataset,
while the two other ones are the Balance Scale andCorporate
Bankruptcy datasets. We use the latter dataset to examine
interpretability more closely.
In the comparison, the IWTM is compared with the vanilla
TM as well as the TM with real-valued weights (RWTM).
Additionally, we contrast performance against the standard


key: chroma:document
string_value:
machine learning techniques Artiﬁcial Neural Networks
(ANNs), Support Vector Machines (SVMs), Decision Trees
(DTs), K-Nearest Neighbor (KNN), Random Forest (RF),
Gradient Boosted Trees (XGBoost) [37], Explainable Boost-
ing Machines (EBMs) [32] along with two recent state-of-the-
art machine learning approaches: Neural Additive Models
[11] and StructureBoost [34]. For comprehensiveness, three
ANN architectures are used: ANN-1 – with one hidden layer


key: chroma:document
string_value:
of 5 neurons; ANN-2 – with two hidden layers of 20 and 50
neurons each, and ANN-3 – with three hidden layers and 20,
150, and 100 neurons.
In the experiments, we use the binarization scheme based
on thresholding proposed in [30] for continuous and categor-
ical features. The results are average measures over 50 inde-
pendent experiment trials. We used 80% of the data for
training and 20% for testing. Hyperparameters were set using
manual binary search.
VOLUME 9, 2021 8239


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 1. Binarizing categorical features in the Bankruptcy dataset.
TABLE 2. Clauses produced by TM, RWTM, and IWTM for m=10.
A. BANKRUPTCY
In ﬁnance, accurate prediction of bankruptcy is important to
mitigate economic loss [38]. However, since the decisions
made related to bankruptcy can have critical consequences,
interpretable machine learning algorithms are often preferred


key: chroma:document
string_value:
over black-box methods.
Consider the historical records of 250 companies in
the Bankruptcy dataset.1Each record consists of six fea-
tures pertinent to predicting bankruptcy: 1) Industrial Risk,
2) Management Risk, 3) Financial Flexibility, 4) Credibility,
5) Competitiveness, and 6) Operation Risk. These are cate-
gorical features where each feature can be in one of three
states: Negative (N), Average (A), or Positive (P). The two
target classes are Bankruptcy and Non-bankruptcy. While


key: chroma:document
string_value:
the class output is binary, the features are ternary. We thus
binarize the features using thresholding [30], as shown in
TABLE 1. Thus, the binarized dataset contains 18 binary
features.
We ﬁrst investigate the behavior of TM, RWTM, and
IWTM with very few clauses (10 clauses). The clauses pro-
duced are summarized in TABLE 2.
Five out of the ten clauses (clauses with odd index) vote
for class 1 and the remaining ﬁve (those with even index)


key: chroma:document
string_value:
vote for class 0. In the TM, the ﬁrst clause contains just
one literal, which is the negation of feature 11. From the
binarized feature set, we recognize that the 11thfeature
isNegative Credibility. Likewise, clauses 2, 4, 6, 8, 10
contain the same two literals – the negation of Average
Competitiveness andNegative Competitiveness non-negated.
The clauses 3, 5, and 9, on the other hand, include Nega-
tive Competitiveness negated. There is also a free vote for


key: chroma:document
string_value:
class 1 from the ‘‘empty’’ clause 7, which is ignored during
classiﬁcation.
1Available from https://archive.ics.uci.edu/ml/datasets/qualitative_
bankruptcy.TABLE 3. Clauses produced by TM, RWTM, and IWTM for m=2.
The clause outputs of the TM in TABLE 2are visualized
in FIGURE 4. From the ﬁgure, it is clear how the trained TM
operates. It uses only two features, Credibility andCompet-
itiveness, and their negations. Further, observe how the TM


key: chroma:document
string_value:
implicitly introduces weighting by duplicating the clauses.
TABLE 2also contains the clauses learnt by RWTM and
IWTM. The most notable difference is that RWTM puts little
emphasis on the clauses for class 0, giving them weight
0.0001. Further, it puts most emphasis on the negation of Neg-
ative Credibility andNegative Competitiveness. The IWTM,
on the other hand, like the TM, focuses on the negation of
Average Competitiveness and non-negated Negative Compet-


key: chroma:document
string_value:
itiveness. Note also that, without loss of accuracy, IWTM
simpliﬁes the set of rules by turning off negated Negative
Credibility by giving clause 5 weight zero. The three literals
remaining are the negation of Average Competitiveness and
Negative Competitiveness , negated and non-negated. Because
Negative Competitiveness implies negated Average Compet-
itiveness, IWTM ends up with the simple classiﬁcation rule
(ignoring the weights):
Outcome={
Bankruptcy ifNegative Competitiveness


key: chroma:document
string_value:
Outcome={
Bankruptcy ifNegative Competitiveness
Non-bankruptcy otherwise.
(12)
By asking the TMs to only produce two clauses, we can
obtain the above rule more directly, as shown in TABLE 3.
As seen, again, TM, RWTM, and IWTM achieve similar
accuracy. Further, IWTM turns off Negative Competitive-
ness negated, producing the simplest rule set of the three
approaches.
The previous accuracy results represent the majority of
experiment trials. However, some of the trials fail to reach


key: chroma:document
string_value:
an optimal TM conﬁguration. Instead of re-running learning
a few times, one can increase the number of clauses for
increased robustness in every trial. This comes at the cost of
reduced interpretability, however. TABLE 4, TABLE 5, and
TABLE 6contain average performance (Precision, Recall,
F1-Score, Accuracy, Speciﬁcity) over 50 experiment trials,
showing how robustness increases with more clauses, up to a
certain point.
TABLE 4reports the results for a standard TM. Our goal is


key: chroma:document
string_value:
to maximize F1-Score, since accuracy can be misleading for
imbalanced datasets. Notice how the F1-Score increases with
the number of clauses, peaking when mequals 2000. At this
point, the average number of literals (include actions) across
the clauses is 3622 (rounded to nearest integer). The RWTM
behaves similarly, as seen in TABLE 5. However, it peaks
with an F1-Score of 0.999 at m=2000. Then 3478 literals
have been included on average.
8240 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
FIGURE 4. TM classification process for the Bankruptcy dataset.
TABLE 4. Performance of TM on Bankruptcy dataset.
TABLE 5. Performance of RWTM on Bankruptcy dataset.
TABLE 6. Performance of IWTM on Bankruptcy dataset.
The IWTM, on the other hand, achieves its best F1-Score
when mis 500. At that point, an average of 379 literals are
included (only considering clauses with a weight larger than


key: chroma:document
string_value:
0), which is signiﬁcantly smaller than what was obtained with
TM and RWTM.
How the number of literals increases with the number
of clauses is shown in FIGURE 5. The IWTM consistently
FIGURE 5. The number of literals included in different TM setups to work
with Bankruptcy dataset.
produces fewer literals than the other two schemes, and the
difference increases with the number of clauses.
We ﬁnally compare the performance of TM, RWTM, and
IWTM against several standard machine learning algorithms,


key: chroma:document
string_value:
namely ANN, DT, SVM, KNN, RF, XGBoost, and EBM.
The performance of all of the techniques is compiled in
TABLE 7. The best F1-Score is obtained by RWTM and
IWTM, which produce identical results expect that IWTM
uses fewer literals, less memory during training, and less
training time per epoch. Also, in terms of Accuracy, RWTM
and IWTM obtains the best average results. Further notice
that all of the TMs achieve a Recall of 1.0. Additionally,
VOLUME 9, 2021 8241


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 7. Performance comparison for Bankruptcy dataset.
FIGURE 6. Sample complexity analysis for the Bankruptcy dataset.
only DT, TM, and IWTM require memory close to zero, both
during training and testing. RWTM uses more memory in
training than the other two TMs since it has to represent the
weights of those 2000 clauses as ﬂoating point numbers.


key: chroma:document
string_value:
We also perform a sample complexity analysis for all
the techniques. As FIGURE 6 manifests, all the techniques
except RF and XGBoost surpasses an F1-Score of 0 .975 when
training on 40 percent of the data. The F1-Scores of RF and
XGBoost start relatively low and ﬂuctuate around 0.950 after
40% of the training samples have been processed.
B. BALANCE SCALE
For the remaining datasets, we focus on TM, RWTM and
IWTM conﬁgurations that provide robust performance over


key: chroma:document
string_value:
interpretability, comparing with selected machine learning
techniques.
We ﬁrst cover the Balance Scale dataset,2which contains
three classes: balance scale tip to the right, tip to the left, or is
in balance. The dataset also contains four features: 1) size of
the weight on the left-hand side, 2) distance from the center to
the weight on the left, 3) size of the weight on the right-hand
side, and 4) distance from the center to the weight on the right.


key: chroma:document
string_value:
To make the output binary, we remove the ‘‘balanced’’ class
ending up with 576 data samples.
TABLE 8, TABLE 9, and TABLE 10contain the results of
TM, RWTM, and IWTM, respectively, with varying m. For
2Available from http://archive.ics.uci.edu/ml/datasets/balance+scale.TABLE 8. Performance of TM on Balance Scale dataset.
TABLE 9. Performance of RWTM on Balance Scale dataset.
TABLE 10. Performance of IWTM on Balance Scale dataset.


key: chroma:document
string_value:
FIGURE 7. The number of literals included in different TM setups to work
with Balance Scale dataset.
the TM, F1-Score peaks at 0.945 when m=200. At the
peak, 790 literals are used on average. RWTM obtains its
8242 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 11. Performance comparison for Balance Scale dataset.
FIGURE 8. Sample complexity analysis for the Balance Scale dataset.
best F1-Score with 500 clauses, using an average of 4406
literals overall. In contrast, IWTM reaches its best F1-Score
using only 120 literals, distributed among 100 clauses. Again,
IWTM uses signiﬁcantly fewer literals than TM and RWTM.


key: chroma:document
string_value:
The average number of literals used for varying number
of clauses is plotted in FIGURE 7. IWTM uses the least
number of literals, with the difference increasing with number
of clauses.
TABLE 11summarises the performance also of the other
machine learning techniques we contrast against. Here, EBM
obtains the highest F1-Score and Accuracy. Out of the three
TMs, IWTM achieves the highest F1-Score and Accuracy,
using similar or less training memory. The training time


key: chroma:document
string_value:
required by IWTM is close to the training time of TM, and
roughly 60 times less compared to RWTM. According to the
sample complexity analysis in FIGURE 8, IWTM reaches an
F1-Score of 0.90 with merely 10% of the training data, and
approaches 0.95 from 20%.
C. BREAST CANCER
The Breast Cancer dataset3covers recurrence of breast can-
cer, and consists of nine features: Age, Menopause, Tumor
Size, Inv Nodes, Node Caps, Deg Malig, Side (left or right),


key: chroma:document
string_value:
the Position of the Breast, and Irradiation Status. The dataset
contains 286 patients (201 with non-recurrence and 85 with
3Available from https://archive.ics.uci.edu/ml/datasets/Breast+CancerTABLE 12. Performance of TM on Breast Cancer dataset.
TABLE 13. Performance of RWTM on Breast Cancer dataset.
TABLE 14. Performance of IWTM on Breast Cancer dataset.
recurrence). However, some of the patient samples miss some
of the feature values. These samples are removed from the


key: chroma:document
string_value:
dataset in the present experiment.
The accuracy and number of literals included for
TM, RWTM, and IWTM are respectively summarized in
TABLE 12, TABLE 13, and TABLE 14. In contrast to the
previous two datasets, the F1-Score peaks at m=2, and then
drops with increasing m. For m=2, the average number of
literals used by TM, RWTM, and IWTM are 21, 4, and 2,
respectively. As seen in FIGURE 9, IWTM requires the least
amount of literals overall.
The performance of the other machine learning techniques


key: chroma:document
string_value:
is similar in terms of F1-Score, with DT, RF, SVM, XGBoost,
and EBM providing the worst performance as summarized
in TABLE 15. The best F1-Score is obtained by TM while
IWTM provides the second-best. Yet, the moderate increase
of F1-Score from 0.511 to 0.531 for TM comes at the cost
of 19 extra literals. The three TMs also uses the least mem-
ory, requiring negligible memory both during training and
VOLUME 9, 2021 8243


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 15. Performance comparison for Breast Cancer dataset.
FIGURE 9. The number of literals included in different TM setups to work
with Breast Cancer dataset.
FIGURE 10. Sample complexity analysis for the Breast Cancer dataset.
testing. Training time per epoch for the TM approaches is also
small, amounting to 0.001 seconds, which is the lowest of


key: chroma:document
string_value:
all the algorithms. The TMs also maintain better F1-Scores
across all training data sizes in comparison with the other
techniques, as seen from the sample complexity analysis in
FIGURE 10.D. LIVER DISORDERS
The Liver Disorders dataset4was created by BUPA Medical
Research and Development Ltd. (hereafter ‘‘BMRDL’’) dur-
ing the 1980s as part of a larger health-screening database.
The dataset consists of 7 attributes, namely Mean Cor-
puscular Volume, Alkaline Phosphotase, Alamine Amino-


key: chroma:document
string_value:
transferase, Aspartate Aminotransferase, Gamma-Glutamyl
Transpeptidase, Number of Half-Pint Equivalents of Alco-
holic Beverages (drunk per day), and Selector (used to split
data into training and testing sets). However, McDermott
and Forsyth [39] claim that many researchers have used
the dataset incorrectly, considering the Selector attribute
as class label. Based on the recommendation of McDer-
mott and Forsythof, we here instead use Number of Half-


key: chroma:document
string_value:
Pint Equivalents of Alcoholic Beverages as the dependent
variable, binarized using the threshold ≥3. The Selec-
tor attribute is discarded. The remaining attributes repre-
sent the results of various blood tests, and we use them as
features.
TABLE 17, TABLE 18, and TABLE 19summarizes
the performance of TM, RWTM, and IWTM, respectively.
As seen, all of the TM F1-Scores peak at m=2. TM uses an
average of 27 literals, RWTM uses 29, while IWTM uses 9.


key: chroma:document
string_value:
FIGURE 11plots how the number of literals increases with
number of clauses, again conﬁrming that IWTM uses fewer
literals overall.
Considering the other machine learning techniques
(TABLE 16), RF produces the highest F1-Score 0.729,
obtained with the smallest memory usage for both training
and testing. However, this performance is comparable to the
DT F1-Score of 0 .728, spending negligible testing memory.
Of the three TM approaches, RWTM obtains the highest


key: chroma:document
string_value:
F1-Score - the fourth highest among all of the techniques.
All three TMs use insigniﬁcant memory during both training
and testing, requiring the same amount of training time per
epoch.
4Available from https://archive.ics.uci.edu/ml/datasets/Liver+Disorders.
8244 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 16. Performance comparison for Liver Disorders dataset.
TABLE 17. Performance of TM on Liver Disorders dataset.
TABLE 18. Performance of RWTM on Liver Disorders dataset.
TABLE 19. Performance of IWTM on Liver Disorders dataset.
With 10% of the training data available, IWTM obtains the
highest F1-Score among all the techniques – a score of 0.725.


key: chroma:document
string_value:
After that, the score ﬂuctuates with increasing training data
size, as depicted in FIGURE 12.
E. HEART DISEASE
The Heart Disease dataset5concerns prediction of heart dis-
ease. To this end, 13 features are available, selected among
75. Out of the 13 features, 6 are real-valued, 3 are binary, 3
are nominal, and one is ordered.
TABLE 21, TABLE 22, and TABLE 23summarize the per-
formance of TM, RWTM, and IWTM on the Heart Disease
dataset. For the TM, the best F1-Score occurs with m=10,


key: chroma:document
string_value:
achieved by using 346 literals on average. The RWTM F1-
Score peaks at m=2000 with 18 528 literals. IWTM peaks
5Available from https://archive.ics.uci.edu/ml/datasets/Statlog+
%28Heart%29.
FIGURE 11. The number of literals included in different TM setups to
work with the Liver Disorders dataset.
FIGURE 12. Sample complexity analysis for the Liver Disorders dataset.
atm=10, with slightly lower F1-Score, however, employing
only 226 literals on average.


key: chroma:document
string_value:
only 226 literals on average.
Considering the number of literals used with increasing
number of clauses (FIGURE 13), TM and IWTM behave
similarly, while RWTM requires signiﬁcantly more literals.
Out of the considered machine learning models, EBM
obtains the best F1-Score, while RWTM, IWTM, and ANN-2
follow closely behind (TABLE 20). However, EBM needs
the highest training time and uses the second largest training
VOLUME 9, 2021 8245


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 20. Performance comparison for Heart Disease dataset.
TABLE 21. Performance of TM on Heart Disease dataset.
TABLE 22. Performance of RWTM on Heart Disease dataset.
TABLE 23. Performance of IWTM on Heart Disease dataset.
memory, while all three TMs use negligible memory during
both training and testing. Apart from DT, all of the machine


key: chroma:document
string_value:
learning models surpass an F1-Score of 0.5 using only 10%
of the training data, as shown in FIGURE 14. From 30%
onward, the F1-Score of all the models behaves similarly,
with EBM being superior after 60%.
F. SUMMARY OF EMPIRICAL EVALUATION
To compare overall performance of the various techniques,
we calculate average F1-Score across the datasets. Further to
evaluate overall interpretability of TM, RWTM and IWTM,
we also report average number of literals used, overall.


key: chroma:document
string_value:
In all brevity, the average F1-Score of ANN-1, ANN-2,
ANN-3, DT, SVM, KNN, RF, TM, XGBoost, EBM, RWTM,
and IWTM are 0.770, 0.757, 0.744, 0 .742, 0.713, 0.737,
0.724 0.728, 0.775, 0.762, 0.774, and 0.777, respectively. Out
of all the considered models, IWTM obtains the best average
F1-Score, which is 0.777. Also notice that increasing ANN
model complexity (from ANN-1 to ANN-3) reduces overall
FIGURE 13. The number of literals included in different TM setups to
work with Heart Disease dataset.


key: chroma:document
string_value:
work with Heart Disease dataset.
FIGURE 14. Sample complexity analysis for the Heart Disease dataset.
F1-Score, which can potentially be explained by the small
size of the datasets.
The F1-Score of RWTM is also competitive, however,
it requires much more literals than IWTM. Indeed, the
average number of literals employed are 961 for TM, 17 670
for RWTM, and 147 for IWTM. That is, IWTM uses 6.5
times fewer literals than TM, and 120 times fewer literals than
RWTM.


key: chroma:document
string_value:
RWTM.
The average combined memory requirement (training +
testing) by TM, RWTM, and IWTM are 3.27 KB, 79.46 KB,
and 3.27, respectively. The combined memory usage of
8246 VOLUME 9, 2021


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
TABLE 24. Performance (in AUC) comparison against recent
state-of-the-art machine learning models.
IWTM is signiﬁcantly less compared to the other models –
ANN-1:≈305 times, ANN-2: ≈1 278 times, ANN-3: ≈
11 096 times, DT:≈34 times, SVM:≈255 times, KNN:≈
106 times, RF:≈45 times, XGBoost:≈877 times, EBM:≈
1226 times, and RWTM: ≈24 times.
G. COMPARISON AGAINST RECENT STATE-OF-THE-ART


key: chroma:document
string_value:
G. COMPARISON AGAINST RECENT STATE-OF-THE-ART
MACHINE LEARNING MODELS
In this section, we compare IWTM accuracy with reported
results on recent state-of-the-art machine learning mod-
els. First, we perform experiments on Fraud Detection and
COMPAS: Risk Prediction in Criminal Justice datasets to
study the performance of IWTM in comparison with Neu-
ral Additive Models [11]. A Neural Additive Model is a
novel member of so-called general adaptive models. In Neu-


key: chroma:document
string_value:
ral Additive Models, the signiﬁcance of each input feature
towards the output is learned by a dedicated neural network.
During the training phase, the complete set of neural networks
are jointly trained to learn complex interactions between
inputs and outputs.
To compare the performance against StructureBoost [34],
we use the CA weather dataset [40]. For simplicity, we use
only the CA-58 subset of the dataset in this study. Struc-
tureBoost is based on gradient boosting and is capable of


key: chroma:document
string_value:
exploiting the structure of categorical variables. Structure-
Boost outperforms established models such as CatBoost and
LightBoost on multiple classiﬁcation tasks [34].
Since the performance of both of the above techniques has
been measured in terms of Area under the ROC Curve (AUC),
we here use a soft TM output layer [41] to calculate AUC. The
performance characteristics are summarized in TABLE 24.
TABLE 24 shows that on Fraud Detection, IWTM outper-


key: chroma:document
string_value:
forms NAMs and all the other techniques mentioned in [11].
On the COMPAS dataset, IWTM exhibits competitive per-
formance compared to NAMs, EBM, XGBoost, and DNNs.
IWTM shows, however, superior performance compared to
Logistic Regression and DT on COMPAS. The performance
of IWTM on CA-20 is better in comparison to Structure-
Boost, LightBoost, and CatBoost models, reported in [34].
IV. CONCLUSION
In this paper, we proposed a novel Tsetlin Machine (TM)


key: chroma:document
string_value:
having integer weights attached to clauses, to addressthe accuracy-interpretability challenge in machine learning.
In our proposed TM (denoted IWTM), the weights are learnt
using the stochastic searching on the line (SSL) automa-
ton. The weights attached to the clauses help the TM to
the represent sub-patterns in a more compact way. Since
integer weights can turn off unimportant clauses by setting
their weight to 0, this allows the TM to create a classi-


key: chroma:document
string_value:
ﬁer with fewer number of literals compared to the vanilla
TM and the Real-Value Weighted TM (RWTM). We have
provided empirical evidence by generating rules for several
datasets. In conclusion, the IWTM obtains on par or better
accuracy compared to the vanilla TM and the RWTM while
using respectively 6.5 and 125 times fewer literals. Further-
more, in terms of average F1-Score, the proposed IWTM
also outperforms several state-of-the-art machine learning
algorithms.


key: chroma:document
string_value:
algorithms.
In our future work, we intend to investigate more advanced
SSL schemes, such as Continuous Point Location with Adap-
tive Tertiary Search (CPL-ATS) [42] and Random Walk-
based Triple Level Learning Algorithm (RWTLA) [43],
including their impact on interpretability.
REFERENCES
[1] C. Molnar, Interpretable Machine Learning . Morrisville, NC, USA: Lulu,
2019.
[2] R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, ‘‘Deep learning


key: chroma:document
string_value:
for healthcare: Review, opportunities and challenges,’’ Brieﬁngs Bioinf.,
vol. 19, no. 6, pp. 1236–1246, Nov. 2018.
[3] B. Baesens, C. Mues, M. De Backer, J. Vanthienen, and R. Setiono,
‘‘Building intelligent credit scoring systems using decision tables,’’ in
Enterprise Information Systems V. Dordrecht, The Netherlands: Springer,
2004, pp. 131–137.
[4] J. Huysmans, K. Dejaeger, C. Mues, J. Vanthienen, and B. Baesens,
‘‘An empirical evaluation of the comprehensibility of decision table, tree


key: chroma:document
string_value:
and rule based predictive models,’’ Decis. Support Syst., vol. 51, no. 1,
pp. 141–154, Apr. 2011.
[5] R. Bellazzi and B. Zupan, ‘‘Predictive data mining in clinical medicine:
Current issues and guidelines,’’ Int. J. Med. Informat., vol. 77, no. 2,
pp. 81–97, Feb. 2008.
[6] S. Mani, W. R. Shankle, and M. J. Pazzani, ‘‘Acceptance of rules generated
by machine learning among medical experts,’’ Methods Inf. Med., vol. 40,
no. 5, pp. 380–385, 2001.


key: chroma:document
string_value:
no. 5, pp. 380–385, 2001.
[7] A. A. Freitas, D. C. Wieser, and R. Apweiler, ‘‘On the importance of
comprehensible classiﬁcation models for protein function prediction,’’
IEEE/ACM Trans. Comput. Biol. Bioinf., vol. 7, no. 1, pp. 172–182,
Jan. 2010.
[8] D. Szafron, P. Lu, R. Greiner, D. S. Wishart, B. Poulin, R. Eisner, Z. Lu,
J. Anvik, C. Macdonell, A. Fyshe, and D. Meeuwis, ‘‘Proteome analyst:
Custom predictions with explanations in a Web-based tool for high-


key: chroma:document
string_value:
throughput proteome annotations,’’ Nucleic Acids Res., vol. 32, no. 2,
pp. W365–W371, Jul. 2004.
[9] E. Lima, C. Mues, and B. Baesens, ‘‘Domain knowledge integration in data
mining using decision tables: Case studies in churn prediction,’’ J. Oper.
Res. Soc., vol. 60, no. 8, pp. 1096–1106, Aug. 2009.
[10] W. Verbeke, D. Martens, C. Mues, and B. Baesens, ‘‘Building compre-
hensible customer churn prediction models with advanced rule induction


key: chroma:document
string_value:
techniques,’’ Expert Syst. Appl., vol. 38, no. 3, pp. 2354–2364, Mar. 2011.
[11] R. Agarwal, N. Frosst, X. Zhang, R. Caruana, and G. E. Hinton, ‘‘Neural
additive models: Interpretable machine learning with neural nets,’’ 2020,
arXiv:2004.13912. [Online]. Available: http://arxiv.org/abs/2004.13912
[12] M. T. Ribeiro, S. Singh, and C. Guestrin, ‘‘Why should I trust you?’
Explaining the predictions of any classiﬁer,’’ in Proc. 22nd ACM SIGKDD


key: chroma:document
string_value:
Int. Conf. Knowl. Discovery Data Mining, 2016, pp. 1135–1144.
[13] C. Rudin, ‘‘Stop explaining black box machine learning models for high
stakes decisions and use interpretable models instead,’’ Nature Mach.
Intell., vol. 1, no. 5, pp. 206–215, May 2019.
VOLUME 9, 2021 8247


key: chroma:document
string_value:
K. D. Abeyrathna et al.: Extending the TM With Integer-Weighted Clauses for Increased Interpretability
[14] R. Agrawal, T. Imieliński, and A. Swami, ‘‘Mining association rules
between sets of items in large databases,’’ ACM SIGMOD Rec., vol. 22,
no. 2, pp. 207–216, Jun. 1993.
[15] T. McCormick, C. Rudin, and D. Madigan, ‘‘A hierarchical model for
association rule mining of sequential events: An approach to automated
medical symptom prediction,’’ MIT Sloan Res. Paper, Jan. 2011. [Online].


key: chroma:document
string_value:
Available: https://ssrn.com/abstract=1736062
[16] V. Feldman, ‘‘Hardness of approximate two-level logic minimization and
PAC learning with membership queries,’’ J. Comput. Syst. Sci., vol. 75,
no. 1, pp. 13–26, Jan. 2009.
[17] L. G. Valiant, ‘‘A theory of the learnable,’’ Commun. ACM, vol. 27, no. 11,
pp. 1134–1142, Nov. 1984.
[18] T. Wang, C. Rudin, F. Doshi-Velez, Y. Liu, E. Klampﬂ, and P. MacNeille,
‘‘A Bayesian framework for learning rule sets for interpretable classiﬁca-


key: chroma:document
string_value:
tion,’’ J. Mach. Learn. Res., vol. 18, no. 1, pp. 2357–2393, 2017.
[19] J. R. Hauser, O. Toubia, T. Evgeniou, R. Befurt, and D. Dzyabura, ‘‘Dis-
junctions of conjunctions, cognitive simplicity, and consideration sets,’’
J. Marketing Res., vol. 47, no. 3, pp. 485–496, Jun. 2010.
[20] Y. Liang and G. Van den Broeck, ‘‘Learning logistic circuits,’’ in Proc. 33rd
AAAI Conf. Artif. Intell., vol. 33, 2019, pp. 4277–4286.
[21] O.-C. Granmo, ‘‘The tsetlin machine—A game theoretic bandit driven


key: chroma:document
string_value:
approach to optimal pattern recognition with propositional logic,’’ 2018,
arXiv:1804.01508. [Online]. Available: http://arxiv.org/abs/1804.01508
[22] A. Phoulady, O.-C. Granmo, S. R. Gorji, and H. A. Phoulady,
‘‘The weighted tsetlin machine: Compressed representations with
weighted clauses,’’ 2019, arXiv:1911.12607. [Online]. Available:
https://arxiv.org/abs/1911.12607
[23] G. T. Berge, O.-C. Granmo, T. O. Tveit, M. Goodwin, L. Jiao, and


key: chroma:document
string_value:
B. V. Matheussen, ‘‘Using the tsetlin machine to learn human-interpretable
rules for high-accuracy text categorization with medical applications,’’
IEEE Access, vol. 7, pp. 115134–115146, 2019.
[24] K. D. Abeyrathna, O.-C. Granmo, X. Zhang, L. Jiao, and M. Goodwin,
‘‘The regression Tsetlin machine: A novel approach to interpretable nonlin-
ear regression,’’ Phil. Trans. Roy. Soc. A, Math., Phys. Eng. Sci., vol. 378,
no. 2164, Feb. 2020, Art. no. 20190165.


key: chroma:document
string_value:
no. 2164, Feb. 2020, Art. no. 20190165.
[25] S. R. Gorji, O.-C. Granmo, A. Phoulady, and M. Goodwin, ‘‘A Tsetlin
machine with multigranular clauses,’’ in Proc. 39th Int. Conf. Innov.
Techn. Appl. Artif. Intell. (SGAI), in Lecture Notes in Computer Science,
vol. 11927. Cham, Switzerland: Springer, 2019, pp. 146–151.
[26] S. Gorji, O. C. Granmo, S. Glimsdal, J. Edwards, and M. Goodwin,
‘‘Increasing the inference and learning speed of Tsetlin machines with


key: chroma:document
string_value:
clause indexing,’’ in Proc. Int. Conf. Ind., Eng. Other Appl. Appl. Intell.
Syst. Cham, Switzerland: Springer, 2020, pp. 695–708.
[27] A. Wheeldon, R. Shaﬁk, A. Yakovlev, J. Edwards, I. Haddadi, and
O.-C. Granmo, ‘‘Tsetlin machine: A new paradigm for pervasive AI,’’
inProc. SCONA Workshop Design, Automat. Test Eur. (DATE), 2020.
[Online]. Available: https://www.date-conference.com/node/468
[28] M. L. Tsetlin, ‘‘On behaviour of ﬁnite automata in random medium,’’


key: chroma:document
string_value:
Avtomatika I Telemekhanika, vol. 22, no. 10, pp. 1345–1354, 1961.
[29] O.-C. Granmo, S. Glimsdal, L. Jiao, M. Goodwin, C. W. Omlin, and
G. T. Berge, ‘‘The convolutional Tsetlin machine,’’ 2019,
arXiv:1905.09688. [Online]. Available: http://arxiv.org/abs/1905.09688
[30] K. D. Abeyrathna, O.-C. Granmo, X. Zhang, and M. Goodwin, ‘‘A scheme
for continuous input to the Tsetlin machine with applications to forecasting
disease outbreaks,’’ in Proc. Int. Conf. Ind., Eng. Other Appl. Appl. Intell.


key: chroma:document
string_value:
Syst. Cham, Switzerland: Springer, 2019, pp. 564–578.
[31] B. J. Oommen, ‘‘Stochastic searching on the line and its applications to
parameter learning in nonlinear optimization,’’ IEEE Trans. Syst. Man,
Cybern. B, Cybern., vol. 27, no. 4, pp. 733–739, Aug. 1997.
[32] H. Nori, S. Jenkins, P. Koch, and R. Caruana, ‘‘InterpretML:
A uniﬁed framework for machine learning interpretability,’’ 2019,
arXiv:1909.09223. [Online]. Available: http://arxiv.org/abs/1909.09223


key: chroma:document
string_value:
[33] Y. Lou, R. Caruana, and J. Gehrke, ‘‘Intelligible models for classiﬁcation
and regression,’’ in Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discovery
Data Mining (KDD), 2012, pp. 150–158.
[34] B. Lucena, ‘‘StructureBoost: Efﬁcient gradient boosting for structured
categorical variables,’’ 2020, arXiv:2007.04446. [Online]. Available:
http://arxiv.org/abs/2007.04446
[35] K. S. Narendra and M. A. Thathachar, Learning Automata: An Introduc-
tion. Chelmsford, MA, USA: Courier Corporation, 2012.


key: chroma:document
string_value:
[36] I. Chivers and J. Sleightholme, ‘‘An introduction to algorithms and the
big O notation,’’ in Introduction to Programming With Fortran. Cham,
Switzerland: Springer, 2015, pp. 359–364.[37] T. Chen and C. Guestrin, ‘‘XGBoost: A scalable tree boosting system,’’
inProc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining,
Aug. 2016, pp. 785–794.
[38] M.-J. Kim and I. Han, ‘‘The discovery of experts’ decision rules from


key: chroma:document
string_value:
qualitative bankruptcy data using genetic algorithms,’’ Expert Syst. Appl.,
vol. 25, no. 4, pp. 637–646, Nov. 2003.
[39] J. McDermott and R. S. Forsyth, ‘‘Diagnosing a disorder in a classiﬁcation
benchmark,’’ Pattern Recognit. Lett., vol. 73, pp. 41–43, Apr. 2016.
[40] B. Lucena, ‘‘Exploiting categorical structure using tree-based meth-
ods,’’ 2020, arXiv:2004.07383. [Online]. Available: http://arxiv.org/
abs/2004.07383
[41] K. D. Abeyrathna, O.-C. Granmo, and M. Goodwin, ‘‘On obtaining classi-


key: chroma:document
string_value:
ﬁcation conﬁdence, ranked predictions and AUC with Tsetlin machines,’’
inProc. IEEE Symp. Series Comput. Intell. (ISSC), 2020.
[42] B. J. Oommen and G. Raghunath, ‘‘Automata learning and intelligent
tertiary searching for stochastic point location,’’ IEEE Trans. Syst. Man,
Cybern. B, Cybern., vol. 28, no. 6, pp. 947–954, Dec. 1998.
[43] W. Jiang, D.-S. Huang, and S. Li, ‘‘Random walk-based solution to triple
level stochastic point location problem,’’ IEEE Trans. Cybern., vol. 46,


key: chroma:document
string_value:
no. 6, pp. 1438–1451, Jun. 2016.
K. DARSHANA ABEYRATHNA received the
B.Sc. degree in mechatronics engineering from
AIT University, Thailand, in 2015, and the M.Sc.
degree from the Big Data Research Group, Tham-
masat University, Thailand, in 2017. He is cur-
rently a Ph.D. Research Fellow with the University
of Agder. He is also with CAIR, working on a
project which develops a global grid that facilitates
real-time compilation, and management and anal-


key: chroma:document
string_value:
real-time compilation, and management and anal-
ysis of spatio-temporal data. His research interests
include artiﬁcial neural networks, data mining, optimization, and operations
research.
OLE-CHRISTOFFER GRANMO received the
master’s and Ph.D. degrees from the University
of Oslo, Norway, in 1999 and 2004, respectively.
He is currently a Professor with the University
of Agder and the Founding Director of the Cen-
tre for Artiﬁcial Intelligence Research (CAIR).


key: chroma:document
string_value:
tre for Artiﬁcial Intelligence Research (CAIR).
He has authored in excess of 140 refereed arti-
cles within machine learning, encompassing learn-
ing automata, bandit algorithms, Tsetlin machines,
Bayesian reasoning, reinforcement learning, and
computational linguistics. Apart from his academic endeavors, he co-
founded the company Anzyz Technologies AS. He is also a Co-Founder of
the Norwegian Artiﬁcial Intelligence Consortium (NORA).
MORTEN GOODWIN received the master’s


key: chroma:document
string_value:
MORTEN GOODWIN received the master’s
degree from the University of Agder, Norway,
in 2005, and the Ph.D. degree from the Depart-
ment of Computer Science, Aalborg University,
Denmark, in 2011, where he applied machine
learning algorithms on eGovernment indicators
which are difﬁcult to measure automatically. He is
currently an Associate Professor with the Univer-
sity of Agder. His research interests include data-
mining, optimization, machine learning, swarm


key: chroma:document
string_value:
mining, optimization, machine learning, swarm
intelligence, deep learning, and adaptive learning. He has written more
than 40 peer-reviewed scientiﬁc publications in this area and supervised more
than 60 student projects at the master’s and Ph.D. level.
8248 VOLUME 9, 2021


key: chroma:document
string_value:
(2022)52:–1746517489 AppliedIntelligence
https://doi.org/10.1007/s10489-022-03281-1
Word-levelhumaninterpretablescoringmechanismfornoveltext
detectionusingTsetlinMachines
BimalBhattarai1·Ole-ChristoﬀerGranmo1·LeiJiao1
Accepted:20January2022
©TheAuthor(s)2022
Abstract
Recent research in novelty detection focuses mainly on document-level classification, employing deep neural networks


key: chroma:document
string_value:
(DNN). However, the black-box nature of DNNs makes it difficult to extract an exact explanation of why a document is
considered novel. In addition, dealing with novelty at the word level is crucial to provide a more fine-grained analysis than
what is available at the document level. In this work, we propose a Tsetlin Machine (TM)-based architecture for scoring
individual words according to their contribution to novelty. Our approach encodes a description of the novel documents using


key: chroma:document
string_value:
the linguistic patterns captured by TM clauses. We then adapt this description to measure how much a word contributes to
making documents novel. Our experimental results demonstrate how our approach breaks down novelty into interpretable
phrases, successfully measuring novelty.
Keywords Natural language processing ·Novelty detection ·Tsetlin Machine ·Interpretable learning ·Explainable AI
1Introduction
The fundamental principle underlying machine learning


key: chroma:document
string_value:
classifiers is a generalization – the ability to form a decision
boundary that differentiates new input into known classes.
When training a supervised classifier, it is common to
assume that the classes to be recognized are present both
in the training and test data [ 49]. However, given an
open world, training on all conceivable classes of input is
impractical. This problem introduces the need for novelty
detection – the task of spotting input classes that one has


key: chroma:document
string_value:
not seen before. The problem is particularly severe in text-
based supervised classification due to the many-faceted
nature of natural language, which gives rise to multiple
application-dependent interpretations. Indeed, researchers
/envelopebackBimal Bhattarai
bimal.bhattarai@uia.no; bobsbimal58@gmail.com
Ole-Christoffer Granmo
ole.granmo@uia.no
Lei Jiao
lei.jiao@uia.no
1Centre for Artificial Intelligence Research (CAIR), University


key: chroma:document
string_value:
of Agder, Grimstad, Norwayhave for a long time tried to address novelty detection in
natural language. So far, no single best model has appeared.
Indeed, the success of each model relies on the properties of
each particular dataset.
The problem of novelty detection arises in many tasks,
such as fault detection [ 16] and handwritten alphabet
recognition [ 54]. In general, one applies novelty detection
when it is required to know whether a given input is similar


key: chroma:document
string_value:
to or significantly different from the training data. For
natural language text, the novelty detector should discern
that a text does not belong to a predefined set of topics.
Several challenges make such novelty detection particularly
difficult:
1. Textual information tends to be diverse, composed of
large vocabularies.
2. Language and topics are typically evolving, making the
novelty detection problem dynamic [ 21].
Lately, the aforementioned challenges have manifested


key: chroma:document
string_value:
when using supervised learning to build chatbots, an
application area that is gaining traction. A chatbot typically
needs to handle the language of a multitude of users with
evolving information requirements. As such, it must be able
to determine when it is capable of answering a query and
when it faces a new topic.
Majority of the existing literature on text-based novelty
detection addresses one of the following granularity levels:/Published online: 2 April 2022


key: chroma:document
string_value:
B.Bhattaraietal.
1. Event-level techniques [ 4] perform topic detection and
tracking on a stream of documents.
2. Document-level techniques [ 17] classify an incoming
document as known or novel based on its content.
3. Sentence-level techniques [ 6] look for novel sentences
within a particular document.
Usually, the sentences/documents are ranked based on
some sort of similarity score, obtained from comparing them
with previously seen sentences/documents. For instance, the


key: chroma:document
string_value:
Maximal Marginal Relevance model (MMR) proposed in
[14] assigns low scores to previously seen sentences/docu-
ments, while assigning high scores to novel ones.
Figure 1illustrates the problem of novelty detection,
contrasting it against anomaly and outlier detection.
Anomaly detection [ 15] concerns discovering anomalies,
which are invalid data points. Outlier detection [ 3,29],
on the other hand, flags legitimate data points that deviate


key: chroma:document
string_value:
significantly from the mean. Finally, novelty detection [ 43]
is the discovery of entirely new types of data points.
In contrast to prior work, we here focus on novelty
detection at the word level. To this end, we propose a
new interpretable machine learning approach for calculating
novelty scores for the words within a sentence. The
calculation is based on the linguistic patterns captured by
a Tsetlin Machine (TM) in the form of AND-rules (i.e.,


key: chroma:document
string_value:
conjunctive clauses). To the best of our knowledge, this is
the first study of its kind on this problem.
Problemdeﬁnition In the supervised classification setting,
ipre-labeled data points D={(v1,y1), (v 2,y2) ,...,
(vi,yi)}is used for training. Here, viis the ithinput
example and yiis its class. The input viis ant-dimensional
real-valued vector (x1,x2,...,x o)∈Rt,w h e r e xorefers
to the othelement of the vector. The class yi∈Y=
{1,2,...,C l}, in turn, is an integer class index referring


key: chroma:document
string_value:
to one out of Clclasses. Learning a classifier entails
constructing a classification function f( v;D),f:Rt→Y,
based on the data D. The function simply assigns a label y
to the data point v. Our emphasis is novelty scoring, which
can be seen as another function z(v;D),z :Rt→R.The function computes a real-valued novelty score for input
data point v, with the purpose of discerning new classes not
found in Y. In this way, a classifier can return the correct


key: chroma:document
string_value:
class label while flagging novel examples. Considering each
element in vto represent a specific word, this paper further
extend the novelty detection by introducing a method for
breaking down the overall score z(v;D)forvinto the
contribution of each element xo. By doing so, we break
down novelty into interpretable phrases.
Paper contributions In this paper, we use the TM to con-
struct conjunctive clauses in propositional logic. In this


key: chroma:document
string_value:
manner, we capture frequent patterns in the data D,w h i c h
we then utilize to characterize the known classes Ycom-
prehensively. The novelty score is then calculated based on
examining the clauses that match the given input. By fur-
ther looking into the composition of each clause, we are
able to break down the novelty score into the contribu-
tion of the different phrases. This decomposition is based
on training clauses for the novel data and then measuring


key: chroma:document
string_value:
the relative frequency of each word inside the clauses for
the known classes, contrasted against the relative frequency
obtained from the clauses of the novel class. These scores
can, in turn, be adopted as input features to machine learn-
ing classifiers for novelty detection. Similarly, contextual
scores can be calculated simply by inspecting each word’s
clauses, providing a local perspective for both novel and
known classes.
The remainder of the paper is organized as follows.


key: chroma:document
string_value:
In Section 2, we first summarize related work before we
present the details of the TM in Section 3. This forms the
basis for our novelty description architecture, covered in
Section 4. In Section 5, we present our empirical results,
concluding the work in the last section.
2Relatedwork
Several studies have been carried out on supervised
multiclass classification in a closed-world setting [ 5]. There
is a dearth of work addressing open-world settings [ 33],


key: chroma:document
string_value:
Outlier decision boundaryOutlier detection Novelty detection
trained example
distant
known classnovel example
decision boundaryAnamoly detection
Anomaly
Fig.1 Visualization of outlier detection, anomaly detection and novelty detection17466


key: chroma:document
string_value:
Word-levelhumaninterpretable...
with distance-based methods being one of the earliest
approaches [ 28]. These approaches rely on nearest neighbor
search, which introduces scalability issues when dealing
with larger datasets. Another class of methods are based on
single-class classifiers. These include One-Class SVM [ 50]
and SVDD [ 55]. Further, the decision score from SVM has
been used to produce a probability distribution for novelty
detection [ 44]. As no negative training samples are used,


key: chroma:document
string_value:
single-class classifiers struggle with maximizing the class
margin. To overcome the problem of One-Class SVMs,
a new learning method named center-based similarity
space (CBS) was proposed in [ 20], which transforms each
document in a closed boundary to a central similarity vector
that can be used in a binary classifier.
Probabilistic methods have also been utilized for novelty
detection [ 43]. In [ 30], a technique to threshold the entropy


key: chroma:document
string_value:
of the estimated class probability distribution is proposed.
In that method, choosing the entropy threshold needs prior
knowledge. Additionally, the class probability distribution
can be misleading when novel data points fall far from
the decision boundary. In [ 32]a n d[ 46], an active learning
model is proposed to both discover and classify novel
classes during training. However, the appearance of novel
instances during testing is not considered.


key: chroma:document
string_value:
instances during testing is not considered.
DNNs have recently been used to address the problem
of novelty detection. In [ 61], a two-class SVM classifier is
adopted to categorize known and novel classes. An adver-
sarial sample generation (ASG) framework [ 23]i su s e d
to generate positive and negative samples. Similarly, [ 37]
employs generative adversarial networks (GANs), where
the generator produces a mixture of known and novel data.


key: chroma:document
string_value:
The generator is trained with so-called feature matching
loss, and the discriminator performs simultaneous clas-
sification and novelty detection. In computer vision, the
problem of novel image detection is addressed by intro-
ducing the concept of open space risk [ 49]. This is
achieved by reducing the half-space of a binary SVM
classifier with two parallel hyperplanes that bound the
positive region. Although the binary SVM reduces the pos-


key: chroma:document
string_value:
itive region to half-spaces, their open space risk is still
infinite. In [ 5], a method called OpenMAX is proposed,
which estimates the probability of an input belonging to
a novel class. In general, the major weaknesses of these
methods are high computational complexity and uninter-
pretable inference. A state-of-the-art GAN-based method
for unsupervised outlier detection called Single-Objective
Generative Adversarial Active Learning (SO-GAAL) and


key: chroma:document
string_value:
Multi-Objective Generative Adversarial Active Learning
(MO-GAAL) was proposed in [ 41]. The method is based
on a min-max game between a generator and a discrimina-
tor. The training process of the generator is paused before
convergence to synthesize outliers, which is subsequently
used to train the discriminator to recognize the outliers.However, the method is primarily designed for high-dimen-
sional data, requiring extensive problem-specific hyper-


key: chroma:document
string_value:
parameter tweaking. The unsupervised learning method
COPOD [ 40] is a more recent approach that is inspired
by copulas for modeling multivariate data distributions. In
comparison to other methods, COPOD is computationally
efficient, interpretable, and is unaffected by feature dimen-
sion. However, the method fails to handle complex features
and intricate nonlinear relations.
Apart from the studies on the document-level novelty
detection, novelty detection at the event level arises from


key: chroma:document
string_value:
topic detection, which focuses on the online event and story
detection [ 38]. The study at the event level primarily con-
sists of clustering algorithms that measure the closeness of
incoming events or stories to one of the clusters depend-
ing on a pre-defined threshold. Novelty detection at the
sentence level was investigated in Text Retrieval Confer-
ences (TREC) by highlighting sentences that include novel
information given a topic and a list of documents [ 52].


key: chroma:document
string_value:
Based on TREC, many studies have been conducted on
novelty detection at sentence level [ 56,63], including term
translations, Principal Component Analysis (PCA) vectors,
Support Vector Machine (SVM) classification, named enti-
ties patterns, etc. Likewise, a few approaches have been
introduced for learning sentence embeddings, including
SkipThought [ 36], Conceptual Sentence Embedding [ 58],
and FastSent [ 31]. However, these approaches on embed-


key: chroma:document
string_value:
dings are very dependent on the domain-specific down-
stream tasks. Recently introduced powerful language mod-
els, such as ELMo [ 42]a n dB E R T[ 18], have been success-
ful for transfer learning and they are able to learn dynamic
sentence embedding in an unsupervised manner.
In [22], a unified attention architecture is proposed to
deal with vector representations of text input in NLP. The
authors investigate how information can be retrieved from


key: chroma:document
string_value:
attention in NLP. Further, [ 51] checks whether the attention
weights provide any interpretability by manipulating the
weights in pretrained text classification models. They used
an intermediate representation erasure method to demon-
strate that attention weights are unreliable predictors of the
relative significance of the specific input. They thus do
not accurately explain the model’s decision-making. Addi-
tionally, [ 53] employed a novel approach for visualizing


key: chroma:document
string_value:
the attention score for each token. This is the first study
on interpretability analysis by visualizing and scoring at
the word level. However, as explained in [ 34], the scoring
acquired using attention methods does not provide a mean-
ingful explanation. A more advanced scoring method known
as Masked Language Model (MLM) [ 48] uses pretrained
MLM to score sentences using pseudo-log-likelihood scores
(PLLs), which involves masking each token one by one.


key: chroma:document
string_value:
The method becomes unsuitable for scoring the entire
tokens of the dataset as the computational complexity rises.17467


key: chroma:document
string_value:
B.Bhattaraietal.
Likewise, recent keyword extraction (KE) algorithms such
as YAKE [ 13] and KeyBERT [ 26] are also used to extract
the top-scoring tokens from the trained model. To the best
of our knowledge, in novelty detection, there exists no such
method to measure each word’s contribution to the novelty.
In this study, we expand the study on novelty detection with
a method for scoring each word’s contribution to the over-
all novelty, which offers a clear view to the researchers for


key: chroma:document
string_value:
the reasoning and the interpretation of the results that the
algorithm offers.
3Tsetlinmachine(TM)architecture
The TM, proposed in [ 24], is a recent approach to pat-
tern classification, regression, and novelty detection [ 1,8,
25]. It captures the frequent patterns of the learning prob-
lem using conjunctive clauses in propositional logic. Each
clause is a conjunction of literals, where a literal is a propo-
sitional/Boolean variable or its negation. Recent research


key: chroma:document
string_value:
reports that the TM performs competitively with state-of-
the-art deep learning networks in text classification [ 7,
47,59,60] along with parallel and asynchronous architec-
ture [ 2] for faster learning across diverse tasks. Further,
theoretical studies have uncovered robust convergence prop-
erties [ 35,62].
A basic TM accepts a vector X=(x1,...,x o)∈{0,1}o
ofoBoolean features as input. For text input, it is typical
to booleanize the text to form a Boolean set of words, as


key: chroma:document
string_value:
suggested in [ 7]. The input features, together with their
negated counterparts, ¯x=¬x=1−x, form a literal
setL={x1,...,x o,¬x1,...,¬xo}. For classification
problems, the sub-patterns associated with the classes are
captured by the TM using mconjunctive clauses C+
jorC−
j.
Thej=1,...,m / 2 subscript denotes the clause index,
while the superscript indicates the polarity of a clause.
In brief, half of the clauses are assigned positive polarity,
i.e.,C+


key: chroma:document
string_value:
i.e.,C+
j, and the other half are assigned negative polarity,
i.e.,C−
j. The positive polarity clauses vote for the input
belonging to the class favored by the TM, while the negative
polarity clauses vote against that class, that is, for other
classes.
Ac l a u s e Cξ
j,ξ∈{ −,+},is formed by ANDing a subset
Lξ
j⊆Lof the literal set. That is, the set of literals for clause
Cξ
jwith polarity ξcan be written as:
Cξ
j(X)=⋀
l∈Lξ
jl=∏
l∈Lξ
jl.( 1 )


key: chroma:document
string_value:
Cξ
j(X)=⋀
l∈Lξ
jl=∏
l∈Lξ
jl.( 1 )
The clause evaluates to 1 if and only if all the literals of the
clause also evaluate to 1. For example, the clause Cξ
j(X)=
x1x2consists of the literals Lξ
j={x1,x2}and outputs 1, ifx1=x2=1. The final classification decision is obtained
by subtracting the negative votes from the positive votes,
and then thresholding the resulting sum using the unit step
function u:
ˆy=u⎛
⎝m/2∑
j=1C+
j(X)−m/2∑
j=1C−
j(X)⎞
⎠.( 2 )


key: chroma:document
string_value:
ˆy=u⎛
⎝m/2∑
j=1C+
j(X)−m/2∑
j=1C−
j(X)⎞
⎠.( 2 )
For example, the classifier ˆy=u(x 1¯x2+¯x1x2−x1x2−¯x1¯x2)
captures the XOR-relation.
For learning, the TM employs a team of Tsetlin Automata
(TA), one TA per literal l∈L. Each TA performs one of two
actions: either include orexclude its designated literal. Each
clause statistically forwards the feedback to its individual
TA. The TM employs Type I and Type II feedback. These
feedback types control the reward, penalty or inaction


key: chroma:document
string_value:
received by TAs depending on six factors: (1) target output
(y=0o ry=1), (2) clause polarity, (3) clause output
(Cj=0 or 1), (4) literals value ( x=1, or¬x=1),
(5) vote sum, and (6) the current state of the TA. Type I
feedback is designed to produce frequent patterns, while
Type II feedback increases the discriminating power of the
patterns (see [ 25] for details). The feedback guides the
complete system of TAs towards a Nash equilibrium. At any


key: chroma:document
string_value:
point in the training process, we have mconjunctive clauses
per class, half of them positive and half of them negative.
These can be retrieved and deployed upon completion of
training.
4Noveltydescription
By novelty description, we mean the task of characterizing
novel textual content at the word level. For instance, the
known content may be reviews of mobile phones, while
the novel content could be reviews of grocery stores. For
this example, one may define the novel content using words


key: chroma:document
string_value:
associated with grocery stores. However, describing novelty
at the word level is nontrivial because the meaning of
words varies depending on the context they appear in.
For example, consider the word “bat”. This word typically
manifests in two distinct contexts- it can denote either
“animal” or “sports”. Likewise, the word “bank” can refer to
“river bank” or “cash bank”. That is, when contextual
meaning is considered, the novelty of the word “bat” and


key: chroma:document
string_value:
“bank” can be different based on their respective uses.
As a result, measuring and describing novel content is a
challenging problem.
In general, one can detect and characterize novel content
by contrasting against the probability of observing textual
content X, given that the content is known. We denote
this probability distribution by pknown(X). Assume that
the corresponding probability distribution pnovel(X) for17468


key: chroma:document
string_value:
Word-levelhumaninterpretable...
novel content also is available. Then, the optimal novelty
detection test for a given false positive rate ( α) can be
obtained by thresholding the likelihood ratio pnovel(X) /
pknown(X)[39].
Since neither pknown(X) orpnovel(X) are available
to us, we must estimate them using training examples.
Inspired by the work in [ 9] on Semi-Supervised Novelty
Detection (SSND), we use two sets of examples. One
set represents known content, while the other represents


key: chroma:document
string_value:
novel content. We obtain these sets by employing a binary
classifier that can distinguish between known and novel
content, such as the one we proposed in [ 8].
4.1Identifyingnovelwordcandidates
In our approach, we begin by training a TM on input
texts represented as Boolean bag-of-words, i.e., as word
sets. A propositional variable represents each word in
the vocabulary, capturing the presence/absence of the
corresponding word in the input text. We group the texts into


key: chroma:document
string_value:
two classes, Known andNovel . The first represents known
content, and the second represents novel content. Our task
is to describe how the second group of text is novel at
the word level. To this end, we begin by identifying novel
word candidates, followed by scoring and ranking the words
based on their contribution to novelty.
Figure 2shows our architecture for identifying novel
word candidates. As seen, upon training, we obtain the
clauses of the two classes, Known andNovel . We extract all


key: chroma:document
string_value:
the words included in the clauses for each class. Each clause
contains a combination of both plain ( PL) and negated
(NL) words. As such, the plain and the negated words
serve two different roles. The plain words characterize the
corresponding class, while the negated words characterize
the other class. We exploit this property as follows, buildingtwo bag-of-words (BOW). The first is a bag of known
words, referred to as BK, and the second is a bag of novel
words, referred to as BN.


key: chroma:document
string_value:
words, referred to as BN.
For class Known , we perform the following procedure:
– We consider the words included in positive clauses first.
Here, the plain words PLare added to the bag of known
wordsBK, while the negated words are placed in the
bag of novel words BN.
– For negative clauses, we do the opposite. The plain
wordsPLare added to the novel words bag BN.T h e
negated words NL, on the other hand, are added to the
known word bag BK.
The above procedure is inverted for class Novel :


key: chroma:document
string_value:
The above procedure is inverted for class Novel :
– For the positive clauses, the plain words PLare added
to the novel word bag BN, while the negated words are
added to the known word bag BK.
– Conversely, for the negative clauses, the plain words are
added toBK, characterizing the known class, while the
negated words NLare added to BN.
4.2Scoringwordnovelty
With the word bags BKandBNavailable, we calculate
novelty scores at the word level as follows. From the


key: chroma:document
string_value:
unique words in the bags BKandBN, we produce
two corresponding word sets, SKandSN. Assume these
respectively contain KandNunique words:
SK={s1,s2,...,s k,...,s K},
SN={s1,s2,...,s n,...,s N}.( 3 )
Here, skrepresents a specific word in the set SK, while sn
represents a specific word in the set SN.
Text Literals
Binary encoded Literals
Positive
clausesPositive
clausesNegative
clausesNegative
clausesknown class novel class
Known BOW list Novel BOW listNovel BOW list Known BOW list


key: chroma:document
string_value:
B.Bhattaraietal.
We next estimate the occurrence probability psiof each
wordsiinSK, from the known class. The estimate is based
on the relative frequency of siin the word bag BKas given
by ( 4):
pK
si=FK
i∑K
k=1FK
k.( 4 )
Here,FK
iis the frequency of word siinBK, i.e., the number
of times that word sihas the appropriate role in one of
the clauses (as defined in the previous section). To prevent
infinite or zero scores, we assume that every word has a


key: chroma:document
string_value:
minimum frequency of 1. In the following, we denote the set
of relative frequencies for the words from BKbypK, while
pNis the set of relative frequencies for the words from BN,
as captured by ( 5):
pK={pK
s1,pK
s2,...,PK
sK},
pN={pN
s1,pN
s2,...,pN
sN}.( 5 )
The calculation of the novelty score for each word depends
on whether si∈SK,si∈SN, or both, as shown in ( 6):
Score(si)=⎧
⎪⎪⎨
⎪⎪⎩pN
si
pKsiifsk∈SK∩SN,
0i f sk∈SK\SN,
∞ ifsk∈SN\SK.(6)
Here, pN
siandpK
sidenote the estimated occurrence prob-


key: chroma:document
string_value:
siandpK
sidenote the estimated occurrence prob-
abilities of the word sifrompNandpK, respectively.
The score defines how much a word contributes in a sen-
tence/document to make it novel. That is, a higher score
signals higher novelty and vice versa. Figure 3shows the
resulting TM-based architecture and flow of information for
the above scoring approach.Additionally, we also propose a contextual scoring
approach to capture multiple word meanings determined


key: chroma:document
string_value:
by context. We presume that words that appear in the
same clause are related semantically, and accordingly, we
use clause co-occurrence of words to measure semantic
relations. The intent is to differentiate between, for example,
the meaning of “apple” in “apple phone” and the meaning of
“apple” in “apple fruit”. We achieve this through leveraging
clauses that capture “apple” and “phone” in combination
with other clauses that capture “apple” and “fruit”.


key: chroma:document
string_value:
The scoring is again performed in two steps:
1. Rather than measuring the frequency of individual
words, we now measure frequency of co-occurrence
among the TM clauses. For instance, let us consider
the word pair (s1,s2)and novel class, associated with a
total number of mclauses. The frequency of the word
pair occurring together in the clauses is then given as:
pN
s1,s2=FN
s1,s2
m.( 7 )
Here,FN
s1,s2is the number of times the word pair occur
together across the mclauses of the novel class.


key: chroma:document
string_value:
together across the mclauses of the novel class.
2. Finally, the contextual score for the word pair ( s1,s2)i n
class Novel can be defined as:
ScoreN
context(s1,s2)=pN
s1,s2
pNs1×pNs2.( 8 )
Above, pN
s2andpN
s1are the individual frequencies of
each word across the novel clauses, from the previous
subsection.
Notice how the above score increases with lower indi-
vidual frequencies and higher joint frequency, measuring
Relative frequency
calculator Relative frequency


key: chroma:document
string_value:
Relative frequency
calculator Relative frequency
calculator Known Class Novel Class
TMKnown Clauses
word list Novel Clauses
word list 
Score Generatorword in word in 
yes yes
word ScoreScore == very high
(novel word)Score == very low
(known word)No No
wordscore
Fig.3 Novelty scoring calculation for each word17470


key: chroma:document
string_value:
Word-levelhumaninterpretable...
dependence over the clauses. In the same way, we can cal-
culate dependence over the clauses for the known class as
well.
4.3Casestudy
We now demonstrate our novelty description approach,
steb-by-step, using two example sentences from the sports
domain. For illustration purposes, we consider the class
Cricket to be Known and the class Rugby to be Novel .
– Class : Cricket (Known)
Te x t : England won the cricket match by hitting six in
the last ball.


key: chroma:document
string_value:
the last ball.
Words :“England” ,“won” ,“cricket” ,“match” ,“hit” ,
“six” ,“ball” .
– Class : Rugby (Novel)
Text : England won the rugby match despite using old ball.
Words :“England” ,“won” ,“rugby” ,“match” ,
“despite” ,“old” ,“ball” .
We first create the set of 10 unique words W={“England” ,
“won” ,“cricket” ,“match” ,“hit” ,“six” ,“ball” ,“rugby” ,
“despite” ,“old”}from the words in the two sentences,
each with a unique index o. From this set, we produce the


key: chroma:document
string_value:
input feature vector for the TM, X=[x1,x2,...,x 10].
Each propositional input xoinXrefers to a particular word.
Jointly, the propositional inputs are used to represent an
input text. If a word wo∈Wis present in the document, the
corresponding propositional input xois set to 1, otherwise,
it is set to 0.
After TM training, we obtain a set of clauses, as exam-
plified in Table 1. The clauses (C+
1)K,(C+
2)K,(C−
1)N,
(C−
2)Nvote for class Known , while (C−
1)K,( C−
2)K,
(C+
1)N,( C+


key: chroma:document
string_value:
1)K,( C−
2)K,
(C+
1)N,( C+
2)Nvote for class Novel . These clauses are
then used to produce two bag-of-words, BKandBN.A l l
the plain words in (C+
1)K,(C+
2)K,(C−
1)N,(C−
2)Nare
placed in BK, while all the negated words are placed in
BN. Since none of the words are negated in the clauses,
we now have BK=(“England” ,“cricket” ,“match” ,“hit” ,
“six” ,“cricket” ,“six” ,“cricket” ,“won” ,“six” ,“ball” ,
“cricket” ,“hit” ,“six” ). Correspondingly, all the plain
words in (C−
1)K,( C−
2)K,( C+


key: chroma:document
string_value:
words in (C−
1)K,( C−
2)K,( C+
1)N,( C+
2)Nare placed in
BN, while all the negated words are placed in BK.Within each bag-of-words, each word occurs with a
certain frequency. For instance, the word “match” occurs
once inBKand twice in BN. Notice that the total number
of word occurrences are different for each class – 14 words
in class Known and 13 words in class Novel . Hence, the
relative frequency for “match” in class Known becomes
pK
match=1
14=0.071 while for class Novel it becomes
pN


key: chroma:document
string_value:
14=0.071 while for class Novel it becomes
pN
match=2
13=0.154. Table 2lists the frequencies of the
words per class.
We are now ready to calculate the novelty score for each
word in W. Let us consider the word “rugby” from the novel
word set and the word “cricket” from the known word set.
For “rugby”, we first calculate its relative frequency ( 4). In
the bag-of-word BNfor class Novel , “rugby” occurs four
times, i.e., FN
rugby=4 .S i n c ew ea s s u m et h a taw o r dh a s


key: chroma:document
string_value:
a minimum frequency of 1, we further have FK
rugby=1,
despite “rugby” not appearing in the text from class Known .
From Table 2, we observe that the total word frequencies
for the known and novel classes are 14 and 13, respectively.
Hence, the relative frequencies for “rugby” becomes
prugby(K)=0.307 for class Known andprugby(N)=
0.071 for class Novel (4).
Because the clauses characterize each class Known and
Novel , notice how “rugby” gets the relatively high novelty


key: chroma:document
string_value:
scoreScore rugby =4.651. That is, its relative frequency
is high in the novel class and low in the known class.
Conversely, the word “cricket” is repeated four times in
BKandonce inBN. Its relative frequencies thus becomes
pcricket(K)=0.28 for class Known andpcricket(N)=0.076
for class Novel . Accordingly, the novelty score becomes
Score cricket =0.271, which is a low score denoting a
strong inclination of the word towards the known class.
Overall, Table 2shows how the words characterizing


key: chroma:document
string_value:
class Known get a relatively low novelty score, while those
characterizing class Novel o b t a i nh i g hs c o r e s .
5Resultsanddiscussions
In this section, we evaluate our proposed novelty descrip-
tion approach on two publicly available datasets: BBC
Sports andTwenty Newsgroups . The performance of the TM
framework for novelty detection was previously investigated
Table 1 Clauses with conjunctive word patterns for known and novel class
Known Clauses Novel Clauses
(C+


key: chroma:document
string_value:
Known Clauses Novel Clauses
(C+
1)K=“England” ∧“cricket” ∧“match” ∧“hit”∧“six” (C+
1)N=“England” ∧“won” ∧“rugby” ∧“old”
(C−
1)K=“won” ∧“rugby” ∧“ball” (C−
1)N=“cricket” ∧“won” ∧“six”∧“ball”
(C+
2)K=“cricket” ∧“six” (C+
2)N=“rugby” ∧“match” ∧“despite” ∧“old”
(C−
2)K=“rugby” ∧“match” (C−
2)N=“cricket” ∧“hit”∧“six”17471


key: chroma:document
string_value:
B.Bhattaraietal.
Table 2 Relative frequency and score for each word
Known Novel
Word Frequency Relative frequency Score Word Frequency Relative frequency Score
England 1 0.071 1.070 England 1 0.076 1.070
Won 1 0.071 2.169 Won 2 0.154 2.169
Cricket 4 0.28 0.271 Rugby 4 0.307 4.651
Match 1 0.071 2.169 Match 2 0.154 2.169
Hit 2 0.142 0.535 Despite 1 0.076 1.15
Six 4 0.28 0.271 Old 2 0.153 2.31
Ball 1 0.071 1.070 Ball 1 0.076 1.070
in [8] and is summarized in Table 3. Notably, as has


key: chroma:document
string_value:
been found across several datasets, a one-class SVM on
the simple mean embeddings established a strong base-
line. Here, we further explore our model’s effectiveness
at producing discriminative novelty scores at the word
level using TM clauses. To obtain robust performance and
ensure that the results are not influenced by the data, we
perform a one-class classification using leave-one-out eval-
uation on 20 Newsgroup dataset. This paper deals with


key: chroma:document
string_value:
the post-processing after novelty detection to deal with the
novelty scoring at the word level. However, the leave-one-
out evaluation is necessary because this study leverage the
performance of the TM framework in terms of novelty
detection. We employ the ROC AUC to quantify the nov-
elty detection performance by using the ground truth labels
during testing. Table 4shows the performance compari-
son of our method and the baseline algorithms, including


key: chroma:document
string_value:
a one-class classifier. In the leave-one-out setup, one of
the classes is considered a known class, while the remain-
ing classes are treated as novel. The training is conducted
using a known class, whereas testing is carried out on
samples from a novel class. The ROC AUC is computed
during testing with the assumption that the samples from
the known class are labeled as y=0 and from novel
class as y=1. Our method outperforms baselines algo-


key: chroma:document
string_value:
rithms in five out of six evaluation setups with a significant
margin.
In the following, we compare the scoring mechanism of
our framework with attention and TF-IDF as a baseline.
To ensure a fair comparison, the attention score for each
word is calculated as described in Section 5.1.1 .F o r
TF-IDF, We calculate TF separately for the known and
novel classes. Conversely, IDF is calculated using all the
documents from both classes (to suppress common words


key: chroma:document
string_value:
such as stop words). Unlike attention and TF-IDF, even if
a word is present in most documents, our scoring considers
both relevance and context. For example, if a word from
class Novel also is present in class Known , our modelcan nevertheless assign more weight to that word. This
happens when a word, while syntactically the same in both
classes, acquires a novel meaning in the novel class due
to its appearance in a novel context. The latter contextual


key: chroma:document
string_value:
information is captured through those clauses of the novel
class that trigger for that word. As such, attention and
TF-IDF are not context-aware. Moreover, these methods
prove especially beneficial on more extensive datasets, such
as 20 Newsgroups and BBC Sports, since they filter out
general language contexts that are less discriminative for
the characterization of a text corpus, making them a strong
baseline for performance comparison.
To provide a comparison, we plot the cumulative


key: chroma:document
string_value:
To provide a comparison, we plot the cumulative
frequency distribution (CFD) for the scores of (1) the words
only found in the novel dataset, (2) the words only found
in the known dataset, and (3) the words shared by both
datasets. In brief, the CFD demonstrates that the word scores
generated by the baseline are relatively similar for both
known and novel classes. Thus, the baseline methods lack
the discriminatory power necessary to distinguish between
the two categories of words.


key: chroma:document
string_value:
the two categories of words.
Table 3 Performance comparison of TM framework with cluster and
outlier-based novelty detection algorithms
Algorithms 20 Newsgroup BBC sports
LOF 52.51 % 47.97 %
Feature Bagging 67.60 % 54.38 %
HBOS 55.03 % 49.53 %
Isolation Forest 52.01 % 49.35%
Average KNN 76.35 % 55.54 %
K-Means clustering 81.00 % 47.70 %
One-class SVM 83.70 % 83.53 %
SO-GAAL 80.2% 83.50%
MO-GAAL 82.9% 86.68%
COPOD 84.4% 86.09%
TM framework 82.50% 89.47%17472


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Table 4 ROC AUC (%) of one-class classification with leave-one-out evaluation on 20 Newsgroup
Normal class ABOD CBLOF HBOS IForest KNN LOF OCSVM COPOD TM
comp 0.506 0.618 0.625 0.62 0.622 0.62 0.614 0.627 0.55
rec 0.508 0.481 0.476 0.483 0.479 0.48 0.48 0.476 0.60
sci 0.50 0.435 0.449 0.454 0.434 0.435 0.433 0.45 0.53
misc 0.511 0.533 0.527 0.534 0.54 0.542 0.534 0.532 0.69
pol 0.492 0.452 0.436 0.445 0.451 0.451 0.454 0.435 0.71


key: chroma:document
string_value:
rel 0.494 0.449 0.437 0.456 0.472 0.47 0.457 0.443 0.63
5.1Baseline
5.1.1Attentionmechanism
We utilize the weights from attention’s layers input repre-
sentation Aof the trained model. The importance of each
token is calculated based on the attention it receives. For
instance, if attention to the token c∈Ais higher than the
token d∈A,t h e n cis assumed to be “more significant”
thandto the model’s output. In our work, the scores are
calculated using scaled-dot product attention mechanism
[57].


key: chroma:document
string_value:
[57].
Let us consider an input sequence of length o,X=
(x1,x2,...,x o),w h e r e xirepresents the ithtoken whose
representation in the attention layer is hi∈Rt. The attention
score for the ithtoken is as follows:
αi=hi×V
β, (9)
where the parameter βis the scaling factor, and V∈Rt
is the context vector that can be seen as a fixed query
requesting the “most important token” from input. Either the
word embedding or the encoder’s output can denote token


key: chroma:document
string_value:
representation hi. The attention weight can be expressed as:
ai=exp(a i)∑
i′exp(a i′). (10)
Finally, the complete input sequence is denoted as:
h=∑
i(aihi). (11)
In our experiment, we retrieve the attention score and
weights for each token using ( 9)a n d( 10) respectively.
We conducted experiments using scaled dot-product
attention ( DP) and additive attention with varying scaling
factors ( β). The attention scores in our experiments are
generated using a Long short-term memory (LSTM) with


key: chroma:document
string_value:
DP and an affine transformation layer as the input encoder.
We used the Adagrad optimizer [ 19] for gradient descent
and used dropout as regularization to prevent over-fitting.
To eliminate the influence of prior knowledge, we learn all
parameters from scratch and initialize the pre-trained word
embeddings with a uniform distribution and dimension d=100. A softmax function is applied over a linear layer for
obtaining the final classification output. The readers are


key: chroma:document
string_value:
referred to [ 53] for a detailed theoretical explanation to
generate the attention scores.
5.1.2Termfrequency-inversedocumentfrequency(TF-IDF)
A commonly used method to analyze the importance of
a word is the term frequency-inverse document frequency
(TF-IDF) [ 45]. TF-IDF weighs each word to statistically
measure the significance of the word in a given document.
To this end, TF-IDF consists of two factors: normalized
term frequency (TF) and inverse document frequency (IDF).


key: chroma:document
string_value:
TF measures the frequency of the word in the document,
whereas IDF measures the uniqueness of the word across
documents:
TF−IDF s=Fs
F×log2|D|
|Ds|+1. (12)
Here,Fsis the frequency of the word sin the target
document, Fis the sum of the target document word
frequencies, |D|is the total number of documents, and |Ds|
is the number of documents containing the word s.
5.1.3Keywordextractionalgorithms
Our method extracts keywords from known and novel


key: chroma:document
string_value:
Our method extracts keywords from known and novel
classes based on the novelty scores. As a result, we also
compare the significant words obtained by our method
to those captured by existing keyword extraction (KE)
algorithms. To do this, we first separate the text documents
from known and novel classes before passing them to
the KE algorithms. Additionally, we present the top
10 keywords captured by these algorithms. For the KE
baselines mentioned below, we use the pkepackage [ 10]:


key: chroma:document
string_value:
– TopicRank [ 12]: This is a graph-based KE method that
depends on the extraction of the top-ranked topic.
– YAKE [ 13]: A lightweight statistical approach for KE.
– MultipartiteRank [ 11]: An unsupervised KE method for
encoding topical information in a multipartite graph
structure.17473


key: chroma:document
string_value:
B.Bhattaraietal.
– BERT-MMR [ 26]: A KE method that leverages Bidi-
rectional Encoder Representations from Transformers
(BERT) embeddings and Maximal Marginal Relevance
(MMR).
5.2Evaluationmeasures
We use the accuracy, Receiver Operating Characteristics
(ROC) curve, precision, and recall to evaluate the perfor-
mance of novelty detection using word scores obtained from
the proposed method. In general, accuracy is a well-known
parameter to measure the effectiveness of novelty detection


key: chroma:document
string_value:
models, which indicates the percentage of correct prediction
by a model in a test set. The accuracy is calculated by:
Accuracy =TP+TN
TP+TN+FP+FN, (13)
where TP,TN,FP,FNdenotes the samples that are
correct novel, correct normal, incorrect novel, and incorrect
normal respectively. And P, N denotes the total novel and
normal samples. The precision is defined by the percentage
of correctly identified novel samples and is give as:
Precision =TP
TP+FP. (14)


key: chroma:document
string_value:
Precision =TP
TP+FP. (14)
Recall is the percentage of the real novel samples identified
and is given as:
Recall =TP
TP+FN. (15)
In general, the higher the precision and the recall, the
better the algorithm. However, the precision and recall are
mutually constrained. For example, if only one novel sample
is detected, the precision is 100%, while the recall is very
low. And if all samples detected are novel, the recall will be
100%, while precision tends to be very low. Therefore, we


key: chroma:document
string_value:
present precision-recall graph in our evaluation.
The ROC is insensitive to the number of novel samples
and is calculated by plotting all potential choices of the
TPrate (the portion of novel data ranked among the total
novel data) against the FPrate (the portion of normal data
ranked among the total novel data). The ROC curve can
be summarized using a single value defined as the area
under ROC curve (AUC). The ROC value ranges between 0


key: chroma:document
string_value:
and 1 and is regarded as average of the recall. The perfect
detection of all test samples would result in ROC value of
1, whereas the null detection would result in ROC value of
0. In general, the greater the ROC AUC value, the better
the algorithm. [ 27] established that the ROC AUC value
corresponds to the probability of a pair (nov, nor) ,w h e r enov is certain true novel samples and nor is certain true
normal samples. The ROC AUC can then be defined by:
ROC AUC =⎧
⎨


key: chroma:document
string_value:
ROC AUC =⎧
⎨
⎩1,ifscore(nov) > score(nor),
0,ifscore(nov) < score(nor),
1/2,ifscore(nov) =score(nor) .
Therefore, the ROC AUC has a direct probabilistic
interpretation. The AUC can be also be defined as:
AUC =∫1
0ROC(T) d T, (16)
where Tdenotes a threshold to control novel samples. The
ROC AUC is the most often used evaluation metric for
novelty detection that provides a ranking. Therefore, in this
paper, we compare the evaluation alongside other methods,


key: chroma:document
string_value:
so it can give different aspects of the performance. To ensure
fairness, effectiveness and reproducibility of the evaluation
results, we use scikit-plot library1to compute ROC and
precision-recall graphs.
5.3BBCsportsdataset
The BBC sports dataset comprises 737 documents from
the BBC sport website organized in five sports article
categories, collected from 2004 to 2005. The resulting
vocabulary encompasses 4 613 terms. For our experiment,
we consider the classes “cricket” and “football” to be


key: chroma:document
string_value:
known and the class “rugby” to be novel, thus creating
an unbalanced dataset. For preprocessing, we perform
tokenization, stopword removal, and lemmatization. We run
the TM for 100 epochs with 10 000 clauses, a voting margin
Tof 50, and a sensitivity sof 25.0.
We present overall novelty score statistics for the words
captured by the clauses in Table 5. The table demonstrates
that words in the class Novel have distinctively higher
average scores than words in the class Known .A l s o ,


key: chroma:document
string_value:
notice that the shared words have the highest mean and
standard deviation. As analyzed further below, this is the
case because the TM will mainly use those words when
forming the decision boundary between the two classes. As
a result, the shared words will appear in more clauses as
characterizing class features. That is, the clauses will either
single out the words in one class or suppress the words in
the other class.
To gain further insight into the properties of the novelty


key: chroma:document
string_value:
score, we plot the CFD for the scores of the novel, known,
and shared words in Fig. 4. We further compare these
CFDs with the corresponding ones obtained using attention
weights in Fig. 5a n dT F - I D Fi nF i g . 10. As can be observed
1https://scikit-plot.readthedocs.io/en/stable/Quickstart.html.17474


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Table 5 Overall word statistics for BBC sport dataset
Category Total word count Average score Standard deviation
Known words 6660 0.74 0.23
Novel words 1941 1.3125 3.75
Shared words 3135 11.30 316.93
from the plot, our approach produces more distinctive
novelty scores than both attention and TF-IDF. The novel
words typically produce high scores, while the known words
produce low scores. In particular, as shown in Fig. 4a, 85%


key: chroma:document
string_value:
of the known words output scores lower than 1.0. On the
other hand, as seen in Fig. 4b, only approximately 45% of
the words unique for the novel class have scores below 1.
The majority of the uniquely novel words produce scores
greater than 1.
We plot the TM and attention scores for each token in
Fig. 6b and a, respectively. Due to the large span of the
TM scores, the y-axis is plotted on a log scale. Nonetheless,
we note that the scores are structured in successive layers,


key: chroma:document
string_value:
with known scores at the bottom, novel scores at the top,and shared scores in the center. We notice that even the
attention score demonstrates a small degree of differenti-
ation between known and novel categories. However, the
variability of the score is quite low when compared to the
score generated by TM as seen in Fig. 7boxplot. Addi-
tionally, the shared word scores produced by the attention
mechanism exhibit a high degree of resemblance to known
word scores.


key: chroma:document
string_value:
word scores.
Finally, we plot the scores for words that are shared
between the known and novel classes in Fig. 4c. As can
be observed, the words that are shared produce both high
and low scores. To cast further light on this observation, we
investigate the words that are shared further in Table 6.W e
see that the words captured frequently by novel clauses have
high scores, whereas the words that are frequent in known
clauses have low scores. Additionally, common words (e.g.,


key: chroma:document
string_value:
stopwords), also have low scores. For example, the word
“Rugby”, which is highly characteristic for class Novel ,
is repeated only 5 times in the clauses representing class
Known . For the clauses that represent class Novel ,o nt h e
other hand, it is repeated 215 times. In other words, the
Fig.4 Cumulative frequency distribution (CFD) graph for word scores in different categories of BBC Sports using TM17475


key: chroma:document
string_value:
B.Bhattaraietal.
Fig.5 Cumulative frequency distribution (CFD) graph for word scores in different categories of BBC Sports using attention weights
shared words constitute words that are either characteristic
for class Known or class Novel . This finding also suggests
that the scores can be calculated accurately even if the
words are present in both categories. We analyze the mostfrequently used words to obtain an intuition of the overall
theme captured by the clauses. We generate such lists by


key: chroma:document
string_value:
counting the top words according to the highest scores
from known and novel classes. Such a list may assist a
Fig.6 Visualization of tokens in known, Novel and Shared categories from BBC Sports17476


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Fig.7 Boxplot of scores in known, Novel and Shared categories from BBC Sports
Table 6 Composition of shared words in BBC Sport
Composition Total word count Average score Standard deviation
Known words 10 0.11 0.070
Novel words 17 1941.13 3919.02
Common words 3051 1.03 0.99
Table 7 Example of top words extracted from KE baselines for the Known class in BBC Sports
TM TopicRank YAKE MultipartiteRank BERT-MMR
Word Score Word Score Word Score Word Score Word Score


key: chroma:document
string_value:
manchester 0.004 players 0.0065 said 2.65 players 0.0053 kickoff 0.25
manager 0.040 ball 0.0050 game 5.16 games 0.0050 fifaasian 0.29
arsenal 0.041 team 0.0050 england 6.48 goal 0.0038 espnstar 0.28
united 0.043 goal 0.0043 chelsea 7.01 chelsea 0.0029 fifa 0.24
cricket 0.046 chelsea 0.0038 players 7.08 team 0.0029 juventus 0.23
chelsea 0.049 wickets 0.0030 team 7.44 manchester 0.0026 matchwinner 0.22
oneday 0.051 oneday 0.0029 oneday 8.35 arsenal 0.0024 sportsweek 0.34


key: chroma:document
string_value:
striker 0.074 england 0.0027 united 8.39 ball 0.0023 goalkick 0.20
batsman 0.114 arsenal 0.0027 league 8.39 england 0.0022 clubmate 0.12
bowler 0.133 tests 0.0026 arsenal 9.62 matches 0.0021 autobiography 0.1317477


key: chroma:document
string_value:
B.Bhattaraietal.
Table 8 Example of top words extracted from KE baselines for the novel class in BBC Sports
TM TopicRank YAKE MultipartiteRank BERT-MMR
Word Score Word Score Word Score Word Score Word Score
rugby 11143.58 nations 0.012 england 0.006 nations 0.0089 nflstyle 0.29
nations 10000 game 0.011 rugby 0.101 england 0.0082 july 0.05
ireland 468.18 player 0.009 wales 0.112 game 0.0074 wednesday 0.08
flyhalf 54.79 side 0.007 nations 0.113 wales 0.0071 rugby 0.17


key: chroma:document
string_value:
lions 38.18 wales 0.007 ireland 0.113 player 0.0064 wordclass 0.06
scrumhalf 25.09 ireland 0.006 game 0.125 side 0.0048 dropkicking 0.12
flanker 19.61 years 0.006 coach 0.146 team 0.0038 goalkickers 0.21
irish 17.55 team 0.005 side 0.182 france 0.0036 sportsweek 0.28
centre 17.19 ball 0.004 players 0.206 win 0.0033 tournament 0.13
squad 9.28 win 0.003 win 0.220 squad 0.0030 kickoff 0.23


key: chroma:document
string_value:
Fig.8 ROC curve and precision-recall of known/novel class classification of BBC Sports using word scores obtained from TM
Fig.9 ROC curve and precision-recall of known/novel class classification of BBC Sports using attention scores17478


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Fig.10 Cumulative frequency distribution (CFD) graph for TF-IDF scores in different categories of BBC Sports
Fig.11 ROC curve and precision-recall of known/novel class classification of BBC Sports using TF-IDF scores17479


key: chroma:document
string_value:
B.Bhattaraietal.
Table 9 Overall word statistics for 20 Newsgroups dataset
Category Total word count Average score Standard deviation
Known words 23133 0.99 0.21
Novel words 6921 1.20 1.04
Shared words 5786 3.04 131.62
Table 10 Composition of shared words in 20 Newsgroups dataset
Composition Total word count Average score Standard deviation
Known words 9 0.14 0.074
Novel words 33 640.75 2378.87
Common words 5697 1.11 0.58


key: chroma:document
string_value:
Common words 5697 1.11 0.58
Fig.12 Cumulative frequency distribution (CFD) graph for word scores in different categories of 20 Newsgroups using TM17480


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Fig.13 Cumulative frequency distribution (CFD) graph for word scores in different categories of 20 Newsgroups using attention weights
user in weighting and selecting relevant words in a specific
application.
Tables 7and8show an example of a top word list for
each class, from which we make the following observations.First, our proposed method assigns low scores to words
belonging to known classes while assigning comparatively


key: chroma:document
string_value:
high scores to words belonging to the novel class. In
general, the words that appear in a novel context are
Table 11 Example of top words extracted from KE baselines for the known class in 20 Newsgroups
TM TopicRank YAKE MultipartiteRank BERT-MMR
Word Score Word Score Word Score Word Score Word Score
program 0.103 people 0.074 image 4.60 people 0.054 lawbook 0.44
gun 0.104 gun 0.041 gun 6.77 guns 0.033 interpolation 0.42
use 0.117 article 0.038 people 7.19 article 0.027 literature 0.41


key: chroma:document
string_value:
write 0.145 fire 0.032 file 7.25 government 0.026 reading 0.41
public 0.154 government 0.032 article 7.41 fire 0.022 writing 0.41
file 0.175 image 0.027 like 10.1 time 0.019 translation 0.41
image 0.187 fbi 0.021 jpeg 13.1 day 0.018 lexidata 0.40
email 0.074 weapons 0.019 think 15.6 weapons 0.017 prisoncamp 0.17
police 0.358 problem 0.018 time 16.6 fbi 0.016 comparable 0.40
fbi 0.231 information 0.017 program 17.4 year 0.014 literate 0.4017481


key: chroma:document
string_value:
B.Bhattaraietal.
Table 12 Example of top words extracted from KE baselines for the novel class in 20 Newsgroups
TM TopicRank YAKE MultipartiteRank BERT-MMR
Word Score Word Score Word Score Word Score Word Score
team 450.66 game 0.082 writes 1.48 game 0.065 baseball 0.38
baseball 243.62 team 0.074 game 1.55 team 0.048 reporters 0.33
pitcher 62.26 player 0.055 article 2.29 pitch 0.037 salaries 0.32
league 55.88 year 0.053 team 2.30 player 0.037 basketball 0.31


key: chroma:document
string_value:
season 40.50 baseball 0.049 last 2.95 baseball 0.034 fittest -0.07
hit 24.63 pitcher 0.042 baseball 3.41 pitcher 0.026 kickoff 0.15
play 22.68 ball 0.037 player 3.44 ball 0.026 nba 0.10
batting 22.25 runs 0.030 time 3.94 runs 0.022 secretive 0.30
pitching 22.00 season 0.030 hit 4.06 season 0.020 jerseys 0.30
player 21.86 braves 0.030 run 4.66 braves 0.019 catchers 0.30
boosted. Second, the words that are most representative
for the respective classes are captured frequently by


key: chroma:document
string_value:
clauses, making them the most repeated ones. Third, the
keywords captured by other KE baselines are comparable
to those extracted by our method and accurately define the
corresponding classes. We observe that TopicRank, YAKE,
and MultipartiteRank all yield words with a high degree
of similarity to our approach. Additionally, we notice that
BERT-MMR exhibits the worst performance. This might
be due to the fact that we utilized pre-trained sentence


key: chroma:document
string_value:
embedding for BERT, and the keywords are extracted from
overall documents. Even though the words are not highly
relevant to the classes, BERT is capable of producing words
relating to the class’s general theme. For example, sports-
related words are included in both classes.
We now investigate the degree of discrimination power
our novelty scoring provides, and therefore uniquely


key: chroma:document
string_value:
describes novelty at the word level. To this end, we employlogistic regression for classifying novel text based on the
word scores obtained from our method. The ROC and
precision-recall curves of the experiment are depicted in
Fig. 8for our novelty scoring mechanism. Our method
provides the competitive ROC value due to its ability to
discriminate novel samples based on their scores. This capa-
bility enables our method to acquire a higher true positive


key: chroma:document
string_value:
TPrate since it makes separate analysis of both correct
novel, i.e., true positive TPand correct normal, i.e., true
negative TN. Figures 9a,10,a n d 11contains corresponding
curves when TF-IDF and attention scores are used instead.
We see that the classification performance for our nov-
elty scores is substantially better than what is obtained
with TF-IDF. Attention score outperforms our approach for
BBC Sports dataset by a small percentage. However, our


key: chroma:document
string_value:
approach outperforms attention in 20 Newsgroups. This can
be attributed to the capability of our approach to deal with a
big dataset.
Fig.14 Visualization of tokens in known, Novel and Shared categories from 20Newsgroups17482


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Fig.15 Boxplot of scores in known, Novel and Shared categories from 20Newsgroups
Fig.16 ROC curve and precision-recall of known/novel class classification of 20 Newsgroups using word scores obtained from TM
Fig.17 ROC curve and precision-recall of known/novel class classification of 20 Newsgroups using attention scores17483


key: chroma:document
string_value:
B.Bhattaraietal.
Fig.18 Cumulative frequency distribution (CFD) graph for TF-IDF scores in different categories of 20 Newsgroups
Fig.19 ROC curve and precision-recall of known/novel class classification of 20 Newsgroups using TF-IDF scores17484


key: chroma:document
string_value:
Word-levelhumaninterpretable...
Table13 Co-occurrence matrix showing the information gain between
words in BBC Sports
Manchester Chelsea Particular Rugby Flyhalf
Manchester 14363.688 6.324 4.738 0.33 0.848
Chelsea 6.324 19801.49 6.18 0.466 1.326
Particular 4.738 6.18 30863.006 2.52 4.968
Rugby 0.33 0.466 2.52 486.758 3.952
Flyhalf 0.848 1.326 4.968 3.952 8888.888
5.420newsgroupsdataset
The 20 Newsgroups dataset contains a total of 18 828
documents partitioned equally into 20 separate classes. In


key: chroma:document
string_value:
our experiments, we treat the two classes “comp.graphics”
and “talk.politics.guns” as Known topics, and then use the
class “rec.sport.baseball” to represent a Novel topic. Again,
we train a TM to produce our clause-based novelty scores.
The overall statistics of the resulting word scores are shown
in Tables 9and10, where we observe similar behavior to
that observed with the BBC Sports dataset.
The CFD plot in Fig. 12presents the score distribution


key: chroma:document
string_value:
among words per group (known, novel, shared). For known
words, in Fig. 12a, we find that 90% of the scores of the
words are below around 1.3. In Fig. 12b, however, only 45%
of the novel word scores fall below approx. 1.3. From the
plots, it is evident that the majority of the novel words have
considerably higher scores than the known words. Note that
some of the novel word’s low scores are attributable to the
presence of common words (e.g., stop words) in the novel


key: chroma:document
string_value:
bag-of-words. Since the common words, as such, do not
signify novelty, the TM clauses do not frequently capture
them. As a result, they receive relatively low scores despite
their appearance among the novel documents.
The CFD plot for attention and TF-IDF both exhibit
similar behaviour to that of BBC Sports, as seen in Fig. 13
and 18, respectively. Finally, we again observe that the
clauses have used the shared words for discrimination (cf.


key: chroma:document
string_value:
Table 10), resulting in a mix of low and high novelty scores,
a ss h o w ni nF i g . 12c.
Table14 Co-occurrence matrix showing the information gain between
words in 20 Newsgroup
Guns Weapon Gather Baseball Player
Guns 12302.96 17.648 15.754 4.036 4.268
Weapon 17.648 13888.888 12.108 4.66 5.102
Gather 15.754 12.108 14610.272 11.854 15.408
Baseball 4.036 4.66 11.854 4003.824 18.566
Player 4.268 5.102 15.408 18.566 9255.402Table15 Co-occurrence matrix showing the similarity between words


key: chroma:document
string_value:
in BBC Sports using Word2Vec
Manchester Chelsea Particular Rugby Flyhalf
Manchester 1 0.782 0.653 0.598 0.718
Chelsea 0.782 1 0.891 0.820 0.829
Particular 0.653 0.891 1 0.821 0.941
Rugby 0.598 0.820 0.821 1 0.706
Flyhalf 0.718 0.829 0.941 0.706 1
Table 11and Table 12provide examples of the highest-
scoring words captured by KE baselines, including TM, for
both classes. The visualization of the scores are presented
in Figs. 14and15. Again, we observe a similar behavior as


key: chroma:document
string_value:
for the BBC Sports dataset. The ROC and precision-recall
curves for our novelty scoring mechanism are illustrated in
Figs. 16a,17a,18,a n d 19a include corresponding graphs
when TF-IDF and attention scores are used instead. Our
method outperforms the ROC value obtained from attention
because of its ability to identify more number of correct
novel samples, i.e., true positives TP. However, the TF-IDF
surprisingly outperforms both of the methods because of its


key: chroma:document
string_value:
straightforward scoring system and the dataset’s moderate
size. We can see that our scoring approach outperforms the
baselines by a wide margin.
5.5Contextualscoring
We also implement a context-based scoring approach to
investigate how multiple words interact to capture novelty.
As detailed in Section 4, we compute the combined novelty
score by measuring word co-occurrence in clauses. That
is, we intend to demonstrate how context can help uncover


key: chroma:document
string_value:
novelty when words have multiple meanings. The context-
based scoring is critical since the context can transform
the word from being novel to known, such as the meaning
of the word “apple” in “apple fruit” and “apple phone”.
For demonstration, we calculate our proposed context-
based novelty score for five words (i.e., two known, two
novel, and one common word) in both datasets. For the
BBC Sports dataset, the pairwise co-occurrence scores


key: chroma:document
string_value:
Table16 Co-occurrence matrix showing the similarity between words
in 20 Newsgroup using Word2Vec
Guns Weapon Gather Baseball Player
Guns 1 0.709 0.726 0.673 0.701
Weapon 0.709 1 0.648 0.454 0.539
Gather 0.726 0.648 1 0.631 0.686
Baseball 0.673 0.454 0.631 1 0.764
Player 0.701 0.539 0.686 0.764 117485


key: chroma:document
string_value:
B.Bhattaraietal.
are presented in Table 13. We see a significant degree
of correspondence between words such as “Manchester”
and “Chelsea” from class Known . Similarly, there is a
high correspondence between words such as “Rugby” and
“Flyhalf” from class Novel . The common word “Particular”,
on the other hand, shows similar correspondence with words
from both of the classes. Similarly, for the 20 Newsgroups
dataset, the co-occurrence scores for five words selected


key: chroma:document
string_value:
from the known, novel, and common word types are shown
in Table 14. The words “Guns” and “Weapon” are from class
Known and manifest strong co-occurrence. Additionally the
words “Baseball” and “Player” from class Novel correspond
strongly as well. The common word “Gather”, on the other
hand, co-occurs within both of the classes. These examples
demonstrate that the words that are most likely to appear in
the same context have a high co-occurrence score. This can


key: chroma:document
string_value:
be explained by the fact that many clauses capture words
that frequently occur together in a similar context.
We compare the contextual scores obtained from our
method with the Word2Vec similarity score. To do this, we
utilize Gensim library to train custom Word2Vec on both
datasets. Gensim library enabled us to create word embed-
dings by training own Word2Vec models on a custom corpus
using either CBOW or skip-grams algorithms. Parameter-
wise, we used an embedding size of 200 and a win-


key: chroma:document
string_value:
wise, we used an embedding size of 200 and a win-
dow size of 5. We compute the cosine similarity between
words by using their word vectors (embeddings). The
findings are included in Tables 15and 16. We notice
a significant degree of resemblance between the correspond-
ing words from the known and the novel classes. However,
unlike our method, the similarity scores are less distinct, and
the common words are not discernible score-wise.
6Conclusion


key: chroma:document
string_value:
6Conclusion
In this work, we propose a Tsetlin Machine (TM)-based
solution for word-level novelty description. First, we
employ the clauses from a trained TM to capture how
the most significant words differentiate a group of novel
documents apart from a group of known documents. Then,
we calculate the score for each word based on the role
it plays in the clauses. The analysis of our empirical
results for BBC Sports and 20 Newsgroups demonstrate


key: chroma:document
string_value:
significantly better novelty discrimination power when
compared to using attention and TF-IDF. Our empirical
results also show that we can capture word relations
through a contextual scoring mechanism that measures co-
occurrence within TM clauses. By capturing non-linear
relationships among words, we can enhance the capability
of measuring novelty at the word level. However, training
a TM is computationally more expensive than calculating


key: chroma:document
string_value:
TF-IDF, particularly for large datasets with an extensivevocabulary. We will address computation speed in our future
work, employing indexing mechanisms and exploiting
feature space sparsity.
Funding Open access funding provided by University of Agder.
Declarations
ConﬂictofInterests The authors declare that they have no conflict of
interest.
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing,


key: chroma:document
string_value:
adaptation, distribution and reproduction in any medium or format, as
long as you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons licence, and indicate
if changes were made. The images or other third party material in this
article are included in the article’s Creative Commons licence, unless
indicated otherwise in a credit line to the material. If material is not
included in the article’s Creative Commons licence and your intended


key: chroma:document
string_value:
use is not permitted by statutory regulation or exceeds the permitted
use, you will need to obtain permission directly from the copyright
holder. To view a copy of this licence, visit http://creativecommons.
org/licenses/by/4.0/ .
References
1. Abeyrathna KD, Granmo O, Jiao L, Goodwin M (2019)
The regression tsetlin machine: A tsetlin machine for con-
tinuous output problems. In: Oliveira PM, Novais P, Reis
LP (eds) Progress in Artificial Intelligence, 19th EPIA Con-


key: chroma:document
string_value:
ference on Artificial Intelligence, EPIA 2019, Vila Real, Por-
tugal, September 3-6, 2019, Proceedings, Part II, Springer,
Lecture Notes in Computer Science, vol 11805, pp 268–280.
https://doi.org/10.1007/978-3-030-30244-3 23
2. Abeyrathna KD, Bhattarai B, Goodwin M, Gorji SR, Granmo
O, Jiao L, Saha R, Yadav RK (2021) Massively parallel and
asynchronous tsetlin machine architecture supporting almost
constant-time scaling. In: Meila M, Zhang T (eds) Proceedings


key: chroma:document
string_value:
of the 38th International Conference on Machine Learning,
ICML 2021, 18-24 July 2021, Virtual Event, PMLR, Proceedings
of Machine Learning Research, vol 139, pp 10–20. http://
proceedings.mlr.press/v139/abeyrathna21a.html
3. Aggarwal CC (2017) An introduction to outlier analysis. In:
Outlier analysis. Springer International Publishing, pp 1–34.
https://doi.org/10.1007/978-3-319-47578-3 1
4. Allan J, Papka R, Lavrenko V (1998) On-line new event detection


key: chroma:document
string_value:
and tracking. In: Croft WB, Moffat A, van Rijsbergen CJ, Wilkin-
son R, Zobel J (eds) SIGIR ’98: Proceedings of the 21st Annual
International ACM SIGIR Conference on Research and Devel-
opment in Information Retrieval, August 24-28, 1998. ACM,
Australia, pp 37-45. https://doi.org/10.1145/290941.290954
5. Bendale A, Boult TE (2016) Towards open set deep net-
works. In: 2016 IEEE Conference on Computer Vision and
Pattern Recognition, CVPR 2016, Las Vegas, NV , USA,


key: chroma:document
string_value:
June 27-30, 2016, IEEE Computer Society, pp 1563–1572.
https://doi.org/10.1109/CVPR.2016.173
6. Bentivogli L, Clark P, Dagan I, Giampiccolo D (2011)
The seventh PASCAL recognizing textual entailment chal-
lenge. In: Proceedings of the Fourth Text Analysis Confer-
ence, TAC 2011, Gaithersburg, Maryland, USA, November 14-
15, 2011, NIST. https://tac.nist.gov/publications/2011/additional.
papers/RTE7 overview.proceedings.pdf17486


key: chroma:document
string_value:
Word-levelhumaninterpretable...
7. Berge GT, Granmo O, Tveit TO, Goodwin M, Jiao L,
Matheussen BV (2019) Using the tsetlin machine to learn
human-interpretable rules for high-accuracy text categorization
with medical applications. IEEE Access 7:115134–115146.
https://doi.org/10.1109/ACCESS.2019.2935416
8. Bhattarai B, Granmo O, Jiao L (2021) Measuring the novelty of
natural language text using the conjunctive clauses of a tsetlin
machine text classifier. In: Rocha AP, Steels L, van den Herik


key: chroma:document
string_value:
HJ (eds) Proceedings of the 13th International Conference on
Agents and Artificial Intelligence, ICAART 2021, V olume 2,
Online Streaming, February 4-6, 2021, SCITEPRESS, pp 410–
417. https://doi.org/10.5220/0010382204100417
9. Blanchard G, Lee G, Scott C (2010) Semi-supervised novelty
detection. J Mach Learn Res 11:2973–3009. http://portal.acm.org/
citation.cfm?id=1953028
10. Boudin F (2016) Pke: an open source python-based keyphrase
extraction toolkit. In: Watanabe H (ed) COLING 2016, 26th


key: chroma:document
string_value:
International Conference on Computational Linguistics, Proceed-
ings of the Conference System Demonstrations, December 11-16,
2016, Osaka, Japan, ACL, pp 69–73. https://aclanthology.org/
C16-2015/
11. Boudin F (2018) Unsupervised keyphrase extraction with mul-
tipartite graphs. In: Walker MA, Ji H, Stent A (eds) Proceed-
ings of the 2018 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language


key: chroma:document
string_value:
Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June
1-6, 2018 Short Papers, vol 2. Association for Computational
Linguistics, pp 667–672. https://doi.org/10.18653/v1/n18-2105
12. Bougouin A, Boudin F, Daille B (2013) Topicrank: Graph-based
topic ranking for keyphrase extraction. In: Sixth International
Joint Conference on Natural Language Processing, IJCNLP 2013,
Nagoya, Japan, October 14-18, 2013, Asian Federation of Natural
Language Processing / ACL, pp 543–551. https://aclanthology.


key: chroma:document
string_value:
org/I13-1062/
13. Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt
A (2020) Yake! keyword extraction from single documents using
multiple local features. Inf Sci 509:257–289. https://doi.org/10.
1016/j.ins.2019.09.013
14. Carbinell J, Goldstein J (2017) The use of mmr, diversity-based
reranking for reordering documents and producing summaries.
SIGIR Forum 51(2):209–210. https://doi.org/10.1145/3130348.
3130369
15. Chandola V, Banerjee A, Kumar V (2012) Anomaly detection


key: chroma:document
string_value:
for discrete sequences: a survey. IEEE Trans Knowl Data Eng
24(5):823–839. https://doi.org/10.1109/TKDE.2010.235
16. Dasgupta D, Nino L (2000) A comparison of negative and posi-
tive selection algorithms in novel pattern detection, Proceedings of
the IEEE International Conference on Systems, Man & Cyber-
netics. In: Cybernetics evolving to systems, humans, organiza-
tions, and their complex interactions, sheraton music city hotel,
nashville, tennessee, USA. 8-11 October, 2000. IEEE, pp 125–


key: chroma:document
string_value:
130. https://doi.org/10.1109/ICSMC.2000.884976
17. Dasgupta T, Dey L (2016) Automatic scoring for innovativeness
of textual ideas. In: Fortuna B, Grobelnik M, ERH Jr, Witbrock
MJ (eds) Knowledge Extraction from Text, Papers from the 2016
AAAI Workshop, Phoenix, Arizona, USA, February 12, 2016,
AAAI Press, AAAI Workshops, vol WS-16-10. http://www.aaai.
org/ocs/index.php/WS/AAAIW16/paper/view/12663
18. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: Pre-


key: chroma:document
string_value:
training of deep bidirectional transformers for language under-
standing. In: Burstein J, Doran C, Solorio T (eds) Proceedings
of the 2019 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Lan-
guage Technologies, NAACL-HLT 2019, Minneapolis, MN, USA,
June 2-7, 2019 vol 1 (Long and Short Papers). Associationfor Computational Linguistics, pp 4171–4186. https://doi.org/10.
18653/v1/n19-1423
19. Duchi JC, Hazan E, Singer Y (2011) Adaptive subgradient


key: chroma:document
string_value:
methods for online learning and stochastic optimization. J
Mach Learn Res 12:2121–2159. http://dl.acm.org/citation.cfm?
id=2021068
20. Fei G, Liu B, Callison-burch C, Su J, Pighin D, Marton Y
(2015) Social media text classification under negative covari-
ate shift. In: M ´arquez L (ed) Proceedings of the 2015 Con-
ference on Empirical Methods in Natural Language Process-
ing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015,
The Association for Computational Linguistics, pp 2347–2356.


key: chroma:document
string_value:
https://doi.org/10.18653/v1/d15-1282
21. Fei G, Liu B (2016) Breaking the closed world assumption in
text classification. In: Knight K, Nenkova A, Rambow O (eds)
NAACL HLT 2016, The 2016 conference of the north american
chapter of the association for computational linguistics: Human
language technologies, san diego california, USA, June 12-17,
2016, The Association for Computational Linguistics, pp 506–
514. https://doi.org/10.18653/v1/n16-1061


key: chroma:document
string_value:
514. https://doi.org/10.18653/v1/n16-1061
22. Galassi A, Lippi M, Torroni P (2021) Attention in natu-
ral language processing. IEEE Trans Neural Networks Learn
Syst 32(10):4291–4308. https://doi.org/10.1109/TNNLS.2020.
3019893
23. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley
D, Ozair S, Courville AC, Bengio Y (2014) Generative adver-
sarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence
ND, Weinberger KQ (eds) Advances in Neural Information Pro-


key: chroma:document
string_value:
cessing Systems 27: Annual Conference on Neural Information
Processing Systems 2014, December 8-13 2014, Montreal, Que-
bec, Canada, pp 2672–2680. https://proceedings.neurips.cc/paper/
2014/hash/5ca3e9b122f61f8f06494c97b1afccf3-Abstract.html
24. Granmo O (2018) The tsetlin machine - A game theoretic bandit
driven approach to optimal pattern recognition with propositional
logic. arXiv: 1804.01508
25. Granmo O, Glimsdal S, Jiao L, Goodwin M, Omlin CW, Berge
GT (2019)


key: chroma:document
string_value:
GT (2019)
26. Grootendorst M (2020) Keybert: Minimal keyword extraction with
bert. https://doi.org/10.5281/zenodo.4461265
27. Hanley JA, McNeil BJ (1982) The meaning and use of the area
under a receiver operating characteristic (roc) curve. Radiology
143(1):29–36. https://doi.org/10.1148/radiology.143.1.7063747
28. Hautam ¨aki V, K ¨arkk ¨ainen I, Fr ¨anti P (2004) Outlier detection
using k-nearest neighbour graph. In: 17th International Con-


key: chroma:document
string_value:
ference on Pattern Recognition, ICPR 2004, Cambridge, UK,
August 23-26, 2004, IEEE Computer Society, pp 430–433.
https://doi.org/10.1109/ICPR.2004.1334558
29. Hawkins DM (1980) Identification of outliers, Monographs on
Applied Probability and Statistics. Springer, Berlin. https://doi.
org/10.1007/978-94-015-3994-4
30. Hendrycks D, Gimpel K (2017) A baseline for detecting
misclassified and out-of-distribution examples in neural net-
works. In: 5th International Conference on Learning Represen-


key: chroma:document
string_value:
tations, ICLR 2017, Toulon, France, April 24-26, 2017, Confer-
ence Track Proceedings, OpenReview.net. https://openreview.net/
forum?id=Hkg4TI9xl
31. Hill F, Cho K, Korhonen A (2016) Learning distributed represen-
tations of sentences from unlabelled data. In: Knight K, Nenkova
A, Rambow O (eds) NAACL HLT 2016, The 2016 conference of
the north american chapter of the association for computational
linguistics: Human language technologies, san diego california,


key: chroma:document
string_value:
USA, June 12-17, 2016, The Association for Computational Lin-
guistics, pp 1367–1377. https://doi.org/10.18653/v1/n16-1162
32. Hospedales TM, Gong S, Xiang T (2011) Finding rare classes:
Adapting generative and discriminative models in active learning.17487


key: chroma:document
string_value:
B.Bhattaraietal.
In: Huang JZ, Cao L, Srivastava J (eds) Advances in Knowledge
Discovery and Data Mining - 15th Pacific-Asia Conference,
PAKDD 2011, Shenzhen, China, May 24-27, 2011, Proceedings,
Part II, Springer, Lecture Notes in Computer Science, vol 6635,
pp 296–308. https://doi.org/10.1007/978-3-642-20847-8 25
33. Jain LP, Scheirer WJ, Boult TE (2014) Multi-class open
set recognition using probability of inclusion. In: Fleet DJ,
Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision -


key: chroma:document
string_value:
ECCV 2014 - 13th European Conference, Zurich, Switzer-
land, September 6-12, 2014, Proceedings, Part III, Springer,
Lecture Notes in Computer Science, vol 8691, pp 393–409.
https://doi.org/10.1007/978-3-319-10578-9 26
34. Jain S, Wallace BC (2019) Attention is not explanation. In:
Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019
Conference of the North American Chapter of the Association
for Computational Linguistics: Human Language Technologies,


key: chroma:document
string_value:
NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019,
vol 1 (Long and Short Papers). Association for Computational
Linguistics, pp 3543–3556. https://doi.org/10.18653/v1/n19-1357
35. Jiao L, Zhang X, Granmo O, Abeyrathna KD (2021) On
the convergence of tsetlin machines for the XOR operator.
arXiv: 2101.02547
36. Kiros R, Zhu Y, Salakhutdinov R, Zemel RS, Urtasun R,
Torralba A, Fidler S (2015) Skip-thought vectors. In: Cortes
C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds)


key: chroma:document
string_value:
Advances in Neural Information Processing Systems 28:
Annual Conference on Neural Information Processing Sys-
tems 2015, December 7-12, 2015, Montreal, Quebec, Canada,
pp 3294–3302. https://proceedings.neurips.cc/paper/2015/hash/
f442d33fa06832082290ad8544a8da27-Abstract.html
37. Kliger M, Fleishman S (2018) Novelty detection with GAN.
arXiv: 1802.10560
38. Kumaran G, Allan J (2004) Text classification and named enti-
ties for new event detection. In: Sanderson M, J ´arvelin K, Allan J,


key: chroma:document
string_value:
Bruza P (eds) SIGIR 2004: Proceedings of the 27th Annual Inter-
national ACM SIGIR Conference on Research and Development
in Information Retrieval, Sheffield, UK, July 25-29, 2004, ACM,
pp 297–304. https://doi.org/10.1145/1008992.1009044
39. LANN E (1959) Testing statistical hypotheses. Wiley, New York
40. Li Z, Zhao Y, Botta N, Ionescu C, Hu X (2020) COPOD: Copula-
based outlier detection. In: Plant C, Wang H, Cuzzocrea A, Zan-
iolo C, Wu X (eds) 20Th IEEE international conference on data


key: chroma:document
string_value:
mining, ICDM 2020, sorrento, italy, november 17-20, 2020, IEEE,
pp 1118-1123. https://doi.org/10.1109/ICDM50108.2020.00135
41. Liu Y, Li Z, Zhou C, Jiang Y, Sun J, Wang M, He X
(2020) Generative adversarial active learning for unsupervised
outlier detection. IEEE Trans Knowl Data Eng 32(8):1517–1528.
https://doi.org/10.1109/TKDE.2019.2905606
42. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K,
Zettlemoyer L (2018) Deep contextualized word representations.


key: chroma:document
string_value:
In: Walker MA, Ji H, Stent A (eds) Proceedings of the 2018
Conference of the North American Chapter of the Association
for Computational Linguistics: Human Language Technologies,
NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-
6, 2018, vol 1 (Long Papers). Association for Computational
Linguistics, pp 2227–2237. https://doi.org/10.18653/v1/n18-1202
43. Pimentel MAF, Clifton DA, Clifton LA, Tarassenko L (2014)
A review of novelty detection. Signal Process 99:215–249.


key: chroma:document
string_value:
https://doi.org/10.1016/j.sigpro.2013.12.026
44. Platt JC (1999) Probabilistic outputs for support vector machines
and comparisons to regularized likelihood methods. In: Advances
in large margin classifiers
45. Ramos J et al (2003) Using tf-idf to determine word relevance
in document queries. In: Proceedings of the first instructional
conference on machine learning, pp 29–4846. Rebuffi S, Kolesnikov A, Sperl G, Lampert CH (2017) icarl:


key: chroma:document
string_value:
Incremental classifier and representation learning. In: 2017 IEEE
Conference on Computer Vision and Pattern Recognition, CVPR
2017, Honolulu, HI, USA, July 21-26, 2017, IEEE Computer
Society, pp 5533–5542. https://doi.org/10.1109/CVPR.2017.587
47. Saha R, Granmo O, Goodwin M (2020) Mining interpretable
rules for sentiment and semantic relation analysis using tsetlin
machines. In: Bramer M, Ellis R (eds) Artificial Intelligence
XXXVII - 40th SGAI International Conference on Artificial


key: chroma:document
string_value:
Intelligence, AI 2020, Cambridge, UK, December 15-17, 2020,
Proceedings, Springer, Lecture Notes in Computer Science, vol
12498, pp 67–78. https://doi.org/10.1007/978-3-030-63799-6 5
48. Salazar J, Liang D, Nguyen TQ, Kirchhoff K (2020) Masked
language model scoring. In: Jurafsky D, Chai J, Schluter N,
Tetreault JR (eds) Proceedings of the 58th Annual Meeting of the
Association for Computational Linguistics, ACL 2020, Online,
July 5-10, 2020, Association for Computational Linguistics,


key: chroma:document
string_value:
pp 2699–2712. https://doi.org/10.18653/v1/2020.acl-main.240
49. Scheirer WJ, de Rezende Rocha A, Sapkota A, Boult TE (2013)
Toward open set recognition. IEEE Trans Pattern Anal Mach Intell
35(7):1757–1772. https://doi.org/10.1109/TPAMI.2012.256
50. Sch ¨olkopf B, Platt JC, Shawe-taylor J, Smola AJ, Williamson
RC (2001) Estimating the support of a high-dimensional distri-
bution. Neural Comput 13(7):1443–1471. https://doi.org/10.1162/
089976601750264965


key: chroma:document
string_value:
089976601750264965
51. Serrano S, Smith NA (2019) Is attention interpretable?. In:
Korhonen A, Traum DR, M ´arquez L (eds) Proceedings of the 57th
Conference of the Association for Computational Linguistics,
ACL 2019, Florence, Italy, July 28- August 2, 2019, vol 1 (Long
Papers). Association for Computational Linguistics, pp 2931-
2951. https://doi.org/10.18653/v1/p19-1282
52. Soboroff I, Harman D (2005) Novelty detection: The TREC
experience. In: HLT/EMNLP 2005, Human Language Technology


key: chroma:document
string_value:
Conference and Conference on Empirical Methods in Natural
Language Processing, Proceedings of the Conference, 6-8 October
2005, Vancouver, British Columbia, Canada, The Association for
Computational Linguistics, pp 105–112. https://aclanthology.org/
H05-1014/
53. Sun X, Lu W (2020) Understanding attention for text classifi-
cation. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds)
Proceedings of the 58th Annual Meeting of the Association


key: chroma:document
string_value:
for Computational Linguistics, ACL 2020, Online, July 5-10,
2020, Association for Computational Linguistics, pp 3418–3428.
https://doi.org/10.18653/v1/2020.acl-main.312
54. Tax DMJ, Duin RPW (1998) Outlier detection using classi-
fier instability. In: Amin A, Dori D, Pudil P, Freeman H (eds)
Advances in pattern recognition, joint IAPR international work-
shops SSPR ’98 and SPR ’98, sydney, NSW, Australia, August 11-
13, 1998 (Proceedings, Springer), Lecture Notes in Computer Sci-


key: chroma:document
string_value:
ence, vol 1451, pp 593-601. https://doi.org/10.1007/BFb0033283
55. Tax DMJ, Duin RPW (2004) Support vector data descrip-
tion. Mach Learn 54(1):45–66. https://doi.org/10.1023/B:MACH.
0000008084.60811.49
56. Tsai FS, Tang W, Chan KL (2010) Evaluation of novelty metrics
for sentence-level novelty mining. Inf Sci 180(12):2359–2374.
https://doi.org/10.1016/j.ins.2010.02.020
57. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez


key: chroma:document
string_value:
AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In:
Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vish-
wanathan SVN, Garnett R (eds) Advances in Neural Information
Processing Systems 30: Annual Conference on Neural Informa-
tion Processing Systems 2017, December 4-9, 2017, Long Beach,
CA, USA, pp 5998–6008. https://proceedings.neurips.cc/paper/
2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html17488


key: chroma:document
string_value:
Word-levelhumaninterpretable...
58. Wang Y, Huang H, Feng C, Zhou Q, Gu J, Gao X (2016) CSE:
Conceptual sentence embeddings based on attention model. In:
Proceedings of the 54th Annual Meeting of the Association for
Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin,
Germany, V olume 1 (Long Papers). The Association for Computer
Linguistics. https://doi.org/10.18653/v1/p16-1048
59. Yadav RK, Jiao L, Granmo O, Goodwin M (2021a) Human-


key: chroma:document
string_value:
level interpretable learning for aspect-based sentiment analysis.
In: In: Thirty-Fifth AAAI Conference on Artificial Intelligence,
AAAI 2021, Thirty-Third Conference on Innovative Applications
of Artificial Intelligence, IAAI 2021, The Eleventh Symposium
on Educational Advances in Artificial Intelligence, EAAI 2021,
Virtual Event, February 2-9, 2021, AAAI Press, pp 14203–14212.
https://ojs.aaai.org/index.php/AAAI/article/view/17671
60. Yadav RK, Jiao L, Granmo OC, Goodwin M (2021b) Enhancing


key: chroma:document
string_value:
interpretable clauses semantically using pretrained word represen-
tation. In: Proceedings of the Fourth BlackboxNLP Workshop on
Analyzing and Interpreting Neural Networks for NLP, Association
for Computational Linguistics, Punta Cana, Dominican Republic,
pp 265–274 https://doi.org/10.18653/v1/2021.blackboxnlp-1.19 .
https://aclanthology.org/2021.blackboxnlp-1.19
61. Yu Y, Qu W, Li N, Guo Z (2017) Open category classification by


key: chroma:document
string_value:
adversarial sample generation. In: Sierra C (ed) Proceedings of the
Twenty-Sixth International Joint Conference on Artificial Intelli-
gence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017,
ijcai.org, pp 3357–3363. https://doi.org/10.24963/ijcai.2017/469
62. Zhang X, Jiao L, Granmo O, Goodwin M (5555) On the
convergence of tsetlin machines for the identity- and not operators.
IEEE Transactions on Pattern Analysis & Machine Intelligence pp
1–1. https://doi.org/10.1109/TPAMI.2021.3085591


key: chroma:document
string_value:
1–1. https://doi.org/10.1109/TPAMI.2021.3085591
63. Zhang Y, Tsai FS (2009) Combining named entities and tags for
novel sentence detection. In: Proceedings of the WSDM’09 Work-
shop on Exploiting Semantic Annotations in Information Retrieval,
Association for Computing Machinery, New York, NY , USA,
ESAIR ’09, p 30–34. https://doi.org/10.1145/1506250. 1506256.
Publisher’s note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.


key: chroma:document
string_value:
Bimal Bhattarai received
the B.Tech. degree in elec-
tronics and communication
engineering from JNTUA,
India, in 2015, and the M.Sc.
degree from the Department
of Information and Com-
munication Engineering,
Chosun University, South
Korea. He is currently a PhD
Scholar with the Department
of Information and Commu-
nication Technology, UiA.
His research interests include
data science, natural language
processing, tsetlin machine,
indoor positioning and navigation, machine learning, and deep


key: chroma:document
string_value:
learning applications.
Ole-Christoﬀer Granmo is
the Founding Director of Cen-
tre for Artificial Intelligence
Research (CAIR), Univer-
sity of Agder, Norway. He
obtained his master’s degree
in 1999 and the PhD degree in
2004, both from the University
of Oslo, Norway. Dr. Granmo
has authored in excess of
150 refereed papers with 6
best paper awards within
machine learning, encompass-
ing learning automata, bandit
algorithms, Tsetlin machines,
Bayesian reasoning, rein-


key: chroma:document
string_value:
Bayesian reasoning, rein-
forcement learning, and computational linguistics. He has further
coordinated 7+ research projects and graduated 55+ master- and 7
PhD students. Dr. Granmo is also a co-founder of the Norwegian
Artificial Intelligence Consortium (NORA). Apart from his academic
endeavours, he co-founded the company Anzyz Technologies AS.
Lei Jiao received the B.E.
degree in telecommunications
engineering from Hunan Uni-
versity, Changsha, China, in
2005, the M.E. degree in


key: chroma:document
string_value:
2005, the M.E. degree in
communication and informa-
tion system from Shandong
U n i v e r s i t y ,J i n a n ,C h i n a ,i n
2008, and the Ph.D. degree in
information and communica-
tion technology from the Uni-
versity of Agder (UiA), Nor-
way, in 2012. He is currently
w o r k i n ga sa nA s s o c i a t eP r o -
fessor with the Department of
Information and Communica-
tion Technology, UiA. His research interests include reinforcement


key: chroma:document
string_value:
learning, Tsetlin machine, resource allocation and performance evalu-
ation for communication and energy systems.17489


