# embedding_metadata

key: chroma:document
string_value:
6. Conclusions
In this paper, we proposed a novel continuous feature representation approach for
Tsetlin Machines (TMs) using Stochastic Searching on the Line (SSL) automata. The SSLs
learn the lower and upper limits of the continuous feature values inside clauses. These
limits decide the Boolean representation of the continuous value inside the clauses. We
have provided empirical evidence to show that the novel way of representing continuous


key: chroma:document
string_value:
features in the TMs can reduce the number of literals included in the learned TM clauses
by 4.3 times compared to the Booleanization scheme without loss of performance. Further,
the new continuous feature representation is able to decrease the total training time from
0.177 s to 0.126 s per epoch, and the combined total memory usage is reduced from 16.35 KB
to 9.45 KB while exhibiting on par or better performance. In terms of the average F1-Score,


key: chroma:document
string_value:
the TM with the proposed feature representation also outperforms several state-of-the-art
machine learning algorithms.
In our future work, we intend to investigate the possibility of applying a similar
feature representation for multi-class and regression versions of the TM.
Author Contributions: Conceptualization, K.D.A. and O.-C.G.; software, K.D.A. and O.-C.G.; vali-
dation, K.D.A. and O.-C.G.; formal analysis, K.D.A. and O.-C.G.; investigation, K.D.A., O.-C.G. and


key: chroma:document
string_value:
M.G.; writing—original draft preparation, K.D.A.; writing—review and editing, K.D.A., O.-C.G.
and M.G.; supervision, O.-C.G. All authors have read and agreed to the published version of the
manuscript.
Funding: This research received no external funding.
Conﬂicts of Interest: The authors declare no conﬂict of interest.
References
1. Miotto, R.; Wang, F.; Wang, S.; Jiang, X.; Dudley, J.T. Deep learning for healthcare: review, opportunities and challenges. Brief.


key: chroma:document
string_value:
Bioinform. 2018 ,19, 1236–1246. [CrossRef] [PubMed]
2. Bellazzi, R.; Zupan, B. Predictive data mining in clinical medicine: Current issues and guidelines. Int. J. Med. Inform. 2008 ,
77, 81–97. [CrossRef]
3. Pazzani, M.J.; Mani, S.; Shankle, W.R. Acceptance of rules generated by machine learning among medical experts. Methods Inf.
Med. 2001 ,40, 380–385.
4. Baesens, B.; Mues, C.; De Backer, M.; Vanthienen, J.; Setiono, R. Building intelligent credit scoring systems using decision tables.


key: chroma:document
string_value:
InEnterprise Information Systems V ; Springer: Berlin/Heidelberg, Germany, 2004; pp. 131–137.
5. Huysmans, J.; Dejaeger, K.; Mues, C.; Vanthienen, J.; Baesens, B. An empirical evaluation of the comprehensibility of decision
table, tree and rule based predictive models. Decis. Support Syst. 2011 ,51, 141–154. [CrossRef]
6. Lima, E.; Mues, C.; Baesens, B. Domain knowledge integration in data mining using decision tables: Case studies in churn


key: chroma:document
string_value:
prediction. J. Oper. Res. Soc. 2009 ,60, 1096–1106. [CrossRef]
7. Verbeke, W.; Martens, D.; Mues, C.; Baesens, B. Building comprehensible customer churn prediction models with advanced rule
induction techniques. Expert Syst. Appl. 2011 ,38, 2354–2364. [CrossRef]
8. Freitas, A.A.; Wieser, D.C.; Apweiler, R. On the importance of comprehensible classiﬁcation models for protein function prediction.
IEEE/ACM Trans. Comput. Biol. Bioinform. 2008 ,7, 172–182. [CrossRef] [PubMed]


key: chroma:document
string_value:
Electronics 2021 ,10, 2107 24 of 25
9. Szafron, D.; Lu, P .; Greiner, R.; Wishart, D.S.; Poulin, B.; Eisner, R.; Lu, Z.; Anvik, J.; Macdonell, C.; Fyshe, A.; et al. Proteome
Analyst: Custom predictions with explanations in a web-based tool for high-throughput proteome annotations. Nucleic Acids Res.
2004 ,32, W365–W371. [CrossRef]
10. Lazreg, M.B.; Goodwin, M.; Granmo, O.C. Deep learning for social media analysis in crises situations. In Proceedings of the 29th


key: chroma:document
string_value:
Annual Workshop of the Swedish Artiﬁcial Intelligence Society (SAIS), Malmö, Sweden, 2–3 June 2016; p. 31.
11. Agarwal, R.; Frosst, N.; Zhang, X.; Caruana, R.; Hinton, G.E. Neural Additive Models: Interpretable Machine Learning with
Neural Nets. arXiv 2020 , arXiv:2004.13912.
12. Molnar, C. Interpretable Machine Learning ; Leanpub: Victoria, BC, Canada, 2019.
13. Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why should i trust you?” Explaining the predictions of any classiﬁer. In Proceedings of the


key: chroma:document
string_value:
22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August
2016; pp. 1135–1144.
14. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.
LeanpubNat. Mach. Intell. 2019 ,1, 206–215. [CrossRef]
15. Agrawal, R.; Imieli ´ nski, T.; Swami, A. Mining Association Rules Between Sets of Items in Large Databases. SIGMOD Rec. 1993 ,
22, 207–216. [CrossRef]


key: chroma:document
string_value:
22, 207–216. [CrossRef]
16. McCormick, T.; Rudin, C.; Madigan, D. A Hierarchical Model for Association Rule Mining of Sequential Events: An Approach to
Automated Medical Symptom Prediction. Ann. Appl. Stat. 2011 . [CrossRef]
17. Feldman, V . Hardness of Approximate Two-Level Logic Minimization and PAC Learning with Membership Queries. J. Comput.
Syst. Sci. 2009 ,75, 13–26. [CrossRef]
18. Valiant, L.G. A Theory of the Learnable. Commun. ACM 1984 ,27, 1134–1142. [CrossRef]


key: chroma:document
string_value:
19. Wang, T.; Rudin, C.; Doshi-Velez, F.; Liu, Y.; Klampﬂ, E.; MacNeille, P . A Bayesian Framework for Learning Rule Sets for
Interpretable Classiﬁcation. J. Mach. Learn. Res. 2017 ,18, 2357–2393.
20. Hauser, J.R.; Toubia, O.; Evgeniou, T.; Befurt, R.; Dzyabura, D. Disjunctions of Conjunctions, Cognitive Simplicity, and
Consideration Sets. J. Mark. Res. 2010 ,47, 485–496. [CrossRef]


key: chroma:document
string_value:
21. Liang, Y.; Van den Broeck, G. Learning logistic circuits. In Proceedings of the 33rd AAAI Conference on Artiﬁcial Intelligence,
Hilton Hawaiian Village, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33, pp. 4277–4286.
22. Nori, H.; Jenkins, S.; Koch, P .; Caruana, R. InterpretML: A Uniﬁed Framework for Machine Learning Interpretability. arXiv 2019 ,
arXiv:1909.09223.


key: chroma:document
string_value:
arXiv:1909.09223.
23. Lou, Y.; Caruana, R.; Gehrke, J. Intelligible models for classiﬁcation and regression. In Proceedings of the 18th ACM SIGKDD
International Conference on Knowledge Discovery and Data Mining, Beijing, China, 12–16 August 2012; pp. 150–158.
24. Granmo, O.C. The Tsetlin Machine—A game theoretic bandit driven approach to optimal pattern recognition with propositional
logic. arXiv 2018 , arXiv:1804.01508.


key: chroma:document
string_value:
logic. arXiv 2018 , arXiv:1804.01508.
25. Phoulady, A.; Granmo, O.C.; Gorji, S.R.; Phoulady, H.A. The Weighted Tsetlin Machine: Compressed Representations with
Clause Weighting. In Proceedings of the Ninth International Workshop on Statistical Relational AI (StarAI2020), New York, NY,
USA, 7 February 2020.
26. Berge, G.T.; Granmo, O.C.; Tveit, T.O.; Goodwin, M.; Jiao, L.; Matheussen, B.V . Using the Tsetlin Machine to learn human-


key: chroma:document
string_value:
interpretable rules for high-accuracy text categorization with medical applications. IEEE Access 2019 ,7, 115134–115146. [CrossRef]
27. Abeyrathna, K.D.; Granmo, O.C.; Zhang, X.; Jiao, L.; Goodwin, M. The Regression Tsetlin Machine—A Novel Approach to
Interpretable Non-Linear Regression. Philos. Trans. R. Soc. A 2019 ,378, 20190165. [CrossRef]
28. Gorji, S.R.; Granmo, O.C.; Phoulady, A.; Goodwin, M. A Tsetlin Machine with Multigranular Clauses. In Lecture Notes in Computer


key: chroma:document
string_value:
Science: Proceedings of the Thirty-ninth International Conference on Innovative Techniques and Applications of Artiﬁcial Intelligence
(SGAI-2019) ; Springer: Berlin/Heidelberg, Germany, 2019; Volume 11927.
29. Gorji, S.R.; Granmo, O.-G.; Glimsdal, S.; Edwards, J.; Goodwin, M. Increasing the Inference and Learning Speed of Tsetlin
Machines with Clause Indexing. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent


key: chroma:document
string_value:
Systems ; Springer: Berlin/Heidelberg, Germany, 2020.
30. Wheeldon, A.; Shaﬁk, R.; Yakovlev, A.; Edwards, J.; Haddadi, I.; Granmo, O.C. Tsetlin Machine: A New Paradigm for Pervasive
AI. In Proceedings of the SCONA Workshop at Design, Automation and Test in Europe (DATE), Grenoble, France, 9–13
March 2020.
31. Tsetlin, M.L. On behaviour of ﬁnite automata in random medium. Avtomat. Telemekh 1961 ,22, 1345–1354.


key: chroma:document
string_value:
32. Yadav, R.K.; Jiao, L.; Granmo, O.C.; Goodwin, M. Interpretability in Word Sense Disambiguation using Tsetlin Machine. In
Proceedings of the 13th International Conference on Agents and Artiﬁcial Intelligence (ICAART), Vienna, Austria, 4 February
2021; INSTICC: Lisboa, Portugal, 2021.
33. Bhattarai, B.; Jiao, L.; Granmo, O.C. Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin


key: chroma:document
string_value:
Machine Text Classiﬁer. In Proceedings of the 13th International Conference on Agents and Artiﬁcial Intelligence (ICAART),
Vienna, Austria, 4 February 2021; INSTICC: Lisboa, Portugal, 2021.
34. Sharma, J.; Yadav, R.; Granmo, O.C.; Jiao, L. Human Interpretable AI: Enhancing Tsetlin Machine Stochasticity with Drop Clause.
arXiv 2021 , arXiv:2105.14506.
35. Lei, J.; Rahman, T.; Shaﬁk, R.; Wheeldon, A.; Yakovlev, A.; Granmo, O.C.; Kawsar, F.; Mathur, A. Low-Power Audio Keyword


key: chroma:document
string_value:
Electronics 2021 ,10, 2107 25 of 25
36. Granmo, O.C.; Glimsdal, S.; Jiao, L.; Goodwin, M.; Omlin, C.W.; Berge, G.T. The Convolutional Tsetlin Machine. arXiv 2019 ,
arXiv:1905.09688.
37. Abeyrathna, K.D.; Granmo, O.C.; Zhang, X.; Goodwin, M. A scheme for continuous input to the Tsetlin Machine with applications
to forecasting disease outbreaks. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent


key: chroma:document
string_value:
Systems ; Springer: Berlin/Heidelberg, Germany, 2019; pp. 564–578.
38. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased
Interpretability. IEEE Access 2021 ,9, 8233–8248. [CrossRef]
39. Wheeldon, A.; Shaﬁk, R.; Rahman, T.; Lei, J.; Yakovlev, A.; Granmo, O.C. Learning Automata based Energy-efﬁcient AI Hardware
Design for IoT. Philos. Trans. R. Soc. A 2020 ,378, 20190593. [CrossRef] [PubMed]


key: chroma:document
string_value:
40. Abeyrathna, K.D.; Bhattarai, B.; Goodwin, M.; Gorji, S.; Granmo, O.C.; Jiao, L.; Saha, R.; Yadav, R.K. Massively Parallel and
Asynchronous Tsetlin Machine Architecture Supporting Almost Constant-Time Scaling. In Proceedings of the Thirty-eighth
International Conference on Machine Learning (ICML2021), Virtual, 18–24 July 2021.
41. Shaﬁk, R.; Wheeldon, A.; Yakovlev, A. Explainability and Dependability Analysis of Learning Automata based AI Hardware. In


key: chroma:document
string_value:
Proceedings of the IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS), Napoli, Italy,
13–15 July 2020.
42. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern
Representation. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems ; Springer:
Berlin/Heidelberg, Germany, 2020.


key: chroma:document
string_value:
Berlin/Heidelberg, Germany, 2020.
43. Zhang, X.; Jiao, L.; Granmo, O.C.; Goodwin, M. On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators.
IEEE Trans. Pattern Anal. Mach. Intell. 2021 , 1. [CrossRef]
44. Jiao, L.; Zhang, X.; Granmo, O.C.; Abeyrathna, K.D. On the Convergence of Tsetlin Machines for the XOR Operator. arXiv 2021 ,
arXiv:2101.02547.


key: chroma:document
string_value:
arXiv:2101.02547.
45. Oommen, B.J. Stochastic searching on the line and its applications to parameter learning in nonlinear optimization. IEEE Trans.
Syst. Man Cybern. Part B (Cybern.) 1997 ,27, 733–739. [CrossRef]
46. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. Adaptive Sparse Representation of Continuous Input for Tsetlin Machines Based
on Stochastic Searching on the Line. 2020. In Preparation.


key: chroma:document
string_value:
47. Lucena, B. StructureBoost: Efﬁcient Gradient Boosting for Structured Categorical Variables. arXiv 2020 , arXiv:2007.04446.
48. Narendra, K.S.; Thathachar, M.A. Learning Automata: An Introduction ; Courier Corporation: Chelmsford, MA, USA, 2012.
49. Thathachar, M.A.L.; Sastry, P .S. Networks of Learning Automata: Techniques for Online Stochastic Optimization ; Kluwer Academic
Publishers: London, UK, 2004.


key: chroma:document
string_value:
Publishers: London, UK, 2004.
50. Chen, T.; Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd Acm Sigkdd International Conference
on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794.
51. Kim, M.J.; Han, I. The discovery of experts’ decision rules from qualitative bankruptcy data using genetic algorithms. Expert Syst.
Appl. 2003 ,25, 637–646. [CrossRef]


key: chroma:document
string_value:
Appl. 2003 ,25, 637–646. [CrossRef]
52. McDermott, J.; Forsyth, R.S. Diagnosing a disorder in a classiﬁcation benchmark. Pattern Recognit. Lett. 2016 ,73, 41–43. [CrossRef]
53. Lucena, B. Exploiting Categorical Structure Using Tree-Based Methods. arXiv 2020 , arXiv:2004.07383.
54. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. On Obtaining Classiﬁcation Conﬁdence, Ranked Predictions and AUC with


key: chroma:document
string_value:
Tsetlin Machines. In Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, ACT,
Australia, 1–4 December 2020; pp. 662–669. [CrossRef]


key: chroma:document
string_value:
Self-timed Reinforcement Learning
using Tsetlin Machine
Adrian Wheeldon, Alex Y akovlev and Rishad Shaﬁk
Microsystems Group, Newcastle University, UK
Email: {a.r.wheeldon2, alex.yakovlev, rishad.shaﬁk}@ncl.ac.uk
Abstract —We present a hardware design for the learning
datapath of the Tsetlin machine algorithm, along with a latency
analysis of the inference datapath. In order to generate a
low energy hardware which is suitable for pervasive artiﬁcial


key: chroma:document
string_value:
intelligence applications, we use a mixture of asynchronous design
techniques—including Petri nets, signal transition graphs, dual-
rail and bundled-data. The work builds on previous design of
the inference hardware, and includes an in-depth breakdown
of the automaton feedback, probability generation and Tsetlin
automata. Results illustrate the advantages of asynchronous de-
sign in applications such as personalized healthcare and battery-


key: chroma:document
string_value:
powered internet of things devices, where energy is limited and
latency is an important ﬁgure of merit. Challenges of static timing
analysis in asynchronous circuits are also addressed.
I. I NTRODUCTION
We present a comprehensive design and analysis for an
asynchronous learning datapath based on the Tsetlin machine
(TM) algorithm. When coupled with asynchronous inference
hardware [1], a complete asynchronous TM capable of online
learning is formed. We also analyze the latency of the infer-


key: chroma:document
string_value:
ence datapath, showing its evolution during training.
The TM [2] algorithm is effective in many large classiﬁca-
tion problems [3, 4, 5]. In addition, the TM’s reinforcement
learning and logic-based inference make it a good candidate
for energy efﬁcient hardware [6]. We design an energy-frugal
TM hardware with a view of use cases in pervasive artiﬁcial
intelligence; e.g. in personal healthcare, accessibility, environ-
mental monitoring and predictive maintenance. We give a brief


key: chroma:document
string_value:
introduction to the TM algorithm in Section II.
We wish to implement the TM using asynchronous circuits
as they can be beneﬁcial for low energy sensing systems [7],
when tightly coupled with analog blocks [8], and when power
delivery is unstable or unpredictable [9]. These features are
often present in the aforementioned applications [10].
The architecture of the hardware is designed using a hy-
brid Petri net model in Section III. The model incorporates
inference and learning components.


key: chroma:document
string_value:
inference and learning components.
TMs use groups of reinforcement automata, called Tsetlin
automata ( TA s), to create an ensemble learning effect. We
decompose the automata reinforcement feedback into three
stages to aid the hardware design in Section IV.
The reinforcement process of the TM involves random
choice with deﬁned probabilities—some ﬁxed and some vary-
ing at runtime. These probabilistic choices enable diversity
of learning in the TA s. We design a quasi delay insensitive


key: chroma:document
string_value:
(QDI ) pseudorandom bit generator ( PRBG ) for this use in theClauses Automaton T eams
TA TA
Majority VotingClause Resultsexclude
Class ConﬁdenceFeedback
ClassiﬁcationInput Feature Vector ( f) PRBG
Learning
Figure 1. Overview of the Tsetlin machine ( TM) architecture.
TM in Section V, and motivate its use in other low power
applications.
Considering the implementation of the state-holding TA s,w e
compare QDI and bundled-data ( BD) design styles suitable for


key: chroma:document
string_value:
the low energy applications in Section VI. The comparison
will show the limitations of the two asynchronous design
methods in terms of area, energy and scalability. We use the
WORKCRAFT framework [11] for speciﬁcation and synthesis
of the QDI implementation using signal transition graphs
(STGs ), as well as veriﬁcation of the STG properties required for
a functional and robust design. For BD, we are inspired by the
desynchronization workﬂow of Cortadella et al. [12] in using


key: chroma:document
string_value:
a commercial synthesis tool designed for synchronous design.
BD circuits are readily-implementable with industrial-quality
tools, in contrast with QDI circuits.
Main contributions of this work:
•Visualization of an asynchronous TM hardware architec-
ture using Petri nets (Section III).
•Latency analysis of the asynchronous TM inference datap-
ath from [1] and its evolution during training (Section III).
•A detailed decomposition of TA reinforcement feedback
into three stages (Section IV).


key: chroma:document
string_value:
into three stages (Section IV).
•Design of a QDI PRBG for probabilistic choice in the TM
and other low power applications (Section V).
•Comparison of QDI and BD implementations of the TAfor
low energy hardware (Section VI).
II. T SETLIN MACHINES IN BRIEF
TMs learn patterns in binary data using propositional logic.
The main inference component of the TM is the clause that
composes an AND expression of the input features and their


key: chroma:document
string_value:
402021 27th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC)
978-1-7281-4132-9/21/$31.00 ©2021 IEEE
DOI 10.1109/ASYNC48570.2021.000142021 27th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC) | 978-1-7281-4132-9/21/$31.00 ©2021 IEEE | DOI: 10.1109/ASYNC48570.2021.00014
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
1 2 n··· n+1 n+2 2n···Action 1 Exclude Action 2 Include
Penalty Reward
Figure 2. Tsetlin automaton state graph.
complements. The TM comprises many clauses, each produc-
ing a vote. The composition of each clause is controlled by a
vector of exclude bits (see Figure 1). These bits are parameters
that are learned by teams of TA s.
Each clause can produce a vote for its class. The algorithm
states that half of the clauses vote positively, while the other


key: chroma:document
string_value:
half of the clauses vote negatively (we will denote this by
the boolean c neg). The inclusion of inhibition in the voting
system enables non-linear decision boundaries in the infer-
ence process. A majority vote gives an indication of class
conﬁdence. This conﬁdence is used to classify the input data
and inﬂuence future decisions of the automata through the
feedback mechanism [2]. In this work we consider only the
single class TM for simplicity.


key: chroma:document
string_value:
single class TM for simplicity.
The TA is a class of ﬁnite reinforcement automaton [13].
It produces an exclude output for states below the midpoint,
and include for states above the midpoint as illustrated in
Figure 2. The TA receives a penalty or reward from the
feedback mechanism based on the current state of the TM.
Continued rewards in the end states cause the TAto saturate.
A penalty in one of the midstates ( norn+1 ) causes the
TA to transition across the decision boundary—inverting its


key: chroma:document
string_value:
output from exclude to include, or vice versa. The feedback
mechanism is described in more detail in Section IV.
III. A SYNCHRONOUS TSETLIN MACHINE
Figure 3 is an architectural diagram for the asynchronous
implementation of the TM shown in Figure 1, demonstrating
the scalability of the design. The diagram can be composed
with the Petri-net-like tiles in Figure 4 to form a visualization
of the complete system. In these diagrams, rectangles represent


key: chroma:document
string_value:
transitions or computations, and circles represent places as in
classical Petri nets. When abutting these tiles, adjoining places
are reduced to a single place. This representation of the system
is not a formalism.
The main structure is a 3×3grid constructed with feature
inputs in rows and clauses in columns. The small size facil-
itates the explanation of the system, but the architecture can
be easily extended by appending clause columns to the right
side, or feature rows below f3.


key: chroma:document
string_value:
side, or feature rows below f3.
The clause tile is detailed in Figure 4a and comprises a
partial clause ( PC) computation for the feature input and its
complement (as introduced in Section II); a partial feedback
calculation for stage three feedback ( FB3 ); and two TA s. The
feedback stages will be discussed later in Section V.
The clause sum is computed in the sum tile at the bottom
of each clause column. Figure 4b details the sum tile. StageClause
TileClause
TileClause
Tile
Clause
TileClause


key: chroma:document
string_value:
TileClause
TileClause
Tile
Clause
TileClause
TileClause
Tile
Clause
TileClause
TileClause
Tile
Sum
TileSum
TileSum
Tilef1
f2
f3
FB1 calcyexp
learnClassiﬁcationConcurrency Opportunity
Figure 3. Architectural diagram for a Tsetlin machine with three feature
inputs (rows) and three clauses (columns).
(a) Clause tile. (b) Sum tile.
Figure 4. Petri-net-like visualizations for the Tsetlin machine system: (a) the
clause tile including partial clause ( PC) and two Tsetlin automata ( TA s ); and


key: chroma:document
string_value:
(b) the sum tile.
two feedback ( FB2 ) for the clause is computed at this stage
using the clause sum.
The diagrams capture the potential for both concurrency and
serialization in the design. As presented, the system operates
in a serialized manner where the feature inputs are passed
through each clause in series. By contracting the places in
the highlighted regions of Figure 4a, tokens are passed to the
right without waiting for PCcalc and FB3 calc , this allowing


key: chroma:document
string_value:
clauses to compute in parallel.
The majority of the system is implemented in a dual-
rail ( DR) style using a reduced completion detection ( RCD )
scheme [7]. However, the TA s are implemented in a single-rail
(SR) style (as indicated in Figure 4a) which will be motivated
and discussed in Section VI.
The RCD scheme is a simpliﬁed scheme which completion
detects only the outputs of a block. This brings beneﬁts of
early propagation and drastically reduced area and power


key: chroma:document
string_value:
overheads versus traditional full completion detection. The
trade-off is an added timing assumption on the return-to-idle
phase of the DR signals. Full details of the RCD scheme are
41
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
fpep
0fnep
1
pcnfnen
0fpen
1
pcp
(a) Partial clause ( PC).
cpcn... ...
(b) Partial clause combiner.HA0 HA1 HA2 HA3
HA4 HA5 HA6 HA7
HA8 FA0 FA1a0a1 a2a3 a4a5 a6a7
s cs c
scs
c
spinvc
y0 y1 y2spinv
y3
(c) Population count with implicit
dual-rail wires and blocks.

(d) Magnitude comparator.
Figure 5. Dual-rail circuits for inference. Spinv = spacer inverter.
available in [7].
A. Inference Circuits
The inference datapath comprises the clauses, majority


key: chroma:document
string_value:
voting and classiﬁcation introduced in Figure 1. Clauses are
separated into partial clause calculations as discussed earlier
in this section and can be implemented by the schematics in
Figures 5a and 5b. Feature inputs occur in DR encoding along
with exclude signals e0ande1. The outputs of the partial clause
are combined using a DR AND tree.
Majority voting calculated the sum of the clause votes ( csum)
and is implemented using DR population count. The schematic


key: chroma:document
string_value:
in Figure 5c shows the implementation based on DR half-
and full-adders. The OR gates and wires in this circuit are
also implicitly DR encoded. Two spacer inverters (spinv) are
required to ensure uniform spacer polarity at the outputs.
For this single class TM example, the output is classiﬁed
using a threshold function in the form of a magnitude com-
parator. The magnitude comparator lends itself to low energy
implementation in DR and saves energy by evaluating single-


key: chroma:document
string_value:
bit comparisons from most signiﬁcant bit to least signiﬁcant
bit only if needed, all within the period of one cycle [1].
To expand the example to two classes or more, an argmax
function would take the place of thresholding.
In the following sections we will explore the inference
circuit delays and analyze the inference latency of the asyn-chronous datapath with differing input operands. For synthesis
we use a commercial, off-the-shelf technology library: UMC


key: chroma:document
string_value:
65 nm LOW-LEAKAGE , herein referred to as UMC 65LL.
B. Delay Analysis
Since we use an RCD scheme which enables early propaga-
tion [1], the delays of the circuits can vary based on the input
operands. We analyze the delay distributions of the inference
circuits using uniformly-distributed random inputs to gain an
understanding of this phenomenon. We will use this to explain
the evolution of latency during training in the next section.
Clause: In the worst-case delay no literals are excluded


key: chroma:document
string_value:
(e=0 ), all feature inputs are logic-1 ( f=1 ); therefore the
delay is bounded by the slowest input and the delay arc is
through the positive rail. In the best-case delay, all but one
literal is excluded and the literal is logic-0; therefore the delay
is bounded by signal propagation through the OR tree in the
negative rail. Since the clause is a large AND tree, there is a
high probability of c=0 , so the delay distribution is mostly
clustered at the lower end as Figure 6 shows. A much smaller


key: chroma:document
string_value:
portion is seen towards the circuit’s maximum delay and is
associated with c=1 .
Population Count: The delay of the circuit is bounded
by the carry chain formed by HA8,FA0and FA1. In the best
case there are no carries and the circuit latency is governed by
the longest logic path which comprises the ﬁrst two layers of
HAs (which have roughly equal paths), through the OR gates
and ﬁnally FA0to the y1output. In the worst case, output y3
must wait for the aforementioned best-case path as well as


key: chroma:document
string_value:
the carry through FA1. This leads to the log-normal-like delay
distribution seen in Figure 6.
Magnitude Comparator: For uniform random inputs and
a normalized worst-case delay of 1, the comparator achieves
a mean delay of 0.05. This is attributed to early propagation
and the evaluation of single-bit comparisons from MSB toLSB .
The delay of the circuit increases as the absolute difference
between operands decreases, because more of the MSBs are


key: chroma:document
string_value:
equal. This circuit is therefore fastest during inference and
the later stages of training. The operand space halves for each
single-bit comparison leading to the negative exponential delay
distribution in Figure 6.
C. Operand-dependent Inference Latency
We investigate the evolution of inference latency during
training by plotting the spacer →codeword latency at different
stages. We use a binarized version of the I RIS Flower dataset1


key: chroma:document
string_value:
to provide feature inputs ( f). Exclude inputs ( e) are retrieved
from a TM pre-trained on the same dataset after the speciﬁed
number of epochs. Figure 7 shows the distributions for the
untrained TM, and the trained TM after 4 epochs and 50 epochs.
The plots show the trend of the mean towards greater circuit
delay as the TM is trained and therefore more clauses become
activated during the computation of each datapoint ( c=1 ).
In this case we see more examples from the right side of


key: chroma:document
string_value:
1Available: https://github.com/cair/TsetlinMachine
42
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
Meanc=0 c=1
00.20.4Clause
Mean00.20.4Population Count
Mean
Decreasing Operand Delta
00.20.4Magnitude Comparator
Mean
00 .20 .40 .60 .8100.20.4
Normalized DelayInference DatapathProbability
Figure 6. Latency distributions of the Tsetlin machine inference datapath and
its components for uniformly-distributed random inputs. Post-synthesis results
using the UMC 65 nm LOW-LEAKAGE library.
Mean
00.20.4Untrained
Mean
00.20.4Trained 4 epochs ( 81 % Accuracy)
Mean
140 160 180 200 22000.20.4


key: chroma:document
string_value:
Mean
140 160 180 200 22000.20.4
Latency ( ps)Trained 50 epochs ( 96 % Accuracy)Probability
Figure 7. Latency distribution of the Tsetlin machine inference datapath for
the I RIS dataset.
the clause delay distribution in Figure 6. This is the main
contributor to the increase in mean latency as the system
becomes more trained. The magnitude comparator offers a
netdecrease in mean latency as training progresses since the
difference between input operands will increase, meaning we


key: chroma:document
string_value:
shift towards the left of the distribution in Figure 7. However
this shift is somewhat smaller than that of the clauses, and
it therefore has a lesser effect on the inference datapath. The
distribution of the population count is much more uniform
than that of the other circuits, and therefore has a negligible
effect during training.IV . D ECOMPOSITION OF FEEDBACK
During learning, feedback is given to each TAto inﬂuence
its future action, and therefore control pattern detection in the


key: chroma:document
string_value:
TM as a whole. The feedback is based on a carefully-designed
payoff matrix [2, p. 10] and is split into two categories: Type
I and Type II. Type I feedback reinforces good decisions and
penalizes false negative outputs. Type II feedback combats
false positives by penalizing TA s that exclude when c=1 .
We decompose the feedback logic into three stages based on
scope: FB1 atTM level, FB2 at clause level, and FB3 atTAlevel.
FB1 and FB2 are assigned types: none, T1or T2; these are based


key: chroma:document
string_value:
on the Type I and Type II concepts introduced previously. FB3
assigns an action to a single TA: penalty, reward or inaction;
and is based on the outcomes of the payoff matrix2. Each
feedback stage’s output is input to the next, until ﬁnally the
output of FB3 is input to the corresponding TA. The following
sections discuss each feedback stage in detail. Finally we will
discuss the synthesis of the feedback circuits in the DR style
in Section IV -D.
A. FB1:TM-level Feedback
Inputs: learn ,yexp


key: chroma:document
string_value:
A. FB1:TM-level Feedback
Inputs: learn ,yexp
Output: Feedback type (none, T1or T2)
The ﬁrst stage broadly enables feedback for the entire TM if
learn is asserted. The feedback type is chosen according to
yexp —the expected class as provided by the labeled data—as
follows:
FB1=⎧
⎪⎨
⎪⎩none ,forlearn =0
T1, forlearn =1,yexp =1
T2, forlearn =1,yexp =0
B. FB2: Clause-level Feedback
Inputs: FB1 ,c neg,pT
Output: Feedback type (none, T1or T2)
The second stage swaps the feedback type from T1 to T2,


key: chroma:document
string_value:
or vice versa if the associated clause is negated (c neg=1 ,
discussed in Section II). Additionally, feedback is stopped
(inaction given) if the number of activated clauses (those
producing a logic-1 at the output) meets the threshold, T. This
is determined by probabilistic choice, pT, which is discussed
at the end of this section. If there is no feedback from the
previous stage ( FB1 =none), then none will be generated
(FB2=none). Parameter c neg is determined at design time,


key: chroma:document
string_value:
therefore the logic for FB1 can be separated and depends only
on FB1 and pT. This will be demonstrated in Section IV -D.
Clause-level feedback can be summarized as follows:
FB2=⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩none ,for FB1=none
none ,for FB1=T1,pT=0
none ,for FB1=T2,pT=1
T1, for c neg=1,FB1=T2,pT=0
T2, for c neg=1,FB1=T1,pT=1
T1, for c neg=0,FB1=T1,pT=1
T2, for c neg=0,FB1=T2,pT=0
2We do not consider boosting of true positive feedback, which is an option
in the original algorithm.
43


key: chroma:document
string_value:
in the original algorithm.
43
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
Table I
TRUTH T ABLE FOR FB3 .×= DON ’TC A R E .
FB2 inc c x p s FB3
none× ××× inaction
T1 10×0 penalty
T1 10×1 inaction
T1 11×0 inaction
T1 11×1 reward
T1 00×0 reward
T1 00×1 inactionFB2inc c x p s FB3
T1 01 0 0 inaction
T1 01 0 1 reward
T1 01 1 0 inaction
T1 01 1 1 penalty
T2 1××× inaction
T2 01 0 × penalty
T2 00×× inaction
The TM algorithm describes probabilities of generating T1
and T2[2, p. 10]. In this work we refer to them as P(T1)and
P(T2)which are based on csum andTas follows:


key: chroma:document
string_value:
P(T2)which are based on csum andTas follows:
P(T1)=T−clamp (csum)
2T,P(T2)=T+clamp (csum)
2T,
where the clamp function restricts its argument to [−T,T]such
that the probabilities lie in the range [0,1]. As the number
of activated clauses (those producing c=1 ) approaches T,
the probability of the TA s in that clause receiving feedback
decreases. In order to generate a randomized choice based on
these probabilities, we could assign booleans as follows:
q1=rand <P(T1),q2=rand <P(T2)


key: chroma:document
string_value:
q1=rand <P(T1),q2=rand <P(T2)
where rand is a real number in the range [0,1]. Since P(T1)
andP(T2)are complementary in the range [0,1], q1and q2
are also complementary in the binary domain and therefore
q2=q1. We therefore introduce a single new signal for FB2 ,
pT, which takes the place of the preceding probabilities. The
generation of this signal will be discussed in Section V.
C. FB3:TA-level Feedback
Inputs: FB2 ,inc,c,x,ps
Output: TAaction (inaction, penalty or reward)


key: chroma:document
string_value:
Output: TAaction (inaction, penalty or reward)
The third and ﬁnal feedback stage translates the feedback
type from FB2 into a TAaction for the speciﬁed TAbased on:
the TA’s current action, encoded as include in signal inc; the
current clause output, c; the feature of complemented feature
(forf) associated with the TA,x; and a probabilistic choice
component, ps.
The FB3 output is chosen according to Table I, where
ps=1 denotes that the higher probability (s−1)/soption is


key: chroma:document
string_value:
chosen from the payoff matrix. ps=0 denotes that the lower
probability 1/sis chosen. The generation of signal pswill be
discussed in Section V.
D. Synthesis of Feedback Circuits
Using one-hot encoding in the feedback circuits main-
tains direct compatibility with the inference datapath and
ensures speed independence, which is important for hazard-
freeness. Signals inc,c,xare already one-hot encoded (dual-
rail) for the inference circuits in Section III, so these en-


key: chroma:document
string_value:
codings are reused here in the learning circuits. Further-
more, we encode FB1 as follows, and FB2 similarly: FB1=
{FB1T2,FB1T1,FB1none}.FB3 describes TAactions and is encoded
as: FB3 ={FB3r,FB3p,FB3i}, for reward, penalty, inactionrespectively. Note that rail orders do not matter as we refer to
the rails by name. In this way, to indicate T2feedback on FB1 ,
we would set FB1T2=1,FB1T1=0,FB1none=0 .
Using the previously deﬁned logic, we can generate circuits


key: chroma:document
string_value:
for the feedback. The implementation of stage two feedback
depends on the clause polarity—it being either negated (c neg=
1) or non-negated (c neg=0 ). Note that the stage one circuit is
instantiated once per TM, the stage two circuit once per clause ,
and the stage three circuit once per TA.
In stage three, the FB2 signal is shared throughout the TA s
within the same clause. For this reason, we carefully design the
logic so that FB2 signals are injected as close to the outputs as


key: chroma:document
string_value:
possible, making the propagation path the shortest. Therefore
when FB2=0 , the computation on the corresponding FB3 rails
will conclude rapidly, and for all TA s in the clause.
V. G ENERA TION OF RANDOM BITSpT,ps
We introduced two distinct probabilistic choice mechanisms
in Section IV. In Section IV -B we introduced the signal pT:
a probabilistic boolean chosen at the clause level which may
force FB2=none. This boolean is dependent on Tand csum.


key: chroma:document
string_value:
The probability of pT=1 varies at runtime according to csum.
In Section IV -C we introduced the signal ps, which is a random
boolean generated for each TA. It is required under certain
circumstances to choose between two TAactions. This boolean
is dependent on sand is ﬁxed during runtime.
From these requirements, we must be able to generate
biased random bits: bits where P(1)̸=0.5(unbiased random
bits have P( 1 )=0 .5, such as those generated by a linear


key: chroma:document
string_value:
feedback shift register ( LFSR )). In addition, we must be able
to vary P(1) at runtime for pT. Probabilistic choices are used
in the TM to ensure diversity of learning. A pseudorandom
generator is sufﬁcient to satisfy these needs, and beneﬁts from
higher energy efﬁciency than a true random generator.
Due to the requirement of PRBGs for each TA, and addition-
ally for each clause, their area and energy consumption are of
utmost importance. We chose to implement a PRBG based on


key: chroma:document
string_value:
the principles of irregular sampling of a regular waveform, as
this allows us to minimize the overheads and take advantage
of the asynchronous nature of the rest of the system.
A. Asynchronous Sampling of Clock with V ariable Duty Cycle
We take advantage of the asynchronous inference logic by
using asynchronous handshakes to sample a regular clock
waveform. A clock with 50 % duty cycle will generate un-
biased random bits. A clock can be generated using a ring


key: chroma:document
string_value:
oscillator ( RO). To ensure random sampling, the clock and
handshake must be uncorrelated .ROs can be gated by adding a
NAND or NOR gate into the ring. However this technique will
disadvantage us in this case as the RO will always start up in
the same phase, therefore the clock and sampling signals may
become correlated.
We take advantage of the properties of a purely inverter-
based RO. The RO can be power gated using header/footer
transistor when entropy generation is not required. Such an RO


key: chroma:document
string_value:
44
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
will start up in a non-deterministic phase according to thermal
noise and other effects in the inverters.
To generate biased bits for pTand ps, we need to vary the
duty cycle of the clock. Agustin and Lopez-V allejo [14] show
how to construct an RO such that each tap has a unique duty
cycle. This is achieved by using inverters with asymmetric rise/
fall times. For example, odd inverters have fast rise and slow
fall times, and even inverters vice versa. This can be done via


key: chroma:document
string_value:
transistor sizing (ie. in the silicon library), or altering supply
voltage to the inverters (ie. at implementation stage). Using
this ROsetup, we can multiplex between taps to obtain different
clock duty cycles, and therefore alter our PRBG probability at
runtime.
We have introduced a clock into our asynchronous circuit,
and this may seem counterintuitive, however the load on this
clock (and therefore its energy consumption) will be low
compared with fully synchronous designs where the clock


key: chroma:document
string_value:
drives large numbers of ﬂip-ﬂops. In Section V -B we will
investigate the optimal number of PRBGs required. By taking
this distributed PRBG approach, we can vastly reduce the circuit
area compared with a naïve approach using one LFSR per TA.
Figure 8 shows the circuit used to select the correct duty
cycle clock and sample it with the asynchronous req input. The
mutex ensures ackpand acknoutputs are mutually exclusive.
These outputs represent the random output bit with a dual-rail


key: chroma:document
string_value:
encoding. That is for {ackp,ackn}:{0,1}represents logic-0,
and{1,0}logic-1.{0,0}is the spacer ornull state used to
separate valid output values temporally.
When the clk and req signals are both low, the output will
be in the spacer state. If a request is made (by asserting req)
during the low period of the clock, the output of the set-
dominant latch is reset, and req wins the mutex, resulting in
acknasserting. If clk rises now, the output of the latch rises,


key: chroma:document
string_value:
however the mutex is still held by req. A following deassertion
ofreq will result in clk gaining the mutex, however ackpwill
be masked by the AND gate.
If a request is made during the high period of clk, the output
of the latch will already be high and clk will have won the
mutex. On the rise of req the AND gate will unmask ackp.I f
clk falls now, the output of the latch remains high since the R
input is inactive, and therefore ackpalso remains high.


key: chroma:document
string_value:
The circuit is synthesized for the UMC 65LL cell library
and the results are summarized in Table II. The asynchronous
handshake shows almost 10×saving in area compared to an 8-
bit LFSR . Power and energy are also drastically decreased. The
asynchronous handshake is well suited to energy-conscious,
pervasive applications.
B. Optimal Number of PRBGs
The original TM algorithm takes a probabilistic choice for
every TAupdate involving s. However Abeyrathna et al. [15]


key: chroma:document
string_value:
show that randomizing every dth TA update can save energy
with a minimal drop in test accuracy. With d= 1000 ,a n
accuracy within 2% ofd=1 was maintained for three of the
four datasets tested.MutexS•
RQclk
dc→
from
RO
reqackpackn
Figure 8. Handshake circuit for the asynchronous pseudorandom bit generator
with dual-rail output.
Table II
PRBG SSYNTHESIZED FOR UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .
Implementation Area Cycle Time [ ns] Power [μ W] Energy [ fJ]
LFSR8 92.20 .38 6.41 2 .44


key: chroma:document
string_value:
LFSR8 92.20 .38 6.41 2 .44
Handshake 9.72 0 .23 0.621 0 .143
Following on, we could hypothesize an optimal number
of PRBGs to fulﬁll the needs of a given TM. If every 100 th
TA update is randomized, we will need to produce at most
NTA s/100 probabilistic choices per learning phase.
This could result in a 100×reduction in PRBGs for an
LFSR -based design, in exchange for arbitration overhead to
share one PRBG between multiple TA s. Similar could be said


key: chroma:document
string_value:
for the asynchronous handshake design, although the already
small size of the design means that arbitration overhead would
almost certainly result in a larger area than instantiating an
asynchronous handshake PRBG for every TA.
VI. A SYNCHRONOUS TSETLIN AUTOMA TON
In a TM, the TA s consume most of the hardware re-
sources [6]. The number of TA s is given by NTA s =
2NClasses NClauses NInputs . Therefore the goal is to create a
lightweight TA implementation with a focus on area and


key: chroma:document
string_value:
energy. The TA hardware must implement the ﬁnite state
automaton described by the state diagram in Figure 2.
This section explores two methods for the design of a TA.
Firstly a QDI approach starting from an STG speciﬁcation; and
secondly a BDapproach starting from a traditional synchronous
speciﬁcation, using matched delays to time the design. The
designs will be compared with a synchronous implementation.
A. Quasi Delay Insensitive Approach


key: chroma:document
string_value:
A. Quasi Delay Insensitive Approach
The STG in Figure 9 represents a one-hot ﬁnite state machine
(FSM ) for a two action, six-state TA(for the deﬁnition of STGs ,
see [16]). The inputs pand rcome from the FB3 rails FB3p
and FB3rwhich were introduced in Section IV -D. The FB3irail
denotes inaction and is only used for completion detection.
State bits are encoded in the x∗∗_0and x∗∗_1places at the
top of the graph. States x1∗(x2∗) are the action 1 (action 2)


key: chroma:document
string_value:
states—in these states the automaton is indicating the action
to exclude (include) the feature or feature complement from
the clause composition. The x∗1(x∗3) states are the closest to
(furthest from) the action decision boundary. As an example,
when x13_1holds a token, the automaton is indicating action
1 (exclude) strongly.
A token at p0denotes the idle state for the automaton. It
offers a choice between penalty and reward represented by
45


key: chroma:document
string_value:
45
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
signal transitions p+and r+respectively. Each subsequent
branch from p+,r+, enables a transition in one of the action
output signals a1,a2. Depending on the current action and the
previous state, a state transition may also occur. The correct
branch is chosen based on read arcs from the x∗∗_1places. For
example in the 2nd left-most branch of r+, we transition from
state x12tox13. This happens by ﬁrstly giving the token from
place x13_0tox13_1, and secondly giving token x12_1tox12_0.


key: chroma:document
string_value:
After state transitions and action output have occurred, the
acknowledge output, ack , makes a positive transition. Finally,
the previously given input and action signals return low,
followed by ack−and the token returns to p0. The automaton
is now ready for the next penalty or reward input.
The internal transitions x∗∗encode some information about
the states in the STG . However, this is not enough to satisfy
complete state coding ( CSC ) required for synthesis of the


key: chroma:document
string_value:
STG to logic gates [16]. In the previous example where we
transition from x12tox13, there is a point where tokens are
held by both x13_1and x12_1. The STG reaches this same state
when transitioning in the opposite direction, from x12tox13.
In order to achieve CSC , we introduce internal signals to
encode the direction of travel of the state: x∗L∗,x∗R∗; for left
and right. We insert these signals into every branch to maintain
uniformity in the STG . Although not all these internal signals


key: chroma:document
string_value:
are strictly required, they ease scalability of the STG and help
the synthesis tool to minimize and share logic efﬁciently.
The STG in Figure 9 passes all veriﬁcation tasks required
for synthesis of a correct and hazard-free circuit in the
WORKCRAFT framework: consistency, deadlock freeness, in-
put properness, and output persistency. The circuit was syn-
thesized and mapped to the target technology by the MPS AT
backend (the P ETRIFY backend did not produce a result in a


key: chroma:document
string_value:
reasonable amount of time).
In order to read the state of the TAwithout giving penalty or
reward (during inference for example), we add a latch to the
output to store one of the porrsignals. The latch is controlled
by the ack handshake.
The STG state space expands exponentially as more signal
transitions are added. This makes it difﬁcult for the synthesis
tool to handle TA s with more than three action states. Addition-
ally there is a high possibility for human error when copying


key: chroma:document
string_value:
and editing branches of the STG . And as the number of TA
action states increases, the graph becomes large and unwieldy
to navigate. For these reasons we explore a BD approach to
asynchronous TAdesign.
B. Bundled-Data Approach
A synchronous one-hot FSM can be constructed using ﬂip-
ﬂops as storage elements. The next state logic can be derived
using the adjacent states and p,rinputs as follows:
x13=x13·r+x12·r
x12=x11·r+x13·p
x11=x12·p+x21·px21=x22·p+x11·p
x22=x21·r+x23·p
x23=x23·r+x22·r


key: chroma:document
string_value:
x22=x21·r+x23·p
x23=x23·r+x22·r
This synchronous circuit is converted into a self-timed one
through the process of desynchronization [12], by decom-posing the ﬂip-ﬂops into master/slave latches and replacing
the clock with handshake controllers and delays matched to
the combinational logic paths. The timing of the resultant BD
design is not as robust as the QDI approach in Section VI-A
since the matched delays are ﬁxed at design time and are


key: chroma:document
string_value:
subject to process, variation and temperature variations.
To reduce the overhead of the handshake controllers, we
group the TA s into an array and use one handshake controller
for many latches. Depending on the overall size of TM for
the target application, the TA s could be grouped in different
ways. A monolithic array will minimize controller overhead,
however there comes a limit where buffers must be introduced
into the latch enable signals to maintain integrity. At this point,


key: chroma:document
string_value:
it may become desirable to group the TA s by clause.
The speciﬁcation of the BD design is easily scaled due to
the use of parameterizable V erilog code. The workﬂow takes
advantage of mature synchronous tools for synthesis.
C. Synthesis Results
Timing and power analyses of asynchronous circuits need
special consideration. In conventional synchronous static tim-
ing analysis, delay through logic gates is calculated using the
input slew and output load capacitance. Power estimations also


key: chroma:document
string_value:
rely on these parameters. Synchronous tools cut combinational
loops arbitrarily, therefore may be optimistic about slews,
gate delays and internal power in asynchronous circuits which
contain such loops.
To obtain results for the QDI design we use a specialized
asynchronous static timing analysis tool [17] which takes into
account the combinational loops in the timing calculation.
The tool operates on the gate-level netlist and propagates


key: chroma:document
string_value:
slews iteratively through circuits with loops, therefore avoiding
optimism in the slews and gate delays. In the case there are
multiple paths in the circuit which begin and end at the same
input/output combination, the tool uses the worst-case path to
determine the cycle time.
Since the UMC 65LL library does not contain C-elements,
the QDI circuit is combinational with feedback loops. Table III
shows a results summary of the TAimplementations. All results


key: chroma:document
string_value:
are synthesis estimations and no layout is performed. In this
case, the QDI circuit trades increased cycle time and energy
for increased robustness to process, variation and temperature
variations. The BD design strikes a good trade-off between
performance, area and elastic timing to integrate with the rest
of the system. The layout process will introduce more power
Table III
RESUL TS FOR THE SIX -ST A TE TSETLIN AUTOMA TON SYNTHESIZED FOR
THE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .


key: chroma:document
string_value:
THE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .
Implementation AreaCycle Time
[ns]Leakage
[nW ]Power
[μW]Energy
[fJ]
Synchronous 87.80 .35 3 .08 4 .58 1 .60
Bundled-data386.00 .41 2 .89 5 .19 2 .13
QDI 123 3 .35 3 .42 1 .58 5 .29
4Averaged over the array.
46
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
	


 








	


Figure 9. Signal transition graph for a Tsetlin automaton with two actions and six states. Inputs: p,r. Outputs: a1,a2,ack . Other signals are internal.
and latency into the synchronous and BD designs due to their
clock trees, which the QDI design will not suffer from.
VII. C ONCLUSION
Our Petri net visualization of the TM architecture enables


key: chroma:document
string_value:
ﬂexible hardware implementations by encapsulating both con-
currency and serialization. By decoupling the TA storage
elements from the clauses, e.g. in a standalone array, the TM
could be serialized further by reusing clause and sum hardware
across multiple cycles, trading off throughput for reduced logic
area (and leakage power).
Early propagation enabled by the RCD scheme used in the DR
circuits can be leveraged to trigger ﬁne-grained power gating


key: chroma:document
string_value:
in the system to further reduce energy consumption.
Distributed generation of probabilistic choices maximizes
concurrency and decreases energy overheads compared to
a centralized approach. A simple PRBG implementation is
sufﬁcient and most suitable for our applications. Features
of a clock source undesirable in synchronous circuits (jitter,
etc.) can be leveraged for stochasticity. Further study of PRBG
properties and their effects on TM learning are a subject for
future work.
ACKNOWLEDGMENT


key: chroma:document
string_value:
future work.
ACKNOWLEDGMENT
The authors thank Christos Sotiriou and colleagues for help
in asynchronous ST A and for access and support for their tool.
We are also grateful to colleague Thomas Bunnam for fruitful
discussions on ROs and RNG techniques. The authors would
like to acknowledge the funding support from EPSRC IAA
grant: Whisperable.
REFERENCES
[1] A. Wheeldon et al. , “Low-Latency Asynchronous Logic Design for
Inference at the Edge,” in Proc. Conf. Des. Autom. Test Eur. , Feb. 2021,


key: chroma:document
string_value:
pp. 370–373.[2] O.-C. Granmo, “The Tsetlin Machine – A Game Theoretic Bandit Driven
Approach to Optimal Pattern Recognition with Propositional Logic,”
arXiv:1804.01508v15 [cs.AI] , Apr. 2018.
[3] G. T. Berge et al. , “Using the Tsetlin Machine to Learn Human-
Interpretable Rules for High-Accuracy Text Categorization with Medical
Applications,” arXiv:1809.04547 [cs, stat] , Sep. 2018.
[4] C. D. Blakely and O.-C. Granmo, “Closed-Form Expressions for Global


key: chroma:document
string_value:
and Local Interpretation of Tsetlin Machines with Applications to
Explaining High-Dimensional Data,” arXiv:2007.13885 [cs, stat] , Jul.
2020.
[5] B. Bhattarai, O.-C. Granmo, and L. Jiao, “Measuring the Novelty of
Natural Language Text Using the Conjunctive Clauses of a Tsetlin
Machine Text Classiﬁer,” arXiv:2011.08755 [cs] , Nov. 2020.
[6] A. Wheeldon et al. , “Learning automata based energy-efﬁcient AI
hardware design for IoT applications,” Philos Trans R Soc Math Phys


key: chroma:document
string_value:
Eng Sci , vol. 378, no. 2182, Oct. 2020.
[7] ——, “Self-timed, minimum latency circuits for the internet of things,”
Integration , vol. 69, pp. 138–146, Nov. 2019.
[8] D. Sokolov et al. , “Automating the Design of Asynchronous Logic
Control for AMS Electronics,” IEEE Trans. Comput.-Aided Design
Integr . Circuits Syst. , vol. 39, no. 5, pp. 952–965, 2020.
[9] X. Zhang et al. , “A Novel Power Delivery Method for Asynchronous
Loads in Energy Harvesting Systems,” J. Emerg. Technol. Comput. Syst. ,


key: chroma:document
string_value:
vol. 7, no. 4, pp. 1–22, Dec. 2011.
[10] D. Balsamo et al. , “Wearable and autonomous computing for future
smart cities: Open challenges,” in 2017 25th Int. Conf. on Software
Telecommun. Comput. Netw. (SoftCOM) , 2017, pp. 1–5.
[11] “The workcraft framework.” [Online]. Available: https://workcraft.org/
[12] J. Cortadella et al. , “Desynchronization: Synthesis of asynchronous
circuits from synchronous speciﬁcations,” IEEE Trans Comput Integr


key: chroma:document
string_value:
Circuits Syst , vol. 25, no. 10, pp. 1904–1921, Oct. 2006.
[13] K. S. Narendra and M. A. L. Thathachar, Learning Automata: An
Introduction . Englewood Cliffs, N.J: Prentice Hall, 1989.
[14] J. Agustin and M. Lopez-V allejo, “An in-depth analysis of ring oscilla-
tors: Exploiting their conﬁgurable duty-cycle,” IEEE Trans Circuits Syst
Regul Pap , vol. 62, no. 10, pp. 2485–2494, Oct. 2015.
[15] K. D. Abeyrathna et al. , “A Novel Multi-Step Finite-State Automaton


key: chroma:document
string_value:
for Arbitrarily Deterministic Tsetlin Machine Learning,” in Proc. 40th
SGAI Int. Conf. Artif. Intell. , vol. 7, Jul. 2020.
[16] J. Cortadella et al. ,Logic synthesis of asynchronous controllers and
interfaces . Springer-V erlag Berlin Heidelberg, 2002.
[17] S. Simoglou et al. , “Graph-based ST A for asynchronous controllers,”
Integration , vol. 75, pp. 91–101, Nov. 2020.
47


key: chroma:document
string_value:
Integration , vol. 75, pp. 91–101, Nov. 2020.
47
Authorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply.


key: chroma:document
string_value:
Measuring the Novelty of Natural Language Text Using the
Conjunctive Clauses of a Tsetlin Machine Text Classiﬁer
Bimal Bhattarai
University of Agder
Grimstad, Norway
bimal.bhattarai@uia.noOle-Christoffer Granmo
University of Agder
Grimstad, Norway
ole.granmo@uia.noLei Jiao
University of Agder
Grimstad, Norway
lei.jiao@uia.no
Abstract
Most supervised text classiﬁcation approaches assume a closed world, counting on all classes


key: chroma:document
string_value:
being present in the data at training time. This assumption can lead to unpredictable behaviour
during operation, whenever novel, previously unseen, classes appear. Although deep learning-
based methods have recently been used for novelty detection, they are challenging to interpret
due to their black-box nature. This paper addresses interpretable open-world text classiﬁcation,
where the trained classiﬁer must deal with novel classes during operation. To this end, we extend


key: chroma:document
string_value:
the recently introduced Tsetlin machine (TM) with a novelty scoring mechanism. The mecha-
nism uses the conjunctive clauses of the TM to measure to what degree a text matches the classes
covered by the training data. We demonstrate that the clauses provide a succinct interpretable
description of known topics, and that our scoring mechanism makes it possible to discern novel
topics from the known ones. Empirically, our TM-based approach outperforms seven other nov-


key: chroma:document
string_value:
elty detection schemes on three out of ﬁve datasets, and performs second and third best on the
remaining, with the added beneﬁt of an interpretable propositional logic-based representation.
1 Introduction
In recent years, deep learning-based techniques have achieved superior performance on many text clas-
siﬁcation tasks. Most of the classiﬁers use supervised learning, assuming a closed-world environment


key: chroma:document
string_value:
(Scheirer et al., 2013). That is, the classes present in the test data (or during operation) are also assumed
to be present in the training data. However, when facing an open-world environment, new classes may
appear after training (Bendale and Boult, 2016). In such cases, assuming a closed world can lead to
unpredictable behaviour. For example, a chatbot interacting with a human user will regularly face new


key: chroma:document
string_value:
user intents that it has not been trained to recognize. A chatbot for banking services may, for instance,
have been trained to recognize the intent of applying for a loan. However, it will provide meaningless
responses if it fails to recognize that asking for a lower interest rate is a new and different user intent. The
problem with neural network-based supervised classiﬁers that use the typical softmax layer is that they


key: chroma:document
string_value:
erroneously force novel input into one of the previously seen classes, by normalizing the class output
scores to produce a distribution that sums to 1.0. Instead, a robust classiﬁer should be able to ﬂag input
as novel, rejecting to label it according to the presently known classes. Recently, many important appli-
cation areas make use of novelty detection such as medical applications, fraud detection (Veeramreddy


key: chroma:document
string_value:
et al., 2011), sensor networks (Zhang et al., 2010) and text analysis (Basu et al., 2004). For a further
study of these classes of techniques, the reader is referred to (Pimentel et al., 2014).
Figure 1 illustrates the problem of novelty detection, i.e., recognizing when the data fed to a classiﬁer
is novel and somehow differs from the data that was available during training. In brief, after training on


key: chroma:document
string_value:
data with known classes (blue data points to the right in Figure 1), the classiﬁer should be able to detect
novel data arising from new, previously unseen, classes (yellow data points to the right in the ﬁgure). We
will refer to the known data points as positive examples and the novel data points as negative examples .
Note that the problem of novelty detection is closely related to so-called outlier detection (Chandola et


key: chroma:document
string_value:
al., 2009)(Pincus, 1995). However, as exempliﬁed to the left in Figure 1, the latter problem involves
ﬂagging data points that are part of an already known class, yet deviating from the other data points, e.g.,
due to measurement errors or anomalies (red point).arXiv:2011.08755v1  [cs.CL]  17 Nov 2020


key: chroma:document
string_value:
Outlier decision boundaryOutlier detection Novelty detection
trained example
distant
known classnovel exampleFigure 1: Visualization of outlier detection and novelty detection.
Problem Deﬁnition: Generally, in multiclass classiﬁcation, we have a set of example data points
X= (xi,yi),xi∈Rs,wherei= 1,2,3,...,N indexes the positive examples, sis the dimensionality
of the data point xi, andyiis the label of xi, assigning it a class. For any data point xi, also referred


key: chroma:document
string_value:
to as a feature vector, a classiﬁer function ˆy=f(x;X)is to assign the data point a predicted class ˆy,
after the function has been ﬁtted to the training data X. Additionally, a novelty scoring function z(x;X),
also ﬁtted on X, calculates a novelty score, so that a threshold σcan be used to recognize novel input.
In other words, the classiﬁer is to return the correct class label while simultaneously being capable of
rejecting novel examples.


key: chroma:document
string_value:
rejecting novel examples.
Under the above circumstances, standard supervised learning would fail, particularly for methods
based on building a discriminant function, such as neural networks. As shown to the left of Figure 1, a
discriminant function captures the discriminating “boundaries” between the known classes and cannot
readily be used to discern novel classes. Still, a traditional way of implementing novelty detection is to


key: chroma:document
string_value:
threshold the entropy of the class probability distribution (Hendrycks and Gimpel, 2016). Such methods
are not actually measuring novelty, but closeness to the decision boundary. Such an approach thus leads
to undetected novel data points when the data points are located far from the decision boundary. Another,
and perhaps more robust, approach is to take advantage of the class likelihood function, which estimates
the probability of the data given the class.


key: chroma:document
string_value:
the probability of the data given the class.
Paper Contributions: Unlike traditional methods, in this paper we leverage the conjunctive clauses
in propositional logic that a TM builds. TM clauses represent frequent patterns in the data, and our
hypothesis is that these frequent patterns characterize the known classes succinctly and comprehensively.
We thus establish a novelty scoring mechanism that is based simply on counting how many clauses match


key: chroma:document
string_value:
the input. This score can, in turn, be thresholded manually to ﬂag novel input. However, for more robust
novelty detection, we train several standard machine learning techniques to ﬁnd accurate thresholds. The
main contributions of our work can thus be summarized as follows:
• We devise the ﬁrst TM-based approach for novelty detection, leveraging the clauses of the TM
architecture.
• We illustrate how the new technique can be used to detect novel topics in text, and compare the


key: chroma:document
string_value:
technique against widely used approaches on ﬁve different datasets.
2 Related Work
The perhaps most common approach to novelty detection is distance-based methods (Hautamaki et al.,
2004), which assumes that the known or seen data are clustered together while novel data have a high
distance to the clusters. The major drawback of these methods is computational complexity when per-
forming clustering or nearest neighbor search in a large dataset. Early work on novelty detection also


key: chroma:document
string_value:
includes one-class SVMs (Sch ¨olkopf et al., 2001), which are only capable of using the positive training
examples to maximize the class margin. This shortcoming is overcome by the Center-Based Similarity


key: chroma:document
string_value:
(CBS) space learning method (Fei and Liu, 2015), which uses binary classiﬁers over vector similarities
of training examples transformed into the center of the class. To build a classiﬁer for detecting novel
class distributions, Chow et al. proposed a conﬁdence score-based method that suggests an optimum
input rejection rule (Chow, 1970). The method is relatively accurate, but it does not scale well to high-
dimensional datasets.


key: chroma:document
string_value:
dimensional datasets.
The novelty detection method OpenMax (Bendale and Boult, 2016) is more recent, and estimates the
probability of the input belonging to a novel class. To achieve this, the method employs an extra layer,
connected to the penultimate layer of the original network. The computational complexity of the method
is however high, and the underlying inference cannot easily be interpreted for quality assurance. Lately,


key: chroma:document
string_value:
Yu et al. (Yu et al., 2017) adopted the Adversarial Sample Generation (ASG) framework (Hautamaki
et al., 2004) to generate positive and negative examples in an unsupervised manner. Then, based on
those examples, they trained an SVM classiﬁer for novelty detection. Furthermore, in computer vision,
Scheirer et al. introduced the concept of open space risk to recognize novel image content (Scheirer
et al., 2013). They proposed a “1-vs-set machine” that creates a decision space using a binary SVM


key: chroma:document
string_value:
classiﬁer, with two parallel hyperplanes bounding the non-novel regions. In Section 4, we compare
the performance of our new TM-based approach with the most widely used approaches among those
mentioned above.
3 Tsetlin Machine-based Novelty Detection
In this section, we propose our approach to novelty detection based on the TM. First, we explain the
architecture of the TM. Then, we show how novelty scores can be obtained from the TM clauses. Finally,


key: chroma:document
string_value:
we integrate the TM with a rule-based classiﬁer for novel text classiﬁcation.
3.1 Tsetlin Machine (TM) Architecture
The TM is a recent approach to pattern classiﬁcation (Granmo, 2018) and regression (Dar-
shana Abeyrathna et al., 2020). It builds on a classic learning mechanism called a Tsetlin automaton
(TA), developed by M. L. Tsetlin in the early 1960s (Tsetlin, 1961). In all brevity, multiple teams of TA


key: chroma:document
string_value:
combine to form the TM. Each team is responsible for capturing a frequent pattern of high precision,
by composing a conjunctive clause. In-built resource allocation principles guide the teams to distribute
themselves across the underlying sup-patterns of the problem. Recently, the TM has performed compet-
itively with the state-of-the-art techniques, including deep neural networks for text classiﬁcation (Berge


key: chroma:document
string_value:
et al., 2018). Further, the convergence of TM learning has been studied in (Zhang et al., 2020). In
the following, we propose a new scheme that extends the TM with the capability of recognizing novel
patterns.
As explored in the following, Figure 2 capture the building blocks of a TM. As seen, a vanilla TM
takes a vector X= (x1,...,x o)of binary features as input (Figure 3). We binarize text by using binary


key: chroma:document
string_value:
features that capture the presence/absence of terms in a vocabulary, akin to a bag of words, as done in
(Berge et al., 2018). However, as opposed to a vanilla TM, our scheme does not output the predicted
class. Instead, it calculates a novelty score per class.
Together with their negated counterparts, ¯xk=¬xk= 1−xk, the features form a literal set L=
{x1,...,x o,¯x1,..., ¯xo}. A TM pattern is formulated as a conjunctive clause Cj, formed by ANDing a
subsetLj⊆Lof the literal set:
Cj(X) =⋀


key: chroma:document
string_value:
subsetLj⊆Lof the literal set:
Cj(X) =⋀
lk∈Ljlk=∏
lk∈Ljlk. (1)
For example, the clause Cj(X) =x1x2consists of the literals Lj={x1,x2}and outputs 1ifx1=
x2= 1. The number of clauses employed is a user set parameter m. Half of the clauses are assigned a
positive polarity and the other half are assigned a negative polarity. The output of a conjunctive clause
is determined by evaluating it on the input literals. When a clause outputs 1, this means that it has


key: chroma:document
string_value:
recognized a pattern in the input. Conversely, the clause outputs 0when no pattern is recognized. The
clause outputs, in turn, are combined into a classiﬁcation decision through summation and thresholding


key: chroma:document
string_value:
using the unit step function u:
ˆy=u
m/2∑
j=1C+
j(X)−m/2∑
j=1C−
j(X)
. (2)
That is, the classiﬁcation is performed based on a majority vote, with the positive clauses voting for
y= 1and the negative for y= 0. The classiﬁer ˆy=u(x1¯x2+ ¯x1x2−x1x2−¯x1¯x2), e.g., captures the
XOR-relation.
 
 
 
Novelty Score 
 
  
 
 
yes NoType I/II feedback
Training clauses
Figure 2: Optimization of clauses in TM for generating novelty score.


key: chroma:document
string_value:
A clause is composed by a team of TA, each TA deciding to Exclude orInclude a speciﬁc literal in the
clause. The TA learns which literals to include based on reinforcement: Type I feedback is designed to
produce frequent patterns, while Type II feedback increases the discriminating power of the patterns (see
(Granmo, 2018) for details).
3.2 Novelty Detection Architecture
For novelty detection, however, we here propose to treat all clause output as positive, disregarding clause


key: chroma:document
string_value:
polarity. This is because both positive and negative clauses capture patterns in the training data, and thus
can be used to detect novel input. We use this sum of absolute clause outputs as a novelty score, which
denotes the resemblance of the input to the patterns formed by clauses during training. The resulting
modiﬁed TM architecture is captured by Figure 3, showing how four different outputs are produced by
the TM, two per class. These outputs form the basis for novelty detection.


key: chroma:document
string_value:
The overall novelty detection architecture is shown in Figure 4. Each TM (one per class) produces
two novelty score for its respective class, one based on the positive polarity clauses and one based on the
negative polarity ones. The novelty scores are normalized and then given to a classiﬁer, such as decision


key: chroma:document
string_value:
Text Input
Binary encoded input
Class 1 Class 2
Positive
Novelty Score
Positive
Novelty ScoreInput
Clauses
Output
Negative
Novelty Score
Negative
Novelty ScoreFigure 3: Multiclass Tsetlin Machine (MTM) framework to produce the novelty score for each class.
Seen Class
 Novel Class
Novelty Score
 Novelty Score
Classifier
Output (0 or 1)
Figure 4: Novelty detection architecture.
tree (DT), K nearest neighbor (KNN), Support vector machine (SVM) and logistic regression (LR). The


key: chroma:document
string_value:
output from these classiﬁers are ”Novel” or ”Not Novel”, i.e. 0or1in the ﬁgure.
For illustration purposes, instead of using a machine learning algorithm to decide upon novelty, one
can instead use a simple rule-based classiﬁer, introducing a classiﬁcation threshold T. Then the novelty
score for input sentence can be compared with Tto detect whether a sentence is novel or not. That is,
Tdecides how many clauses must match to qualify the input as non-novel. The classiﬁcation function


key: chroma:document
string_value:
F(X)for a single class can accordingly be given as:
F(X) =

1,if∑m/2
j=1C+
j(X)>T,
1,if∑m/2
j=1C−
j(X)<−T,
0,otherwise.
4 Experiments and Results
In this section, we present the experimental evaluation of our proposed TM model and compare it with
other baseline models across ﬁve datasets.


key: chroma:document
string_value:
4.1 Datasets
We used the following datasets for evaluation.
• 20-newsgroup dataset: The dataset contains 20 classes with a total of 18,828 documents. In our ex-
periment, we consider the two classes “comp.graphics” and“talk.politics.guns” as known classes
and the class “rec.sport.baseball” as novel. We take 1000 samples from both known and novel
classes for novelty classiﬁcation.
• CMU Movie Summary corpus: The dataset contains 42,306 movie plot summaries extracted from


key: chroma:document
string_value:
Wikipedia and metadata extracted from Freebase. In our experiment, we consider the two movie
categories “action” and“horror” as known classes and “fantasy” as novel.
• Spooky Author Identiﬁcation dataset: The dataset contains 3,000 public domain books from the
following horror ﬁction authors: Edgar Allan Poe (EAP), HP Lovecraft (HPL) , and Mary Shelley
(MS) . We train on written texts from EAP andHPL while treating texts from MSas novel.


key: chroma:document
string_value:
• Web of Science dataset (Kowsari et al., 2017): This dataset contains 5,736 published papers, with
eleven categories organized under three main categories. We use two of the main categories as
known classes, and the third as a novel class.
• BBC sports dataset: This dataset contains 737 documents from the BBC Sport website, organized
in ﬁve sports article categorie and collected from 2004 to 2005. In our work, we use “cricket” and
“football” as the known classes and “rugby” as novel.


key: chroma:document
string_value:
4.2 A Case Study
To cast light on the interpretability of our scheme, we here use substrings from the 20 Newsgroup dataset,
demonstrating novelty detection on a few simple cases. First, we form an indexed vocabulary set V
including all literals from the dataset. The input text is binarized based upon the index of the literals in
V. For example, if a word in the text substring has been assigned index 5inV, the5thposition of the


key: chroma:document
string_value:
input vector is set to 1. If a word is absent from the substring, its corresponding feature is set to 0. Let
us consider substrings from the two known classes and the novel class from the 20 Newsgroup dataset.
•Class : comp.graphics (known)
Text: Presentations are solicited on all aspects of Navy-related scientiﬁc visualization and virtual
reality.
Literals :“Presentations” ,“solicited” ,“aspects” ,“Navy” ,“related” ,“scientiﬁc” ,“visualiza-
tion” ,“virtual” ,“reality” .


key: chroma:document
string_value:
tion” ,“virtual” ,“reality” .
•Class : talk.politics.guns (known)
Text: Last year the US suffered almost 10,000 wrongful or accidental deaths by handguns alone. In
the same year, the UK suffered 35 such deaths.
Literals :“Last ” ,“year” ,“US” ,“suffered” ,“wrongful” ,“accidental” ,“deaths” ,“handguns” ,
“UK” ,“suffered” .
•Class : rec.sport.baseball (Novel)
Text: The top 4 are the only true contenders in my mind. One of these 4 will deﬁnitely win the
division unless it snows in Maryland.


key: chroma:document
string_value:
division unless it snows in Maryland.
Literals :“top ” ,“only” ,“true” ,“contenders” ,“mind” ,“win” ,“division” ,“unless” ,“snows” ,
“Maryland”
After training, the two known classes form conjunctive clauses that capture literal patterns reﬂecting the
textual content. For the above example, we get the following clauses:
•C+
1=“Presentations”∧“aspects”∧“Navy”∧“scientiﬁc”∧“virtual”,“reality”∧“year”∧“US”
∧“mind”∧“division”


key: chroma:document
string_value:
•C−
1=¬(“suffered”∧“accidental”∧“unless”∧“snows” )
•C+
2=“last”∧“year”∧“US”∧“wrongful”∧“deaths”,“accidental”∧“handguns”∧“Navy”
∧“snows”∧“Maryland”∧“divisions”
•C−
2=¬(“presentations”∧“solicited”∧“virtual”∧“top”∧“win” )
Here, the clauses from each class captures the frequent patterns from the class. However, it may also
contain certain literals from other classes. The positive polarity clauses provide evidence on the presence


key: chroma:document
string_value:
of a class, while negative polarity clauses provide evidence on the absence of the class. The novelty
score for each class is calculated based on the propositional formula formed by the clauses. In general,
for input from known classes, the novelty scores are higher. This is because the clauses have been trained
to vote for or against input from the known class. For example, when we pass a known class to our model,
the clauses might produce scores as in Table 1 (for illustration purposes).


key: chroma:document
string_value:
ClassC+
1C−
1C+
2C−
2
Known +6 -3 +3 -5
Novel +2 -1 +1 -2
Table 1: Novelty score example when known text sentence is passed to the model.
The scores are then used as features to prepare a dataset for employing machine learning classiﬁers to
enhance novelty detection. As exampliﬁed in the table, novelty scores for known classes are relatively
higher than those of novel classes. This allows the ﬁnal classiﬁer to robustly recognize novel input, as
explored empirically below.


key: chroma:document
string_value:
explored empirically below.
4.3 Empirical Evaluation
We divided the task into two experiments i.e., 1) Novelty score calculation 2) Novelty/Known Class
classiﬁcation. In the ﬁrst experiment, we employed the known classes to train the TM. The TM runs
for 100 epochs with hyperparameter setting of 5000 clauses, a threshold Tof 25, and a sensitivity s
of 15.0. Then, we use the clauses formed by the trained TM model to calculate the novelty scores for


key: chroma:document
string_value:
both the known and novel classes. We adopt an equal number of examples to gather the novelty score
from both known and novel classes. In the second experiment, the novelty score generated from the ﬁrst
experiment is forwarded as input to standard machine learning classiﬁers, such as DT, KNN, SVM, LR,
NB, and MLP to classify whether a text is novel.
Dataset DT KNN SVM LR NB MLP
20 Newsgroup 72.5 % 82 % 78.0 % 72.75 % 69.0 % 82.50 %


key: chroma:document
string_value:
Spooky action author 53.42 % 57.89 % 63.15 % 52.63 % 58.68 % 63.15 %
CMU movie 61.05 % 64.73 % 62.10 % 55.00 % 58.94 % 68.68 %
BBC sports 84.21 % 85.96 % 75.43% 70.17 % 73.68 % 89.47 %
Web of Science 64.70 % 67.97 % 69.93 % 67.10 % 62.09 % 70.37 %
Table 2: Accuracy of different machine learning classiﬁers to detect novel class in various dataset.
The experimental results for all datasets are shown in Table 2. As seen, multilayer perceptron (MLP)


key: chroma:document
string_value:
(hidden layer sizes 100,30 and relu activation with stochastic gradient descent) is superior for all of the
datasets. In our experiments, the 20 Newsgroups and BBC sports datasets yielded better results than the
three other data sets, arguably because of the sharp distinctiveness of examples in the known and novel
classes. We further believe that the novelty scores are clustered based on known and novel classes, thus,


key: chroma:document
string_value:
distance-based methods seem effective in classiﬁcation. We plotted the novelty score of thousand text
samples from known and novel classes using our framework, which shows how scores differ signiﬁcantly
for each sample as visualized in Figure 3. Moreover, we can see from the graph that when the texts from


key: chroma:document
string_value:
/s50/s48/s48 /s52/s48/s48 /s54/s48/s48 /s56/s48/s48 /s49/s48/s48/s48/s48/s49/s48/s48/s50/s48/s48/s51/s48/s48/s52/s48/s48/s53/s48/s48/s83/s99/s111/s114/s101
/s83/s97/s109/s112/s108/s101/s32/s83/s101/s101/s110/s32/s99/s108/s97/s115/s115
/s32/s78/s111/s118/s101/s108/s32/s99/s108/s97/s115/s115Figure 5: Visualization of differences in novelty score for known and novel classes
the novel and known classes are discriminative, the TM produces distinct novelty scores, thus improving


key: chroma:document
string_value:
the ability of machine learning classiﬁers to detect novel texts. In all brevity, we believe that the clauses
of a TM capture frequent patterns in the training data, and thus novel samples will reveal themselves by
not ﬁtting with a sufﬁciently large number of clauses. The number of triggered clauses can therefore be
utilized to measure novelty score. Also, the clauses formed by trained TM models should only to a small
degree trigger on novel classes, producing distinctively low scores.


key: chroma:document
string_value:
Algorithms 20 Newsgroup Spooky action author CMU movie BBC sports WOS
LOF 52.51 % 50.66 % 48.84 % 47.97 % 55.61 %
Feature Bagging 67.60 % 62.70 % 64.73 % 54.38 % 69.64 %
HBOS 55.03 % 48.55 % 48.57 % 49.53 % 55.09 %
Isolation Forest 52.01 % 48.66% 49.10 % 49.35% 54.70 %
Average KNN 76.35 % 57.76 % 56.21 % 55.54 % 79.22 %
K-Means clustering 81.00 % 61.30 % 49.20 % 47.70 % 41.31 %
One-class SVM 83.70 % 43.56 % 51.94 % 83.53 % 36.32 %
TM framework 82.50 % 63.15% 68.15 % 89.47 % 70.37 %


key: chroma:document
string_value:
Table 3: Performance comparison of proposed TM framework with cluster and outlier based novelty
detection algorithms.
We compared the performance of our TM framework with different clustering and outlier detection
algorithms such as Cluster-based Local Outlier Factor (CBLOF), Feature Bagging ( neighbors,
n= 35 ), Histogram-base Outlier Detection (HBOS), Isolation Forest, Average KNN, K-Means clus-


key: chroma:document
string_value:
tering, and One-class SVM. The evaluation was performed on the same preprocessed datasets for a fair
comparison. To make comparision more robust, we preprocessed the data for the baseline algorithms
using count vectorizer, term frequency-inverse document frequency (TFIDF) and Principle component
analysis (PCA). Additionally, we utilized maximum possible outlier fraction (i.e., 0.5) for these methods.


key: chroma:document
string_value:
The performance comparison is given in Table 3, which shows that our framework surpasses the other al-
gorithms on three of the datasets and performs competitively in the remaining two. However, in datasets
likeWOS , where there are many similar words shared between known and novel classes, our method is
surprisingly surpassed by the distance-based algorithm. One-class SVM closely follows the performance


key: chroma:document
string_value:
of our TM framework, which may be due to its linear structure that prevents overﬁtting on imbalanced
and small datasets. In our paper, we are not just trying to build a state-of-art novelty detection algorithm.
5 Conclusions
In this paper, we studied a problem of novelty detection in multiclass text classiﬁcation. We proposed a
score-based TM framework for novel class detection. We ﬁrst used the clauses of the TM to produce a


key: chroma:document
string_value:
novelty score, distinguishing between known and novel classes. Then, a machine learning classiﬁer is
adopted for novelty classiﬁcation using the novelty scores provided by the TM. The experimental results
on various datasets demonstrate the effectiveness of our proposed framework. Our future work includes
using a large text corpus with multiple classes for experimentation and studying the properties of the
novelty score theoretically.
References


key: chroma:document
string_value:
novelty score theoretically.
References
Sugato Basu, Mikhail Bilenko, and Raymond J. Mooney. 2004. A probabilistic framework for semi-supervised
clustering. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and
Data Mining , KDD ’04, page 59–68, New York, NY , USA. Association for Computing Machinery.
Abhijit Bendale and Terrance E. Boult. 2016. Towards open set deep networks. In The IEEE Conference on
Computer Vision and Pattern Recognition (CVPR) , June.


key: chroma:document
string_value:
Geir Thore Berge, Ole-Christoffer Granmo, Tor Oddbjørn Tveit, Morten Goodwin, Lei Jiao, and Bernt Viggo
Matheussen. 2018. Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text
Categorization with Medical Applications. CoRR , abs/1809.04547.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM Comput. Surv. ,
41(3), July.


key: chroma:document
string_value:
41(3), July.
C. K. Chow. 1970. On optimum recognition error and reject tradeoff. IEEE Trans. Information Theory , 16:41–46.
K. Darshana Abeyrathna, Ole-Christoffer Granmo, Xuan Zhang, Lei Jiao, and Morten Goodwin. 2020. The
regression Tsetlin machine: a novel approach to interpretable nonlinear regression. Philosophical Transactions
of the Royal Society A: Mathematical, Physical and Engineering Sciences , 378(2164):20190165.


key: chroma:document
string_value:
Geli Fei and Bing Liu. 2015. Social media text classiﬁcation under negative covariate shift. In Proceedings of the
2015 Conference on Empirical Methods in Natural Language Processing , pages 2347–2356, Lisbon, Portugal,
September. Association for Computational Linguistics.
Ole-Christoffer Granmo. 2018. The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal
Pattern Recognition with Propositional Logic. CoRR , abs/1804.01508.


key: chroma:document
string_value:
V . Hautamaki, I. Karkkainen, and P. Franti. 2004. Outlier detection using k-nearest neighbour graph. In Proceed-
ings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. , volume 3, pages 430–433
V ol.3.
Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassiﬁed and out-of-distribution examples
in neural networks. CoRR , abs/1610.02136.
Kamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, and


key: chroma:document
string_value:
Laura E. Barnes. 2017. Hdltex: Hierarchical deep learning for text classiﬁcation. CoRR , abs/1709.08267.
Marco A. F. Pimentel, David A. Clifton, Lei Clifton, and Lionel Tarassenko. 2014. Review: A review of novelty
detection. Signal Process. , 99:215–249, June.
R. Pincus. 1995. Barnett, v., and lewis t.: Outliers in statistical data. 3rd edition. j. wiley & sons 1994, xvii. 582
pp., £49.95. Biometrical Journal , 37(2):256–256.


key: chroma:document
string_value:
W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult. 2013. Toward open set recognition. IEEE
Transactions on Pattern Analysis and Machine Intelligence , 35(7):1757–1772, July.
Bernhard Sch ¨olkopf, John Platt, John Shawe-Taylor, Alexander Smola, and Robert Williamson. 2001. Estimating
support of a high-dimensional distribution. Neural Computation , 13:1443–1471, 07.
M. L. Tsetlin. 1961. On the behavior of ﬁnite automata in random media. Avtomatika i Telemekhanika , 22:1345–
1354.


key: chroma:document
string_value:
1354.
Jyothsna Veeramreddy, V . Prasad, and Koneti Prasad. 2011. A review of anomaly based intrusion detection
systems. International Journal of Computer Applications , 28:26–35, 08.
Yang Yu, Wei-Yang Qu, Nan Li, and Zimin Guo. 2017. Open-category classiﬁcation by adversarial sample
generation. CoRR , abs/1705.08722.
Y . Zhang, N. Meratnia, and P. Havinga. 2010. Outlier detection techniques for wireless sensor networks: A survey.
IEEE Communications Surveys Tutorials , 12(2):159–170, Second.


key: chroma:document
string_value:
Xuan Zhang, Lei Jiao, Ole-Christoffer Granmo, and Morten Goodwin. 2020. On the convergence of tsetlin
machines for the identity-and not operators. arXiv preprint arXiv:2007.14268 .


key: chroma:document
string_value:
Self-timed Reinforcement Learning
using Tsetlin Machine
Adrian Wheeldon, Alex Yakovlev and Rishad Shaﬁk
Microsystems Group, Newcastle University, UK
Email: {a.r.wheeldon2, alex.yakovlev, rishad.shaﬁk}@ncl.ac.uk
Abstract —We present a hardware design for the learning
datapath of the Tsetlin machine algorithm, along with a latency
analysis of the inference datapath. In order to generate a
low energy hardware which is suitable for pervasive artiﬁcial


key: chroma:document
string_value:
powered internet of things devices, where energy is limited and
latency is an important ﬁgure of merit. Challenges of static timing
analysis in asynchronous circuits are also addressed.
I. I NTRODUCTION
We present a comprehensive design and analysis for an
asynchronous learning datapath based on the Tsetlin machine
(TM) algorithm. When coupled with asynchronous inference
hardware [1], a complete asynchronous TMcapable of online
learning is formed. We also analyze the latency of the infer-


key: chroma:document
string_value:
ence datapath, showing its evolution during training.
The TM[2] algorithm is effective in many large classiﬁca-
tion problems [3, 4, 5]. In addition, the TM’s reinforcement
learning and logic-based inference make it a good candidate
for energy efﬁcient hardware [6]. We design an energy-frugal
TMhardware with a view of use cases in pervasive artiﬁcial
intelligence; e.g. in personal healthcare, accessibility, environ-
mental monitoring and predictive maintenance. We give a brief


key: chroma:document
string_value:
introduction to the TMalgorithm in Section II.
We wish to implement the TMusing asynchronous circuits
as they can be beneﬁcial for low energy sensing systems [7],
when tightly coupled with analog blocks [8], and when power
delivery is unstable or unpredictable [9]. These features are
often present in the aforementioned applications [10].
The architecture of the hardware is designed using a hy-
brid Petri net model in Section III. The model incorporates
inference and learning components.


key: chroma:document
string_value:
inference and learning components.
TMs use groups of reinforcement automata, called Tsetlin
automata ( TAs), to create an ensemble learning effect. We
decompose the automata reinforcement feedback into three
stages to aid the hardware design in Section IV.
The reinforcement process of the TMinvolves random
choice with deﬁned probabilities—some ﬁxed and some vary-
ing at runtime. These probabilistic choices enable diversity
of learning in the TAs. We design a quasi delay insensitive


key: chroma:document
string_value:
(QDI) pseudorandom bit generator ( PRBG ) for this use in the
Clauses Automaton Teams
TA TA
Majority VotingClause Resultsexclude
Class ConﬁdenceFeedback
ClassiﬁcationInput Feature Vector ( f) PRBG
LearningFigure 1. Overview of the Tsetlin machine ( TM) architecture.
TMin Section V, and motivate its use in other low power
applications.
Considering the implementation of the state-holding TAs, we
compare QDIand bundled-data ( BD) design styles suitable for


key: chroma:document
string_value:
the low energy applications in Section VI. The comparison
will show the limitations of the two asynchronous design
methods in terms of area, energy and scalability. We use the
WORKCRAFT framework [11] for speciﬁcation and synthesis
of the QDI implementation using signal transition graphs
(STGs), as well as veriﬁcation of the STGproperties required for
a functional and robust design. For BD, we are inspired by the
desynchronization workﬂow of Cortadella et al. [12] in using


key: chroma:document
string_value:
a commercial synthesis tool designed for synchronous design.
BDcircuits are readily-implementable with industrial-quality
tools, in contrast with QDIcircuits.
Main contributions of this work:
•Visualization of an asynchronous TMhardware architec-
ture using Petri nets (Section III).
•Latency analysis of the asynchronous TMinference datap-
ath from [1] and its evolution during training (Section III).
•A detailed decomposition of TAreinforcement feedback
into three stages (Section IV).


key: chroma:document
string_value:
into three stages (Section IV).
•Design of a QDI PRBG for probabilistic choice in the TM
and other low power applications (Section V).
•Comparison of QDIand BDimplementations of the TAfor
low energy hardware (Section VI).
II. T SETLIN MACHINES IN BRIEF
TMslearn patterns in binary data using propositional logic.
The main inference component of the TMis the clause that
composes an AND expression of the input features and theirarXiv:2109.00846v1  [eess.SP]  2 Sep 2021


key: chroma:document
string_value:
1 2 n · · · n+ 1 n+ 2 2n · · ·Action 1 Exclude Action 2 Include
Penalty RewardFigure 2. Tsetlin automaton state graph.
complements. The TMcomprises many clauses, each produc-
ing a vote. The composition of each clause is controlled by a
vector of exclude bits (see Figure 1). These bits are parameters
that are learned by teams of TAs.
Each clause can produce a vote for its class. The algorithm
states that half of the clauses vote positively, while the other


key: chroma:document
string_value:
half of the clauses vote negatively (we will denote this by
the boolean c neg). The inclusion of inhibition in the voting
system enables non-linear decision boundaries in the infer-
ence process. A majority vote gives an indication of class
conﬁdence. This conﬁdence is used to classify the input data
and inﬂuence future decisions of the automata through the
feedback mechanism [2]. In this work we consider only the
single class TMfor simplicity.


key: chroma:document
string_value:
single class TMfor simplicity.
The TAis a class of ﬁnite reinforcement automaton [13].
It produces an exclude output for states below the midpoint,
and include for states above the midpoint as illustrated in
Figure 2. The TAreceives a penalty or reward from the
feedback mechanism based on the current state of the TM.
Continued rewards in the end states cause the TAto saturate.
A penalty in one of the midstates ( norn+ 1) causes the
TAto transition across the decision boundary—inverting its


key: chroma:document
string_value:
output from exclude to include, or vice versa. The feedback
mechanism is described in more detail in Section IV.
III. A SYNCHRONOUS TSETLIN MACHINE
Figure 3 is an architectural diagram for the asynchronous
implementation of the TMshown in Figure 1, demonstrating
the scalability of the design. The diagram can be composed
with the Petri-net-like tiles in Figure 4 to form a visualization
of the complete system. In these diagrams, rectangles represent


key: chroma:document
string_value:
side, or feature rows below f3.
The clause tile is detailed in Figure 4a and comprises a
partial clause ( PC) computation for the feature input and its
complement (as introduced in Section II); a partial feedback
calculation for stage three feedback ( FB3); and two TAs. The
feedback stages will be discussed later in Section V.
The clause sum is computed in the sum tile at the bottom
of each clause column. Figure 4b details the sum tile. Stage
Clause
TileClause
TileClause
Tile
Clause
TileClause


key: chroma:document
string_value:
TileClause
TileClause
Tile
Clause
TileClause
TileClause
Tile
Clause
TileClause
TileClause
Tile
Sum
TileSum
TileSum
Tilef1
f2
f3
FB1calcyexp
learnClassiﬁcationConcurrency OpportunityFigure 3. Architectural diagram for a Tsetlin machine with three feature
inputs (rows) and three clauses (columns).
(a) Clause tile.
 (b) Sum tile.
Figure 4. Petri-net-like visualizations for the Tsetlin machine system: (a) the
clause tile including partial clause ( PC) and two Tsetlin automata ( TAs); and


key: chroma:document
string_value:
(b) the sum tile.
two feedback ( FB2) for the clause is computed at this stage
using the clause sum.
The diagrams capture the potential for both concurrency and
serialization in the design. As presented, the system operates
in a serialized manner where the feature inputs are passed
through each clause in series. By contracting the places in
the highlighted regions of Figure 4a, tokens are passed to the
right without waiting for PCcalc and FB3calc, this allowing
clauses to compute in parallel.


key: chroma:document
string_value:
clauses to compute in parallel.
The majority of the system is implemented in a dual-
rail ( DR) style using a reduced completion detection ( RCD)
scheme [7]. However, the TAsare implemented in a single-rail
(SR) style (as indicated in Figure 4a) which will be motivated
and discussed in Section VI.
The RCD scheme is a simpliﬁed scheme which completion
detects only the outputs of a block. This brings beneﬁts of
early propagation and drastically reduced area and power


key: chroma:document
string_value:
overheads versus traditional full completion detection. The
trade-off is an added timing assumption on the return-to-idle
phase of the DRsignals. Full details of the RCD scheme are


key: chroma:document
string_value:
fpep
0fnep
1
pcnfnen
0fpen
1
pcp(a) Partial clause
(PC).
cpcn. . . . . .
(b) Partial clause com-
biner.
HA0 HA1 HA2 HA3
HA4 HA5 HA6 HA7
HA8 FA0 FA1a0a1 a2a3a4a5 a6a7
s cs c
scs
c
spinvc
y0 y1 y2spinv
y3(c) Population count with implicit
dual-rail wires and blocks.
class 0class 1
(d) Magnitude comparator.
Figure 5. Dual-rail circuits for inference. Spinv = spacer inverter.
available in [7].
A. Inference Circuits
The inference datapath comprises the clauses, majority


key: chroma:document
string_value:
voting and classiﬁcation introduced in Figure 1. Clauses are
separated into partial clause calculations as discussed earlier
in this section and can be implemented by the schematics in
Figures 5a and 5b. Feature inputs occur in DRencoding along
with exclude signals e0ande1. The outputs of the partial clause
are combined using a DR AND tree.
Majority voting calculated the sum of the clause votes ( csum)
and is implemented using DRpopulation count. The schematic


key: chroma:document
string_value:
in Figure 5c shows the implementation based on DRhalf-
and full-adders. The ORgates and wires in this circuit are
also implicitly DRencoded. Two spacer inverters (spinv) are
required to ensure uniform spacer polarity at the outputs.
For this single class TMexample, the output is classiﬁed
using a threshold function in the form of a magnitude com-
parator. The magnitude comparator lends itself to low energy
implementation in DRand saves energy by evaluating single-


key: chroma:document
string_value:
bit comparisons from most signiﬁcant bit to least signiﬁcant
bit only if needed, all within the period of one cycle [1].
To expand the example to two classes or more, an argmax
function would take the place of thresholding.In the following sections we will explore the inference
circuit delays and analyze the inference latency of the asyn-
chronous datapath with differing input operands. For synthesis
we use a commercial, off-the-shelf technology library: UMC


key: chroma:document
string_value:
(e= 0), all feature inputs are logic-1 ( f= 1); therefore the
delay is bounded by the slowest input and the delay arc is
through the positive rail. In the best-case delay, all but one
literal is excluded and the literal is logic-0; therefore the delay
is bounded by signal propagation through the ORtree in the
negative rail. Since the clause is a large AND tree, there is a
high probability of c= 0, so the delay distribution is mostly
clustered at the lower end as Figure 6 shows. A much smaller


key: chroma:document
string_value:
portion is seen towards the circuit’s maximum delay and is
associated with c= 1.
Population Count: The delay of the circuit is bounded
by the carry chain formed by HA8,FA0and FA1. In the best
case there are no carries and the circuit latency is governed by
the longest logic path which comprises the ﬁrst two layers of
HAs (which have roughly equal paths), through the ORgates
and ﬁnally FA0to the y1output. In the worst case, output y3
must wait for the aforementioned best-case path as well as


key: chroma:document
string_value:
the carry through FA1. This leads to the log-normal-like delay
distribution seen in Figure 6.
Magnitude Comparator: For uniform random inputs and
a normalized worst-case delay of 1, the comparator achieves
a mean delay of 0.05. This is attributed to early propagation
and the evaluation of single-bit comparisons from MSB toLSB.
The delay of the circuit increases as the absolute difference
between operands decreases, because more of the MSBs are


key: chroma:document
string_value:
equal. This circuit is therefore fastest during inference and
the later stages of training. The operand space halves for each
single-bit comparison leading to the negative exponential delay
distribution in Figure 6.
C. Operand-dependent Inference Latency
We investigate the evolution of inference latency during
training by plotting the spacer →codeword latency at different
stages. We use a binarized version of the I RISFlower dataset1


key: chroma:document
string_value:
to provide feature inputs ( f). Exclude inputs ( e) are retrieved
from a TMpre-trained on the same dataset after the speciﬁed
number of epochs. Figure 7 shows the distributions for the
untrained TM, and the trained TMafter 4 epochs and 50 epochs.
The plots show the trend of the mean towards greater circuit
delay as the TMis trained and therefore more clauses become
1Available: https://github.com/cair/TsetlinMachine


key: chroma:document
string_value:
Meanc=0 c=1
00.20.4Clause
Mean00.20.4Population Count
Mean
Decreasing Operand Delta
00.20.4Magnitude Comparator
Mean
0 0.2 0.4 0.6 0.8 100.20.4
Normalized DelayInference DatapathProbabilityFigure 6. Latency distributions of the Tsetlin machine inference datapath and
its components for uniformly-distributed random inputs. Post-synthesis results
using the UMC 65 nm LOW-LEAKAGE library.
Mean
00.20.4Untrained
Mean
00.20.4Trained 4 epochs ( 81 % Accuracy)
Mean
140 160 180 200 22000.20.4


key: chroma:document
string_value:
Mean
140 160 180 200 22000.20.4
Latency ( ps)Trained 50 epochs ( 96 % Accuracy)Probability
Figure 7. Latency distribution of the Tsetlin machine inference datapath for
the I RISdataset.
activated during the computation of each datapoint ( c= 1).
In this case we see more examples from the right side of
the clause delay distribution in Figure 6. This is the main
contributor to the increase in mean latency as the system
becomes more trained. The magnitude comparator offers a


key: chroma:document
string_value:
netdecrease in mean latency as training progresses since the
difference between input operands will increase, meaning we
shift towards the left of the distribution in Figure 7. However
this shift is somewhat smaller than that of the clauses, and
it therefore has a lesser effect on the inference datapath. The
distribution of the population count is much more uniform
than that of the other circuits, and therefore has a negligible
effect during training.IV. D ECOMPOSITION OF FEEDBACK


key: chroma:document
string_value:
During learning, feedback is given to each TAto inﬂuence
its future action, and therefore control pattern detection in the
TMas a whole. The feedback is based on a carefully-designed
payoff matrix [2, p. 10] and is split into two categories: Type
I and Type II. Type I feedback reinforces good decisions and
penalizes false negative outputs. Type II feedback combats
false positives by penalizing TAsthat exclude when c= 1.
We decompose the feedback logic into three stages based on


key: chroma:document
string_value:
scope: FB1atTMlevel, FB2at clause level, and FB3atTAlevel.
FB1and FB2are assigned types: none, T1orT2; these are based
on the Type I and Type II concepts introduced previously. FB3
assigns an action to a single TA: penalty, reward or inaction;
and is based on the outcomes of the payoff matrix2. Each
feedback stage’s output is input to the next, until ﬁnally the
output of FB3is input to the corresponding TA. The following
sections discuss each feedback stage in detail. Finally we will


key: chroma:document
string_value:
discuss the synthesis of the feedback circuits in the DRstyle
in Section IV-D.
A.FB1:TM-level Feedback
Inputs: learn ,yexp
Output: Feedback type (none, T1orT2)
The ﬁrst stage broadly enables feedback for the entire TMif
learn is asserted. The feedback type is chosen according to
yexp —the expected class as provided by the labeled data—as
follows:
FB1=

none,forlearn = 0
T1, forlearn = 1,yexp = 1
T2, forlearn = 1,yexp = 0
B.FB2: Clause-level Feedback
Inputs: FB1, cneg,pT


key: chroma:document
string_value:
B.FB2: Clause-level Feedback
Inputs: FB1, cneg,pT
Output: Feedback type (none, T1orT2)
The second stage swaps the feedback type from T1toT2,
or vice versa if the associated clause is negated (c neg= 1,
discussed in Section II). Additionally, feedback is stopped
(inaction given) if the number of activated clauses (those
producing a logic-1 at the output) meets the threshold, T. This
is determined by probabilistic choice, pT, which is discussed


key: chroma:document
string_value:
at the end of this section. If there is no feedback from the
previous stage ( FB1=none), then none will be generated
(FB2=none). Parameter c negis determined at design time,
therefore the logic for FB1can be separated and depends only
onFB1and pT. This will be demonstrated in Section IV-D.
Clause-level feedback can be summarized as follows:
FB2=

none,for FB1=none
none,for FB1=T1,pT= 0
none,for FB1=T2,pT= 1
T1, for c neg= 1,FB1=T2,pT= 0
T2, for c neg= 1,FB1=T1,pT= 1


key: chroma:document
string_value:
T2, for c neg= 1,FB1=T1,pT= 1
T1, for c neg= 0,FB1=T1,pT= 1
T2, for c neg= 0,FB1=T2,pT= 0
2We do not consider boosting of true positive feedback, which is an option
in the original algorithm.


key: chroma:document
string_value:
Table I
TRUTH TABLE FOR FB3.×=DON ’T CARE .
FB2 inc c x p s FB3
none× ××× inaction
T1 1 0×0 penalty
T1 1 0×1 inaction
T1 1 1×0 inaction
T1 1 1×1 reward
T1 0 0×0 reward
T1 0 0×1 inactionFB2inc c x p s FB3
T10 1 0 0 inaction
T10 1 0 1 reward
T10 1 1 0 inaction
T10 1 1 1 penalty
T21××× inaction
T20 1 0 × penalty
T20 0×× inaction
The TMalgorithm describes probabilities of generating T1
and T2[2, p. 10]. In this work we refer to them as P(T1)and
P(T2)which are based on csumandTas follows:


key: chroma:document
string_value:
P(T2)which are based on csumandTas follows:
P(T1) =T−clamp (csum)
2T, P(T2) =T+clamp (csum)
2T,
where the clamp function restricts its argument to [−T, T]such
that the probabilities lie in the range [0,1]. As the number
of activated clauses (those producing c= 1) approaches T,
the probability of the TAsin that clause receiving feedback
decreases. In order to generate a randomized choice based on
these probabilities, we could assign booleans as follows:
q1=rand< P(T1),q2=rand< P(T2)


key: chroma:document
string_value:
q1=rand< P(T1),q2=rand< P(T2)
where rand is a real number in the range [0,1]. Since P(T1)
andP(T2)are complementary in the range [0,1], q1and q2
are also complementary in the binary domain and therefore
q2=q1. We therefore introduce a single new signal for FB2,
pT, which takes the place of the preceding probabilities. The
generation of this signal will be discussed in Section V.
C.FB3:TA-level Feedback
Inputs: FB2,inc,c,x,ps
Output: TAaction (inaction, penalty or reward)


key: chroma:document
string_value:
Output: TAaction (inaction, penalty or reward)
The third and ﬁnal feedback stage translates the feedback
type from FB2into a TAaction for the speciﬁed TAbased on:
the TA’s current action, encoded as include in signal inc; the
current clause output, c; the feature of complemented feature
(forf) associated with the TA,x; and a probabilistic choice
component, ps.
The FB3 output is chosen according to Table I, where
ps= 1 denotes that the higher probability (s−1)/soption is


key: chroma:document
string_value:
chosen from the payoff matrix. ps= 0 denotes that the lower
probability 1/sis chosen. The generation of signal pswill be
discussed in Section V.
D. Synthesis of Feedback Circuits
Using one-hot encoding in the feedback circuits main-
tains direct compatibility with the inference datapath and
ensures speed independence, which is important for hazard-
freeness. Signals inc,c,xare already one-hot encoded (dual-
rail) for the inference circuits in Section III, so these en-


key: chroma:document
string_value:
codings are reused here in the learning circuits. Further-
more, we encode FB1as follows, and FB2similarly: FB1=
{FB1T2,FB1T1,FB1none}.FB3describes TAactions and is encoded
as: FB3={FB3r,FB3p,FB3i}, for reward, penalty, inactionrespectively. Note that rail orders do not matter as we refer to
the rails by name. In this way, to indicate T2feedback on FB1,
we would set FB1T2= 1,FB1T1= 0,FB1none= 0.
Using the previously deﬁned logic, we can generate circuits


key: chroma:document
string_value:
for the feedback. The implementation of stage two feedback
depends on the clause polarity—it being either negated (c neg=
1) or non-negated (c neg= 0). Note that the stage one circuit is
instantiated once per TM, the stage two circuit once per clause ,
and the stage three circuit once per TA.
In stage three, the FB2signal is shared throughout the TAs
within the same clause. For this reason, we carefully design the
logic so that FB2signals are injected as close to the outputs as


key: chroma:document
string_value:
possible, making the propagation path the shortest. Therefore
when FB2= 0, the computation on the corresponding FB3rails
will conclude rapidly, and for all TAsin the clause.
V. G ENERATION OF RANDOM BITSpT,ps
We introduced two distinct probabilistic choice mechanisms
in Section IV. In Section IV-B we introduced the signal pT:
a probabilistic boolean chosen at the clause level which may
force FB2=none. This boolean is dependent on Tand csum.


key: chroma:document
string_value:
The probability of pT= 1varies at runtime according to csum.
In Section IV-C we introduced the signal ps, which is a random
boolean generated for each TA. It is required under certain
circumstances to choose between two TAactions. This boolean
is dependent on sand is ﬁxed during runtime.
From these requirements, we must be able to generate
biased random bits: bits where P(1)̸= 0.5(unbiased random
bits have P(1) = 0 .5, such as those generated by a linear


key: chroma:document
string_value:
feedback shift register ( LFSR )). In addition, we must be able
to vary P(1)at runtime for pT. Probabilistic choices are used
in the TMto ensure diversity of learning. A pseudorandom
generator is sufﬁcient to satisfy these needs, and beneﬁts from
higher energy efﬁciency than a true random generator.
Due to the requirement of PRBGs for each TA, and addition-
ally for each clause, their area and energy consumption are of
utmost importance. We chose to implement a PRBG based on


key: chroma:document
string_value:
the principles of irregular sampling of a regular waveform, as
this allows us to minimize the overheads and take advantage
of the asynchronous nature of the rest of the system.
A. Asynchronous Sampling of Clock with Variable Duty Cycle
We take advantage of the asynchronous inference logic by
using asynchronous handshakes to sample a regular clock
waveform. A clock with 50 % duty cycle will generate un-
biased random bits. A clock can be generated using a ring


key: chroma:document
string_value:
oscillator ( RO). To ensure random sampling, the clock and
handshake must be uncorrelated .ROscan be gated by adding a
NAND orNOR gate into the ring. However this technique will
disadvantage us in this case as the ROwill always start up in
the same phase, therefore the clock and sampling signals may
become correlated.
We take advantage of the properties of a purely inverter-
based RO. The ROcan be power gated using header/footer
transistor when entropy generation is not required. Such an RO


key: chroma:document
string_value:
will start up in a non-deterministic phase according to thermal
noise and other effects in the inverters.
To generate biased bits for pTand ps, we need to vary the
duty cycle of the clock. Agustin and Lopez-Vallejo [14] show
how to construct an ROsuch that each tap has a unique duty
cycle. This is achieved by using inverters with asymmetric rise/
fall times. For example, odd inverters have fast rise and slow
fall times, and even inverters vice versa. This can be done via


key: chroma:document
string_value:
drives large numbers of ﬂip-ﬂops. In Section V-B we will
investigate the optimal number of PRBGs required. By taking
this distributed PRBG approach, we can vastly reduce the circuit
area compared with a naïve approach using one LFSR per TA.
Figure 8 shows the circuit used to select the correct duty
cycle clock and sample it with the asynchronous reqinput. The
mutex ensures ackpand acknoutputs are mutually exclusive.
These outputs represent the random output bit with a dual-rail


key: chroma:document
string_value:
encoding. That is for {ackp,ackn}:{0,1}represents logic-0,
and{1,0}logic-1.{0,0}is the spacer ornull state used to
separate valid output values temporally.
When the clkand reqsignals are both low, the output will
be in the spacer state. If a request is made (by asserting req)
during the low period of the clock, the output of the set-
dominant latch is reset, and reqwins the mutex, resulting in
acknasserting. If clkrises now, the output of the latch rises,


key: chroma:document
string_value:
however the mutex is still held by req. A following deassertion
ofreqwill result in clkgaining the mutex, however ackpwill
be masked by the AND gate.
If a request is made during the high period of clk, the output
of the latch will already be high and clkwill have won the
mutex. On the rise of reqthe AND gate will unmask ackp. If
clkfalls now, the output of the latch remains high since the R
input is inactive, and therefore ackpalso remains high.


key: chroma:document
string_value:
The circuit is synthesized for the UMC 65LL cell library
and the results are summarized in Table II. The asynchronous
handshake shows almost 10×saving in area compared to an 8-
bitLFSR . Power and energy are also drastically decreased. The
asynchronous handshake is well suited to energy-conscious,
pervasive applications.
B. Optimal Number of PRBGs
The original TMalgorithm takes a probabilistic choice for
every TAupdate involving s. However Abeyrathna et al. [15]


key: chroma:document
string_value:
show that randomizing every dthTAupdate can save energy
with a minimal drop in test accuracy. With d= 1000 , an
accuracy within 2 %ofd= 1 was maintained for three of the
four datasets tested.
MutexS•
RQclk
dc→
from
RO
reqackpacknFigure 8. Handshake circuit for the asynchronous pseudorandom bit generator
with dual-rail output.
Table II
PRBG SSYNTHESIZED FOR UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .
Implementation Area Cycle Time [ ns] Power [µ W] Energy [ fJ]
LFSR8 92.2 0 .38 6 .41 2 .44


key: chroma:document
string_value:
LFSR8 92.2 0 .38 6 .41 2 .44
Handshake 9.72 0 .23 0 .621 0 .143
Following on, we could hypothesize an optimal number
ofPRBGs to fulﬁll the needs of a given TM. If every 100th
TAupdate is randomized, we will need to produce at most
NTAs/100probabilistic choices per learning phase.
This could result in a 100×reduction in PRBGs for an
LFSR -based design, in exchange for arbitration overhead to
share one PRBG between multiple TAs. Similar could be said


key: chroma:document
string_value:
for the asynchronous handshake design, although the already
small size of the design means that arbitration overhead would
almost certainly result in a larger area than instantiating an
asynchronous handshake PRBG for every TA.
VI. A SYNCHRONOUS TSETLIN AUTOMATON
In a TM, the TAs consume most of the hardware re-
sources [6]. The number of TAs is given by NTAs =
2NClassesNClausesNInputs. Therefore the goal is to create a
lightweight TAimplementation with a focus on area and


key: chroma:document
string_value:
energy. The TAhardware must implement the ﬁnite state
automaton described by the state diagram in Figure 2.
This section explores two methods for the design of a TA.
Firstly a QDIapproach starting from an STGspeciﬁcation; and
secondly a BDapproach starting from a traditional synchronous
speciﬁcation, using matched delays to time the design. The
designs will be compared with a synchronous implementation.
A. Quasi Delay Insensitive Approach


key: chroma:document
string_value:
A. Quasi Delay Insensitive Approach
The STGin Figure 9 represents a one-hot ﬁnite state machine
(FSM) for a two action, six-state TA(for the deﬁnition of STGs,
see [16]). The inputs pand rcome from the FB3rails FB3p
and FB3rwhich were introduced in Section IV-D. The FB3irail
denotes inaction and is only used for completion detection.
State bits are encoded in the x∗∗_0and x∗∗_1places at the
top of the graph. States x1∗(x2∗) are the action 1 (action 2)


key: chroma:document
string_value:
states—in these states the automaton is indicating the action
to exclude (include) the feature or feature complement from
the clause composition. The x∗1(x∗3) states are the closest to
(furthest from) the action decision boundary. As an example,
when x13_1holds a token, the automaton is indicating action
1 (exclude) strongly.
A token at p0denotes the idle state for the automaton. It
offers a choice between penalty and reward represented by


key: chroma:document
string_value:
After state transitions and action output have occurred, the
acknowledge output, ack, makes a positive transition. Finally,
the previously given input and action signals return low,
followed by ack−and the token returns to p0. The automaton
is now ready for the next penalty or reward input.
The internal transitions x∗∗encode some information about
the states in the STG. However, this is not enough to satisfy
complete state coding ( CSC) required for synthesis of the


key: chroma:document
string_value:
STG to logic gates [16]. In the previous example where we
transition from x12tox13, there is a point where tokens are
held by both x13_1and x12_1. The STGreaches this same state
when transitioning in the opposite direction, from x12tox13.
In order to achieve CSC, we introduce internal signals to
encode the direction of travel of the state: x∗L∗,x∗R∗; for left
and right. We insert these signals into every branch to maintain
uniformity in the STG. Although not all these internal signals


key: chroma:document
string_value:
are strictly required, they ease scalability of the STGand help
the synthesis tool to minimize and share logic efﬁciently.
The STG in Figure 9 passes all veriﬁcation tasks required
for synthesis of a correct and hazard-free circuit in the
WORKCRAFT framework: consistency, deadlock freeness, in-
put properness, and output persistency. The circuit was syn-
thesized and mapped to the target technology by the MPS AT
backend (the P ETRIFY backend did not produce a result in a


key: chroma:document
string_value:
reasonable amount of time).
In order to read the state of the TAwithout giving penalty or
reward (during inference for example), we add a latch to the
output to store one of the porrsignals. The latch is controlled
by the ackhandshake.
The STG state space expands exponentially as more signal
transitions are added. This makes it difﬁcult for the synthesis
tool to handle TAswith more than three action states. Addition-
ally there is a high possibility for human error when copying


key: chroma:document
string_value:
and editing branches of the STG. And as the number of TA
action states increases, the graph becomes large and unwieldy
to navigate. For these reasons we explore a BDapproach to
asynchronous TAdesign.
B. Bundled-Data Approach
A synchronous one-hot FSM can be constructed using ﬂip-
ﬂops as storage elements. The next state logic can be derived
using the adjacent states and p,rinputs as follows:
x13=x13·r+x12·r
x12=x11·r+x13·p
x11=x12·p+x21·px21=x22·p+x11·p
x22=x21·r+x23·p
x23=x23·r+x22·r


key: chroma:document
string_value:
x22=x21·r+x23·p
x23=x23·r+x22·r
This synchronous circuit is converted into a self-timed one
through the process of desynchronization [12], by decom-posing the ﬂip-ﬂops into master/slave latches and replacing
the clock with handshake controllers and delays matched to
the combinational logic paths. The timing of the resultant BD
design is not as robust as the QDIapproach in Section VI-A
since the matched delays are ﬁxed at design time and are


key: chroma:document
string_value:
subject to process, variation and temperature variations.
To reduce the overhead of the handshake controllers, we
group the TAsinto an array and use one handshake controller
for many latches. Depending on the overall size of TMfor
the target application, the TAscould be grouped in different
ways. A monolithic array will minimize controller overhead,
however there comes a limit where buffers must be introduced
into the latch enable signals to maintain integrity. At this point,


key: chroma:document
string_value:
it may become desirable to group the TAsby clause.
The speciﬁcation of the BDdesign is easily scaled due to
the use of parameterizable Verilog code. The workﬂow takes
advantage of mature synchronous tools for synthesis.
C. Synthesis Results
Timing and power analyses of asynchronous circuits need
special consideration. In conventional synchronous static tim-
ing analysis, delay through logic gates is calculated using the
input slew and output load capacitance. Power estimations also


key: chroma:document
string_value:
rely on these parameters. Synchronous tools cut combinational
loops arbitrarily, therefore may be optimistic about slews,
gate delays and internal power in asynchronous circuits which
contain such loops.
To obtain results for the QDIdesign we use a specialized
asynchronous static timing analysis tool [17] which takes into
account the combinational loops in the timing calculation.
The tool operates on the gate-level netlist and propagates


key: chroma:document
string_value:
slews iteratively through circuits with loops, therefore avoiding
optimism in the slews and gate delays. In the case there are
multiple paths in the circuit which begin and end at the same
input/output combination, the tool uses the worst-case path to
determine the cycle time.
Since the UMC 65LL library does not contain C-elements,
the QDIcircuit is combinational with feedback loops. Table III
shows a results summary of the TAimplementations. All results


key: chroma:document
string_value:
are synthesis estimations and no layout is performed. In this
case, the QDIcircuit trades increased cycle time and energy
for increased robustness to process, variation and temperature
variations. The BDdesign strikes a good trade-off between
performance, area and elastic timing to integrate with the rest
of the system. The layout process will introduce more power
Table III
RESULTS FOR THE SIX -STATE TSETLIN AUTOMATON SYNTHESIZED FOR
THE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .


key: chroma:document
string_value:
THE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .
Implementation AreaCycle Time
[ns]Leakage
[nW]Power
[µW]Energy
[fJ]
Synchronous 87.8 0 .35 3 .08 4 .58 1 .60
Bundled-data386.0 0 .41 2 .89 5 .19 2 .13
QDI 123 3 .35 3 .42 1 .58 5 .29
4Averaged over the array.


key: chroma:document
string_value:
Decision Boundary Transitions Rewards in End-states
One-hot Encoded States
Action 1
(exclude)
StatesAction 2
(include)
States
Decision BoundaryFigure 9. Signal transition graph for a Tsetlin automaton with two actions and six states. Inputs: p,r. Outputs: a1,a2,ack. Other signals are internal.
and latency into the synchronous and BDdesigns due to their
clock trees, which the QDIdesign will not suffer from.
VII. C ONCLUSION
Our Petri net visualization of the TMarchitecture enables


key: chroma:document
string_value:
ﬂexible hardware implementations by encapsulating both con-
currency and serialization. By decoupling the TAstorage
elements from the clauses, e.g. in a standalone array, the TM
could be serialized further by reusing clause and sum hardware
across multiple cycles, trading off throughput for reduced logic
area (and leakage power).
Early propagation enabled by the RCD scheme used in the DR
circuits can be leveraged to trigger ﬁne-grained power gating


key: chroma:document
string_value:
in the system to further reduce energy consumption.
Distributed generation of probabilistic choices maximizes
concurrency and decreases energy overheads compared to
a centralized approach. A simple PRBG implementation is
sufﬁcient and most suitable for our applications. Features
of a clock source undesirable in synchronous circuits (jitter,
etc.) can be leveraged for stochasticity. Further study of PRBG
properties and their effects on TMlearning are a subject for
future work.
ACKNOWLEDGMENT


key: chroma:document
string_value:
future work.
ACKNOWLEDGMENT
The authors thank Christos Sotiriou and colleagues for help
in asynchronous STAand for access and support for their tool.
We are also grateful to colleague Thomas Bunnam for fruitful
discussions on ROsand RNG techniques. The authors would
like to acknowledge the funding support from EPSRC IAA
grant: Whisperable.
REFERENCES
[1] A. Wheeldon et al. , “Low-Latency Asynchronous Logic Design for
Inference at the Edge,” in Proc. Conf. Des. Autom. Test Eur. , Feb. 2021,


key: chroma:document
string_value:
Eng Sci , vol. 378, no. 2182, Oct. 2020.
[7] ——, “Self-timed, minimum latency circuits for the internet of things,”
Integration , vol. 69, pp. 138–146, Nov. 2019.
[8] D. Sokolov et al. , “Automating the Design of Asynchronous Logic
Control for AMS Electronics,” IEEE Trans. Comput.-Aided Design
Integr. Circuits Syst. , vol. 39, no. 5, pp. 952–965, 2020.
[9] X. Zhang et al. , “A Novel Power Delivery Method for Asynchronous
Loads in Energy Harvesting Systems,” J. Emerg. Technol. Comput. Syst. ,


key: chroma:document
string_value:
Circuits Syst , vol. 25, no. 10, pp. 1904–1921, Oct. 2006.
[13] K. S. Narendra and M. A. L. Thathachar, Learning Automata: An
Introduction . Englewood Cliffs, N.J: Prentice Hall, 1989.
[14] J. Agustin and M. Lopez-Vallejo, “An in-depth analysis of ring oscilla-
tors: Exploiting their conﬁgurable duty-cycle,” IEEE Trans Circuits Syst
Regul Pap , vol. 62, no. 10, pp. 2485–2494, Oct. 2015.
[15] K. D. Abeyrathna et al. , “A Novel Multi-Step Finite-State Automaton


key: chroma:document
string_value:
for Arbitrarily Deterministic Tsetlin Machine Learning,” in Proc. 40th
SGAI Int. Conf. Artif. Intell. , vol. 7, Jul. 2020.
[16] J. Cortadella et al. ,Logic synthesis of asynchronous controllers and
interfaces . Springer-Verlag Berlin Heidelberg, 2002.
[17] S. Simoglou et al. , “Graph-based STA for asynchronous controllers,”
Integration , vol. 75, pp. 91–101, Nov. 2020.


key: chroma:document
string_value:
Tsetlin Machine Embedding: Representing Words Using Logical Expressions
Bimal Bhattarai ,Ole-Christoffer Granmo ,Lei Jiao ,Rohan Yadav andJivitesh Sharma
Centre for AI Research (CAIR), University of Agder, Norway
{bimal.bhattarai, ole.granmo, lei.jiao, rohan.yadav, jivitesh.sharma }@uia.no
Abstract
Embedding words in vector space is a fundamen-
tal ﬁrst step in state-of-the-art natural language pro-
cessing (NLP). Typical NLP solutions employ pre-
deﬁned vector representations to improve gener-


key: chroma:document
string_value:
deﬁned vector representations to improve gener-
alization by co-locating similar words in vector
space. For instance, Word2Vec is a self-supervised
predictive model that captures the context of words
using a neural network. Similarly, GLoVe is a
popular unsupervised model incorporating corpus-
wide word co-occurrence statistics. Such word em-
bedding has signiﬁcantly boosted important NLP
tasks, including sentiment analysis, document clas-
siﬁcation, and machine translation. However, the


key: chroma:document
string_value:
siﬁcation, and machine translation. However, the
embeddings are dense ﬂoating-point vectors, mak-
ing them expensive to compute and difﬁcult to in-
terpret. In this paper, we instead propose to rep-
resent the semantics of words with a few deﬁn-
ing words that are related using propositional logic.
To produce such logical embeddings, we introduce
a Tsetlin Machine-based autoencoder that learns
logical clauses self-supervised. The clauses con-
sist of contextual words like “black,” “cup,” and


key: chroma:document
string_value:
sist of contextual words like “black,” “cup,” and
“hot” to deﬁne other words like “coffee,” thus be-
ing human-understandable. We evaluate our em-
bedding approach on several intrinsic and extrinsic
benchmarks, outperforming GLoVe on six classiﬁ-
cation tasks. Furthermore, we investigate the inter-
pretability of our embedding using the logical rep-
resentations acquired during training. We also vi-
sualize word clusters in vector space, demonstrat-


key: chroma:document
string_value:
ing how our logical embedding co-locate similar
words.1
1 Introduction
The success of natural language processing (NLP) relies on
advances in word, sentence, and document representation. By
capturing word semantics and similarities, such representa-
tions boost the performance of downstream tasks [Borgeaud
1The Tsetlin Machine Autoencoder and logical word embedding
implementation is available here: https://github.com/cair/tmu.et al. , 2022 ], including clustering, topic modelling [Angelov,


key: chroma:document
string_value:
2020 ], searching, and text mining [Huang et al. , 2020 ].
While straightforward, the traditional bag-of-words encod-
ing does not consider the words’ position, semantics, and
context within a document. Distributed word representa-
tion[Bengio et al. , 2000; Bojanowski et al. , 2017 ]addresses
this lack by encoding words as low-dimensional vectors, re-
ferred to as embeddings . The purpose is to co-locate simi-
lar or contextually relevant words in vector space. There are


key: chroma:document
string_value:
many algorithms for learning word embeddings. Contem-
porary self-supervised techniques like Word2Vec [Mikolov
et al. , 2013 ], FastText [Bojanowski et al. , 2017 ], and
GloVe [Pennington et al. , 2014 ]have demonstrated how to
build embeddings from word co-occurrence, utilizing mas-
sive training data. Introducing context-dependent embed-
dings, the more sophisticated language models BERT [De-
vlin et al. , 2019 ]and ELMO [Peters et al. , 2018 ]now per-


key: chroma:document
string_value:
form remarkably well in downstream tasks [Reimers and
Gurevych, 2019 ]. However, they require signiﬁcant compu-
tation power [Schwartz et al. , 2020 ].
The above approaches represent words as dense ﬂoating
point vectors. Word2Vec, for instance, typically builds a 300-
dimensional vector per word. The size and density of these
vectors make them expensive to compute and difﬁcult to in-
terpret. Consider, for example, the word “queen.” Represent-


key: chroma:document
string_value:
ing it with 300 ﬂoats seems inefﬁcient compared to the Ox-
ford Language deﬁnition for the same word: “the female ruler
of an independent state, especially one who inherits the posi-
tion by right of birth.” From this perspective, it appears ad-
vantageous to create embeddings directly from words rather
than from arbitrary ﬂoating-point values. Such interpretable
embeddings would capture the multiple meanings of a word
using a few deﬁning words, simplifying both computation
and interpretation.


key: chroma:document
string_value:
and interpretation.
In this paper, we propose a Tsetlin Machine (TM)
[Granmo, 2018 ]based autoencoder for creating interpretable
embeddings. The autoencoder builds propositional logic ex-
pressions with context words that identify each target word.
The term “coffee” can, for instance, be represented by “one,”
“hot,” “cup,” “table,” and “black.” In this manner, the TM
builds contextual representations from a vast text corpus,
which model the semantics of each word. In contrast to neu-


key: chroma:document
string_value:
ral network-based embedding, the logical TM embedding is
sparse. The embedding space consists of, e.g., 500truth val-arXiv:2301.00709v1  [cs.CL]  2 Jan 2023


key: chroma:document
string_value:
ues, where each truth value is a logical expression over words.
For contextual representation, each target word links to less
than ten percent of these expressions. Despite the sparsity and
crispness of this representation, it is competitive with neural
network-based embedding.
The contributions of our work are summarized below:
• We propose the ﬁrst TM-based Autoencoder to learn ef-
ﬁcient encodings in a self-supervised manner.
• We introduce TM-based word embedding that builds


key: chroma:document
string_value:
human-comprehensible contextual representations from
unlabeled data.
• We compare our embedding with state-of-the-art ap-
proaches on several intrinsic and extrinsic benchmarks,
outperforming GloVe on six downstream classiﬁcation
tasks.
2 Related Work
The majority of self-supervised embedding approaches pro-
duce dense word representations based on the distributional
hypothesis [Harris, 1954 ], which states that words that oc-
cur in the same context are likely to have similar meaning.


key: chroma:document
string_value:
Word2Vec [Mikolov et al. , 2013 ]is one of the best-known
models. It builds embeddings from word co-occurrence using
a neural network, leveraging the hidden layer output weights.
GloVe [Pennington et al. , 2014 ], on the other hand, em-
beds by factorizing a word co-occurrence matrix. Similarly,
canonical correlation analysis (CCA) is used in [Dhillon et
al., 2015 ]for embedding words to maximise context corre-
lation. In [Levy et al. , 2015 ], it is demonstrated how pre-


key: chroma:document
string_value:
cise factorization-based SVD can compete with neural em-
bedding. However, all of these methods are challenging to
train because they involve tweaking algorithms and hyperpa-
rameters toward particular applications [Lample et al. , 2016 ],
limiting their wider applicability.
Building upon word embedding, several studies focus on
sentence embedding [Arora et al. , 2017; Logeswaran and
Lee, 2018 ]. Recent advances in sentence embedding in-
clude supervised data inference [Reimers and Gurevych,


key: chroma:document
string_value:
2019 ], multitask learning [Ceret al. , 2018 ], contrastive learn-
ing[Zhang et al. , 2020 ], and pretrained large language mod-
els[Liet al. , 2020 ]. However, the majority of sentence
embedding techniques overlook intrinsic evaluations such as
similarity tasks, and instead largely focus on extrinsic eval-
uations involving downstream performance. The most re-
cent building block for embedding originates from the trans-
former approach [Vaswani et al. , 2017 ]. Transformers pro-


key: chroma:document
string_value:
vide context awareness by utilizing stacks of self-attention
layers. BERT [Kenton and Toutanova, 2019 ], for instance,
employs the transformer architecture to carry out extensive
self-supervised training, making it capable of producing text
embedding. Other embedding models use a contrastive loss
function to perform supervised ﬁne-tuning on positive and
negative text pairs [Wang et al. , 2021 ]. Despite the large
variety of text embedding models, they all share three main


key: chroma:document
string_value:
drawbacks: i) they are computationally demanding to train;
ii) they are intrinsically complex because they are trained on
a large amount of data to tune a huge amount of parameters;
Clause Pool+4 -5Input Output
+1 +2 -7 +6Figure 1: Tsetlin Machine Autoencoder. In this illustration, x1is
masked by replacing it with value 1 for inferring ˆx1.
and iii) the embeddings produced from these models are not
easily interpreted by humans.
To improve interpretability, Faruqui et al. introduced


key: chroma:document
string_value:
“Sparse Overcomplete Word Vectors” (SPOWV) which cre-
ate a sparse non-negative projection of word embedding us-
ing dictionary learning [Faruqui et al. , 2015 ]. Similarly,
SParse Interpretable Neural Embeddings (SPINE) employs
a k-sparse denoising autoencoder to generate sparse embed-
dings [Subramanian et al. , 2018 ]. However, these methods
are unable to distinguish between multiple context-dependent
word meanings. To address this problem, another avenue of


key: chroma:document
string_value:
research focuses on composing linear combinations of dense
vectors from Word2Vec and GloVe [Arora et al. , 2018 ]. How-
ever, the assumption of linearity does not hold for real-world
data, yielding linear coefﬁcients that are difﬁcult to compre-
hend [Muet al. , 2017 ].
The logical embedding approach we present here is most
closely related to Naive Bayes word sense induction and topic
modeling [Charniak and others, 2013; Lau et al. , 2014 ]. This


key: chroma:document
string_value:
approach learns word meanings from local contexts by con-
sidering each instance of the word in a document as a pseudo-
document. However, the approach is not scalable because
it requires training a single topic per target word. Our ap-
proach, on the other hand, is scalable and builds non-linear
(non-naive) logical embeddings that capture word composi-
tions. To build the logical embeddings, we propose a novel
human-interpretable algorithm based on the TM that provides


key: chroma:document
string_value:
logical rules describing contexts. The TM has recently per-
formed competitively with other deep learning techniques in
many NLP tasks, including novelty detection [Bhattarai et
al., 2022a ], sentiment analysis [Yadav et al. , 2021 ], and fake
news detection [Bhattarai et al. , 2022b ]. Furthermore, the
local and global interpretability of TMs have been explored
through direct manipulation of the logical rules [Blakely and
Granmo, 2021 ].
3 Tsetlin Machine Autoencoder


key: chroma:document
string_value:
Granmo, 2021 ].
3 Tsetlin Machine Autoencoder
We here detail the TM Autoencoder based on the Coalesced
TM [Glimsdal and Granmo, 2021 ], extended with input
masking and freezing of masked variables. For ease of ex-
planation, we use three inputs. Adding more inputs follows
trivially.


key: chroma:document
string_value:
3.1 Architecture
Input and Output. As seen in Figure 1, the TM Autoen-
coder digests and outputs propositional values: (x1,x2,x3)∈
{0,1}3→(ˆx1,ˆx2,ˆx3)∈ {0,1}3. For our purposes, the
propositional variables x1,x2, andx3each represents a word,
for example, “Brilliant,” “Actor,” and “Awful.” The value 1
means that the word occurs in the input text, while the value
0means that it does not. I.e., we represent natural language
text as a set of words . Notice also that the input variables have


key: chroma:document
string_value:
corresponding output variables ˆx1,ˆx2, andˆx3. In short,ˆx1
is to be predicted from x2andx3,ˆx2fromx1andx3, and
so on. Continuing our example, ˆx1predicts the presence of
“Brilliant” based on knowing the occurrence of “Actor” and
“Awful.”
Clause Pool. A pool ofnconjunctive clauses, denoted
Cj,j∈ {1,2,...,n},encodes the input in order to pre-
dict the output. A conjunctive clause Cjis simply an And-
expression over a given subset Lj⊆{x1,x2,x3}of the input


key: chroma:document
string_value:
(our autoencoder does not use the input negations ¬x1,¬x2,
and¬x3):
Cj(x1,x2,x3) =⋀
xk∈Ljxk. (1)
For example, the input subset L1={x1,x2}gives the clause
C1(x1,x2,x3) =x1∧x2in the ﬁgure. This clause matches
the input ifx1andx2both are 1. In our example, the clause
accordingly encodes the concept “Brilliant Actor”.
Weights. An integer weight matrix Wconnects each of the
nclauses to the three outputs ˆx1,ˆx2, andˆx3:
W=[w11···w1n
w21···w2n
w31···w3n]
∈Z3×n. (2)


key: chroma:document
string_value:
W=[w11···w1n
w21···w2n
w31···w3n]
∈Z3×n. (2)
The row index is an output while the column index is a clause.
The weight w12, for instance, connects output ˆx1to clause
C2. In Figure 1, six weights connect two clauses and three
outputs:[+4−5
+1 +2
−7 +6]
. (3)
Consider, for example, the weights (+4,−5)of outputˆx1in
the ﬁgure. The weight +4states that clause C1(x1,x2,x3) =
x1∧x2favoursˆx1being 1, while clause C2(x1,x2,x3) =
x2∧x3opposes it. For example, the concept “Awful Actor”


key: chroma:document
string_value:
opposes output “Brilliant.”
3.2 Inference
Let us consider the prediction of ˆx1ﬁrst. The autoencoder
predictsˆx1from the clauses and weights:
ˆx1= 0≤n∑
j=1wj1Cj(1,x2,x3). (4)
That is, each clause Cjis multiplied by its weight wj1for
outputˆx1. The outcomes are then summed up to decide the
output. If the sum is larger than or equal to zero, the output
isˆx1= 1. Otherwise, it is ˆx1= 0. Clauses with positive
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten 4 
 
3 
 
2


key: chroma:document
string_value:
ForgottenMemorized
Forgotten 4 
 
3 
 
2 
 
1Figure 2: Tsetlin Machine memory for single clause.
weight thus promote output ˆx1= 1 while clauses with neg-
ative weight encourage ˆx1= 0. Notice that x1is masked by
replacing it with value 1. Accordingly, the autoencoder infers
outputˆx1from the remaining inputs x2andx3.
Correspondingly, ˆx2andˆx3are calculated by respectively
maskingx2andx3:
ˆx2= 0≤n∑
j=1wj2Cj(x1,1,x3), (5)
ˆx3= 0≤n∑
j=1wj3Cj(x1,x2,1). (6)


key: chroma:document
string_value:
ˆx3= 0≤n∑
j=1wj3Cj(x1,x2,1). (6)
Example. Assume that the input is always either (1,1,0)
or(0,1,1). The input (1,1,0)could for instance represent
“Brilliant Actor” and (0,1,1)“Awful Actor.” Then notice
how Eq. (4) correctly determines the masked input x1with
outputˆx1in Figure 1, both for input (1,1,0)and(0,1,1).
3.3 Learning
We next consider how to learn the variable subsets Ljfor the
clausesCj,j∈{1,2,...,n},as well as how to determine
the weightswjiof the weight matrix W.


key: chroma:document
string_value:
the weightswjiof the weight matrix W.
Clause Memory. Each clause Cjhas a graded memory that
contains the input variables, shown in Figure 2. The graded
memory enables incremental learning of the variable subsets
from data. Observe how each variable is in one of four mem-
ory positions (the number of memory positions is a user-
conﬁgurable parameter). Positions 1−2means Forgotten.
Positions 3−4means Memorized. Memorized variables take
part in the clause, while Forgotten ones do not. The memory


key: chroma:document
string_value:
in Figure 2 thus gives the clause Cj(x1,x2,x3) =x1∧x2.
Learning Step. The TM Autoencoder learns incrementally
using three kinds of memory and weight updates: Type Ia,
Type Ib, and Type II. Each training example has the form
[k,(x1,x2,x3),xk],1≤k≤3. The ﬁrst element is an index
that identiﬁes which input to mask and which output to pre-
dict. The second element is an input vector (x1,x2,x3)and
the third element is the target value for output ˆxk, which is


key: chroma:document
string_value:
xk. We describe the update procedure step-by-step below for
index 1examples (output ˆx1prediction). The update proce-
dure forˆx2andˆx3follows trivially.
Clause Update Probability. First, we calculate the
weighted clause sum for ˆx1from Eqn. (4): v1=∑n
j=1wj1Cj(1,x2,x3). The sum is then compared with a
marginT(hyper-parameter) to calculate a summation error ϵ.
The error depends on the x1-value:
ϵ={T−clip(v1,−T,T), x 1= 1,
T+ clip(v1,−T,T), x 1= 0.(7)


key: chroma:document
string_value:
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1Figure 3: Type Ia (Recognize) Feedback for input (1,1,0). The
masked variable x1is frozen.
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1
Figure 4: Type Ib (Erase) Feedback for input (0,0,1). The masked
variable x1is frozen.
That is, for x1-value 1the weighted clause sum should be-
comeT, while forx1-value 0the sum should become −T.


key: chroma:document
string_value:
The goal of the learning is thus to reach the margin for all
inputs (x1,x2,x3), ensuring correct output from Eqn. (4).
To reach this goal, each clause Cjis updated randomly with
probabilityϵ
2Tin each round. In other words, the update
probability drops with the error towards zero.
Update Types. The kind of update depends on the values
ofx1,Cj(1,x2,x3), andwj1. We ﬁrst consider clauses with
positive weight, wj1≥0. According to Eqn. 4, they are to


key: chroma:document
string_value:
recognize patterns for x1= 1. Note that in all of the below
updates, the masked variable x1is frozen, leaving it unaf-
fected by the update.
•Type Ia (Recognize) Feedback occurs when x1=
1andCj(1,x2,x3) = 1 . Then one can say that
Cj(1,x2,x3) = 1 is atrue positive because it correctly
predicts the masked x1-value. The Type Ia feedback re-
inforces this successful match by updating the memory
ofCjto further mimic the input (see Figure 3). That is,


key: chroma:document
string_value:
1-valued variables move one step upwards in memory,
with probability 1.0.2Conversely, 0-valued inputs move
one step downwards, however, randomly with probabil-
ity1
s. Here,sis a hyperparameter called speciﬁcity ,
meaning that a larger smakes the clauses more speciﬁc.
The clause overall is also reinforced by incrementing its
weightwj1by1.
•Type Ib (Erase) Feedback occurs when x1= 1 and
Cj(1,x2,x3) = 0 . Then we call Cj(1,x2,x3) = 0
afalse negative because it fails to promote x1= 1 .


key: chroma:document
string_value:
In that case, all inputs randomly move one step down-
wards in memory (see Figure 4). Again, each downward
move happens with probability1
s. Here, the purpose is
to eliminate the false negative outcome by erasing vari-
ables from the clause.
2Originally, the increment probability iss−1
s, which can be
boosted to 1.0to enhance learning of true positive patterns [Granmo,
2018 ].
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2


key: chroma:document
string_value:
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1Figure 5: Type II (Reject) Feedback for input (0,1,0). The masked
variable x1is frozen.
•Type II (Reject) Feedback occurs when x1= 0 and
Cj(1,x2,x3) = 1 . Then, one can say that Cj(1,x2) = 1
is afalse positive because it promotes x1= 1 when in
fact we have x1= 0. Then all Forgotten 0-valued in-
puts move one step upwards in memory. The purpose
is to eventually eliminate the current false positive out-


key: chroma:document
string_value:
come by injecting 0-valued variables into the clause.
The clause is further diminished by decrementing its
weightwj1by1. Note that the latter decrement can
switch the weight from positive to negative. In effect,
the clause then changes role, now training to recognize
x1= 0instead.
ClausesCjwith negative weights, wj1<0, are updated
the same way. However, they are to recognize patterns for
x1= 0. To achieve this, x1= 0 is treated as x1= 1 and
x1= 1 is treated as x1= 0 when updating the memories.


key: chroma:document
string_value:
Furthermore, the weight updates are reversed. Increments be-
comes decrements, and vice versa.
Algorithm 1 TM word embedding
Require: V ocabularyV; DocumentsD∈G ,D⊆V ; Accumula-
tionu; Clauses n; Margin T; Speciﬁcity s; Rounds r
1: TMCreate (n, T, s ) ⊿Create TM with nclauses.
2:forrrounds do
3: forword k∈Vdo ⊿Create one example per word.
4: qk←Select({0,1}) ⊿Random target value.
5: ifqk= 1then
6:Gk←{D| word k∈D,D∈G} ⊿Documents with
word k.
7: else
8:Gk←{D| word k/∈D,D∈G} ⊿Documents


key: chroma:document
string_value:
7: else
8:Gk←{D| word k/∈D,D∈G} ⊿Documents
without word k.
9:Sk←SelectN(Gk, u) ⊿Random subset of size u.
10:Uk←⋃
D∈S kD ⊿Union of selected documents.
11: xk←(x1, x2, . . . , x m), xi={
1,word i∈Uk
0,word i/∈Uk
12: TMUpdate( k,xk, qk)⊿Update TM Autoencoder for
output index k, input xk, and target value ˆxk=qk.
13:C,W←TMGetState() ⊿Clauses Cj∈Cwith weights W.
14:E←clip(W,0, T) ⊿Elementwise clip of negative values
produces weighted logical word embeddings.


key: chroma:document
string_value:
produces weighted logical word embeddings.
15:B←(W>0) ⊿Elementwise comparison with zero
produces purely logical word embeddings.
16:returnC,E,B


key: chroma:document
string_value:
4 Logical Embedding Procedure
We now use the TM Autoencoder to build logical embed-
dings. LetV={word 1,word 2,..., wordm}be the target
vocabulary consisting of munique words.
Pre-processing. The ﬁrst step is to pre-process the docu-
ment corpus. To this end, each document is represented by a
subset of wordsD⊆V . For example, the document “The ac-
tor was brilliant” becomes the set D={“actor”, “brilliant”,
“the”, “was”}. The setG, in turn, contains all the documents,


key: chroma:document
string_value:
D∈G . Finally, in propositional vector form, the word set D
becomes:
x= (x1,x2,...,xt),xi={1,wordi∈D,
0,wordi/∈D.(8)
Embedding. Algorithm 1 speciﬁes the procedure for em-
bedding the mvocabulary words from Vby usingnclauses,
Cj,1≤j≤n, forming a clause set C. Each round of
training produces a training example [k,(x1,x2,...,xm),qk]
perwordkinV. First, a target value qkfor the word is set
randomly to either 0or1. This random selection balances
the dataset. If qkbecomes 1, we randomly select udoc-


key: chroma:document
string_value:
uments that contain wordkand assign them to the set Sk
(positive examples). Otherwise, we randomly select udoc-
uments that does notcontain the word (negative examples).
Next, the randomly selected documents are merged by OR-
ing them together, yielding the uniﬁed document Uk. The
purpose of ORing multiple documents is to increase the fre-
quency of rare context words. Then, picking up characteristic
ones becomes easier. After that, the propositional vector form


key: chroma:document
string_value:
(x1,x2,...,xm)ofUkis obtained. Finally, the TM Autoen-
coder is updated with [k,(x1,x2,...,xm),qk]following the
training procedure in Section 3.
Vector Space Representation. The weighted logical em-
bedding of wordk∈V can now be obtained from row kof
matrix E(returned from Algorithm 1), while the the purely
logical embedding is found in row kof matrix B. Letekde-
note thek’th row of E, and let eldenote thel’th row. We can
then compare the similarity of two words wordkand wordl


key: chroma:document
string_value:
using cosine similarity (CS) between their E-embedding:
CS(wordk,wordl) =ek·el
||ek||||el||. (9)
5 Empirical Evaluation
We here evaluate our logical embedding scheme, comparing
it with neural network approaches.
5.1 Datasets and Setup
We ﬁrst evaluate our logical embedding intrinsically, fol-
lowed by an extrinsic evaluation using classiﬁcation tasks.
Intrinsic Evaluation. We use word similarity and catego-
rization benchmarks for intrinsic evaluation. That is, we ex-


key: chroma:document
string_value:
amine to what degree our approach retains semantic word
relations. To this end, we measure how semantic relations
manifest in vector space using six datasets: SimLex-999,
WordSim-353, MEN, MTurk-287, Mturk-771, and RG-65.
Each dataset consists of human-scored word pairs, which arecompared with the corresponding vector space similarities.
The categorization tasks evaluate how well we can group
words into distinct word categories, only based on their em-


key: chroma:document
string_value:
bedding. We here use three datasets: AP, BLESS, and ESS-
LLI.3As baselines, we chose Word2Vec, GloVe, and FastText
because of their wide use.
Extrinsic Evaluation. In our extrinsic evaluation, we in-
vestigate how well our logical embedding supports down-
stream NLP classiﬁcation tasks. Using the word embeddings
as feature vectors, the performance of supervised classiﬁca-
tion models gives insight into the embedding quality. We
employ six standard text classiﬁcation datasets from SentE-


key: chroma:document
string_value:
val[Conneau and Kiela, 2018 ]: R8, R52, TREC, SUBJ, SST-
2, and SST-5. For supervised learning, we use the standard
attention-based BiLSTM model with the Adam optimizer and
cross-entropy loss function. In this manner, we directly con-
trast GloVe embedding against the logical TM approach.
Embedding Datasets. For extrinsic evaluation with BiL-
STM, we use standard 300-dimensional GloVe embeddings,
pre-trained on the Wikipedia 2014 + Gigaword 5 datasets (6B


key: chroma:document
string_value:
tokens).4The purpose is to compare the TM embedding per-
formance against widely used and successful GloVe embed-
dings on downstream tasks. To directly compare the intrinsic
properties of Word2Vec, GloVe, FastText, and TM embed-
ding, we also train them from scratch using the One Billion
Word dataset [Chelba et al. , 2014 ]. For training the TM, we
use Algorithm 1 with r= 2000 training rounds, produc-
ing2000 examples per word by accumulating u= 25 con-


key: chroma:document
string_value:
texts per example. We use the following hyperparameters: a
pool ofn= 600 clauses, margin T= 1200 , and speciﬁcity
s= 5.0.5Word2Vec Skip-Gram is trained with 10passes
over the data, using separated embeddings for the input and
output contexts. The window size is 5and we use ﬁve nega-
tive samples per example. Similarly, GloVe is trained for 30
epochs with a window size of 10and a learning rate of 0.05.6
5.2 Results and Discussion
As presented in Section 5.1, we employ two kinds of evalu-


key: chroma:document
string_value:
ation: intrinsic and extrinsic. Table 1 contains the intrinsic
evaluation results from the six word similarity tasks. We here
compute the Spearman correlation, the Kendall coefﬁcient,
and the cosine similarity between the human-set similarity
scores and the predicted similarity scores per dataset. Con-
sidering Spearman and Kendall score, Word2Vec and GloVe
are marginally better than the comparable FastText and TM
embedding. However, as reported in [Rastogi et al. , 2015 ],


key: chroma:document
string_value:
3To obtain the categorization accuracy, we use KMeans
clustering from sklearn on the word embeddings and exam-
ine the cluster quality by calculating the purity score from
(https://github.com/purity).
4The pre-trained GloVe embeddings can be found here:
https://nlp.stanford.edu/projects/glove/
5The TM Autoencoder and logical word embedding implemen-
tation can be found here: https://github.com/cair/tmu.
6Word2Vec and FastText have been trained using


key: chroma:document
string_value:
6Word2Vec and FastText have been trained using
the standard gensim library (https://github.com/RaRe-
Technologies/gensim/tree/develop/gensi). GloVe has been trained
using https://github.com/maciejkula/glove-python.


key: chroma:document
string_value:
DatasetW2V FastText TM GloVe
Spearman Kendall Cosine Spearman Kendall Cosine Spearman Kendall Cosine Spearman Kendall Cosine
WS-353 0.53 0.37 0.87 0.46 0.32 0.79 0.45 0.31 0.90 0.41 0.28 0.90
SIM999 0.26 0.18 0.79 0.23 0.16 0.79 0.14 0.10 0.76 0.25 0.17 0.80
MEN 0.71 0.50 0.91 0.71 0.51 0.94 0.64 0.45 0.94 0.73 0.53 0.95
MTURK287 0.66 0.47 0.77 0.63 0.44 0.93 0.63 0.44 0.92 0.66 0.47 0.86
MTURK717 0.57 0.39 0.86 0.52 0.36 0.93 0.48 0.32 0.91 0.58 0.40 0.94


key: chroma:document
string_value:
RG65 0.72 0.58 0.89 0.67 0.49 0.88 0.75 0.63 0.92 0.78 0.62 0.93
Average 0.58 0.42 0.85 0.54 0.38 0.88 0.52 0.38 0.89 0.57 0.42 0.90
Table 1: Performance comparison of TM embedding with baseline algorithms on the similarity task.
Target words ClausesClauses
 student
surgery
heart
baseball
football
queen
princess
coffee
tea
 
Figure 6: Interpretability of clauses capturing distinct meanings of target words in the TM embedding.
Dataset W2V FastText TM GloVe
AP 0.50 0.35 0.41 0.41


key: chroma:document
string_value:
AP 0.50 0.35 0.41 0.41
BLESS 0.64 0.66 0.62 0.66
ESSLI 0.63 0.60 0.57 0.56
Average 0.59 0.54 0.53 0.54
Table 2: Performance comparison of TM embedding with baseline
embeddings on the categorization task.
DatasetGloVe TM TM hybrid
Acc. F1 Acc. F1 Acc. F1
R8 96.31 0.88 96.10 0.88 97.80 0.94
TREC 95.20 0.95 96.40 0.96 96.80 0.96
R52 90.34 0.58 91.23 0.62 94.23 0.68
SUBJ 86.20 0.86 85.80 0.85 86.70 0.87
SST-2 76.38 0.75 75.61 0.74 79.30 0.78
SST-5 47.47 0.46 47.80 0.43 49.75 0.44


key: chroma:document
string_value:
SST-5 47.47 0.46 47.80 0.43 49.75 0.44
Table 3: Performance comparison of our embedding with standard
GloVe embedding on the classiﬁcation task.
small differences in correlation-based measures are not nec-
essarily signiﬁcant for smaller datasets. To more robustly
assess performance, we therefore also use cosine similarity
to compare predicted word similarities with the human-set
similarities. In terms of cosine score, our model outperforms


key: chroma:document
string_value:
Word2Vec and FastText on the majority of the datasets, whileperforming competitively with GloVe. This means that the
angles between the human-set similarities and the GloVe/TM-
predicted similarities are quite similar. Finally, Table 2 shows
the outcome for the word categorization tasks. As seen, the
performance of the selected embedding techniques are com-
parable, with Word2Vec being slightly ahead.
Previous research indicates that intrinsic word similarity


key: chroma:document
string_value:
performance is minimally or even negatively correlated with
downstream NLP performance [Wang et al. , 2021 ]. There-
fore, we also include an extrinsic evaluation with six down-
stream classiﬁcation tasks. To avoid overﬁtting and robustly
assess downstream properties, we keep our experimental
setup from above. Table 3 reports the outcome of the eval-
uation, where the embeddings have been fed to an attention-
based BiLSTM model. The ﬁrst conﬁguration (GloVe) uses


key: chroma:document
string_value:
the pre-trained GloVe embeddings from the Wikipedia 2014
+ Gigaword 5 datasets. The second conﬁguration consists
of our purely logical TM embedding from One Billion Word
(embedding Bfrom Algorithm 1). Being ﬁve times smaller,
the One Billion Word dataset only provides about 80percent
of the vocabulary required for the classiﬁcation tasks. We em-
bed the remaining 20percent of the words randomly. Hence,
the TM approach can potentially have a disadvantage in the


key: chroma:document
string_value:
evaluation. In the third conﬁguration (TM hybrid ), we replace
the20percent random embeddings with the corresponding
GloVe embeddings (approximately 80% TM + 20% GloVe).
We note that the downstream accuracy of BiLSTM is simi-
lar for both TM and GloVe. Speciﬁcally, the TM embedding


key: chroma:document
string_value:
exceeds GloVe by a small margin on TREC, R52, and SST-
5. The hybrid embedding, on the other hand, clearly outper-
forms the other two. In particular, for R52, SST-2, and SST-5,
the hybrid embedding is able to surpass GloVe by a substan-
tial margin of roughly 2−4%. Given that the datasets are
not completely balanced, we also compute F1 macro scores.
We again observe that the TM embedding either outperforms
or is competive with GloVe. For R8 and R52, the hybrid em-


key: chroma:document
string_value:
bedding surpasses GloVe by a large margin, respectively by
around 6%and10%. Based on these results, we conjecture
that logical TM embedding can successfully replace neural
network embedding. Even with 20% of the vocabulary miss-
ing, trained on ﬁve times smaller data, the logical embedding
perform competitively with GloVe. Interestingly, the hybrid
approach performed even better. One possible explanation of
this higher performance can be the extra information added


key: chroma:document
string_value:
by the larger vocabulary. Additionally, there may be synergy
between the neural and logical representations that manifest
in the hybrid approach.
5.3 Interpretability and Visualization
In this section, we investigate the nature of the TM embed-
dings in more detail, focusing on interpretability. Our em-
bedding consists of the positive clause weights E, or, alter-
natively, the propositional version B, explained by the set
of clausesC. As demonstrated in Figure 6, each clause in C


key: chroma:document
string_value:
captures a facet of a context. The dotted lines in the ﬁgure
showcase the connection between the target words and their
clauses from matrix B(and, accordingly, E). Each target
word gets its own color to more easily discern the connec-
tions. In the ﬁgure, we provide an excerpt of 18connections
fromB, involving 8target words and the 11most triggered
clauses for these words. Consider for example the target
words surgery andheart . These two target words share two


key: chroma:document
string_value:
clauses: [ went∧hospital ] and [ old∧disease∧patient ].
The two clauses capture two joint contexts, both related to
health. The clauses thus represent commonality between the
target words, providing information on one particular mean-
ing of the words.
The two target words are also semantically different. The
differences are captured by the clauses they do not share. The
target word heart , for example, also relates to the meaning
[woman∧love], which surgery does not. Surgery , on the


key: chroma:document
string_value:
other hand, connects with [ injury∧game∧racing ]. In this
manner, the unique meanings and relations between words
are represented through sharing of logical expressions. Ac-
cordingly, it is feasible to capture a wide range of possible
contextual representations with concise logical expressions.
As such, the logical embedding provides a sparse representa-
tion of words and their relations. Indeed, at most 10% of the
clauses connect to each word in our experiments. As shown in


key: chroma:document
string_value:
the intrinsic evaluations from the previous subsection, these
contextual representations are effective for measuring word
similarity and categorizing words. Similarly, we observed
that the logical embedding is boosting downstream NLP clas-
siﬁcation tasks.
To cast further light on the TM embedding approach, we
visualize the embedding of 400words from the SimLex-999
dataset in Figure 7, plotted using t-SNE. The ﬁgure indicates
Figure 7: TM embedding visualization plotted using t-SNE.


key: chroma:document
string_value:
that we are able to cluster contextually similar words in vec-
tor space. To scrutinize the clusters, we zoom in on two
of them. Consider the upper right cluster ﬁrst. Notice how
the words in the cluster relate to hospital , such as heart and
diseases . As seen, the word embeddings are closely located
in vector space. Similarly, we can observe that terminology
connected to weather and geography are grouped together in
the bottom cluster. From these two examples, it seems clear


key: chroma:document
string_value:
that the TM embedding incorporates semantic relationships
among words.
6 Conclusion and Future Work
In this work, we ﬁrst discussed the challenge and necessity
of ﬁnding computationally simpler and more interpretable
word embedding approaches. We then motivated an efﬁcient
self-supervised approach, namely, a TM-based autoencoder,
for producing sparse and interpretable logical word embed-
dings. We evaluated our approach on a wide range of intrin-


key: chroma:document
string_value:
sic and extrinsic tasks, demonstrating that it is competitive
with dense neural network-based embedding schemes such
as Word2Vec, GloVe, and FastText. Further, we investigated
the interpretability our embedding through visualization and
a case study. Our conclusion from the study is that the log-
ical embedding is able to represent words with logical ex-
pressions. This structure makes the representation sparse, en-
abling a clear-cut decomposition of each word into sets of
semantic concepts.


key: chroma:document
string_value:
semantic concepts.
Future work includes scaling up our implementation using
GPUs to support building of large scale vocabularies from
more massive datasets. Also, we intend to investigate how
sentence-level and document-level embedding can be created
using clauses, for instance applicable for downstream sen-
tence similarity tasks.


key: chroma:document
string_value:
References
[Angelov, 2020 ]Dimo Angelov. Top2vec: Distributed rep-
resentations of topics. arXiv preprint arXiv:2008.09470 ,
2020.
[Arora et al. , 2017 ]Sanjeev Arora, Yingyu Liang, and
Tengyu Ma. A simple but tough-to-beat baseline for sen-
tence embeddings. In International conference on learning
representations , 2017.
[Arora et al. , 2018 ]Sanjeev Arora, Yuanzhi Li, Yingyu
Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic
structure of word senses, with applications to polysemy.


key: chroma:document
string_value:
Transactions of the Association for Computational Lin-
guistics , 6:483–495, 2018.
[Bengio et al. , 2000 ]Yoshua Bengio, R ´ejean Ducharme, and
Pascal Vincent. A neural probabilistic language model.
Advances in neural information processing systems , 13,
2000.
[Bhattarai et al. , 2022a ]Bimal Bhattarai, Ole-Christoffer
Granmo, and Lei Jiao. A Tsetlin Machine Framework for
Universal Outlier and Novelty Detection. In International
Conference on Agents and Artiﬁcial Intelligence , pages


key: chroma:document
string_value:
250–268. Springer, 2022.
[Bhattarai et al. , 2022b ]Bimal Bhattarai, Ole-Christoffer
Granmo, and Lei Jiao. Explainable Tsetlin Machine
framework for fake news detection with credibility score
assessment. In LREC , 2022.
[Blakely and Granmo, 2021 ]Christian D. Blakely and Ole-
Christoffer Granmo. Closed-Form Expressions for Global
and Local Interpretation of Tsetlin Machines. In
34th International Conference on Industrial, Engineer-
ing and Other Applications of Applied Intelligent Systems


key: chroma:document
string_value:
(IEA/AIE 2021) . Springer, 2021.
[Bojanowski et al. , 2017 ]Piotr Bojanowski, Edouard Grave,
Armand Joulin, and Tomas Mikolov. Enriching word vec-
tors with subword information. Transactions of the asso-
ciation for computational linguistics , 5:135–146, 2017.
[Borgeaud et al. , 2022 ]Sebastian Borgeaud, Arthur Men-
sch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie
Millican, George Bm Van Den Driessche, Jean-Baptiste
Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving


key: chroma:document
string_value:
language models by retrieving from trillions of tokens.
InInternational conference on machine learning , pages
2206–2240. PMLR, 2022.
[Ceret al. , 2018 ]Daniel Cer, Yinfei Yang, Sheng-yi Kong,
Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Con-
stant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar,
et al. Universal sentence encoder for English. In Proceed-
ings of the 2018 conference on empirical methods in nat-
ural language processing: system demonstrations , pages
169–174, 2018.


key: chroma:document
string_value:
169–174, 2018.
[Charniak and others, 2013 ]Eugene Charniak et al. Naive
Bayes word sense induction. In Proceedings of the 2013
Conference on Empirical Methods in Natural Language
Processing , pages 1433–1437, 2013.[Chelba et al. , 2014 ]Ciprian Chelba, Tomas Mikolov, Mike
Schuster, Qi Ge, T. Brants, Phillip Todd Koehn, and Tony
Robinson. One billion word benchmark for measuring
progress in statistical language modeling. In Interspeech ,
2014.
[Conneau and Kiela, 2018 ]Alexis Conneau and Douwe


key: chroma:document
string_value:
Kiela. SentEval: An Evaluation Toolkit for Universal Sen-
tence Representations. In Proceedings of the Eleventh In-
ternational Conference on Language Resources and Eval-
uation (LREC 2018) , 2018.
[Devlin et al. , 2019 ]Jacob Devlin, Ming-Wei Chang, Ken-
ton Lee, and Kristina Toutanova. BERT: Pre-training
of Deep Bidirectional Transformers for Language Under-
standing. In NAACL , 2019.
[Dhillon et al. , 2015 ]Paramveer S Dhillon, Dean P Foster,


key: chroma:document
string_value:
and Lyle H Ungar. Eigenwords: spectral word embed-
dings. J. Mach. Learn. Res. , 16:3035–3078, 2015.
[Faruqui et al. , 2015 ]Manaal Faruqui, Yulia Tsvetkov, Dani
Yogatama, Chris Dyer, and Noah A Smith. Sparse Over-
complete Word Vector Representations. In Proceedings of
the 53rd Annual Meeting of the Association for Computa-
tional Linguistics and the 7th International Joint Confer-
ence on Natural Language Processing (Volume 1: Long
Papers) , pages 1491–1500, 2015.


key: chroma:document
string_value:
Papers) , pages 1491–1500, 2015.
[Glimsdal and Granmo, 2021 ]Sondre Glimsdal and Ole-
Christoffer Granmo. Coalesced Multi-Output Tsetlin Ma-
chines with Clause Sharing. Submitted to ICML’23. Avail-
able as an arXiv preprint, arXiv:2108.07594 , 2021.
[Granmo, 2018 ]Ole-Christoffer Granmo. The Tsetlin Ma-
chine - A Game Theoretic Bandit Driven Approach to Op-
timal Pattern Recognition with Propositional Logic. arXiv
preprint arXiv:1804.01508 , 2018.


key: chroma:document
string_value:
preprint arXiv:1804.01508 , 2018.
[Harris, 1954 ]Zellig S Harris. Distributional structure.
Word , 10(2-3):146–162, 1954.
[Huang et al. , 2020 ]Jui-Ting Huang, Ashish Sharma, Shuy-
ing Sun, Li Xia, David Zhang, Philip Pronin, Janani
Padmanabhan, Giuseppe Ottaviano, and Linjun Yang.
Embedding-based retrieval in Facebook search. In Pro-
ceedings of the 26th ACM SIGKDD International Con-
ference on Knowledge Discovery & Data Mining , pages
2553–2561, 2020.


key: chroma:document
string_value:
2553–2561, 2020.
[Kenton and Toutanova, 2019 ]Jacob Devlin Ming-
Wei Chang Kenton and Lee Kristina Toutanova. Bert:
Pre-training of deep bidirectional transformers for lan-
guage understanding. In Proceedings of NAACL-HLT ,
pages 4171–4186, 2019.
[Lample et al. , 2016 ]Guillaume Lample, Miguel Balles-
teros, Sandeep Subramanian, Kazuya Kawakami, and
Chris Dyer. Neural Architectures for Named Entity
Recognition. In Proceedings of NAACL-HLT , pages 260–
270, 2016.


key: chroma:document
string_value:
270, 2016.
[Lauet al. , 2014 ]Jey Han Lau, Paul Cook, Diana McCarthy,
Spandana Gella, and Timothy Baldwin. Learning word
sense distributions, detecting unattested senses and identi-
fying novel senses using topic models. In Proceedings of


key: chroma:document
string_value:
the 52nd Annual Meeting of the Association for Compu-
tational Linguistics (Volume 1: Long Papers) , pages 259–
270, 2014.
[Levy et al. , 2015 ]Omer Levy, Yoav Goldberg, and Ido Da-
gan. Improving distributional similarity with lessons
learned from word embeddings. Transactions of the as-
sociation for computational linguistics , 3:211–225, 2015.
[Liet al. , 2020 ]Bohan Li, Hao Zhou, Junxian He, Mingx-
uan Wang, Yiming Yang, and Lei Li. On the Sentence


key: chroma:document
string_value:
Embeddings from Pre-trained Language Models. In Pro-
ceedings of the 2020 Conference on Empirical Methods
in Natural Language Processing (EMNLP) , pages 9119–
9130, 2020.
[Logeswaran and Lee, 2018 ]Lajanugen Logeswaran and
Honglak Lee. An efﬁcient framework for learning
sentence representations. In International Conference on
Learning Representations , 2018.
[Mikolov et al. , 2013 ]Tomas Mikolov, Kai Chen, Gre-
gory S. Corrado, and Jeffrey Dean. Efﬁcient Estimation


key: chroma:document
string_value:
of Word Representations in Vector Space. In International
Conference on Learning Representations , 2013.
[Muet al. , 2017 ]Jiaqi Mu, Suma Bhat, and Pramod
Viswanath. Geometry of polysemy. In 5th International
Conference on Learning Representations, ICLR 2017 ,
2017.
[Pennington et al. , 2014 ]Jeffrey Pennington, Richard
Socher, and Christopher D Manning. Glove: Global vec-
tors for word representation. In Proceedings of the 2014
conference on empirical methods in natural language


key: chroma:document
string_value:
processing (EMNLP) , pages 1532–1543, 2014.
[Peters et al. , 2018 ]Matthew E Peters, Mark Neumann, Mo-
hit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee,
and Luke Zettlemoyer. Deep contextualized word repre-
sentations. arXiv preprint arXiv:1802.05365 , 2018.
[Rastogi et al. , 2015 ]Pushpendre Rastogi, Benjamin
Van Durme, and Raman Arora. Multiview LSA: Repre-
sentation learning via generalized CCA. In Proceedings
of the 2015 conference of the North American chapter of


key: chroma:document
string_value:
the Association for Computational Linguistics: human
language technologies , pages 556–566, 2015.
[Reimers and Gurevych, 2019 ]Nils Reimers and Iryna
Gurevych. Sentence-BERT: Sentence Embeddings using
Siamese BERT-Networks. In Proceedings of the 2019
Conference on Empirical Methods in Natural Language
Processing and the 9th International Joint Conference on
Natural Language Processing (EMNLP-IJCNLP) , pages
3982–3992, 2019.
[Schwartz et al. , 2020 ]Roy Schwartz, Jesse Dodge, Noah


key: chroma:document
string_value:
Smith, and Oren Etzioni. Green AI. Communications of
the ACM , 63:54 – 63, 2020.
[Subramanian et al. , 2018 ]Anant Subramanian, Danish
Pruthi, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and
Eduard Hovy. Spine: Sparse interpretable neural em-
beddings. In Proceedings of the AAAI Conference on
Artiﬁcial Intelligence , volume 32, 2018.[Vaswani et al. , 2017 ]Ashish Vaswani, Noam Shazeer, Niki
Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,


key: chroma:document
string_value:
Łukasz Kaiser, and Illia Polosukhin. Attention is all you
need. Advances in neural information processing systems ,
30, 2017.
[Wang et al. , 2021 ]Kexin Wang, Nils Reimers, and Iryna
Gurevych. Tsdae: Using transformer-based sequential de-
noising auto-encoder for unsupervised sentence embed-
ding learning. arXiv preprint arXiv:2104.06979 , 2021.
[Yadav et al. , 2021 ]Rohan Yadav, Lei Jiao, Ole-Christoffer
Granmo, and Morten Goodwin. Human-Level Inter-


key: chroma:document
string_value:
Granmo, and Morten Goodwin. Human-Level Inter-
pretable Learning for Aspect-Based Sentiment Analysis.
InProceedings of AAAI , 2021.
[Zhang et al. , 2020 ]Yan Zhang, Ruidan He, Zuozhu Liu,
Kwan Hui Lim, and Lidong Bing. An Unsupervised Sen-
tence Embedding Method by Mutual Information Max-
imization. In Proceedings of the 2020 Conference
on Empirical Methods in Natural Language Processing
(EMNLP) , pages 1601–1610, 2020.


key: source
string_value:
source_documents\NeurIPS-2022-tsetlin-machine-for-solving-contextual-bandit-problems-Paper-Conference.pdf


key: chroma:document
string_value:
Tsetlin Machine for Solving Contextual Bandit
Problems
Raihan Seraj
Electrical and Computer Engineering
McGill University
Canada
raihan.seraj@mail.mcgill.caJivitesh Sharma
Center for Artificial Intelligence Research (CAIR)
University of Agder
Norway
jivitesh.sharma@uia.no
Ole-Christoffer Granmo
Center for Artificial Intelligence Research (CAIR)
University of Agder
Norway
ole.granmo@uia.no
Abstract
This paper introduces an interpretable contextual bandit algorithm using Tsetlin


key: chroma:document
string_value:
Machines, which is an inherently interpretable machine learning method that
solves complex pattern recognition tasks using propositional (Boolean) logic. The
proposed bandit learning algorithm relies on straightforward bit manipulation, thus
simplifying computation and interpretation. We then present a mechanism for
performing Thompson sampling with Tsetlin Machine, given its non-parametric
nature. Our empirical analysis shows that Tsetlin Machine as a base contextual


key: chroma:document
string_value:
bandit learner outperforms other popular comparable base learners on eight out of
nine datasets. We further analyze the interpretability of our learner, investigating
how arms are selected based on propositional expressions that model the context1.
1 Introduction
Contextual bandits play a fundamental role in many applications involving sequential decision making,
ranging from personalized recommendations [ 19] to designing effective treatment allocation strategies


key: chroma:document
string_value:
in clinical trials [ 11,7]. Algorithms for contextual bandits have additionally gained significant interest
because of their theoretical elegance. In brief, a decision-maker selects one of multiple bandit arms
over a sequence of rounds, taking into account an observed context. Each round, the arm chosen
elicits feedback in the form of a reward signal associated with the success of selecting that arm
(such as a user purchasing a recommended product). The contextual bandit problem is particularly


key: chroma:document
string_value:
intriguing because the decision-maker must maximize the expected reward in as few rounds as
possible, trading exploitation against exploration to identify the optimal arm.
In this paper, we recast the Tsetlin Machine ( TM) [14] as a contextual bandit algorithm and study
the resulting scheme empirically. TMis a recent machine learning approach to pattern recognition
that employs a team of non-contextual bandit algorithms, in the form of Tsetlin automata [ 27], to


key: chroma:document
string_value:
learn patterns expressed in propositional logic. TMs have been shown to obtain competitive accuracy,
memory footprint, and learning speed on several benchmark datasets [ 3,18]. They have been
particularly successful in natural language processing, including explainable aspect-based sentiment
analysis [ 28] and robust text classification with ANDed word negations [ 29]. Being based on finite
state automata, they further support Markov chain-based convergence analysis [30]. Leveraging the


key: chroma:document
string_value:
non-linear pattern recognition capability of TMs, our proposed scheme thus addresses the contextual
1The code is available online on: github
36th Conference on Neural Information Processing Systems (NeurIPS 2022).


key: chroma:document
string_value:
bandit problem using a team of non-contextual bandit algorithms.
Although existing algorithms for contextual bandits provide theoretical guarantees, they are either
difficult to interpret or make assumptions that limit their usability in real-world settings. Motivated
by these limitations, our work contributes as follows:
•We investigate TMas a base learner for contextual bandit problems and empirically demon-
strate its effectiveness compared to other popular comparable base learners.


key: chroma:document
string_value:
•We propose how the Thompson sampling scheme can be leveraged by TMs through boot-
strapping [12].
•We provide interpretability analysis that shows how algorithms using TMs select arms based
on propositional expressions of context features.
To the best of our knowledge, this is the first reported work on learning arm selection strategies for
contextual bandits expressed in propositional logic. The paper is organized as follows. Section 2 and


key: chroma:document
string_value:
3 present the problem formulation and introduces the TMalgorithm, respectively. In Section 4, we
show how TMcan be used as a base learner for contextual bandits. The numerical results and the
interpretability analysis are then presented in Section 5 and 7, respectively. We conclude the paper
and provide directions for future work in Section 9. A literature overview on contextual bandits and
different algorithms for solving them can be found in Appendix A.
2 Problem Formulation


key: chroma:document
string_value:
2 Problem Formulation
We consider an online stochastic contextual bandit setup where at time ta context-reward pair denoted
by(st, rt)is sampled independently from past data distribution D. Here, st∈ S represents an M
dimensional context vector and rt= ((rt(1), . . . , r t(K))∈ {0,1}Kis the reward vector for K
possible actions. The learner chooses an arm ut∈ {1, . . . , K }after observing the context stand


key: chroma:document
string_value:
receives a reward rt(ut)for the chosen arm. The objective of the learner is to perform a sequence of
actions in order to minimize the cumulative expected regret given by
Regret =EhTX
t=1
rt(π∗(st))−rt(ut)i
. (1)
Here, π∗= arg max
π∈ΠE(s,r)∼D[r(π(s))], where Πdenotes the set of large (possibly infinite) policies
andπ:S 7→ { 1, . . . , K }. For our analysis we consider maximizing the expected total reward which
is equivalent to minimizing regret.
Action 1 Action 2
Penatly Reward


key: chroma:document
string_value:
Action 1 Action 2
Penatly Reward
Figure 1: A two-action Tsetlin Automaton with 2Nstates.
3 Tsetlin Machine
Structure. ATMin its simplest form takes a feature vector x= [x1, x2, . . . , x o]∈ {0,1}oofo
propositional values as input and assigns the vector a class ˆy∈ {0,1}. To minimize classification
error, the TM produces nself-contained patterns. In brief, the input vector xprovides the literal set


key: chroma:document
string_value:
L={l1, l2, . . . , l 2o}={x1, x2, . . . , x o,¬x1,¬x2, . . . ,¬xo}, consisting of the input features and
their negations. By selecting subsets Lj⊆Lof the literals, the TM can build arbitrarily complex
patterns, ANDing the selected literals to form conjunctive clauses:
Cj(x) =^
lk∈Ljlk. (2)
2


key: chroma:document
string_value:
Input  
Type I feedback Type I feedbackinclude exclude include exclude
1 N N+1 2 N1 N N+1 2 N
TAstransition
Step 1Step tFigure 2: TM learning dynamics for an XOR-gate training sample, with input ( x1= 0, x2= 1) and
output target y= 1.
Above, j∈ {1,2, . . . , n }refers to a particular clause Cjandk∈ {1,2, . . . , 2o}refers to a particular
literal lk. As an example, the clause Cj(x) =x1∧ ¬x2consists of the literals Lj={x1,¬x2}and
evaluates to 1when x1= 1andx2= 0.


key: chroma:document
string_value:
evaluates to 1when x1= 1andx2= 0.
The TM assigns one Tsetlin Automata ( TA) per literal lkper clause Cjto build the clauses. The TA
assigned to literal lkof clause Cjdecides whether lkisExcluded orIncluded inCj. Figure 1 depicts
a two-action TAwith2Nstates. For states 1toN, the TAperforms action Exclude (Action 1), while
for states N+ 1to2Nit performs action Include (Action 2). As feedback to the action performed,


key: chroma:document
string_value:
the environment responds with either a Reward or a Penalty. If the TA receives a Reward, it moves
deeper into the side of the action. If it receives a Penalty, it moves towards the middle and eventually
switches action.
With nclauses and 2oliterals, we get n×2oTAs. We organize the states of these in an n×2o
matrix A= [aj
k]∈ {1,2, . . . , 2N}n×2o. We will use the function g(·)to map the automaton
stateaj
kto Action 0(Exclude ) for states 1toNand to Action 1(Include ) for states N+ 1to2N:
g(aj


key: chroma:document
string_value:
g(aj
k) =aj
k> N . We can connect the states aj
kof the TAs assigned to clause Cjwith its composition
as follows:
Cj(x) =^
lk∈Ljlk=2o^
k=1h
g(aj
k)⇒lki
. (3)
Here, lkis one of the literals and aj
kis the state of its TAin clause Cj. The logical imply operator ⇒
implements the Exclude /Include action. That is, the imply operator is always 1ifg(aj
k) = 0 (Exclude ),
while if g(aj
k) = 1 (Include ) the truth value is decided by the truth value of the literal. The complexity


key: chroma:document
string_value:
of the algorithm is proportional (linear) to the number of features (input bits). E.g., doubling the
number of features doubles the number of computations[1].
Classification. Classification is performed as a majority vote. The odd-numbered half of the clauses
vote for class ˆy= 1and the even-numbered half vote for ˆy= 0:
ˆy= 0≤n−1X
j=1,3,...2o^
k=1h
g(aj
k)⇒lki
−nX
j=2,4,...2o^
k=1h
g(aj
k)⇒lki
. (4)
As such, the odd-numbered clauses have positive polarity, while the even-numbered ones have


key: chroma:document
string_value:
negative polarity. As an example, consider the input vector x= [0,1]in the lower part of Figure 2.
3


key: chroma:document
string_value:
The figure depicts two clauses of positive polarity, C1(x) =x1∧ ¬x2andC3(x) =¬x1∧ ¬x2
(the negative polarity clauses are not shown). Both of the clauses evaluate to zero, leading to class
prediction ˆy= 1.
INPUTCLAUSE 1 0
LITERAL 1 0 1 0
INCLUDE LITERALP(R EWARD )s−1
sNA 0 0
P(INACTION )1
sNAs−1
ss−1
s
P(P ENALTY ) 0 NA1
s1
s
EXCLUDE LITERALP(R EWARD ) 01
s1
s1
s
P(INACTION )1
ss−1
ss−1
ss−1
s
P(P ENALTY )s−1
s0 0 0
Table 1: Type I FeedbackINPUTCLAUSE 1 0
LITERAL 1 0 1 0


key: chroma:document
string_value:
LITERAL 1 0 1 0
INCLUDE LITERALP(R EWARD ) 0 NA 0 0
P(INACTION )1.0 NA 1.0 1.0
P(P ENALTY ) 0 NA 0 0
EXCLUDE LITERALP(R EWARD ) 0 0 0 0
P(INACTION )1.0 0 1.0 1.0
P(P ENALTY ) 0 1.0 0 0
Table 2: Type II Feedback
Learning. The upper part of Figure 2 illustrates learning. A TM learns online, processing one
training example (x, y)at a time. Based on (x, y), the TM rewards and penalizes its TAs, which
amounts to incrementing and decrementing their states. There are two kinds of feedback: Type


key: chroma:document
string_value:
I Feedback produces frequent patterns and Type II Feedback increases the discrimination power
of the patterns. Type I feedback is given stochastically to clauses with positive polarity when
y= 1 and to clauses with negative polarity when y= 0. Conversely, Type II Feedback is
given stochastically to clauses with positive polarity when y= 0 and to clauses with negative
polarity when y= 1. The probability of a clause being updated is based on the vote sum v:
v=Pn−1
j=1,3,...V2o
k=1h
g(aj
k)⇒lki


key: chroma:document
string_value:
v=Pn−1
j=1,3,...V2o
k=1h
g(aj
k)⇒lki
−Pn
j=2,4,...V2o
k=1h
g(aj
k)⇒lki
. The voting error is calculated
as:
ϵ=T−v y = 1
T+v y = 0.(5)
Here, Tis a user-configurable voting margin yielding an ensemble effect. The probability of updating
each clause is P(Feedback) =ϵ
2T. After random sampling from P(Feedback) has decided which
clauses to update, the following TAstate updates can be formulated as matrix additions, subdividing
Type I Feedback into feedback Type Ia and Type Ib:
A∗


key: chroma:document
string_value:
A∗
t+1=At+FII+FIa−FIb. (6)
Here, At= [aj
k]∈ {1,2, . . . , 2N}n×2ocontains the states of the TAs at time step tandA∗
t+1
contains the updated state for time step t+ 1(before clipping). The matrices FIa∈ {0,1}n×2oand
FIb∈ {0,1}n×2ocontains Type I Feedback. A zero-element means no feedback and a one-element
means feedback. As shown in Table 1, two rules govern Type I feedback:
•Type Ia Feedback is given with probabilitys−1
swhenever both clause and literal are


key: chroma:document
string_value:
swhenever both clause and literal are
1-valued.2It penalizes Exclude actions and rewards Include actions. The purpose is to
remember and refine the patterns manifested in the current input x. This is achieved by
increasing selected TAstates. The user-configurable parameter scontrols pattern frequency,
i.e., a higher sproduces less frequent patterns.
•Type Ib Feedback is given with probability1
swhenever either clause or literal is 0-valued.


key: chroma:document
string_value:
swhenever either clause or literal is 0-valued.
This feedback rewards Exclude actions and penalizes Include actions to coarsen patterns,
combating overfitting. Thus, the selected TA states are decreased.
The matrix FII∈ {0,1}n×2ocontains Type II Feedback to the TAs, given per Table 2.
•Type II Feedback penalizes Exclude actions to make the clauses more discriminative,
combating false positives. That is, if the literal is 0-valued and the clause is 1-valued, TA


key: chroma:document
string_value:
states below N+ 1are increased. Eventually the clause becomes 0-valued for that particular
input, upon inclusion of the 0-valued literal.
The final updating step for training example (x, y)is to clip the state values to make sure that they
stay within value 1and2N:
At+1=clip
A∗
t+1,1,2N
. (7)
2Note that the probabilitys−1
sis replaced by 1when boosting true positives.
4


key: chroma:document
string_value:
For example, both of the clauses in Figure 2 receives Type I Feedback over several training examples,
making them resemble the input associated with y= 1.
Weighted Tsetlin Machine: The learning of weights is based on increasing the weight of clauses
that receive Type Ia feedback (due to true positive output) and decreasing the weight of clauses that
receive Type II feedback (due to false positive output). The overall rationale is to determine which


key: chroma:document
string_value:
clauses are inaccurate and thus must team up to obtain high accuracy as a team (low weight clauses),
and which clauses are sufficiently accurate to operate more independently (high weight clauses). The
weight updating procedure is summarized in Algorithm 1. Here, wiis the weight of clause Ciat the
nthtraining round (ignoring polarity to simplify notation). The first step of a training round is to
calculate the clause output as per Equation 2. The weight of a clause is only updated if the clause


key: chroma:document
string_value:
output Ciis 1 and the clause has been selected for feedback ( Pi= 1). Then the polarity of the clause
and the class label ydecide the type of feedback given. That is, like a regular TM, positive polarity
clauses receive Type Ia feedback if the clause output is a true positive and Type II feedback if the
clause output is a false positive. For clauses with negative polarity, the feedback types switch roles.


key: chroma:document
string_value:
When clauses receive Type Ia or Type II feedback, their weights are updated accordingly. We use the
stochastic searching on the line (SSL) automaton to learn appropriate weights. SSL is an optimization
scheme for unknown stochastic environments pioneered by Oommen [ 22]. The goal is to find an
unknown location λ∗within a search interval [0,1]. In order to find λ∗, the only available information
for the Learning Mechanism (LM) is the possibly faulty feedback from its attached environment E.


key: chroma:document
string_value:
In SSL, the search space λis discretized into Npoints, {0,1/N,2/N, ..., (N−1)/N,1}with N being
the discretization resolution. During the search, the LM has a location λ∈ {0,1/N,2/N, ..., (N−
1)/N,1}, and can freely move to the left or to the right from its current location. The environment E
provides two types of feedback: E= 1is the environment suggestion to increase the value of λby
one step, and E= 0is the environment suggestion to decrease the value of λby one step. The next


key: chroma:document
string_value:
location of λ, i.e.λn+1, can thus be expressed as follows:
λn+1=λn+ 1/N, ifEn= 1,
λn−1/N, ifEn= 0.(8)
λn+1=λn,ifλn= 1andEn= 1,
λn,ifλn= 0andEn= 0.(9)
Asymptotically, the learning mechanics is able to find a value arbitrarily close to λ∗when N→ ∞
andn→ ∞ . In our case, the search space of clause weights is [0,∞], so we use resolution N= 1,
with no upper bound for λ. Accordingly, we operate with integer weights. As described in Algorithm


key: chroma:document
string_value:
1, if the clause output is a true positive, we simply increase the weight by 1. Conversely, if the clause
output is a false positive, we decrease the weight by 1.
By following the above procedure, the goal is to make low precision clauses team up by giving
them low weights, so that they together can reach the summation target T. By teaming up, precision
increases due to the resulting ensemble effect. Clauses with high precision, however, gets a higher


key: chroma:document
string_value:
weight, allowing them to operate more independently.
The above weighting scheme has several advantages. First of all, increment and decrement operations
on integers are computationally less costly than multiplication based updates of real-valued weights.
Additionally, a clause with an integer weight can be seen as multiple copies of the same clause,
making it more interpretable than real-valued weighting, as studied in the next section. Additionally,


key: chroma:document
string_value:
clauses can be turned completely off by setting their weights to 0if they do not contribute positively
to the classification task.
4 Contextual Bandits with Tsetlin Machines
We use TMas a contextual bandit learner that learns a mapping from context to actions incrementally
with streaming data. Since both inputs, patterns and outputs of TM are represented as bits [ 14],
eachMdimensional context stis binarized with appropriate number of bits. This results in a B


key: chroma:document
string_value:
dimensional binarized context with B≥Mwhich is fed to each TMlearner. We outline appropriate
choices of bits for binarization for different datasets in the next section. The TMlearner, being
non-parametric, has the advantage that it makes few or no assumption about the underlying functions
to be learned; hence it is adaptive and exhibit high degree of flexibility. Another popular non-
parametric learner for contextual bandits is the decision tree, which has been thoroughly studied in


key: chroma:document
string_value:
the literature [ 12,26,13]. The TM, in contrast, learns a linear combination of conjunctive clauses
5


key: chroma:document
string_value:
Algorithm 1 Complete WTM learning process
1:Input: Training data batch (B, x, y )▷B≥1
2:Initialize: Random initialization of TAs
3:Begin: nthtraining round
4:fori= 1, ..., m do ifpi= 1
5: if(y= 1andiis odd) or(y= 0andiis even) then
6: ifci= 1then
7: wi←wi+ 1
8: forfeature k= 1, ...,2odo
9: iflk= 1then
10: Type Ia Feedback
11: else:
12: Type Ib Feedback
13: end if
14: end for
15: else:
16: wi←wi▷[No Change]
17: Type Ib Feedback
18: end if
19: else: (y= 1andiis even) or(y= 0andiis odd)


key: chroma:document
string_value:
19: else: (y= 1andiis even) or(y= 0andiis odd)
20: ifci= 1then
21: ifwi>0then
22: wi←wi−1
23: end if
24: forfeature k= 1, ...,2odo
25: iflk= 0then
26: Type II Feedback
27: else:
28: Inaction
29: end if
30: end for
31: else:
32: wi←wi▷[No Change]
33: Inaction
34: end if
35: end if
36:end for
in propositional logic by producing decision rules similar to the branches in decision trees [ 1], with
the added advantage of being memory and energy efficient, computationally simple and not having


key: chroma:document
string_value:
a tendency to overfit the training data [ 18]. We consider two contextual bandit learning algorithms
using TM: (i) TM with epsilon greedy arm selection; and (ii) TM with Thompson sampling.
Epsilon greedy TM:The exploration-exploitation trade-off is a fundamental problem in learning to
make decisions under uncertainty. In a multi-armed bandit setting, the ε−greedy algorithm is one
of the simplest ones. The learner either chooses the empirically best arm with probability (1−ε)


key: chroma:document
string_value:
(exploitation) or a random arm with probability ε(exploration). In the contextual bandit setting,
given a set of contexts, the learner chooses to select the current empirically best arm with some high
probability, maximizing immediate rewards. Otherwise, it selects a randomized arm with the hope
of improving future rewards given the context. Variations of epsilon greedy algorithms have been
explored in the literature, including decaying the εparameter and eventually dropping the probability


key: chroma:document
string_value:
of choosing a random arm to zero. In our setting, we consider a fixed small value of εfor each TM
learner associated with the arm.
TMwith Thompson sampling: We now show how Thompson sampling can be achieved with TM.
Our approach is similar to that presented in the Tree Bootstrap Algorithm in [ 12]. Since a TM
learner is non-parametric, we use bootstrapping to simulate the behavior of sampling from a posterior


key: chroma:document
string_value:
distribution. At each time instant t,Ncontext reward pairs are bootstrapped from the Dt,uwith
replacement, where Dt,urepresents the set of observations (context reward pairs) for arm uand
N=|Dt,u|. ATMlearner is fitted to each of these bootstrapped datasets and at each time t, the
6


key: chroma:document
string_value:
bootstrapping algorithm selects the arm utthat has the maximum probability of success ˆp. For ease
of exposition, we outline Thompson Sampling with bootstrapping for TMs in Algorithm 2.
Algorithm 2 Thompson Sampling with TM
fort= 1 toTdo
get context st
foru= 1, . . . , K do
Sample bootstrapped dataset ˜Dt,uof size Nfrom Dt,uwith replacement.
Fit Tsetlin Machine TM t,uto˜Dt,u
end for
Choose action ut= arg max
u(ˆp(TM t,u, st))
Update Dt,utwith(st, rt,ut)
end for
5 Empirical Analysis


key: chroma:document
string_value:
end for
5 Empirical Analysis
Given the interactive aspect of contextual bandit algorithms, it is often difficult to evaluate them on
real datasets except for a handful number of tasks [ 6]. Therefore, we use standard supervised learning
classification datasets for our evaluation. We consider both binary as well as multiclass classification
datasets. Each unique label in these datasets is considered as an arm of the equivalent contextual


key: chroma:document
string_value:
bandit problem, where a separate learner associated with each arm is trained independently. Our
numerical analysis contrasts the performance of TMwithε−greedy arm selection and TMwith
Thompson sampling against Tree Bootstrap [ 12], Linear UCB [ 8], Neutral Network ( ε−greedy) and
Logistic regression ( ε−greedy). For our analysis we consider two scenarios described as follows.
Scenario 1: In this scenario, we consider five standard supervised learning classification datasets from


key: chroma:document
string_value:
UCI machine learning repository [ 10]:Iris, Breast Cancer Wisconsin (Diagnostic), Adult, Covertype ,
andStatlog (Shuttle) . The response variables for each of these datasets are, Iris types, Diagnosis,
Occupation , and Covertypes , respectively, where as for the Statlog(Shuttle) dataset, we consider the
last column as the response variable. Additionally we considered two other classification datasets:


key: chroma:document
string_value:
MNIST [9] and Noisy XOR , where the respective response variables are digits and the XOR output .
The noisy XOR dataset contains the XOR operation of 12bit input, where the output is flipped with
probability 0.4. These classification datasets are converted into a contextual bandit problem where
the learner receives a reward of +1for correctly identifying the target value and 0for incorrect
classification. The datasets are processed by removing entries with missing values.


key: chroma:document
string_value:
Scenario 2: In this scenario, we consider two datasets: Movielens 100 K [15] and Simulated
Article [23]. Both these datasets simulate a recommender system. The reward function for the
Movielens dataset is the user’s rating for a particular movie while for Simulated Article dataset, a
reward of +1is received if the recommended article is clicked by the user. The Movielens dataset
is preprocessed, so that top 10 rated movies are selected. One important takeaway from these two


key: chroma:document
string_value:
datasets is that only the reward function for correct recommendation is provided. For instance the
Simulated Article dataset provides recommended articles that has been clicked by the user. Similarly,
the Movielens dataset provides user movie ratings, where the users only rated the movies they have
watched. In such circumstances, the rewards are partial and sparse since there are no information on


key: chroma:document
string_value:
the ratings of all movies or user click information for each article. To circumvent this, we perform
singular value decomposition (SVD) for both the datasets which is a popular approach in collaborative
filtering [ 17]. For the Movielens dataset, let Si,jbe the rating of user ion movie j. Low rank SVD
of this matrix results in two matrices WandX, where S≈W∗X. The ithrow of Wrepresents
the context features for each user iand the rows of Xrepresents the actions or the movie to be


key: chroma:document
string_value:
recommended. The reward for recommending a movie jto user iis then the dot product of the
corresponding row of WiandXj. We perform SVD for Simulated Article dataset in a similar manner.
In order to obtain binarized rewards for the actions, the maximum reward corresponding to an arm
that yields a reward of 1, and the rest a reward of 0. For our analysis we consider rank 10for both the
Movielens and the Simulated Article dataset.


key: chroma:document
string_value:
Movielens and the Simulated Article dataset.
For both the scenarios we use appropriate binarization before fitting the TMlearners. Details of
binarization and parameter choices are provided in Appendix B. The results for both the scenarios
are presented in Figures 3. These results show that for eight datasets, TMwith Thompson sampling
outperforms all other algorithms except for Linear UCB on Covertype. Further, TMwithε−greedy
7


key: chroma:document
string_value:
Figure 3: Performance comparison of TM with different base learners.
(a) Iris
0 200 400 600 800 1000
Rounds0.30.40.50.60.70.80.91.0Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (b) Breast Cancer
0 200 400 600 800 1000
Rounds0.40.50.60.70.80.91.0Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (c) MNIST
0 200 400 600 800 1000


key: chroma:document
string_value:
(c) MNIST
0 200 400 600 800 1000
Rounds0.00.10.20.30.40.50.60.7Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
(d) Adult
0 500 1000 1500 2000
Rounds0.00.20.40.60.8Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (e) Covertype
0 500 1000 1500 2000
Rounds0.10.20.30.40.50.6Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy


key: chroma:document
string_value:
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (f) MovieLens
0 200 400 600 800 1000
Rounds0.00.10.20.30.40.50.60.70.8Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
(g) Statlog
0 500 1000 1500 2000
Rounds0.20.40.60.81.0Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (h) Noisy XOR


key: chroma:document
string_value:
Neural Network -greedy
 (h) Noisy XOR
0 200 400 600 800 1000
Rounds0.30.40.50.60.70.80.91.0Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
 (i) Simulated Article
0 500 1000 1500 2000
Rounds0.00.20.40.60.81.0Mean RewardLinear UCB
Tree bootstrap
TM Thompson Sampling
TM -greedy
Logistic Regression -greedy
Neural Network -greedy
provides competitive performance when compared with popular contextual bandit algorithms in the


key: chroma:document
string_value:
literature. Also, the TMlearner learns at a faster rate compared to other learners. These experiments
were performed with 10independent runs for each dataset. The average result across the independent
runs are reported.
6 Choice of Binarization Parameters
The working principle of TM relies on binarization of the context features. We use gap-based
thresholding as described in [ 2], each threshold yielding one bit. The binarization is performed with


key: chroma:document
string_value:
the parameter which controls the maximum number of bits allowed to represent each feature. We
varied the number of bits allowed per feature for binarization and performed an ablation study across
these datasets, in order to observe the effects of sub-optimal binarization. Our analysis shows that
TM as a base learner is robust to the choice of maximum bits per feature allowable for binarization.
Detailed evaluation results of TM with epsilon greedy and TM with Thompson sampling are provided


key: chroma:document
string_value:
in Appendix C.
7 Interpretability
Unlike neural networks and some other complex machine learning approaches, one of the advantages
ofTMs is that they produce propositional logic expressions. These are in flat AND-form, which have
proven to be human interpretable [ 21]. As explained in Section 3, each propositional expression is a
conjunctive clause, consisting of features, in their original or negated forms, interacting with each


key: chroma:document
string_value:
other using logical AND operations. These clauses can form a simplified representation of the arm
selection policy by combining them into a single Disjunctive Normal Form (DNF) expression. Since
8


key: chroma:document
string_value:
clauses are assigned to each arm of the multi-armed contextual bandit problem, we produce a single
DNF expression for each arm. These DNF expressions are propositional logic expressions made up
of the context. The TMis able to produce these interpretations demonstrating how it interprets the
context with respect to each arm.
Here, we show the simplified propositional expressions for each arm, obtained from TMs trained
with Thompson Sampling on the Iris dataset:
Arm-1: x10∨x14∨x15∨x3


key: chroma:document
string_value:
Arm-1: x10∨x14∨x15∨x3
Arm-2: ¬x1∨x12∨ ¬x13∨ ¬x14∨x16∨x8∨ ¬x9∨(x10∧x11∧x15∧x2∧ ¬x4∧x5∧x6∧
¬x7)∨(x10∧x11∧x15∧x3∧ ¬x4∧x5∧ ¬x7)
Arm-3: ¬x10∨ ¬x11∨ ¬x15∨(x1∧ ¬x12∧x13∧x14∧ ¬x16∧x2∧x3∧x5∧ ¬x8∧x9)∨
(¬x12∧ ¬x16∧x4)
The next set of propositional expressions represent the arms for the Simulated Article dataset:
Arm-1: ¬x4∨(x1∧ ¬x10∧ ¬x11∧x12∧x13∧x14∧x15∧x16∧x17∧x18∧x19∧x2∧x20∧
¬x21∧x3∧x5∧x6∧x7∧x8∧x9)
Arm-2: (x1∧ ¬x10∧ ¬x11∧x12∧x13∧x14∧x15∧x16∧x17∧x18∧x19∧x2∧x20∧ ¬x21∧


key: chroma:document
string_value:
x3∧x4∧x5∧x6∧x7∧x8∧x9)∨(¬x1∧x10∧x11∧x12∧x13∧x14∧x15∧x16∧
x17∧x18∧x19∧x2∧ ¬x20∧ ¬x21∧x3∧x4∧x5∧x6∧ ¬x7∧x8∧x9)
Arm-3: (x1∧ ¬x10∧ ¬x11∧x12∧x13∧x14∧x15∧x16∧x17∧x19∧x2∧x20∧ ¬x21∧x3∧
x4∧x5∧x6∧x7∧x9)∨(¬x11∧x12∧x13∧x14∧x16∧x17∧x18∧x19∧x2∧x20∧
¬x21∧x3∧x4∧x5∧x8∧x9)
Arm-4: x10∧x11∧x12∧x13∧x14∧x15∧x16∧x17∧x18∧x19∧x2∧x3∧x4∧x5∧x6∧
x8∧x9∧ ¬x1∧ ¬x20∧ ¬x21∧ ¬x7
The propositional expressions shown above exhibit how the TMlearns interactions between proposi-


key: chroma:document
string_value:
tional input features (literals xi). As seen, the interactions are learned with logical AND operations
between original and negated features. These learnt interactions (clauses) are combined with the
logical ORoperation (or by addition for T > 1). From the propositional expressions, we can see
which arm is selected by TMby simply plugging in the input values. An arm is selected if its
propositional expression evaluates to True (or it obtains the largest net sum for T > 1). This is


key: chroma:document
string_value:
arguably an important advantage of TMs — a trained TMmodel can be reduced down to simple
propositional expressions. Of course, as such, any machine learning algorithm can be converted into
logical form. However, the complexity of such representations can be immense for the competitive
classic machine learning models, which the TMstill outperforms by a large margin [ 24]. Note that,
although the length of expressions can increase with the increase in dimensionality of data, there are


key: chroma:document
string_value:
simple ways of reducing the length of expressions in O(n)time [20].
8 Limitations
One of the limitations of using TMis that it requires the contexts to be binarized. While such
binarization leads to a loss of information, our empirical analysis shows that TMwith Thompson
sampling performs substantially better than the other evaluated learners on the majority of the datasets
and TM learners are quite robust to sub-optimal binarization. While our work does not provide any


key: chroma:document
string_value:
theoretical guarantees on performance or regret bounds for TM learners, we believe the latter will
form an interesting direction of research.
9 Conclusion
In this paper, we presented an interpretable and practical contextual bandit learner using Tsetlin
Machine (TM). Our analysis showed that TMas a contextual bandit learner provides competitive
performance compared to other popular contextual bandit algorithms. We then presented how


key: chroma:document
string_value:
Thompson sampling can be implemented using TM, where our approach is derived from the Tree
Bootstrap algorithm. Finally, we perform interpretability analysis where the arm selection strategy
can be characterized by a propositional function of the contexts.. Having such promising empirical
results, we aim at providing theoretical performance guarantees for algorithms with TMas future
work.
9


key: chroma:document
string_value:
References
[1]K Darshana Abeyrathna, Ole-Christoffer Granmo, and Morten Goodwin. Extending the tsetlin
machine with integer-weighted clauses for increased interpretability. IEEE Access , 9:8233–8248,
2021.
[2]K Darshana Abeyrathna, Ole-Christoffer Granmo, Xuan Zhang, and Morten Goodwin. Adaptive
continuous feature binarization for tsetlin machines applied to forecasting dengue incidences in
the philippines. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI) , pages


key: chroma:document
string_value:
2084–2092. IEEE, 2020.
[3]Kuruge Darshana Abeyrathna, Bimal Bhattarai, Morten Goodwin, Saeed Rahimi Gorji, Ole-
Christoffer Granmo, Lei Jiao, Rupsa Saha, and Rohan K Yadav. Massively parallel and asyn-
chronous tsetlin machine architecture supporting almost constant-time scaling. In International
Conference on Machine Learning , pages 10–20. PMLR, 2021.
[4]Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear


key: chroma:document
string_value:
payoffs. In International Conference on Machine Learning , pages 127–135. PMLR, 2013.
[5]Robin Allesiardo, Rapha ¨el F´eraud, and Djallel Bouneffouf. A neural networks committee for
the contextual bandit problem. In International Conference on Neural Information Processing ,
pages 374–381. Springer, 2014.
[6]Alberto Bietti, Alekh Agarwal, and John Langford. A contextual bandit bake-off. Journal of
Machine Learning Research , 22(133):1–49, 2021.


key: chroma:document
string_value:
Machine Learning Research , 22(133):1–49, 2021.
[7]Djallel Bouneffouf, Irina Rish, and Charu Aggarwal. Survey on applications of multi-armed
and contextual bandits. In 2020 IEEE Congress on Evolutionary Computation (CEC) , pages
1–8. IEEE, 2020.
[8]Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff
functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence


key: chroma:document
string_value:
and Statistics , pages 208–214. JMLR Workshop and Conference Proceedings, 2011.
[9]Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE
Signal Processing Magazine , 29(6):141–142, 2012.
[10] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[11] Audrey Durand, Charis Achilleos, Demetris Iacovides, Katerina Strati, Georgios D Mitsis,
and Joelle Pineau. Contextual bandits for adapting treatment in a mouse model of de novo


key: chroma:document
string_value:
carcinogenesis. In Machine learning for healthcare conference , pages 67–82. PMLR, 2018.
[12] Adam N. Elmachtoub, Ryan McNellis, Sechan Oh, and Marek Petrik. A practical method
for solving contextual bandit problems using decision trees. In Gal Elidan, Kristian Kersting,
and Alexander T. Ihler, editors, Proceedings of the Thirty-Third Conference on Uncertainty
in Artificial Intelligence, @articleTsetlin1961, author = Tsetlin, Michael Lvovitch, journal =


key: chroma:document
string_value:
Avtomat. i Telemekh, number = 10, pages = 1345–1354, title = On behaviour of finite automata
in random medium, volume = 22, year = 1961 UAI 2017, Sydney, Australia, August 11-15, 2017 .
AUAI Press, 2017.
[13] Rapha ¨el F´eraud, Robin Allesiardo, Tanguy Urvoy, and Fabrice Cl ´erot. Random forest for the
contextual bandit problem. In Artificial intelligence and statistics , pages 93–101. PMLR, 2016.
[14] Ole-Christoffer Granmo. The tsetlin machine–a game theoretic bandit driven approach to


key: chroma:document
string_value:
optimal pattern recognition with propositional logic. arXiv preprint arXiv:1804.01508 , 2018.
[15] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm
transactions on interactive intelligent systems (tiis) , 5(4):1–19, 2015.
[16] Insaf Ismath, KB Shashika Manosha, Samad Ali, Nandana Rajatheva, and Matti Latva-aho. Deep
contextual bandits for fast initial access in mmwave based user-centric ultra-dense networks. In


key: chroma:document
string_value:
[17] Yehuda Koren, Robert Bell, and Chris V olinsky. Matrix factorization techniques for recom-
mender systems. Computer , 42(8):30–37, 2009.
[18] Jie Lei, Adrian Wheeldon, Rishad Shafik, Alex Yakovlev, and Ole-Christoffer Granmo. From
arithmetic to logic based ai: A comparative analysis of neural networks and tsetlin machine. In
2020 27th IEEE International Conference on Electronics, Circuits and Systems (ICECS) , pages
1–4, 2020.


key: chroma:document
string_value:
1–4, 2020.
[19] Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to
personalized news article recommendation. In Proceedings of the 19th international conference
on World wide web , pages 661–670, 2010.
[20] Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ond ˇrejˇCert´ık, Sergey B. Kirpichev,
Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rath-


key: chroma:document
string_value:
nayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta,
Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, ˇStˇep´an
Rouˇcka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Sco-
patz. Sympy: symbolic computing in python. PeerJ Computer Science , 3:e103, January
2017.
[21] I. Noveck, R. B. Lea, George M. Davidson, and D. O’brien. Human reasoning is both logical


key: chroma:document
string_value:
and pragmatic. Intellectica , 11:81–109, 1991.
[22] B.J. Oommen. Stochastic searching on the line and its applications to parameter learning
in nonlinear optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B
(Cybernetics) , 27(4):733–739, 1997.
[23] Dattaraj Rao. Contextual bandits for adapting to changing user preferences over time. arXiv
preprint arXiv:2009.10073 , 2020.
[24] Jivitesh Sharma, Rohan Kumar Yadav, Ole-Christoffer Granmo, and Lei Jiao. Drop clause:


key: chroma:document
string_value:
Enhancing performance, interpretability and robustness of the tsetlin machine. CoRR ,
abs/2105.14506, 2022.
[25] Yilin Shen, Yue Deng, Avik Ray, and Hongxia Jin. Interactive recommendation via deep
neural memory augmented contextual bandits. In Proceedings of the 12th ACM Conference on
Recommender Systems , pages 122–130, 2018.
[26] Dennis Soemers, Tim Brys, Kurt Driessens, Mark Winands, and Ann Now ´e. Adapting to


key: chroma:document
string_value:
concept drift in credit card transaction data streams using contextual bandits and decision trees.
InProceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.
[27] Michael Lvovitch Tsetlin. On behaviour of finite automata in random medium. Avtomat. i
Telemekh , 22(10):1345–1354, 1961.
[28] Rohan Kumar Yadav, Lei Jiao, Ole-Christoffer Granmo, and Morten Goodwin. Human-level
interpretable learning for aspect-based sentiment analysis. In The Thirty-Fifth AAAI Conference


key: chroma:document
string_value:
on Artificial Intelligence (AAAI-21). AAAI , 2021.
[29] Rohan Kumar Yadav, Lei Jiao, Ole Christoffer Granmo, and Morten Goodwin. Robust Inter-
pretable Text Classification against Spurious Correlations Using AND-rules with Negation. In
The 31st International Joint Conference on Artificial Intelligence (IJCAI) , 2022.
[30] Xuan Zhang, Lei Jiao, Ole-Christoffer Granmo, and Morten Goodwin. On the Convergence
of Tsetlin Machines for the IDENTITY- and NOT Operators. IEEE Transactions on Pattern


key: chroma:document
string_value:
Analysis and Machine Intelligence , 2021.
[31] Dongruo Zhou, Lihong Li, and Quanquan Gu. Neural contextual bandits with ucb-based
exploration. In International Conference on Machine Learning , pages 11492–11502. PMLR,
2020.
11


key: chroma:document
string_value:
Checklist
1. For all authors...
(a)Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 8
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d)Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes]
2. If you are including theoretical results...


key: chroma:document
string_value:
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a)Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)? [Yes] See footnote on
first page.


key: chroma:document
string_value:
first page.
(b)Did you specify all the training details (e.g., data splits, hyperparameters, how they
were chosen)? [Yes]
(c)Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)? [No]
(d)Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [No]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...


key: chroma:document
string_value:
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c)Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d)Did you discuss whether and how consent was obtained from people whose data you’re
using/curating? [N/A]
(e)Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content? [N/A]


key: chroma:document
string_value:
information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a)Did you include the full text of instructions given to participants and screenshots, if
applicable? [N/A]
(b)Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable? [N/A]
(c)Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation? [N/A]
12


key: chroma:document
string_value:
Tsetlin Machine Embedding: Representing Words Using Logical
Expressions
Anonymous ACL submission
Abstract
Embedding words in vector space is a fun- 001
damental first step in state-of-the-art natural 002
language processing (NLP). Typical NLP so- 003
lutions employ pre-defined vector representa- 004
tions to improve generalization by co-locating 005
similar words in vector space. For instance, 006
Word2Vec is a self-supervised predictive model 007


key: chroma:document
string_value:
that captures the context of words using a neu- 008
ral network. Similarly, GLoVe is a popular 009
unsupervised model incorporating corpus-wide 010
word co-occurrence statistics. Such word em- 011
bedding has significantly boosted important 012
NLP tasks, including sentiment analysis, doc- 013
ument classification, and machine translation. 014
However, the embeddings are dense floating- 015
point vectors, making them expensive to com- 016
pute and difficult to interpret. In this paper, we 017


key: chroma:document
string_value:
instead propose to represent the semantics of 018
words with a few defining words that are related 019
using propositional logic. To produce such 020
logical embeddings, we introduce a Tsetlin 021
Machine-based autoencoder that learns logi- 022
cal clauses self-supervised. The clauses consist 023
of contextual words like “black”, “cup”, and 024
“hot” to define other words like “coffee”, thus 025
being human-understandable. We evaluate our 026
embedding approach on several intrinsic and 027


key: chroma:document
string_value:
embedding approach on several intrinsic and 027
extrinsic benchmarks, outperforming GLoVe 028
on six classification tasks. Furthermore, we in- 029
vestigate the interpretability of our embedding 030
using the logical representations acquired dur- 031
ing training. We also visualize word clusters 032
in vector space, demonstrating how our logical 033
embedding co-locate similar words.1034
1 Introduction 035
The success of natural language processing (NLP) 036


key: chroma:document
string_value:
relies on advances in word, sentence, and docu- 037
ment representation. By capturing word semantics 038
1The Tsetlin Machine Autoencoder and logical word em-
bedding implementation is available here: XXXX (Removed
for anonymity).and similarities, such representations boost the per- 039
formance of downstream tasks (Borgeaud et al., 040
2022), including clustering, topic modelling (An- 041
gelov, 2020), searching, and text mining (Huang 042
et al., 2020). 043


key: chroma:document
string_value:
et al., 2020). 043
While straightforward, the traditional bag-of- 044
words encoding does not consider the words’ posi- 045
tion, semantics, and context within a document. 046
Distributed word representation (Bengio et al., 047
2000; Bojanowski et al., 2017) addresses this lack 048
by encoding words as low-dimensional vectors, 049
referred to as embeddings . The purpose is to 050
co-locate similar or contextually relevant words 051
in vector space. There are many algorithms for 052


key: chroma:document
string_value:
learning word embeddings. Contemporary self- 053
supervised techniques like Word2Vec (Mikolov 054
et al., 2013), FastText (Bojanowski et al., 2017), 055
and GloVe (Pennington et al., 2014) have demon- 056
strated how to build embeddings from word co- 057
occurrence, utilizing massive training data. Intro- 058
ducing context-dependent embeddings, the more 059
sophisticated language models BERT (Devlin et al., 060
2019) and ELMO (Peters et al., 2018) now perform 061


key: chroma:document
string_value:
remarkably well in downstream tasks (Reimers and 062
Gurevych, 2019). However, they require significant 063
computation power (Schwartz et al., 2020). 064
The above approaches represent words as dense 065
floating point vectors. Word2Vec, for instance, typ- 066
ically builds a 300-dimensional vector per word. 067
The size and density of these vectors make them ex- 068
pensive to compute and difficult to interpret. Con- 069
sider, for example, the word “queen.” Representing 070


key: chroma:document
string_value:
it with 300 floats seems inefficient compared to 071
the Oxford Language definition for the same word: 072
“the female ruler of an independent state, especially 073
one who inherits the position by right of birth.” 074
From this perspective, it appears advantageous to 075
create embeddings directly from words rather than 076
from arbitrary floating-point values. Such inter- 077
pretable embeddings would capture the multiple 078
meanings of a word using a few defining words, 079
1


key: chroma:document
string_value:
simplifying both computation and interpretation. 080
In this paper, we propose a Tsetlin Ma- 081
chine (TM) (Granmo, 2018) based autoencoder 082
for creating interpretable embeddings. The autoen- 083
coder builds propositional logic expressions with 084
context words that identify each target word. The 085
term “coffee” can, for instance, be represented by 086
“one,” “hot,” “cup,” “table,” and “black.” In this 087
manner, the TM builds contextual representations 088


key: chroma:document
string_value:
from a vast text corpus, which model the seman- 089
tics of each word. In contrast to neural network- 090
based embedding, the logical TM embedding is 091
sparse and energy efficient (Maheshwari et al., 092
2023; Abeyrathna et al., 2023). The embedding 093
space consists of, e.g., 500truth values, where each 094
truth value is a logical expression over words. For 095
contextual representation, each target word links to 096
less than ten percent of these expressions. Despite 097


key: chroma:document
string_value:
the sparsity and crispness of this representation, it 098
is competitive with neural network-based embed- 099
ding. 100
The contributions of our work are summarized 101
below: 102
•We propose the TM-based Autoencoder to 103
learn efficient encodings in a self-supervised 104
manner. To the best of our knowledge, it is 105
the first logic-based word embedding. 106
•We introduce TM-based word embedding that 107
builds human-comprehensible contextual rep- 108
resentations from unlabeled data. 109


key: chroma:document
string_value:
resentations from unlabeled data. 109
•We compare our embedding with state-of-the- 110
art approaches on several intrinsic and extrin- 111
sic benchmarks, outperforming GloVe on six 112
downstream classification tasks. 113
2 Related Work 114
The majority of self-supervised embedding ap- 115
proaches produce dense word representations 116
based on the distributional hypothesis (Harris, 117
1954), which states that words that occur in the 118
same context are likely to have similar meanings. 119


key: chroma:document
string_value:
Word2Vec (Mikolov et al., 2013) is one of the best- 120
known models. It builds embeddings from word co- 121
occurrence using a neural network, leveraging the 122
hidden layer output weights. GloVe (Pennington 123
et al., 2014), on the other hand, embeds by factoriz- 124
ing a word co-occurrence matrix. Similarly, canon- 125
ical correlation analysis (CCA) is used in (Dhillon 126
et al., 2015) for embedding words to maximize con- 127


key: chroma:document
string_value:
text correlation. In (Levy et al., 2015), it is demon- 128strated how precise factorization-based SVD can 129
compete with neural embedding. However, all of 130
these methods are challenging to train because they 131
involve tweaking algorithms and hyperparameters 132
toward particular applications (Lample et al., 2016), 133
limiting their wider applicability. 134
Building upon word embedding, several stud- 135
ies focus on sentence embedding (Arora et al., 136


key: chroma:document
string_value:
2017; Logeswaran and Lee, 2018). Recent ad- 137
vances in sentence embedding include supervised 138
data inference (Reimers and Gurevych, 2019), mul- 139
titask learning (Cer et al., 2018), contrastive learn- 140
ing (Zhang et al., 2020), and pretrained large lan- 141
guage models (Li et al., 2020). However, the ma- 142
jority of sentence embedding techniques overlook 143
intrinsic evaluations such as similarity tasks, and 144
instead largely focus on extrinsic evaluations in- 145


key: chroma:document
string_value:
volving downstream performance. The most re- 146
cent building block for embedding originates from 147
the transformer approach (Vaswani et al., 2017). 148
Transformers provide context awareness by utiliz- 149
ing stacks of self-attention layers. BERT (Kenton 150
and Toutanova, 2019), for instance, employs the 151
transformer architecture to carry out extensive self- 152
supervised training, making it capable of producing 153
text embedding. Other embedding models use a 154


key: chroma:document
string_value:
text embedding. Other embedding models use a 154
contrastive loss function to perform supervised fine- 155
tuning on positive and negative text pairs (Wang 156
et al., 2021). Despite the large variety of text em- 157
bedding models, they all share three main draw- 158
backs: i) they are computationally demanding to 159
train; ii) they are intrinsically complex because they 160
are trained on a large amount of data to tune a huge 161
amount of parameters; and iii) the embeddings pro- 162


key: chroma:document
string_value:
duced from these models are not easily interpreted 163
by humans. 164
To improve interpretability, Faruqui et al. 165
introduced “Sparse Overcomplete Word Vec- 166
tors" (SPOWV) which create a sparse non-negative 167
projection of word embedding using dictionary 168
learning (Faruqui et al., 2015). Similarly, SParse In- 169
terpretable Neural Embeddings (SPINE) employs a 170
k-sparse denoising autoencoder to generate sparse 171
embeddings (Subramanian et al., 2018). However, 172


key: chroma:document
string_value:
these methods are unable to distinguish between 173
multiple context-dependent word meanings. To 174
address this problem, another avenue of research 175
focuses on composing linear combinations of dense 176
vectors from Word2Vec and GloVe (Arora et al., 177
2018). However, the assumption of linearity does 178
not hold for real-world data, yielding linear coeffi- 179
cients that are difficult to comprehend (Mu et al., 180
2


key: chroma:document
string_value:
Clause Pool+4 -5Input Output
+1 +2 -7 +6Figure 1: Tsetlin Machine Autoencoder. In this illus-
tration, x1is masked by replacing it with value 1 for
inferring ˆx1.
2017). 181
The logical embedding approach we present 182
here is most closely related to Naive Bayes word 183
sense induction and topic modeling (Charniak et al., 184
2013; Lau et al., 2014). This approach learns word 185
meanings from local contexts by considering each 186
instance of the word in a document as a pseudo- 187


key: chroma:document
string_value:
document. However, the approach is not scalable 188
because it requires training a single topic per target 189
word. Our approach, on the other hand, is scalable 190
and builds non-linear (non-naive) logical embed- 191
dings that capture word compositions. To build the 192
logical embeddings, we propose a novel human- 193
interpretable algorithm based on the TM that pro- 194
vides logical rules describing contexts. The TM 195
has recently performed competitively with other 196


key: chroma:document
string_value:
deep learning techniques in many NLP tasks, in- 197
cluding novelty detection (Bhattarai et al., 2022a,c), 198
sentiment analysis (Abeyrathna et al., 2023; Yadav 199
et al., 2021), knowledge representation (Bhattarai 200
et al., 2023), and fake news detection (Bhattarai 201
et al., 2022b). Furthermore, the local and global 202
interpretability of TMs have been explored through 203
direct manipulation of the logical rules (Blakely 204
and Granmo, 2021). In addition, TM has been 205


key: chroma:document
string_value:
and Granmo, 2021). In addition, TM has been 205
shown to be hardware-friendly for low-power IoT 206
devices (Maheshwari et al., 2023). 207
3 Tsetlin Machine Autoencoder 208
We here detail the TM Autoencoder based on the 209
Coalesced TM (Glimsdal and Granmo, 2021), ex- 210
tended with input masking and freezing of masked 211
variables. For ease of explanation, we use three 212
inputs. Adding more inputs follows trivially. 213
3.1 Architecture 214
Input and Output. As seen in Figure 1, the 215


key: chroma:document
string_value:
Input and Output. As seen in Figure 1, the 215
TM Autoencoder digests and outputs propositional 216values: (x1, x2, x3)∈ {0,1}3→(bx1,bx2,bx3)∈ 217
{0,1}3. For our purposes, the propositional vari- 218
ables x1,x2, andx3each represent a word, for ex- 219
ample, “Brilliant,” “Actor,” and “Awful.” The value 220
1means that the word occurs in the input text, while 221
the value 0means that it does not. I.e., we repre- 222
sent natural language text as a set of words . Notice 223


key: chroma:document
string_value:
also that the input variables have corresponding 224
output variables bx1,bx2, andbx3. In short, bx1is to 225
be predicted from x2andx3,bx2from x1andx3, 226
and so on. Continuing our example, bx1predicts 227
the presence of “Brilliant” based on knowing the 228
occurrence of “Actor” and “Awful.” 229
Clause Pool. A pool of nconjunctive clauses, 230
denoted Cj, j∈ {1,2, . . . , n },encodes the input 231
in order to predict the output. A conjunctive clause 232


key: chroma:document
string_value:
Cjis simply an And-expression over a given subset 233
Lj⊆ {x1, x2, x3}of the input (our autoencoder 234
does not use the input negations ¬x1,¬x2, and 235
¬x3): 236
Cj(x1, x2, x3) =^
xk∈Ljxk. (1) 237
For example, the input subset L1={x1, x2}gives 238
the clause C1(x1, x2, x3) =x1∧x2in the figure. 239
This clause matches the input if x1andx2both 240
are1. In our example, the clause accordingly en- 241
codes the concept “Brilliant Actor”. 242
Weights. An integer weight matrix Wconnects 243


key: chroma:document
string_value:
Weights. An integer weight matrix Wconnects 243
each of the nclauses to the three outputs bx1,bx2, 244
andbx3: 245
W=
w11···w1n
w21···w2n
w31···w3n
∈Z3×n. (2) 246
The row index is an output while the column index 247
is a clause. The weight w12, for instance, connects 248
outputbx1to clause C2. In Figure 1, six weights 249
connect two clauses and three outputs: 250

+4−5
+1 +2
−7 +6
. (3) 251
Consider, for example, the weights (+4,−5)of 252


key: chroma:document
string_value:
Consider, for example, the weights (+4,−5)of 252
outputbx1in the figure. The weight +4states that 253
clause C1(x1, x2, x3) =x1∧x2favoursbx1being 254
1, while clause C2(x1, x2, x3) =x2∧x3opposes it. 255
For example, the concept “Awful Actor” opposes 256
output “Brilliant.” 257
3


key: chroma:document
string_value:
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten 4 
 
3 
 
2 
 
1Figure 2: Tsetlin Machine memory for single clause.
3.2 Inference 258
Let us consider the prediction of bx1first. The au- 259
toencoder predicts bx1from the clauses and weights: 260
bx1= 0≤nX
j=1w1jCj(1, x2, x3). (4) 261
That is, each clause Cjis multiplied by its weight 262
w1jfor output bx1. The outcomes are then summed 263
up to decide the output. If the sum is larger than or 264


key: chroma:document
string_value:
equal to zero, the output is bx1= 1. Otherwise, it is 265
bx1= 0. Clauses with positive weight thus promote 266
outputbx1= 1while clauses with negative weight 267
encourage bx1= 0. Notice that x1is masked by 268
replacing it with value 1. Accordingly, the autoen- 269
coder infers output bx1from the remaining inputs 270
x2andx3. 271
Correspondingly, bx2andbx3are calculated by 272
respectively masking x2andx3: 273
bx2= 0≤nX
j=1w2jCj(x1,1, x3), (5) 274
bx3= 0≤nX
j=1w3jCj(x1, x2,1). (6) 275


key: chroma:document
string_value:
bx3= 0≤nX
j=1w3jCj(x1, x2,1). (6) 275
Example. Assume that the input is always either 276
(1,1,0)or(0,1,1). The input (1,1,0)could for 277
instance represent “Brilliant Actor” and (0,1,1) 278
“Awful Actor.” Then notice how Eq. (4) correctly 279
determines the masked input x1with output bx1in 280
Figure 1, both for input (1,1,0)and(0,1,1). 281
3.3 Learning 282
We next consider how to learn the variable subsets 283
Ljfor the clauses Cj, j∈ {1,2, . . . , n },as well 284


key: chroma:document
string_value:
as how to determine the weights wijof the weight 285
matrix W. 286
Clause Memory. Each clause Cjhas a graded 287
memory that contains the input variables, shown 288
in Figure 2. The graded memory enables incre- 289
mental learning of the variable subsets from data. 290Observe how each variable is in one of four mem- 291
ory positions (the number of memory positions is 292
a user-configurable parameter). Positions 1−2 293
means Forgotten. Positions 3−4means Memo- 294


key: chroma:document
string_value:
means Forgotten. Positions 3−4means Memo- 294
rized. Memorized variables take part in the clause, 295
while Forgotten ones do not. The memory in Figure 296
2 thus gives the clause Cj(x1, x2, x3) =x1∧x2. 297
Learning Step. The TM Autoencoder learns 298
incrementally using three kinds of memory 299
and weight updates: Type Ia, Type Ib, and 300
Type II. Each training example has the form 301
[k,(x1, x2, x3), xk],1≤k≤3. The first element 302
is an index that identifies which input to mask and 303


key: chroma:document
string_value:
which output to predict. The second element is an 304
input vector (x1, x2, x3)and the third element is 305
the target value for output bxk, which is xk. We 306
describe the update procedure step-by-step below 307
for index 1examples (output bx1prediction). The 308
update procedure for bx2andbx3follows trivially. 309
Clause Update Probability. First, we calculate 310
the weighted clause sum for bx1from Eqn. (4): 311
v1=Pn
j=1w1jCj(1, x2, x3). The sum is then 312


key: chroma:document
string_value:
v1=Pn
j=1w1jCj(1, x2, x3). The sum is then 312
compared with a margin T(hyper-parameter) to 313
calculate a summation error ϵ. The error depends 314
on the x1-value: 315
ϵ=(
T−clip(v1,−T, T), x 1= 1,
T+ clip( v1,−T, T), x 1= 0.(7) 316
That is, for x1-value 1the weighted clause sum 317
should become T, while for x1-value 0the sum 318
should become −T. The goal of the learning is 319
thus to reach the margin for all inputs (x1, x2, x3), 320
ensuring correct output from Eqn. (4). To reach 321


key: chroma:document
string_value:
this goal, each clause Cjis updated randomly with 322
probabilityϵ
2Tin each round. In other words, the 323
update probability drops with the error toward zero. 324
Update Types. The kind of update depends on 325
the values of x1,Cj(1, x2, x3), and w1j. We first 326
consider clauses with positive weight, w1j≥0. 327
According to Eqn. 4, they are to recognize pat- 328
terns for x1= 1. Note that in all of the below 329
updates, the masked variable x1is frozen, leaving 330


key: chroma:document
string_value:
it unaffected by the update. 331
•Type Ia (Recognize) Feedback occurs when 332
x1= 1 andCj(1, x2, x3) = 1 . Then one 333
can say that Cj(1, x2, x3) = 1 is atrue posi- 334
tivebecause it correctly predicts the masked 335
x1-value. The Type Ia feedback reinforces 336
this successful match by updating the memory 337
4


key: chroma:document
string_value:
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1Figure 3: Type Ia (Recognize) Feedback for input
(1,1,0). The masked variable x1is frozen.
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1
Figure 4: Type Ib (Erase) Feedback for input (0,0,1).
The masked variable x1is frozen.
ofCjto further mimic the input (see Figure 338
3). That is, 1-valued variables move one step 339
upwards in memory, with probability 1.0.2340


key: chroma:document
string_value:
upwards in memory, with probability 1.0.2340
Conversely, 0-valued inputs move one step 341
downwards, however, randomly with proba- 342
bility1
s. Here, sis a hyperparameter called 343
specificity , meaning that a larger smakes the 344
clauses more specific (Zhang et al., 2022). 345
The clause overall is also reinforced by in- 346
crementing its weight wj1by1. 347
•Type Ib (Erase) Feedback occurs when x1= 348
1andCj(1, x2, x3) = 0 . Then we call 349
Cj(1, x2, x3) = 0 afalse negative because 350


key: chroma:document
string_value:
Cj(1, x2, x3) = 0 afalse negative because 350
it fails to promote x1= 1. In that case, all 351
inputs randomly move one step downwards 352
in memory (see Figure 4). Again, each down- 353
ward move happens with probability1
s. Here, 354
the purpose is to eliminate the false negative 355
outcome by erasing variables from the clause. 356
2Originally, the increment probability iss−1
s, which can
be boosted to 1.0to enhance the learning of true positive
patterns (Granmo, 2018).
Maximally
Memorized


key: chroma:document
string_value:
patterns (Granmo, 2018).
Maximally
Memorized
Maximally
ForgottenMemorized
Forgotten
 4 
 
3 
 
2 
 
1
Figure 5: Type II (Reject) Feedback for input (0,1,0).
The masked variable x1is frozen.•Type II (Reject) Feedback occurs when 357
x1= 0 andCj(1, x2, x3) = 1 . Then, one 358
can say that Cj(1, x2) = 1 is afalse positive 359
because it promotes x1= 1when in fact we 360
havex1= 0. Then all Forgotten 0-valued in- 361
puts move one step upwards in memory. The 362


key: chroma:document
string_value:
puts move one step upwards in memory. The 362
purpose is to eventually eliminate the current 363
false positive outcome by injecting 0-valued 364
variables into the clause. The clause is further 365
diminished by decrementing its weight w1jby 366
1. Note that the latter decrement can switch 367
the weight from positive to negative. In effect, 368
the clause then changes role, now training to 369
recognize x1= 0instead. 370
Clauses Cjwith negative weights, w1j<0, are 371


key: chroma:document
string_value:
Clauses Cjwith negative weights, w1j<0, are 371
updated the same way. However, they are to recog- 372
nize patterns for x1= 0. To achieve this, x1= 0is 373
treated as x1= 1andx1= 1is treated as x1= 0 374
when updating the memories. Furthermore, the 375
weight updates are reversed. Increments become 376
decrements and vice versa. 377
Algorithm 1 TM word embedding
Require: V ocabulary V; Documents D ∈ G ,D ⊆ V ; Accu-
mulation u; Clauses n; Margin T; Specificity s; Rounds
r


key: chroma:document
string_value:
r
1: TMCreate (n, T, s ) ▷Create TM with nclauses.
2:forrrounds do
3: forword k∈ Vdo▷Create one example per word.
4: qk←Select( {0,1}) ▷Random target value.
5: ifqk= 1then
6: Gk← {D| word k∈ D,D ∈ G} ▷
Documents with word k.
7: else
8: Gk← {D| word k/∈ D,D ∈ G} ▷
Documents without word k.
9: Sk←SelectN( Gk, u)▷Random subset of size
u.
10: Uk←S
D∈S kD▷Union of selected documents.
11: xk ← (x1, x2, . . . , x m), xi =(
1,word i∈ Uk
0,word i/∈ Uk
12: TMUpdate( k,xk, qk) ▷Update


key: chroma:document
string_value:
0,word i/∈ Uk
12: TMUpdate( k,xk, qk) ▷Update
TM Autoencoder for output index k, input xk, and target
valuebxk=qk.
13:C,W←TMGetState() ▷Clauses Cj∈ Cwith
weights W.
14:E←clip(W,0, T) ▷Elementwise clip of negative
values produces weighted logical word embeddings.
15:B←(W>0)▷Elementwise comparison with zero
produces purely logical word embeddings.
16:return C,E,B
4 Logical Embedding Procedure 378
We now use the TM Autoencoder to 379
build logical embeddings. Let V = 380
5


key: chroma:document
string_value:
RG65 0.72 0.58 0.89 0.67 0.49 0.88 0.75 0.63 0.92 0.78 0.62 0.93
Average 0.58 0.42 0.85 0.54 0.38 0.88 0.52 0.38 0.89 0.57 0.42 0.90
Table 1: Performance comparison of TM embedding with baseline algorithms on the similarity task.
{word 1,word 2, . . . , word m}be the target 381
vocabulary consisting of munique words. 382
Pre-processing. The first step is to pre-process 383
the document corpus. To this end, each document 384
is represented by a subset of words D ⊆ V . For 385


key: chroma:document
string_value:
example, the document “The actor was brilliant” 386
becomes the set D={“actor”, “brilliant”, “the”, 387
“was”}. The set G, in turn, contains all the doc- 388
uments, D ∈ G . Finally, in propositional vector 389
form, the word set Dbecomes: 390
x= (x1, x2, . . . , x t), xi=(
1,word i∈D,
0,word i/∈D.
(8) 391
Embedding. Algorithm 1 specifies the procedure 392
for embedding the mvocabulary words from Vby 393
using nclauses, Cj,1≤j≤n, forming a clause 394


key: chroma:document
string_value:
using nclauses, Cj,1≤j≤n, forming a clause 394
setC. Each round of training produces a training 395
example [k,(x1, x2, . . . , x m), qk]perword kinV. 396
First, a target value qkfor the word is set randomly 397
to either 0or1. This random selection balances 398
the dataset. If qkbecomes 1, we randomly select 399
udocuments that contain word kand assign them 400
to the set Sk(positive examples). Otherwise, we 401
randomly select udocuments that do notcontain 402


key: chroma:document
string_value:
randomly select udocuments that do notcontain 402
the word (negative examples). Next, the randomly 403
selected documents are merged by ORing them to- 404
gether, yielding the unified document Uk. The pur- 405
pose of ORing multiple documents is to increase 406
the frequency of rare context words. Then, picking 407
up characteristic ones becomes easier. After that, 408
the propositional vector form (x1, x2, . . . , x m)of 409
Ukis obtained. Finally, the TM Autoencoder is up- 410


key: chroma:document
string_value:
dated with [k,(x1, x2, . . . , x m), qk]following the 411
training procedure in Section 3. 412
Vector Space Representation. The weighted 413
logical embedding of word k∈ V can now be ob- 414
tained from row kof a matrix E(returned from 415
Algorithm 1), while the purely logical embedding 416
is found in row kof the matrix B. Letekdenote 417
thek’th row of E, and let eldenote the l’th row. 418
We can then compare the similarity of two words 419Dataset W2V FastText TM GloVe
AP 0.50 0.35 0.41 0.41


key: chroma:document
string_value:
AP 0.50 0.35 0.41 0.41
BLESS 0.64 0.66 0.62 0.66
ESSLI 0.63 0.60 0.57 0.56
Average 0.59 0.54 0.53 0.54
Table 2: Performance comparison of TM embedding
with baseline embeddings on the categorization task.
word kandword lusing cosine similarity (CS) be- 420
tween their E-embedding: 421
CS(word k,word l) =ek·el
||ek|| ||el||. (9) 422
5 Empirical Evaluation 423
We here evaluate our logical embedding scheme, 424
comparing it with neural network approaches. The 425


key: chroma:document
string_value:
details of the evaluation and training setup are pro- 426
vided in the appendix. 427
DatasetGloVe TM TM hybrid
Acc. F1 Acc. F1 Acc. F1
R8 96.31 0.88 96.10 0.88 97.80 0.94
TREC 95.20 0.95 96.40 0.96 96.80 0.96
R52 90.34 0.58 91.23 0.62 94.23 0.68
SUBJ 86.20 0.86 85.80 0.85 86.70 0.87
SST-2 76.38 0.75 75.61 0.74 79.30 0.78
SST-5 47.47 0.46 47.80 0.43 49.75 0.44
Table 3: Performance comparison of our embedding
with standard GloVe embedding on the classification
task.
5.1 Results and Discussion 428


key: chroma:document
string_value:
task.
5.1 Results and Discussion 428
We employ two kinds of evaluation: intrinsic and 429
extrinsic. Table 1 contains the intrinsic evalu- 430
ation results from the six-word similarity tasks. 431
We here compute the Spearman correlation, the 432
Kendall coefficient, and the cosine similarity be- 433
tween the human-set similarity scores and the pre- 434
dicted similarity scores per dataset. Considering 435
Spearman and Kendall score, Word2Vec and GloVe 436


key: chroma:document
string_value:
are marginally better than the comparable FastText 437
and TM embedding. However, as reported in (Ras- 438
togi et al., 2015), small differences in correlation- 439
based measures are not necessarily significant for 440
6


key: chroma:document
string_value:
Target words ClausesClauses
 student
surgery
heart
baseball
football
queen
princess
coffee
tea
 Figure 6: Interpretability of clauses capturing distinct meanings of target words in the TM embedding.
Figure 7: TM embedding visualization plotted using
t-SNE.
smaller datasets. To more robustly assess perfor- 441
mance, we therefore also use cosine similarity 442
to compare predicted word similarities with the 443
human-set similarities. In terms of cosine score, 444


key: chroma:document
string_value:
our model outperforms Word2Vec and FastText 445
on the majority of the datasets, while performing 446
competitively with GloVe. This means that the 447
angles between the human-set similarities and the 448
GloVe/TM-predicted similarities are quite similar. 449
Finally, Table 2 shows the outcome for the word 450
categorization tasks. As seen, the performance of 451
the selected embedding techniques are comparable, 452
with Word2Vec being slightly ahead. 453


key: chroma:document
string_value:
with Word2Vec being slightly ahead. 453
Previous research indicates that intrinsic word 454
similarity performance is minimally or even neg- 455
atively correlated with downstream NLP perfor- 456
mance (Wang et al., 2021). Therefore, we also in- 457
clude an extrinsic evaluation with six downstream 458
classification tasks. To avoid overfitting and ro- 459bustly assess downstream properties, we keep our 460
experimental setup from above. Table 3 reports 461


key: chroma:document
string_value:
the outcome of the evaluation, where the embed- 462
dings have been fed to an attention-based BiLSTM 463
model. The first configuration (GloVe) uses the 464
pre-trained GloVe embeddings from the Wikipedia 465
2014 + Gigaword 5 datasets. The second configu- 466
ration consists of our purely logical TM embedding 467
from One Billion Word (embedding Bfrom Algo- 468
rithm 1). Being five times smaller, the One Billion 469
Word dataset only provides about 80percent of the 470


key: chroma:document
string_value:
vocabulary required for the classification tasks. We 471
embed the remaining 20percent of the words ran- 472
domly. Hence, the TM approach can potentially 473
have a disadvantage in the evaluation. In the third 474
configuration (TM hybrid ), we replace the 20per- 475
cent random embeddings with the corresponding 476
GloVe embeddings (approximately 80% TM + 20% 477
GloVe). We note that the downstream accuracy of 478
BiLSTM is similar for both TM and GloVe. Specifi- 479


key: chroma:document
string_value:
cally, the TM embedding exceeds GloVe by a small 480
margin on TREC, R52, and SST-5. The hybrid em- 481
bedding, on the other hand, clearly outperforms the 482
other two. In particular, for R52, SST-2, and SST-5, 483
the hybrid embedding is able to surpass GloVe by 484
a substantial margin of roughly 2−4%. Given that 485
the datasets are not completely balanced, we also 486
compute F1 macro scores. We again observe that 487
the TM embedding either outperforms or is com- 488


key: chroma:document
string_value:
petitive with GloVe. For R8 and R52, the hybrid 489
embedding surpasses GloVe by a large margin, re- 490
spectively by around 6%and10%. Based on these 491
results, we conjecture that logical TM embedding 492
can successfully replace neural network embedding. 493
Even with 20% of the vocabulary missing, trained 494
on five times smaller data, the logical embedding 495
performs competitively with GloVe. Interestingly, 496
7


key: chroma:document
string_value:
the hybrid approach performed even better. One 497
possible explanation for this higher performance 498
can be the extra information added by the larger 499
vocabulary. Additionally, there may be synergy 500
between the neural and logical representations that 501
manifest in the hybrid approach. 502
5.2 Interpretability and Visualization 503
In this section, we investigate the nature of the 504
TM embeddings in more detail, focusing on inter- 505


key: chroma:document
string_value:
pretability. Our embedding consists of the positive 506
clause weights E, or, alternatively, the proposi- 507
tional version B, explained by the set of clauses 508
C. As demonstrated in Figure 6, each clause in C 509
captures a facet of a context. The dotted lines in the 510
figure showcase the connection between the target 511
words and their clauses from matrix B(and, ac- 512
cordingly, E). Each target word gets its own color 513
to more easily discern the connections. In the fig- 514


key: chroma:document
string_value:
ure, we provide an excerpt of 18connections from 515
B, involving 8target words and the 11most trig- 516
gered clauses for these words. Consider for exam- 517
ple the target words surgery andheart . These two 518
target words share two clauses: [ went∧hospital ] 519
and [ old∧disease ∧patient ]. The two clauses 520
capture two joint contexts, both related to health. 521
The clauses thus represent commonality between 522
the target words, providing information on one par- 523


key: chroma:document
string_value:
ticular meaning of the words. 524
The two target words are also semantically dif- 525
ferent. The differences are captured by the clauses 526
they do not share. The target word heart , for exam- 527
ple, also relates to the meaning [ woman ∧love], 528
which surgery does not. Surgery , on the other hand, 529
connects with [ injury ∧game ∧racing ]. In 530
this manner, the unique meanings and relations be- 531
tween words are represented through the sharing 532


key: chroma:document
string_value:
of logical expressions. Accordingly, it is feasible 533
to capture a wide range of possible contextual rep- 534
resentations with concise logical expressions. As 535
such, the logical embedding provides a sparse rep- 536
resentation of words and their relations. Indeed, at 537
most 10% of the clauses connect to each word in 538
our experiments. As shown in the intrinsic evalua- 539
tions from the previous subsection, these contextual 540
representations are effective for measuring word 541


key: chroma:document
string_value:
similarity and categorizing words. Similarly, we 542
observed that the logical embedding is boosting 543
downstream NLP classification tasks. 544
To cast further light on the TM embedding ap- 545
proach, we visualize the embedding of 400words 546
from the SimLex-999 dataset in Figure 7, plot- 547ted using t-SNE. The figure indicates that we are 548
able to cluster contextually similar words in vec- 549
tor space. To scrutinize the clusters, we zoom in 550


key: chroma:document
string_value:
on two of them. Consider the upper right cluster 551
first. Notice how the words in the cluster relate 552
tohospital , such as heart anddiseases . As seen, 553
the word embeddings are closely located in vector 554
space. Similarly, we can observe that terminology 555
connected to weather and geography are grouped 556
together in the bottom cluster. From these two 557
examples, it seems clear that the TM embedding 558
incorporates semantic relationships among words. 559


key: chroma:document
string_value:
6 Conclusion and Future Work 560
In this work, we first discussed the challenge and 561
necessity of finding computationally simpler and 562
more interpretable word embedding approaches. 563
We then motivated an efficient self-supervised ap- 564
proach, namely, a TM-based autoencoder, for pro- 565
ducing sparse and interpretable logical word em- 566
beddings. We evaluated our approach on a wide 567
range of intrinsic and extrinsic tasks, demonstrating 568


key: chroma:document
string_value:
that it is competitive with dense neural network- 569
based embedding schemes such as Word2Vec, 570
GloVe, and FastText. Further, we investigated the 571
interpretability of our embedding through visual- 572
ization and a case study. Our conclusion from the 573
study is that logical embedding is able to repre- 574
sent words with logical expressions. This structure 575
makes the representation sparse, enabling a clear- 576
cut decomposition of each word into sets of seman- 577


key: chroma:document
string_value:
tic concepts. Future work includes scaling up our 578
implementation using GPUs to support the build- 579
ing of large-scale vocabularies from more massive 580
datasets. 581
7 Limitations 582
The primary purpose of the conducted experiments 583
in the context of the downstream classification task 584
is to thoroughly analyze and comprehend the prac- 585
tical implementation of our embedding approach. 586
Consequently, the evaluation did not involve a com- 587


key: chroma:document
string_value:
parison of performance against other contemporary 588
transformer-based large language models, such as 589
BERT, which are considered the state of the art. 590
Further, we intend to investigate how sentence- 591
level and document-level embedding can be created 592
using clauses, for instance, applicable for down- 593
stream sentence similarity tasks. 594
8


key: chroma:document
string_value:
References 595
K. Darshana Abeyrathna, Ahmed Abdulrahem Oth- 596
man Abouzeid, Bimal Bhattarai, Charul Giri, Sondre 597
Glimsdal, Ole-Christoffer Granmo, Lei Jiao, Rupsa 598
Saha, Jivitesh Sharma, Svein Anders Tunheim, and 599
Xuan Zhang. 2023. Building concise logical patterns 600
by constraining tsetlin machine clause size. In In- 601
ternational joint conference on artificial intelligence 602
(IJCAI) . 603
Dimo Angelov. 2020. Top2vec: Distributed representa- 604


key: chroma:document
string_value:
tions of topics. arXiv preprint arXiv:2008.09470 . 605
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, 606
and Andrej Risteski. 2018. Linear algebraic struc- 607
ture of word senses, with applications to polysemy. 608
Transactions of the Association for Computational 609
Linguistics , 6:483–495. 610
Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A 611
simple but tough-to-beat baseline for sentence em- 612
beddings. In Proceedings of ICLR . 613


key: chroma:document
string_value:
beddings. In Proceedings of ICLR . 613
Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. 614
2000. A neural probabilistic language model. Ad- 615
vances in neural information processing systems , 13. 616
Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. 617
2022a. A Tsetlin Machine Framework for Universal 618
Outlier and Novelty Detection. In International Con- 619
ference on Agents and Artificial Intelligence , pages 620
250–268. Springer. 621


key: chroma:document
string_value:
250–268. Springer. 621
Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. 622
2022b. Explainable Tsetlin Machine framework for 623
fake news detection with credibility score assessment. 624
InLREC . 625
Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. 626
2022c. Word-level human interpretable scoring 627
mechanism for novel text detection using tsetlin ma- 628
chines. Applied Intelligence . 629
Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. 630


key: chroma:document
string_value:
2023. An interpretable knowledge representation 631
framework for natural language processing with 632
cross-domain application. In Advances in Informa- 633
tion Retrieval: 45th European Conference on Infor- 634
mation Retrieval, ECIR 2023, Dublin, Ireland, April 635
2–6, 2023, Proceedings, Part I , pages 167–181. 636
Christian D. Blakely and Ole-Christoffer Granmo. 2021. 637
Closed-Form Expressions for Global and Local Inter- 638
pretation of Tsetlin Machines. In 34th International 639


key: chroma:document
string_value:
Conference on Industrial, Engineering and Other Ap- 640
plications of Applied Intelligent Systems (IEA/AIE 641
2021) . Springer. 642
Piotr Bojanowski, Edouard Grave, Armand Joulin, and 643
Tomas Mikolov. 2017. Enriching word vectors with 644
subword information. Transactions of the associa- 645
tion for computational linguistics , 5:135–146. 646
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- 647
mann, Trevor Cai, Eliza Rutherford, Katie Milli- 648


key: chroma:document
string_value:
can, George Bm Van Den Driessche, Jean-Baptiste 649Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. 650
Improving language models by retrieving from tril- 651
lions of tokens. In Proceedings of ICML , pages 2206– 652
2240. PMLR. 653
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, 654
Nicole Limtiaco, Rhomni St John, Noah Constant, 655
Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, 656
et al. 2018. Universal sentence encoder for English. 657
InProceedings of EMNLP , pages 169–174. 658


key: chroma:document
string_value:
InProceedings of EMNLP , pages 169–174. 658
Eugene Charniak et al. 2013. Naive Bayes word sense 659
induction. In Proceedings of EMNLP , pages 1433– 660
1437. 661
Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, 662
T. Brants, Phillip Todd Koehn, and Tony Robinson. 663
2014. One billion word benchmark for measuring 664
progress in statistical language modeling. In Inter- 665
speech . 666
Alexis Conneau and Douwe Kiela. 2018. SentEval: An 667


key: chroma:document
string_value:
Evaluation Toolkit for Universal Sentence Represen- 668
tations. In Proceedings of LREC . 669
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and 670
Kristina Toutanova. 2019. BERT: Pre-training of 671
Deep Bidirectional Transformers for Language Un- 672
derstanding. In NAACL . 673
Paramveer S Dhillon, Dean P Foster, and Lyle H Ungar. 674
2015. Eigenwords: spectral word embeddings. J. 675
Mach. Learn. Res. , 16:3035–3078. 676
Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris 677


key: chroma:document
string_value:
Dyer, and Noah A Smith. 2015. Sparse Overcom- 678
plete Word Vector Representations. In Proceedings 679
of ACL, (Long Papers) , pages 1491–1500. 680
Sondre Glimsdal and Ole-Christoffer Granmo. 2021. 681
Coalesced Multi-Output Tsetlin Machines with 682
Clause Sharing. Available as an arXiv preprint, 683
arXiv:2108.07594 . 684
Ole-Christoffer Granmo. 2018. The tsetlin machine - 685
a game theoretic bandit driven approach to optimal 686
pattern recognition with propositional logic. arXiv 687


key: chroma:document
string_value:
preprint arXiv:1804.01508 . 688
Zellig S Harris. 1954. Distributional structure. Word , 689
10(2-3):146–162. 690
Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, 691
David Zhang, Philip Pronin, Janani Padmanab- 692
han, Giuseppe Ottaviano, and Linjun Yang. 2020. 693
Embedding-based retrieval in Facebook search. In 694
Proceedings of ACM SIGKDD , pages 2553–2561. 695
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina 696
Toutanova. 2019. Bert: Pre-training of deep bidirec- 697


key: chroma:document
string_value:
tional transformers for language understanding. In 698
Proceedings of NAACL-HLT , pages 4171–4186. 699
Guillaume Lample, Miguel Ballesteros, Sandeep Sub- 700
ramanian, Kazuya Kawakami, and Chris Dyer. 2016. 701
Neural Architectures for Named Entity Recognition. 702
InProceedings of NAACL-HLT , pages 260–270. 703
9


key: chroma:document
string_value:
Jey Han Lau, Paul Cook, Diana McCarthy, Spandana 704
Gella, and Timothy Baldwin. 2014. Learning word 705
sense distributions, detecting unattested senses and 706
identifying novel senses using topic models. In Pro- 707
ceedings of ACL, (Long Papers) , pages 259–270. 708
Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Im- 709
proving distributional similarity with lessons learned 710
from word embeddings. Transactions of the associa- 711
tion for computational linguistics , 3:211–225. 712


key: chroma:document
string_value:
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, 713
Yiming Yang, and Lei Li. 2020. On the Sentence 714
Embeddings from Pre-trained Language Models. In 715
Proceedings of EMNLP , pages 9119–9130. 716
Lajanugen Logeswaran and Honglak Lee. 2018. An 717
efficient framework for learning sentence representa- 718
tions. In Proceedings of ICLR . 719
Sidharth Maheshwari, Tousif Rahman, Alex Yakovlev, 720
Ashur Rafiev, Lei Jiao, Ole-Christoffer Granmo, et al. 721


key: chroma:document
string_value:
2023. REDRESS: Generating compressed models 722
for edge inference using Tsetlin machines. IEEE 723
Transactions on Pattern Analysis and Machine Intel- 724
ligence . 725
Tomas Mikolov, Kai Chen, Gregory S. Corrado, and 726
Jeffrey Dean. 2013. Efficient Estimation of Word 727
Representations in Vector Space. In Proceedings of 728
ICLR . 729
Jiaqi Mu, Suma Bhat, and Pramod Viswanath. 2017. 730
Geometry of polysemy. In Proceedings of ICLR . 731


key: chroma:document
string_value:
Jeffrey Pennington, Richard Socher, and Christopher D 732
Manning. 2014. Glove: Global vectors for word 733
representation. In Proceedings of EMNLP , pages 734
1532–1543. 735
Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt 736
Gardner, Christopher Clark, Kenton Lee, and Luke 737
Zettlemoyer. 2018. Deep contextualized word repre- 738
sentations. arXiv preprint arXiv:1802.05365 . 739
Pushpendre Rastogi, Benjamin Van Durme, and Raman 740
Arora. 2015. Multiview LSA: Representation learn- 741


key: chroma:document
string_value:
ing via generalized CCA. In Proceedings of NAACL , 742
pages 556–566. 743
Nils Reimers and Iryna Gurevych. 2019. Sentence- 744
BERT: Sentence Embeddings using Siamese BERT- 745
Networks. In Proceedings of EMNLP-IJCNLP , 746
pages 3982–3992. 747
Roy Schwartz, Jesse Dodge, Noah Smith, and Oren 748
Etzioni. 2020. Green AI. Communications of the 749
ACM , 63:54 – 63. 750
Anant Subramanian, Danish Pruthi, Harsh Jhamtani, 751
Taylor Berg-Kirkpatrick, and Eduard Hovy. 2018. 752


key: chroma:document
string_value:
Spine: Sparse interpretable neural embeddings. In 753
Proceedings of AAAI , volume 32. 754Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob 755
Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz 756
Kaiser, and Illia Polosukhin. 2017. Attention is all 757
you need. Advances in neural information processing 758
systems , 30. 759
Kexin Wang, Nils Reimers, and Iryna Gurevych. 2021. 760
Tsdae: Using transformer-based sequential denoising 761
auto-encoder for unsupervised sentence embedding 762


key: chroma:document
string_value:
learning. arXiv preprint arXiv:2104.06979 . 763
Rohan Yadav, Lei Jiao, Ole-Christoffer Granmo, and 764
Morten Goodwin. 2021. Human-Level Interpretable 765
Learning for Aspect-Based Sentiment Analysis. In 766
Proceedings of AAAI . 767
Xuan Zhang, Lei Jiao, Ole-Christoffer Granmo, and 768
Morten Goodwin. 2022. On the Convergence of 769
Tsetlin Machines for the IDENTITY-and NOT Op- 770
erators. IEEE Transactions on Pattern Analysis and 771
Machine Intelligence , 44(10):6345–6359. 772


key: chroma:document
string_value:
Machine Intelligence , 44(10):6345–6359. 772
Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, 773
and Lidong Bing. 2020. An Unsupervised Sentence 774
Embedding Method by Mutual Information Maxi- 775
mization. In Proceedings of EMNLP , pages 1601– 776
1610. 777
A Appendix 778
Datasets and Setup We first evaluate our logical 779
embedding intrinsically, followed by an extrinsic 780
evaluation using classification tasks. 781
Intrinsic Evaluation. We use word similarity 782


key: chroma:document
string_value:
Intrinsic Evaluation. We use word similarity 782
and categorization benchmarks for intrinsic eval- 783
uation. That is, we examine to what degree our 784
approach retains semantic word relations. To this 785
end, we measure how semantic relations manifest 786
in vector space using six datasets: SimLex-999, 787
WordSim-353, MEN, MTurk-287, Mturk-771, and 788
RG-65. Each dataset consists of human-scored 789
word pairs, which are compared with the corre- 790


key: chroma:document
string_value:
sponding vector space similarities. The catego- 791
rization tasks evaluate how well we can group 792
words into distinct word categories, only based 793
on their embedding. We here use three datasets: 794
AP, BLESS, and ESSLLI.3As baselines, we chose 795
Word2Vec, GloVe, and FastText because of their 796
wide use. 797
Extrinsic Evaluation. In our extrinsic evalua- 798
tion, we investigate how well our logical embed- 799
ding supports downstream NLP classification tasks. 800


key: chroma:document
string_value:
Using the word embeddings as feature vectors, the 801
3To obtain the categorization accuracy, we use KMeans
clustering from sklearn on the word embeddings and exam-
ine the cluster quality by calculating the purity score from
(https://github.com/purity).
10


key: chroma:document
string_value:
performance of supervised classification models 802
gives insight into the embedding quality. We em- 803
ploy six standard text classification datasets from 804
SentEval (Conneau and Kiela, 2018): R8, R52, 805
TREC, SUBJ, SST-2, and SST-5. For supervised 806
learning, we use the standard attention-based BiL- 807
STM model with the Adam optimizer and cross- 808
entropy loss function. In this manner, we directly 809
contrast GloVe embedding against the logical TM 810
approach. 811


key: chroma:document
string_value:
approach. 811
Embedding Datasets. For extrinsic evaluation 812
with BiLSTM, we use standard 300-dimensional 813
GloVe embeddings, pre-trained on the Wikipedia 814
2014 + Gigaword 5 datasets (6B tokens).4The pur- 815
pose is to compare the TM embedding performance 816
against widely used and successful GloVe embed- 817
dings on downstream tasks. To directly compare 818
the intrinsic properties of Word2Vec, GloVe, Fast- 819
Text, and TM embedding, we also train them from 820


key: chroma:document
string_value:
scratch using the One Billion Word dataset (Chelba 821
et al., 2014). For training the TM, we use r= 2000 822
training rounds, producing 2000 examples per 823
word by accumulating u= 25 contexts per exam- 824
ple. We use the following hyperparameters: a pool 825
ofn= 600 clauses, margin T= 1200 , and speci- 826
ficity s= 5.0.5Word2Vec Skip-Gram is trained 827
with 10passes over the data, using separated em- 828
beddings for the input and output contexts. The 829


key: chroma:document
string_value:
window size is 5and we use five negative samples 830
per example. Similarly, GloVe is trained for 30 831
epochs with a window size of 10and a learning 832
rate of 0.05.6833
4The pre-trained GloVe embeddings can be found here:
https://nlp.stanford.edu/projects/glove/
5The TM Autoencoder and logical word embedding im-
plementation can be found here: XXXX (Removed for
anonymity).
6Word2Vec and FastText have been trained using
the standard gensim library (https://github.com/RaRe-


key: chroma:document
string_value:
Technologies/gensim/tree/develop/gensim). GloVe has been
trained using https://github.com/maciejkula/glove-python.
11


key: chroma:document
string_value:
The Tsetlin Machine – A Game Theoretic Bandit Driven
Approach to Optimal Pattern Recognition with
Propositional Logic∗
Ole-Christoﬀer Granmo†
Abstract
Although simple individually, artiﬁcial neurons provide state-of-the-art performance
when interconnected in deep networks. Arguably, the Tsetlin Automaton is an even simpler
and more versatile learning mechanism, capable of solving the multi-armed bandit problem.


key: chroma:document
string_value:
Merely by means of a single integer as memory, it learns the optimal action in stochastic
environments through increment and decrement operations. In this paper, we introduce
the Tsetlin Machine, which solves complex pattern recognition problems with propositional
formulas, composed by a collective of Tsetlin Automata. To eliminate the longstanding
problem of vanishing signal-to-noise ratio, the Tsetlin Machine orchestrates the automata


key: chroma:document
string_value:
using a novel game. Further, both inputs, patterns, and outputs are expressed as bits, while
recognition and learning rely on bit manipulation, simplifying computation. Our theoret-
ical analysis establishes that the Nash equilibria of the game align with the propositional
formulas that provide optimal pattern recognition accuracy. This translates to learning
without local optima, only global ones. In ﬁve benchmarks, the Tsetlin Machine provides


key: chroma:document
string_value:
competitive accuracy compared with SVMs, Decision Trees, Random Forests, Naive Bayes
Classiﬁer, Logistic Regression, and Neural Networks. We further demonstrate how the
propositional formulas facilitate interpretation. In conclusion, we believe the combination
of high accuracy, interpretability, and computational simplicity makes the Tsetlin Machine
a promising tool for a wide range of domains.
Keywords: Bandit Problem, Game Theory, Interpretable Pattern Recognition, Propo-


key: chroma:document
string_value:
sitional Logic, Tsetlin Automata Games, Learning Automata, Frequent Pattern Mining,
Resource Allocation.
1 Introduction
Although simple individually, artiﬁcial neurons provide state-of-the-art performance when in-
terconnected in deep networks [1]. However, deep neural networks often require huge amounts
of training data and extensive computational resources. Unknown to many, there exists an
arguably even more fundamental and versatile learning mechanism than the artiﬁcial neuron,


key: chroma:document
string_value:
namely, the Tsetlin Automaton, developed by M.L. Tsetlin in the Soviet Union in the early
1960s [2]. In this paper, we propose a novel technique for large-scale and complex pattern
recognition based on Tsetlin Automata.
1.1 The Tsetlin Automaton
Tsetlin Automata have been used to model biological systems, and have attracted considerable
interest because they can learn the optimal action when operating in unknown stochastic


key: chroma:document
string_value:
environments [2, 3]. Furthermore, they combine rapid and accurate convergence with low
computational complexity.
∗Source code and datasets for this paper can be found at https://github.com/cair/TsetlinMachine and
https://github.com/cair/pyTsetlinMachine.
†Author’s status: Professor . The author can be contacted at: Centre for Artiﬁcial Intelligence Research
(https://cair.uia.no), University of Agder, Grimstad, Norway. E-mail: ole.granmo@uia.no
1arXiv:1804.01508v15  [cs.AI]  2 Jan 2021


key: chroma:document
string_value:
In all brevity, the Tsetlin Automaton is one of the pioneering solutions to the well-known
multi-armed bandit problem [4, 5] and the ﬁrst Learning Automaton [3]. It performs actions
αz,z∈{1,2}, sequentially in an environment. Each action αztriggers either a reward or a
penalty. That is, the action is rewarded with probability pz, otherwise it is penalized. The
reward probabilities are unknown to the automaton and may change over time. Under such


key: chroma:document
string_value:
challenging conditions, the goal is to identify the action with the highest reward probability
using as few attempts as possible.
12NN+2N+12NAction 1Action 2……RewardPenalty
Figure 1: A Tsetlin Automaton for two-action environments.
The mechanism driving the Tsetlin Automaton is surprisingly simple. Informally, as illus-
trated in Figure 1, a Tsetlin Automaton is simply a ﬁxed ﬁnite-state automaton [6] with an
unusual interpretation:


key: chroma:document
string_value:
unusual interpretation:
•The current state of the automaton decides which action to perform. The automaton in
the ﬁgure has 2 Nstates. Action 1 ( α1) is performed in the states with index 1 to N,
while Action 2 ( α2) is performed in the states with index N+ 1 to 2N.
•The state transitions of the automaton govern learning. One set of state transitions is
activated on reward (solid lines in the ﬁgure), and one set of state transitions is activated


key: chroma:document
string_value:
on penalty (dotted lines in the ﬁgure). As seen, rewards and penalties trigger speciﬁc
transitions from one state to another, designed to reinforce successful actions (those
eliciting rewards).
Formally, a Two-Action Tsetlin Automaton can be deﬁned as a quintuple [3]:
{Φ,α,β,F(·,·),G(·)}.
Φ={φ1,φ2,...,φ 2N}is the set of internal states. α={α1,α2}is the set of automaton actions.
β={βPenalty,βReward}is the set of inputs that can be given to the automaton. An output


key: chroma:document
string_value:
functionG(φu) determines the next action performed by the automaton, given the current
automaton state φu:
G(φu) ={
α1,if1≤u≤N
α2,ifN+ 1≤u≤2N.(1)
Finally, a transition function F(φu,βv) determines the new automaton state from: (i) the
current automaton state φuand (ii) the response βvof the environment to the action performed
by the automaton:
F(φu,βv) =

φu+1,if1≤u≤Nandv= Penalty
φu−1,ifN+ 1≤u≤2Nandv= Penalty
φu−1,if1<u≤Nandv= Reward
φu+1,ifN+ 1≤u<2Nandv= Reward
φu,otherwise.(2)


key: chroma:document
string_value:
Implementation-wise, a Tsetlin Automaton simply maintains an integer (the state index),
and learning is performed through increment and decrement operations, according to the tran-
sitions speciﬁed by F(φu,βv) (and depicted in Figure 1). The Tsetlin Automaton is thus
extremely simple computationally, with a very small memory footprint.
1.2 State-of-the-art in the Field of Learning Automata
Learning Automata have attracted considerable interest because they can learn the optimal


key: chroma:document
string_value:
action when operating in unknown stochastic environments [3]. As such, they have been used
for pattern classiﬁcation over more than four decades. Early work includes the stochastic
Learning Automata-based classiﬁer of Barto and Anandan [7], as well as the games of Learning
Automata proposed by Narendra and Thathachar [3]. These approaches can learn the optimal
classiﬁer for speciﬁc forms of discriminant functions (e.g., linear classiﬁers), also when feedback


key: chroma:document
string_value:
is noisy. Along the same lines, Sastry and Thathachar have provided several algorithms based
on cooperating systems of Learning Automata [8]. More recently, Zahiri proposed hyperplane-
and rule-based classiﬁers, which performed comparably to other well-known methods in the
literature [9, 10]. Recent research also includes the noise-tolerant learning algorithm by Sastry
et al., built upon a team of continuous-action Learning Automata [11]. Further, Goodwin et al.


key: chroma:document
string_value:
proposed a Learning Automata-guided random walk on a grid to construct robust discriminant
functions [12], while Motieghader et al. introduced a hybrid scheme that combines a genetic
algorithm with Learning Automata for classiﬁcation [13].
In general, however, previous Learning Automata schemes have mainly addressed relatively
small-scale pattern recognition tasks. The above solutions are further primarily based on so-


key: chroma:document
string_value:
called variable structure learning automata [14]. Although still simple, these are signiﬁcantly
more complex than the Tsetlin Automaton because they need to maintain an action probability
vector for sampling actions. Their success in pattern recognition is additionally limited by
constrained pattern representation capability (linearly separable classes and simple decision
trees).
The Tsetlin Automaton has formed the core of more advanced learning automata designs as


key: chroma:document
string_value:
well. This includes decentralized control [15], searching on the line [16], equi-partitioning [17],
streaming sampling for social activity networks [18], faulty dichotomous search [19], learning
in deceptive environments [20], and routing in telecommunication networks [21]. The strength
of these ﬁnite state learning automata solutions is that they have provided state-of-the-art
performance when problem properties are unknown and stochastic, while the problem must be


key: chroma:document
string_value:
solved as quickly as possible through trial and error.
1.3 The Vanishing Signal-to-Noise Ratio Problem
The ability to handle stochastic and unknown environments for a wide range of problems,
combined with their computational simplicity and small memory footprint, make Learning
Automata an attractive building block for complex machine learning tasks. However, a par-
ticularly adverse challenge has hindered the success of Learning Automata. Complex problem


key: chroma:document
string_value:
solving requires a team of interacting automata, and unfortunately, each team member in-
troduces noise. This challenge manifests sharply in the Goore Game, which employs a simple
uni-modal stochastic objective function to be optimized by a Learning Automata team [22, 15].
The challenge manifests as follows. Consider a team of W+ 1 two-action variable structure
Learning Automata. Each automaton maintains an action probability p. In any round of the


key: chroma:document
string_value:
Goore Game, an automaton performs its ﬁrst action with probability pand its second action
with probability 1 −p. Now, let µbe the average value with which a single automaton can
modify the game’s objective function by switching action. From the perspective of a single
automaton, each other automaton in the team then increases the objective function variance,
amounting to p(1−p)µ2[22]. In particular, consider the case of a deterministic objective


key: chroma:document
string_value:
from 0.5. Then the best-case signal-to-noise ratio (SNR) becomes:
SNR =µ2
σ2=µ2
Wp(1−p)µ2=1
Wp(1−p). (3)
That is, as the number of automata Wgrows, SNR drops to zero (vanishes).
For Tsetlin Automata, which are deterministic, the stochasticity of the action selection
arises from the automata changing state. These state changes are stochastic since they are
driven by stochastic feedback. Thus, again, as the number of automata grows, so does the


key: chroma:document
string_value:
variance of the feedback. Even for a simple uni-modal objective function, such as the one used
in the Goore Game, this eﬀect is problematic. Indeed, as explored by Kleinrock and Tung in
1996, a team of Tsetlin Automata solving the Goore Game needs to maintain an increasingly
larger state-space as the number of Tsetlin Automata grows, and in the end an inﬁnite number
of states per Tsetlin Automaton [15].
In general, this problem is inherent to decision-making with Learning Automata because


key: chroma:document
string_value:
of their decentralized and stochastic nature. The automata independently decide upon their
actions, directly based on the feedback from the environment. This is on one hand a strength
because it allows problems to be solved in a decentralized manner. On the other hand, as the
number of automata grows, the level of noise increases. In the following, we will refer to this
eﬀect as the vanishing signal-to-noise ratio problem .
0 0 * 1 * 0 0 0
* 0 * 1 * 0 0 0
0 * * 1 * * * 0
0 * * * * 0 0 *


key: chroma:document
string_value:
* 0 * 1 * 0 0 0
0 * * 1 * * * 0
0 * * * * 0 0 *
0 0 0 * * 0 0 0
0 * 0 * * * 0 0
0 0 * 1 * * * 0
0 0 0 * 1 * * *
Table 1: A bit pattern produced by the Tsetlin Machine for handwritten digits ’1’. The ’*’
symbol can either take the value ’0’ or ’1’. The remaining bit values require strict matching.
The pattern is relatively easy to interpret for humans compared to, e.g., the weights of a
neural network. It is also eﬃcient to evaluate for computers. Despite this simplicity, the


key: chroma:document
string_value:
Tsetlin Machine produces bit patterns that deliver competitive pattern recognition accuracy
for several datasets, as reported in Section 5.
1.4 Paper Contributions
In this paper, we attack the limited pattern expression capability and vanishing signal-to-noise
ratio of Learning Automata-based pattern recognition, introducing the Tsetlin Machine . The
contributions of the paper can be summarized as follows:
•We introduce the Tsetlin Machine, which solves complex pattern recognition problems


key: chroma:document
string_value:
with propositional formulas , composed by a collective of Tsetlin Automata.
•We eliminate the longstanding vanishing signal-to-noise ratio problem with a unique
decentralized learning scheme based on game theory [23, 2]. The game we have designed
allows millions of Tsetlin Automata to successfully cooperate.
•The game orchestrated by the Tsetlin Machine is based on resource allocation principles
[24], in inter-play with frequent pattern mining [25]. By allocating sparse pattern rep-


key: chroma:document
string_value:
resentation resources according to the frequency of the patterns, the Tsetlin Machine is
able to capture intricate unlabelled sub-patterns, for instance addressing the so-called
Noisy XOR-problem.
4


key: chroma:document
string_value:
•Our theoretical analysis establishes that the Nash equilibria of the Tsetlin Machine game
are aligned with the propositional formulas that provide optimal pattern recognition
accuracy. This translates to learning without local optima, only global ones.
•The propositional formulas are represented as bit patterns. These bit patterns are rela-
tively easy to interpret, compared to e.g. a neural network (see Table 1 for an example


key: chroma:document
string_value:
bit pattern). This facilitates human quality assurance and scrutiny, which for instance
can be important in safety-critical domains such as medicine.
•The Tsetlin Machine is a new approach to global construction of decision rules [26, 27].
We demonstrate that decision rules for large-scale pattern recognition can be learned
on-line, under particularly noisy conditions.
•The Tsetlin Machine is particularly suited for digital computers, being directly based on


key: chroma:document
string_value:
bit manipulation with AND-, OR-, and NOT operators.
•In our empirical evaluation on ﬁve datasets, the Tsetlin Machine provides competitive per-
formance in comparison with Multilayer Perceptron Networks, Support Vector Machines,
Decision Trees, Random Forests, the Naive Bayes Classiﬁer, and Logistic Regression.
•We demonstrate how the Tsetlin Machine can be used as a building block to create more
advanced architectures.


key: chroma:document
string_value:
advanced architectures.
In conclusion, we believe the combination of high accuracy, interpretability, and computational
simplicity makes the Tsetlin Machine a promising tool for a wide range of domains.
1.5 Paper Organization
The paper is organized as follows. In Section 2, we deﬁne the exact nature of the pattern
recognition problem we are going to solve, also introducing the crucial concept of sub-patterns.


key: chroma:document
string_value:
Then, in Section 3, we cover the Tsetlin Machine in detail. We ﬁrst present the Tsestlin
Machine inference structure, before we introduce the Tsetlin Automata teams that write for-
mulas in propositional logic. These Tsetlin Automata teams are in turn organized to recognize
complex patterns. We conclude the section by presenting the Tsetlin Machine game that we
use to coordinate millions of Tsetlin Automata, eliminating the vanishing signal-to-noise ratio
problem.


key: chroma:document
string_value:
problem.
In Section 4, we analyze pertinent properties of the Tsetlin Machine formally, and establish
that the Nash equilibria of the game are aligned with the propositional formulas that solve the
pattern recognition problem at hand. This allows the Tsetlin Machine as a whole to robustly
and accurately uncover complex patterns with propositional logic.
In our empirical evaluation in Section 5, we evaluate the performance of the Tsetlin Machine


key: chroma:document
string_value:
on ﬁve datasets: Flower categorization, digit recognition, board game planning, the Noisy XOR
Problem with Non-informative Features, as well as MNIST.
The Tsetlin Machine has been designed to act as a building block in more advanced ar-
chitectures, and in Section 6 we demonstrate how four distinct architectures can be built by
interconnecting multiple Tsetlin Machines.
As the ﬁrst step in a new research direction, the Tsetlin Machine also opens up a range of


key: chroma:document
string_value:
new research questions. In Section 7, we summarize our main ﬁndings and provide pointers to
some of the open research problems.
2 The Pattern Recognition Problem
The accuracy of a machine learning technique is bounded by its pattern representation capa-
bility. The Naive Bayes Classiﬁer, for instance, assumes that input variables are independent
given the output category. In general, compared to the representation capability of the underly-


key: chroma:document
string_value:
P(y = 0 | X ) > P(y = 1 | X )Sub-pattern X1
Sub-pattern X0Sub-pattern X0
Sub-pattern X1P(y = 0 | X ) <= P(y = 1 | X )1
21
2Figure 2: A partitioning of the input space according to the posterior probability of the output
variabley, highlighting distinct sub-patterns in the input space, X∈X. Sub-patterns most
likely belonging to output y= 1 can be found on the right side, while sub-patterns most likely
belonging to y= 0 on the left.


key: chroma:document
string_value:
belonging to y= 0 on the left.
appear somewhat limited, neural networks being one of the exceptions. With propositional
logic/Boolean algebra1as the starting point, we here deﬁne the pattern recognition problem
to be solved by the Tsetlin Machine.
Input and Output. Consider an input vector of opropositional variables: X= (x1,...,x o)∈
X,X={0,1}o. Together with their negated counterparts, ¯ xk=¬xk= 1−xk, the variables


key: chroma:document
string_value:
form a literal set L={l1,...,l 2o}={x1,...,x o,¯x1,..., ¯xo}. From the literals, we are to pro-
duce an output vector of mpropositional variables: Y= (y1,y2,...,ym)∈Y,Y={0,1}m.
For the sake of simple notation, we will in the remainder of the paper represent any particular
output variable yi,i= 1,...,m , as a single output variable y, without loss of generality.
Patterns. In Tsetlin Machine learning, a pattern is expressed as a conjunctive clause in


key: chroma:document
string_value:
propositional logic, denoted Cj, withjbeing the index of the clause. A conjunctive clause is
formed by ANDing a subset Lj⊆Lof the literal set:
Cj(X) =⋀
lk∈Ljlk. (4)
or, equivalently, as a product of the literals:
Cj(X) =∏
lk∈Ljlk. (5)
For example, the clause Cj(X) =x1∧x2=x1x2consists of the literals Lj={x1,x2}and
outputs 1 iﬀ x1=x2= 1.
Note that any formula in propositional logic can be reformulated as a disjunction of con-


key: chroma:document
string_value:
junctive clauses (referred to as disjunctive normal form). With oinput variables, one can
express no less than 22ounique propositional formulas, making conjunctive clauses a powerful
building block for expressing complex non-linear patterns. Arguably, they are also particularly
easy for humans to comprehend [28].
1We found the Tsetlin Machine on propositional logic, which can be mapped to Boolean algebra, and vice
versa.
6


key: chroma:document
string_value:
Learning Problem and Sub-Patterns. As illustrated in Figure 2, the input space Xcan
be partitioned into two parts, X1={X|P(y= 0|X)≤P(y= 1|X)}andX0={X|P(y=
0|X)>P(y= 1|X)}. For input vectors in X1, classiﬁcation accuracy is maximized by assigning
ythe value 1. Conversely, for input vectors in X0, accuracy is maximized by assigning ythe
value 0. Accordingly, the learning problem that we address consists of ﬁnding the partitioning
of the input space that maximizes classiﬁcation accuracy .


key: chroma:document
string_value:
We now come to the crucial concept of unlabelled sub-patterns. As Figure 2 exempliﬁes,
we decomposeX0andX1inton/2 sub-partitions each, respectively, X0
j,j∈{1,...,n/ 2},
andX1
j,j∈{1,...,n/ 2}. Each sub-partition Xω
j,ω∈{0,1},is deﬁned by a corresponding
conjunctive clause, containing all the inputs that make the clause evaluate to 1, and only those
inputs. Consider for instance the XOR-relation. Then X0decomposes intoX0
1={X|x1x2= 1}
andX0
2={X|¯x1¯x2= 1}, whileX1decomposes intoX1


key: chroma:document
string_value:
andX0
2={X|¯x1¯x2= 1}, whileX1decomposes intoX1
1={X|x1¯x2= 1}andX1
2={X|¯x1x2=
1}.
During pattern learning, however, we only observe training examples ( X,y) sampled from
an unknown input-output distribution P(X,y). Which sub-partition Xwas sampled from is
unavailable to us. What we know is that (i) each sub-partition can be described by a conjunctive
clause, and (ii) the probability of obtaining a training example from a speciﬁc sub-partition
is maximally close to a known value1


key: chroma:document
string_value:
is maximally close to a known value1
s,s≥1, but not equal to or below. That is, we are
targeting sub-patterns with a certain frequency. Formally, the pattern recognition problem
that we address can be deﬁned as follows:
Deﬁnition 1. Letybe a random output variable taking values ω∈ {0,1}and letX=
(x1,...,x o)refer toorandom input variables with domain X={0,1}o. Given the joint input-
output distribution P(X,y), the input domain Xpartitions into two parts, X1={X|P(y=


key: chroma:document
string_value:
0|X)≤P(y= 1|X)}andX0={X|P(y= 0|X)> P(y= 1|X)}. Assume that the joint
probability distribution P(X,y)relatesytonsub-patterns inX,n/2sub-patterns per y-
value. Let each sub-pattern be deﬁned by a corresponding conjunctive clause Qω
j(X),j∈
{1,...,n/ 2},ω∈{0,1}. That is, clause Qω
j(X)delineates a subset Xω
jof inputs withinX:
Xω
j={X|Qω
j(X) = 1}. Let these subsets have the following properties:
1. Conditioned on Xω, the probability of input subset Xω
jis larger than a given constant
1


key: chroma:document
string_value:
jis larger than a given constant
1
s,s∈[1,∞):P(Xω
j|Xω)>1
s.
2. If clause Qω
j(X)is extended with additional literals, the resulting input subset Xω′
jbecomes
a subset ofXω
j,Xω′
j⊂Xω
j, and occurs with probability less than1
s, conditioned onXω:
P(Xω′
j|Xω)<1
s.
3. Input outside the Xω
j-subsets cannot satisfy property (2) above: P(X\⋃
jXω
j|Xω)<1
s.
4. Each input Xfrom the subsetXω
jis a noisy predictor of the associated y-valueω,P(y=
ω|Xω
j)>0.5, implyingXω
j⊆Xω.


key: chroma:document
string_value:
ω|Xω
j)>0.5, implyingXω
j⊆Xω.
Under these assumptions, given nands, the pattern recognition problem consists of recovering
the clauses Qω
j(X),j∈{1,...n/ 2},ω∈{0,1}, only by observing input-output samples (X,y)
from the unknown distribution P(X,y).
Note that in practice, for real-world data, nandsmay not be known. They can then be
considered as hyper-parameters, optimized through a hyper-parameter search.
3 The Tsetlin Machine


key: chroma:document
string_value:
3 The Tsetlin Machine
The Tsetlin Machine decomposes problems into self-contained patterns that are expressed as
conjunctive clauses in propositional logic. In this section, we ﬁrst cover the clause-based infer-
ence structure, before we proceed to present the Tsetlin Automata teams, which are responsible
for composing the clauses. The teams are coordinated by means of a game that optimizes accu-
racy using so-called Type I and Type II Feedback, combined with resource allocation dynamics.


key: chroma:document
string_value:
0/1
………+-+-
∑⌵⌵⌵⌵0 1 0 … 1 0 10/1∑0/10/1
0/10/1Output
InputThreshold FunctionSummationConjunctiveClausesFigure 3: The Tsetlin Machine inference structure, introducing clause polarity, a summation
operator collecting ”votes”, and a threshold function arbitrating the ﬁnal output.
3.1 Tsetlin Machine Inference Structure
The Tsetlin Machine inference structure is shown in Figure 3. As seen, an input vector X=
(x1,...,x o)∈X,X={0,1}o,ofopropositional variables is fed to several conjunctive clauses


key: chroma:document
string_value:
for evaluation.
The number of clauses employed is a user set parameter n. Half of the clauses are assigned
positive polarity, denoted by upper index 1: C1
j,j∈{1,...,n/ 2}. The other half is assigned
negative polarity, denoted by upper index 0: C0
j,j∈{1,...,n/ 2}. In the ﬁgure, polarity is
indicated with a ’+’ or ’-’ sign, attached to each clause.
The role of the clauses is to capture sub-patterns in Xper Eq. 4, however, with a slight
adjustment. We allow clauses without literals, i.e., Lω


key: chroma:document
string_value:
j=∅,j∈{1,...,n/ 2},ω∈{0,1}. These
output 1 during learning and 0 during classiﬁcation:
Cω
j(X) =

1, ifLω
j=∅during learning ,
0, ifLω
j=∅during classiﬁcation ,⋀
lk∈Lω
jlk,ifLω
j̸=∅.(6)
In other words, clauses without literals play no role during classiﬁcation, and can be pruned
from the inference structure. During learning, however, outputting 1 stimulates further updat-
ing of the clause, as explored in Section 3.3.


key: chroma:document
string_value:
ing of the clause, as explored in Section 3.3.
The clause evaluations, in turn, are combined into a ﬁnal output decision through summa-
tion and unit step thresholding, u(v) = 1 ifv≥0else 0:
ˆy=u
n/2∑
j=1C1
j(X)−n/2∑
j=1C0
j(X)
. (7)
That is, output is decided based on a majority vote, with the positive polarity clauses voting
fory= 1 and the negative ones for y= 0. The purpose is to reach a balanced output


key: chroma:document
string_value:
decision, weighting positive against negative evidence, found in the input. The classiﬁer ˆ y=
u(x1¯x2+ ¯x1x2−x1x2−¯x1¯x2), for instance, captures the XOR-relation.
3.2 The Tsetlin Automata Team for Composing Clauses
Recall that a Tsetlin Automaton can be represented as a quintuple {Φ,α,β,F(·,·),G(·)}. Each
clauseCω
j,ω∈{0,1},j∈{1,...,n/ 2}, of a Tsetlin Machine is composed by a team of such
Tsetlin Automata. The team consists of 2 oautomata, one assigned per literal lk∈L. Let
Φω


key: chroma:document
string_value:
IncludeExcludex2x1¬x2¬x1⌵⌵⌵x2x1¬x2¬x1⌵⌵⌵Clause +Clause —
X = (x1, x2)Majority votingFeedback Type I or IIClause outputClause output
Input feature vectorDecided classFeedback Type I or II+—
123456
123456
123456
123456
123456
123456
123456
123456Figure 4: Two Tsetlin Automata teams, each producing a conjunctive clause. The overall
output is based on majority voting.
clauseCω
j,ω∈{0,1},j∈{1,...,n/ 2}. It is this state that decides whether the literal is


key: chroma:document
string_value:
excluded (action α1) or included (action α2) in the clause.
Figure 4 depicts two clauses, one with positive polarity ( C1) and one with negative polarity
(C0). They are both assigned four Tsetlin Automata, each with six states, controlling the
inclusion of literals L={x1,x2,¬x1,¬x2}. The leftmost Tsetlin Automaton in the ﬁgure
has for example decided to include literal x1because it is in state φ4. The second Tsetlin


key: chroma:document
string_value:
Automaton, being in state φ3, has decided to exclude literal x2. Given the decisions of the four
Tsetlin Automata of C1, the clause has the form x1¯x2. Similarly, clause C0obtains the form
x1x2.
3.3 The Tsetlin Machine Game for Learning Conjunctive Clauses
We here introduce the game theoretic learning mechanism that guides the Tsetlin Automata
towards solving the pattern recognition problem from Deﬁnition 1. The game is designed


key: chroma:document
string_value:
to deal with the problem of vanishing signal-to-noise ratio for learning automata games (cf.
Section 1.3).
3.3.1 Tsetlin Automata Games
A game of Tsetlin Automata involves multiple automata and is played over several rounds [3].
In each round of the game, the participating Tsetlin Automata independently decide upon
their next action from α. Thus, if two actions are available to each automaton, they jointly
select among 2Wunique action conﬁgurations, with Wbeing the number of automata. After


key: chroma:document
string_value:
the Tsetlin Automata have made their decisions, the round ends with the automata being
individually penalized/rewarded based on the jointly selected action conﬁguration. The next
round of the game then starts.
To fully specify the game, we need to assign a reward probability per Tsetlin Automaton for
each unique conﬁguration of actions. We refer to these reward probabilities as the payoﬀ matrix
of the game. With Wtwo-action automata, we accordingly need W2Wreward probabilities


key: chroma:document
string_value:
3.3.2 Design of the Payoﬀ Matrix
The potential complexity of the Tsetlin Machine game is immense, because the decisions of
every single Tsetlin Automaton jointly decide the behaviour of the Tsetlin Machine as a whole.
Indeed, under the right conditions, a single Tsetlin Automaton has the power to completely
disrupt a clause by introducing a contradiction. The payoﬀs of the game must therefore be
carefully designed so that the Tsetlin Automata are guided towards building the clauses that


key: chroma:document
string_value:
solve the pattern recognition problem at hand. To complicate further, an explicit enumeration
of the payoﬀs is impractical due to the potentially tremendous size of the payoﬀ matrix.
To address the above challenges, we decompose the payoﬀ matrix based on the notion of
true positive, false negative, and false positive clause output, treating clauses as individual
classiﬁers. True negative clause output is ignored to increase the freedom of the automata. By


key: chroma:document
string_value:
progressively suppressing false negative and false positive clause output, and reinforcing true
positive clause output, we intend to guide the Tsetlin Automata towards maximizing pattern
recognition accuracy. This guiding is based on what we will refer to as Type I and Type II
Feedback. Based on the principles exposed in this section, we will deﬁne the resulting game
formally in Section 4.
3.3.3 Type I Feedback – Combating False Negative Output


key: chroma:document
string_value:
In each round of the game, the Tsetlin Machine receives an input-output sample ( X,y). Type I
Feedback is then given to the Tsetlin Automata of clauses Cω
jwheny=ω. Consider a particular
Tsetlin Automaton with state Φω
jk, controlling inclusion of literal lkin clauseCω
j, i.e., thejth
clause of polarity ω. Feedback is then decided by three factors:
1. The decisions of the Tsetlin Automata team for clause Cω
j(X) as a whole, manifested by
the output of the clause.


key: chroma:document
string_value:
the output of the clause.
2. The truth value of the literal lkassigned to the automaton.
3. The action αz=G(Φω
jk),z∈{1,2},decided by the automaton.
Table 2 contains the probabilities that we use to generate Type I Feedback, looking up the
above three factors. For instance, assume that:
1. ClauseCω
j(X) evaluates to 1,
2. Literallkis 1, and
3. Automaton state Φω
jkproduces the Include action (α2).
By examining the corresponding cell in Table 2, we observe that the probability of receiving a


key: chroma:document
string_value:
rewardP(Reward) iss−1
s. Further, the probability of inaction P(Inaction) is1
s. Finally, the
probability of receiving a penalty P(Penalty) is zero.
Note that the Inaction feedback is a novel extension to the Tsetlin Automaton, which
traditionally receives either a Reward or a Penalty. When receiving the Inaction feedback, the
Tsetlin Automaton is simply left unchanged.
Brief Analysis of Type I Feedback. Notice how the Type I Feedback probabilities


key: chroma:document
string_value:
force production of clauses that contain many literals, up to a certain point. We achieve this
by reinforcing Include actions more strongly than Exclude actions, using reward- and penalty
probabilities favouring Include . As seen, the hyper-parameter s,s≥1, controls how strongly
we favour Include . In eﬀect, sdecides how ”ﬁne-grained” patterns the clauses are going to
capture. The larger the value of s, the more the Tsetlin Automata are stimulated to include


key: chroma:document
string_value:
literals in their clauses. The countering force is non-matching input, i.e., input that makes the
clause evaluate to 0. For such input, the Exclude action is rewarded and the Include action
is penalized. Clearly, the probability of encountering non-matching input grows as sincreases
(more literals are included). When the above two counter-acting forces are in balance, we have
10


key: chroma:document
string_value:
a Nash equilibrium as discussed further in Section 4. These dynamics are a critical part of the
Tsetlin Machine, allowing learning of any sub-pattern, no matter how infrequent, as decided
bys.
Boosting of True Positive Feedback (Column 1 in Table 2). The feedback proba-
bilities in Table 2 have been selected based on mathematical derivations (see Section 4). For
certain real-life data sets, however, it turns out that further boosting of Include actions can


key: chroma:document
string_value:
be beneﬁcial. That is, pattern recognition accuracy can be enhanced by boosting rewarding of
these actions when they produce true positive outcomes. Penalizing of Exclude actions is then
adjusted accordingly. In all brevity, we boost rewarding in this manner by replacings−1
swith
1.0 and1
swith 0.0 in Column 1 of Table 2.
3.3.4 Type II Feedback – Combating False Positive Output
Table 3 covers Type II Feedback, that is, feedback that combats false positive output. Type II


key: chroma:document
string_value:
Feedback is given to the Tsetlin Automata of clauses Cω
jwheny̸=ω. The purpose of this
feedback is to introduce candidate literals that increase discrimination power. That is, a clause
Cω
jis supposed to output 0 when y̸=ω, to discriminate between the outputs ω∈{0,1}.
However, if it erroneously evaluates to 1, we correct this by identifying the Tsetlin Automata
that have excluded a zero-valued literal from the clause. It is suﬃcient to merely include one


key: chroma:document
string_value:
of these literals to make the oﬀending clause evaluate to 0 instead. This is because the clauses
are conjunctive.
Brief Analysis of Type II Feedback. Observe that Type II Feedback only penalizes the
exclusion of literals of value 0 when a clause outputs 1 (false positive output). When a clause
outputs 0 (the two leftmost columns in the table), only Inaction-feedback is given. This is to
avoid local minima by leaving it to Type I feedback to reinforce Include actions. Accordingly,


key: chroma:document
string_value:
together, Type I Feedback and Type II Feedback interact to minimize the expected output
error, moving towards a global optimum as explored in Section 4.
Truth Value of Clause Cω
j 1 0
Truth Value of Literal lk 1 0 1 0
Include Literal ( α2→lk∈Lω
j)P(Reward)s−1
sNA 0 0
P(Inaction)1
sNAs−1
ss−1
s
P(Penalty) 0 NA1
s1
s
Exclude Literal ( α1→lk/∈Lω
j)P(Reward) 01
s1
s1
s
P(Inaction)1
ss−1
ss−1
ss−1
s
P(Penalty)s−1
s0 0 0


key: chroma:document
string_value:
ss−1
ss−1
ss−1
s
P(Penalty)s−1
s0 0 0
Table 2: Type I Feedback — Feedback from the perspective of a single Tsetlin Automaton
deciding to either Include orExclude a given literal lkin the clause Cω
j. Type I Feedback is
triggered to increase the number of clauses that correctly evaluates to 1 for a given input X.
Truth Value of Clause Cω
j 1 0
Truth Value of Literal lk1 0 1 0
Include Literal ( α2→lk∈Lω
j)P(Reward) 0 NA 0 0
P(Inaction) 1.0 NA 1.0 1.0
P(Penalty) 0 NA 0 0


key: chroma:document
string_value:
P(Inaction) 1.0 NA 1.0 1.0
P(Penalty) 0 NA 0 0
Exclude Literal ( α1→lk/∈Lω
j)P(Reward) 0 0 0 0
P(Inaction) 1.0 0 1.0 1.0
P(Penalty) 0 1.0 0 0
Table 3: Type II Feedback — Feedback from the perspective of a single Tsetlin Automaton
deciding to either Include orExclude a given literal lkin the clause Cω
j. Type II Feedback is
triggered to increase the discrimination power of the clauses.
11


key: chroma:document
string_value:
3.3.5 Allocating Clauses to Sub-Patterns
For the Tsetlin Machine output to be correct it is suﬃcient that the summation part of Eqn. 7,∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X), has the correct sign. That is, it is suﬃcient that a single clause,
with the appropriate polarity, outputs 1. In the case of noisy data, however, it can be beneﬁcial
to introduce a margin T, referred to as the target of the summation. The intent is to make the


key: chroma:document
string_value:
available clauses distribute themselves across the sub-patterns in the data, so that Tclauses
of the correct polarity evaluate to 1 for any particular input X. Additionally, the summation
targetTopens up for interplay between the clauses, including rectiﬁcation of special cases.
The resource allocation mechanism that we introduce here is inspired by a ﬁnite state
automaton-based solution to the fractional knapsack problem in unknown and stochastic en-


key: chroma:document
string_value:
vironments [29]. The purpose of the mechanism is to ensure that the Tsetlin Machine only
spends a few of the available clauses to represent each speciﬁc sub-pattern. This is achieved
by randomly selecting clauses to receive feedback, reducing intensity when approaching the
summation target T.
Generating Type I Feedback. For Type I Feedback, we randomly select clauses Cω
j,ω=
y. Among these, the probability of receiving Type I Feedback is
T−clip(∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(8)


key: chroma:document
string_value:
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(8)
wheny= 1, and
T+clip(∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(9)
wheny= 0. Here, the clip operation restricts the sum to lie between −TandT.
Generating Type II Feedback. For Type II Feedback, we randomly select clauses
Cω
j,ω̸=y. The probability of receiving Type II Feedback is:
T+clip(∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(10)
wheny= 1, and
T−clip(∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(11)
wheny= 0.


key: chroma:document
string_value:
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
2T(11)
wheny= 0.
Notice how the feedback vanishes as the number of triggering clauses correctly approaches
T/−T. This is pertinent for eﬀective use of the available pattern representation capacity. To
exemplify, assume that the correct output is y= 1 for an input X. If the clause output sum
accumulates to a total of Tor more, neither rewards nor penalties are provided to the involved


key: chroma:document
string_value:
Tsetlin Automata. This leaves the Tsetlin Automata free to learn other sub-patterns.
3.3.6 The Tsetlin Machine Algorithm
The step-by-step procedure for learning conjunctive clauses can be found in Algorithm 1. The
algorithm takes a set of training examples ( X,y)∈Sas input. It then produces a collection of
conjunctive clauses for predicting the output y. We will now take a closer look at the algorithm,
line-by-line.


key: chroma:document
string_value:
line-by-line.
Lines 2-3. From the perspective of game theory, we form one team of Tsetlin Automata
per clause, denoted A0
j,A1
j,j∈{1,...,n/ 2}. A team consists of 2 oautomata, each assigned
a user speciﬁed number of states, Φω
jk∈{φ1,φ2,...,φ 2N},j∈{1,...,n/ 2},k∈{1,..., 2o},
ω∈{0,1}. The initial state is then set to Φω
jk←φN(the Exclude action).
Line 5. The learning process is driven by a set of training examples S, sampled from the


key: chroma:document
string_value:
input-output distribution P(X,y). Each training example ( X,y) is fed to the Tsetlin Machine,
one at a time, facilitating online learning.
12


key: chroma:document
string_value:
Algorithm 1 Tsetlin Machine
Input Training data ( X,y)∈S∼P(X,y), Number of clauses n, Number of inputs o,
Precisions, TargetT
Output Trained conjunctive clauses C1
j,C0
j,j= 1,...,n/ 2
1:function TrainTsetlinMachine (S,n,o,s,T )
2:A1
1,...,A1
n/2←CreateTATeams( n/2,2o,φN)⊿Createn/2 teams of 2 oTsetlin
Automata (TA), one team A1
jper
clauseC1
j. Each TA is initialized to
stateφN.
3:A0
1,...,A0
n/2←CreateTATeams( n/2,2o,φN)⊿Create another n/2 teams of 2 oTA,
one teamA0
jper clause C0
j. Each


key: chroma:document
string_value:
one teamA0
jper clause C0
j. Each
TA is initialized to state φN.
4: repeat
5:X,y←GetTrainingExample( S) ⊿Mini-batches, random selection, etc.
6:C1
1,...,C1
n/2←ObtainClauses(A1
1,...,A1
n/2)⊿Each TA teamA1
jproduces a
clauseC1
j.
7:C0
1,...,C0
n/2←ObtainClauses(A0
1,...,A0
n/2)⊿Each TA teamA0
jproduces a
clauseC0
j.
8:v←∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X) ⊿Calculate sum of clause outputs.
9: forj←1,...,n/ 2do ⊿Give feedback to the TA teams.
10: ify= 1then
11: ifRandom()≤T−clip(v,−T,T)
2Tthen


key: chroma:document
string_value:
11: ifRandom()≤T−clip(v,−T,T)
2Tthen
12: GenerateTypeIFeedback( X,C1
j(X),A1
j,o)⊿Give Type I Feedback to
TA of clause C1
j.
13: end if
14: ifRandom()≤T−clip(v,−T,T)
2Tthen
15: GenerateTypeIIFeedback( X,C0
j(X),A0
j,o)⊿Give Type II Feedback to
TA of clause C0
j.
16: end if
17: else ify= 0then
18: ifRandom()≤T+clip(v,−T,T)
2Tthen
19: GenerateTypeIIFeedback( X,C1
j(X),A1
j,o)⊿Give Type II Feedback to
TA of clause C1
j.
20: end if
21: ifRandom()≤T+clip(v,−T,T)
2Tthen


key: chroma:document
string_value:
20: end if
21: ifRandom()≤T+clip(v,−T,T)
2Tthen
22: GenerateTypeIFeedback( X,C0
j(X),A0
j,o)⊿Give Type I Feedback to
TA of clause C0
j.
23: end if
24: end if
25: end for
26: until StopCriteria(S,C0
1,...,C0
n/2,C1
1,...,C1
n/2)
27: return PruneAllExcludeClauses( C0
1,...,C0
n/2,C1
1,...,C1
n/2)⊿Return completely
trained conjunctive clauses after pruning clauses where all literals have been excluded.
28:end function
13


key: chroma:document
string_value:
Algorithm 2 Type I Feedback
Input Input vector Xof sizeo, Clause output cω
j, Tsetlin Automata team Aω
j
1:procedure GenerateTypeIFeedback (X,cω
j,Aω
j,o)
2: fork←1,..., 2odo ⊿Reward/Penalize all Tsetlin Automata in Aω
j.
3:lk←ObtainLiteral( X,k) ⊿Get literallkfrom input X.
4:αu←G(Φω
jk)⊿Obtain corresponding automaton action according to Eqn. 1.
5:βv←SampleTypeIFeedback( αu,lk,cω
j)⊿Sample Type I Feedback from Table 2.
6: Φω
jk←F(Φω
jk,βv)⊿Update state of Tsetlin Automaton according to Eqn. 2.


key: chroma:document
string_value:
7: end for
8:end procedure
Algorithm 3 Type II Feedback
Input Input vector Xof sizeo, Clause output cω
j, Tsetlin Automata team Aω
j
1:procedure GenerateTypeIIFeedback (X,cω
j,Aω
j,o)
2: fork←1,..., 2odo ⊿Penalize all Tsetlin Automata in Aω
j.
3:lk←ObtainLiteral( X,k) ⊿Get literallkfrom input X.
4:αu←G(Φω
jk)⊿Obtain corresponding automaton action according to Eqn. 1.
5:βv←SampleTypeIIFeedback( αu,lk,cω
j)⊿Sample Type II Feedback from Table 3.
6: Φω
jk←F(Φω


key: chroma:document
string_value:
6: Φω
jk←F(Φω
jk,βv)⊿Update state of Tsetlin Automaton according to Eqn. 2.
7: end for
8:end procedure
Line 6-7. In each iteration, the Tsetlin Automata decide whether to Include orExclude
their assigned literals in their respective clauses. The result is a new set of clauses, C0
1,...,C0
n/2,
C1
1,...,C1
n/2, for predicting y.
Line 8. Based on the new clauses, the current example Xis evaluated, producing a clause


key: chroma:document
string_value:
sumvaccording to Eqn. 7. The unit step function uis not used here because examples are
not classiﬁed during training.
Lines 9-25. The next step is to randomly determine the clauses whose Tsetlin Automata
are to receive Type I Feedback (Eqn. 8-9) and Type II Feedback (Eqn. 10-11), based on the
clipped diﬀerence between vandT.
After Type I or Type II Feedback have been triggered for a clause, the Tsetlin Automata


key: chroma:document
string_value:
of the clause are rewarded/penalized according to Algorithm 2 and Algorithm 3. In all brevity,
rewarding/penalizing is directly based on Table 2 and Table 3.
Line 26. The above steps are iterated until a stopping criteria is fulﬁlled (for instance a
certain number of iterations over the dataset). The current clauses are then returned as the
output of the learning process, after the clauses without literals have been removed.
3.4 Implementation Using Bitwise Operators


key: chroma:document
string_value:
3.4 Implementation Using Bitwise Operators
Small memory footprint and speed of operation can be crucial in complex and large scale
pattern recognition. Being based on propositional formula, the Tsetlin Machine architecture
can naturally be represented with bits and manipulated upon using bitwise operators. However,
it is not straightforward how to represent and update the Tsetlin Automata themselves. First


key: chroma:document
string_value:
of all, the state index of each Tsetlin Automaton is an integer value. Further, the action of
an automaton is decided upon using a smaller-than operator, while feedback is processed by
means of increment and decrement operations.
One approach to bitwise operation is to jointly represent the state indexes of all of the
Tsetlin Automata of a clause Cω
jwith multiple sequences of bits. Sequence 1 then contains the


key: chroma:document
string_value:
ﬁrst bit of each state index, sequence 2 contains the second bit, and so on, as exempliﬁed in
Figure 5 for 24 Tsetlin Automata. The beneﬁt of this representation is that the action of each
Tsetlin Automaton is readily available from the most signiﬁcant bit (sequence 8 in the ﬁgure).
14


key: chroma:document
string_value:
Thus, the output of the clause can be obtained from the input based on fast bitwise operators
(NOT, AND, and CMP - comparison).
In the ﬁgure, a setup for the Noisy XOR dataset from Section 5 is used for illustration
purposes. First, the 12-bit input is extended to 24 bits by concatenating the original input
with the original input inverted. The resulting 24 bits are in turn connected with 24 Tsetlin
Automata. The ﬁrst 12 control the non-negated input, while the second 12 control the negated


key: chroma:document
string_value:
input.
Employing the latter bit-based representation reduces memory usage four times, compared
to using a full 32-bit integer to represent the state of each and every Tsetlin Automaton.
More importantly, it is possible to increment/decrement the states of all of the automata in
parallel with bitwise operations through customized increment/decrement procedures, signiﬁ-
cantly increasing learning speed. As an example, for the MNIST dataset (cf. Section 5), the


key: chroma:document
string_value:
overall memory usage is approximately ten times smaller, learning speed 3.5 times faster, and
classiﬁcation speed 8 times faster with the bit-based representation.
When deployed after training, only the state bit sequence containing the most signiﬁcant
bit is required. The other bit sequences can be discarded because these bits are only used to
keep track of the learning. This provides a further reduction in memory usage.


key: chroma:document
string_value:
State of 24 Tsetlin Automata 110101010011110101010011001010101100NOTAND100000000000 010000000000101000100000 010011010000111000010001 010010011011…State Bits1-78CMPOutput
Input0
Figure 5: Bit-based representation of a clause for the Noisy XOR dataset (see Section 5).
The bit-based representation of the Tsetlin Automata states allows the actions of all of the
automata to be obtained directly from the most signiﬁcant bit (bit 8 in the ﬁgure).
4 Theoretical Analysis


key: chroma:document
string_value:
4 Theoretical Analysis
In this section, we analyse the Tsetlin Machine game formally. We base the analysis on par-
titioning the input space Xinto subsets that isolate the impact a single Tsetlin Automaton
has on classiﬁcation, mediated through its clause. We then study each of these subsets for the
two output scenarios y= 0 andy= 1. From this investigation, we identify the Nash equi-
libria of Tsetlin Machine learning and compare the Nash equilibria clauses with the optimal


key: chroma:document
string_value:
sub-patterns from Deﬁnition 1.
4.1 The Payoﬀ Matrix of the Tsetlin Machine Game
Based on the learning dynamics described algorithmically in Section 3, we now introduce a
more succinct description of Tsetlin Machine learning by formally deﬁning the payoﬀ matrix
of the Tsetlin Machine game.
Deﬁnition 2. Consider a pattern recognition problem per Deﬁnition 1 and a Tsetlin Ma-
chine with nclausesCω
j(X),j∈ {1,...,n/ 2},ω∈ {0,1}. The resulting game consists of
15


key: chroma:document
string_value:
2noTsetlin Automata (players), one automaton per literal lk∈L={x1,...,x o,¯x1,..., ¯xo},
per clause Cω
j(X). LetAω
jkbe a Tsetlin Automaton with actions aω
jk∈ {α1,α2}, control-
ling the inclusion of literal lkin clauseCω
j(α1represents Exclude and α2represents In-
clude). Each cell in the payoﬀ matrix of the game then refers to a unique action conﬁguration
α= (a0
1,1,...,a0
n/2,2o,a1
1,1...,a1
n/2,2o)∈{α1,α2}2no. Accordingly, the payoﬀ matrix consists of


key: chroma:document
string_value:
22nocells. For every cell, we deﬁne the payoﬀ of each aω
jk-action inαusing the four stochastic
variablesX,y,Uω
j, andRω
jk:
•Each game round, the distribution P(X,y)randomly produces a training example (X,y).
•Let the binary stochastic variable Uω
jrefer to whether clause Cω
jis updated ( Uω
j= 1)
or ignored ( Uω
j= 0) in the given game round. The distribution of Uω
jis decided as
follows. First, the clipped clause sum vc=clip(∑n/2
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
is


key: chroma:document
string_value:
j=1C1
j(X)−∑n/2
j=1C0
j(X),−T,T)
is
determined. This clipped clause sum is calculated from the clauses produced by the joint
action conﬁguration αof the payoﬀ matrix cell, evaluated upon X. ClauseCω
jis then
updated with probability P(Uω
j= 1) =T−vc
2Tify= 1 and with probability P(Uω
j= 1) =
T+vc
2Tify= 0.
•Finally, we decide the payoﬀ Rω
jk∈{− 1,0,+1}(Penalty, Inaction, Reward) for each
automaton Aω
jk. IfUω
j= 0 the payoﬀ is 0. Otherwise, Rω
jkis randomly set based on


key: chroma:document
string_value:
jkis randomly set based on
Table 2 ify=ω(Type I Feedback) and Table 3 if y̸=ω(Type II Feedback). We obtain
the distribution of Rω
jkfrom the designated table by looking up: (1) the action selected
(α1orα2), (2) the value of literal lkfromX, and (3) the value of the clause Cω
j(X).
Accordingly, the payoﬀ of each action aω
jkinαbecomesUω
j·Rω
jk.
In the following, we analyze the dynamics of the game using the above payoﬀ matrix.
Figure 6: The subsets X0,X1,Xω
j, andXω


key: chroma:document
string_value:
Figure 6: The subsets X0,X1,Xω
j, andXω
jk, pertinent for the theoretical analysis.
4.2 Pertinent Input Space Subsets
The input space subsets ( X-subsets) of interest are listed in Table 4 and Table 5. Observe that
all of theX-subsets in the tables are derived from the following four subsets: X0,X1,X1
j, and
X1
jk, illustrated in Figure 6.
The subsetX1={X|P(y= 0|X)≤P(y= 1|X)}is the subset of the input space where it
is optimal to output y= 1. Conversely, the subset X0={X|P(y= 0|X)> P(y= 1|X)}is


key: chroma:document
string_value:
the compliment of X, where it is optimal to output y= 0. The subsetX1
jk={X|Cω
jk(X) = 1},
Lω
jk⊆L,lk∈Ljk, is the part of the input space in which the clause Cω
jk(X) with literals
Lω
jk,lk∈Lω
jk,evaluates to 1. Finally, the subset X1
j={X|Cω
j(X) = 1},Lω
j=Lω
jk\lk, is a
superset ofX1
jk, where the literal lkhas been removed from the conjunction.
16


key: chroma:document
string_value:
Recall that it is a single Tsetlin Automaton Aω
jkthat decides whether to Include orExclude
literallkabove, which means that it in eﬀect decides between clause Cω
jk(X) (literal included)
and clause Cω
j(X) (literal excluded).
Further note that we in this section ignore polarity to simplify notation. That is, we analyze
how the clauses with positive polarity in the Tsetlin Machine game capture the sub-patterns
Q1


key: chroma:document
string_value:
Q1
h,h∈{1,...,n/ 2},from Deﬁnition 1. The analysis for negative polarity clauses follows the
same structure.
4.3 Expected Payoﬀ for the Actions of a Single Tsetlin Automaton
As deﬁned in the payoﬀ matrix of the Tsetlin Machine game, the stochasticity in Tsetlin
Machine learning comes from: (i) the training samples ( X,y), which are drawn randomly
according to an unknown distribution P(X,y); (ii) the random selection of clauses for updating;


key: chroma:document
string_value:
and (iii) the random generation of the rewards and penalties from Type I feedback (Table 2).
We ﬁrst focus on what happens with a speciﬁc Tsetlin Automaton each time its clause activates.
Thus, for the moment, we ignore stochasticity stemming from random clause selection, studying
a single clause.
We have collected the information needed to calculate the expected payoﬀ for the Exclude
action in Table 4 and for the Include action in Table 5. The ﬁrst column of the table enumerates


key: chroma:document
string_value:
the possible y-values. The second column speciﬁes a partitioning of the input space Xfor each
y-value. The third column contains the expected payoﬀ of the action covered by the table,
for the givenX-subset and output y. The expected payoﬀ has been obtained from Table 2
and Table 3. The fourth column states the probability of obtaining a training example ( X,y)
yX-Subset R Probability Noise 1−γ
0X1
jk∩X10P(y= 0|X1
jk∩X1)P(X1
jk∩X1) (1−γ)P(X1
jk∩X1)
0(X1
j\X1
jk)∩X1−1P(y= 0|[X1
j\X1
jk]∩X1)P([X1


key: chroma:document
string_value:
0(X1
j\X1
jk)∩X1−1P(y= 0|[X1
j\X1
jk]∩X1)P([X1
j\X1
jk]∩X1)(1−γ)P([X1
j\X1
jk]∩X1)
0X1\X1
j 0P(y= 0|X1\X1
j)P(X1\X1
j) (1−γ)P(X1\X1
j)
0X1
jk∩X00P(y= 0|X1
jk∩X0)P(X1
jk∩X0) γP(X1
jk∩X0)
0(X1
j\X1
jk)∩X0−1P(y= 0|[X1
j\X1
jk]∩X0)P([X1
j\X1
jk]∩X0)γP([X1
j\X1
jk]∩X0)
0X0\X1
j 0P(y= 0|X0\X1
j)P(X0\X1
j) γP(X0\X1
j)
1X1
jk∩X1−s−1
sP(y= 1|X1
jk∩X1)P(X1
jk∩X1) γP(X1
jk∩X1)
1(X1
j\X1
jk)∩X1 1
sP(y= 1|[X1
j\X1
jk]∩X1)P([X1
j\X1
jk]∩X1)γP([X1
j\X1
jk]∩X1)
1X1\X1
j1
sP(y= 1|X1\X1
j)P(X1\X1
j) γP(X1\X1
j)


key: chroma:document
string_value:
1X1\X1
j1
sP(y= 1|X1\X1
j)P(X1\X1
j) γP(X1\X1
j)
1X1
jk∩X0−s−1
sP(y= 1|X1
jk∩X0)P(X1
jk∩X0) (1−γ)P(X1
jk∩X0)
1(X1
j\X1
jk)∩X0 1
sP(y= 1|[X1
j\X1
jk]∩X0)P([X1
j\X1
jk]∩X0)(1−γ)P([X1
j\X1
jk]∩X0)
1X0\X1
j1
sP(y= 1|X0\X1
j)P(X0\X1
j) (1−γ)P(X0\X1
j)
Table 4: Expected payoﬀ for the Exclude action.
yX-Subset R Probability Noise 1−γ
0X1
jk∩X10P(y= 0|X1
jk∩X1)P(X1
jk∩X1) (1−γ)P(X1
jk∩X1)
0(X1
j\X1
jk)∩X10P(y= 0|[X1
j\X1
jk]∩X1)P([X1
j\X1
jk]∩X1)(1−γ)P([X1
j\X1
jk]∩X1)
0X1\X1
j 0P(y= 0|X1\X1
j)P(X1\X1


key: chroma:document
string_value:
j\X1
jk]∩X1)
0X1\X1
j 0P(y= 0|X1\X1
j)P(X1\X1
j) (1−γ)P(X1\X1
j)
0X1
jk∩X00P(y= 0|X1
jk∩X0)P(X1
jk∩X0) γP(X1
jk∩X0)
0(X1
j\X1
jk)∩X00P(y= 0|[X1
j\X1
jk]∩X0)P([X1
j\X1
jk]∩X0)γP([X1
j\X1
jk]∩X0)
0X0\X1
j 0P(y= 0|X0\X1
j)P(X0\X1
j) γP(X0\X1
j)
1X1
jk∩X1 s−1
sP(y= 1|X1
jk∩X1)P(X1
jk∩X1) γP(X1
jk∩X1)
1(X1
j\X1
jk)∩X1−1
sP(y= 1|[X1
j\X1
jk]∩X1)P([X1
j\X1
jk]∩X1)γP([X1
j\X1
jk]∩X1)
1X1\X1
j−1
sP(y= 1|X1\X1
j)P(X1\X1
j) γP(X1\X1
j)
1X1
jk∩X0 s−1
sP(y= 1|X1
jk∩X0)P(X1
jk∩X0) (1−γ)P(X1
jk∩X0)
1(X1
j\X1


key: chroma:document
string_value:
jk∩X0)P(X1
jk∩X0) (1−γ)P(X1
jk∩X0)
1(X1
j\X1
jk)∩X0−1
sP(y= 1|[X1
j\X1
jk]∩X0)P([X1
j\X1
jk]∩X0)(1−γ)P([X1
j\X1
jk]∩X0)
1X0\X1
j−1
sP(y= 1|X0\X1
j)P(X0\X1
j) (1−γ)P(X0\X1
j)
Table 5: Expected payoﬀ for the Include action.
17


key: chroma:document
string_value:
for the corresponding X-subset and output y. The ﬁfth column speciﬁes these probabilities
assuming a static output noise probability P(y= 0|X1) =P(y= 1|X0) = 1−γ,γ > 0.5. If
for instance γ= 0.75, the output is erroneously y= 0 forX∈X1andy= 1 forX∈X0,
on average, 25% of the time. The assumption of static noise is not a limitation of the Tsetlin
Machine, however, the assumption simpliﬁes the theoretical analysis that follows.


key: chroma:document
string_value:
From Table 4 and Table 5, we can calculate the expected payoﬀ of each action by summing
up the rewards (positive polarity) and penalties (negative polarity) in the R-column, each
multiplied with the probability of the X-subset and y-value of its row. The addend from the
ﬁrst row in Table 4 is for instance 0 ·P(y= 0|X1
jk∩X1)P(X1
jk∩X1), which becomes zero
because the reward is zero.
Lemma 1. Consider a literal lkand clause Cω
j. The expected payoﬀ of excluding lkfromCω
j


key: chroma:document
string_value:
j. The expected payoﬀ of excluding lkfromCω
j
in the Tsetlin Machine game (Deﬁnition 2) is:
1
s·γ·P(X1\X1
jk) +1
s·(1−γ)·P(X0\X1
jk)−s−1
s·γ·P(X1
jk∩X1)−
s−1
s·(1−γ)·P(X1
jk∩X0)−(1−γ)·P([X1
j\X1
jk]∩X1)−γ·P([X1
j\X1
jk]∩X0).(12)
Proof. We prove Lemma 1 based on Table 4. The expected payoﬀ of the Exclude action can
be calculated as follows, assuming static noise probability 1 −γ:
1
s·γ·P([X1
j\X1
jk]∩X1) +
1
s·γ·P(X1\X1
j) +
1
s·(1−γ)·P([X1
j\X1
jk]∩X0) +
1
s·(1−γ)·P(X0\X1
j)−
s−1
s·γ·P(X1


key: chroma:document
string_value:
j\X1
jk]∩X0) +
1
s·(1−γ)·P(X0\X1
j)−
s−1
s·γ·P(X1
jk∩X1)−
s−1
s·(1−γ)·P(X1
jk∩X0)−
(1−γ)·P([X1
j\X1
jk]∩X1)−
γ·P([X1
j\X1
jk]∩X0). (13)
Since (X1
j\X1
jk)∩(X1\X1
j) =∅and (X1
j\X1
jk)∪(X1\X1
j) =X1\X1
jk, we can simplify to:
1
s·γ·P(X1\X1
jk) +
1
s·(1−γ)·P(X0\X1
jk)−
s−1
s·γ·P(X1
jk∩X1)−
s−1
s·(1−γ)·P(X1
jk∩X0)−
(1−γ)·P([X1
j\X1
jk]∩X1)−
γ·P([X1
j\X1
jk]∩X0). (14)
Lemma 2. Consider a literal lkand clause Cω
j. The expected payoﬀ of including lkinCω
jin
the Tsetlin Machine game (Deﬁnition 2) is:


key: chroma:document
string_value:
jin
the Tsetlin Machine game (Deﬁnition 2) is:
s−1
s·γ·P(X1
jk∩X1) +s−1
s·(1−γ)·P(X1
jk∩X0)−
1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0\X1
jk). (15)
18


key: chroma:document
string_value:
Proof. The expected payoﬀ of the Include action can be derived from Table 5:
s−1
s·γ·P(X1
jk∩X1) +
s−1
s·(1−γ)·P(X1
jk∩X0)−
1
s·γ·P([X1
j\X1
jk]∩X1)−
1
s·γ·P(X1\X1
j)−
1
s·(1−γ)·P([X1
j\X1
jk]∩X0)−
1
s·(1−γ)·P(X0\X1
j), (16)
which again simpliﬁes to:
s−1
s·γ·P(X1
jk∩X1) +
s−1
s·(1−γ)·P(X1
jk∩X0)−
1
s·γ·P(X1\X1
jk)−
1
s·(1−γ)·P(X0\X1
jk). (17)
4.4 Tsetlin Automaton Convergence Criteria
Most bandit algorithms attempt to learn the action with the largest expected payoﬀ. A Tsetlin


key: chroma:document
string_value:
Automaton, however, can only learn this action if the expected payoﬀ of the action is posi-
tive (i.e., the probability of receiving a reward is larger than the probability of receiving a
penalty) [3]. If the latter criterion is not fulﬁlled, the Tsetlin Automaton instead moves to-
wards its center states ( φNandφN+1), that is, towards indecision. This property is crucial for
eliminating sub-optimal Nash equilibria in Tsetlin Machine learning, and is one of the reasons


key: chroma:document
string_value:
why it is not straightforward to replace the Tsetlin Automaton with another bandit algorithm,
without loss in learning accuracy. So, in order to have a stable Nash equilibrium in Tsetlin
Machine learning, each Tsetlin Automaton action in the equilibrium must not only have the
largest expected payoﬀ, the expected payoﬀ must also be positive.
Lemma 3. By increasing the number of states, a two-action Tsetlin Automaton converges to


key: chroma:document
string_value:
performing an action αz,z∈{0,1},with probability arbitrarily close to unity iﬀ:
1. The action has positive expected payoﬀ: E[R|αz]>0.
2. The action has the largest expected payoﬀ: E[R|αz]>E[R|α1−z].
Proof. The lemma follows trivially from the Tsetlin Automaton convergence proof found in [3].
4.5 Nash Equilibria Without Noise
Lemma 4. Consider a pattern recognition problem per Deﬁnition 1 without noise, P(y̸=
ω|Xω) = 1−γ= 0, and a Tsetlin Machine game per Deﬁnition 2 with a single clause C1(X)of


key: chroma:document
string_value:
positive polarity. The sub-patterns Q1
h(X),h∈{1,...n/ 2},of the pattern recognition problem
are then all Nash equilibria of the game. Additionally, the expected payoﬀ of each player action
in each equilibrium is positive, hence not rejected by the Tsetlin Automata players.
19


key: chroma:document
string_value:
Proof. We ﬁrst analyse the case when the Exclude action is optimal, before addressing the
Include action.
Exclude is Optimal. From Property 1 and Property 2 of Deﬁnition 1, we have P(X1
jk|X1)<
1
s<P(X1
j|X1)⇔P(X1
jk∩X1)
P(X1)<1
s<P(X1
j∩X1)
P(X1), which simpliﬁes toP(X1
jk)
P(X1)<1
s<P(X1
j)
P(X1)because
of Property 4. From Property 4 we know in particular that [ X1
j\X1
jk]∩X0=∅. Under these
conditions, we must verify that Exclude is also the preferred action according to Lemma 3.


key: chroma:document
string_value:
We ﬁrst verify that the expected payoﬀ of Exclude (Eqn. 12) is positive to accommodate for
Criterion 1 of Lemma 3. We here assume no noise, i.e., γ= 1, and thus can remove the addends
containing the factor (1 −γ) from Eqn. 12. We further know that P(X1
j\X1
jk]∩X0) = 0 because
[X1
j\X1
jk]∩X0=∅. Hence, we can remove addends including that factor as well, ending up
with:
1
s·P(X1\X1
jk)−s−1
s·P(X1
jk∩X1)>0. (18)
BecauseX1
jis a subset ofX1(Property 4), we have:
1
s·P(X1\X1
jk)−s−1
s·P(X1


key: chroma:document
string_value:
1
s·P(X1\X1
jk)−s−1
s·P(X1
jk)>0. (19)
Dividing by P(X1) on each side we get:
1
s·P(X1\X1
jk)
P(X1)−s−1
s·P(X1
jk)
P(X1)>0. (20)
WithP(X1
jk)
P(X1)<1
s, we haveP(X1\X1
jk)
P(X1)>s−1
s. This clearly entails that the left side is positive.
We next verify that the expected payoﬀ of Exclude is larger than that of Include to ensure
Criterion 2 of Lemma 3. Performing the same simpliﬁcations as above, however, for Eqn. 15,
we end up with the following condition:
s−1
s·P(X1
jk)
P(X1)−1
s·P(X1\X1
jk)


key: chroma:document
string_value:
s−1
s·P(X1
jk)
P(X1)−1
s·P(X1\X1
jk)
P(X1)<0. (21)
For the same reasons that the expected payoﬀ of the Exclude action was positive, the expected
payoﬀ of Include is negative, which clearly is inferior.
Include is Optimal. Conversely, given that Include is optimal, we have:P(X1
jk)
P(X1)>1
s
(Property 1) and X1
jk∩X0=∅(from Property 4). With no noise we can again simplify the
calculation of the expected payoﬀ of Include , reducing Eqn. 15 to:
s−1
s·P(X1
jk)
P(X1)−1
s·P(X1\X1
jk)
P(X1)>0. (22)


key: chroma:document
string_value:
s·P(X1
jk)
P(X1)−1
s·P(X1\X1
jk)
P(X1)>0. (22)
WithP(X1
jk)
P(X1)>1
s(Property 1), we must haveP(X1\X1
jk)
P(X1)<s−1
s. Accordingly, the expected
payoﬀ of Include is positive.
We now turn to investigating whether the expected payoﬀ of Include is larger than the
expected payoﬀ of Exclude (Criterion 2 of Lemma 3), which we show by verifying that the
expected payoﬀ of Exclude is negative. We can again simplify Eqn. 12, however, this time
without knowing that [ X1
j\X1
jk]∩X0=∅:
1
s·P(X1\X1
jk)−s−1


key: chroma:document
string_value:
j\X1
jk]∩X0=∅:
1
s·P(X1\X1
jk)−s−1
s·P(X1
jk)−P([X1
j\X1
jk]∩X0)<0. (23)
Dividing by P(X1) on each side we get:
1
s·P(X1\X1
jk)
P(X1)−s−1
s·P(X1
jk)
P(X1)−P([X1
j\X1
jk]∩X0)
P(X1)<0. (24)
20


key: chroma:document
string_value:
Again, withP(X1
jk)
P(X1)>1
s, we haveP(X1\X1
jk)
P(X1)<s−1
s. From this result alone, the left side is
negative. Additionally, the addendP([X1
j\X1
jk]∩X0)
P(X1)is negative as well, reducing the expected
payoﬀ of Exclude even further. So, changing from Include toExclude gives a net expected
loss.
Lemma 5. Consider a pattern recognition problem per Deﬁnition 1 without noise, P(y̸=
ω|Xω) = 1−γ= 0, and a Tsetlin Machine game per Deﬁnition 2 with a single clause C1(X)of


key: chroma:document
string_value:
positive polarity. If clause C1(X)deviates from all of the sub-patterns Q1
h(X),h∈{1,...n/ 2},
of the pattern recognition problem, it is either not a Nash equilibrium in the game or it is
produced by an action with negative expected payoﬀ.
Proof. There are three kinds of sub-optimal conﬁgurations for a clause: (i) The clause is too
infrequent; (ii) The clause is too frequent; or (iii) The clause outputs 1 for input vectors from


key: chroma:document
string_value:
X0. We must show that none of these fulﬁll both criteria of Lemma 3.
In case (i) there exists at least one literal lkthat has been included in the clause, for which
P(X1
jk∩X1)
P(X1)<1
s. That is, less than1
sofX1is covered by the clause, in frequency. It is suﬃcient
to show that the expected payoﬀ of Include (Eqn. 15) is negative for γ= 1:
s−1
s·P(X1
jk∩X1)−1
s·P(X1\X1
jk)<0. (25)
Dividing by P(X1) on each side we get:
s−1
s·P(X1
jk∩X1)
P(X1)−1
s·P(X1\X1
jk)
P(X1)<0. (26)
WithP(X1
jk∩X1)


key: chroma:document
string_value:
s·P(X1\X1
jk)
P(X1)<0. (26)
WithP(X1
jk∩X1)
P(X1)<1
s, we must haveP(X1\X1
jk)
P(X1)>s−1
s. We can thus conclude that the expected
payoﬀ of Include is negative.
We now turn to case (ii), that is, the case where the clause is too frequent. Then there
must exist an excluded literal lksuch thatP(X1
jk∩X1)
P(X1)>1
s. It is then suﬃcient to show that the
expected payoﬀ of Exclude is negative:
1
s·P(X1\X1
jk)
P(X1)−s−1
s·P(X1
jk∩X1)
P(X1)−P([X1
j\X1
jk]∩X0)
P(X1)<0. (27)
WithP(X1
jk∩X1)
P(X1)>1


key: chroma:document
string_value:
jk]∩X0)
P(X1)<0. (27)
WithP(X1
jk∩X1)
P(X1)>1
s, we haveP(X1\X1
jk)
P(X1)<s−1
s. Again, the left side is negative.
Apart from the sub-optimal conﬁgurations where too many or too few literals are included
in the clause, we must also consider case (iii), that is, scenarios where P([X1
j\X1
jk]∩X0)>0.
This means that the corresponding clause C1
j(X) outputs 1 for input vectors X∈[X1
j\X1
jk]∩X0.
Accordingly, it outputs 1 when P(y= 1|X)<P(y= 0|X), which is sub-optimal and violates


key: chroma:document
string_value:
Property 4 of Deﬁnition 1. It must then clearly exist a literal lkthat has been excluded
from the clause and that is zero for input vectors X∈[X1
j\X1
jk]∩X0. If not, an optimal
conﬁguration would not exist, which it does by problem deﬁnition. Including lkin the clause
would make the clause output 0 for input vectors X∈[X1
j\X1
jk]∩X0. This will thus eliminate
X∈[X1
j\X1
jk]∩X0from the input space captured by the clause. Accordingly, we must ensure


key: chroma:document
string_value:
that excluding lk= 0 gives a negative expected payoﬀ:
1
s·P(X1\X1
jk)−s−1
s·P(X1
jk∩X1)−P([X1
j\X1
jk]∩X0)<0. (28)
We can assume that1
s·P(X1\X1
jk) ands−1
s·P(X1
jk∩X1) are roughly equal. If they are not,
we have already determined in the previous two paragraphs that the resulting clause is not a
Nash equilibrium. Thus, what remains is to show that P([X1
j\X1
jk]∩X0)>0, which is stated
by case (iii) itself.
21


key: chroma:document
string_value:
Theorem 1. Consider a pattern recognition problem per Deﬁnition 1 without noise, P(y̸=
ω|Xω) = 1−γ= 0, and a Tsetlin Machine game per Deﬁnition 2 with a single clause C1(X)of
positive polarity. The sub-patterns Q1
h(X),h∈{1,...n/ 2},are then the only Nash equilibria
of the game, where the expected payoﬀ of each player action is positive, hence not rejected by
the Tsetlin Automata players.
Proof. This theorem follows from Lemma 3, Lemma 4 and Lemma 5.
4.6 Nash Equilibria with Static Noise


key: chroma:document
string_value:
4.6 Nash Equilibria with Static Noise
Lemma 6. Consider a pattern recognition problem per Deﬁnition 1 with static noise, P(y̸=
ω|Xω) = 1−γ,0.5< γ < 1.0, and output classes of equal size, P(X0) =P(X1) = 0.5.
Consider further a Tsetlin Machine game per Deﬁnition 2 with a single clause C1(X)of positive
polarity. Each sub-pattern Q1
h(X),h∈{1,...n/ 2},of the pattern recognition problem is a Nash
equilibrium in the game when sis scaled by a factor t,
b1
γ<t<b2
b2
b3(1−γ) + 2(γ−0.5), (29)


key: chroma:document
string_value:
b1
γ<t<b2
b2
b3(1−γ) + 2(γ−0.5), (29)
where,b2= min j,kP(X1)
P(X1
jk)·s,b2>1,andb3= max j,kP(X1)
P(X1
j)·s,0< b 3<1,deﬁne a worst-case
upper bound for twhen Exclude is optimal. Further, b1= max j,kP(X1)
P(X1
jk)·s,0< b1<1,deﬁnes
a worst-case lower-bound when Include is optimal. Additionally, the expected payoﬀ of each
player action in each equilibrium is positive, hence not rejected by the Tsetlin Automata players.


key: chroma:document
string_value:
As seen, increasing noise (reduced γ) tightens the lower bound, but loosens the upper.
Further, an increasing b2loosens the upper bound, while a decreasing b1loosens the lower
bound.
Proof. We ﬁrst analyse the case when Exclude is optimal, before addressing Include .
Exclude is Optimal. Since Exclude is optimal, we have (follows from Deﬁnition 1):
P(X1
jk∩X0) = 0,P([X1
j\X1
jk]∩X0) = 0,P(X1
jk∩X1) =P(X1
jk),P([X1
j\X1
jk]∩X1) =
P(X1
j\X1
jk), andP(X0\X1


key: chroma:document
string_value:
j\X1
jk]∩X1) =
P(X1
j\X1
jk), andP(X0\X1
jk) =P(X0). We consider Criterion 1 of Lemma 3 ﬁrst, i.e., that
the expected payoﬀ from Eqn. 12 is positive, which simpliﬁes to:
1
s·γ·P(X1\X1
jk) +1
s·(1−γ)·P(X0)−
s−1
s·γ·P(X1
jk)−(1−γ)·P(X1
j\X1
jk)>0. (30)
Dividing by P(X1) on each side, we get:
1
s·γ·P(X1\X1
jk)
P(X1)+1
s·(1−γ)·P(X0)
P(X1)−
s−1
s·γ·P(X1
jk)
P(X1)−(1−γ)·P(X1
j\X1
jk)
P(X1)>0. (31)
Clearly, to maintain positive expected payoﬀ after the introduction of static noise, we need to


key: chroma:document
string_value:
compensate for the addends introduced by the noise, i.e., the (1 −γ)-addends. To this end, we
artiﬁcially scale up the speciﬁcity parameter sby a factor t. Simplifying further, we replace
P(X0)
P(X1)with 1:
1
ts·γ·P(X1\X1
jk)
P(X1)+1
ts·(1−γ)−
ts−1
ts·γ·P(X1
jk)
P(X1)−(1−γ)·P(X1
j\X1
jk)
P(X1)>0. (32)
22


key: chroma:document
string_value:
Since we consider the case where excluding literal lkis optimal, we have b2>1 such that:
P(X1
jk)
P(X1)=1
b2s⇒b2=P(X1)
P(X1
jk)·s. That is, b2is an unknown factor that measures exactly how
much including lkconstrains the clause beyond1
s, according to Property 1 and Property 2 of
Deﬁnition 1. This in turn, means thatP(X1\X1
jk)
P(X1)= 1−P(X1
jk)
P(X1)=b2s−1
b2s. We ﬁnally introduce
b3,0<b3<1. The role of b3is to exactly determine how much excluding literal lkloosens up
the clause beyond1
s:P(X1


key: chroma:document
string_value:
the clause beyond1
s:P(X1
j)
P(X1)=1
b3s⇒b3=P(X1)
P(X1
j)·s. This leads toP(X1
j\X1
jk)
P(X1)=P(X1
j)
P(X1)−P(X1
jk)
P(X1)=
1
b3s−1
b2s. With these amendments, we revisit the condition that the expected payoﬀ of Exclude
must be positive:
1
ts·γ·b2s−1
b2s+1
ts·(1−γ)−ts−1
ts·γ·1
b2s−(1−γ)·(1
b3s−1
b2s)
>0.(33)
Through a sequence of standard algebraic operations, the above condition can be simpliﬁed as
follows:
γ·b2s−1
b2s+ (1−γ)−(ts−1)·γ·1
b2s−ts·(1−γ)·(1
b3s−1
b2s)
>0 (34)
γ·b2s−1


key: chroma:document
string_value:
b2s−ts·(1−γ)·(1
b3s−1
b2s)
>0 (34)
γ·b2s−1
b2s+ (1−γ)−(ts−1)·γ·1
b2s−ts·(1−γ)·b2−b3
b2b3s>0 (35)
γ·(b2s−1) +b2s·(1−γ)−(ts−1)·γ−ts·(1−γ)·b2−b3
b3>0 (36)
b2sγ−γ+b2s−b2sγ−tsγ+γ−ts·(1−γ)·b2−b3
b3>0 (37)
b2γ+b2−b2γ−tγ−t·(1−γ)·b2−b3
b3>0 (38)
b2γ+b2−b2γ−tγ−t·(1−γ)·(b2
b3−1)
>0 (39)
b2γ+b2−b2γ−tγ−(
t·(1−γ)·b2
b3−t·(1−γ))
>0 (40)
b2γ+b2−b2γ−tγ−t·(1−γ)·b2
b3+t·(1−γ)>0 (41)
b2−tγ−t·(1−γ)·b2
b3+t·(1−γ)>0 (42)
b2−tγ−b2
b3t+b2
b3tγ+t−tγ > 0 (43)
b2−t(
γ+b2
b3−b2
b3γ−1 +γ)
>0 (44)
b2
γ+b2
b3−b2


key: chroma:document
string_value:
b2−t(
γ+b2
b3−b2
b3γ−1 +γ)
>0 (44)
b2
γ+b2
b3−b2
b3γ−1 +γ> t (45)
b2
b2
b3(1−γ) + 2(γ−1
2)> t (46)
This means that it is possible to increase tup to a certain boundary, weakening reinforcement
ofExclude , without making the expected payoﬀ of Exclude turn negative.
We now show that the expected payoﬀ of Include is negative, fulﬁlling Criterion 2 of
Lemma 3:
s−1
s·γ·P(X1
jk∩X1) +s−1
s·(1−γ)·P(X1
jk∩X0)−
1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0\X1
jk)<0. (47)


key: chroma:document
string_value:
1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0\X1
jk)<0. (47)
Since Exclude is optimal, we can simplify as follows (per Deﬁnition 1):
s−1
s·γ·P(X1
jk)−1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0)<0. (48)
23


key: chroma:document
string_value:
Introducing the scaling factor t>1 forsand dividing by P(X1) we get:
ts−1
ts·γ·P(X1
jk)
P(X1)−1
ts·γ·P(X1\X1
jk)
P(X1)−1
ts·(1−γ)·P(X0)
P(X1)<0. (49)
Because we have established that Eqn. 32 is positive, it follows that the above equation is
negative. This is due to the fact Eqn. 32 is still positive after removing the last addend, and
the above equation is simply the negated version of the latter.
Include is Optimal. To fulﬁll Criterion 1 of Lemma 3 we must show that the expected


key: chroma:document
string_value:
payoﬀ of Include is positive:
s−1
s·γ·P(X1
jk∩X1) +s−1
s·(1−γ)·P(X1
jk∩X0)−
1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0\X1
jk)>0. (50)
Since Include is optimal, we have: P(X1
jk∩X0) = 0,P(X1
jk∩X1) =P(X1
jk) andP(X0\X1
jk) =
P(X0), again per Property 4 of Deﬁnition 1. Accordingly, we can simplify the above expression
to:
s−1
s·γ·P(X1
jk)−1
s·γ·P(X1\X1
jk)−1
s·(1−γ)·P(X0)>0. (51)
To compensate for the addends introduced by noise, we again increase the s-parameter by


key: chroma:document
string_value:
artiﬁcially multiplying it with a factor t>1. Dividing by P(X1) we get:
ts−1
ts·γ·P(X1
jk)
P(X1)−1
ts·γ·P(X1\X1
jk)
P(X1)−1
ts·(1−γ)·P(X0)
P(X1)>0. (52)
Now we introduce 0 < b1<1, which tells exactly how much largerP(X1
jk)
P(X1)is compared to1
s,
i.e.,P(X1
jk)
P(X1)=1
b1s⇒b1=P(X1)
P(X1
jk)·sas determined by Property 1 and Property 2 of Deﬁnition 1,
we can simplify the condition as follows, also replacingP(X0)
P(X1)with 1:
ts−1
ts·γ·1
b1s−1
ts·γ·b1s−1
b1s−1
ts·(1−γ)>0 (53)
ts−1
t·γ·1
b1−1


key: chroma:document
string_value:
ts·γ·b1s−1
b1s−1
ts·(1−γ)>0 (53)
ts−1
t·γ·1
b1−1
t·γ·b1s−1
b1−s
t·(1−γ)>0 (54)
(ts−1)·γ−γ·(b1s−1)−b1s·(1−γ)>0 (55)
tsγ−γ−b1sγ+γ−b1s+b1sγ > 0 (56)
tsγ−b1s > 0 (57)
b1
γ< t (58)
This leads to the following range for t, recovering a Nash equilibrium corresponding to the
sub-patterns of the pattern recognition problem:
b1
γ< t <b2
b2
b3(1−γ) + 2(γ−1
2)(59)
That is, scaling sby a factor tallows us to recover from some degree of static noise, as governed
byb1,b2,andb3.


key: chroma:document
string_value:
byb1,b2,andb3.
We ﬁnally need to verify that the expected payoﬀ of Exclude is negative, in order to fulﬁll
Criterion 2 of Lemma 3:
1
s·γ·P(X1\X1
jk) +1
s·(1−γ)·P(X0)−s−1
s·γ·P(X1
jk)−
(1−γ)·P([X1
j\X1
jk]∩X1)−γ·P([X1
j\X1
jk]∩X0)<0. (60)
24


key: chroma:document
string_value:
Again, introducing the scaling factor t>1 forsand dividing by P(X1) we get:
1
ts·γ·P(X1\X1
jk)
P(X1)+1
ts·(1−γ)·P(X0)
P(X1)−ts−1
ts·γ·P(X1
jk)
P(X1)−
(1−γ)·P([X1
j\X1
jk]∩X1)
P(X1)−γ·P([X1
j\X1
jk]∩X0)
P(X1)<0. (61)
Since we have established that Eqn. 52 is positive, it follows that the above equation is negative.
This is due to the above equation simply being the negated version of the latter, subtracting
two extra addends, clearly making it negative.


key: chroma:document
string_value:
two extra addends, clearly making it negative.
Lemma 7. Consider a pattern recognition problem per Deﬁnition 1 with static noise, P(y̸=
ω|Xω) = 1−γ,0.5< γ < 1.0, and output classes of equal size, P(y= 0) =P(y= 1) = 0.5.
Consider further a Tsetlin Machine game per Deﬁnition 2 with only a single clause C1(X)of
positive polarity. If clause C1(X)deviates from all of the sub-patterns Q1
h(X),h∈{1,...n/ 2},
of the pattern recognition problem, it is either not a Nash equilibrium in the game or it is


key: chroma:document
string_value:
produced by an action with negative payoﬀ.
Proof. For too infrequent or too frequent clauses, the proof follows the structure of the corre-
sponding proof for the noise-free cases (i) and (ii). However, we need to introduce the scaling
factort, as demonstrated above.
Case (iii) requires some more scrutiny. That is, we need to show that when P([X1
j\X1
jk]∩
X0)>0, the expected payoﬀ of Exclude is negative:
1
ts·γ·P(X1\X1
jk) +
1
ts·(1−γ)·P(X0)−
ts−1
ts·γ·P(X1
jk)−
(1−γ)·P([X1
j\X1
jk]∩X1)−


key: chroma:document
string_value:
ts−1
ts·γ·P(X1
jk)−
(1−γ)·P([X1
j\X1
jk]∩X1)−
γ·P([X1
j\X1
jk]∩X0)<0. (62)
Again we can assume thatts−1
ts·γ·P(X1
jk)−1
ts·γ·P(X1\X1
jk)−1
ts·(1−γ)·P(X0) is close
to zero. Otherwise, we did not have a Nash equilibrium in the ﬁrst place, due to case (i) and
case (ii). Thus, we only need to verify that:
(1−γ)·P([X1
j\X1
jk]∩X1) +γ·P([X1
j\X1
jk]∩X0)>0.
The above condition is clearly true because we have P([X1
j\X1
jk]∩X0)>0 by case (iii) itself.
Additionally, because we have noise, (1 −γ)·P([X1


key: chroma:document
string_value:
Additionally, because we have noise, (1 −γ)·P([X1
j\X1
jk]∩X1) is strictly positive as well.
Theorem 2. Consider a pattern recognition problem per Deﬁnition 1 with static noise, P(y̸=
ω|Xω) = 1−γ,0.5< γ < 1.0, and output classes of equal size, P(y= 0) =P(y= 1) = 0.5.
Consider further a Tsetlin Machine game per Deﬁnition 2 with only a single clause C1(X)of
positive polarity. The sub-patterns Q1
h(X),h∈{1,...n/ 2},of the pattern recognition problem


key: chroma:document
string_value:
are then the only Nash equilibria of the game, where the expected payoﬀ of each player action
is positive, hence not rejected by the Tsetlin Automata players.
Proof. This theorem follows from Lemma Lemma 3, Lemma 6, and Lemma 7.
4.7 Coordination of Multiple Tsetlin Automata Teams
Theorem 3. Consider a pattern recognition problem per Deﬁnition 1 without noise, P(y̸=
ω|Xω) = 1−γ= 0, and a Tsetlin Machine game per Deﬁnition 2 with T·nclauses. Then the


key: chroma:document
string_value:
only Nash equilibrium of the Tsetlin Machine game is a conﬁguration where TTsetlin Machine
clausesCω
j,ω∈{0,1},j∈{1,...,T·n/2}, duplicate each sub-pattern Qω
h,ω∈{0,1},h∈
{1,...,n/ 2}.
25


key: chroma:document
string_value:
Proof. We already known from Theorem 1 that if a clause Cω
jis not duplicating any of the sub-
patternsQω
h, the Tsetlin Automata of the clause are not in a Nash Equilibrium. We therefore
only consider the case where all clauses Cω
jare duplicating some sub-pattern Qω
h. Let us ﬁrst
consider positive polarity clauses ( ω= 1). Assume that more than Tof these clauses duplicate
a particular sub-pattern Q1
h. Then, clearly, there are fewer than Tclauses duplicating some
other sub-pattern Q1


key: chroma:document
string_value:
other sub-pattern Q1
h′,h′̸=h. The Tsetlin Automata that take part in duplicating Q1
hwill then
only receive Inaction-feedback when facing sub-pattern Q1
h, as seen from Eqn. 8. However,
when they observe sub-pattern Q1
h′they will receive Type I Feedback with probability larger
than 0. Type I Feedback pulls the Tsetlin Automata away from Q1
hand towards Q1
h′. Hence,
we do not have a Nash equilibrium. For the same reasons, we do not have a Nash equilibrium


key: chroma:document
string_value:
when more than Tnegative polarity clauses capture a sub-pattern Q0
hdue to Eqn. 9. When
there areTclauses per sub-pattern Qω
h, on the other hand, the whole system is in equilibrium
due to Theorem 1 and Eqns. 8-11.
The proof for static noise follows the above structure, however, relies on Theorem 2 instead.
The proof is left out here for the sake of brevity.
Remark 1. Note that a Tsetlin Machine in practice also learns robustly when the total


key: chroma:document
string_value:
number of clauses available is less than T·n. This is because of the gradually increasing
probability of receiving feedback the farther away one is from T/−T, which leads to a load
balancing eﬀect among the clauses. This eﬀect dynamically distributes the clauses among the
available sub-patterns.
Remark 2. A traditional Learning Automata approach to pattern recognition would use
classiﬁcation accuracy as payoﬀ (utility) function, to give feedback to the Learning Automata.


key: chroma:document
string_value:
In general, this leads to the Vanishing Signal to Noise Ratio Problem described in Section 1.3.
This is due to the relatively small eﬀect each single Learning Automaton has on overall classi-
ﬁcation accuracy, and due to the noise introduced by the random behaviour of the team as a
whole. In the Tsetlin Machine game, on the other hand, each single automaton gets feedback
directly from the value of its literal and the value of its clause, as laid out in Table 2 and Ta-


key: chroma:document
string_value:
ble 3. We will now show that this local feedback scheme can be a generalized ordinal potential
game [30], with classiﬁcation accuracy being the global payoﬀ (utility) function.
Theorem 4. LetMrefer to the payoﬀ matrix of the Tsetlin Machine game from Deﬁnition 2.
Further, let a′
ianda′′
irefer to the two actions available to a single automaton, while a−irefers
to the remaining actions in the action conﬁguration α. Finally, let the summation target Tbe1,


key: chroma:document
string_value:
only considering positive clauses. The resulting Tsetlin Machine game is a generalized ordinal
potential game with regards to classiﬁcation accuracy, under the constraints of Deﬁnition 1.
That is, we have: P(y= ˆy;a′
i,a−i)−P(y= ˆy;a′′
i,a−i)>0⇒M(a′
i,a−i)−M(a′′
i,a−i)>0.
Proof. Classiﬁcation accuracy is clearly maximised only when C1
j,j∈{1,...,n/ 2},are du-
plicatingQ1
h,h∈{1,...,n/ 2}. Any mismatch will either produce false positive output if the
Tsetlin Machine clauses C1


key: chroma:document
string_value:
Tsetlin Machine clauses C1
jare missing literals, or false negative output if not all of the sub-
patternsQ1
hare completely covered. Thus, during learning, there are two ways classiﬁcation
accuracy can increase by switching action from a′′
itoa′
i: either by a decrease in the rate of
false negatives, P(y= 1,ˆy= 0), or by a decrease in the rate of false positives, P(y= 0,ˆy= 1).
Let us ﬁrst consider decreasing P(y= 1,ˆy= 0). This can only be achieved by turning


key: chroma:document
string_value:
false negative output into true positive output. Clearly, for a particular input X, we have false
negative output if there exists an hsuch thatQ1
h(X) = 1, with C1
j(X) = 0 for all j. LetL1
j
be the literals of clause C1
jandK1
hbe the literals of Q1
h. We can correct the false negative
error if and only if we have ajandkso thatL1
j\{lk}⊆K1
h,lk/∈K1
h, implyinglk= 0. The
false negative error can then be corrected by excluding lkfrom clause C1
j. Examining Feedback
Type I forC1


key: chroma:document
string_value:
j. Examining Feedback
Type I forC1
j(X) = 0,lk= 0,in Table 2, which applies for y= 1, we observe that the expected
payoﬀ of excluding lkis larger that the expected payoﬀ of including it.
26


key: chroma:document
string_value:
We consider decreasing P(y= 0,ˆy= 1) next. This can be achieved by turning false
positive output into true negative output. Clearly, for a particular input X, we have false
positive output if and only if there exists a jsuch thatC1
j(X) = 1, with Q1
h(X) = 0 for all
h. LetL1
jbe the literals of clause C1
jandK1
hbe the literals of Q1
h. We can correct the false
positive output if and only if there exists only onejsuch thatC1
j(X) = 1, and if this particular
C1


key: chroma:document
string_value:
j(X) = 1, and if this particular
C1
j(X) has left out a literal lkof value 0, lk= 0,lk/∈L1
j. Then the false positive output can
be corrected by including lk= 0 inC1
j. Examining Feedback Type II for C1
j(X) = 1,lk= 0,in
Table 3, which applies for y= 0, we observe that the expected payoﬀ of including lkis larger
that the expected payoﬀ of excluding it.
Remark 4. The ordinal potential game is generalized because the Tsetlin Machine game


key: chroma:document
string_value:
matrixMrewards action switches that have no impact on classiﬁcation accuracy. That is,
we do not have: M(a′
i,a−i)−M(a′′
i,a−i)>0⇒P(y= ˆy;a′
i,a−i)−P(y= ˆy;a′′
i,a−i)>0.
In particular, consider false negative output that cannot be corrected by excluding a single
literal from a single clause. Yet, Mstill rewards excluding literals of value 0 ,lk= 0,when
C1
j(X) = 0, eventually correcting the false negative output. Similarly, when multiple clauses


key: chroma:document
string_value:
output 1, producing a false positive output, no single Include action can alone correct the
false positive output. Yet, including literals of value 0 is still rewarded in the game matrix
M, again, eventually ﬁxing the false positive output. Thus, in this sense, the Tsetlin Machine
game matrix has a longer planning horizon than just greedily optimizing classiﬁcation accuracy
in single steps. Note that these latter dynamics do not produce any additional Nash equilibria,


key: chroma:document
string_value:
as established by Theorem 3. This means that there is a one-to-one correspondence between
the equilibria in the potential game and in optimizing classiﬁcation accuracy.
Remark 5. As summarized in [3], the behaviour of ﬁnite state Learning Automata collectives
have been studied by several researchers, showing that rational behaviour can be obtained in
certain cases, when memory is inﬁnite and the game’s payoﬀ matrix is constrained [31, 15, 32,


key: chroma:document
string_value:
33, 34]. Proofs for variable structure Learning Automata are more general, on the other hand.
That is, for identical payoﬀ games, the Learning Automata collective converges to a Nash
equilibrium with probability arbitrarily close to unity [3]. Further, for zero-sum games with
a saddle point, there exists Learning Automata collectives that can obtain the Von Neumann
value [35, 36]. In the next section, we study the behaviour of Tsetlin Machine games empirically


key: chroma:document
string_value:
on various datasets, including the artiﬁcial Noisy XOR dataset where the optimal classiﬁcation
accuracy and the corresponding Nash equilibria are known.
5 Empirical Results
In this section, we evaluate the Tsetlin Machine empirically using ﬁve datasets:
•Binary Iris Dataset. This is the classical Iris Dataset, however, with features in binary
form.
•Binary Digits Dataset. This is the classical digits dataset, again with features in
binary form.


key: chroma:document
string_value:
binary form.
•Axis & Allies Board Game Dataset. This new dataset involves optimal move pre-
diction in a minimalistic, yet intricate, mini-game from the Axis & Allies board game.
•Noisy XOR Dataset with Non-informative Features. This artiﬁcial dataset is de-
signed to reveal particular ”blind zones” of pattern recognition algorithms. The dataset
captures the renowned XOR-relation. Furthermore, the dataset contains a large number


key: chroma:document
string_value:
of random non-informative features to measure susceptibility towards the curse of dimen-
sionality [37]. To examine robustness towards noise we have further randomly inverted
40% of the outputs.
27


key: chroma:document
string_value:
•MNIST Dataset. The MNIST dataset is a larger scale dataset used extensively to
benchmark machine learning algorithms. We have included this dataset to investigate the
scalability of the Tsetlin Machine, as well as the behaviour of longer learning processes.
For these datasets, we form ensembles of 50 to 1000 independent replications with diﬀerent
random number streams. We do this to minimize the variance of the reported results and to


key: chroma:document
string_value:
provide the foundation for a statistical analysis of the merits of the diﬀerent schemes evaluated.
Together with the Tsetlin Machine, we also evaluate several classical machine learning tech-
niques using the same random number streams. This includes Multilayer Perceptron Networks,
the Naive Bayes Classiﬁer, Support Vector Machines, and Logistic Regression. Where appro-
priate, the diﬀerent schemes are optimized by means of relatively light hyper-parameter grid


key: chroma:document
string_value:
searches. As an example, Figure 7 captures the impact the sparameter of the Tsetlin Ma-
chine has on mean accuracy, for the Noisy XOR Dataset. Each point in the plot measures the
mean accuracy of 100 diﬀerent replications of the XOR-experiment for a particular value of s.
Clearly, accuracy increases with sup to a certain point, before it degrades gradually. Based
on the plot, for the Noisy XOR-experiment, we decided to use an svalue of 3.9.
 0.5 0.6 0.7 0.8 0.9 1


key: chroma:document
string_value:
0.5 0.6 0.7 0.8 0.9 1
 1 1.5  2 2.5  3 3.5  4 4.5  5 5.5  6Accuracy
sTsetlin Machine
Figure 7: The mean accuracy of the Tsetlin Machine (y-axis) on the Noisy XOR Dataset for
diﬀerent values of the parameter s(x-axis).
5.1 The Binary Iris Dataset
We ﬁrst evaluate the Tsetlin Machine on the classical Iris dataset2. This dataset consists of
150 examples with four inputs (Sepal Length, Sepal Width, Petal Length and Petal Width),
and three possible outputs (Setosa, Versicolour, and Virginica).


key: chroma:document
string_value:
We increase the challenge by transforming the four input values into one consecutive se-
quence of 16 bits, four bits per ﬂoat. It is thus necessary to also learn how to segment the 16
2UCI Machine Learning Repository [https://archive.ics.uci.edu/ml/datasets/iris].
28


key: chroma:document
string_value:
bits into four partitions, and extract the numeric information. We refer to the new dataset as
the Binary Iris Dataset.
We partition this dataset into a training set and a test set, with 80 percent of the data
being used for training. We here randomly produce 1000 training and test data partitions. For
each ensemble, we also randomly reinitialize the competing algorithms, to gain information on
stability and robustness. The results are reported in Table 6.


key: chroma:document
string_value:
Technique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 95.0±0.2 86.7 100.0 80.0100.0
Naive Bayes 91.6±0.3 83.3 96.7 70.0100.0
Logistic Regression 92.6±0.2 86.7 100.0 76.7100.0
Multilayer Perceptron Networks 93.8±0.2 86.7 100.0 80.0100.0
SVM 93.6±0.3 86.7 100.0 76.7100.0
Table 6: The Binary Iris Dataset – accuracy on test data.
The Tsetlin Machine3used here employs 300 clauses, and uses an s-value of 3.0 and a


key: chroma:document
string_value:
summation target Tof 10. Furthermore, the individual Tsetlin Automata each has 100 states.
This Tsetlin Machine is run for 500 epochs, and it is the accuracy after the ﬁnal epoch that is
reported. Propositional formulas with higher test accuracy are often found in preceding epochs
because of the random exploration of the Tsetlin Machine. However, to avoid overﬁtting
to the test set by handpicking the best conﬁguration found, we instead simply use the last
conﬁguration produced.


key: chroma:document
string_value:
conﬁguration produced.
In Table 6, we list mean accuracy with 95% conﬁdence intervals, 5 and 95 percentiles, as
well as the minimum and maximum accuracy obtained, across the 1000 experiment runs we
executed. As seen, the Tsetlin Machine provides the highest mean accuracy. For the 95 %ile
scores, however, most of the schemes obtain 100% accuracy. This can be explained by the
small size of the test set, which merely contains 30 examples. Thus it is easier to stumble


key: chroma:document
string_value:
upon a random conﬁguration that happens to provide fault-free classiﬁcation. Since the test
set is merely a sample of the corresponding real-world problem, it is reasonable to assume that
higher mean accuracy translates to more robust performance overall.
The training set, on the other hand, reveals subtler diﬀerences between the schemes. The
results obtained on the training set are shown in Table 7. As seen, the SVM here provides the


key: chroma:document
string_value:
highest mean accuracy, while the Tsetlin Machine provides the second highest. However, the
large drop in accuracy from the training data to the test data for the SVM indicates overﬁtting
on the training data.
Technique Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 96.6±0.05 95.0 98.3 94.2 99.2
Naive Bayes 92.4±0.08 90.0 94.2 85.8 97.5
Logistic Regression 93.8±0.07 92.5 95.8 90.0 97.5
Multilayer Perceptron Network 95.0±0.07 93.3 96.7 92.5 98.3
SVM 96.7±0.05 95.8 98.3 95.8 99.2


key: chroma:document
string_value:
SVM 96.7±0.05 95.8 98.3 95.8 99.2
Table 7: The Binary Iris Dataset – accuracy on training data.
5.2 The Binary Digits Dataset
We next evaluate the Tsetlin Machine on the classical Pen-Based Recognition of Handwritten
Digits Dataset4. The original dataset consists of 250 handwritten digits from 44 diﬀerent
writers, for a total number of 10992 instances. We increase the challenge by removing the


key: chroma:document
string_value:
3In this experiment, we use a Multi-Class Tsetlin Machine, described in Section 6.1. We also apply Boosting
of True Positive Feedback to Include actions as described in Section 3.3.3.
4UCI Machine Learning Repository [http://archive.ics.uci.edu/ml/datasets/Pen-
Based+Recognition+of+Handwritten+Digits].
29


key: chroma:document
string_value:
individual pixel value structure, transforming the 64 diﬀerent input features into a sequence
of 192 bits, 3 bits per pixel. We refer to the modiﬁed dataset as the Binary Digits Dataset.
Again we partition the dataset into training and test sets, keeping 80 percent of the data for
training.
The Tsetlin Machine5used here contains 1000 clauses, and uses a speciﬁcity sof 3.0 and a
summation target Tof 10. Furthermore, the individual Tsetlin Automata each has 1000 states.


key: chroma:document
string_value:
The Tsetlin Machine is run for 300 epochs, and it is the accuracy after the ﬁnal epoch that is
reported.
Table 8 reports mean accuracy with 95% conﬁdence intervals, 5 and 95 percentiles, as well
as the minimum and maximum accuracy obtained, across the 100 experiment runs we executed.
As seen, the Tsetlin Machine again clearly provides the highest accuracy on average, also when
taking the 95% conﬁdence intervals into account. For this dataset, the Tsetlin Machine is


key: chroma:document
string_value:
also superior when it comes to the maximal accuracy found across the 100 replications of the
experiment, as well as for the 95 %ile results.
Technique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 95.7±0.2 93.9 97.2 92.5 98.1
Naive Bayes 91.3±0.3 88.9 93.6 87.2 94.4
Logistic Regression 94.0±0.2 91.9 95.8 90.8 96.9
Multilayer Perceptron Network 93.5±0.2 91.7 95.3 90.6 96.7
SVM 50.5±2.2 30.3 67.4 25.8 77.8
Table 8: The Binary Digits Dataset – accuracy on test data.


key: chroma:document
string_value:
Performing poor on the test data and well on the training data indicates susceptibility to
overﬁtting. Table 9 reveals that the other techniques, apart from the Naive Bayes Classiﬁer,
perform signiﬁcantly better on the training data, unable to transfer this performance to the
test data.
Table 1 visualizes one of the clauses produced by the Tsetlin Machine, capturing handwrit-
ten digit ’1’. As seen, the pattern should be relatively easy to interpret for humans compared


key: chroma:document
string_value:
to, e.g., a neural network.
Technique Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 100.0±0.01 99.9 100.0 99.8100.0
Naive Bayes 92.9±0.07 92.4 93.5 91.3 93.7
Logistic Regression 99.6±0.02 99.4 99.7 99.3 99.9
Multilayer Perceptron Network 100.0±0.0 100.0 100.0 100.0100.0
SVM 100.0±0.0 100.0 100.0 100.0100.0
Table 9: The Binary Digits Dataset – accuracy on training data.
5.3 The Axis & Allies Board Game Dataset


key: chroma:document
string_value:
5.3 The Axis & Allies Board Game Dataset
Besides the two classical datasets, we also have built a new dataset based on the board game
Axis & Allies6. We designed this dataset to exhibit intricate pattern structures, involving
optimal move prediction in a subgame of Axis & Allies . In Axis & Allies , every piece on the
board are potentially moved each turn. Additionally, new pieces are introduced throughout


key: chroma:document
string_value:
the game, as a result of earlier decisions. This arguably yields a larger search tree than the
ones we ﬁnd in Go and chess. Finally, the outcome of battles are determined by dice, rendering
the game stochastic.
The Axis & Allies Board Game Dataset consists of 10 000 board game positions, exempliﬁed
in Figure 8. Player 1 owns the ”Caucasus” territory in the ﬁgure, while Player 2 owns ”Ukraine”


key: chroma:document
string_value:
5In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1. We also apply Boosting
of True Positive Feedback to Include actions as described in Section 3.3.3.
6http://avalonhill.wizards.com/games/axis-and-allies
30


key: chroma:document
string_value:
and ”West Russia”. At start-up, each player is randomly assigned 0-10 tanks and 0-20 infantry
each. These units are the starting forces. For Player 2, the units are randomly distributed
among his two territories. The game consists of two rounds. First Player 1 attacks. This is
followed by a counter attack by Player 2. In order to win, Player 1 needs to capture both of
”Ukraine” and ”West Russia”. Player 2, on the other hand, only needs to take ”Caucasus”.
At Start Optimal Attack


key: chroma:document
string_value:
At Start Optimal Attack
Caucasus W. Russia Ukraine W. Russia Ukraine
Inf Tnk Inf Tnk Inf Tnk Inf Tnk Inf Tnk
16 4 11 4 5 4 0 0 3 4
19 3 6 1 6 3 7 2 12 1
9 1 1 3 0 5 0 0 0 0
Table 10: The Axis & Allies Board Game Dataset.
To produce the dataset, we built an Axis & Allies Board Game simulator. This allowed
us to ﬁnd the optimal attack for each assignment of starting forces. The resulting input and
output variables are shown in Table 10. The at start forces are to the left, while the optimal


key: chroma:document
string_value:
attack forces can be found to the right. In the ﬁrst row, for instance, it is optimal for Player
1 to launch a preemptive strike against the armor in Ukraine (armor is better oﬀensively than
defensively), to destroy oﬀensive power, while keeping the majority of forces for defense.
We use 25% of the data for training, and 75% for testing, randomly producing 100 diﬀerent
partitions of the dataset. The Tsetlin Machine employed here contains 10 000 clauses, and


key: chroma:document
string_value:
uses ans-value of 40 .0 and a summation target Tof 10. Furthermore, the individual Tsetlin
Automata each has 1000 states. The Tsetlin machine is run for 200 epochs, and it is the
accuracy after the ﬁnal epoch that is reported.
Table 11 reports the results from predicting output bit 5 among the 20 output bits (as
representative for all of the bits). In the table, we list mean accuracy with 95% conﬁdence
intervals, 5 and 95 percentiles, as well as the minimum and maximum accuracy obtained,


key: chroma:document
string_value:
across the 100 experiment runs we executed. As seen in the table, apparently only the Tsetlin
Technique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 87.7±0.0 87.4 88.0 87.2 88.1
Naive Bayes 80.1±0.0 80.1 80.1 80.1 80.1
Logistic Regression 77.7±0.0 77.7 77.7 77.7 77.7
Multilayer Perceptron Network 87.6±0.1 87.1 88.1 86.6 88.3
SVM 83.7±0.0 83.7 83.7 83.7 83.7
Random Forest 83.1±0.1 82.3 83.8 81.6 84.1
Table 11: The Axis & Allies Dataset – accuracy on test data.


key: chroma:document
string_value:
Machine and the neural network are capable of properly handling the complexity of the dataset,
providing statistically similar performance. The Tsetlin Machine is quite stable performance-
wise, while the neural network performance varies more.
However, the number of clauses needed to achieve the above performance is quite high
for the Tsetlin Machine, arguably due to its ﬂat one-layer architecture. Another reason that


key: chroma:document
string_value:
can explain the need for a large number of clauses can be the intricate nature of the mini-
game of Axis & Allies. Since we need an s-value as large as 40, clearly, some of the pertinent
sub-patterns must be quite ﬁne-grained. Because the s-value is global, all patterns, even the
coarser ones, must be learned at this ﬁne granularity. A possible next step in the research on
the Tsetlin Machine could therefore be to investigate the eﬀect of having clauses with diﬀerent


key: chroma:document
string_value:
s-values – some with smaller values for the rougher patterns, and some with larger values for
the ﬁner patterns.
As a ﬁnal observation, Table 12 reports performance on the training data. Random Forest
distinguishes itself by almost perfect predictions for the training data, thus clearly overﬁtting,
31


key: chroma:document
string_value:
but still performing well on the test set. The other techniques provide slightly improved
performance on the training data, as expected.
Technique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 96.2±0.1 95.7 96.8 95.5 97.0
Naive Bayes 81.2±0.0 81.2 81.2 81.2 81.2
Logistic Regression 78.8±0.0 78.8 78.8 78.8 78.8
Multilayer Perceptron Network 92.6±0.1 91.5 93.6 90.7 94.2
SVM 85.2±0.0 85.2 85.2 85.2 85.2
Random Forest 99.1±0.0 98.8 99.4 98.6 99.7


key: chroma:document
string_value:
Random Forest 99.1±0.0 98.8 99.4 98.6 99.7
Table 12: The Axis & Allies Dataset – accuracy on training data.
Figure 8: The Axis & Allies mini game.
5.4 The Noisy XOR Dataset with Non-informative Features
We now turn to an artiﬁcal dataset, constructed to uncover ”blind zones” caused by XOR-like
relations. Furthermore, the dataset contains a large number of random non-informative features
to measure susceptibility towards the curse of dimensionality [37]. To examine robustness


key: chroma:document
string_value:
towards noise, we have further randomly inverted 40% of the outputs.
x1x2x3x4x5x6x7x8x9x10x11x12y
0 1 0 1 1 0 1 1 0 1 1 0 1
1 1 1 0 1 0 1 1 0 0 1 1 0
0 0 1 1 0 1 1 1 1 0 1 0 0
1 1 1 0 1 1 1 0 1 1 0 0 1
Table 13: The Noisy XOR Dataset with Non-informative Features.
The dataset consists of 10 000 examples with twelve binary inputs, X= [x1,x2,...,x 12],
and a binary output, y. Ten of the inputs are completely random. The two remaining inputs,


key: chroma:document
string_value:
however, are related to the output ythrough an XOR-relation, y= XOR(xk1,xk2). Finally,
40% of the outputs are inverted. Table 13 shows four examples from the dataset, demonstrating
the high level of noise. We partition the dataset into training and test data, using 50% of the
data for training.
32


key: chroma:document
string_value:
The Tsetlin Machine7used here contains 20 clauses, and uses an s-value of 3.9 and a
summation target Tof 15. Furthermore, the individual Tsetlin Automata each has 100 states.
The Tsetlin Machine is run for 200 epochs, and it is the accuracy after the ﬁnal epoch, that
we report.
Table 14 contains four of the clauses produced by the Tsetlin Machine. Notice how the noisy
dataset from Table 13 has been turned into informative propositional formulas that capture
the structure of the dataset.


key: chroma:document
string_value:
the structure of the dataset.
The empirical results are found in Table 15. Again, we report mean accuracy with 95%
conﬁdence intervals, 5 and 95 percentiles, as well as the minimum and maximum accuracy
obtained, across the 100 replications of the experiment. Note that for the test data, the output
values are unperturbed. As seen, the XOR-relation, as expected, makes Logistic Regression
and the Naive Bayes Classiﬁer incapable of predicting the output value y, resorting to random


key: chroma:document
string_value:
guessing. Both the neural network and the Tsetlin Machine, on the other hand, see through
the noise and captures the underlying XOR pattern. SVM performs slightly better than the
Naive Bayes Classiﬁer and Logistic Regression, however, is clearly distracted by the added non-
informative features (the SVM performs much better with fewer non-informative features).
Figure 9 shows how accuracy degrades with less data, when we vary the dataset size from


key: chroma:document
string_value:
1000 examples to 20 000 examples. As expected, Naive Bayes and Logistic Regression guess
blindly for all the diﬀerent data sizes. The main observation, however, is that the accuracy
advantage the Tsetlin Machine has over neural networks increases with less training data.
Indeed, it turns out that the Tsetlin Machine performs robustly with small training data sets
in all of our experiments.
5.5 The MNIST Dataset


key: chroma:document
string_value:
in all of our experiments.
5.5 The MNIST Dataset
We next evaluate the Tsetlin Machine on the MNIST Dataset of Handwritten Digits8[38], also
investigating how learning progresses, epoch-by-epoch, in terms of accuracy. Note that the
experimental results reported here can be reproduced with the demo found at
https://github.com/cair/fast-tsetlin-machine-with-mnist-demo.
The original dataset consists of 60 000 training examples, and 10 000 test examples. We


key: chroma:document
string_value:
binarize this dataset by replacing pixel values larger than 0 .3 with 1. Pixel values below or
equal to 0.3 are replaced with 0.
The Tsetlin Machine9employed here contains 40 000 clauses, 4000 clauses per class, uses an
7In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1.
8http://www.pymvpa.org/datadb/mnist.html
9In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1. We also applied
No. Sign Clause Learned
1 +¬x1∧x2
2−¬x1∧¬x2


key: chroma:document
string_value:
No. Sign Clause Learned
1 +¬x1∧x2
2−¬x1∧¬x2
3 +x1∧¬x2
4− x1∧x2
Table 14: Example of four clauses composed by the Tsetlin Machine for the XOR Dataset with
Non-informative Features.
Technique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.
Tsetlin Machine 99.3±0.3 95.9 100.0 91.6100.0
Naive Bayes 49.8±0.2 48.3 51.0 41.3 52.7
Logistic Regression 49.8±0.3 47.8 51.1 41.1 53.1
Multilayer Perceptron Network 95.4±0.5 90.1 98.6 88.2 99.9
SVM 58.0±0.3 56.4 59.2 55.4 66.5


key: chroma:document
string_value:
SVM 58.0±0.3 56.4 59.2 55.4 66.5
Table 15: The Noisy XOR Dataset with Non-informative Features – accuracy on test data.
33


key: chroma:document
string_value:
0.5 0.6 0.7 0.8 0.9 1
 0  2000  4000  6000  8000  10000  12000  14000  16000  18000  20000Accuracy
# Training ExamplesTsetlin Machine
Neural Networks
SVM
Logistic Regression
Naive BayesFigure 9: Accuracy (y-axis) for the Noisy XOR Dataset for diﬀerent training dataset sizes
(x-axis).
 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
 0  50  100  150  200  250  300  350  400Accuracy
EpochTest Accuracy w/10000 Clauses, s=10.0, Threshold=25
Training Accuracy w/10000 Clauses, s=10.0, Threshold=25


key: chroma:document
string_value:
Test Accuracy w/20000 Clauses, s=10.0, Threshold=50
Training Accuracy w/20000 Clauses, s=10.0, Threshold=50
Test Accuracy w/40000 Clauses, s=10.0, Threshold=100
Training Accuracy w/40000 Clauses, s=10.0, Threshold=100
Figure 10: The mean test- and training accuracy per epoch for the Tsetlin Machine on the
MNIST Dataset.
s-value of 10 .0, and a summation target Tof 50. Furthermore, the individual Tsetlin Automata


key: chroma:document
string_value:
each has 256 states. The Tsetlin Machine is run for 400 epochs, and it is the accuracy after
the ﬁnal epoch that is reported.
Boosting of True Positive Feedback to Include actions, as described in Section 3.3.3.
34


key: chroma:document
string_value:
As seen in Figure 10, both mean test- and training accuracy increase almost monotonically
across the epochs, however, aﬀected by random ﬂuctuation. Perhaps most notably, while the
mean accuracy on the training data approaches 99 .9%, accuracy on the test data continues to
increase as well, hitting 98 .2% after 400 epochs. This is quite diﬀerent from what occurs with
back-propagation on a neural network, where accuracy on test data starts to drop at some


key: chroma:document
string_value:
point due to overﬁtting, without proper regularization mechanisms.
The ﬁgure also shows how varying the number of clauses and the summation target T
aﬀects accuracy and learning stability. With more clauses available to express patterns, in
combination with a higher summation target T, both learning speed, stability and accuracy
increases, however, at the expense of larger computational cost.
Technique Accuracy ( %)
2-layer NN, 800 HU, Cross-Entropy Loss 98.6
Tsetlin Machine (95 %ile) 98.3


key: chroma:document
string_value:
Tsetlin Machine (95 %ile) 98.3
Tsetlin Machine (Mean) 98.2±0.0
Tsetlin Machine (5 %ile) 98.1
K-nearest-neighbors, L3 97.2
3-layer NN, 500+150 hidden units 97.1
40 PCA + quadratic classiﬁer 96.7
1000 RBF + linear classiﬁer 96.4
Logistic regression 91.5
Linear classiﬁer (1-layer NN) 88.0
Decision tree 87.8
Multinomial Naive Bayes 83.2
Table 16: A comparison of vanilla machine learning algorithms with the Tsetlin Machine,
directly on the original unenhanced MNIST dataset (NN - Neural Network).


key: chroma:document
string_value:
Table 16 reports the mean accuracy of the Tsetlin Machine, across the 50 experiment runs
we executed. As points of reference, results for other well-known algorithms have been obtained
from http://yann.lecun.com/exdb/mnist/ and included in the table (in italic). Only the vanilla
version of the algorithms, that has been applied directly on unenhanced MNIST data, is in-
cluded here. The purpose of this selection is to strictly compare algorithmic performance.


key: chroma:document
string_value:
In other words, we do not consider the eﬀect of enhancing the dataset (e.g., warping, distor-
tion, deskewing), combining diﬀerent algorithms (e.g., neural network combined with nearest
neighbor, convolution schemes), or applying meta optimization techniques (boosting, ensemble
learning, etc.). With such techniques, it is possible to signiﬁcantly increase accuracy, with the
best currently reported results being an accuracy of 99 .79% [39]. Enhancing the vanilla Tsetlin


key: chroma:document
string_value:
Machine with such techniques is further work.
Additionally, as a further point of reference, we train and evaluate logistic regression, deci-
sion trees, and multinomial Naive Bayes on the binarized MNIST dataset used by the Tsetlin
Machine.
As seen in the table, the Tsetlin Machine provides competitive accuracy, outperforming
e.g. K-nearest neighbor and a 3-layer neural network. It is outperformed by a 2-layer neural


key: chroma:document
string_value:
network with 800 hidden nodes, using cross entropy loss. However, note that the Tsetlin
Machine operates upon the binarized MNIST data (the grey tone value of each pixel is either
set to 0 or 1), and thus has a disadvantage. Improved binarization techniques for the Tsetlin
Machine is further work.
6 The Tsetlin Machine as a Building Block in More Advanced
Architectures
We have designed the Tsetlin Machine to facilitate building of more advanced architectures.


key: chroma:document
string_value:
architectures.
6.1 The Multi-Class Tsetlin Machine
In some pattern recognition problems the task is to assign one of nclasses to each observed
patternX. That is, one needs to decide upon a single output value, y∈{1,...,m}. Such a
multi-class pattern recognition problem can be handled by the Tsetlin Machine by represent-
ingyas bits, using multiple outputs y. In this section, however, we present an alternative
architecture that addresses the multi-class pattern recognition problem more directly.


key: chroma:document
string_value:
………+-+-∑⌵⌵⌵⌵0 1 0 … 1 0 10/1∑0/11…n0/10/1Selected class
InputArgmax operatorSummationConjunctiveClausesArgmax
Figure 11: The Multi-Class Tsetlin Machine.
Figure 11 depicts the Multi-Class Tsetlin Machine10which replaces the threshold function
of each output y,i∈{1,...,m}, with a single argmax operator. With the argmax operator,
the indexiof the largest sum∑n/2
j=1C1,i
j(X)−∑n/2
j=1C0,i
j(X) is outputted as the ﬁnal output
of the Tsetlin Machine:
y= argmaxi=1,...,m
n/2∑
j=1C1,i
j(X)−n/2∑


key: chroma:document
string_value:
y= argmaxi=1,...,m
n/2∑
j=1C1,i
j(X)−n/2∑
j=1C0,i
j(X)
. (63)
Training is done as described in Section 3, apart from one critical modiﬁcation. Assume
we havey=ifor the current observation ( X,y). Then the Tsetlin Automata teams associated
with classiare trained as per y= 1 in the original Algorithm 1. Additionally, a random class
q̸=iis selected. The Tsetlin Automata teams of class qare then trained in accordance with
y= 0 in the original algorithm.


key: chroma:document
string_value:
y= 0 in the original algorithm.
6.2 The Fully Connected Deep Tsetlin Machine
Another architectural family is the Fully Connected Deep Tsetlin Machine [40], illustrated
in Figure 12. The purpose of this architecture is to build composite propositional formulas,
combining the propositional formula composed at one layer into more complex formula at the
next. As exempliﬁed in the ﬁgure, we here connect multiple Tsetlin Machines in a sequence.


key: chroma:document
string_value:
The clause output from each Tsetlin Machine in the sequence is provided as input to the
next Tsetlin Machine in the sequence. In this manner we build a multi-layered system. For
instance, if layer tproduces two clauses ( P∧¬Q) and (¬P∧Q), layert+ 1 can manipulate
these further, treating them as inputs. Layer t+ 1 could then form more complex formulas like
¬(P∧¬Q)∧(P∧¬Q), which can be rewritten as ( ¬P∨Q)∧(P∧¬Q).


key: chroma:document
string_value:
One simple approach for training such an architecture is indicated in the ﬁgure. As il-
lustrated, each layer is trained independently, directly from the output target y, exactly as
described in Section 3.3. The training procedure is thus similar to the strategy Hinton et al.
10An implementation of the Multi-Class Tsetlin Machine can be found at
https://github.com/cair/TsetlinMachine.
36


key: chroma:document
string_value:
………+-+-
∑⌵⌵⌵⌵0 1 0 … 1 0 10/1∑0/10/1
0/10/1
………+-+-
∑⌵⌵⌵⌵0/1∑0/10/1
0/10/10/1
0/1Figure 12: The fully connected Deep Tsetlin Machine.
used to train their pioneering Deep Belief Networks, layer-by-layer, in 2006 [41]. Such an ap-
proach can be eﬀective when each layer produces abstractions, in the form of clauses, that can
be taken advantage of in the following layer.
6.3 The Convolutional Tsetlin Machine
We next demonstrate how self-contained and independent Tsetlin Machines can interact to


key: chroma:document
string_value:
build a Convolutional Tsetlin Machine [42], illustrated in Figure 13. The Convolutional Tsetlin
Figure 13: The Convolutional Tsetlin Machine.
Machine is a deep architecture based on mathematical convolution, akin to Convolutional
Neural Networks [38]. For illustration purposes, consider 2D images of size 100 ×100 as input.
At the core of a Convolutional Tsetlin Machine we ﬁnd a kernel Tsetlin Machine with a small


key: chroma:document
string_value:
receptive ﬁeld. Each layer tof the Convolutional Tsetlin Machine operates as follows:
1. A convolution is performed over the input from the previous Tsetlin Machine layer,
producing one feature map per output y. Here, the Tsetlin Machine acts as a kernel
37


key: chroma:document
string_value:
in the convolution. In this manner, we reduce complexity by reusing the same Tsetlin
Machine across the whole image, focusing on a small image patch at a time.
2. The feature maps produced are then down-sampled using a pooling operator, in a similar
fashion as done in a Convolutional Neural Network, before the next layer and a new
Tsetlin Machine takes over. Here, the purpose of the pooling operation is to gradually
increase the abstraction level of the clauses, layer by layer.


key: chroma:document
string_value:
A simple approach for training a Convolutional Tsetlin Machine is indicated in the ﬁgure. In
brief, the feedback to the Tsetlin Machine kernel is directly provided from the desired end
outputy, exactly as described in Section 3. The only diﬀerence is the fact that the input to
layert+ 1 comes from the down-scaled feature map produced by layer t. Again, this is useful
when each layer produces abstractions, in the form of clauses, that can be taken advantage of
at the next layer.


key: chroma:document
string_value:
at the next layer.
6.4 The Recurrent Tsetlin Machine
The ﬁnal example is the Recurrent Tsetlin Machine [43] (Figure 14). In all brevity, the same
Tsetlin Machine is here reused from time step to time step. By taking the output from the
Tsetlin Machine of the previous time step as input, together with an external input from the
current time step, an inﬁnitely deep sequence of Tsetlin Machines is formed. This is quite


key: chroma:document
string_value:
similar to the family of Recurrent Neural Networks [44]. Again, the architecture can be trained
External inputMemory
Input from memoryTime step t
Time step t-1
External inputInput from memory
Figure 14: The Recurrent Tsetlin Machine.
layer by layer, directly from the target output y(t) of the current time step t. However, to learn
more advanced sequential patterns, there is a need for rewarding and penalizing that propagate


key: chroma:document
string_value:
back in time. How to design such a propagation scheme is presently an open research question.
7 Conclusion and Further Work
In this paper we proposed the Tsetlin Machine, an alternative to neural networks. The Tsetlin
Machine solves the vanishing signal-to-noise ratio of collectives of Tsetlin Automata. This
allows it to coordinate millions of Tsetlin Automata. By equipping teams of Tsetlin Automata
38


key: chroma:document
string_value:
with the ability to express patterns in propositional logic, we have enabled them to recognize
complex patterns. Furthermore, we proposed a novel decentralized feedback orchestration
mechanism. The mechanism is based on resource allocation principles, with the intent of
maximizing eﬀectiveness of sparse pattern recognition capacity. This mechanism eﬀectively
provides the Tsetlin Machine with the ability to capture unlabelled sub-patterns.


key: chroma:document
string_value:
Our theoretical analysis reveals that the Tsetlin Machine game have Nash equilibria that
maps to propositional formulas that maximize pattern recognition accuracy. In other words,
there are no local optima in the learning process, only global ones. This explains how the
collectives of Tsetlin Automata are able to accurately converge towards complex propositional
formulas that capture the essence of ﬁve diverse pattern recognition problems.


key: chroma:document
string_value:
Overall, the Tsetlin Machine seems to be particularly suited for digital computers, being
based on simple bit manipulation with AND-, OR-, and NOT gates. Both input, hidden
patterns, and output can be expressed as bit patterns.
In our empirical evaluations on ﬁve benchmarks, the Tsetlin Machine provided competitive
accuracy with respect to both Multilayer Perceptron Networks, Support Vector Machines,
Decision Trees, Random Forests, the Naive Bayes Classiﬁer and Logistic Regression. It further


key: chroma:document
string_value:
turns out that the Tsetlin Machine requires less data than neural networks, outperforming the
Naive Bayes Classiﬁer in data sparse environments.
By demonstrating that the longstanding problem of vanishing signal-to-noise ratio can be
solved, the Tsetlin Machine further provides a novel game theoretic framework for recasting the
problem of pattern recognition. This framework can thus provide opportunities for introducing


key: chroma:document
string_value:
bandit algorithms into large-scale pattern recognition. It could for instance be interesting to in-
vestigate the eﬀect of replacing the Tsetlin Automaton with alternative bandit algorithms, such
as algorithms based on Thompson Sampling [45, 46, 47, 48] or Upper Conﬁdence Bounds [49].
The more advanced Fully Connected Deep Tsetlin Machine, the Convolution Tsetlin Ma-
chine, and the Recurrent Tsetlin Machine architectures also form a starting point for further


key: chroma:document
string_value:
exploration. These architectures can potentially improve pattern representation compactness
and learning speed. However, it is currently unclear how these architectures can be most
eﬀectively trained.
Lastly, the high accuracy of the Tsetlin Machine, combined with its ability to produce self-
contained easy-to-interpret propositional formulas for pattern recognition, makes it attractive
for applied research, such as in the safety-critical medical domain.
Acknowledgements


key: chroma:document
string_value:
Acknowledgements
I thank my colleagues from the Centre for Artiﬁcial Intelligence Research (CAIR), Lei Jiao,
Xuan Zhang, Geir Thore Berge, Darshana Abeyrathna, Saeed Rahimi Gorji, Sondre Glimsdal,
Rupsa Saha, Bimal Bhattarai, Rohan K. Yadev, Bernt Viggo Matheussen, Morten Good-
win, Christian Omlin, Vladimir Zadorozhny (University of Pittsburgh), Jivitesh Sharma, and
Ahmed Abouzeid, for their contributions to the development of the Tsetlin machine family of


key: chroma:document
string_value:
techniques. I would also like to thank our House of CAIR partners, Alex Yakovlev, Rishad
Shaﬁk, Adrian Wheeldon, Jie Lei, Tousif Rahman (Newcastle University), Jonny Edwards
(Temporal Computing), Marco Wiering (University of Groningen), Christian D. Blakely (PwC
Switzerland), Adrian Phoulady, Anders Refsdal Olsen, Halvor Smørvik, and Erik Mathisen for
their many contributions.
Code Availability
Source code and datasets for the Tsetlin Machine, available under the MIT Licence, can be


key: chroma:document
string_value:
Data Availability
The datasets generated during and/or analysed during the current study are available from the
corresponding author on reasonable request.
References
[1] Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” pp. 436–444, 2015.
[2] M. L. Tsetlin, “On behaviour of ﬁnite automata in random medium,” Avtomat. i Telemekh ,
vol. 22, no. 10, pp. 1345–1354, 1961.
[3] K. S. Narendra and M. A. L. Thathachar, Learning Automata: An Introduction . Prentice-
Hall, Inc., 1989.


key: chroma:document
string_value:
Hall, Inc., 1989.
[4] H. Robbins, “Some aspects of the sequential design of experiments,” Bulletin of the Amer-
ican Mathematical Society , 1952.
[5] J. Gittins, “Bandit processes and dynamic allocation indices,” Journal of the Royal Sta-
tistical Society, Series B (Methodological) , vol. 41, no. 2, pp. 148–177, 1979.
[6] J. Carroll, Theory of Finite Automata With an Introduction to Formal Languages . Pren-
tice Hall, 1989.


key: chroma:document
string_value:
tice Hall, 1989.
[7] A. G. Barto and P. Anandan, “Pattern-recognizing stochastic learning automata,” IEEE
Transactions on Systems, Man, and Cybernetics , vol. 15, no. 3, pp. 360–375, 1985.
[8] P. Sastry and M. Thathachar, “Learning automata algorithms for pattern classiﬁcation,”
Sadhana , vol. 24, no. 4-5, pp. 261–292, 1999.
[9] S.-H. Zahiri, “Learning automata based classiﬁer,” Pattern Recognition Letters , vol. 29,
no. 1, pp. 40–48, 2008.


key: chroma:document
string_value:
no. 1, pp. 40–48, 2008.
[10] ——, “Classiﬁcation rule discovery using learning automata,” International Journal of
Machine Learning and Cybernetics , vol. 3, no. 3, pp. 205–213, 2012.
[11] P. Sastry, G. Nagendra, and N. Manwani, “A team of continuous-action learning automata
for noise-tolerant learning of half-spaces,” IEEE Transactions on Systems, Man, and Cy-
bernetics, Part B (Cybernetics) , vol. 40, no. 1, pp. 19–28, 2009.


key: chroma:document
string_value:
[12] M. Goodwin, A. Yazidi, and T. M. Jonassen, “Distributed learning automata for solving
a classiﬁcation task,” in 2016 IEEE congress on evolutionary computation (CEC) . IEEE,
2016, pp. 3999–4006.
[13] H. Motieghader, A. Najaﬁ, B. Sadeghi, and A. Masoudi-Nejad, “A hybrid gene selec-
tion algorithm for microarray cancer classiﬁcation using genetic algorithm and learning
automata,” Informatics in Medicine Unlocked , vol. 9, pp. 246–254, 2017.


key: chroma:document
string_value:
[14] M. A. L. Thathachar and P. S. Sastry, Networks of Learning Automata: Techniques for
Online Stochastic Optimization . Kluwer Academic Publishers, 2004.
[15] B. Tung and L. Kleinrock, “Using Finite State Automata to Produce Self-Optimization
and Self-Control,” IEEE Transactions on Parallel and Distributed Systems , vol. 7, no. 4,
pp. 47–61, 1996.
[16] B. J. Oommen, “Stochastic Searching on the Line and its Applications to Parameter Learn-


key: chroma:document
string_value:
ing in Nonlinear Optimization,” IEEE Transactions on Systems, Man, and Cybernetics,
Part B , vol. 27, no. 4, pp. 733–739, 1997.
40


key: chroma:document
string_value:
[17] B. J. Oommen and D. C. Ma, “Deterministic Learning Automata Solutions to The
Equipartitioning Problem,” IEEE Transactions on Computers , vol. 37, no. 1, pp. 2–13,
1988.
[18] M. Ghavipour and M. R. Meybodi, “A streaming sampling algorithm for social activity
networks using ﬁxed structure learning automata,” Applied Intelligence , 2018.
[19] A. Yazidi and B. John Oommen, “On the analysis of a random walk-jump chain with tree-


key: chroma:document
string_value:
based transitions and its applications to faulty dichotomous search,” Sequential Analysis ,
vol. 37, pp. 31–46, jan 2018.
[20] J. Zhang, Y. Wang, C. Wang, and M. Zhou, “Symmetrical Hierarchical Stochastic Search-
ing on the Line in Informative and Deceptive Environments,” IEEE Transactions on Cy-
bernetics , vol. 47, no. 3, pp. 626 – 635, jul 2016.
[21] B. J. Oommen, S. Misra, and O.-C. Granmo, “Routing bandwidth-guaranteed paths in


key: chroma:document
string_value:
MPLS traﬃc engineering: a multiple race track learning approach,” Computers, IEEE
Transactions on , vol. 56, no. 7, pp. 959–976, 2007.
[22] O.-C. Granmo and S. Glimsdal, “Accelerated Bayesian learning for decentralized two-
armed bandit based decision making with applications to the Goore Game,” Applied In-
telligence , vol. 38, no. 4, 2013.
[23] J. Von Neumann and O. Morgenstern, Theory of Games and Economic Behavior , 1947.


key: chroma:document
string_value:
[24] O.-C. Granmo, B. J. Oommen, S. A. Myrer, and M. G. Olsen, “Learning Automata-based
Solutions to the Nonlinear Fractional Knapsack Problem withApplications to Optimal
Resource Allocation,” IEEE Transactions on Systems, Man, and Cybernetics, Part B ,
vol. 37, no. 1, pp. 166–175, 2007.
[25] V. Haugland, M. Kjølleberg, S.-E. Larsen, and O.-C. Granmo, “A two-armed bandit
collective for hierarchical examplar based mining of frequent itemsets with applications


key: chroma:document
string_value:
to intrusion detection,” Transactions on Computational Collective Intelligence XIV , vol.
8615, pp. 1–19, 2014.
[26] Guolong Su, Kush R. Varshney, and Dmitry M. Malioutov, “Interpretable Two-Level
Boolean Rule Learning for Classiﬁcation,” in ICML Workshop on Human Interpretability
in Machine Learning (WHI 2016) , 2016, pp. 66–70.
[27] T. Wang, C. Rudin, F. Doshi-Velez, Y. Liu, E. Jones, E. Klampﬂ, P. Macneille, and


key: chroma:document
string_value:
M. Gupta, “A Bayesian Framework for Learning Rule Sets for Interpretable Classiﬁcation,”
Journal of Machine Learning Research , 2017.
[28] L. G. Valiant, “A Theory of the Learnable,” Communications of the ACM , vol. 27, no. 11,
pp. 1134–1142, 1984.
[29] O.-C. Granmo, B. J. Oommen, S. A. Myrer, and M. G. Olsen, “Learning Automata-based
Solutions to the Nonlinear Fractional Knapsack Problem with Applications to Optimal


key: chroma:document
string_value:
Resource Allocation,” IEEE Transactions on Systems, Man, and Cybernetics, Part B ,
vol. 37, no. 1, pp. 166–175, 2007.
[30] D. Monderer and L. Shapley, “Potential games,” Games and Economic Behavior , vol. 14,
no. 1, pp. 124–143, 1996. [Online]. Available: https://EconPapers.repec.org/RePEc:eee:
gamebe:v:14:y:1996:i:1:p:124-143
[31] V. U. Krylov and M. Tsetlin, “Games Between Automata,” Automation and Remote
Control , vol. 24, pp. 889–899, 1963.
41


key: chroma:document
string_value:
[32] V. A. Borovikov and V. I. Bryzgalov, “The Simplest Symmetric Game of Many Automata,”
Avtomatika i Telemekhanika , vol. 26, pp. 637–687, 1965.
[33] V. A. Volkonskii, “Asymptotic Properties of the Behavior of Elementary Automata in a
Game,” Problemy Peredachi Informatsii , vol. 1, pp. 36–53, 1965.
[34] B. G. Pittel, “The Asymptotic Properties of One Form of Goore Game,” Problemy
Peredachi Informatsii , vol. 1, pp. 99–112, 1965.


key: chroma:document
string_value:
Peredachi Informatsii , vol. 1, pp. 99–112, 1965.
[35] R. Viswanathan and K. S. Narendra, “Games of stochastic automata,” IEEE Transactions
on Systems, Man, and Cybernetics , vol. SMC-4, no. 1, pp. 131–135, 1974.
[36] K. S. Narendra, “Competitive and cooperative games of variable-structure stochastic au-
tomata,” Journal of Cybernetics , vol. 3, no. 1, pp. 1–23, 1973.
[37] R. Duda, P. Hart, and D. Stork, Pattern Classiﬁcation , 2nd ed. New York, NY: John
Wiley and Sons, Inc., 2000.


key: chroma:document
string_value:
Wiley and Sons, Inc., 2000.
[38] Y. LeCun, L. Bottou, Y. Bengio, and P. Haﬀner, “Gradient-based learning applied to
document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278 – 2324, 1998.
[39] L. Wan, M. Zeiler, S. Zhang, Y. LeCun, and R. Fergus, “Regularization of neural networks
using dropconnect,” International Conference on Machine Learning (ICML) , 2013.
[40] O.-C. Granmo, “The Fully Connected Deep Tsetlin Machine,” In Preparation , 2019.


key: chroma:document
string_value:
[41] G. E. Hinton, S. Osindero, and Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief
Nets,” Neural Computation , vol. 18, no. 7, pp. 1527 – 1554, 2006.
[42] O.-C. Granmo, “The Convolutional Tsetlin Machine,” In Preparation , 2019.
[43] ——, “The Recurrent Tsetlin Machine,” In Preparation , 2019.
[44] J. Schmidhuber, “Deep Learning in neural networks: An overview,” Neural Networks ,
vol. 61, pp. 85–117, 2015.


key: chroma:document
string_value:
vol. 61, pp. 85–117, 2015.
[45] W. R. Thompson, “On the likelihood that one unknown probability exceeds another in
view of the evidence of two samples,” Biometrika , vol. 25, no. 3/4, pp. 285–294, 1933.
[46] O.-C. Granmo, “Solving Two-Armed Bernoulli Bandit Problems Using a Bayesian Learn-
ing Automaton,” International Journal of Intelligent Computing and Cybernetics , vol. 3,
no. 2, pp. 207–234, 2010.
[47] B. C. May, N. Korda, A. Lee, and D. S. Leslie, “Optimistic Bayesian sampling in


