# embeddings_queue

created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47bc-934e-11ee-bb25-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 28, "chroma:document": "highest mean accuracy, while the Tsetlin Machine provides the second highest. However, the\nlarge drop in accuracy from the training data to the test data for the SVM indicates over\ufb01tting\non the training data.\nTechnique Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 96.6\u00b10.05 95.0 98.3 94.2 99.2\nNaive Bayes 92.4\u00b10.08 90.0 94.2 85.8 97.5\nLogistic Regression 93.8\u00b10.07 92.5 95.8 90.0 97.5\nMultilayer Perceptron Network 95.0\u00b10.07 93.3 96.7 92.5 98.3\nSVM 96.7\u00b10.05 95.8 98.3 95.8 99.2"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47bd-934e-11ee-8cdb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 28, "chroma:document": "SVM 96.7\u00b10.05 95.8 98.3 95.8 99.2\nTable 7: The Binary Iris Dataset \u2013 accuracy on training data.\n5.2 The Binary Digits Dataset\nWe next evaluate the Tsetlin Machine on the classical Pen-Based Recognition of Handwritten\nDigits Dataset4. The original dataset consists of 250 handwritten digits from 44 di\ufb00erent\nwriters, for a total number of 10992 instances. We increase the challenge by removing the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47be-934e-11ee-bf1f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 28, "chroma:document": "3In this experiment, we use a Multi-Class Tsetlin Machine, described in Section 6.1. We also apply Boosting\nof True Positive Feedback to Include actions as described in Section 3.3.3.\n4UCI Machine Learning Repository [http://archive.ics.uci.edu/ml/datasets/Pen-\nBased+Recognition+of+Handwritten+Digits].\n29"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47bf-934e-11ee-9515-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "individual pixel value structure, transforming the 64 di\ufb00erent input features into a sequence\nof 192 bits, 3 bits per pixel. We refer to the modi\ufb01ed dataset as the Binary Digits Dataset.\nAgain we partition the dataset into training and test sets, keeping 80 percent of the data for\ntraining.\nThe Tsetlin Machine5used here contains 1000 clauses, and uses a speci\ufb01city sof 3.0 and a\nsummation target Tof 10. Furthermore, the individual Tsetlin Automata each has 1000 states."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c0-934e-11ee-90a6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "The Tsetlin Machine is run for 300 epochs, and it is the accuracy after the \ufb01nal epoch that is\nreported.\nTable 8 reports mean accuracy with 95% con\ufb01dence intervals, 5 and 95 percentiles, as well\nas the minimum and maximum accuracy obtained, across the 100 experiment runs we executed.\nAs seen, the Tsetlin Machine again clearly provides the highest accuracy on average, also when\ntaking the 95% con\ufb01dence intervals into account. For this dataset, the Tsetlin Machine is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c1-934e-11ee-91aa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "also superior when it comes to the maximal accuracy found across the 100 replications of the\nexperiment, as well as for the 95 %ile results.\nTechnique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 95.7\u00b10.2 93.9 97.2 92.5 98.1\nNaive Bayes 91.3\u00b10.3 88.9 93.6 87.2 94.4\nLogistic Regression 94.0\u00b10.2 91.9 95.8 90.8 96.9\nMultilayer Perceptron Network 93.5\u00b10.2 91.7 95.3 90.6 96.7\nSVM 50.5\u00b12.2 30.3 67.4 25.8 77.8\nTable 8: The Binary Digits Dataset \u2013 accuracy on test data."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c2-934e-11ee-ab2b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "Performing poor on the test data and well on the training data indicates susceptibility to\nover\ufb01tting. Table 9 reveals that the other techniques, apart from the Naive Bayes Classi\ufb01er,\nperform signi\ufb01cantly better on the training data, unable to transfer this performance to the\ntest data.\nTable 1 visualizes one of the clauses produced by the Tsetlin Machine, capturing handwrit-\nten digit \u20191\u2019. As seen, the pattern should be relatively easy to interpret for humans compared"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c3-934e-11ee-800c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "to, e.g., a neural network.\nTechnique Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 100.0\u00b10.01 99.9 100.0 99.8100.0\nNaive Bayes 92.9\u00b10.07 92.4 93.5 91.3 93.7\nLogistic Regression 99.6\u00b10.02 99.4 99.7 99.3 99.9\nMultilayer Perceptron Network 100.0\u00b10.0 100.0 100.0 100.0100.0\nSVM 100.0\u00b10.0 100.0 100.0 100.0100.0\nTable 9: The Binary Digits Dataset \u2013 accuracy on training data.\n5.3 The Axis & Allies Board Game Dataset"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c4-934e-11ee-bb42-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "5.3 The Axis & Allies Board Game Dataset\nBesides the two classical datasets, we also have built a new dataset based on the board game\nAxis & Allies6. We designed this dataset to exhibit intricate pattern structures, involving\noptimal move prediction in a subgame of Axis & Allies . In Axis & Allies , every piece on the\nboard are potentially moved each turn. Additionally, new pieces are introduced throughout"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c5-934e-11ee-9c39-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "the game, as a result of earlier decisions. This arguably yields a larger search tree than the\nones we \ufb01nd in Go and chess. Finally, the outcome of battles are determined by dice, rendering\nthe game stochastic.\nThe Axis & Allies Board Game Dataset consists of 10 000 board game positions, exempli\ufb01ed\nin Figure 8. Player 1 owns the \u201dCaucasus\u201d territory in the \ufb01gure, while Player 2 owns \u201dUkraine\u201d"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c6-934e-11ee-b44e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 29, "chroma:document": "5In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1. We also apply Boosting\nof True Positive Feedback to Include actions as described in Section 3.3.3.\n6http://avalonhill.wizards.com/games/axis-and-allies\n30"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c7-934e-11ee-8411-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "and \u201dWest Russia\u201d. At start-up, each player is randomly assigned 0-10 tanks and 0-20 infantry\neach. These units are the starting forces. For Player 2, the units are randomly distributed\namong his two territories. The game consists of two rounds. First Player 1 attacks. This is\nfollowed by a counter attack by Player 2. In order to win, Player 1 needs to capture both of\n\u201dUkraine\u201d and \u201dWest Russia\u201d. Player 2, on the other hand, only needs to take \u201dCaucasus\u201d.\nAt Start Optimal Attack"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c8-934e-11ee-acca-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "At Start Optimal Attack\nCaucasus W. Russia Ukraine W. Russia Ukraine\nInf Tnk Inf Tnk Inf Tnk Inf Tnk Inf Tnk\n16 4 11 4 5 4 0 0 3 4\n19 3 6 1 6 3 7 2 12 1\n9 1 1 3 0 5 0 0 0 0\nTable 10: The Axis & Allies Board Game Dataset.\nTo produce the dataset, we built an Axis & Allies Board Game simulator. This allowed\nus to \ufb01nd the optimal attack for each assignment of starting forces. The resulting input and\noutput variables are shown in Table 10. The at start forces are to the left, while the optimal"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47c9-934e-11ee-b513-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "attack forces can be found to the right. In the \ufb01rst row, for instance, it is optimal for Player\n1 to launch a preemptive strike against the armor in Ukraine (armor is better o\ufb00ensively than\ndefensively), to destroy o\ufb00ensive power, while keeping the majority of forces for defense.\nWe use 25% of the data for training, and 75% for testing, randomly producing 100 di\ufb00erent\npartitions of the dataset. The Tsetlin Machine employed here contains 10 000 clauses, and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ca-934e-11ee-806d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "uses ans-value of 40 .0 and a summation target Tof 10. Furthermore, the individual Tsetlin\nAutomata each has 1000 states. The Tsetlin machine is run for 200 epochs, and it is the\naccuracy after the \ufb01nal epoch that is reported.\nTable 11 reports the results from predicting output bit 5 among the 20 output bits (as\nrepresentative for all of the bits). In the table, we list mean accuracy with 95% con\ufb01dence\nintervals, 5 and 95 percentiles, as well as the minimum and maximum accuracy obtained,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47cb-934e-11ee-ae47-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "across the 100 experiment runs we executed. As seen in the table, apparently only the Tsetlin\nTechnique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 87.7\u00b10.0 87.4 88.0 87.2 88.1\nNaive Bayes 80.1\u00b10.0 80.1 80.1 80.1 80.1\nLogistic Regression 77.7\u00b10.0 77.7 77.7 77.7 77.7\nMultilayer Perceptron Network 87.6\u00b10.1 87.1 88.1 86.6 88.3\nSVM 83.7\u00b10.0 83.7 83.7 83.7 83.7\nRandom Forest 83.1\u00b10.1 82.3 83.8 81.6 84.1\nTable 11: The Axis & Allies Dataset \u2013 accuracy on test data."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47cc-934e-11ee-a0b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "Machine and the neural network are capable of properly handling the complexity of the dataset,\nproviding statistically similar performance. The Tsetlin Machine is quite stable performance-\nwise, while the neural network performance varies more.\nHowever, the number of clauses needed to achieve the above performance is quite high\nfor the Tsetlin Machine, arguably due to its \ufb02at one-layer architecture. Another reason that"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47cd-934e-11ee-a35f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "can explain the need for a large number of clauses can be the intricate nature of the mini-\ngame of Axis & Allies. Since we need an s-value as large as 40, clearly, some of the pertinent\nsub-patterns must be quite \ufb01ne-grained. Because the s-value is global, all patterns, even the\ncoarser ones, must be learned at this \ufb01ne granularity. A possible next step in the research on\nthe Tsetlin Machine could therefore be to investigate the e\ufb00ect of having clauses with di\ufb00erent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ce-934e-11ee-a975-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 30, "chroma:document": "s-values \u2013 some with smaller values for the rougher patterns, and some with larger values for\nthe \ufb01ner patterns.\nAs a \ufb01nal observation, Table 12 reports performance on the training data. Random Forest\ndistinguishes itself by almost perfect predictions for the training data, thus clearly over\ufb01tting,\n31"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47cf-934e-11ee-b0e8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 31, "chroma:document": "but still performing well on the test set. The other techniques provide slightly improved\nperformance on the training data, as expected.\nTechnique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 96.2\u00b10.1 95.7 96.8 95.5 97.0\nNaive Bayes 81.2\u00b10.0 81.2 81.2 81.2 81.2\nLogistic Regression 78.8\u00b10.0 78.8 78.8 78.8 78.8\nMultilayer Perceptron Network 92.6\u00b10.1 91.5 93.6 90.7 94.2\nSVM 85.2\u00b10.0 85.2 85.2 85.2 85.2\nRandom Forest 99.1\u00b10.0 98.8 99.4 98.6 99.7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d0-934e-11ee-a4a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 31, "chroma:document": "Random Forest 99.1\u00b10.0 98.8 99.4 98.6 99.7\nTable 12: The Axis & Allies Dataset \u2013 accuracy on training data.\nFigure 8: The Axis & Allies mini game.\n5.4 The Noisy XOR Dataset with Non-informative Features\nWe now turn to an arti\ufb01cal dataset, constructed to uncover \u201dblind zones\u201d caused by XOR-like\nrelations. Furthermore, the dataset contains a large number of random non-informative features\nto measure susceptibility towards the curse of dimensionality [37]. To examine robustness"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d1-934e-11ee-93a3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 31, "chroma:document": "towards noise, we have further randomly inverted 40% of the outputs.\nx1x2x3x4x5x6x7x8x9x10x11x12y\n0 1 0 1 1 0 1 1 0 1 1 0 1\n1 1 1 0 1 0 1 1 0 0 1 1 0\n0 0 1 1 0 1 1 1 1 0 1 0 0\n1 1 1 0 1 1 1 0 1 1 0 0 1\nTable 13: The Noisy XOR Dataset with Non-informative Features.\nThe dataset consists of 10 000 examples with twelve binary inputs, X= [x1,x2,...,x 12],\nand a binary output, y. Ten of the inputs are completely random. The two remaining inputs,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d2-934e-11ee-89ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 31, "chroma:document": "however, are related to the output ythrough an XOR-relation, y= XOR(xk1,xk2). Finally,\n40% of the outputs are inverted. Table 13 shows four examples from the dataset, demonstrating\nthe high level of noise. We partition the dataset into training and test data, using 50% of the\ndata for training.\n32"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d3-934e-11ee-94bd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "The Tsetlin Machine7used here contains 20 clauses, and uses an s-value of 3.9 and a\nsummation target Tof 15. Furthermore, the individual Tsetlin Automata each has 100 states.\nThe Tsetlin Machine is run for 200 epochs, and it is the accuracy after the \ufb01nal epoch, that\nwe report.\nTable 14 contains four of the clauses produced by the Tsetlin Machine. Notice how the noisy\ndataset from Table 13 has been turned into informative propositional formulas that capture\nthe structure of the dataset."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d4-934e-11ee-8e28-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "the structure of the dataset.\nThe empirical results are found in Table 15. Again, we report mean accuracy with 95%\ncon\ufb01dence intervals, 5 and 95 percentiles, as well as the minimum and maximum accuracy\nobtained, across the 100 replications of the experiment. Note that for the test data, the output\nvalues are unperturbed. As seen, the XOR-relation, as expected, makes Logistic Regression\nand the Naive Bayes Classi\ufb01er incapable of predicting the output value y, resorting to random"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d5-934e-11ee-ad38-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "guessing. Both the neural network and the Tsetlin Machine, on the other hand, see through\nthe noise and captures the underlying XOR pattern. SVM performs slightly better than the\nNaive Bayes Classi\ufb01er and Logistic Regression, however, is clearly distracted by the added non-\ninformative features (the SVM performs much better with fewer non-informative features).\nFigure 9 shows how accuracy degrades with less data, when we vary the dataset size from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d6-934e-11ee-9943-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "1000 examples to 20 000 examples. As expected, Naive Bayes and Logistic Regression guess\nblindly for all the di\ufb00erent data sizes. The main observation, however, is that the accuracy\nadvantage the Tsetlin Machine has over neural networks increases with less training data.\nIndeed, it turns out that the Tsetlin Machine performs robustly with small training data sets\nin all of our experiments.\n5.5 The MNIST Dataset"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d7-934e-11ee-a6a8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "in all of our experiments.\n5.5 The MNIST Dataset\nWe next evaluate the Tsetlin Machine on the MNIST Dataset of Handwritten Digits8[38], also\ninvestigating how learning progresses, epoch-by-epoch, in terms of accuracy. Note that the\nexperimental results reported here can be reproduced with the demo found at\nhttps://github.com/cair/fast-tsetlin-machine-with-mnist-demo.\nThe original dataset consists of 60 000 training examples, and 10 000 test examples. We"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d8-934e-11ee-8e7e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "binarize this dataset by replacing pixel values larger than 0 .3 with 1. Pixel values below or\nequal to 0.3 are replaced with 0.\nThe Tsetlin Machine9employed here contains 40 000 clauses, 4000 clauses per class, uses an\n7In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1.\n8http://www.pymvpa.org/datadb/mnist.html\n9In this experiment, we used a Multi-Class Tsetlin Machine, described in Section 6.1. We also applied\nNo. Sign Clause Learned\n1 +\u00acx1\u2227x2\n2\u2212\u00acx1\u2227\u00acx2"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47d9-934e-11ee-82f6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "No. Sign Clause Learned\n1 +\u00acx1\u2227x2\n2\u2212\u00acx1\u2227\u00acx2\n3 +x1\u2227\u00acx2\n4\u2212 x1\u2227x2\nTable 14: Example of four clauses composed by the Tsetlin Machine for the XOR Dataset with\nNon-informative Features.\nTechnique/Accuracy ( %) Mean 5 %ile 95 %ileMin. Max.\nTsetlin Machine 99.3\u00b10.3 95.9 100.0 91.6100.0\nNaive Bayes 49.8\u00b10.2 48.3 51.0 41.3 52.7\nLogistic Regression 49.8\u00b10.3 47.8 51.1 41.1 53.1\nMultilayer Perceptron Network 95.4\u00b10.5 90.1 98.6 88.2 99.9\nSVM 58.0\u00b10.3 56.4 59.2 55.4 66.5"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47da-934e-11ee-9e48-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 32, "chroma:document": "SVM 58.0\u00b10.3 56.4 59.2 55.4 66.5\nTable 15: The Noisy XOR Dataset with Non-informative Features \u2013 accuracy on test data.\n33"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47db-934e-11ee-9a58-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 33, "chroma:document": "0.5 0.6 0.7 0.8 0.9 1\n 0  2000  4000  6000  8000  10000  12000  14000  16000  18000  20000Accuracy\n# Training ExamplesTsetlin Machine\nNeural Networks\nSVM\nLogistic Regression\nNaive BayesFigure 9: Accuracy (y-axis) for the Noisy XOR Dataset for di\ufb00erent training dataset sizes\n(x-axis).\n 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1\n 0  50  100  150  200  250  300  350  400Accuracy\nEpochTest Accuracy w/10000 Clauses, s=10.0, Threshold=25\nTraining Accuracy w/10000 Clauses, s=10.0, Threshold=25"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47dc-934e-11ee-a99e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 33, "chroma:document": "Test Accuracy w/20000 Clauses, s=10.0, Threshold=50\nTraining Accuracy w/20000 Clauses, s=10.0, Threshold=50\nTest Accuracy w/40000 Clauses, s=10.0, Threshold=100\nTraining Accuracy w/40000 Clauses, s=10.0, Threshold=100\nFigure 10: The mean test- and training accuracy per epoch for the Tsetlin Machine on the\nMNIST Dataset.\ns-value of 10 .0, and a summation target Tof 50. Furthermore, the individual Tsetlin Automata"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47dd-934e-11ee-93c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 33, "chroma:document": "each has 256 states. The Tsetlin Machine is run for 400 epochs, and it is the accuracy after\nthe \ufb01nal epoch that is reported.\nBoosting of True Positive Feedback to Include actions, as described in Section 3.3.3.\n34"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47de-934e-11ee-815d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "As seen in Figure 10, both mean test- and training accuracy increase almost monotonically\nacross the epochs, however, a\ufb00ected by random \ufb02uctuation. Perhaps most notably, while the\nmean accuracy on the training data approaches 99 .9%, accuracy on the test data continues to\nincrease as well, hitting 98 .2% after 400 epochs. This is quite di\ufb00erent from what occurs with\nback-propagation on a neural network, where accuracy on test data starts to drop at some"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47df-934e-11ee-a6c2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "point due to over\ufb01tting, without proper regularization mechanisms.\nThe \ufb01gure also shows how varying the number of clauses and the summation target T\na\ufb00ects accuracy and learning stability. With more clauses available to express patterns, in\ncombination with a higher summation target T, both learning speed, stability and accuracy\nincreases, however, at the expense of larger computational cost.\nTechnique Accuracy ( %)\n2-layer NN, 800 HU, Cross-Entropy Loss 98.6\nTsetlin Machine (95 %ile) 98.3"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e0-934e-11ee-86a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "Tsetlin Machine (95 %ile) 98.3\nTsetlin Machine (Mean) 98.2\u00b10.0\nTsetlin Machine (5 %ile) 98.1\nK-nearest-neighbors, L3 97.2\n3-layer NN, 500+150 hidden units 97.1\n40 PCA + quadratic classi\ufb01er 96.7\n1000 RBF + linear classi\ufb01er 96.4\nLogistic regression 91.5\nLinear classi\ufb01er (1-layer NN) 88.0\nDecision tree 87.8\nMultinomial Naive Bayes 83.2\nTable 16: A comparison of vanilla machine learning algorithms with the Tsetlin Machine,\ndirectly on the original unenhanced MNIST dataset (NN - Neural Network)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e1-934e-11ee-a464-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "Table 16 reports the mean accuracy of the Tsetlin Machine, across the 50 experiment runs\nwe executed. As points of reference, results for other well-known algorithms have been obtained\nfrom http://yann.lecun.com/exdb/mnist/ and included in the table (in italic). Only the vanilla\nversion of the algorithms, that has been applied directly on unenhanced MNIST data, is in-\ncluded here. The purpose of this selection is to strictly compare algorithmic performance."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e2-934e-11ee-8f55-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "In other words, we do not consider the e\ufb00ect of enhancing the dataset (e.g., warping, distor-\ntion, deskewing), combining di\ufb00erent algorithms (e.g., neural network combined with nearest\nneighbor, convolution schemes), or applying meta optimization techniques (boosting, ensemble\nlearning, etc.). With such techniques, it is possible to signi\ufb01cantly increase accuracy, with the\nbest currently reported results being an accuracy of 99 .79% [39]. Enhancing the vanilla Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e3-934e-11ee-85a4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "Machine with such techniques is further work.\nAdditionally, as a further point of reference, we train and evaluate logistic regression, deci-\nsion trees, and multinomial Naive Bayes on the binarized MNIST dataset used by the Tsetlin\nMachine.\nAs seen in the table, the Tsetlin Machine provides competitive accuracy, outperforming\ne.g. K-nearest neighbor and a 3-layer neural network. It is outperformed by a 2-layer neural"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e4-934e-11ee-8c9c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "network with 800 hidden nodes, using cross entropy loss. However, note that the Tsetlin\nMachine operates upon the binarized MNIST data (the grey tone value of each pixel is either\nset to 0 or 1), and thus has a disadvantage. Improved binarization techniques for the Tsetlin\nMachine is further work.\n6 The Tsetlin Machine as a Building Block in More Advanced\nArchitectures\nWe have designed the Tsetlin Machine to facilitate building of more advanced architectures."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e5-934e-11ee-a3dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 34, "chroma:document": "We will here exemplify di\ufb00erent ways of connecting multiple Tsetlin Machines in more advanced\n35"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e6-934e-11ee-970c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "architectures.\n6.1 The Multi-Class Tsetlin Machine\nIn some pattern recognition problems the task is to assign one of nclasses to each observed\npatternX. That is, one needs to decide upon a single output value, y\u2208{1,...,m}. Such a\nmulti-class pattern recognition problem can be handled by the Tsetlin Machine by represent-\ningyas bits, using multiple outputs y. In this section, however, we present an alternative\narchitecture that addresses the multi-class pattern recognition problem more directly."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e7-934e-11ee-9941-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "\u2026\u2026\u2026+-+-\u2211\u2335\u2335\u2335\u23350 1 0 \u2026 1 0 10/1\u22110/11\u2026n0/10/1Selected class\nInputArgmax operatorSummationConjunctiveClausesArgmax\nFigure 11: The Multi-Class Tsetlin Machine.\nFigure 11 depicts the Multi-Class Tsetlin Machine10which replaces the threshold function\nof each output y,i\u2208{1,...,m}, with a single argmax operator. With the argmax operator,\nthe indexiof the largest sum\u2211n/2\nj=1C1,i\nj(X)\u2212\u2211n/2\nj=1C0,i\nj(X) is outputted as the \ufb01nal output\nof the Tsetlin Machine:\ny= argmaxi=1,...,m\uf8eb\n\uf8edn/2\u2211\nj=1C1,i\nj(X)\u2212n/2\u2211"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e8-934e-11ee-bcd8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "y= argmaxi=1,...,m\uf8eb\n\uf8edn/2\u2211\nj=1C1,i\nj(X)\u2212n/2\u2211\nj=1C0,i\nj(X)\uf8f6\n\uf8f8. (63)\nTraining is done as described in Section 3, apart from one critical modi\ufb01cation. Assume\nwe havey=ifor the current observation ( X,y). Then the Tsetlin Automata teams associated\nwith classiare trained as per y= 1 in the original Algorithm 1. Additionally, a random class\nq\u0338=iis selected. The Tsetlin Automata teams of class qare then trained in accordance with\ny= 0 in the original algorithm."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47e9-934e-11ee-a7f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "y= 0 in the original algorithm.\n6.2 The Fully Connected Deep Tsetlin Machine\nAnother architectural family is the Fully Connected Deep Tsetlin Machine [40], illustrated\nin Figure 12. The purpose of this architecture is to build composite propositional formulas,\ncombining the propositional formula composed at one layer into more complex formula at the\nnext. As exempli\ufb01ed in the \ufb01gure, we here connect multiple Tsetlin Machines in a sequence."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ea-934e-11ee-9dea-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "The clause output from each Tsetlin Machine in the sequence is provided as input to the\nnext Tsetlin Machine in the sequence. In this manner we build a multi-layered system. For\ninstance, if layer tproduces two clauses ( P\u2227\u00acQ) and (\u00acP\u2227Q), layert+ 1 can manipulate\nthese further, treating them as inputs. Layer t+ 1 could then form more complex formulas like\n\u00ac(P\u2227\u00acQ)\u2227(P\u2227\u00acQ), which can be rewritten as ( \u00acP\u2228Q)\u2227(P\u2227\u00acQ)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47eb-934e-11ee-ba1b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 35, "chroma:document": "One simple approach for training such an architecture is indicated in the \ufb01gure. As il-\nlustrated, each layer is trained independently, directly from the output target y, exactly as\ndescribed in Section 3.3. The training procedure is thus similar to the strategy Hinton et al.\n10An implementation of the Multi-Class Tsetlin Machine can be found at\nhttps://github.com/cair/TsetlinMachine.\n36"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ec-934e-11ee-a80e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 36, "chroma:document": "\u2026\u2026\u2026+-+-\n\u2211\u2335\u2335\u2335\u23350 1 0 \u2026 1 0 10/1\u22110/10/1\n0/10/1\n\u2026\u2026\u2026+-+-\n\u2211\u2335\u2335\u2335\u23350/1\u22110/10/1\n0/10/10/1\n0/1Figure 12: The fully connected Deep Tsetlin Machine.\nused to train their pioneering Deep Belief Networks, layer-by-layer, in 2006 [41]. Such an ap-\nproach can be e\ufb00ective when each layer produces abstractions, in the form of clauses, that can\nbe taken advantage of in the following layer.\n6.3 The Convolutional Tsetlin Machine\nWe next demonstrate how self-contained and independent Tsetlin Machines can interact to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ed-934e-11ee-be2f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 36, "chroma:document": "build a Convolutional Tsetlin Machine [42], illustrated in Figure 13. The Convolutional Tsetlin\nFigure 13: The Convolutional Tsetlin Machine.\nMachine is a deep architecture based on mathematical convolution, akin to Convolutional\nNeural Networks [38]. For illustration purposes, consider 2D images of size 100 \u00d7100 as input.\nAt the core of a Convolutional Tsetlin Machine we \ufb01nd a kernel Tsetlin Machine with a small"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ee-934e-11ee-b235-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 36, "chroma:document": "receptive \ufb01eld. Each layer tof the Convolutional Tsetlin Machine operates as follows:\n1. A convolution is performed over the input from the previous Tsetlin Machine layer,\nproducing one feature map per output y. Here, the Tsetlin Machine acts as a kernel\n37"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ef-934e-11ee-bb61-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 37, "chroma:document": "in the convolution. In this manner, we reduce complexity by reusing the same Tsetlin\nMachine across the whole image, focusing on a small image patch at a time.\n2. The feature maps produced are then down-sampled using a pooling operator, in a similar\nfashion as done in a Convolutional Neural Network, before the next layer and a new\nTsetlin Machine takes over. Here, the purpose of the pooling operation is to gradually\nincrease the abstraction level of the clauses, layer by layer."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f0-934e-11ee-95e8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 37, "chroma:document": "A simple approach for training a Convolutional Tsetlin Machine is indicated in the \ufb01gure. In\nbrief, the feedback to the Tsetlin Machine kernel is directly provided from the desired end\noutputy, exactly as described in Section 3. The only di\ufb00erence is the fact that the input to\nlayert+ 1 comes from the down-scaled feature map produced by layer t. Again, this is useful\nwhen each layer produces abstractions, in the form of clauses, that can be taken advantage of\nat the next layer."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f1-934e-11ee-8fdd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 37, "chroma:document": "at the next layer.\n6.4 The Recurrent Tsetlin Machine\nThe \ufb01nal example is the Recurrent Tsetlin Machine [43] (Figure 14). In all brevity, the same\nTsetlin Machine is here reused from time step to time step. By taking the output from the\nTsetlin Machine of the previous time step as input, together with an external input from the\ncurrent time step, an in\ufb01nitely deep sequence of Tsetlin Machines is formed. This is quite"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f2-934e-11ee-ae2b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 37, "chroma:document": "similar to the family of Recurrent Neural Networks [44]. Again, the architecture can be trained\nExternal inputMemory\nInput from memoryTime step t\nTime step t-1\nExternal inputInput from memory\nFigure 14: The Recurrent Tsetlin Machine.\nlayer by layer, directly from the target output y(t) of the current time step t. However, to learn\nmore advanced sequential patterns, there is a need for rewarding and penalizing that propagate"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f3-934e-11ee-a59b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 37, "chroma:document": "back in time. How to design such a propagation scheme is presently an open research question.\n7 Conclusion and Further Work\nIn this paper we proposed the Tsetlin Machine, an alternative to neural networks. The Tsetlin\nMachine solves the vanishing signal-to-noise ratio of collectives of Tsetlin Automata. This\nallows it to coordinate millions of Tsetlin Automata. By equipping teams of Tsetlin Automata\n38"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f4-934e-11ee-8a81-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "with the ability to express patterns in propositional logic, we have enabled them to recognize\ncomplex patterns. Furthermore, we proposed a novel decentralized feedback orchestration\nmechanism. The mechanism is based on resource allocation principles, with the intent of\nmaximizing e\ufb00ectiveness of sparse pattern recognition capacity. This mechanism e\ufb00ectively\nprovides the Tsetlin Machine with the ability to capture unlabelled sub-patterns."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f5-934e-11ee-9495-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "Our theoretical analysis reveals that the Tsetlin Machine game have Nash equilibria that\nmaps to propositional formulas that maximize pattern recognition accuracy. In other words,\nthere are no local optima in the learning process, only global ones. This explains how the\ncollectives of Tsetlin Automata are able to accurately converge towards complex propositional\nformulas that capture the essence of \ufb01ve diverse pattern recognition problems."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f6-934e-11ee-95b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "Overall, the Tsetlin Machine seems to be particularly suited for digital computers, being\nbased on simple bit manipulation with AND-, OR-, and NOT gates. Both input, hidden\npatterns, and output can be expressed as bit patterns.\nIn our empirical evaluations on \ufb01ve benchmarks, the Tsetlin Machine provided competitive\naccuracy with respect to both Multilayer Perceptron Networks, Support Vector Machines,\nDecision Trees, Random Forests, the Naive Bayes Classi\ufb01er and Logistic Regression. It further"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f7-934e-11ee-8349-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "turns out that the Tsetlin Machine requires less data than neural networks, outperforming the\nNaive Bayes Classi\ufb01er in data sparse environments.\nBy demonstrating that the longstanding problem of vanishing signal-to-noise ratio can be\nsolved, the Tsetlin Machine further provides a novel game theoretic framework for recasting the\nproblem of pattern recognition. This framework can thus provide opportunities for introducing"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f8-934e-11ee-920a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "bandit algorithms into large-scale pattern recognition. It could for instance be interesting to in-\nvestigate the e\ufb00ect of replacing the Tsetlin Automaton with alternative bandit algorithms, such\nas algorithms based on Thompson Sampling [45, 46, 47, 48] or Upper Con\ufb01dence Bounds [49].\nThe more advanced Fully Connected Deep Tsetlin Machine, the Convolution Tsetlin Ma-\nchine, and the Recurrent Tsetlin Machine architectures also form a starting point for further"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47f9-934e-11ee-b5f8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "exploration. These architectures can potentially improve pattern representation compactness\nand learning speed. However, it is currently unclear how these architectures can be most\ne\ufb00ectively trained.\nLastly, the high accuracy of the Tsetlin Machine, combined with its ability to produce self-\ncontained easy-to-interpret propositional formulas for pattern recognition, makes it attractive\nfor applied research, such as in the safety-critical medical domain.\nAcknowledgements"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47fa-934e-11ee-8bdd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "Acknowledgements\nI thank my colleagues from the Centre for Arti\ufb01cial Intelligence Research (CAIR), Lei Jiao,\nXuan Zhang, Geir Thore Berge, Darshana Abeyrathna, Saeed Rahimi Gorji, Sondre Glimsdal,\nRupsa Saha, Bimal Bhattarai, Rohan K. Yadev, Bernt Viggo Matheussen, Morten Good-\nwin, Christian Omlin, Vladimir Zadorozhny (University of Pittsburgh), Jivitesh Sharma, and\nAhmed Abouzeid, for their contributions to the development of the Tsetlin machine family of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47fb-934e-11ee-ac46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "techniques. I would also like to thank our House of CAIR partners, Alex Yakovlev, Rishad\nSha\ufb01k, Adrian Wheeldon, Jie Lei, Tousif Rahman (Newcastle University), Jonny Edwards\n(Temporal Computing), Marco Wiering (University of Groningen), Christian D. Blakely (PwC\nSwitzerland), Adrian Phoulady, Anders Refsdal Olsen, Halvor Sm\u00f8rvik, and Erik Mathisen for\ntheir many contributions.\nCode Availability\nSource code and datasets for the Tsetlin Machine, available under the MIT Licence, can be"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47fc-934e-11ee-bb9d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 38, "chroma:document": "found at https://github.com/cair/TsetlinMachine and https://github.com/cair/pyTsetlinMachine.\n39"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47fd-934e-11ee-858d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "Data Availability\nThe datasets generated during and/or analysed during the current study are available from the\ncorresponding author on reasonable request.\nReferences\n[1] Y. Lecun, Y. Bengio, and G. Hinton, \u201cDeep learning,\u201d pp. 436\u2013444, 2015.\n[2] M. L. Tsetlin, \u201cOn behaviour of \ufb01nite automata in random medium,\u201d Avtomat. i Telemekh ,\nvol. 22, no. 10, pp. 1345\u20131354, 1961.\n[3] K. S. Narendra and M. A. L. Thathachar, Learning Automata: An Introduction . Prentice-\nHall, Inc., 1989."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47fe-934e-11ee-b82f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "Hall, Inc., 1989.\n[4] H. Robbins, \u201cSome aspects of the sequential design of experiments,\u201d Bulletin of the Amer-\nican Mathematical Society , 1952.\n[5] J. Gittins, \u201cBandit processes and dynamic allocation indices,\u201d Journal of the Royal Sta-\ntistical Society, Series B (Methodological) , vol. 41, no. 2, pp. 148\u2013177, 1979.\n[6] J. Carroll, Theory of Finite Automata With an Introduction to Formal Languages . Pren-\ntice Hall, 1989."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d47ff-934e-11ee-824d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "tice Hall, 1989.\n[7] A. G. Barto and P. Anandan, \u201cPattern-recognizing stochastic learning automata,\u201d IEEE\nTransactions on Systems, Man, and Cybernetics , vol. 15, no. 3, pp. 360\u2013375, 1985.\n[8] P. Sastry and M. Thathachar, \u201cLearning automata algorithms for pattern classi\ufb01cation,\u201d\nSadhana , vol. 24, no. 4-5, pp. 261\u2013292, 1999.\n[9] S.-H. Zahiri, \u201cLearning automata based classi\ufb01er,\u201d Pattern Recognition Letters , vol. 29,\nno. 1, pp. 40\u201348, 2008."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4800-934e-11ee-a37a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "no. 1, pp. 40\u201348, 2008.\n[10] \u2014\u2014, \u201cClassi\ufb01cation rule discovery using learning automata,\u201d International Journal of\nMachine Learning and Cybernetics , vol. 3, no. 3, pp. 205\u2013213, 2012.\n[11] P. Sastry, G. Nagendra, and N. Manwani, \u201cA team of continuous-action learning automata\nfor noise-tolerant learning of half-spaces,\u201d IEEE Transactions on Systems, Man, and Cy-\nbernetics, Part B (Cybernetics) , vol. 40, no. 1, pp. 19\u201328, 2009."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4801-934e-11ee-969b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "[12] M. Goodwin, A. Yazidi, and T. M. Jonassen, \u201cDistributed learning automata for solving\na classi\ufb01cation task,\u201d in 2016 IEEE congress on evolutionary computation (CEC) . IEEE,\n2016, pp. 3999\u20134006.\n[13] H. Motieghader, A. Naja\ufb01, B. Sadeghi, and A. Masoudi-Nejad, \u201cA hybrid gene selec-\ntion algorithm for microarray cancer classi\ufb01cation using genetic algorithm and learning\nautomata,\u201d Informatics in Medicine Unlocked , vol. 9, pp. 246\u2013254, 2017."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4802-934e-11ee-a3c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "[14] M. A. L. Thathachar and P. S. Sastry, Networks of Learning Automata: Techniques for\nOnline Stochastic Optimization . Kluwer Academic Publishers, 2004.\n[15] B. Tung and L. Kleinrock, \u201cUsing Finite State Automata to Produce Self-Optimization\nand Self-Control,\u201d IEEE Transactions on Parallel and Distributed Systems , vol. 7, no. 4,\npp. 47\u201361, 1996.\n[16] B. J. Oommen, \u201cStochastic Searching on the Line and its Applications to Parameter Learn-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4803-934e-11ee-89d3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 39, "chroma:document": "ing in Nonlinear Optimization,\u201d IEEE Transactions on Systems, Man, and Cybernetics,\nPart B , vol. 27, no. 4, pp. 733\u2013739, 1997.\n40"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4804-934e-11ee-9233-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "[17] B. J. Oommen and D. C. Ma, \u201cDeterministic Learning Automata Solutions to The\nEquipartitioning Problem,\u201d IEEE Transactions on Computers , vol. 37, no. 1, pp. 2\u201313,\n1988.\n[18] M. Ghavipour and M. R. Meybodi, \u201cA streaming sampling algorithm for social activity\nnetworks using \ufb01xed structure learning automata,\u201d Applied Intelligence , 2018.\n[19] A. Yazidi and B. John Oommen, \u201cOn the analysis of a random walk-jump chain with tree-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4805-934e-11ee-855f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "based transitions and its applications to faulty dichotomous search,\u201d Sequential Analysis ,\nvol. 37, pp. 31\u201346, jan 2018.\n[20] J. Zhang, Y. Wang, C. Wang, and M. Zhou, \u201cSymmetrical Hierarchical Stochastic Search-\ning on the Line in Informative and Deceptive Environments,\u201d IEEE Transactions on Cy-\nbernetics , vol. 47, no. 3, pp. 626 \u2013 635, jul 2016.\n[21] B. J. Oommen, S. Misra, and O.-C. Granmo, \u201cRouting bandwidth-guaranteed paths in"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4806-934e-11ee-8c0f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "MPLS tra\ufb03c engineering: a multiple race track learning approach,\u201d Computers, IEEE\nTransactions on , vol. 56, no. 7, pp. 959\u2013976, 2007.\n[22] O.-C. Granmo and S. Glimsdal, \u201cAccelerated Bayesian learning for decentralized two-\narmed bandit based decision making with applications to the Goore Game,\u201d Applied In-\ntelligence , vol. 38, no. 4, 2013.\n[23] J. Von Neumann and O. Morgenstern, Theory of Games and Economic Behavior , 1947."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4807-934e-11ee-9b55-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "[24] O.-C. Granmo, B. J. Oommen, S. A. Myrer, and M. G. Olsen, \u201cLearning Automata-based\nSolutions to the Nonlinear Fractional Knapsack Problem withApplications to Optimal\nResource Allocation,\u201d IEEE Transactions on Systems, Man, and Cybernetics, Part B ,\nvol. 37, no. 1, pp. 166\u2013175, 2007.\n[25] V. Haugland, M. Kj\u00f8lleberg, S.-E. Larsen, and O.-C. Granmo, \u201cA two-armed bandit\ncollective for hierarchical examplar based mining of frequent itemsets with applications"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4808-934e-11ee-a87c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "to intrusion detection,\u201d Transactions on Computational Collective Intelligence XIV , vol.\n8615, pp. 1\u201319, 2014.\n[26] Guolong Su, Kush R. Varshney, and Dmitry M. Malioutov, \u201cInterpretable Two-Level\nBoolean Rule Learning for Classi\ufb01cation,\u201d in ICML Workshop on Human Interpretability\nin Machine Learning (WHI 2016) , 2016, pp. 66\u201370.\n[27] T. Wang, C. Rudin, F. Doshi-Velez, Y. Liu, E. Jones, E. Klamp\ufb02, P. Macneille, and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4809-934e-11ee-ba7f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "M. Gupta, \u201cA Bayesian Framework for Learning Rule Sets for Interpretable Classi\ufb01cation,\u201d\nJournal of Machine Learning Research , 2017.\n[28] L. G. Valiant, \u201cA Theory of the Learnable,\u201d Communications of the ACM , vol. 27, no. 11,\npp. 1134\u20131142, 1984.\n[29] O.-C. Granmo, B. J. Oommen, S. A. Myrer, and M. G. Olsen, \u201cLearning Automata-based\nSolutions to the Nonlinear Fractional Knapsack Problem with Applications to Optimal"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480a-934e-11ee-bfd3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 40, "chroma:document": "Resource Allocation,\u201d IEEE Transactions on Systems, Man, and Cybernetics, Part B ,\nvol. 37, no. 1, pp. 166\u2013175, 2007.\n[30] D. Monderer and L. Shapley, \u201cPotential games,\u201d Games and Economic Behavior , vol. 14,\nno. 1, pp. 124\u2013143, 1996. [Online]. Available: https://EconPapers.repec.org/RePEc:eee:\ngamebe:v:14:y:1996:i:1:p:124-143\n[31] V. U. Krylov and M. Tsetlin, \u201cGames Between Automata,\u201d Automation and Remote\nControl , vol. 24, pp. 889\u2013899, 1963.\n41"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480b-934e-11ee-a20e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "[32] V. A. Borovikov and V. I. Bryzgalov, \u201cThe Simplest Symmetric Game of Many Automata,\u201d\nAvtomatika i Telemekhanika , vol. 26, pp. 637\u2013687, 1965.\n[33] V. A. Volkonskii, \u201cAsymptotic Properties of the Behavior of Elementary Automata in a\nGame,\u201d Problemy Peredachi Informatsii , vol. 1, pp. 36\u201353, 1965.\n[34] B. G. Pittel, \u201cThe Asymptotic Properties of One Form of Goore Game,\u201d Problemy\nPeredachi Informatsii , vol. 1, pp. 99\u2013112, 1965."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480c-934e-11ee-8518-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "Peredachi Informatsii , vol. 1, pp. 99\u2013112, 1965.\n[35] R. Viswanathan and K. S. Narendra, \u201cGames of stochastic automata,\u201d IEEE Transactions\non Systems, Man, and Cybernetics , vol. SMC-4, no. 1, pp. 131\u2013135, 1974.\n[36] K. S. Narendra, \u201cCompetitive and cooperative games of variable-structure stochastic au-\ntomata,\u201d Journal of Cybernetics , vol. 3, no. 1, pp. 1\u201323, 1973.\n[37] R. Duda, P. Hart, and D. Stork, Pattern Classi\ufb01cation , 2nd ed. New York, NY: John\nWiley and Sons, Inc., 2000."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480d-934e-11ee-bf47-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "Wiley and Sons, Inc., 2000.\n[38] Y. LeCun, L. Bottou, Y. Bengio, and P. Ha\ufb00ner, \u201cGradient-based learning applied to\ndocument recognition,\u201d Proceedings of the IEEE , vol. 86, no. 11, pp. 2278 \u2013 2324, 1998.\n[39] L. Wan, M. Zeiler, S. Zhang, Y. LeCun, and R. Fergus, \u201cRegularization of neural networks\nusing dropconnect,\u201d International Conference on Machine Learning (ICML) , 2013.\n[40] O.-C. Granmo, \u201cThe Fully Connected Deep Tsetlin Machine,\u201d In Preparation , 2019."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480e-934e-11ee-bcb6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "[41] G. E. Hinton, S. Osindero, and Y.-W. Teh, \u201cA Fast Learning Algorithm for Deep Belief\nNets,\u201d Neural Computation , vol. 18, no. 7, pp. 1527 \u2013 1554, 2006.\n[42] O.-C. Granmo, \u201cThe Convolutional Tsetlin Machine,\u201d In Preparation , 2019.\n[43] \u2014\u2014, \u201cThe Recurrent Tsetlin Machine,\u201d In Preparation , 2019.\n[44] J. Schmidhuber, \u201cDeep Learning in neural networks: An overview,\u201d Neural Networks ,\nvol. 61, pp. 85\u2013117, 2015."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d480f-934e-11ee-b15c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "vol. 61, pp. 85\u2013117, 2015.\n[45] W. R. Thompson, \u201cOn the likelihood that one unknown probability exceeds another in\nview of the evidence of two samples,\u201d Biometrika , vol. 25, no. 3/4, pp. 285\u2013294, 1933.\n[46] O.-C. Granmo, \u201cSolving Two-Armed Bernoulli Bandit Problems Using a Bayesian Learn-\ning Automaton,\u201d International Journal of Intelligent Computing and Cybernetics , vol. 3,\nno. 2, pp. 207\u2013234, 2010.\n[47] B. C. May, N. Korda, A. Lee, and D. S. Leslie, \u201cOptimistic Bayesian sampling in"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4810-934e-11ee-b1a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "contextual-bandit problems,\u201d Journal of Machine Learning Research , vol. 13, pp. 2069\u2013\n2106, 2012.\n[48] O. Chapelle and L. Li, \u201cAn Empirical Evaluation of Thompson Sampling,\u201d in Advances in\nNeural Information Processing Systems 24 , J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett,\nF. Pereira, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2011, pp. 2249\u20132257.\n[49] P. Auer, \u201cUsing con\ufb01dence bounds for exploitation-exploration trade-o\ufb00s,\u201d in Journal of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4811-934e-11ee-9ad8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\1804.01508.pdf", "page": 41, "chroma:document": "Machine Learning Research , vol. 3, no. 3, 2003, pp. 397\u2013422.\n42"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4887-934e-11ee-b008-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 0, "chroma:document": "https://doi.or g/10.1007/s10844-021-00682-5\nArelationaltsetlinmachinewithapplications\ntonaturallanguageunderstanding\nRupsaSaha1\u00b7Ole-Christo\ufb00erGranmo1\u00b7VladimirI.Zadorozhny1,2\u00b7\nMortenGoodwin1\nReceived:19March2021/Revised:11October2021/Accepted:11October2021/\n\u00a9TheAuthor(s)2021\nAbstract\nTsetlin machines (TMs) are a pattern recognition approach that uses finite state machines\nfor learning and propositional logic to represent patterns. In addition to being natively"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4888-934e-11ee-94c5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 0, "chroma:document": "interpretable, they have provided competitive accuracy for various tasks. In this paper, we\nincrease the computing power of TMs by proposing a first-order logic-based framework\nwith Herbrand semantics. The resulting TM is relational and can take advantage of logical\nstructures appearing in natural language, to learn rules that represent how actions and con-\nsequences are related in the real world. The outcome is a logic program of Horn clauses,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4889-934e-11ee-844e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 0, "chroma:document": "bringing in a structured view of unstructured data. In closed-domain question-answering,\nthe first-order representation produces 10 \u00d7more compact KBs, along with an increase\nin answering accuracy from 94.83% to 99.48%. The approach is further robust towards\nerroneous, missing, and superfluous information, distilling the aspects of a text that are\nimportant for real-world understanding\nKeywords Tsetlin machine \u00b7Natural language processing \u00b7Logic programming \u00b7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488a-934e-11ee-8196-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 0, "chroma:document": "Knowledge representation \u00b7Question answering\n1 Introduction\nUsing Artificial Intelligences (AI) to answer natural language questions has long been an\nactive research area, considered as an essential aspect in machines ultimately achieving\nhuman-level world understanding. Large-scale structured knowledge bases (KBs), such as\nFreebase (Bollacker et al., 2008 ), have been a driving force behind successes in this field."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488b-934e-11ee-8eff-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 0, "chroma:document": "The KBs encompass massive ever-growing amounts of information, which enable easier\nhandling of Open-Domain Question-Answering (QA) (Prager, 2006 ) by organizing a large\n/envelopebackRupsa Saha\nrupsa.saha@uia.no\n1Centre for AI Research, Department of IKT, University of Agder, Agder, Norway\n2School of Computing and Information, University of Pittsburgh, Pittsburgh, PA, USAPublished online: 1 January 2022Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488c-934e-11ee-91f7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "variety of answers in a structured format. The difficulty arises in successfully interpret-\ning natural language by artificially intelligent agents, both to build the KBs from naturallanguage text resources and to interpret the questions asked.\nGeneralization beyond the information stored in a KB further complicates the QA prob-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488d-934e-11ee-8ff9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "lem. Human-level world understanding requires abstracting from specific examples to buildmore general concepts and rules. When the information stored in the KB is error-freeand consistent, generalization becomes a standard inductive reasoning problem. How-ever, abstracting world-knowledge entails dealing with uncertainty, vagueness, exceptions,errors, and conflicting information. This is particularly the case when relying on Artifi-cial Intelligences (AI) approaches to extract and structure"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488e-934e-11ee-8f9f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "(AI) approaches to extract and structure information, which is notoriouslyerror-prone."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d488f-934e-11ee-a8b2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "This paper addresses the above QA challenges by proposing a Relational Tsetlin machine\n(TM) that builds non-recursive first-order Horn clauses from specific examples, distilling\ngeneral concepts and rules.\nTsetlin Machines (Granmo, 2018) are a pattern recognition approach to constructing\nhuman-understandable patterns from given data, founded on propositional logic. While theidea of Tsetlin automaton (TA) (Tsetlin, 1961) have been around since 1960s, using them"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4890-934e-11ee-b43b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "in pattern recognition is relatively new. TMs have successfully addressed several machinelearning tasks, including natural language understanding (Yadav et al., 2021a, b;B h a t -\ntarai et al., 2021;S a h ae ta l . ,2020; Berge et al., 2019), image analysis (Granmo et al.,\n2019), classification (Abeyrathna et al., 2021), regression (Abeyrathna et al., 2019), and\nspeech understanding (Lei et al., 2021). The propositional clauses constructed by a Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4891-934e-11ee-83db-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "machine (TM) have high discriminative power and constitute a global description of thetask learnt (Blakely & Granmo, 2020;S a h ae ta l . ,2020). Apart from maintaining accuracy\ncomparable to state-of-the-art machine learning techniques, the method also has provided asmaller memory footprint and faster inference than more traditional neural network-basedmodels (Wheeldon et al., 2020; Lei et al., 2020; Abeyrathna et al., 2021; Lei et al., 2021;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4892-934e-11ee-afc3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "Phoulady et al., 2019). Furthermore, Shafik et al. ( 2020) shows that TMs can be fault-\ntolerant, able to mask stuck-at faults. However, although TMs can express any propositionalformula by using disjunctive normal form, first-order logic is required to obtain the com-puting power equivalent to a universal Turing machine. In this paper, we take the first stepstowards increasing the computing power of TM by introducing a first order TM framework"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4893-934e-11ee-8af3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "with Herbrand semantics, referred to as the Relational TM. Accordingly, we will in the\nfollowing denote the original approach as Propositional TM .\nClosed-DomainQuestion-Answering: As proof-of-concept, we apply our proposed Rela-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4894-934e-11ee-987d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "tional TM to so-called Closed-Domain QA. Closed-Domain QA assumes a text (single ormultiple sentences) followed by a question which refers to some aspect of the precedingtext. Accordingly, the amount of information that must be navigated is less than for openquestion-answering. Yet, answering closed-domain questions poses a significant naturallanguage understanding challenge.\nConsider the following example of information, taken from Rajpurkar et al. ( 2016): \u201cThe"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4895-934e-11ee-8dcf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "Black Death is thought to have originated in the arid plains of Central Asia, where it thentravelled along the Silk Road, reaching Crimea by 1343. From there, it was most likelycarried by Oriental rat fleas living on the black rats that were regular passengers on merchantships.\u201dOne can then have questions such as \u201cWhere did the black death originate?\u201d or \u201cHowdid the black death make it to the Mediterranean and Europe?\u201d. These questions can be122 Journal of Intelligent Information Systems (2022)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4896-934e-11ee-a4da-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 1, "chroma:document": "Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4897-934e-11ee-94d5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "answered completely with just the information provided, hence it is an example of closed-\ndomain question answering. However, mapping the question to the answer requires not onlynatural language processing, but also a fair bit of language understanding.\nHere is a much simpler example: \u201cBob went to the garden. Sue went to the cafe. Bob"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4898-934e-11ee-a382-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "walked to the office.\u201d This information forms the basis for questions like \u201cWhere is Bob?\u201d or\u201cWhere is Sue?\u201d. Taking it a step further, given the previous information and questions, onecan envision a model that learns to answer similar questions based on similar information,even though the model has never seen the specifics of the information before (i.e., the namesand the locations).\nWith QA being such an essential area of Natural Language Understanding, there has been"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d4899-934e-11ee-9823-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "a lot of different approaches proposed. Common methods to QA include the following:\n\u2013 Linguistic techniques, such as tokenization, POS tagging and parsing that transform\nquestions into a precise query that merely extracts the respective response from a\nstructured database;\n\u2013 Statistical techniques such as Support Vector Machines, Bayesian Classifiers, and\nmaximum entropy models, trained on large amount of data, specially for open QA;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489a-934e-11ee-8a81-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "\u2013 Pattern matching using surface text patterns with templates for response generation.\nMany methods use a hybrid approach encompassing more than one of these approaches for\nincreased accuracy. Most QA systems suffer from a lack of generality, and are tuned forperformance in restricted use cases. Lack of available explainabilty also hinders researchers\u2019quest to identify pain points and possible major improvements (Soares & Parreiras, 2020;\nPundge et al., 2016)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489b-934e-11ee-a3a6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "Pundge et al., 2016).\nPaperContributions: Our main contributions in this paper are as follows:\n\u2013 We introduce a Relational TM, as opposed to a propositional one, founded on non-\nrecursive Horn clauses and capable of processing relations, variables and constants.\n\u2013 We propose an accompanying relational framework for efficient representation and\nprocessing of the QA problem.\n\u2013 We provide empirical evidence uncovering that the Relational TM produces at least one"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489c-934e-11ee-806a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "order of magnitude more compact KBs than the Propositional TM. At the same time,answering accuracy increases from 94.83% to 99.48% because of more general rules.\n\u2013 We provide a model-theoretical interpretation for the proposed framework.\nOverall, our Relational TM unifies knowledge representation, learning, and reasoning in a\nsingle framework.\nPaper Organization: The paper is organized as follows. In Section 2, we present related"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489d-934e-11ee-8ed3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 2, "chroma:document": "work on Question Answering. Section 3focuses on the background of the Propositional TM\nand the details of the new Relational TM. In Sections 4and5, we describe how we employ\nRelational TM in QA and related experiments.\n2 Backgroundandrelatedwork\nThe problem of QA is related to numerous aspects of Knowledge Engineering and DataManagement.123 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489e-934e-11ee-ac10-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "Knowledge engineering deals with constructing and maintaining knowledge bases to\nstore knowledge of the real world in various domains. Automated reasoning techniques"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d489f-934e-11ee-894b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "use this knowledge to solve problems in domains that ordinarily require human logicalreasoning. Therefore, the two key issues in knowledge engineering are how to constructand maintain knowledge bases, and how to derive new knowledge from existing knowledgeeffectively and efficiently. Automated reasoning is concerned with the building ofcomputing systems that automate this process. Although the overall goal is to automatedifferent forms of reasoning, the term has largely been identified with"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a0-934e-11ee-a0a3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "the term has largely been identified with valid deductivereasoning as conducted in logical systems. This is done by combining known (yet possi-bly incomplete) information with background knowledge and making inferences regardingunknown or uncertain information."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a1-934e-11ee-9c87-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "Typically, such a system consists of subsystems like knowledge acquisition system, the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a2-934e-11ee-99f0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "knowledge base itself, inference engine, explanation subsystem and user interface. Theknowledge model has to represent the relations between multiple components in a symbolic,machine understandable form, and the inference engine has to manipulate those symbolsto be capable of reasoning. The \u201cway to reason\u201d can range from earlier versions that weresimple rule-based systems to more complex and recent approaches based on machine learn-ing, especially on Deep Learning. Recently, transformer based"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a3-934e-11ee-82dc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "on Deep Learning. Recently, transformer based systems have significantlyimproved question-answering performance in general, even more so when they can be sup-plied with a very large amount of training data (Lu et al., 2020; Lewis et al., 2019;W o l f"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a4-934e-11ee-8f8c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "et al., 2019; Jung & Kim, 2020;S r i n i v a s a ne ta l . , 2021). Typically, rule-based systems suf-\nfered from lack of generality, and the need for human experts to create rules in the firstplace. On the other hand most machine learning based approaches have the disadvantage ofnot being able to justify decisions taken by them in a human understandable form Ludwig(2010) and Cyras et al. (2020).\nWhile databases have long been a mechanism of choice for storing information, they"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a5-934e-11ee-b3e5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "only had inbuilt capability to identify relations between various components, and did nothave the ability to support reasoning based on such relations. Efforts to combine formallogic programming with relational databases led to the advent of deductive databases. Infact, the field of QA is said to have arisen from the initial goal of performing deduc-tive reasoning on a set of given facts (Green, 1969). In deductive databases, the semantics"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a6-934e-11ee-9855-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "of the information are represented in terms of mathematical logic. Queries to deductivedatabases also follow the same logical formulation (Gallaire et al., 1989). One such exam-\nple is ConceptBase (Jarke et al., 1995), which used the Prolog-inspired language O-Telos\nfor logical knowledge representation and querying using deductive object-oriented databaseframework.\nWith the rise of the internet, there came a need for unification of information on the web."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a7-934e-11ee-b122-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "The Semantic Web (SW) proposed by W3C is one of the approaches that bridges the gapbetween the Knowledge Representation and the Web Technology communities. However,reasoning and consistency checking is still not very well developed, despite the underlyingformalism that accompanies the semantic web. One way of introducing reasoning is viadescriptive logic. It involves concepts (unary predicates) and roles (binary predicates) andthe idea is that implicitly captured knowledge can be inferred from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a8-934e-11ee-943b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "captured knowledge can be inferred from the given descriptionsof concepts and roles (Dong et al., 2003; Turhan, 2011)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48a9-934e-11ee-a889-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 3, "chroma:document": "One of the major learning exercises is carried out by the NELL mechanism proposed\nby Mitchell et al. ( 2018), which aims to learn many semantic categories from primarily\nunlabeled data. At present, NELL uses simple frame-based knowledge representation, aug-mented by the PRA reasoning system. The reasoning system performs tractable, but limited124 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48aa-934e-11ee-a720-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "types of reasoning based on restricted Horn clauses. NELL\u2019s capabilities is already lim-\nited in part by its lack of more powerful reasoning components; for example, it currentlylacks methods for representing and reasoning about time and space. Hence, core ArtificialIntelligence (AI) problems of representation and tractable reasoning are also core researchproblems for never-ending learning agents.\nWhile other approaches such as neural networks are considered to provide attribute-based"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48ab-934e-11ee-85b2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "learning, Inductive Logic Programming (ILP) is an attempt to overcome their limitations bymoving the learning away from the attributes themselves and more towards the level of first-order predicate logic. ILP builds upon the theoretical framework of logic programming andlooks to construct a predicate logic given background knowledge, positive examples andnegative examples. One of the main advantages of ILP over attribute-based learning is ILP\u2019sgenerality of representation for background"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48ac-934e-11ee-aff1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "ILP\u2019sgenerality of representation for background knowledge. This enables the user to provide, ina more natural way, domain-specific background knowledge to be used in learning. The useof background knowledge enables the user both to develop a suitable problem representationand to introduce problem-specific constraints into the learning process. Over the years, ILPhas evolved from depending on hand-crafted background knowledge only, to employingdifferent technologies in order to learn the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48ad-934e-11ee-b8f7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "technologies in order to learn the background knowledge as part of the process. Incontrast to typical machine learning, which uses feature vectors, ILP requires the knowledgeto be in terms of facts and rules governing those facts. Predicates can either be supplied ordeducted, and one of the advantages of this method is that newer information can be addedeasily, while previously learnt information can be maintained as required (Cropper et al.,2020). Probabilistic inductive logic programming is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48ae-934e-11ee-8ef1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "Probabilistic inductive logic programming is an extension of ILP, where logic rules,as learnt from the data, are further enhanced by learning probabilities associated with suchrules (Bratko & Muggleton, 1995; Nickles & Mileo, 2014; De Raedt & Kersting, 2008;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48af-934e-11ee-87b9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "Kramer et al., 2001).\nTo sum up, none of the above approaches can be efficiently and systematically applied to\nthe QA problem, especially in uncertain and noisy environments. In this paper we proposea novel approach to tackle this problem. Our approach is based on relational representationof QA, and using a novel Relational TM technique for answering questions. We elaborateon the proposed method in the next two sections.\n3 Buildingarelationaltsetlinmachine\n3.1 Tsetlinmachinefoundation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48b0-934e-11ee-858f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "3.1 Tsetlinmachinefoundation\nA Tsetlin Automaton (Tsetlin Automaton (TA)) is a deterministic automaton that learns theoptimal action among the set of actions offered by an environment. It performs the actionassociated with its current state, which triggers a reward or penalty based on the groundtruth. The state is updated accordingly, so that the Tsetlin automaton (TA) progressivelyshifts focus towards the optimal action (Tsetlin, 1961). A 2-action Tsetlin automaton (TA),"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d48b1-934e-11ee-ab7c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "as shown in Fig. 1a, has 2N states, such that, a final state between 1 and Nindicates Action\n1, while final state between N+1a n d2 Nindicates Action 2. A reward reinforces a par-\nticular action by changing state away from the central state, i.e. moving \u201cdeeper\u201d towardsAction 1 or 2, as the case maybe. The opposite happens in the case of a penalty, where theresultant state after an iteration is closer to the central states ( N/N+1). A TM consists"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7329-934e-11ee-a6ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 4, "chroma:document": "of a collection of such TAs, which together create complex propositional formulas usingconjunctive clauses.125 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732a-934e-11ee-8daa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 5, "chroma:document": "3.1.1 Classi\ufb01cation\nA TM takes a vector X=(x1,...,x f)of propositional features as input, to be classified\ninto one of two classes, y=0o ry =1. Together with their negated counterparts, \u00afxk=\n\u00acxk=1\u2212xk, the features form a literal set L={x1,...,x f,\u00afx1,...,\u00afxf}. We refer to\nthis \u201cregular\u201d TM as a Propositional TM, due to the input it works with and the output it\nproduces.\nA TM pattern is formulated as a conjunctive clause Cj, formed by ANDing a subset\nLj\u2286Lof the literal set:\nCj(X)=\u22c0\nlk\u2208Ljlk=\u220f"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732b-934e-11ee-a5e4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 5, "chroma:document": "Lj\u2286Lof the literal set:\nCj(X)=\u22c0\nlk\u2208Ljlk=\u220f\nlk\u2208Ljlk.( 1)\nE.g., the clause Cj(X)=x1\u2227x2=x1x2consists of the literals Lj={x1,x2}and outputs\n1i f fx 1=x2=1.\nEach clause has access to 2 \u00d7fTsetlin automaton (TAs), one per literal k. The bidirec-\ntional action of each TA decides which of the literals form a part of a clause. The TA (TA)states from 1 to Nmap to the exclude action, which means that the corresponding literal"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732c-934e-11ee-9770-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 5, "chroma:document": "Fig.1 Clause Formation in a Tsetlin Machine126 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732d-934e-11ee-944d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "is excluded from the clause. For states from N+1t o2N, the decision becomes include,\ni.e., the literal is included instead. The states of all the TAs in all of the clauses are jointly\nstored in the matrix A:A=(aj,k)\u03f5{1,...,2N }m\u00d72fwithjandkreferring to the clause\nand the literal respectively. Thus II\nj={k|aj,k>N , 1\u2264k\u22642f}encompasses the literal\nindices contained in II\nj. Figure 1b shows the structure of the clauses and the voting scheme\ndiagrammatically."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732e-934e-11ee-8748-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "diagrammatically.\nThe number of clauses employed is a user set parameter m.H a l fo ft h em clauses are\nassigned positive polarity (C+\nj). The other half is assigned negative polarity ( C\u2212\nj). The\nclause outputs, in turn, are combined into a classification decision through summation:\nv=\u2211m/2\nj=1C+\nj(X)\u2212\u2211m/2\nj=1C\u2212\nj(X).( 2)\nIn effect, the positive clauses vote for y=1a n dt h en e g a t i v ef o ry =0. Classification is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d732f-934e-11ee-8ca4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "performed based on a majority vote, using the unit step function: \u02c6y=u(v)=1ifv\u2265\n0else0. The classifier \u02c6y=u(x1\u00afx2+\u00afx1x2\u2212x1x2\u2212\u00afx1\u00afx2), for instance, captures the XOR-\nrelation.\n3.1.2 Learning\nAlg. 1 encompasses the entire learning procedure. We observe that learning is performed bya team of 2f Tsetlin automaton (TAs) per clause, one Tsetlin automaton (TA) per literal l\nk"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7330-934e-11ee-a0b5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "k\n(Alg. 1, Step 2). Each Tsetlin automaton (TA) has two actions \u2013 Include or Exclude \u2013 anddecides whether to include its designated literal l\nkin its clause.\nTM learn on-line, processing one training example (X,y) at a time (Step 7). The Tsetlin\nautomaton (TAs) first produce a new configuration of clauses (Step 8), C+\n1,...,C\u2212\nn/2,\nfollowed by calculating a voting sum v(Step 9).\nFeedback are then handed out stochastically to each Tsetlin automaton (TAs) team. The"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7331-934e-11ee-ae8e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "difference \u03f5between the clipped voting sum vcand a user-set voting target Tdecides the\nprobability of each Tsetlin automaton (TAs) team receiving feedback (Steps 12-20). Notethat the voting sum is clipped to normalize the feedback probability. The voting target fory=1i sTand for y=0i ti s\u2212T. Observe that for any input X, the probability of reinforc-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7332-934e-11ee-a7c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "ing a clause gradually drops to zero as the voting sum approaches the user-set target. Thisensures that clauses distribute themselves across the frequent patterns, rather than missingsome and over-concentrating on others.\nClauses receive two types of feedback. Type I feedback produces frequent patterns, while\nType II feedback increases the discrimination power of the patterns.\nType I feedback is given stochastically to clauses with positive polarity when y=1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7333-934e-11ee-a22e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "and to clauses with negative polarity when y=0. Each clause, in turn, reinforces its Tsetlin\nautomaton (TAs) based on: (1) its output C\nj(X); (2) the action of the Tsetlin automaton (TA)\n\u2013 Include or Exclude; and (3) the value of the literal lkassigned to the Tsetlin automaton\n(TA). Two rules govern Type I feedback:\n\u2013 Include is rewarded and Exclude is penalized with probabilitys\u22121\nswhenever Cj(X)=\n1andlk=1. This reinforcement is strong (triggered with high probability) and makes"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7334-934e-11ee-884e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 6, "chroma:document": "the clause remember and refine the pattern it recognizes in X.1\n\u2013 Include is penalized and Exclude is rewarded with probability1\nswhenever Cj(X)=\n0orlk=0. This reinforcement is weak (triggered with low probability) and coarsens\ninfrequent patterns, making them frequent.\n1Note that the probabilitys\u22121\nsis replaced by 1 when boosting true positives.127 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7335-934e-11ee-80e4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 7, "chroma:document": "Above, the user-configurable parameter scontrols pattern frequency, i.e., a higher s\nproduces less frequent patterns.\nType II feedback is given stochastically to clauses with positive polarity when y=0\nand to clauses with negative polarity when y=1. It penalizes Exclude whenever Cj(X)=\n1andlk=0. Thus, this feedback produces literals for discriminating between y=0a n d\ny=1, by making the clause evaluate to 0 when facing its competing class. Further details"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7336-934e-11ee-af38-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 7, "chroma:document": "can be found in Granmo (2018).128 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7337-934e-11ee-a243-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "3.2 Relationaltsetlinmachine\nIn this section, we introduce the Relational TM, which is a major contribution of this paper.\nIt is designed to take advantage of logical structures appearing in natural language andprocess them in a way that leads to a compact, logic-based representation, which can ulti-mately reduce the gap between structured and unstructured data. While the PropositionalTM operates on propositional input variables X=(x\n1...,x f), building propositional con-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7338-934e-11ee-873a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "1...,x f), building propositional con-\njunctive clauses, the Relational TM processes relations, variables and constants, buildingHorn clauses. Based on Fig. 2and Alg. 2, we here describe how the Relational TM builds"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7339-934e-11ee-b941-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "upon the original TM in three steps. First, we establish an approach for dealing with rela-tions and constants. This is done by mapping the relations to propositional inputs, allowingthe use of a standard TM. We then introduce Horn clauses with variables, showing howthis representation detaches the TM from the constants, allowing for a more compact rep-resentation compared to only using propositional clauses. We finally introduce a novelconvolution scheme that effectively manages multiple"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733a-934e-11ee-b273-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "scheme that effectively manages multiple possible mappings from constants tovariables. While the mechanism of convolution remains the same as in the original (Granmoet al., 2019), what we wish to attain from using it in a Relational TM context is completely"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733b-934e-11ee-876d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "different, as explained in the following.\n3.2.1 Model-theoreticalinterpretation\nThe concept of Relational TM can be grounded in the model-theoretical interpretation of alogic program without functional symbols and with a finite Herbrand model (Lloyd, 1984;\nKowalski, 2014). The ability to represent learning in the form of Horn clauses is extremely\nuseful due to the fact that Horn clauses are both simple, as well as powerful enough todescribe any logical formula (Kowalski, 2014)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733c-934e-11ee-b9dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "N e x t ,w ed e f i n et h eHerbrand model of a logic program. A Herbrand Base (HB) is the set\nof all possible ground atoms, i.e., atomic formulas without variables, obtained using predi-cate names and constants in a logic program P.A Herbrand Interpretation i sas u b s e t Iof\nthe Herbrand Base (I\u2286HB). To introduce the Least Herbrand Model we define the imme-\ndiate consequence operator TP:P(HB)\u2192P(HB),w h i c hf o ra nHerbrand Interpretation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733d-934e-11ee-a2d5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "Iproduces the interpretation that immediately follows from Iby the rules (Horn clauses)\nin the program P:\nTP(I) ={A\n0\u2208HB|A0\u2190A1,...,A n\n\u2208ground(P) \u2227{A1,...,A n}\u2286I}\u222aI.\nThe least fixed point lfp(TP) of the immediate consequence operator with respect to\nsubset-inclusion is the Least Herbrand Model (LHM) of the program P.LHM identifies the\nsemantics of the program P:i ti st h e Herbrand Interpretation that contains those and only\nthose atoms that follow from the program:\n\u2200A\u2208HB:P|=A\u21d4A\u2208LHM."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733e-934e-11ee-b4b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 8, "chroma:document": "\u2200A\u2208HB:P|=A\u21d4A\u2208LHM.\nAs an example, consider the following program P:\np(a) .q(c).\nq(X)\u2190p(X) .\nIts Herbrand base is\nHB={p(a),p(c),q(a),q(c)},129 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d733f-934e-11ee-8f4a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 9, "chroma:document": "Fig.2 Relational TM processing steps130 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7340-934e-11ee-95cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "and its Least Herbrand Model is:\nLHM =lfp(TP)={p(a),q(a),q(c)},\nwhich is the set of atoms that follow from the program P.\n3.2.2 Learningproblem\nLetA={a1,a2,...,a q}be a finite set of constants and let R={r1,r2,...,r p}be a\nfinite set of relations of arity wu\u22651,u\u2208{1,2,...,p }, which forms the alphabet /Sigma1.T h e\nHerbrand base\nHB ={r1(a1,a2,...,a w1),r1(a2,a1,...,a w1),\n...,r p(a1,a2,...,a wp),rp(a2,a1,...,a wp),...} (3)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7341-934e-11ee-8f24-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "is then also finite, consisting of all the ground atoms that can be expressed using AandR.\nWe also have a logic program P, with program rules expressed as Horn clauses without\nrecursion. Each Horn clause has the form:\nB0\u2190B1,B2,\u00b7\u00b7\u00b7,Bd.( 4)\nHere,Bl,l\u2208{0,...,d },is an atom ru(Z1,Z2,...,Z wu)with variables Z1,Z2,...,\nZwu, or its negation \u00acru(Z1,Z2,...,Z wu). The arity of ruis denoted by wu.\nNow, letXbe a subset of the LHM of P,X\u2286lfp(TP),a n dl e t Ybe the subset of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7342-934e-11ee-9d93-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "LHM that follows from Xdue to the Horn clauses in P. Further assume that atoms are\nrandomly removed and added to XandYto produce a possibly noisy observation (\u02dcX,\u02dcY),\ni.e.,\u02dcXand\u02dcYare not necessarily subsets of lfp (TP).The learning problem is to predict the\natoms inYfrom the atoms in \u02dcXby learning from a sequence of noisy observations (\u02dcX,\u02dcY),\nthus uncovering the underlying program P.\n3.2.3 Relationaltsetlinmachinewithconstants"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7343-934e-11ee-84df-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "3.2.3 Relationaltsetlinmachinewithconstants\nWe base our Relational TM on mapping the learning problem to a Propositional TM pattern\nrecognition problem. We consider Horn clauses without variables first. In brief, we mapevery atom in HBto a propositional input x\nk, obtaining the propositional input vector X=\n(x1,...,x o)(cf. Section 3.1). That is, consider the w-arity relation ru\u2208R,w h i c ht a k e s\nwsymbols from Aas input. This relation can thus take qwunique input combinations. As"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7344-934e-11ee-87bd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "an example, with the constants A={a1,a2}and the binary relations R={r1,r2},w e\nget 8 propositional inputs: x1,1\n1\u2261r1(a1,a1);x1,2\n1\u2261r1(a1,a2);x2,1\n1\u2261r1(a2,a1);x2,2\n1\u2261\nr1(a2,a2);x1,1\n2\u2261r2(a1,a1);x1,2\n2\u2261r2(a1,a2);x2,1\n2\u2261r2(a2,a1);a n dx2,2\n2\u2261r2(a2,a2).\nCorrespondingly, we perform the same mapping to get the propositional output vector Y.\nFinally, obtaining an input (\u02dcX,\u02dcY), we set the propositional input xkto true iff its corre-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7345-934e-11ee-a8ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "sponding atom is in \u02dcX, otherwise it is set to false. Similarly, we set the propositional output\nvariable ymto true iff its corresponding atom is in \u02dcY, otherwise it is set to false.\nClearly, after this mapping, we get a Propositional TM pattern recognition problem that\ncan be solved as described in Section 3.1for a single propositional output ym.T h i si s\nillustrated as Step 1 in Fig. 2.\n3.2.4 DetachingtherelationalTMfromconstants"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7346-934e-11ee-993d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 10, "chroma:document": "3.2.4 DetachingtherelationalTMfromconstants\nThe TM can potentially deal with thousands of propositional inputs. However, we now detachour Relational TM from the constants, introducing Horn clauses with variables. Our intent131 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7347-934e-11ee-9be7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "is to provide a more compact representation of the program and to allow generalization\nbeyond the data. Additionally, the detachment enables faster learning even with less data.\nLetZ={Z1,Z2,...,Z z}bezvariables representing the constants appearing in an\nobservation (\u02dcX,\u02dcY). Here, zis the largest number of unique constants involved in any\nparticular observation (\u02dcX,\u02dcY), each requiring its own variable.\nSeeking Horn clauses with variables instead of constants, we now only need to consider"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7348-934e-11ee-822a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "atoms over variable configurations (instead of over constant configurations). Again, wemap the atoms to propositional inputs to construct a propositional TM learning problem.That is, each propositional input x\nkrepresents a particular atom with a specific variable\nconfiguration: xk\u2261ru(Z\u03b11,Z\u03b12,...,Z \u03b1wu), with wubeing the arity of ru. Accordingly,\nthe number of constants in Ano longer affects the number of propositional inputs xkneeded"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7349-934e-11ee-bb51-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "to represent the problem. Instead, this is governed by the number of variables in Z(and,\nagain, the number of relations in R). That is, the number of propositional inputs is bounded\nbyO(zw), with wbeing the largest arity of the relations in R.\nTo detach the Relational TM from constants, we first replace the constants in \u02dcYwith vari-\nables, from left to right. Accordingly, the corresponding constants in \u02dcXis also replaced with"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734a-934e-11ee-8a0f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "the same variables (Step 2, Fig. 2). Finally, the constants now remaining in \u02dcXis arbitrarily\nreplaced with additional variables (Step 3, Fig. 2).\n3.2.5 Relationaltsetlinmachineconvolutionovervariableassignmentpermutations"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734b-934e-11ee-a044-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "Since there may be multiple ways of assigning variables to constants, the above approachmay produce redundant rules. One may end up with equivalent rules whose only differ-ence is syntactic, i.e., the same rules are expressed using different variable symbols. Thisis illustrated in Step 3 of Fig. 2, where variables can be assigned to constants in two ways."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734c-934e-11ee-b1c7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "To avoid redundant rules, the Relational TM produces all possible permutations of variableassignments. To process the different permutations, we finally perform a convolution overthe permutations in Step 4, employing a TM convolution operator (Granmo et al., 2019).\nThe target value of the convolution is the truth value of the consequent (Step 5).\n3.2.6 Walk-throughofalgorithmwithexamplelearningproblem\nFigure 2contains an example of the process of detaching a Relational TM from constants."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734d-934e-11ee-ad4d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "We use the parent-grandparent relationship as an example, employing the following Hornclause.\ngrandparent(Z\n1,Z2)\u2190parent(Z 1,Z3),parent(Z 3,Z2).\nWe replace the constants in each training example with variables, before learning theclauses. Thus the Relational TM never \u201csees\u201d the constants, just the generic variables.\nAssume the first training example isIn put :parent(Bob, Mary) =1;\nTarge to u t pu t :child(Mary, Bob) =1.\nThen Mary is replaced with Z\n1and Bob with Z2in the target output:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734e-934e-11ee-84d1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 11, "chroma:document": "1and Bob with Z2in the target output:\nIn put :parent(Bob, Mary) =1;\nTarge to u t pu t :child(Z 1,Z 2)=1.\nWe perform the same exchange for the input, getting:In put :parent(Z\n2,Z 1)=1;\nTarge to u t pu t :child(Z 1,Z 2)=1.\nHere, \u201cparent(Z 2,Z1)\u201d is treated as an input feature by the Relational TM. That is,\n\u201cparent(Z 2,Z1)\u201d is seen as a single propositional variable that is either 0 or 1, and the name132 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d734f-934e-11ee-8ffd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "of the variable is simply the string \u201cparent( Z2,Z1)\u201d. The constants may be changing from\nexample to example, so next time it may be Mary and Ann. However, they all end up as Z1\norZ2after being replaced by the variables. After some time, the Relational TM would then\nlearn the following clause:\nchild(Z 1,Z 2)\u2190parent(Z 2,Z1).\nThis is because the feature \u201cparent( Z2,Z1)\u201d predicts \u201cchild( Z1,Z2)\u201d perfectly. Other\ncandidate features like \u201cparent( Z1,Z1)\u201d or \u201cparent( Z2,Z3)\u201d are poor predictors of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7350-934e-11ee-8c27-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "\u201cchild(Z 1,Z2)\u201d and will be excluded by the TM. Here, Z3is a free variable representing\nsome other constant, different from Z1andZ2.\nThen the next training example comes along:\nIn put :parent(Bob, Mary) =1;\nTarge to u t pu t :child(Jane, Bob) =0.\nAgain, we start with replacing the constants in the target output with variables:In put :parent(Bob, Mary) =1;\nTarge to u t pu t :child(Z\n1,Z 2)=0\nwhich is then completed for the input:In put :parent(Z\n2,M a r y ) =1;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7351-934e-11ee-b2ee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "2,M a r y ) =1;\nTarge to u t pu t :child(Z 1,Z 2)=0.\nThe constant Mary was not in the target output, so we introduce a free variable Z3for\nrepresenting Mary:\nIn put :parent(Z 2,Z 3)=1;\nTarge t o u t pu t :child(Z 1,Z 2)=0.\nThe currently learnt clause was:child(Z\n1,Z 2)\u2190parent(Z 2,Z1).\nThe feature \u201cparent(Z 2,Z1)\u201d is not present in the input in the second training example,\nonly \u201cparent(Z 2,Z3)\u201d. Assuming a closed world, we thus have\nparent(Z 2,Z1)=0."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7352-934e-11ee-9884-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "parent(Z 2,Z1)=0.\nAccordingly, the learnt clause correctly outputs 0.For some inputs, there can be many different ways variables can be assigned to constants\n(for the output, variables are always assigned in a fixed order, from Z\n1toZz). Returning to\nour grandparent example in Fig. 2,i fw eh a v e :\nIn put :parent(Bob,Mary) ;parent(Mary,Peter) ; parent(Bob,Jane)\nTarge to u t pu t :grandparent(Bob,Peter) .\nreplacing Bob with Z1and Peter with Z2,w eg e t :"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7353-934e-11ee-bf33-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "replacing Bob with Z1and Peter with Z2,w eg e t :\nIn put :parent(Z 1,Mary) ;parent(Mary,Z 2);\nparent(Z 1,Jan e)\nTarge t o u t pu t :grandparent(Z 1,Z2).\nAbove, both Mary and Jane are candidates for being Z3. One way to handle this ambigu-\nity is to try both, and pursue those that make the clause evaluate correctly, which is exactlyhow the TM convolution operator works (Granmo et al., 2019).\nNote that above, there is an implicit existential quantifier over Z\n3.T h a ti s ,\u2200Z 1,Z2"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7354-934e-11ee-8857-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "3.T h a ti s ,\u2200Z 1,Z2\n(\u2203Z 3(parent(Z 1,Z3)\u2227parent(Z 3,Z2)\u2192grandparent(Z 1,Z2)).\nA practical view of how the TM learns in such a scenario is shown in Fig. 3. Continuing"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7355-934e-11ee-b1b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 12, "chroma:document": "in the same vein as the previous examples, the Input Text in the figure is a set of state-ments, each followed by a question. The text is reduced to a set of relations, which act as thefeatures for the TM to learn from. The figure illustrates how the TM learns relevant infor-mation (while disregarding the irrelevant), in order to successfully answer a new question133 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7356-934e-11ee-aeff-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 13, "chroma:document": "Fig.3 The Relational TM in operation134 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7357-934e-11ee-91b2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "(Test Document). The input text is converted into a features vector which indicates the pres-\nence or absence of relations ( R1,R2) in the text, where R1andR2are respectively MoveTo\n(in the statements) and WhereIs (in the question) relations. For further simplification and\ncompactness of representation, instead of using person and location names, those specificdetails are replaced by (P\n1,P2)a n d( L1,L2), respectively. In each sample, the person name"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7358-934e-11ee-805d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "that occurs first is termed P1throughout, the second unique name is termed P2, and so on,\nand similarly for the locations. As seen in the figure, the TM reduces the feature-set rep-resentation of the input into a set of logical conditions or clauses, all of which togetherdescribe scenarios in which the answer is L\n2(or Location 2).\nRemark 1 We now return to the implicit existential and universal quantifiers of the Horn"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7359-934e-11ee-bab1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "clauses, exemplified in: \u2200Z1,Z2(\u2203Z 3(parent (Z1,Z3)\u2227parent (Z3,Z2)\u2192 grandparent\n(Z1,Z2)). A main goal of the procedure in Fig. 2is to correctly deal with the quantifiers\n\u201cfor all\u201d and \u201cexists\u201d. \u201cFor all\u201d maps directly to the TM architecture because the TM isoperating with conjunctive clauses and the goal is to make these evaluate to 1 (True) when-ever the learning target is 1. \u201cFor all\u201d quantifiers are taken care of in Step 3 of the relationallearning procedure."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735a-934e-11ee-97f3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "Remark 2 \u201cExists\u201d is more difficult because it means that we are looking for a specific value\nfor the variables in the scope of the quantifier that makes the expression evaluate to 1. This\nis handled in the Steps 4-6 in Fig. 2, by evaluating all alternative values (all permutations"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735b-934e-11ee-90a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "with multiple variables involved). Some values make the expression evaluate to 0 (False)and some make the expression become 1. If none makes the expression 1, the output ofthe clause is 0. Otherwise, the output is 1. Remarkably, this is exactly the behavior of theTM convolution operator defined in Granmo et al. ( 2019), so we have an existing learning"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735c-934e-11ee-84cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "procedure in place to deal with the \u201cExists\u201d quantifier. (If there exists a patch in the imagethat makes the clause evaluate to 1, the clause evaluates to 1).\n4 QAinarelationalTMframework\nIn this section, we describe the general pipeline to reduce natural language text into amachine-understandable relational representation to facilitate question answering. Fig. 4"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735d-934e-11ee-a0fa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "shows the pipeline diagrammatically, with a small example. Throughout this section, wemake use of two toy examples in order to illustrate the steps. One of them is derivedfrom a standard question answering dataset (Weston et al., 2015). The other is a simple\nhandcrafted dataset, inspired by Kowalski ( 1979), that refers to parent-child relationships"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735e-934e-11ee-a1a6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "among a group of people. Both datasets consist of instances, where each instance is a setof two or more statements, followed by a query. The expected output for each instanceis the answer to the query based on the statements. We would like to clarify here, thatwe have chosen to highlight the details of our method with toy examples in this work,due to ease of explainability. The ease arises primarily from the fact that the relationsthat form a part of these two sets are well-understood, simple"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d735f-934e-11ee-8a07-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 14, "chroma:document": "of these two sets are well-understood, simple to describe and have well-defined interactions between them. Further expansions on more advanced datasets, such asSQUAD/SQUADv2.0 (which involves question-answering from longer context texts), aswell as on other real-world data is currently work in progress and beyond the scope of thisparticular paper.135 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7360-934e-11ee-bf58-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "Fig.4 General Pipeline\n4.1 Relationextraction\nAs a first step, we need to extract the relation(s) present in the text. A relation here is a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7361-934e-11ee-8c8b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "connection between two (or more) elements of a text. As discussed before, relations occurin natural language, and reducing a text to its constituent relations makes it more structuredwhile ignoring superfluous linguistic elements, leading to easier understanding. We assumethat our text consists of simple sentences, that is, each sentence contains only one relation.The relation found in the query is either equal to, or linguistically related to the relationsfound in the statements preceding the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7362-934e-11ee-abc6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "relationsfound in the statements preceding the query."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7363-934e-11ee-b036-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "Table 1shows examples of Relation Extraction on our two datasets. In Example:Movement,\neach statement has the relation \u201cMoveTo\u201d, while the query is related to \u201cLocation\u201d. The\u201cLocation\u201d can be thought of as a result of the \u201cMoveTo\u201d relations. Example:Parentage has\u201cParent\u201d relations as the information and \u201cGrandparent\u201d as the query.\n4.2 Entityextraction"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7364-934e-11ee-ba3c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "Once the relations have been identified, the next step is to identify the elements of the text(or the entities) that are participating in the respective relations. Doing so allows us to furtherenrich the representation with the addition of restrictions (often real-world ones), whichallow the Relational TM to learn rules that best represent actions and their consequences in aconcise, logical form. Since the datasets we are using here consist only of simple sentences,each relation is limited to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7365-934e-11ee-b241-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "of simple sentences,each relation is limited to having a maximum of two entities (the relations are unary or binary)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7366-934e-11ee-8358-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 15, "chroma:document": "Table1 Relation extraction\nSentence Relation\nMary went to the of\ufb01ce. MoveTo\nJohn moved to the hall MoveTo\nWhere is Mary? Location\nExample:MovementSentence Relation\nBob is a parent of Mary. Parent\nBob is a parent of Jane. Parent\nMary is a parent of Peter Parent\nIs Bob a grandparent of Peter? Grandparent\nExample:Parentage136 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7367-934e-11ee-bbed-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "T\nable2 Entity extractionSentence Relation Entities\nMary went to the of\ufb01ce. MoveTo Mary, of\ufb01ce\nJohn moved to the hall MoveTo John, hall\nWhere is Mary? Location Mary, ?\nExample:Movement\nSentence Relation Entities\nBob is a parent of Mary. Parent Bob, Mary\nBob is a parent of Jane. Parent Bob, Jane\nMary is a parent of Peter Parent Mary, Peter\nIs Bob a grandparent of Peter? Grandparent Bob, Peter\nExample:Parentage\nIn this step, the more external world knowledge that can be combined with the extracted"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7368-934e-11ee-804f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "entities, the richer the resultant representation. In Table 2, Example:Movement, we could\nadd the knowledge that \u201cMoveTo\u201d relation always involves a \u201cPerson\u201d and a \u201cLocation\u201d. Or\nin Example:Parentage, \u201cParent\u201d is always between a \u201cPerson\u201d and a \u201cPerson\u201d. This could,for example, prevent questions like \u201cJean-Joseph Pasteur was the father of Louis Pasteur.Louis Pasteur is the father of microbiology. Who is the grandfather of microbiology?\u201d"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7369-934e-11ee-93ef-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "Note that, as per Fig. 4, it is only possible to start answering the query after both Rela-\ntion Extraction and Entity Extraction have been performed, and not before. Knowledgeof the Relation also allows us to narrow down possible entities for answering the querysuccessfully.\n4.3 Entitygeneralization"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736a-934e-11ee-8faa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "One of the drawbacks of the relational representation is that there is a huge increase in thenumber of possible relations as more and more examples are processed. One way to reducethe spread is to reduce individual entities from their specific identities to a more gener-alised identity. Let us consider two instances : \u201cMary went to the office. John moved to thehall. Where is Mary?\u201d and \u201cSarah moved to the garage. James went to the kitchen. Whereis Sarah?\u201d. Without generalization, we end up with"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736b-934e-11ee-86f7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "Sarah?\u201d. Without generalization, we end up with six different relations : MoveTo(Mary,Office), MoveTo(John, Hall), Location(Mary), MoveTo(Sarah, Garage), MoveTo(James,Kitchen), Location(Sarah). However, to answer either of the two queries, we only need therelations pertaining to the query itself. Taking advantage of that, we can generalize bothinstances to just 3 relations: MoveTo(Person"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736c-934e-11ee-92e2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "1, Location 1), MoveTo(Person 2, Location 2)a n d\nLocation(Person 1).\nIn order to prioritise, the entities present in the query relation are the first to be general-\nized. All occurrences of those entities in the relations preceding the query are also replacedby suitable placeholders (Table 3).\nThe entities present in the other relations are then replaced by what can be considered as\nfree variables (since they do not play a role in answering the query) (Table 4)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736d-934e-11ee-baea-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "In the next section we explain how this relational framework is utilized for question\nanswering using TM.\nWe have not gone into the details of the relation extraction and entity extraction processes"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736e-934e-11ee-b2fc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "in this work. Depending upon the initial data, there can be different strategies for the same.In a related work using TM on natural text data, the researchers show that a TM basedclassification mechanism successfully identifies various semantic relation the text. Such asystem could be a possible way to identify and extract various relations from the text. Onthe other hand, there are also existing methods that can be used for relation identification.137 Journal of Intelligent Information Systems"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d736f-934e-11ee-b13c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 16, "chroma:document": "Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7370-934e-11ee-9aad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "Table3 Entity generalization : Part 1\nSentence Relation Entities Reduced Relation\nMary went to the of\ufb01ce. MoveTo Mary, of\ufb01ce Source MoveTo (X,of\ufb01ce)\nJohn moved to the hall. MoveTo John, hall Source MoveTo (John, hall)\nWhere is Mary? Location Mary, ? Target Location(X,? )\nExample:Movement\nSentence Relation Entities Reduced Relation\nBob is a parent of Mary. Parent Bob, Mary Source Parent (X,Mary)\nBob is a parent of Jane. Parent Bob, Jane Source Parent (X,Jane)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7371-934e-11ee-90b5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "Mary is a parent of Peter Parent Mary, Peter Source Parent (Mary, Y)\nIs Bob a grandparent\nof Peter?Grand-parentBob, Peter Target Grandparent (X, Y )\nExample:Parentage\nFor example, in one of our ongoing work, we use data derived from the PULSAR system,\nwhere raw natural language text on various human rights violations from across the worldhas already been processed by experts into a database-based ecosystem (Park et al., 2020)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7372-934e-11ee-8679-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "The relational aspect of a database system allows for access to relations and entities thathave already been extracted with a certain level of confidence, and leaves our system tofocus on enhancing existing logical constructs or extracting newer ones based on the giveninformation. A similar approach would be to use available knowledge-bases, e.g. NELL orFreebase, to use extracted relations as an initial \u2019seed\u2019 set to infer or extract newer ones.\n4.4 ComputationalcomplexityinrelationTM"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7373-934e-11ee-8d69-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "4.4 ComputationalcomplexityinrelationTM\nOne of the primary differences between the relational framework proposed in this paperversus the existing TM framework lies in Relation Extraction and Entity Generalization.The reason for these steps is that they allow us more flexibility in terms of what the TMlearns, while keeping the general learning mechanism unchanged.\nExtracting relations allows the TM to focus only on the major operations expressed via"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7374-934e-11ee-9be3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "language, without getting caught up in multiple superfluous expressions of the same thing.It also enables to bridge the gap between structured and unstructured data. Using relationshelps the resultant clauses be closer to real-world phenomena, since they model actions andconsequences, rather than the interactions between words.\nEntity Generalization allows the clauses to be succinct and precise, adding another layer"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7375-934e-11ee-b07f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "of abstraction away from specific literals, much like Relation Extraction. It also gives the\nTable4 Entity Generalization : Part 2\nSentence Relation Entities Reduced Relation\nMary went to the of\ufb01ce. MoveTo Mary, of\ufb01ce Source MoveTo (X, A )\nJohn moved to the hall. MoveTo John, hall Source MoveTo (Y , B)\nWhere is Mary? Location Mary, ? Target Location(X,? )\nExample:Movement\nSentence Relation Entities Reduced Relation\nBob is a parent of Mary. Parent Bob, Mary Source Parent (X, Z )"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7376-934e-11ee-8b43-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 17, "chroma:document": "Bob is a parent of Jane. Parent Bob, Jane Source Parent (X,W )\nMary is a parent of Peter Parent Mary, Peter Source Parent (Z, Y)\nIs Bob a grandparent\nof Peter?Grand-parentBob, Peter Target Grandparent (X, Y )\nExample:Parentage138 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7377-934e-11ee-b56c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "added benefit of making the learning of the TM more generalized. In fact, due to this pro-\ncess, the learning reflects \u2018concepts\u2019 gleaned from the training examples, rather than theexamples themselves.\nTo evaluate computational complexity, we employ the three costs \u03b1,\u03b2,a n d\u03b3,w h e r e\nin terms of computational cost, \u03b1is cost to perform the conjunction of two bits, \u03b2is cost\nof computing the summation of two integers, and \u03b3is cost to update the state of a single"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7378-934e-11ee-a428-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "automaton. In a Propositional TM, the worst case scenario would be the one in which all theclauses are updated. Therefore, a TM with mclauses and an input vector of ofeatures, needs\nto perform (2o+1)\u00d7mnumber of Tsetlin automaton (TA) updates for a single training\nsample. For a total of dtraining samples, the total cost of updating is d\u00d7\u03b3\u00d7(2o+1)\u00d7m.\nOnce weight updates have been successfully performed, the next step is to calculate"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7379-934e-11ee-bbd7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "the clause outputs. Here the worst case is represented by all the clauses including all thecorresponding literals, giving us a cost of \u03b1\u00d72o\u00d7m(for a single sample).\nThe last step involves calculating the difference in votes from the clause outputs, thus\nincurring a per-sample cost of \u03b2\u00d7(m\u22121).\nTaken together, the total cost function for a Propositional TM can be expressed as:\nf( d)=d\u00d7[(\u03b3\u00d7(2o+1)\u00d7m)+(\u03b1\u00d72o\u00d7m)+(\u03b2\u00d7(m\u22121))] (5)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737a-934e-11ee-a242-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "f( d)=d\u00d7[(\u03b3\u00d7(2o+1)\u00d7m)+(\u03b1\u00d72o\u00d7m)+(\u03b2\u00d7(m\u22121))] (5)\nExpanding this calculation to the Relation TM scenario, we need to account for the extra"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737b-934e-11ee-8e94-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "operations being performed, as detailed earlier: Relation Extraction and Entity Generaliza-tion. The number of features per sample is restricted by the number of possible relations,both in the entirety of the training data, as well as only in the context of a single sample.For example, in the experiments involving \u201cMoveTo\u201d relations, we have restricted our datato have 3 statements, followed by a question (elaborated further in the next section). Eachstatement gives rise to a single \u201cMoveTo\u201d"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737c-934e-11ee-a9b9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "Eachstatement gives rise to a single \u201cMoveTo\u201d relation, which has 2 entities (a location and aperson)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737d-934e-11ee-ab2e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "When using the textual constants (i.e., without Entity Generalization), the maximum num-\nber of possible features thus becomes equal to the number of possible combination between\nthe unique entities in the relations. Thus if each sample contains rRelations,\n and a Rela-\ntionRinvolves edifferent entities (E 1,E2,...,E e), and cardinality of sets E1,E2,...,E ebe\nrepresented as |E1|,|E2|,...,|Ee|, the number of features in Eq. 5can be re-written as\no=[ |E1|\u00d7|E2|\u00d7...\u00d7|Ee|] \u00d7r."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737e-934e-11ee-98dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "o=[ |E1|\u00d7|E2|\u00d7...\u00d7|Ee|] \u00d7r.\nAs discussed earlier in Section 4.3as well as shown in the previous paragraph, this\nresults in a large o, since it depends on the number of unique textual elements in each of the\nentity sets. Using Entity Generalization, the number of features no longer depends on thecardinality of set E\nn,1\u2264n\u2264ein the context of the whole dataset. Instead, it only depends\non the context of the single sample. Thus, if each sample contains rRelations, and a Relation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d737f-934e-11ee-b6a3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "Rinvolves edifferent entities (E 1,E2,...,E e), and maximum possible cardinality of sets\nE1,E2,...,E eare|E1|=|E2|=... =|En|=r , the number of features simply become :\no=r(e+1).\nIn most scenarios, this number is much lower than the one obtained without the use of\nEntity Generalization.\nA little further modification is required when using the convolutional approach. In cal-\nculating f( d) , the measure of oremains the same as we just showed with/without Entity"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7380-934e-11ee-9d0e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "Generalization. However the second term in the Eq. 5, which refers to the calculation of\nclause outputs ( d\u00d7\u03b1\u00d72o\u00d7m), changes due to the difference in the mechanism of cal-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7381-934e-11ee-8610-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 18, "chroma:document": "culating outputs for convolutional and non-convolutional approaches. In the convolutionalapproach, with Entity Generalization, we need to consider free and bound variables in thefeature representation. Bound variables are the ones which are linked by appearance in the139 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7382-934e-11ee-86aa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "source and target relations, while the other variables, which are independent of that restric-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7383-934e-11ee-9dd7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "tion, can be referred to as the free variables. Each possible permutation of the free variablesin different positions are used by the convolutional approach to determine the best genericrule that describes the scenario. In certain scenarios, it may be possible to have certain per-mutations among the bound variables as well, without violating the restrictions added bythe relations. One such scenario is detailed with an example in Section 5.2, where a bound"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7384-934e-11ee-a356-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "\u201cPerson\u201d entity can be permuted to allow any other \u201cPerson\u201d entity, as long as the order of\u201cMoveTo\u201d relations are not violated. However, it is difficult to get a generic measure forthe same, which would work irrespective of the nature of the relation (or their restrictions).Therefore, for the purpose of this calculation, we only take into account the permutationsafforded to us by the free variable. Using the same notation as before, if each sample con-tainsrRelations, and a Relation Rinvolves"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7385-934e-11ee-af51-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "con-tainsrRelations, and a Relation Rinvolves edifferent entities, the total number of variables"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7386-934e-11ee-83ff-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "isR\u00d7e.O ft h e s e ,i f vis the number of free variables, then they can be arranged in v!differ-\nent ways. Assuming vis constant for all samples, the worst case f( d) from Eq. 5can thus\nbe rewritten as:\nf( d)=d\u00d7[(\u03b3\u00d7(2o+1)\u00d7m)+(v!\u00d7\u03b1\u00d72o\u00d7m)+(\u03b2\u00d7(m\u22121))] (6)\n5 Experimentalstudy\nTo further illustrate how the TM based logic modelling works practically, we employ exam-ples from a standard question answering dataset (Weston et al., 2015), which defines a set"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7387-934e-11ee-a087-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "of prerequisite tasks as part of QA. For the scope of this work, we limit ourselves to thefirst subtask as defined in the dataset, viz. a question that can be answered by the precedingcontext, and the context contains a single supporting fact.\nTo start with, there is a set of statements, followed by a question, as discussed previously."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7388-934e-11ee-85c1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "For this particular subtask, the answer to the question is obtained by a single statement outof the set of statements provided (hence the term, single supporting fact). Choosing sampleswhere the context contained a maximum of 6 sentences, this resulted in a set of 6000 uniquesentences.\nInput:William moved to the office. Susan went to the garden. William walked to the\npantry. Where is William?\nOutput: pantry\nWe assume the following knowledge to help us construct the task:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7389-934e-11ee-adf1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "1. All statements only contain information pertaining to relation MoveTo2. All questions only relate to information pertaining to relation CurrentlyAt3. Relation MoveTo involves 2 entities, such that MoveTo(a,b) :a\u2208{Persons },\nb\u2208{Locations }\n4. Relation CurrentlyAt involves 2 entities, such that CurrentlyAt(a,b) :a\u2208\n{Persons },b\u2208{Locations }\n5. MoveTo is a time-bound relation, it\u2019s effect is superseded by a similar action performed"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738a-934e-11ee-9071-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 19, "chroma:document": "at a later time. In this scenario, \u2018later time\u2019 is indicated by position in text.\n5.1 Withoutentitygeneralization\nThe size of the set of statements from which the model has to identify the correct answerinfluences the complexity of the task. For the purpose of this experiment, the data is capped140 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738b-934e-11ee-9766-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "to have a maximum of three statements per input, and over all has five possible locations.\nThis means that the task for the TM model is reduced to classifying the input into one offive possible classes.\nTo prepare the data for the TM, the first step involves reducing the input to relation-entity"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738c-934e-11ee-9f8a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "bindings. These bindings form the basis of our feature set, which is used to train the TM.We encode sentence order into the features as well, since MoveTo is time-sensitive (asmentioned in assumed world knowledge previously). Consider the following input example:\nInput => MoveT o (S1,William ,Office), MoveT o (S2,Susan,Garden ), MoveT o(S 3,\nWilliam,Pantry) ,Q(William) .\nSince the TM requires binary features, each input is converted to a vector, where each"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738d-934e-11ee-abc0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "element represents the presence (or absence) of the relationship instances. Secondly, the listof possible answers is obtained from the data, which is the list of class labels. Continuingour example, possible answers to the question could be:\nPossible Answers : [Office, Pantry, Garden, Foyer, Kitchen]\nOnce training is complete, we can use the inherent interpretability of the TM to get"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738e-934e-11ee-a4b1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "an idea about how the model learns to discriminate the information given to it. The setof all clauses arrived at by the TM at the end of training represents a global view of thelearning, i.e. what the model has learnt in general. The global view can also be thought ofas a description of the task itself, as understood by the machine. We also have access to alocal snapshot, which is particular to each input instance. The local snapshot involves onlythose clauses that help in arriving at the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d738f-934e-11ee-9db2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "onlythose clauses that help in arriving at the answer for that particular instance."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7390-934e-11ee-990b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "Table 5shows the local snapshot obtained for the above example. As mentioned earlier,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7391-934e-11ee-927b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "the TM model depends on two sets of clauses for each class, a positive set and a negative set.The positive set represents information favour of the class, while the negative set representsthe opposite. The sum of the votes given by these two sets thus represent the final class themodel decides on. As seen in the example, all the classes other than \u201cPantry\u201d receive morenegative votes than positive, making it the winning class. The clauses themselves allow us"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7392-934e-11ee-ba1f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "Table 5 Local Snapshot of Clauses for example \u201cWilliam moved to the office. Susan went to the garden.\nWilliam walked to the pantry. Where is William?\u201d The winning vote is indicated in bold\nClass Clause +/- VotesTotal\nVotes\nfor Class\nOf\ufb01ceQ(William) AND MoveTo(S1,William, Of\ufb01ce) + 12-35Q(William) AND MoveTo(S1,William, Of\ufb01ce)\nAND MoveTo(S3,William, Pantry)- 47\nPantryQ(William) AND MoveTo(S1,William, Of\ufb01ce)\nAND MoveTo(S3,William, Pantry)+ 64+49\nQ(William) AND MoveTo(S1,William, Of\ufb01ce) - 15"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7393-934e-11ee-8451-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 20, "chroma:document": "Q(William) AND MoveTo(S1,William, Of\ufb01ce) - 15\nGardenMoveTo(S2,Susan, Garden) + 12-36Q(William) AND MoveTo(S1,William, Of\ufb01ce)AND MoveTo(S3,William, Pantry)- 48\nFoyer- + 0-106Q(William) AND MoveTo(S1,William, Of\ufb01ce)AND MoveTo(S3,William, Pantry)AND MoveTo(S2,Susan, Garden)\n- 106\nKitchen- + 0-113Q(William) AND MoveTo(S1,William, Of\ufb01ce)AND MoveTo(S3,William, Pantry)AND MoveTo(S2,Susan, Garden)\n- 113141 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7394-934e-11ee-8880-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "to peek into the learning mechanism. For the class \u201cOffice\u201d, a clause captures the informa-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7395-934e-11ee-a732-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "tion that (a) the question contains \u201cWilliam\u201d, and (b) the relationship MoveTo(S1,William,Office) is available. This clause votes in support of the class, i.e. this is an evidence thatthe answer to the question \u201cWhere is William?\u201d may be \u201cOffice\u201d. However, another clauseencapsulates the previous two pieces of information, as well as something more : (c) the rela-tionship MoveTo(S3,William, Pantry) is available. Not only does this clause vote against theclass \u201cOffice\u201d, it also ends up with a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7396-934e-11ee-a010-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "against theclass \u201cOffice\u201d, it also ends up with a larger share of votes than the clause voting positively,thus pushing \u201cOffice\u201d out of contention."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7397-934e-11ee-91d2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "The accuracy obtained over 100 epochs for this experiment was 94.83%, with a F-score\nof 94.80.\n5.1.1 Allowingnegativeliteralsinclauses"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7398-934e-11ee-8251-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "5.1.1 Allowingnegativeliteralsinclauses\nAbove results were obtained by only allowing positive literals in the clauses. The descriptivepower of the TM goes up if negative literals are also added. However, the drawback to thatis, while the TM is empowered to make more precise rules (and by extension, decisions),the overall complexity of the clauses increase, making them less readable. Using the aboveexample again, if we do allow negative literals the positive evidence for Class Office lookslike:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d7399-934e-11ee-9e00-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "Q(William) AND Not(Q(Susan)) AND MoveT o(S 1,William,Office) AND\nNot(MoveTo(S 3,William,Garden)) AND Not(MoveTo(S 3,William,Foyer))\nAND Not(MoveT o(S 3,William,Kitchen)) AND\nNot(MoveTo(S 3,Susan ,Garden ))AN DNot(MoveTo(S 3,Susan,Office))\nAND Not(MoveT o(S 3,Susan,Pantry)) AND Not(MoveTo(S 3,Susan, Foyer))\nAND Not(MoveT o(S 3,Susan,Kitchen)) .\nAt this point, we can see that the use of constants lead to a large number of repetitive"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739a-934e-11ee-917d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "information in terms of the rules learnt by the TM. Generalizing the constants as per theirentity type can prevent this.\n5.2 Entitygeneralization"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739b-934e-11ee-88b6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "Given a set of sentences and a following question, the first step remains the same as in theprevious subsection, i.e. reducing the input to relation-entity bindings. In the second step,we carry out a grouping by entity type, in order to generalize the information. Once theconstants have been replaced by general placeholders, we continue as previously, collectinga list of possible outputs (to be used as class labels), and further, training a TM based modelwith binary feature vectors."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739c-934e-11ee-a955-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "As before, the data is capped to have a maximum of three statements per input.\nContinuing with the same example as above,\nInput:William moved to the office. Susan went to the garden. William walked to the\npantry. Where is William?\nOutput: pantry\n1. Reducing to relation-entity bindings:Input=> MoveT o(S 1,William,Office) ,MoveTo(S 2,Susan,Garden) ,\nMoveTo(S 3,William,Pantry) ,Q(William) .\n2. Generalizing bindings:Modified=> MoveT o(S 1,Per 1,L oc 1),MoveTo(S 2,Per 2,L oc 2),"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739d-934e-11ee-a570-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 21, "chroma:document": "MoveTo(S 3,Per 1,L oc 3),Q(Per 1)\n3. Possible Answers : [Loc1, Loc2, Loc3]142 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739e-934e-11ee-ad1e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "T\nable6 Clause snapshot for \u201dWilliam moved to the office. Susan went to the garden. William walked to the\npantry. Where is William?\u201d after generalization. The winning vote is indicated in bold\nClass Clause +/- VotesTotal\nVotes\nfor Class\nLoc1Q(Per1) AND MoveTo(S1,Per1, Loc1) + 3-44Q(Per1) AND MoveTo(S1,Per1, Loc1)\nAND MoveTo(S3,Per1, Loc3)- 47\nLoc2MoveTo(S1,Per1, Loc1) + 2-88Q(Per1) AND MoveTo(S3,Per1, Loc3) - 90\nLoc3Q(Per1) AND MoveTo(S1,Per1, Loc1)\nAND MoveTo(S3,Per1, Loc3)+ 51+51\n- - 0"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d739f-934e-11ee-8231-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "AND MoveTo(S3,Per1, Loc3)+ 51+51\n- - 0\nThe simplifying effect of generalization is seen right away: even though there are 5 pos-\nsible location in the whole dataset, for any particular instance there are always maximum of\nthree possibilities, since there are maximum three statements per instance.\nAs seen from the local snapshot (Table 6), the clauses formed are much more compact"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a0-934e-11ee-adf0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "and easily understandable. The generalization also releases the TM model from the restric-tion of having had to seen definite constants before in order to make a decision. The modelcan process \u201cRory moved to the conservatory. Rory went to the cinema. Cecil walked to theschool. Where is Rory?\u201d, without needing to have encountered constants \u201cRory\u201d, \u201cCecil\u201d,\u201cschool\u201d, \u201ccinema\u201d and \u201cconservatory\u201d."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a1-934e-11ee-934f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "Accuracy for this experiment over 100 epochs was 99.48% , with a F-score of 92.53.A logic based understanding of the relation \u201cMove\u201d could typically be expressed as :\u2192 MoveTo(S 1,William,office) +MoveTo(S 2,Susan ,garden ) +\nMoveTo(S 3,William,pantry)\n\u2192MoveTo(S 1,P\n1,office) +MoveTo(S 2,P 2,garden ) +MoveTo(S 3,P 1,pan try)\n\u2192MoveTo(S 1,P 1,L1)+MoveTo(S 2,P 2,L2)+MoveTo(S 3,P 1,L3)\n\u2192MoveTo(S 1,P 1,L1)+\u2217+MoveTo(Sn,P 1,Ln)\n=\u21d2Location(P 1,Ln)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a2-934e-11ee-9be0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "=\u21d2Location(P 1,Ln).\nFrom the above two subsections, we can see that with more and more generalization, the\nlearning encapsulated in the TM model can approach what could possibly be a human-levelunderstanding of the world.\nMoreover, as shown in Table 7, compared to the Propositional TM, not only does the\nRelational TM give a more compact and easily understandable explanation, it also has the\nTable7 Representative Winning Clauses for \u201cWilliam moved to the office"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a3-934e-11ee-b75b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "Propositional TM Relational TM Relational TM with entity\ngeneralization\nQ(William) AND Q(Where) AND\nS1William AND S1 Office AND\nS1moved AND S3 William\nAND S3 Pantry AND S3 walked\nAND NOT(S3 Garden) AND\nNOT(S3 Susan)Q(William) ANDMoveTo (S1,William,\nOffice) AND MoveTo(S3,William, Pantry)Q(Per1) AND\nMoveTo(S1,Per1,\nLoc1) ANDMoveTo(S3,Per1, Loc3)\nSusan went to the garden. William walked to the pantry. Where is William?\u201d with Propositional, Relational"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a4-934e-11ee-abf4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 22, "chroma:document": "and Relational TM with Entity Generalization143 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a5-934e-11ee-991f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "added advantage of allowing reasoning to be built on top of obtained explanations (as dis-\ncussed above). How different entities interact with each other in a given environment, andwhat is the result of those interactions, is something that is very important to be able tosuccessfully model the world around us, rather than just the presence (or absence) of cer-tain words in a textual description of that environment. This can only be captured by theRelational TM, and not by the propositional one."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a6-934e-11ee-8f8e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "5.3 Variablepermutationandconvolution\nAs described in Section 3.2.5, we can produce all possible permutations of the available\nvariables in each sample (after Entity Generalization) as long as the relation constraints arenot violated. Doing this gives us more information per sample:\nInput:William moved to the office. Susan went to the garden. William walked to the\npantry. Where is William?\nOutput: pantry"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a7-934e-11ee-95fe-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "pantry. Where is William?\nOutput: pantry\n1. Reducing to relation-entity bindings:Input => MoveT o(S 1,William,Office) ,MoveTo(S 2,Susan ,Garden ) ,\nMoveTo(S 3,William,Pantry) ,Q(William) .\n2. Generalizing bindings:Modified=> MoveT o(S 1,Per 1,L oc 1), MoveTo(S 2,Per 2,L oc 2),\nMoveTo(S 3,Per 1,L oc 3),Q(Per 1)\n3. Performing Permutations:Permuted=> MoveT o(S 1,Per 2,L oc 1), MoveTo(S 2,Per 1,L oc 2),\nMoveTo(S 3,Per 2,L oc 3),Q(Per 2)\n4. Possible Answers : [Loc1, Loc2, Loc3]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a8-934e-11ee-b9a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "4. Possible Answers : [Loc1, Loc2, Loc3]\nThis has two primary benefits. Firstly, in a scenario where the given data does not encom-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73a9-934e-11ee-a34a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "pass all possible structural differences in which a particular information maybe represented,using the permutations allows the TM to view a closer-to-complete representation fromwhich to build it\u2019s learning (and hence, explanations). Moreover, since the TM can learndifferent structural permutations from a single sample, it ultimately requires fewer clausesto learn effectively. In our experiments, permutations using Relational TM Convolutionallowed for up to 1.5 times less clauses than using a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73aa-934e-11ee-a3e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "for up to 1.5 times less clauses than using a non-convolutional approach."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ab-934e-11ee-867d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "As detailed in Section 4.4, convolutional and non-convolutional approaches have differ-\nent computational complexity. Hence, the convolutional approach makes sense only whenthe reduction in complexity from fewer clauses balance the increase due to processing theconvolutional window itself.\n5.4 ComparisonwithsomeotherQAapproaches"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ac-934e-11ee-8aba-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "5.4 ComparisonwithsomeotherQAapproaches\nWe use the following related approaches as a guide for judging the performance of theproposed method on the task (Table 8). All the approaches were trained (where applicable)\nand tested on the same set as described earlier for our proposed approach, i.e. collection\nTable8 Comparing accuracies with other QA approaches\nMethod Relational TM RTM+ Entity\nGen.Mem Net Cross- encoders BART BERT+ Bi-\nEncoder"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ad-934e-11ee-96f6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 23, "chroma:document": "Encoder\nAccuracy 94.8 99.5 98.2 99.6 98.5 84.5144 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ae-934e-11ee-abce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "of 6000 samples from the single supporting fact dataset. Implementations of the same have\nbeen suitably modified from the ParlAI platform (Miller et al., 2017).\n\u2013 MemNet (Weston et al., 2014) integrates basic inference with long-term memory\ncomponent that is used for prediction of output (or answer).\n\u2013 Cross-encoders (Wolf et al., 2019) perform full (cross) self-attention over a given input\nand label candidate (with pre-trained representations). This is a neural network basedmodel."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73af-934e-11ee-8559-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "\u2013 BART (Lewis et al., 2019) adds noise to the text and trains by learning a model to\nreconstruct the original. This is a Tranformer-based neural machine translation model.\n\u2013 BERT-based Bi-encoder (Lu et al., 2020) which uses two separate representations of\nthe document and query based on a BERT-like model. We use the pre-trained largelanguage model as available and do not train it specifically on our data."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b0-934e-11ee-9a43-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "While detailed comparisons in terms of memory usage and training times with these modelsare beyond the scope of this work, other researchers have favourably compared the perfor-mance of Tsetlin Machine based systems to similar models (Yadav et al., 2021b; Bhattarai\net al., 2021). As an example, while the MemNet takes an average of 220 seconds to train on\nthe current dataset, the RTM takes 218 seconds (both averaged over 100 separate runs).\n5.5 Noisetolerance"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b1-934e-11ee-8b9c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "5.5 Noisetolerance\nTo verify our claims of noise tolerance as shown by the TM based architecture, the aboveexperiments were repeated, but with increasing amount of noise artificially introduced intothe training data. The results are shown in Table 9. We observe that with 1%, 2%, 5% and\n10% of noise, the testing accuracy fell only by approximately 1.1 percentage points eachtime when entity generalization was used."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b2-934e-11ee-a1c7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "Along with the obvious strengths of having a transparent explainable system, the noise\ntolerance is one of the more compelling pros of the TM based solution. Typical methodsare more isomorphic to the text provided, along with its imperfections, and not capable ofgeneralizing like the TM. Comparing with accuracies obtained from a few other standardclassification techniques, viz. CNN-LSTM, Naive Bayes, and Random Forests, the drop ison average 3-5%.\n5.6 Hornclauserepresentation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b3-934e-11ee-9ecd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 24, "chroma:document": "5.6 Hornclauserepresentation\nThe example elaborated in the previous section can be formulated as the following Hornclause representation:\n1.Person(Susan).\n2.Person(William) .\n3.Location(Office) .\n4.Location(Garden) .\nTable9 Average accuracy on test with increase in error in training data\nErrorRate 0 %1 %2 %5 %1 0%\nAccuracy 99.48 98.79 98.24 97.02 95.08145 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b4-934e-11ee-9b4c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "5.Location(Pantry) .\n6.CurrentlyAt(Susan, Pantry).\n7.CurrentlyAt(William,Pantry) .\n8.MoveT o(Susan,Garden) \u2190Person(Susan),Location(Garden), not\nCurrentlyAt(Susan, Garden)\n9.MoveTo(William,Office) \u2190Person(William), Location(Off ice),\nnot CurrentlyAt(William, Off ice).\nAfter generalization, we substitute ground rules 8 and 9 with the following rule:\n10.MoveTo(P,L) \u2190Person(P),Location(L),not CurrentlyAt(P,L).\nThe above set of Horn clauses define the immediate consequences operator whose LFP"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b5-934e-11ee-b830-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "represents the Herbrand interpretation of our QA framework.\n6 Conclusion\nMaking interpretable logical decisions in question answering system is an area of active"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b6-934e-11ee-9513-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "research. In this work, we propose a novel relational logic based TM framework to approachQA tasks systematically. Our proposed method takes advantage of noise tolerance showedby TMs to work in uncertain or ambiguous contexts. We reduce the context-question-answertuples to a set of logical arguments, which is used by the TM to determine rules that mimicreal-world actions and consequences.\nThe resulting TM is relational (as opposed to the previously propositional TM) and can"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b7-934e-11ee-bb39-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "take work on logical structures that occur in natural language in order to encode rules rep-resenting actions and effects in the form of Horn clauses. We show initial results using theRelational TM on artificial datasets of closed-domain question answering, and those resultsare extremely promising. The use of first-order representations, as described in this paper,allows KBs to be up to 10 times smaller, while at the same time showing an answeringaccuracy increase of almost 5% to 99.48%."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b8-934e-11ee-8aa4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "Further work on this framework will involve a larger number of relations, with greater"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73b9-934e-11ee-8b40-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "inter-dependencies, and analyzing how well the TM can learn the inherent logical structuregoverning such dependencies. We also intend to introduce recursive Horn clauses to makethe computing power of the Relational TM equivalent to a universal Turing machine. More-over, we wish to experiment with this framework on real-world natural language datasets,rather than on toy ones. A prominent example is exploring large corpus of documents relatedto human rights violation and using them to assess"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ba-934e-11ee-a8a3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "human rights violation and using them to assess risks of social instability. We expect thatthe resultant logic structures will be large and complicated, however, once obtained, can beused to effectively translate to and fro between the machine world and the real world."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73bb-934e-11ee-8ca6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "Data A vailability The toy tasks dataset analysed during the current study are available publicly in the\nBABI repository (http://www.thespermwhale.com/jaseweston/babi/tasks 1-20 v1-2.tar.gz ). The data refer-\nring to parent-child relationships, generated for analysis during the current study are available from the\ncorresponding author on reasonable request.\nDeclarations"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73bc-934e-11ee-90dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 25, "chroma:document": "Declarations\nC o n f l i c to fI n t e r e s t The Authors declare that there is no conflict of interest.146 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73bd-934e-11ee-84bc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "Open\nAccess This article is licensed under a Creative Commons Attribution 4.0 International License, which\npermits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73be-934e-11ee-91d2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence,and indicate if changes were made. The images or other third party material in this article are included in thearticle\u2019s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material isnot included in the article\u2019s Creative Commons licence and your intended use is not permitted by statutory"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73bf-934e-11ee-a2c4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.\nTo view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .\nReferences\nAbeyrathna, K. D., Granmo, O. C., & Goodwin, M. (2021). extending the tsetlin machine with Integer-\nWeighted clauses for increased interpretability. IEEE Access,9 .\nAbeyrathna, K. D., Granmo, O. C., Zhang, X., Jiao, L., & Goodwin, M. (2019). The regression tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c0-934e-11ee-977e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "machine - a novel approach to interpretable Non-Linear regression. Philosophical Transactions of the\nRoyal Society A, 378.\nBerge, G. T., Granmo, O. C., Tveit, T. O., Goodwin, M., Jiao, L., & Matheussen, B.V . (2019). Using the\nTsetlin Machine to learn human-interpretable rules for high-accuracy text categorization with medical\napplications. IEEE Access, 7, 115134\u2013115146.\nBhattarai, B., Jiao, L., & Granmo, O.C. (2021). Measuring the Novelty of Natural Language Text Using the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c1-934e-11ee-af8d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "Conjunctive Clauses of a Tsetlin Machine Text Classifier. In 13Th international conference on agents\nand artificial intelligence (ICAART 2021) . INSTICC.\nBlakely, C. D., & Granmo, O. C. (2020). Closed-Form Expressions for Global and Local Interpretation of\nTsetlin Machines with Applications to Explaining High-Dimensional Data. arXiv: 2007.13885.\nBollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: a collaboratively created graph"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c2-934e-11ee-97d3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international\nconference on Management of data (pp. 1247\u20131250).\nBratko, I., & Muggleton, S. (1995). Applications of inductive logic programming. Communications of the\nACM ,38(11), 65\u201370.\nCropper, A., Duman \u02c7ci\u00b4c, S., & Muggleton, S.H. (2020). Turning 30: New ideas in inductive logic program-\nming. arXiv:2002.11002."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c3-934e-11ee-8c84-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "ming. arXiv:2002.11002.\nCyras, K., Badrinath, R., Mohalik, S. K., Mujumdar, A., Nikou, A., Previti, A., Sundararajan, V ., & Feljan,\nA. V . (2020). Machine reasoning explainability. arXiv: 2009.00418.\nDe Raedt, L., & Kersting, K. (2008). Probabilistic inductive logic programming. In Probabilistic inductive\nlogic programming (pp. 1\u201327). Springer.\nDong, J. S., Sun, J., & Wang, H. (2003). Checking and reasoning about semantic web through alloy. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c4-934e-11ee-9109-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "International symposium of formal methods europe (pp. 796\u2013813). Springer.\nGallaire, H., Minker, J., & Nicolas, J. M. (1989). Logic and databases: a deductive approach. In Readings in\nartificial intelligence and databases (pp. 231\u2013247). Elsevier.\nGranmo, O. C. (2018). The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern\nRecognition with Propositional Logic. arXiv: 1804.01508."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c5-934e-11ee-bc3a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "Granmo, O. C., Glimsdal, S., Jiao, L., Goodwin, M., Omlin, C. W., & Berge, G.T. (2019). The Convolutional\nTsetlin Machine. arXiv:1905.09688.\nGreen, C. (1969). Theorem proving by resolution as a basis for question-answering systems. Machine\nIntelligence, 4, 183\u2013205.\nJarke, M., Gallersd \u00a8orfer, R., Jeusfeld, M. A., Staudt, M., & Eherer, S. (1995). Conceptbase\u2014a deductive\nobject base for meta data management. Journal of Intelligent Information Systems ,4(2), 167\u2013192."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c6-934e-11ee-841e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "Jung, H., & Kim, W. (2020). Automated conversion from natural language query to sparql query. Journal of\nIntelligent Information Systems , 1\u201320.\nKowalski, R. (1979). Algorithm= logic+ control. Communications of the ACM, 22(7), 424\u2013436.\nKowalski, R. (2014). Logic programming. In J. H. Siekmann (Ed.) Computational logic, handbook of\nthe history of logic , (V ol. 9 pp. 523\u2013569). North-holland. https://doi.org/10.1016/B978-0-444-51624-\n4.50012-5 ."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c7-934e-11ee-ab4d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 26, "chroma:document": "4.50012-5 .\nKramer, S., Lavra \u02c7c, N., & Flach, P. (2001). Propositionalization approaches to relational data mining.\nRelational data mining, 262\u2013291.\nLei, J., Rahman, T., Shafik, R., Wheeldon, A., Yakovlev, A., Granmo, O. C., Kawsar, F., & Mathur, A. (2021).\nLow-Power Audio Keyword Spotting using Tsetlin Machines. arXiv: 2101.11336.147 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c8-934e-11ee-adf1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Lei, J., Wheeldon, A., Shafik, R., Yakovlev, A., & Granmo, O.C. (2020). From arithmetic to logic based AI:\na comparative analysis of neural networks and tsetlin machine. In 27Th IEEE international conference\non electronics circuits and systems (ICECS2020). IEEE.\nLewis, M., Liu, Y ., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V ., & Zettlemoyer, L.\n(2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73c9-934e-11ee-95d4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "and comprehension. arXiv:1910.13461.\nLloyd, J. (1984). F oundations of logic programming . New York: Springer.\nLu, W., Jiao, J., & Zhang, R. (2020). Twinbert: Distilling knowledge to twin-structured compressed bert mod-\nels for large-scale retrieval. In Proceedings of the 29th ACM International Conference on Information &\nKnowledge Management (pp. 2645\u20132652).\nLudwig, S. A. (2010). Comparison of a deductive database with a semantic web reasoning engine.\nKnowledge-Based Systems ,23(6), 634\u2013642."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ca-934e-11ee-bd48-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Knowledge-Based Systems ,23(6), 634\u2013642.\nMiller, A. H., Feng, W., Fisch, A., Lu, J., Batra, D., Bordes, A., Parikh, D., & Weston, J. (2017). Parlai: A\ndialog research software platform. arXiv:1705.06476.\nMitchell, T., Cohen, W., Hruschka, E., Talukdar, P., Yang, B., Betteridge, J., Carlson, A., Dalvi, B., Gardner,\nM., Kisiel, B., & et al. (2018). Never-ending learning. Communications of the ACM, 61(5), 103\u2013115."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73cb-934e-11ee-9752-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Nickles, M., & Mileo, A. (2014). Probabilistic inductive logic programming based on answer set program-\nming. arXiv:1405.0720.\nPark, B., Greene, K., & Colaresi, M. (2020). How to teach machines to read human rights reports and identify\njudgments at scale. Journal of Human Rights ,19(1), 99\u2013116.\nPhoulady, A., Granmo, O. C., Gorji, S. R., & Phoulady, H.A. (2019). The Weighted Tsetlin Machine:\nCompressed Representations with Weighted Clauses. arXiv:1911.12607."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73cc-934e-11ee-b7f3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Prager, J. M. (2006). Open-domain question-answering. F ound Trends Inf Retr ,1(2), 91\u2013231.\nPundge, A. M., Khillare, S., & Mahender, C. N. (2016). Question answering system, approaches and\ntechniques: a review. International Journal of Computer Applications ,141(3), 0975\u20138887.\nRajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine\ncomprehension of text. arXiv: 1606.05250."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73cd-934e-11ee-bb30-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "comprehension of text. arXiv: 1606.05250.\nSaha, R., Granmo, O. C., & Goodwin, M. (2020). Mining Interpretable Rules for Sentiment and Seman-\ntic Relation Analysis using Tsetlin Machines. In Lecture Notes in Computer Science: Proceedings of\nthe 40th International Conference on Innovative T echniques and Applications of Artificial Intelligence(SGAI-2020) . Springer.\nShafik, R., Wheeldon, A., & Yakovlev, A. (2020). Explainability and Dependability Analysis of Learning"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73ce-934e-11ee-a6c7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Automata based AI Hardware. In IEEE 26Th international symposium on on-line testing and robust\nsystem design (IOLTS). IEEE.\nSoares, M. A. C., & Parreiras, F. S. (2020). A literature review on question answering techniques, paradigms\nand systems. Journal of King Saud University-Computer and Information Sciences ,32(6), 635\u2013646.\nSrinivasan, V ., Santhanam, S., & Shaikh, S. (2021). Using reinforcement learning with external rewards for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73cf-934e-11ee-bd2d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "open-domain natural language generation. Journal of Intelligent Information Systems ,56(1), 189\u2013206.\nTsetlin, M. L. (1961). On behaviour of finite automata in random medium. Avtom I T elemekhanika ,22(10),\n1345\u20131354.\nTurhan, A. Y . (2011). Description logic reasoning for semantic web ontologies. In Proceedings of the\nInternational Conference on W eb Intelligence, Mining and Semantics (pp. 1\u20135)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73d0-934e-11ee-80b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merri \u00a8enboer, B., Joulin, A., & Mikolov, T. (2015).\nTowards ai-complete question answering: A set of prerequisite toy tasks. arXiv: 1502.05698.\nWeston, J., Chopra, S., & Bordes, A. (2014). Memory networks. arXiv:1410.3916 .\nWheeldon, \nA., Shafik, R., Yakovlev, A., Edwards, J., Haddadi, I., & Granmo, O.C. (2020). Tsetlin machine:\na new paradigm for pervasive AI. In SCONA W orkshop at design, automation and test in europe (DATE\n2020)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73d1-934e-11ee-9b87-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "2020).\nWolf, T., Sanh, V ., Chaumond, J., & Delangue, C. (2019). Transfertransfo: A transfer learning approach for\nneural network based conversational agents. arXiv:1901.08149.\nYadav, R. K., Jiao, L., Granmo, O. C., & Goodwin, M. (2021). Human-Level interpretable learning for\nAspect-Based sentiment analysis. In The thirty-fifth AAAI conference on artificial intelligence (AAAI-\n21). AAAI.\nYadav, R. K., Jiao, L., Granmo, O. C., & Goodwin, M. (2021). Interpretable classification of word sense dis-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d73d2-934e-11ee-9587-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\s10844-021-00682-5.pdf", "page": 27, "chroma:document": "ambiguation using tsetlin machine. In 13Th international conference on agents and artificial intelligence\n(ICAART 2021). INSTICC.\nPublisher\u2019snote Springer Nature remains neutral with regard to jurisdictional claims in published maps\nand institutional affiliations.148 Journal of Intelligent Information Systems (2022) 59:121\u2013148"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb3-934e-11ee-a89f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "Coalesced Multi-Output Tsetlin Machines with Clause Sharing\u2217\nSondre Glimsdal\u2020and Ole-Christo\ufb00er Granmo\u2021\nAugust 18, 2021\nAbstract\nUsing \ufb01nite-state machines to learn patterns, Tsetlin machines (TMs) have obtained\ncompetitive accuracy and learning speed across several benchmarks, with frugal memory-\nand energy footprint. A TM represents patterns as conjunctive clauses in propositional\nlogic (AND-rules), each clause voting for or against a particular output. While e\ufb03cient for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb4-934e-11ee-b14e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "single-output problems, one needs a separate TM per output for multi-output problems.\nEmploying multiple TMs hinders pattern reuse because each TM then operates in a silo.\nIn this paper, we introduce clause sharing, merging multiple TMs into a single one. Each\nclause is related to each output by using a weight. A positive weight makes the clause vote\nfor output 1, while a negative weight makes the clause vote for output 0. The clauses thus"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb5-934e-11ee-8e71-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "coalesce to produce multiple outputs. The resulting coalesced Tsetlin machine (CoTM)\nsimultaneously learns both the weights and the composition of each clause by employing\ninteracting Stochastic Searching on the Line (SSL) and Tsetlin automata (TAs) teams.\nOur empirical results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST show that CoTM\nobtains signi\ufb01cantly higher accuracy than TM on 50- to 1K-clause con\ufb01gurations, indicating"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb6-934e-11ee-8ab2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "an ability to repurpose clauses. E.g., accuracy goes from 71 .99% to 89.66% on Fashion-\nMNIST when employing 50 clauses per class (22 Kb memory). While TM and CoTM\naccuracy is similar when using more than 1K clauses per class, CoTM reaches peak accuracy\n3\u02c6faster on MNIST with 8K clauses. We further investigate robustness towards imbalanced\ntraining data. Our evaluations on imbalanced versions of IMDb- and CIFAR10 data show"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb7-934e-11ee-be32-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "that CoTM is robust towards high degrees of class imbalance. Being able to share clauses,\nwe believe CoTM will enable new TM application domains that involve multiple outputs,\nsuch as learning language models and auto-encoding.\n1 Introduction\nTMs [19] have recently demonstrated competitive accuracy-, memory footprint-, energy-, and\nlearning speed on several benchmarks, spanning tabular data [2, 40], images [20, 37], regres-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb8-934e-11ee-9100-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "sion [4], natural language [6, 45, 46, 9, 44], and speech [27]. By not relying on minimizing\noutput error, TMs are less prone to over\ufb01tting. Instead, they use frequent pattern mining\nand resource allocation principles to extract common patterns in the data. Unlike the inter-\ntwined nature of pattern representation in neural networks, a TM decomposes problems into\nself-contained patterns. These are expressed using conjunctive clauses in propositional logic."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bb9-934e-11ee-9cb8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "That is, each pattern is an AND-rule, such as: ifinput x satis\ufb01es conditionAand not\nconditionBthen outputy\u201c1. The clause outputs, in turn, are combined into a classi\ufb01cation\ndecision through a majority vote, akin to logistic regression, however, with binary weights and\na unit step output function. Being based on the human-interpretable disjunctive normal form\n[39], like Karnaugh maps [23], a TM can map an exponential number of input feature value\ncombinations to an appropriate output [19]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bba-934e-11ee-8bd3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 0, "chroma:document": "combinations to an appropriate output [19].\n\u2217Source code for this paper can be found at https://github.com/cair/PyCoalescedTsetlinMachineCUDA.\n\u2020Author\u2019s status: Lead Researcher at Forsta. E-mail: sondre.glimsdal@forsta.com\n\u2021Author\u2019s status: Professor . The author can be contacted at: Centre for Arti\ufb01cial Intelligence Research\n(https://cair.uia.no), University of Agder, Grimstad, Norway. E-mail: ole.granmo@uia.no\n1arXiv:2108.07594v1  [cs.AI]  17 Aug 2021"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bbb-934e-11ee-8e52-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "\u2026\u2026\u2026+-+-\n\u2211\u2335\u2335\u2335\u23350 1 0 \u2026 1 0 1\u2211\nOutput\nInputThreshold FunctionSummationConjunctiveClausesFigure 1: The Tsetlin Machine inference structure, introducing clause polarity, a summation\noperator collecting \u201cvotes\u201d, and a threshold function arbitrating the \ufb01nal output [19].\nRecent progress on TMs. Recent research reports several distinct TM properties. The\nTM can be used in convolution [20], providing competitive performance on MNIST, Fashion-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bbc-934e-11ee-aedd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "MNIST, and Kuzushiji-MNIST, in comparison with CNNs, K-Nearest Neighbor, Support Vec-\ntor Machines, Random Forests [11], XGBoost [13], BinaryConnect [15], Logistic Circuits [28]\nand ResNet [21]. The TM has also achieved promising results in text classi\ufb01cation [6, 44], word\nsense disambiguation [46], novelty detection [9, 8], fake news detection [7], semantic relation\nanalysis [35], and aspect-based sentiment analysis [45] using the conjunctive clauses to capture"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bbd-934e-11ee-9c4b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "textual patterns. Recently, regression TMs [4] compared favorably with Regression Trees [12],\nRandom Forest Regression [11], and Support Vector Regression [16].\nThe above TM approaches have further been enhanced by various techniques. By introduc-\ning real-valued clause weights, it turns out that the number of clauses can be reduced by up to\n50\u02c6without loss of accuracy [34]. Also, the logical inference structure of TMs makes it possible"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bbe-934e-11ee-b8eb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "to index the clauses on the features that falsify them, increasing inference- and learning speed\nby up to an order of magnitude [17]. Multi-granular clauses simplify the hyper-parameter search\nby eliminating the pattern speci\ufb01city parameter [18]. In [2], SSL automata [33] learn integer\nclause weights, performing on-par or better than Random Forests [11], XGBoost [13], Neural\nAdditive Models [5], StructureBoost [29], and Explainable Boosting Machines [32]. Closed"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bbf-934e-11ee-bd08-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "form formulas for both local and global TM interpretation, akin to SHAP [30], were proposed\nby Blakely et al. [10].\nComputationally, TMs are natively parallel [1] and hardware near [41, 40, 42], allowing\nenergy usage to be traded o\ufb00 against accuracy by making inference deterministic [3]. Ad-\nditionally, Sha\ufb01k et al. show that TMs can be fault-tolerant, completely masking stuck-at\nfaults [36].\nRecent theoretical work proves convergence to the correct operator for \u201cidentity\u201d and \u201cnot\u201d."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc0-934e-11ee-bd01-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "It is further shown that arbitrarily rare patterns can be recognized using a quasi-stationary\nMarkov chain-based analysis. The work \ufb01nally proves that when two patterns are incompatible,\nthe most accurate pattern is selected [47]. Convergence for the \u201cXOR\u201d operator has also\nrecently been proven by Jiao et al. [22].\nPaper Contributions. Figure 1 depicts the original multi-output TM architecture from [19]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc1-934e-11ee-935e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "A TM represents patterns as a collection of conjunctive clauses in propositional logic. Each\nclause vote for or against a particular output, arbitrated by a majority vote. As further depicted\nin the \ufb01gure, there is one separate TM per output, each maintaining its own set of clauses.\nThis hinders reuse of patterns among the outputs, because each TM operates in a silo.\nIn this paper, we introduce a new TM architecture that employs a shared pool of clauses,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc2-934e-11ee-8841-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 1, "chroma:document": "illustrated in Figure 2. Each clause in the pool is related to each output by using a weight.\nA positive weight makes the clause vote for output 1, while a negative weight makes it vote\n2"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc3-934e-11ee-9582-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 2, "chroma:document": "\u2026\u2026+-\n\u2211\u2335\u23350 1 0 \u2026 1 0 1\u2211\nOutput\nInputThreshold FunctionSummationConjunctiveClauses+-Figure 2: The Coalesced Tsetlin Machine (CoTM) inference structure, introducing clause shar-\ning.\nfor output 0. The magnitude of the weight decides the impact of the vote. The clauses thus\ncoalesce to produce multiple outputs. The resulting CoTM simultaneously learns both the\nweights and the composition of each clause. The weights are learnt using one SSL automaton"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc4-934e-11ee-ba57-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 2, "chroma:document": "per clause-output pair. Simultaneously, a team of TAs con\ufb01gures each clause to maximise\noutput prediction accuracy.\nPaper Organization. In Section 2, we introduce the basics of TM inference and learning,\nproviding the basis for CoTM. Then, in Section 3, we describe the CoTM in detail, including\nthe architecture for coalescing clauses and learning weights. Our empirical results are presented\nin Section 4, where we evaluate the CoTM on 2D Noisy XOR, MNIST, Fashion-MNIST, K-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc5-934e-11ee-bc46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 2, "chroma:document": "MNIST, CIFAR10, and IMDb comparing with various baselines. We conclude the paper in\nSection 5 and discuss ideas for further work.\n2 Tsetlin Machine Basics\nIn this section, we provide an introduction to the principles of TM inference and learning,\nwhich we formalize in Section 3 when presenting the CoTM.\n2.1 Inference\nFigure 1 depicts a multi-output TM architecture. As seen, the architecture uses a simple\npattern matching scheme to decide upon the output:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc6-934e-11ee-bd50-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 2, "chroma:document": "\u2022One TM is associated with each output, assembling a set of patterns. Each pattern, in\nturn, is an AND-rule, called a conjunctive clause .\n\u2022The AND-operator ( ^in the \ufb01gure) binds together propositional inputs. These are either\nFalse or True (0 or 1 in the \ufb01gure).\n\u2022For each TM, half of the clauses can vote for output value True. We refer to these as\npositive clauses. The other half can vote for output value False. We call these negative\nclauses."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc7-934e-11ee-9049-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 2, "chroma:document": "clauses.\n\u2022To decide upon a particular output, we count the positive clauses that match the input.\nMatching negative clauses, on the other hand, decrease the match count.\n\u2022A majority vote decides the \ufb01nal output, indicated by the threshold functions in the\n\ufb01gure. If there are more matching negative clauses than positive clauses, the output is\nFalse. Otherwise the output is True.\n3"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc8-934e-11ee-a36d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "As an example, consider the task of determining the sentiment of a collection of movie\nreviews. To this end, we employ a Set of Words (SoW) representation. That is, each word\nin the vocabulary is a propositional input, representing the presence or absence of the word\nin a particular review. Consider, for instance, the review \u201cThe movie was good, and I had\npopcorn\u201d . Here, the words \u201cthe\u201d, \u201cmovie\u201d, \u201cwas\u201d, \u201cgood\u201d, \u201cand\u201d, \u201cI\u201d, \u201chad\u201d, \u201cpopcorn\u201d are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bc9-934e-11ee-a17c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "present. These thus take the input value True. Any other words are absent, taking the input\nvalue False.\nBased on the SoW, the task is to decide upon the nature of each review, whether it is\nPositive orNegative . The task thus has two outputs and we consider each output to either be\nFalse or True.1\nClause TM: Positive TM: Negative\n#1 \u201dgood\u201d AND \u201dmovie\u201d \u00d1+1 \u201dbad\u201d\u00d1+1\n#2 NOT \u201dbad\u201d\u00d1+1 NOT \u201dgood\u201d \u00d1+1\n#3 \u201dhorrible\u201d AND \u201dpopcorn\u201d \u00d1-1 \u201dgood\u201d\u00d1-1\n#4 \u201dbad\u201d\u00d1-1 \u201dhigh value\u201d\u00d1-1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bca-934e-11ee-978c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "#4 \u201dbad\u201d\u00d1-1 \u201dhigh value\u201d\u00d1-1\nTable 1: Eight example clauses for classifying IMDb movie reviews.\nTable 1 contains two example TMs for the above task, one for output Positive and one for\noutput Negative . The \ufb01rst column speci\ufb01es the clause number; the second column contains\nthe clauses for the TM assigned to output Positive ; and the third column covers the Negative\noutput TM. Clause #1 and Clause #2 can vote for output True (\u2019 `1\u2019-votes), while Clause"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bcb-934e-11ee-963f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "#3 and Clause #4 can vote for output False (\u2019 \u00b41\u2019-votes). For instance, Clause #1 for TM\nPositive says that if the input review contains the words \u201cgood\u201d and \u201cmovie\u201d, the clause votes\nfor output True. Notice that Clause #2 consists of a negated input. That is, if the input review\ndoes not contain the word \u201cbad\u201d, this also counts as a vote towards Positive output True. Such\nnegation is useful because one can strengthen the belief in Positive from the absence of negative\nsentiment words."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bcc-934e-11ee-8c25-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "sentiment words.\nInference proceeds as follows. Again, consider the example review: \u201cThe movie was good,\nand I had popcorn\u201d . To classify this review, we \ufb01rst identify clauses that are matching the\ntext. For TM Positive , Clause #1 matches because both \u201dgood\u201d and \u201dmovie\u201d are in the text.\nClause #2 also matches the text, since \u201dbad\u201d is absent. Therefore, the score for TM Positive\nbecomes Clause #1 + Clause #2 \u201c1`1\u201c2. For TM Negative only Clause #3 is active,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bcd-934e-11ee-88cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "providing a score of \u00b41. From a propositional perspective, if a TM obtains a non-negative\nscore it outputs True. Otherwise, it outputs False. Accordingly, the output of TM Positive\nbecomes True, while the output of TM Negative becomes False.\nMaximally\nMemorized\nMaximally\nForgottenMemorized\nForgottenN+12N\n1N\nFigure 3: Sticky memory with depth Nfor the clause: \u201cgood\u201d AND \u201cmovie\u201d.\n1Note that for this particular task, a movie review cannot be both Positive andNegative at the same time."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bce-934e-11ee-88bb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 3, "chroma:document": "Accordingly, we could have modelled the problem as a multi-class problem instead of a multi-output problem [19].\n4"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bcf-934e-11ee-a388-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 4, "chroma:document": "2.2 Learning\nSticky Memory. Memorizing input is the basis of TM learning, such as remembering the\nessence of: \u201cThe movie was good, and I had popcorn\u201d . To this end, each clause has its own\nmemory where it stores its AND-pattern. In the IMDb case, the memory contains the truth\nvalue of words from the SoW. TM Positive Clause #1 stores True for \u201cmovie\u201d and \u201cgood\u201d,\nfor instance. However, instead of simply storing the truth values as is, the memory simulates"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bd0-934e-11ee-816c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 4, "chroma:document": "memorization and forgetting. I.e., every time a clause observes a particular truth value, it\nremembers it longer. Oppositely, without observations, it eventually forgets the value. Ac-\ncordingly, each memory entry does not contain a truth value but an integer in the range from 1\nto 2N. The resulting memory is illustrated in Figure 3. Note that the user sets Nto a speci\ufb01c\nvalue to control maximal memorization and forgetting. From N`1 to 2N, the word\u2019s truth"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bd1-934e-11ee-ae35-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 4, "chroma:document": "value is part of the clause\u2019s AND-pattern. Integer N`1 means in memory but easily forgotten.\nInteger 2Nmeans maximally memorized. A truth value is not part of the pattern in the span\n1 toN. Integer 1 means maximally forgotten, while integer Nmeans almost memorized. The\nclause in the \ufb01gure will, for instance, remember \u201cgood\u201d for a longer time than \u201cmovie\u201d because\n\u201cgood\u201d is more deeply stored. It is close to memorizing \u201chad\u201d, which is close to N`1.2\nMaximally\nMemorized\nMaximally"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152d9bd2-934e-11ee-a344-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 4, "chroma:document": "Maximally\nMemorized\nMaximally\nForgottenMemorized\nForgottenN+12N\n1N\nFigure 4: The memorize (x,s) operator triggered by input \u201cThe movie was good, and I had\npopcorn.\u201d\nMaximally\nMemorized\nMaximally\nForgottenMemorized\nForgottenN+12N\n1N\nFigure 5: The forget (s) operator triggered by input \u201cWas good, and I had popcorn.\u201d\nMemory Updating. Now, let xbe one particular input to the TM. We then have three\noperators for updating the memory of each clause."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2b5-934e-11ee-b1d2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 4, "chroma:document": "operators for updating the memory of each clause.\n1.memorize(x ,s)strengthens the memory of x. However, we do not perform a plain\ncopy. Instead, each truth value in xincreases the corresponding integer in the memory.\n2Note that each memory entry can be seen as a Tsetlin Automaton [38], hence the name Tsetlin Machine.\n5"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2b6-934e-11ee-8657-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "Maximally\nMemorized\nMaximally\nForgottenMemorized\nForgottenN+12N\n1NFigure 6: The invalidate (x) operator applied to input \u201cThe movie was bad, and I had pop-\ncorn.\u201d\nAs exempli\ufb01ed in Figure 4, \u201cgood\u201d in the input increases the integer of \u201cgood\u201d. Further,\n\u201cbad\u201d is missing in the input, which increases the integer of not\u201cbad\u201d. Simultaneously,\nthe truth values that con\ufb02ict with the input have their integers decreased. However,\nthis decrease is randomized, happening with probability1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2b7-934e-11ee-acb9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "s. For instance, not\u201cgood\u201d is\ncon\ufb02icting with the input. Hence, its integer is decreased randomly with probability1\ns.\nThe intuition is that memorization must be stronger than forgetting so that patterns can\nbe retained over time. The parameter sis set by the user so that the user can control\nhow quickly truth values are forgotten. In e\ufb00ect, increasing smakes the patterns \ufb01ner,\nwhile decreasing smakes them coarser."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2b8-934e-11ee-9cad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "while decreasing smakes them coarser.\n2.forget(s)performs pure forgetting, as described above. Figure 5 provides an example.\nAs seen, all the truth values have their integer randomly decreased, with probability1\ns.\nPerforming this operator several times eventually makes all the truth values maximally\nforgotten.\n3.invalidate(x) changes the clause so that it eventually rejects the input x. This operator\nincreases the integer of all False truth values, illustrated in Figure 6 for input \u201cThe movie"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2b9-934e-11ee-a0b9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "was bad, and I had popcorn.\u201d The word \u201cbad\u201d, for instance, appears in the input. Hence,\nnot\u201cbad\u201d gets its integer increased.\nLearning Single Clauses. Four di\ufb00erent prediction outcomes guide the learning of each\nTM clause:\n\u2022True Positive. The clause correctly votes for its assigned output. For instance, we have\na True Positive outcome when Clause #1 for TM Positive matches a review of positive\nsentiment. In this case, the clause performs memorize( x,s). This feedback makes the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ba-934e-11ee-a638-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "clause remember and re\ufb01ne the pattern it recognizes in x.\n\u2022False Negative. The clause fails to vote for its assigned output. If Clause #1 for TM\nPositive does not match a review of positive sentiment, that would be a False Nega-\ntive outcome. In this case, the clause performs forget( s). This reinforcement coarsens\ninfrequent patterns, making them frequent.\n\u2022False Positive. The clause incorrectly votes for its assigned output. If Clause #1 for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2bb-934e-11ee-acad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 5, "chroma:document": "TMPositive matches a review of negative sentiment, we have a False Positive. The clause\nthen performs invalidate( x). This feedback makes the clause more discriminative.\n\u2022True Negative . The clause correctly refrains from voting for its assigned output. That\nwould be the case if Clause #1 for TM Positive does not match a review of negative\nsentiment. This outcome does not trigger any memory updates.\n6"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2bc-934e-11ee-9092-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "Learning Multiple Clauses. The clauses must learn to coordinate, taking di\ufb00erent roles\nin providing correct output. The TM achieves this by introducing a voting margin that we\ncallt. We use this parameter to specify how many votes we want the winning output to win\nby. That is, we want the winning output to win by tvotes, but not more. The voting margin\nmakes sure that a su\ufb03cient number of clauses support each output. Simultaneously, we do not"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2bd-934e-11ee-8191-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "want the winning output to win by much more than the margin either. That would mean that\nwe use more clauses than necessary. So, the voting margin also ensures prudent usage of the\navailable clauses. Learning of multiple clauses is coordinated as follows:\n1. Obtain next training example. The training example consists of the input truth values x\nas well the correct output truth value y.\n2. Evaluate each clause on input x."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2be-934e-11ee-b600-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "2. Evaluate each clause on input x.\n3. Calculate a voting sum for the clauses that evaluate to True (clauses matching x):\n(a) Add up the votes in favour of y, i.e., the correct truth value for the output.\n(b) Subtract the votes in favour of noty, i.e., the incorrect truth value for the output.\n(c) We call the summation outcome v.\n(d) Setvtotif larger than tand to\u00b4tif smaller than \u00b4t.\n4. Go through each clause and update it if Randpq\u010ft\u00b4v\n2t, where Randpqdraws a random"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2bf-934e-11ee-8042-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "2t, where Randpqdraws a random\nvalue uniformly from the interval r0,1s:\n(a) Perform memorize (x,s) if the clause matches xand belongs to output truth value y.\n(True Positive)\n(b) Perform forget (s) if the clause does not match xand belongs to output truth\nvaluey. (False Negative)\n(c) Perform invalidate (x) if the clause matches xand belongs to output truth value noty.\n(False Positive)\nObserve that if we are far from achieving the voting margin for a particular example, we update"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c0-934e-11ee-81e4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "clauses more aggressively. E.g., if the voting sum is \u00b4tor smaller, we update all the clauses. If\nthe voting sum is zero, we randomly update each clause with probability 0 .5. If we are close to\nt, we calm down the updating. When reaching or surpassing t, we update none of the clauses.\nIn this way, clauses individually and gradually assign themselves to training examples that\nhave not yet reached the voting margin. As a result, inference accuracy increases over time as"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c1-934e-11ee-8fb1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "the clauses specialize on di\ufb00erent subsets of the training examples.\n3 Coalesced Tsetlin Machine\nWe here introduce the CoTM, formulated in terms of propositional- and linear algebra. While\nwe cover all the key structures and formulas here, the full details are found in Appendix A.\n3.1 Structure\nA CoTM can be de\ufb01ned as a quadruple:\ntX,Y,C,Wu. (1)\nWe start with introducing each element of the quadruple, with reference to Figure 7."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c2-934e-11ee-a662-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 6, "chroma:document": "\u2022Theinput space of the CoTM is denoted X. The input space consists of vectors xof\nopropositional inputs: x\u201c rx1,...,x os PX,X\u201c t0,1uo. In the \ufb01gure, we have two\npropositional inputs x\u201crx1,x2sPX,X\u201ct0,1u2.\n7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c3-934e-11ee-856d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 7, "chroma:document": "Summation\nClausesClause W eightingMargin\nt=5 -3 +4Thresholding  \n Output\nInput8 2 1 7\n2 7 8 1\n7 1 7 2\n1 7 2 8Clause 1\nClause 2\nClause 3\nClause 4Clause Memory\nN=4Action Mapping+4\nClause 1\nClause 2\nClause 3\nClause 44 -3 4\n3 -5 5\n-4 5 4\n-3 -4 -4Figure 7: The Coalesced Tsetlin Machine structure with inference example.\n\u2022Theoutput space is denoted Y. The output space contains vectors yofmpropositional\noutputs: y\u201c ry1,y2,...,yms PY,Y\u201c t0,1um. In the \ufb01gure, we have three outputs\ny\u201cry1,y2,y3sPY,Y\u201ct0,1u3."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c4-934e-11ee-b360-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 7, "chroma:document": "y\u201cry1,y2,y3sPY,Y\u201ct0,1u3.\n\u2022Thememory space C\u201c t1,2,..., 2Nun\u02c62ois the space of memory matrices C. A\nmemory matrix Cformalizes the sticky memory from Section 2. Each of the nrows\nrepresents the pattern memory of a single clause. The columns, in turn, represent the o\ninputs in xand their negations. Together, the inputs and their negations are referred to\nas literals. In the \ufb01gure, we have four literals: x1,\u2423x1,x2, and\u2423x2. With four clauses,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c5-934e-11ee-95ad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 7, "chroma:document": "four literals, and memory depth N\u201c4, we get the memory space C\u201ct1,2,..., 8u4\u02c64.\n\u2022Theweight space W\u201c t...,\u00b42,\u00b41,1,2,...um\u02c6nconsists of weight matrices W. A\nweight matrix Wrelates each clause in Cto an output in y. Consider the output and\nclause of a speci\ufb01c entry in the weight matrix. A positive weight assigns the clause to\noutput value 1. A negative weight assigns it to output value 0. The magnitude of the\nweight decides the impact of the assignment. With the three outputs and four clauses"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c6-934e-11ee-af3e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 7, "chroma:document": "in the \ufb01gure, we get the weight space WPW\u201c t...,\u00b42,\u00b41,1,2,...u3\u02c64. As seen,\nClause #1 is assigned output values y1\u201c1,y2\u201c0, andy3\u201c1.\nObserve how the pattern matrix Ccombined with the weight matrix Wcon\ufb01gure a complete\nCoTM.\n8"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c7-934e-11ee-ad1e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "3.2 Output Prediction\nBased on the above quadruple, we can predict the output yfrom input xusing propositional-\nand linear algebra as shown in Eqn. 2:\n\u02c6y\u201cUpW\u00a8AndpImplypGpCq,xqqqT. (2)\nAbove, \u02c6yis the multi-output prediction. As speci\ufb01ed, it is calculated from the input xusing\nthe following operators:\n\u2022Gp\u00a8qmaps the memory matrix Celement-wise to Exclude and Include actions, respec-\ntively 0 and 1. Each entry is mapped using the function:\ngpcq\u201c#\n0,if1\u010fc\u010fN\n1,ifN`1\u010fc\u010f2N.(3)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c8-934e-11ee-bd01-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "gpcq\u201c#\n0,if1\u010fc\u010fN\n1,ifN`1\u010fc\u010f2N.(3)\n\u2022Imply is an element-wise logical imply operator (\u00f1), which implements the Exclude and\nInclude actions. It takes a matrix of Exclude/Include actions raj,ksas input, with j\nreferring to a clause and kreferring to a literal. If aj,kis 0 (Exclude), then aj,k\u00f1xkis\nalways 1. Accordingly, the xkvalue does not impact the result. Conversely, the value of\naj,k\u00f1xkis decided solely by the xkvalue ifajkis 1 (Include)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2c9-934e-11ee-aa9e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "\u2022And is a row-wise AND operator. The operator takes a matrix of truth values as input.\nFor each row, it ANDs together all the truth values of that row.\n\u2022Uis an element-wise unit step thresholding operator. Utakes a vector as input and\napplies the unit step function v\u011b0 on each vector entry.\nThese operators are fully speci\ufb01ed in Appendix A, while Figure 7 provides a prediction example.\nIn the \ufb01gure, we trace the input x\u201cr0,1sthrough each operator. The clause memory in"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ca-934e-11ee-97f0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "the \ufb01gure is of depth N\u201c4. So, for example, Clause #1 includes x1and\u2423x2because their\nrespective memory matrix entries are 8 and 7 (larger than or equal to 5). It excludes \u2423x1and\nx2, having memory entries 2 and 1 (equal to or smaller than 4). Notice how the memory and\nweight matrices implement XOR for y1, AND fory2, and OR for y3. Hence, input x\u201cr0,1s\nprovides output y\u201cr1,0,1s.\n3.3 Updating of Memory Matrix"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2cb-934e-11ee-996d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "3.3 Updating of Memory Matrix\nThe memory matrix is updated based on a training example px,yqusing three kinds of feedback\nmatrices:FIa\ni,FIb\ni, andFII\ni. Further, which clauses are eligible for feedback are compiled in\nthe matrices RIandRII. The clauses are randomly selected according to the voting margin\nof each output values yiinyas described in Section 2. In brief, the memory matrix CT`1for\ntime stepT`1 is calculated from the memory matrix CTof time step T:\nC\u02da\nT`1\u201cCT`m\u00ff\ni\u201c1\u00b4\nQII\ni\u02ddFII\ni`QI"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2cc-934e-11ee-906c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "C\u02da\nT`1\u201cCT`m\u00ff\ni\u201c1\u00b4\nQII\ni\u02ddFII\ni`QI\ni\u02ddFIa\ni\u00b4QI\ni\u02ddFIb\ni\u00af\n. (4)\nAs seen, each output yiof output vector yis considered one at a time. Both FIa\niandFII\ni\nincrements memory entries, while FIb\nidecrements them. The matrix QI\nimaps thei-rows of\nmatrixRIfrom the clause level to the literal level. Similarly, QII\nimaps thei-rows of matrix\nRII. These single out which clauses to update using the Hadamard product \u02dd. The \ufb01nal step\nis to clip the entries in Cto make sure that they stay within 1 and 2 N:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2cd-934e-11ee-be0d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 8, "chroma:document": "CT`1\u201cclip`\nC\u02da\nT`1,1,2N\u02d8\n. (5)\nThe feedback matrixes can be summarized as follows:\n9"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ce-934e-11ee-9dcc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "\u2022Type Ia Feedback. This feedback matrix is denoted FIa\niand concerns output yiof\noutput vector y. It operates on clauses that both match the input xand that are assigned\nto output value yi(by a negative clause-output weight for yi\u201c0 and a positive clause-\noutput weight for yi\u201c1).FIa\nionly a\ufb00ects literals that are True and tunes the clauses\nto represent the current input xmore \ufb01nely, reinforcing Include actions.\n\u2022Type Ib Feedback. This feedback matrix is denoted FIb\niand again concerns output yi."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2cf-934e-11ee-9293-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "iand again concerns output yi.\nIt operates on all the clauses that are assigned to output value yiby the clause-output\nweights. For those clauses that does not match the input x, it operates on all of the\nliterals. For clauses that match the input, it only a\ufb00ects False literals. In e\ufb00ect, the\nmatrix coarsens the clauses by making them forget literals, reinforcing Exclude actions.\n\u2022Type II Feedback. This feedback matrix is denoted FII\niand concerns output yi. It"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d0-934e-11ee-a262-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "iand concerns output yi. It\noperates on clauses that both match the input xand that are assigned to the negated\noutput value, notyi. Only literals that are False are a\ufb00ected. This matrix increases the\ndiscrimination power of the matching clauses by introducing literals that invalidate the\nmatching.\nTogether,FIa\niandFIb\niimplement forget andmemorize from Section 2. FII\niimplements\ninvalidate . See Appendix A for a formal de\ufb01nition of the feedback matrices.\n+1 +2 ...True Positive Clause Output"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d1-934e-11ee-8795-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "+1 +2 ...True Positive Clause Output\nFalse Positive Clause Output-2 -1 ...\nFigure 8: Type Ia Feedback changes the weights in one-step increments away from 0. Type II\nFeedback changes the weights in one-step decrements towards 0.\n3.4 Updating of Weight Matrix\nAs shown in Figure 8, the updating of weights is quite straightforward. Weights are only\nupdated in the case of Type Ia or Type II feedback. Type Ia Feedback changes the weights in"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d2-934e-11ee-b05a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "one-step increments away from 0. Type II Feedback changes the weights in one-step decrements\ntowards 0. Type Ib Feedback leaves the weights unchanged. The following equation captures\nthe updating of the weight matrix from time step TtoT`1:\nWT`1\u201cWT`pRI`RIIq\u02dd\u00a8\n\u02da\u02da\u02da\u02dd\u00bb\n\u2014\u2014\u2014\u2013c1y1c2y1\u00a8\u00a8\u00a8cny1\nc1y2c2y2\u00a8\u00a8\u00a8cny2\n............\nc1ymc2ym\u00a8\u00a8\u00a8cnym\ufb01\n\ufb03\ufb03\ufb03\ufb02\u00b4\u00bb\n\u2014\u2014\u2014\u2013c1\u00afy1c2\u00afy1\u00a8\u00a8\u00a8cn\u00afy1\nc1\u00afy2c2\u00afy2\u00a8\u00a8\u00a8cn\u00afy2\n............\nc1\u00afymc2\u00afym\u00a8\u00a8\u00a8cn\u00afym\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02db\n\u2039\u2039\u2039\u201a.(6)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d3-934e-11ee-9e8f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 9, "chroma:document": "............\nc1\u00afymc2\u00afym\u00a8\u00a8\u00a8cn\u00afym\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02db\n\u2039\u2039\u2039\u201a.(6)\nThe matrices RIandRIIare non-overlapping, singling out which clause rows jare eligible for\nType I and Type II Feedback. For this purpose, we again use the Hadamard product \u02dd. Let\ncj,jP t1,...,nurefer to the output of the clause in row jinC. Together with the output\nvalueyithe clause output value controls the direction of the weight update.\n4 Empirical Results\nIn this section, we evaluate the CoTM on six di\ufb00erent datasets.\n10"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d4-934e-11ee-bff5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "2D Noisy XOR. The 2D Noisy XOR dataset contains 4 \u02c64 binary images, 2500 training\nexamples and 10 000 test examples. The image bits have been set randomly, except for the\n2\u02c62 patch in the upper right corner, which reveals the class of the image. A diagonal line\nis associated with class 1, while a horizontal or vertical line is associated with class 0. Thus\nthe dataset models a 2D version of the XOR-relation. Furthermore, the dataset contains a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d5-934e-11ee-b9cc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "large number of random non-informative features to measure susceptibility towards the curse\nof dimensionality. To examine robustness towards noise we have further randomly inverted\n40% of the outputs in the training data.\nIMDB. The IMDb dataset contains 50 000 highly polar movie reviews for binary sentiment\nclassi\ufb01cation [31].\nCIFAR10. The CIFAR-10 dataset consists of 60 000 32 \u02c632 colour images. There are 10\nclasses with 6 000 images per class [24]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d6-934e-11ee-9ea0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "classes with 6 000 images per class [24].\nMNIST. The MNIST dataset has been used extensively to benchmark machine learning\nalgorithms, consisting of 28 \u02c628 grey scale images of handwritten digits [26].\nKuzushiji-MNIST. This dataset contains 28 \u02c628 grayscale images of Kuzushiji charac-\nters, cursive Japanese. Kuzushiji-MNIST is more challenging than MNIST because there are\nmultiple distinct ways to write some of the characters [14]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d7-934e-11ee-96a4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "Fashion-MNIST. This dataset contains 28 \u02c628 grayscale images of articles from the Za-\nlando catalogue, such as t-shirts, sandals, and pullovers [43]. This dataset is quite challenging,\nwith a human accuracy of 83 .50%.\nThe latter three datasets contain 60 000 training examples and 10 000 test examples. We\nbinarize these datasets using an adaptive Gaussian thresholding procedure with window size\n11 and threshold value 2. Accordingly, the CoTM operates on images with 1 bit per pixel. All"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d8-934e-11ee-840b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "experiments are repeated 10 times and we report average results for the last 25 epochs, unless\notherwise noted. We ran the CoTM on a NDVIDIA Tesla V100 GPU, calculating the clause\noutputs and updates in parallel.\n#Clauses per class 50 100 250 500 1K 2K 4K 8K\nVoting Margin 625 1250 3125 6.25K 12.5K 25K 5K 10K\nSpeci\ufb01city 10.0 10.0 10.0 10.0 10.0 10.0 5.0 5.0\nWeighted, Ac. (%) 97.86 98.43 98.82 98.98 99.14 99.22 99.28 99.33\nCoalesced, Ac. (%) 98.33 98.80 99.03 99.14 99.18 99.21 99.26 99.29"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2d9-934e-11ee-9ee8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "Training Time (s) 21.6 21.0 21.1 21.3 20.9 24.1 42.6 86.7\nTesting Time (s) 1.7 1.6 1.6 1.6 1.5 1.6 4.0 6.5\nModel Size (kB) 22 44 110 220 439 879 1758 3516\nTable 2: Weighted Convolutional TM and Convolutional CoTM MNIST mean test accuracy\nand execution time per epoch, for an increasing number of clauses.\nMNIST Results. We consider scalability \ufb01rst, investigating how the number of clauses\na\ufb00ects performance. Table 2 reports performance on unaugmented MNIST, for various number"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2da-934e-11ee-b6bf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "of clauses per class. We see that CoTM outperforms TM accuracy-wise for 50 to 1000 clauses.\nIndeed, CoTM operates at a similar level as TM with half the number of clauses, indicating\nan ability to repurpose clauses towards multiple classes. For 2000 to 8000 clauses, CoTM\nperforms competitively. As further seen in the table, going from 50 clauses to 8000 clauses\n(a\u02c6160 increase) increases testing time 3 .8 times. Similarly, training time increases 4 times."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2db-934e-11ee-8d59-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 10, "chroma:document": "Finally, model size increases proportionally with the number of clauses. Employing 50 clauses\nper class gives a model of size 22 kB, providing a test accuracy of 98 .26. As the number of\nclauses increases, so does test accuracy, with 8000 clauses per class giving a test accuracy of\n99.29.\nFashion-MNIST and Kuzushiji-MNIST Results. We observe similar behaviour for\nFashion-MNIST in Table 3 and Kuzushiji-MNIST in Table 4. However, the accuracy di\ufb00erence\n11"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2dc-934e-11ee-a4ee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "between the weighted TM and CoTM is even larger for these datasets.\n#Clauses per class 50 100 250 500 1K 2K 4K 8K\nVoting Margin 625 1250 3125 6.25K 12.5K 2.5K 5K 10K\nSpeci\ufb01city 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0\nWeighted, Ac. (%) 82.33 83.73 88.25 88.79 89.42 89.89 90.65 91.18\nCoalesced, Ac. (%) 86.79 87.4 88.0 89.20 89.83 90.00 90.71 91.18\nTraining Time (s) 25.1 24.1 24.3 23.3 23.7 25.3 45.6 81.9\nTesting Time (s) 1.6 1.6 1.6 1.5 1.7 1.6 4.1 6.4"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2dd-934e-11ee-9bd6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "Testing Time (s) 1.6 1.6 1.6 1.5 1.7 1.6 4.1 6.4\nModel Size (kB) 22 44 110 220 439 879 1758 3516\nTable 3: Weighted Convolutional TM and Convolutional CoTM Fashion-MNIST mean test\naccuracy and execution time per epoch, for an increasing number of clauses.\n#Clauses per class 50 100 250 500 1K 2K 4K 8K\nVoting Margin 625 1250 3125 6.25K 1.25K 2.5K 5K 10K\nSpeci\ufb01city 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0\nWeighted, Ac. (%) 71.99 89.19 92.75 93.86 94.89 95.40 95.85 96.08"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2de-934e-11ee-a565-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "Coalesced, Ac. (%) 89.66 91.92 93.71 94.65 95.05 95.77 96.17 96.33\nTraining Time (s) 24.2 23.4 23.4 23.7 23.7 25.6 47.3 85.1\nTesting Time (s) 1.7 1.6 1.6 1.6 1.6 1.6 4.1 6.4\nModel Size (kB) 22 44 110 220 439 879 1758 3516\nTable 4: Weighted Convolutional TM and Convolutional CoTM Kuzushiji-MNIST mean test\naccuracy and execution time per epoch, for an increasing number of clauses.\nLearning Speed. Figure 9 plots MNIST test accuracy per epoch for CoTM and weighted"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2df-934e-11ee-a113-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "TM. As seen, CoTM accuracy climbs much faster, surpassing 99 .3 in epoch 19. The weighted\nTM achieves this after 57 epochs, three times slower.\nBaselines. Table 5 contains a comparison between the CoTM with 64 000 clauses and other\nbaselines. Here, CoTM clearly outperforms traditional machine learning techniques, like SVM,\nRandom Forest, and Gradient Boosting. Also, it performs competitively with widely used deep\nlearning architectures."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e0-934e-11ee-aeb7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "learning architectures.\nClass Imbalance Robustness. We here investigate whether the clause sharing of the CoTM\nprovides any robustness towards imbalanced training data. Table 6 contains robustness results\nfor the IMDb dataset. We evaluate robustness by removing a progressively larger fraction\nof the positive sentiment training examples. The TM test accuracy drops signi\ufb01cantly with\nincreasing class imbalance. However, CoTM test accuracy is almost una\ufb00ected, indicating high\nrobustness."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e1-934e-11ee-bb7d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "robustness.\nThe \ufb01nal robustness evaluation is done with highly imbalanced CIFAR10 training data. We\nmake the data imbalanced by ordering the classes. We then use 0 .5rof the training data for the\nclass of rank rin the ordering. Table 7 contains the class-wise F1-scores for TM and CoTM.\nWe report the mean F1 score over epochs 90 \u00b4100, averaged over 10 independent trials. As\nseen, the F1 scores of TM are more severely a\ufb00ected by class imbalance than for CoTM.\n5 Conclusions and Further Work"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e2-934e-11ee-9873-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 11, "chroma:document": "5 Conclusions and Further Work\nIn this paper, we proposed a new TM architecture where clauses are shared among multiple\noutputs. We achieved this by merging multiple TMs into a single one, and then relating each\nclause to each output by weighting. A positive weight makes the clause vote for output 1,\n12"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e3-934e-11ee-b318-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 12, "chroma:document": "051015202530354045505560657075808590951009595.59696.59797.59898.59999.5100\nEpochAccuracy (%)\nWeighted TM\nCoalesced TM\nFigure 9: Single-run test accuracy per epoch for CoTM and TM on MNIST.\nwhile a negative weight makes it vote for output 0. By means of a weight matrix, the clauses\ncoalesce to produce multiple outputs.\nOur empirical evaluations support the following main conclusions:\n\u2022CoTM seems to make signi\ufb01cantly better use of few clauses, both for MNIST, Fashion-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e4-934e-11ee-985f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 12, "chroma:document": "MNIST, and Kuzushiji-MNIST. The di\ufb00erence is the largest for con\ufb01gurations with 50\nclauses per class, leading to high accuracy with frugal memory footprint and faster learn-\ning and inference.\n\u2022Peak accuracy is reached faster, for instance 3 times faster on MNIST with 8 000 clauses\nper class.\n\u2022CoTM is highly robust towards imbalanced training data, indicating an ability to repur-\npose clauses from one class to another. We believe the repurposing helps classify more\ndata sparse classes."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e5-934e-11ee-93a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 12, "chroma:document": "data sparse classes.\nIn our further work, we intend to use the CoTM to build self-supervised rule-based language\nmodels. Language models are usually based on predicting individual words or tokens from large\nvocabularies, requiring a large number of outputs. Further, words may have similar meanings,\nand we will investigate whether clause reuse can capture semantic relationships among words.\nWe also see opportunities for using the CoTM as an auto-encoder, supporting applications such"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e6-934e-11ee-9d01-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 12, "chroma:document": "as turning low-resolution images into high-resolution ones.\nReferences\n[1] K. D. Abeyrathna, B. Bhattarai, M. Goodwin, S. Gorji, O.-C. Granmo, L. Jiao, R. Saha,\nand R. K. Yadav. Massively Parallel and Asynchronous Tsetlin Machine Architecture\nSupporting Almost Constant-Time Scaling. In The Thirty-eighth International Conference\non Machine Learning (ICML 2021) . ICML, 2021.\n[2] K. D. Abeyrathna, O.-C. Granmo, and M. Goodwin. Extending the Tsetlin Machine With"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e7-934e-11ee-815e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 12, "chroma:document": "Integer-Weighted Clauses for Increased Interpretability. IEEE Access , 9:8233 \u2013 8248, 2021.\n13"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e8-934e-11ee-ae46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "Model 2D N-XOR MNIST K-MNIST F-MNIST\n4-Nearest Neighbour [14, 43] 61.62 97.14 91.56 85.40\nSVM [14] 94.63 98.57 92.82 89.7\nRandom Forest [28] 70.73 97.3 - 81.6\nGradient Boosting Classi\ufb01er [43] 87.15 96.9 - 88.0\nSimple CNN [14, 28] 91.05 99.06 95.12 90.7\nBinaryConnect [15] - 98.99 - -\nFPGA-accelerated BNN [25] - 98.70 - -\nLogistic Circuit (binary) [28] - 97.4 - 87.6\nLogistic Circuit (real-valued) [28] - 99.4 - 91.3\nPreActResNet-18 [14] - 99.56 97.82 92.0"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2e9-934e-11ee-87cc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "PreActResNet-18 [14] - 99.56 97.82 92.0\nResNet18 + VGG Ensemble [14] - 99.60 98.90 -\nTM [20] 99.12 98.57 92.03 90.09\nCoTM (Mean) 99.99\u02d80.099.31\u02d80.096.65\u02d80.01 91.83\u02d80.01\nCoTM (95 %ile) 100.0 99.34 96.70 91.93\nCoTM (Peak) 100.0 99.36 96.71 92.0\nTable 5: Comparison with other machine learning techniques \u2013 test accuracy in percent.\nFraction Removed 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9\n\u2206 TM 0.0 1.0 4.3 7.9 10.3 17.2 24.8 27.3 29.5 30.5\n\u2206 CoTM 0.0 0.5 0.2 0.0 0.2 0.4 0.4 0.8 1.4 2.8"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ea-934e-11ee-ab51-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "\u2206 CoTM 0.0 0.5 0.2 0.0 0.2 0.4 0.4 0.8 1.4 2.8\nTable 6: Reduction in IMDb test accuracy after removing a fraction of the training examples\nwith positive sentiment.\n[3] K. D. Abeyrathna, O.-C. Granmo, R. Sha\ufb01k, A. Yakovlev, A. Wheeldon, J. Lei, and\nM. Goodwin. A Novel Multi-Step Finite-State Automaton for Arbitrarily Deterministic\nTsetlin Machine Learning. In Proceedings of the 40th International Conference on In-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2eb-934e-11ee-9301-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "novative Techniques and Applications of Arti\ufb01cial Intelligence (SGAI), Cambridge, UK .\nSpringer International Publishing, 2020.\n[4] K. D. Abeyrathna, O.-C. Granmo, X. Zhang, L. Jiao, and M. Goodwin. The Regression\nTsetlin Machine - A Novel Approach to Interpretable Non-Linear Regression. Philosophical\nTransactions of the Royal Society A , 378, 2020.\n[5] R. Agarwal, N. Frosst, X. Zhang, R. Caruana, and G. E. Hinton. Neural additive models:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ec-934e-11ee-81cc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "Interpretable machine learning with neural nets. arXiv preprint arXiv:2004.13912 , 2020.\n[6] G. T. Berge, O.-C. Granmo, T. O. Tveit, M. Goodwin, L. Jiao, and B. V. Matheussen.\nUsing the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text\nCategorization with Medical Applications. IEEE Access , 7:115134\u2013115146, 2019.\n[7] B. Bhattarai, O.-C. Granmo, and L. Jiao. Explainable Tsetlin Machine framework for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ed-934e-11ee-97af-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "fake news detection with credibility score assessment. arXiv preprint arXiv:2105.09114 ,\n2021.\n[8] B. Bhattarai, O.-C. Granmo, and L. Jiao. Word-level Human Interpretable Scoring Mech-\nanism for Novel Text Detection Using Tsetlin Machines. arXiv preprint arXiv:2105.04708 ,\n2021.\n[9] B. Bhattarai, L. Jiao, and O.-C. Granmo. Measuring the Novelty of Natural Language Text\nUsing the Conjunctive Clauses of a Tsetlin Machine Text Classi\ufb01er. In 13th International"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ee-934e-11ee-90e1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 13, "chroma:document": "Conference on Agents and Arti\ufb01cial Intelligence (ICAART), Vienna , Austria . INSTICC,\n2021.\n14"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ef-934e-11ee-9d59-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "Class airplane automobile bird cat deer dog frog horse ship truck\n#examples 5000 2500 1250 625 315 157 80 40 20 10\nCoTM : F1 0.47 0.50 0.36 0.36 0.26 0.15 0.02 0.04 0.00 0.00\nTM : F1 0.39 0.42 0.25 0.20 0.10 0.00 0.00 0.00 0.00 0.00\nTable 7: The class-wise F1 scores for a highly imbalanced version of the CIFAR10 dataset.\n[10] C. D. Blakely and O.-C. Granmo. Closed-Form Expressions for Global and Local Interpre-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f0-934e-11ee-8833-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "tation of Tsetlin Machines. In 34th International Conference on Industrial, Engineering\nand Other Applications of Applied Intelligent Systems (IEA/AIE 2021) . Springer, 2021.\n[11] L. Breiman. \u201drandom forests\u201d. Machine Learning , 45(1):5\u201332, 2001.\n[12] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. \u201dClassi\ufb01cation and Regression\nTrees\u201d . Wadsworth and Brooks, Monterey, CA, 1984.\n[13] T. Chen and C. Guestrin. \u201dxgboost: A scalable tree boosting system\u201d. In \u201dProceedings"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f1-934e-11ee-9326-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data\nMining\u201d , pages 785\u2013794, 2016.\n[14] T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha. Deep\nLearning for Classical Japanese Literature. arXiv:1812.01718 , Dec 2018.\n[15] M. Courbariaux, Y. Bengio, and J.-P. David. Binaryconnect: Training deep neural net-\nworks with binary weights during propagations. In Advances in neural information pro-\ncessing systems , pages 3123\u20133131, 2015."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f2-934e-11ee-81da-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "cessing systems , pages 3123\u20133131, 2015.\n[16] H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik. \u201dsupport vector\nregression machines\u201d. In M. C. Mozer, M. Jordan, and T. Petsche, editors, Advances in\nNeural Information Processing Systems , volume 9. MIT Press, 1997.\n[17] S. Gorji, O. C. Granmo, S. Glimsdal, J. Edwards, and M. Goodwin. Increasing the In-\nference and Learning Speed of Tsetlin Machines with Clause Indexing. In International"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f3-934e-11ee-97e1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "Conference on Industrial, Engineering and Other Applications of Applied Intelligent Sys-\ntems (IEA/AIE), kitakyushu, Japan . Springer, 2020.\n[18] S. R. Gorji, O.-C. Granmo, A. Phoulady, and M. Goodwin. A Tsetlin Machine with\nMultigranular Clauses. In Proceedings of the Thirty-ninth International Conference on\nInnovative Techniques and Applications of Arti\ufb01cial Intelligence (SGAI), Cambridge, UK ,\nvolume 11927. Springer International Publishing, 2019."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f4-934e-11ee-a000-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "[19] O.-C. Granmo. The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to\nOptimal Pattern Recognition with Propositional Logic. arXiv:1804.01508 , Apr 2018.\n[20] O.-C. Granmo, S. Glimsdal, L. Jiao, M. Goodwin, C. W. Omlin, and G. T. Berge. The\nConvolutional Tsetlin Machine. arXiv preprint arXiv:1905.09688 , 2019.\n[21] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f5-934e-11ee-b433-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 14, "chroma:document": "Proceedings of the IEEE conference on computer vision and pattern recognition , pages\n770\u2013778, 2016.\n[22] L. Jiao, X. Zhang, O.-C. Granmo, and K. D. Abeyrathna. On the Convergence of Tsetlin\nMachines for the XOR Operator. arXiv preprint arXiv:2101.02547 , 2021.\n[23] M. Karnaugh. The map method for synthesis of combinational logic circuits. Transactions\nof the American Institute of Electrical Engineers, Part I: Communication and Electronics ,\n72(5):593\u2013599, 1953.\n15"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f6-934e-11ee-b30d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "[24] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report,\n2009.\n[25] C. Lammie, W. Xiang, and M. Rahimi Azghadi. Accelerating Deterministic and Stochastic\nBinarized Neural Networks on FPGAs Using OpenCL. arXiv:1905.06105 , May 2019.\n[26] Y. LeCun, L. Bottou, Y. Bengio, P. Ha\ufb00ner, et al. Gradient-based learning applied to\ndocument recognition. Proceedings of the IEEE , 86(11):2278\u20132324, 1998."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f7-934e-11ee-80b5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "[27] J. Lei, T. Rahman, R. Sha\ufb01k, A. Wheeldon, A. Yakovlev, O.-C. Granmo, F. Kawsar, and\nA. Mathur. Low-Power Audio Keyword Spotting Using Tsetlin Machines. Journal of Low\nPower Electronics and Applications , 11, 2021.\n[28] Y. Liang and G. Van den Broeck. Learning logistic circuits. In Proceedings of the 33rd\nConference on Arti\ufb01cial Intelligence (AAAI) , jan 2019.\n[29] B. Lucena. \u201dstructureboost: E\ufb03cient gradient boosting for structured categorical vari-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f8-934e-11ee-bfa4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "ables\u201d. arXiv preprint arXiv:2007.04446 , 2020.\n[30] S. M. Lundberg and S.-I. Lee. A uni\ufb01ed approach to interpreting model predictions.\nIn I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and\nR. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Cur-\nran Associates, Inc., 2017.\n[31] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2f9-934e-11ee-8ff4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association\nfor Computational Linguistics: Human Language Technologies , pages 142\u2013150, Portland,\nOregon, USA, June 2011. Association for Computational Linguistics.\n[32] H. Nori, S. Jenkins, P. Koch, and R. Caruana. \u201dinterpretml: A uni\ufb01ed framework for\nmachine learning interpretability\u201d. arXiv preprint arXiv:1909.09223 , 2019."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2fa-934e-11ee-8368-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "[33] B. J. Oommen. Stochastic searching on the line and its applications to parameter learning\nin nonlinear optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B\n(Cybernetics) , 27(4):733\u2013739, 1997.\n[34] A. Phoulady, O.-C. Granmo, S. R. Gorji, and H. A. Phoulady. The Weighted Tsetlin\nMachine: Compressed Representations with Clause Weighting. In Proceedings of the Ninth\nInternational Workshop on Statistical Relational AI (StarAI), New York, USA , 2020."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2fb-934e-11ee-8629-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "[35] R. Saha, O.-C. Granmo, and M. Goodwin. Mining Interpretable Rules for Sentiment\nand Semantic Relation Analysis using Tsetlin Machines. In Proceedings of the 40th Inter-\nnational Conference on Innovative Techniques and Applications of Arti\ufb01cial Intelligence\n(SGAI), Cambridge, UK . Springer International Publishing, 2020.\n[36] R. Sha\ufb01k, A. Wheeldon, and A. Yakovlev. Explainability and Dependability Analysis\nof Learning Automata based AI Hardware. In IEEE 26th International Symposium on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2fc-934e-11ee-9da3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 15, "chroma:document": "On-Line Testing and Robust System Design (IOLTS), Naples, Italy . IEEE, 2020.\n[37] J. Sharma, R. Yadav, O.-C. Granmo, and L. Jiao. Human Interpretable AI: Enhancing\nTsetlin Machine Stochasticity with Drop Clause. arXiv preprint arXiv:2105.14506 , 2021.\n[38] M. L. Tsetlin. On behaviour of \ufb01nite automata in random medium. Avtomat. i Telemekh ,\n22(10):1345\u20131354, 1961.\n[39] L. G. Valiant. A Theory of the Learnable. Communications of the ACM , 27(11):1134\u20131142,\n1984.\n16"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2fd-934e-11ee-b372-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 16, "chroma:document": "[40] A. Wheeldon, R. Sha\ufb01k, T. Rahman, J. Lei, A. Yakovlev, and O.-C. Granmo. Learning\nAutomata based Energy-e\ufb03cient AI Hardware Design for IoT. Philosophical Transactions\nof the Royal Society A , 2020.\n[41] A. Wheeldon, R. Sha\ufb01k, A. Yakovlev, J. Edwards, I. Haddadi, and O.-C. Granmo. Tsetlin\nMachine: A New Paradigm for Pervasive AI. In Proceedings of the SCONA Workshop at\nDesign, Automation and Test in Europe (DATE) , 2020."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2fe-934e-11ee-b3ad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 16, "chroma:document": "[42] A. Wheeldon, A. Yakovlev, R. Sha\ufb01k, and J. Morris. Low-Latency Asynchronous Logic\nDesign for Inference at the Edge. In 2021 Design, Automation and Test in Europe Con-\nference (DATE21) , pages 370\u2013373, 2021.\n[43] H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking\nmachine learning algorithms. arXiv:1708.07747 , 2017.\n[44] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Distributed Word Representation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc2ff-934e-11ee-bb4d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 16, "chroma:document": "in Tsetlin Machine. arXiv preprint arXiv:2104.06901 , 2021.\n[45] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Human-Level Interpretable Learn-\ning for Aspect-Based Sentiment Analysis. In The Thirty-Fifth AAAI Conference on Arti-\n\ufb01cial Intelligence (AAAI-21) . AAAI, 2021.\n[46] R. K. Yadav, L. Jiao, O.-C. Granmo, and M. Goodwin. Interpretability in Word Sense\nDisambiguation using Tsetlin Machine. In 13th International Conference on Agents and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc300-934e-11ee-8429-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 16, "chroma:document": "Arti\ufb01cial Intelligence (ICAART), Vienna, Austria . INSTICC, 2021.\n[47] X. Zhang, L. Jiao, O.-C. Granmo, and M. Goodwin. On the Convergence of Tsetlin Ma-\nchines for the IDENTITY- and NOT Operators. IEEE Transactions on Pattern Analysis\nand Machine Intelligence , 2021.\nA Appendix\nA.1 Structure and Prediction\nA CoTM can be de\ufb01ned as a quadruple:\ntX,Y,C,Wu. (7)\nWe start with de\ufb01ning each element of the quadruple.\nA.1.1 Input Space"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc301-934e-11ee-9e1f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 16, "chroma:document": "A.1.1 Input Space\nThe input space of the CoTM is denoted X. The input space consists of vectors xofo\npropositional inputs: x\u201crx1,...,x osPX,X\u201ct0,1uo.\nA.1.2 Output Space\nThe output space is denoted Y. The output space contains vectors yofmpropositional\noutputs: y\u201cry1,y2,...,ymsPY,Y\u201ct0,1um.\n17"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc302-934e-11ee-9fb7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "A.1.3 Memory Matrix\nWe de\ufb01ne Cas a space of matrices Cformed from n\u02c62ointeger entries, CPC\u201ct1,2,..., 2Nun\u02c62o:\nC\u201cx1x2\u00a8\u00a8\u00a8xo \u00afx1 \u00afx2\u00a8\u00a8\u00a8 \u00afxo\u00bb\n\u2014\u2014\u2013\ufb01\n\ufb03\ufb03\ufb02c1,1c1,2\u00a8\u00a8\u00a8c1,oc1,o`1c1,o`2\u00a8\u00a8\u00a8c1,2oclause 1\nc2,1c2,2\u00a8\u00a8\u00a8c2,oc2,o`1c2,o`2\u00a8\u00a8\u00a8c2,2oclause 2\n........................\ncn,1cn,2\u00a8\u00a8\u00a8cn,ocn,o`1cn,o`2\u00a8\u00a8\u00a8cn,2oclausen(8)\n\u201c rcj,ksPt1,2,..., 2Nun\u02c62o. (9)\nThis is the pattern memory of the CoTM, with each row specifying a clause. The nmatrix"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc303-934e-11ee-bebe-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "rows represent nclauses while the 2 ocolumns speci\ufb01es each pattern. That is, each column\nrefers to an input xkor its negation \u00af xk(we use \u00afxkas shorthand notation for \u2423xk). The entries\nin the matrix take values from t1,2,..., 2Nu, specifying the composition of the nclauses as\nwell as the con\ufb01dence of the composition (how strongly the pattern is memorized).\nA.1.4 Pattern Construction\nACmatrix provides npatterns as follows. The \ufb01rst step is to map each entry in Cto one of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc304-934e-11ee-9639-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "two actions aPt0,1u, obtaining the matrix APt0,1un\u02c62obelow. The function Gp\u00a8qperforms\nthis mapping by selecting action 0 for entries in Cwith values from 1 to Nand by selecting\naction 1 for entries with values from N`1 to 2N:\ngpcq \u201c#\n0,if1\u010fc\u010fN\n1,ifN`1\u010fc\u010f2N.(10)\nA\u201cGpCq \u201c\u00bb\n\u2014\u2014\u2014\u2013gpc1,1qgpc1,2q \u00a8\u00a8\u00a8gpc1,2oq\ngpc2,1qgpc2,2q \u00a8\u00a8\u00a8gpc2,2oq\n............\ngpcn,1qgpcn,2q \u00a8\u00a8\u00a8gpcn,2oq\ufb01\n\ufb03\ufb03\ufb03\ufb02(11)\n\u201c\u00bb\n\u2014\u2014\u2014\u2013a1,1a1,2\u00a8\u00a8\u00a8a1,2o\na2,1a2,2\u00a8\u00a8\u00a8a2,2o\n............\nan,1an,2\u00a8\u00a8\u00a8an,2o\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201craj,ksPt0,1un\u02c62o. (12)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc305-934e-11ee-b55b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "an,1an,2\u00a8\u00a8\u00a8an,2o\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201craj,ksPt0,1un\u02c62o. (12)\nValues close to NandN`1 signify low-con\ufb01dence actions while values close to 1 and 2 N\nsignify high-con\ufb01dence actions (deeply memorized).\nAfter the actions A\u201cGpCqhave been decided, the next step is to formulate the patterns.\nEach pattern is formed by ANDing selected entries xkin the input vector x, or their negations\n\u00afxk.3The pattern\nx1^x2^\u00afx3, (13)\nfor instance, ANDs the values x1,x2, and \u00afx3. It evaluates to 1 if and only if x1\u201c1,x2\u201c1,\nandx3\u201c0."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc306-934e-11ee-ae9d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "andx3\u201c0.\nThe actions from GpCqdecide which inputs to AND, and whether to negate them. For\n1\u010fk\u010fo, if actionaj,kis 1 then input xktakes part in the AND of pattern j. Foro`1\u010fk\u010f2o,\n\u00afxk\u00b4otakes part in the AND instead. If aj,kis 0, on the other hand, the corresponding input\ndoes not play a role in the evaluation of the jth clause and can be ignored.\nThe imply (\u00f1) operator captures the above dynamics. That is, if ajkis 0, thenaj,k\u00f1xk"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc307-934e-11ee-b309-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 17, "chroma:document": "is always 1. Accordingly, the xkvalue does not impact the result. Conversely, the value of\n3Such an expression is referred to as a conjunctive clause in propositional logic, with xkand \u00afxkbeing called\nliterals .\n18"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc308-934e-11ee-a2d0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 18, "chroma:document": "aj,k\u00f1xkis decided solely by the xkvalue ifajkis 1. We thus de\ufb01ne an Imply operator to\ncalculate the e\ufb00ect of each input on each clause:\nImplyrA,xs\u201c\u00bb\n\u2014\u2014\u2014\u2013a1,1\u00f1x1\u00a8\u00a8\u00a8a1,o\u00f1xoa1,o`1\u00f1\u00afx1\u00a8\u00a8\u00a8a1,2o\u00f1\u00afxo\na2,1\u00f1x1\u00a8\u00a8\u00a8a2,o\u00f1xoa2,o`1\u00f1\u00afx1\u00a8\u00a8\u00a8a2,2o\u00f1\u00afxo\n..................\nan,1\u00f1x1\u00a8\u00a8\u00a8an,o\u00f1xoan,o`1\u00f1\u00afx1\u00a8\u00a8\u00a8an,2o\u00f1\u00afxo\ufb01\n\ufb03\ufb03\ufb03\ufb02. (14)\nFinally, we de\ufb01ne the operator And for evaluating the clauses:\nAndrZs\u201c\u00bb\n\u2014\u2014\u2014\u2013\u01792o\nk\u201c1z1,k\u01792o\nk\u201c1z2,k\n...\u01792o\nk\u201c1zn,k\ufb01\n\ufb03\ufb03\ufb03\ufb02. (15)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc309-934e-11ee-9738-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 18, "chroma:document": "k\u201c1z1,k\u01792o\nk\u201c1z2,k\n...\u01792o\nk\u201c1zn,k\ufb01\n\ufb03\ufb03\ufb03\ufb02. (15)\nAccordingly, the value of each clause can be calculated from the input vector xand the pattern\nmemoryC, providing a pattern value column vector c:\nc\u201c\u00bb\n\u2014\u2014\u2014\u2013c1\nc2\n...\ncn\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201cAndpImplypGpCq,xqq. (16)\nAs seen, we employ the npatterns from GpCqwhen processing the input vector x.\nA.1.5 Output Prediction\nFor predicting the output vector, the CoTM uses m\u02c6ninteger weights. These capture how"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30a-934e-11ee-980a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 18, "chroma:document": "each of the nclauses relates to each of the moutputs. The weights are organized as a matrix:\nW\u201c\u00bb\n\u2014\u2014\u2014\u2013w1,1w1,2\u00a8\u00a8\u00a8w1,n\nw2,1w2,2\u00a8\u00a8\u00a8w2,n\n............\nwm,1wm,2\u00a8\u00a8\u00a8wm,n\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201crwi,jsPt...,\u00b42,\u00b41,1,2,...um\u02c6n. (17)\nWe have one weight per output iper clausej. In the following, we will refer to row iofWas\nwi:\nwi\u201c\u201c\nwi,1wi,2\u00a8\u00a8\u00a8wi,n\u2030\n. (18)\nThe weights decide the e\ufb00ect a clause value cj\u201c1 from chas on predicting the output vector."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30b-934e-11ee-8dfa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 18, "chroma:document": "If the weight wi,jis positive and cj\u201c1, thejth clause casts wi,jvotes in favour of output\nyi\u201c1. If the weight is negative and cj\u201c1, clausejcasts\u00b4wi,jvotes in favour of output\nyi\u201c0 instead. If cj\u201c0, on the other hand, no votes are cast from that pattern. The overall\ncalculation can be formulated as a product between the matrix Wand the column vector c,\nobtaining a column vector vof votes:\nv\u201cWc. (19)\nFinally, the output vector prediction is decided by a majority vote using a unit step operator U:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30c-934e-11ee-8a69-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 18, "chroma:document": "Upvq\u201c\u00bb\n\u2014\u2014\u2014\u2013v1\u011b0\nv2\u011b0\n...\nvm\u011b0\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201cruisPt0,1um\u02c61. (20)\nCombining the di\ufb00erent steps, we predict the output yfrom input xas follows:\n\u02c6y\u201cUpW\u00a8AndpImplypGpCq,xqqqT. (21)\n19"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30d-934e-11ee-96b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "A.2 Learning Patterns and Weights\nThe memory matrix Cand the weight matrix Ware learnt online from a set Sof training\nexamplespx,yqPS. The goal is to maximize prediction accuracy Pp\u02c6yi\u201cyiqper entry in y.\nWe here consider the procedure for updating CandWfrom a single training example px,yq.\nTo learn from a stream of training examples, this procedure is repeated for every example. We\n\ufb01rst extend the TM clause coordination mechanism to stimulate production of diverse clauses"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30e-934e-11ee-af6d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "for multiple outputs. This mechanism activates clause rows in Cfor updating based on the\nvote sums vand the example outputs y. We then go through the details of how we update each\nactivated clause in Cbased onpx,yq. Finally, we consider the updating of the corresponding\nweights inW, learning the relationship between each clause in Cand each output in y.\nA.2.1 Resource Allocation\nTo ensure that the nclause rows of Care diverse and robust, the CoTM employs a voting"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc30f-934e-11ee-8e49-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "margintfor each entry yiiny. The voting margin is a user-con\ufb01gurable constant that puts\na max on how many clauses from Ccoalesce to predict a particular output yiwhen facing a\ntraining example px,yq. While the original TM considers a single output y, we here present a\nscheme for handling a vector yof outputs.\nFor CoTM, when an entry yiin the training output yis 0, the summation margin is \u00b4t.\nConversely, when the entry is 1 the summation margin is t. The resulting nsummation margins"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc310-934e-11ee-9fad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "foryare organized in the column vector q:\nq\u201cyt\u00b4\u00afyt\u201c\u00bb\n\u2014\u2014\u2014\u2013y1t\u00b4\u00afy1t\ny2t\u00b4\u00afy2t\n...\nymt\u00b4\u00afymt\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201crqisPt\u00b4t,tum\u02c61. (22)\nAfter the summation margins qfor the training example px,yqhave been set, the next step\nis to decide which clause rows in Cto update. This is done by comparing each vote sum vi\nfromv(Eqn. 19) with the corresponding margin yit\u00b4\u00afyitinq. By clipping the vote sum to fall\nbetween\u00b4tandt, we calculate an update probability per output yiiny:abs\u00b4\nqi\u00b4clippvi,\u00b4t,tq\n2t\u00af\n."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc311-934e-11ee-aebf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "qi\u00b4clippvi,\u00b4t,tq\n2t\u00af\n.\nThe update probability is 0 .0 when the summation margin tis met or surpassed, and gradually\nincreases to 1 .0 towards the maximum distance 2 t. The update probabilities are organized in\nthe column vector d:\nd\u201cabspq\u00b4clippv,\u00b4t,tqq\u00a81\n2t\u201cabs\u00a8\n\u02da\u02da\u02da\u02da\u02dd\u00bb\n\u2014\u2014\u2014\u2014\u2013q1\u00b4clippv1,\u00b4t,tq\n2tq2\u00b4clippv2,\u00b4t,tq\n2t...\nqm\u00b4clippvm,\u00b4t,tq\n2t\ufb01\n\ufb03\ufb03\ufb03\ufb03\ufb02\u02db\n\u2039\u2039\u2039\u2039\u201a\u201crdisPtxPR|0\u010fx\u010f1um\u02c61.\n(23)\nFrom the update probabilities in d, we next randomly select which clause rows in Cto update."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc312-934e-11ee-847c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "In brief, many clauses are updated when the vote sum viis far from its margin yit\u00b4\u00afyit, and\ncomes to a complete standstill when the margin is met or surpassed. Accordingly, we attain a\nnatural balancing of pattern representation resources.\nA.2.2 Type I Feedback \u2013 Suppressing False Negative Output\nThere are two kinds of updates, Type I and Type II. We consider the nature of these next.\nType I Feedback reinforces the output entries yifrom the training example px,yq, considering"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc313-934e-11ee-83fa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 19, "chroma:document": "one output entry yiat a time. Each output entry yiis either 0 or 1. As seen from Eqn. 21,\noutputyi\u201c1 can be reinforced by identifying the clause rows jinCthat has a positive weight\nwi,j\u011b0, and then making those evaluate to 1. Output yi\u201c0, on the other hand, can be\nreinforced by making clauses with negative weight wi,j\u01030 evaluate to 1. In other words,\nType I Feedback is given when yidpwi,j\u011b0q, wheredis the XNOR-operator.\n20"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc314-934e-11ee-8bbd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "The selection of clauses to update is done by sampling a random value 0 or 1 per output-\nclause pairpi,jq, withireferring to an entry in yandjreferring to a row in C. The probability\nof updating from a pair pi,jqis decided by the update probability diind. We draw m\u02c6n\nrandom values \u03c0i,juniformly in the range r0,1sto decide the updating. The updating performed\nper pair is the same as for the original TM, except that the CoTM combines multiple outputs yi."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc315-934e-11ee-8e25-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "We organize the selection criterion yidpwi,j\u011b0qin onem\u02c6nmatrix and the random\nsampling in another. The two matrices are combined by element-wise multiplication using the\nHadamard product \u02dd:\nRI\u201c\u00bb\n\u2014\u2014\u2014\u2013y1dpw1,1\u011b0qy1dpw1,2\u011b0q \u00a8\u00a8\u00a8y1dpw1,n\u011b0q\ny2dpw2,1\u011b0qy1dpw2,2\u011b0q \u00a8\u00a8\u00a8y2dpw2,n\u011b0q\n............\nymdpwm,1\u011b0qy1dpwm,2\u011b0q \u00a8\u00a8\u00a8ymdpwm,n\u011b0q\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02dd (24)\n\u00bb\n\u2014\u2014\u2014\u2013\u03c01,1\u010fd1\u03c01,2\u010fd1\u00a8\u00a8\u00a8\u03c01,n\u010fd1\n\u03c02,1\u010fd2\u03c02,2\u010fd2\u00a8\u00a8\u00a8\u03c02,n\u010fd2\n............\n\u03c0m,1\u010fdm\u03c0m,2\u010fdm\u00a8\u00a8\u00a8\u03c0m,n\u010fdm\ufb01\n\ufb03\ufb03\ufb03\ufb02(25)\n\u201c rrI\ni,jsPt0,1um\u02c6n. (26)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc316-934e-11ee-92ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "\ufb03\ufb03\ufb03\ufb02(25)\n\u201c rrI\ni,jsPt0,1um\u02c6n. (26)\nIn other words, matrix RIpicks out which rows in Cto update, per output yi. Like for the\noriginal TM, the actual updates subdivides into Type Ia and Type Ib feedback, which we\nconsider next.\nType Ia Feedback \u2013 Re\ufb01ning Clauses. Type Ia Feedback is given to clause rows jin\nCthat evaluate to 1. Reinforcement is given per variable xkor its negation \u00af xk, by updating\nmatrixC. In brief, action a\u201c1 is reinforced by increasing cj,kwhenevercj\u201c1andxk\u201c1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc317-934e-11ee-a3b4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "for 1\u011bk\u011boandcj\u201c1and \u00afxk\u00b4o\u201c1 foro`1\u011bk\u011b2o. This reinforcement is strong and\nmakes the clause remember and re\ufb01ne the pattern it recognizes in x.4Which entries cj,kinC\nto increase in this manner are organized in the matrix FIa\ni, per output yi:\nFIa\ni\u201c\u201c\nc\u00a8\u00a8\u00a8c\u2030\n\u02dd\u00bb\n\u2014\u2014\u2014\u2013x1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\nx1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\n..................\nx1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\ufb01\n\ufb03\ufb03\ufb03\ufb02(27)\n\u201c\u00bb\n\u2014\u2014\u2014\u2013c1c1\u00a8\u00a8\u00a8c1\nc2c2\u00a8\u00a8\u00a8c2\n............\ncncn\u00a8\u00a8\u00a8cn\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02dd\u00bb\n\u2014\u2014\u2014\u2013x1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\nx1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\n..................\nx1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\ufb01\n\ufb03\ufb03\ufb03\ufb02. (28)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc318-934e-11ee-85cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "..................\nx1\u00a8\u00a8\u00a8xo\u00afx1\u00a8\u00a8\u00a8 \u00afxo\ufb01\n\ufb03\ufb03\ufb03\ufb02. (28)\nType Ib Feedback \u2013 Coarsening Clauses. Type Ib Feedback reinforces actions aj,k\u201c0.\nThat is,aj,k\u201c0 in matrix Ais reinforced by reducing cj,kin matrixC. The reinforcement\nhappens randomly with probability1\nswhenevercj\u201c0orxk\u201c0 for 1\u011bk\u011boand whenever\ncj\u201c0or\u00afxk\u00b4o\u201c0 foro`1\u011bk\u011b2o. This reinforcement is weak (triggered with low\nprobability) and coarsens infrequent patterns, making them frequent. The user-con\ufb01gurable"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc319-934e-11ee-a259-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 20, "chroma:document": "parameterscontrols pattern frequency, i.e., a higher sproduces less frequent patterns. Which\nentriescj,kinCto reduce in this manner are organized in the matrix FIb\ni, per output entry yi.\nWe \ufb01rst randomly identify which entries in Care candidates for updating. We draw n\u02c62o\nrandom values \u03c0i,juniformly in the range r0,1sto decide this updating, organized in the n\u02c62o\n4Note that we here apply boosting of true positives. The standard setup is to apply the update with\nprobabilitys\u00b41\ns.\n21"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31a-934e-11ee-915c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 21, "chroma:document": "matrixBi:\nBi\u201c\u00bb\n\u2014\u2014\u2014\u2013\u03c01,1\u010f1\ns\u03c01,2\u010f1\ns\u00a8\u00a8\u00a8\u03c01,2o\u010f1\ns\n\u03c02,1\u010f1\ns\u03c02,2\u010f1\ns\u00a8\u00a8\u00a8\u03c02,2o\u010f1\ns............\n\u03c0n,1\u010f1\ns\u03c0n,2\u010f1\ns\u00a8\u00a8\u00a8\u03c0n,2o\u010f1\ns\ufb01\n\ufb03\ufb03\ufb03\ufb02\u201crbi\nj,ksPt0,1un\u02c62o. (29)\nIn this manner, which entries in Cto decrease is organized in the m\u02c62omatrixFIb\nifor the\nparticular output yi:\nFIb\ni\u201cBi\u02dd\u00bb\n\u2014\u2014\u2014\u2013\u00afx1_\u00afc1\u00a8\u00a8\u00a8 \u00afxo_\u00afc1x1_\u00afc1\u00a8\u00a8\u00a8xo_\u00afc1\n\u00afx1_\u00afc2\u00a8\u00a8\u00a8 \u00afxo_\u00afc2x1_\u00afc2\u00a8\u00a8\u00a8xo_\u00afc2\n..................\n\u00afx1_\u00afcn\u00a8\u00a8\u00a8 \u00afxo_\u00afcnx1_\u00afcn\u00a8\u00a8\u00a8xo_\u00afcn\ufb01\n\ufb03\ufb03\ufb03\ufb02. (30)\nA.2.3 Type II Feedback \u2013 Increasing Discrimination Power"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31b-934e-11ee-b00c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 21, "chroma:document": "Type II Feedback is given to clause rows jinCthat outputs false positives. That is, when yi\u201c1\nthe a\ufb04icted clauses are the ones with negative weight wi,j\u01030, considering each output-clause\npair (j,i). Conversely, when yi\u201c0 clauses with positive weight wi,j\u011b0 are targeted. More\nconcisely stated, Type II Feedback is given when yi\u2018pwi,j\u011b0q, where\u2018is the XOR-operator.\nIn addition to this selection criteria, rows jinCare selected randomly for updating per pair"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31c-934e-11ee-8e24-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 21, "chroma:document": "pi,jqdecided by the update probability diind. Uniquely for Type II Feedback, dis further\nmultiplied by a scalar e, which usually is set to 1 .0. However, for one-hot-encoded multi-class\nproblems,eis set to 1.0{pm\u00b41q. The two selection criteria are combined through element-wise\nmultiplication using the Hadamard product, and organized in the m\u02c6nmatrixRII:\nRII\u201c\u00bb\n\u2014\u2014\u2014\u2013y1\u2018pw1,1\u011b0qy1\u2018pw1,2\u011b0q \u00a8\u00a8\u00a8y1\u2018pw1,n\u011b0q\ny2\u2018pw2,1\u011b0qy1\u2018pw2,2\u011b0q \u00a8\u00a8\u00a8y2\u2018pw2,n\u011b0q\n............\nym\u2018pwm,1\u011b0qy1\u2018pwm,2\u011b0q \u00a8\u00a8\u00a8ym\u2018pwm,n\u011b0q\ufb01"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31d-934e-11ee-9d45-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 21, "chroma:document": "ym\u2018pwm,1\u011b0qy1\u2018pwm,2\u011b0q \u00a8\u00a8\u00a8ym\u2018pwm,n\u011b0q\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02dd (31)\n\u00bb\n\u2014\u2014\u2014\u2013\u03c01,1\u010fd1e \u03c0 1,2\u010fd1e\u00a8\u00a8\u00a8\u03c01,n\u010fd1e\n\u03c02,1\u010fd2e \u03c0 2,2\u010fd2e\u00a8\u00a8\u00a8\u03c02,n\u010fd2e\n............\n\u03c0m,1\u010fdme \u03c0 m,2\u010fdme\u00a8\u00a8\u00a8\u03c0m,n\u010fdme\ufb01\n\ufb03\ufb03\ufb03\ufb02(32)\n\u201c rrII\ni,jsPt0,1um\u02c6n. (33)\nLike Type Ia Feedback, we only update clause rows jinCthat evaluates to 1, i.e., cj\u201c1.\nFinally, only Exclude actions for literals of value 0 are given feedback, reinforcing Include .\nThat is, state cj,kincreases when \u00af xk\u00afaj,k\u201c1, for 1\u011bk\u011bo. Foro`1\u011bk\u011b2o, statecj,k"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31e-934e-11ee-adb5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 21, "chroma:document": "increases when xk\u00b4o\u00afaj,k\u201c1. These criteria are ANDed in the FII\nimatrix through element-wise\nmultiplication:\nFII\ni\u201c\u201c\nc\u00a8\u00a8\u00a8c\u2030\n\u02dd\u00bb\n\u2014\u2014\u2014\u2013\u00afx1\u00afa1,1\u00a8\u00a8\u00a8 \u00afxo\u00afa1,ox1\u00afa1,o`1\u00a8\u00a8\u00a8xo\u00afa1,2o\n\u00afx1\u00afa2,1\u00a8\u00a8\u00a8 \u00afxo\u00afa2,ox1\u00afa2,o`1\u00a8\u00a8\u00a8xo\u00afa2,2o\n..................\n\u00afx1\u00afan,1\u00a8\u00a8\u00a8 \u00afxo\u00afan,ox1\u00afan,o`1\u00a8\u00a8\u00a8xo\u00afan,2o\ufb01\n\ufb03\ufb03\ufb03\ufb02(34)\n\u201c\u00bb\n\u2014\u2014\u2014\u2013c1c1\u00a8\u00a8\u00a8c1\nc2c2\u00a8\u00a8\u00a8c2\n............\ncncn\u00a8\u00a8\u00a8cn\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02dd\u00bb\n\u2014\u2014\u2014\u2013\u00afx1\u00afa1,1\u00a8\u00a8\u00a8 \u00afxo\u00afa1,ox1\u00afa1,o`1\u00a8\u00a8\u00a8xo\u00afa1,2o\n\u00afx1\u00afa2,1\u00a8\u00a8\u00a8 \u00afxo\u00afa2,ox1\u00afa2,o`1\u00a8\u00a8\u00a8xo\u00afa2,2o\n..................\n\u00afx1\u00afan,1\u00a8\u00a8\u00a8 \u00afxo\u00afan,ox1\u00afan,o`1\u00a8\u00a8\u00a8xo\u00afan,2o\ufb01\n\ufb03\ufb03\ufb03\ufb02(35)\n22"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc31f-934e-11ee-a45e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 22, "chroma:document": "A.2.4 Updating the Memory Matrix\nThe memory matrix Cis updated based the training example px,yqusing the Type I and Type\nII feedback matrices FIa\ni,FIa\ni, andFII\ni. This updating is done per output yiiny, simply by\nsummation:\nC\u02da\nT`1\u201cCT`m\u00ff\ni\u201c1\u00b4\nQII\ni\u02ddFII\ni`QI\ni\u02ddFIa\ni\u00b4QI\ni\u02ddFIb\ni\u00af\n. (36)\nAbove,QI\nimapsRI\nidown to the literals:\nQI\ni\u201c\u00bb\n\u2014\u2013rI\ni...\nrI\ni\ufb01\n\ufb03\ufb02T\n\u201c\u00bb\n\u2014\u2014\u2014\u2013rI\ni,1rI\ni,1\u00a8\u00a8\u00a8rI\ni,1\nrI\ni,2rI\ni,2\u00a8\u00a8\u00a8rI\ni,2............\nrI\ni,nrI\ni,n\u00a8\u00a8\u00a8rI\ni,n\ufb01\n\ufb03\ufb03\ufb03\ufb02. (37)\nCorrespondingly, QII\nideals withRII\ni:\nQII\ni\u201c\u00bb"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc320-934e-11ee-bc66-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 22, "chroma:document": "Correspondingly, QII\nideals withRII\ni:\nQII\ni\u201c\u00bb\n\u2014\u2013rII\ni...\nrII\ni\ufb01\n\ufb03\ufb02T\n\u201c\u00bb\n\u2014\u2014\u2014\u2013rII\ni,1rII\ni,1\u00a8\u00a8\u00a8rII\ni,1\nrII\ni,2rII\ni,2\u00a8\u00a8\u00a8rII\ni,2............\nrII\ni,nrII\ni,n\u00a8\u00a8\u00a8rII\ni,n\ufb01\n\ufb03\ufb03\ufb03\ufb02. (38)\nThe \ufb01nal step is to clip the state values in Cto make sure that they stay within 1 and 2 N:\nCt`1\u201cclip`\nC\u02da\nt`1,1,2N\u02d8\n. (39)\nA.2.5 Learning Weights\nThe perhaps most novel part of CoTM is the fact that the clauses are not initially related to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc321-934e-11ee-abeb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 22, "chroma:document": "any particular output yiiny. By learning how to relate each clause to each output, the CoTM\nopens up for clause sharing. That is, the weight matrix Wis initialized randomly with values\nfromt\u00b41,1u, thus randomly assigning each clause to a truth value for yi. The original TM,\non the other hand, operates with a \ufb01xed weight matrix Wwhere each clause is preassigned to\na particular output. That is, the clause rows are \ufb01rst partitioned into mpartitions of equal"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc322-934e-11ee-8121-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 22, "chroma:document": "size. Half of the clause rows inside a partition is then assigned the weight \u00b41 and the other\nhalf is assigned the weight `1. For other outputs the weight is set to 0, thus, blocking sharing\nof clauses between the outputs.\nThe updating of weights is quite straightforward. Type Ia Feedback changes the weight\nin one-step increments away from 0, while Type II Feedback changes the weight in one-step\ndecrements towards 0. Type Ib Feedback leaves the weights unchanged. This updating of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152dc323-934e-11ee-8b6c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2108.07594.pdf", "page": 22, "chroma:document": "weight matrix WfromWttoWt`1is captured by the following equation:\nWt`1\u201cWt`pRI`RIIq\u02dd\u00a8\n\u02da\u02da\u02da\u02dd\u00bb\n\u2014\u2014\u2014\u2013c1y1c2y1\u00a8\u00a8\u00a8cny1\nc1y2c2y2\u00a8\u00a8\u00a8cny2\n............\nc1ymc2ym\u00a8\u00a8\u00a8cnym\ufb01\n\ufb03\ufb03\ufb03\ufb02\u00b4\u00bb\n\u2014\u2014\u2014\u2013c1\u00afy1c2\u00afy1\u00a8\u00a8\u00a8cn\u00afy1\nc1\u00afy2c2\u00afy2\u00a8\u00a8\u00a8cn\u00afy2\n............\nc1\u00afymc2\u00afym\u00a8\u00a8\u00a8cn\u00afym\ufb01\n\ufb03\ufb03\ufb03\ufb02\u02db\n\u2039\u2039\u2039\u201a.(40)\nNote that the 1-entries of RIandRIIare non-overlapping, singling out which clause rows j\nare eligible for Type Ia and Type II Feedback. The value of yicontrols the direction of the\nweight update.\n23"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1315-934e-11ee-a252-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 0, "chroma:document": "Chapter 1Your First Tsetlin MachineIn this chapter, you build your \ufb01rst Tsetlin machine. Using toy data onvehicles and real-life data on breast cancer, you learn how Tsetlin machinesunify three powerful strategies for pattern recognition:1.First, you explore how to use simple if-then rules to recognize regular-ities in data \u2013 so-called frequent patterns. Frequent patterns captureknowledge like: \u201cmost cars have four wheels\u201d.2.Next, you discover how to distinguish between di\u21b5erent types of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1316-934e-11ee-abfe-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 0, "chroma:document": "how to distinguish between di\u21b5erent types of ob-jects. For instance, both cars and planes may have four wheels. How-ever, planes have wings, while cars do not. You will \ufb01nd out howTsetlin machines learn patterns that are both frequent and able todi\u21b5erentiate, demonstrated on the vehicle- and breast cancer data.3.Finally, you uncover how a Tsetlin machine dissects the data to con-struct multiple rules, making them interact to predict breast cancer.At the end of this chapter, you know how a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1317-934e-11ee-a500-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 0, "chroma:document": "cancer.At the end of this chapter, you know how a standard Tsetlin machine works.You have also learned how to implement one.The rest of the book introduces several variants of the above mecha-nisms, like regression and convolution. After mastering Chapter 1, thosevariations will be easy to pick up because they all build on the standardTsetlin machine.6"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1318-934e-11ee-ade2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 1, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo\nObservationIf-then Rules Section 1.2Boolean Features Section 1.1Rule Coordination Section 1.5-1.6Plane\nRule Learning Section 1.3-1.4Output"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1319-934e-11ee-b8ab-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 1, "chroma:document": "Figure 1.1: Overview of Tsetlin machine and chapter structure.Figure1.1shows the overall structure of a Tsetlin machine, which youexplore in Chapter 1. First, in Section1.1,y o ul e a r nh o wt op r e p a r et h einput to a Tsetlin machine using so-calledBoolean features.T h e n y o udiscover how to describe patterns with if-then rules in Section1.2. Usingthe if-then rules, Section1.3explains how you uncover frequent patterns indata, while Section1.4covers how to make the if-then rules"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131a-934e-11ee-a8aa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 1, "chroma:document": "Section1.4covers how to make the if-then rules discriminative.Finally, you study the complete Tsetlin machine algorithm in Section1.5and Section1.6, investigating how the Tsetlin machine combines multipleif-then rules.1.1 Boolean FeaturesA Tsetlin machine observes objects (or phenomena), which it recognizesbased on the objects\u2019 features. Each feature is an object property that eitherisTrueorFalse,m a k i n gi tBoolean.C o n s i d e r ,f o re x a m p l e ,ad a t a b a s e o nvehicles of di\u21b5erent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131b-934e-11ee-9086-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 1, "chroma:document": "m p l e ,ad a t a b a s e o nvehicles of di\u21b5erent types, like cars, boats, trains, and planes. Vehicles ofthe same kind will naturally have some properties in common. The cars7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131c-934e-11ee-8062-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 2, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmodepicted in Figure1.2,f o ri n s t a n c e ,a l lh a v ef o u rw h e e l s ,l a c kw i n g s ,a n dtransport people. At the same time, they are di\u21b5erent in various ways. Oneis a sedan, the others are a pickup, an SUV, and a van. They also appearin di\u21b5erent colors. All these properties are examples of Boolean features.For instance, a vehicle is either yellow or not yellow.Table1.1lists six example vehicles and their \ufb01ve Boolean"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131d-934e-11ee-8ec1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 2, "chroma:document": "six example vehicles and their \ufb01ve Boolean features:FourWheels,Transports People,Wings,Yellow,a n dBlue.T h e r e i s o n e c o l u m nper feature, each entry taking the valueTrueorFalse,d e n o t e d\u2018\u2022\u2019a n d\u2018\u00b7\u2019,Tsetlin Machine OriginsThe Tsetlin machine is named after Michael Lvovitch Tsetlin (22 Septem-ber 1924 \u2013 30 May 1966), a Soviet mathematician and physicist. Tsetlininvented a learning algorithm that optimizes its behavior online, embeddedin a random environment. Tsetlin\u2019s algorithm is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131e-934e-11ee-8626-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 2, "chroma:document": "a random environment. Tsetlin\u2019s algorithm is particularly fascinatingbecause it keeps track of previous experience only using a single integer.Further, it is adaptive, capable of \u201cchanging its mind\u201d. Based on this al-gorithm, Tsetlin introduced algorithm collectives for solving complex prob-lems. The Tsetlin machine builds upon the work of Tsetlin, dealing withadvanced pattern recognition tasks.Machine Learning BasicsMachine learning deals with how computers can improve from experienceand what"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e131f-934e-11ee-b047-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 2, "chroma:document": "how computers can improve from experienceand what constitutes the fundamental laws of learning. A typical machinelearning system consists of:\u2022Atasksuch as classi\ufb01cation or regression, for instance, photo clas-si\ufb01cation or breast cancer recurrence prediction.\u2022Training datawith individual cases, which is the experience.\u2022Analgorithmthat learnssupervisedfrom labelled training data,unsupervisedfrom unlabelled data, or fromreinforcementsuchas penalties and rewards.\u2022Testing datafor evaluating the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1320-934e-11ee-890e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 2, "chroma:document": "and rewards.\u2022Testing datafor evaluating the performance of the algorithm aftertraining.8"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1321-934e-11ee-b418-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 3, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo#Four Wheels Transports People Wings Yellow BlueCar1.\u2022\u2022 \u00b7 \u00b7 \u2022\u20222.\u2022\u2022 \u00b7 \u2022 \u00b7\u20223.\u2022\u2022 \u00b7 \u2022 \u00b7\u20224.\u2022\u2022 \u2022 \u00b7 \u2022\u00b75.\u2022\u00b7 \u2022 \u2022 \u00b7\u00b76.\u00b7\u2022 \u2022 \u00b7 \u2022\u00b7Table 1.1: A table of three cars and three planes, with \ufb01ve Boolean features.There is one column per feature, each entry taking the valueTrue(\u2022)o rFalse(\u00b7). The \ufb01nal column decides type of vehicle,Car(\u2022)o rPlane(\u00b7).respectively. The last column contains type of vehicle:Car(\u2022)o rPlane(\u00b7).Vehicle #1, for instance, is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1322-934e-11ee-8816-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 3, "chroma:document": "rPlane(\u00b7).Vehicle #1, for instance, is of typeCar,h a sFour Wheels,Transports People,does not haveWings,i sBlue,b u tn o tYellow.S o m eo ft h ev e h i c l ef e a t u r e sare essential for recognition, such asFour Wheels. Others are peripheral,Tsetlin Machines vs Black BoxesRecent research has brought increasingly accurate learning algorithms andpowerful computation platforms. However, the accuracy gains come withescalating computation costs, and models are getting too complicated forhumans to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1323-934e-11ee-9b4e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 3, "chroma:document": "models are getting too complicated forhumans to comprehend. Mounting computation costs make AI an assetfor the few and impact the environment. Simultaneously, the obscurity ofAI-driven decision-making raises ethical concerns. We are risking unfair,erroneous, and, in high-stakes domains, fatal decisions. Tsetlin machinesaddress the following key challenges:\u2022They are universal function approximators, like neural networks.\u2022They are rule-based, like decision trees.\u2022They are summation-based, like"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1324-934e-11ee-a06a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 3, "chroma:document": "decision trees.\u2022They are summation-based, like Naive Bayes classi\ufb01er and logisticregression.\u2022They are hardware-near, with low energy- and memory footprint.As such, the Tsetlin machine is a general-purpose, interpretable, and low-energy machine learning approach.9"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1325-934e-11ee-94e2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 4, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1326-934e-11ee-b776-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 4, "chroma:document": "Figure 1.2: Four di\u21b5erent vehicles - van, pickup, sedan, and SUV.PropertyValuesDescriptionMenopauselt40, ge40, premenoPre- or postmenopausal statusat time of diagnosisInv-nodes0-2, 3-5, 6-8Axillary lymph nodes containingvisible metastatic breast cancerDeg-malig1, 2, 3Degree of tumor malignancyRecurrenceyes, noPatient recurrence statusTable 1.2: Four properties and their values from the breast cancer"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1327-934e-11ee-bbc6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 4, "chroma:document": "and their values from the breast cancer data.#MenopauseInv-nodesDeg-maligRecurrence1.ge403-53yes2.lt400-23no3.ge406-83yes4.ge400-22no5.premeno0-23yes6.premeno0-21noTable 1.3: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.such asBlue. Later, you discover how a Tsetlin machine distinguishesessential features from more peripheral ones.10"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1328-934e-11ee-94d5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 5, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMenopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 3Recurrence1.\u00b7\u2022 \u00b7\u00b7\u2022\u00b7\u00b7\u00b7 \u2022\u20222.\u2022\u00b7 \u00b7\u2022\u00b7\u00b7\u00b7\u00b7 \u2022\u00b73.\u00b7\u2022 \u00b7\u00b7\u00b7\u2022\u00b7\u00b7 \u2022\u20224.\u00b7\u2022 \u00b7\u2022\u00b7\u00b7\u00b7\u2022 \u00b7\u00b75.\u00b7\u00b7 \u2022\u2022\u00b7\u00b7\u00b7\u00b7 \u2022\u20226.\u00b7\u00b7 \u2022\u2022\u00b7\u00b7\u2022\u00b7 \u00b7\u00b7Table 1.4: A Booleanized version of Table1.3with one Boolean feature pervalue ofMenopause,Inv-nodes,Deg-malig,a n dRecurrence.E a c h c o l u m nis a Boolean feature, each entry taking the value True (\u2022)o rF a l s e(\u00b7).Breast Cancer Recurrence.In your \ufb01rst task in this book,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1329-934e-11ee-9085-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 5, "chroma:document": "Cancer Recurrence.In your \ufb01rst task in this book, you predictbreast cancer recurrence using a real-life dataset. The dataset containsinformation on breast cancer patients \ufb01ve years after surgery.1You willfocus on six patients and four (out of nine) multi-valued properties, whichis enough data to lay out how a Tsetlin machine operates.The four properties are:Menopause,Inv-nodes,Deg-malig,a n dRecur-rence.Y o u \ufb01 n d a d e s c r i p t i o n o f t h e s e p r o p e r t i e s a n d t h e v a l u e"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132a-934e-11ee-a6d9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 5, "chroma:document": "h e s e p r o p e r t i e s a n d t h e v a l u e s t h e y t a k e i nTable1.2.F o r i n s t a n c e ,Menopausetakes the valueslt40,ge40,o rpremeno,describing menopausal status at the time of diagnosis.Table1.3contains the six selected patients, our objects of study. The\ufb01rst object, marked #1, is a patient withRecurrence.Y o uo b t a i nt h ec l a s sfrom the last column, whereRecurrenceisyesfor patient #1. Moreover,this patient hasMenopausetypege40,Inv-nodesvalue3-5,a n dDeg-maligvalue3.B y b"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132b-934e-11ee-965c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 5, "chroma:document": "n dDeg-maligvalue3.B y b e i n g m u l t i - v a l u e d , t h e p a t i e n t p r o p e r t i e s a r e n o t y e t r e a d yfor use by the Tsetlin machine and need to be transformed by Booleanizingthe data.Data Booleanization.The \ufb01rst step of building a Tsetlin machine isto Booleanize your data by making suitable Boolean features. The breastcancer data demonstrates the starting point. You start by inspecting the1This breast cancer dataset was obtained from the University Medical Centre,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132c-934e-11ee-a122-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 5, "chroma:document": "was obtained from the University Medical Centre, In-stitute of Oncology, Ljubljana, Slovenia. Thanks go to M. Zwitter and M. Soklic forproviding the data, found athttps://archive.ics.uci.edu/ml/datasets/Breast+Cancer.11"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132d-934e-11ee-9a08-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 6, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmovalues in Table1.2.F o re a c hp r o p e r t y ,y o uc r e a t eo n eB o o l e a nf e a t u r ep e rvalue.Menopause,f o ri n s t a n c e ,g i v e st h eB o o l e a nf e a t u r e slt40,ge40,a n dpremeno. After Booleanization, you obtain Table1.4,w h i c he n c o m p a s s e snine Boolean features. You get nine features because the three originalproperties each takes three values. The table provides the same informationon each"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132e-934e-11ee-a3de-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 6, "chroma:document": "The table provides the same informationon each patient as the information in Table1.3.T h e d i \u21b5 e r e n c e i s t h a tTable1.4only contains values that are eitherTrueorFalse. Notice howtheRecurrencecases all have aDeg-maligvalue of3.Y o ue x p l o i tt h i sa n dother data regularities later in this chapter.Advantages of Boolean FeaturesUsing Boolean features has several advantages:\u2022Boolean features \ufb01t well with a computer because you can storethem individually as bits.Truebecomes1,a n"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e132f-934e-11ee-be02-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 6, "chroma:document": "storethem individually as bits.Truebecomes1,a n dFalsebecomes0.Hardware-near bitwise operators then dramatically increase infer-ence speed. Further, the resulting small energy- and memory foot-print is ideal for edge computing and Internet of Things (IoT).\u2022You can use Boolean algebra \u2013 the language of computers \u2013 to buildpatterns. If you know a programming language, you have probablyused Boolean algebra. You write Boolean expressions to control the\ufb02ow of a program, for example, with if-then"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1330-934e-11ee-985b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 6, "chroma:document": "the\ufb02ow of a program, for example, with if-then constructs.\u2022Boolean algebra is the same as propositional logic, which models log-ical reasoning withTrueandFalsestatements. Such logical reason-ing is easy for humans to understand, making the Tsetlin machinetransparent.I n c o m p a r i s o n , s o - c a l l e dblack boxmachine learningmodels are too complicated for human comprehension."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1332-934e-11ee-96ee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 7, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo1.2 Creating Patterns with AND and NOTThe Tsetlin machine solves pattern recognition problems by buildingif-thenrulesfrom object observations. Each rule has the formifconditionthenclass.Theconditionpart is a placeholder for a Boolean expression that describesap a t t e r ni nt h ed a t a ,t ob el e a r n tb yt h eT s e t l i nm a c h i n e . L e tu sn o wreturn to our vehicle data from Table1.1. As you can see from the table,the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1333-934e-11ee-80eb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 7, "chroma:document": "from Table1.1. As you can see from the table,the conditionFour WheelsandTransports Peoplecharacterises the three cars (object #1, #2, and #3) and one of the planes(object #4).Notice how a Tsetlin machine uses theand-operator to combine severalfeatures. Usingandmeans that all of the features must beTruefor theoverall condition to beTrue.I n t h i s i n s t a n c e , t h e r u l ematchesthe prop-erties of the observed object. On the other hand, if one or more of thefeatures areFalse,t h eo v e r"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1334-934e-11ee-ba22-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 7, "chroma:document": "one or more of thefeatures areFalse,t h eo v e r a l lc o n d i t i o nb e c o m e sFalsebecause the rule nolonger matches the object\u2019s properties.Theclasspart is a placeholder for a type of object, such asCar.T h i s i sthe type of object that the rule predicts when the rule\u2019s condition matchesthe object\u2019s properties. For example, the ruleifFour WheelsandTransports PeoplethenCarpredictsCarwhen it sees an object withFour WheelsthatTransportsPeople.Negation.The Tsetlin machine also uses"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1335-934e-11ee-8608-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 7, "chroma:document": "Tsetlin machine also uses thenot-operator to tell whichfeatures a classdoes nothave. One says thatnotnegatesthe feature. Forinstance, a car does not haveWings.Y o u c a n i n c l u d e t h i s f a c t i n t h e r u l efor recognizing cars by usingnot:ifFour WheelsandTransports Peopleand notWingsthenCar.The negated featurenotWingsmakes the rule able to distinguish betweenclassCarandPlane.C o r r e s p o n d i n g l y ,c l a s sPlanegets its own rule:ifWingsthenPlane.13"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1336-934e-11ee-9084-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 8, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoLiterals.Taken together, features and negated features are calledliterals,as customary in Boolean algebra. The three literals of ourCarexamplerule are:Four Wheels,Transports People,a n dnotWings. Using the wordliteralis convenient because Tsetlin machines treat features and negatedfeatures the same way.Remark.Note that a Tsetlin machine only uses the above kind ofand-expressions. Using pureand-expressions makes it easier to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1337-934e-11ee-a1e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 8, "chroma:document": "Using pureand-expressions makes it easier to understand therules that the Tsetlin machine builds, increasing transparency.1.3 Learning Frequent Patterns withRecognize and Erase FeedbackAT s e t l i nm a c h i n el e a r n sb yo b s e r v i n ge x a m p l e so fo b j e c t so fd i \u21b5 e r e n tt y p e s .It also gets to know the type of each object. It sees a plane, then a car, andso on. However, the Tsetlin machine perceives the object\u2019s features, notthe object itself. For example, it observes"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1338-934e-11ee-a6d8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 8, "chroma:document": "notthe object itself. For example, it observes aBluevehicle of typeCarthatTransports People,h a sFour Wheelsand does not haveWings.T h et a s ko fthe Tsetlin machine is then to learn to recognize and distinguish betweenthe di\u21b5erent types of objects. Learning from such labeled examples goesunder the namesupervised learning.AT s e t l i nm a c h i n el e a r n sb ym e m o r i z i n gf e a t u r e s . T h el e a r n i n gi si n -spired by how humans remember. If you observe something multiple"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1339-934e-11ee-983f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 8, "chroma:document": "remember. If you observe something multiple times,you remember it better. However, without observations, forgetting sets inafter some time. Also, general properties tend to stick while \ufb01ne detailsdisappear. You remember that your neighbor\u2019s car is blue and has fourwheels but may forget the kind of tire. After some time, you may forgetthe color as well. It appears as if everything memorized is in some state offorgetting.Rule MemoryIn traditional computer memory, everything sticks."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133a-934e-11ee-a8fc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 8, "chroma:document": "traditional computer memory, everything sticks. Figure1.3visualizesmemory that stores rule14"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133b-934e-11ee-b0e6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 9, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoIn MemoryNot in MemoryFigure 1.3: Traditional memory for the rule:ifFour WheelsandTransports PeopleandBluethenCar.MaximallyMemorized"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133c-934e-11ee-917a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 9, "chroma:document": "MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.4: Example of Tsetlin machine memory for the rule:ifFour WheelsandTransports PeopleandBluethenCar.ifFour WheelsandTransports PeopleandBluethenCarin the traditional way. The literalsFour Wheels,Transports People,a n dBluestay in memory until you delete them. Notice that the example rulefor predictingCarincludes the literalBluein its condition. If aCarofanother color appears, the rule will not recognize the object asCar.Y o"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133d-934e-11ee-82ed-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 9, "chroma:document": "the rule will not recognize the object asCar.Y o uwill discover how Tsetlin machines \ufb01x too specialized rules below.A Tsetlin machine simulates forgetting and memorization. Figure1.4shows the Tsetlin machine memory of our example rule. The y-axis rangesfrom 1 to 10 and measures how deeply the memory stores each literal:\u2022Values 1 to 5 stand forForgotten.V a l u e 1 m e a n s m a x i m a l l y f o r g o t t e nwhile value 5 means almost memorized. Literals in this rangedo nottake part in the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133e-934e-11ee-8b0e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 9, "chroma:document": "Literals in this rangedo nottake part in the rule\u2019s condition.15"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e133f-934e-11ee-a7b8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 10, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo\u2022Values 6 to 10 meanMemorized.V a l u e 6 s t a n d s f o r l i g h t l y r e t a i n e d ,and value 10 represents maximally memorized. Literals in this rangetake part in the condition of the rule.In Figure1.4,Four WheelsandTransports Peopleare deeply memorized,whileBlueis lightly memorized. The other literals are forgotten to varyingdegrees, measured by their position in memory (from 1 to 5).Remark.Observe how the negated and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1340-934e-11ee-9c4c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 10, "chroma:document": "(from 1 to 5).Remark.Observe how the negated and non-negated versions of each fea-ture co-exist in memory. Both versions are candidates for being memo-rized, however, memorizing both gives a contradiction. A vehicle cannotboth haveFour WheelsandnothaveFour Wheelsat the same time. Thatis, ifFour WheelsisTruethennotFour WheelsbecomesFalse,a n dv i c eversa. Accordingly, the conditionFour Wheelsand notFour WheelsisalwaysFalse. As explored below, Tsetlin machines avoid contradictions byforgetting"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1341-934e-11ee-892e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 10, "chroma:document": "machines avoid contradictions byforgetting literal combinations that evaluate toFalse.Learning of a Single RuleThe key to understanding how a complete Tsetlin machine learns is to studyhow a single if-then rule learns by itself. By learning independently, eachrule becomes more self-contained and simpler to understand. As a sidebene\ufb01t, independent learning also accommodates parallel processing. Laterin this chapter, you will see how multiple rules coordinate indirectly withoutbeing aware of each"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1342-934e-11ee-ae14-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 10, "chroma:document": "coordinate indirectly withoutbeing aware of each other, providing a simple and e\u0000cient algorithm fordissecting the complete dataset.Rule Initialization.Ar u l es t a r t su pw i t ha l lt h el i t e r a l si nm e m o r yposition 5. That is, the literals are about to beMemorizedbut currentlyForgotten. Figure1.5shows the initial memory for your example rule. Notethat you can initialize the rule any way you like. The initialization doesnot a\u21b5ect the end result of learning because the Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1343-934e-11ee-a9ca-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 10, "chroma:document": "the end result of learning because the Tsetlin machine is self-correcting. This is in contrast to so-called neural network learning, whichis more sensitive to initialisation.Algorithm \u2013 Learning Steps for Single Rule.Let us \ufb01rst look atwhat happens when a rule faces an object of its own class. That would16"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1344-934e-11ee-9142-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 11, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1345-934e-11ee-993d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 11, "chroma:document": "MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.5: Initialization of example rule for predicting classCar.be the case when our example rule for recognizingCarobserves a vehiclelabeled asCar.L e a r n i n gt h e np r o c e e d si nt h r e es t e p s :1.Rule Evaluation.Evaluate the condition part of the rule by assess-ing the object\u2019s literals.2.Recognize Feedback.Perform this step if the condition part isTrue.a)Memorize the literals that areTruefor the object. You"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1346-934e-11ee-9101-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 11, "chroma:document": "the literals that areTruefor the object. You memorizeal i t e r a lb yi n c r e m e n t i n gi t sp o s i t i o ni nm e m o r yu n l e s sa l r e a d yMaximally Memorizedin position 10.b)Forget theFalseliterals by pushing them towards beingMaxi-mally Forgotten.Y o u f o r g e t a l i t e r a l b y d e c r e m e n t i n g i t s p o s i t i o nin memory unless alreadyMaximally Forgottenin position 1.The name of this kind of feedback is Recognize Feedback because itmakes the condition part of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1347-934e-11ee-a77b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 11, "chroma:document": "because itmakes the condition part of the rule mimic the observed object.3.Erase Feedback.Perform this step if the condition part isFalse.Then forget all of the literals, independently of their truth values.Again, you forget literals by pushing them towards beingMaximally17"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1348-934e-11ee-b662-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoForgotten. The name of this type of feedback is Erase Feedback be-cause it erases the condition part of the rule.2Randomization.Because coincidences happen and events sometimes oc-cur by chance, learning should be \ufb02exible. A simple way to achieve such\ufb02exibility is randomization. Then no coincidence or chance event can makethe learning get stuck. The Tsetlin machine, therefore, randomizes theincrements and decrements. Before"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1349-934e-11ee-a72b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "randomizes theincrements and decrements. Before performing an increment, you draw arandom \ufb02oating-point value between 0.0a n d1.0. Skip the increment if thevalue is above 0.5. The value 0.5g e t st h en a m eMemorize Value.Y o ua l s orandomize the decrements. Again, you draw a random \ufb02oating-point valuebefore decrementing. A value above 0.5l e a v e so u tt h ed e c r e m e n t . T h i ss e c -ond 0.5v a l u ei sc o i n e dForget Value. Note that the randomized updatingfurther diversi\ufb01es the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134a-934e-11ee-a1a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "the randomized updatingfurther diversi\ufb01es the rules and boosts exploration.Memorization and Forgetting Speed.The Memorize Value decideshow quickly the rule memorizes literals. Conversely, the Forget Value de-cides how quickly the rule forgets in the absence of observations. If youincrease the Memorize Value and reduce the Forget Value, the rule willremember literals longer. Remembering literals longer allows the rule toretain more details. In the extreme, a Memorize Value of 1.0a n daF o r g e"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134b-934e-11ee-a080-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "extreme, a Memorize Value of 1.0a n daF o r g e tValue of 0.0m a k et h er u l em e m o r i z eevery Trueliteral of one single object.Conversely, Memorize Value 0.0a n dF o r g e tV a l u e1.0r e n d e rt h er u l ei n c a -pable of memorizing anything at all. It is usual to keep the relationship:Forget Value + Memorize Value = 1.0. The reason is that you then canobtain one from the other, and therefore you only need to specify one ofthem. Alternatively, you can \ufb01x the Memorize Value to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134c-934e-11ee-ae0f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "Alternatively, you can \ufb01x the Memorize Value to 1.0.ExamplesUpdating of Memory \u2013 Example 1.The \ufb01rst example traverses thedata from Table1.1, using Memorize Value 0.1a n dF o r g e tV a l u e0.9. Con-sider a rule for predictingCar. After initialization, the rule \ufb01rst faces vehi-cle #1 in the table. The vehicle is of typeCar,h e n c ee i t h e rR e c o g n i z eo r2In the research literature, Recognize and Erase Feedback are typically called TypeIa and Type Ib Feedback. In this book, I use the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134d-934e-11ee-95cb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 12, "chroma:document": "and Type Ib Feedback. In this book, I use the former names to make it easier toremember the roles of the feedback.18"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134e-934e-11ee-acc1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 13, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized\nMaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Memorize Value 0.1Forget Value 0.9Skipped Memorize"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e134f-934e-11ee-88b1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 13, "chroma:document": "Figure 1.6: Memory updating for newly initialized rule when facing aBluecar thatTransports Peopleand doesnothaveWings.Erase Feedback applies. It has \ufb01veTrueliterals:Four Wheels,TransportsPeople,notWings,notYellow, andBlue.T h e r e m a i n i n g l i t e r a l s a r eFalse.Learning proceeds as follows:\u2022All the literals of the rule startForgotten, as shown in Figure1.5.S o ,the rule\u2019s condition is empty. An empty condition does not specifyany literal requirements so it is alwaysTrue. Accordingly,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1350-934e-11ee-b48f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 13, "chroma:document": "requirements so it is alwaysTrue. Accordingly, you proceedwith Recognize Feedback.\u2022The solid arrows in Figure1.6visualize how Recognize Feedback in-crements theTrueliterals randomly. Because of the small Mem-orize Value of 0.1, only a few increments will happen, hence theshort arrows. Recall how each increment is skipped whenever a ran-dom \ufb02oating-point value between 0.0a n d1.0s u r p a s s e st h eM e m o r i z eValue. For the sake of the example, execute the increments"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e1351-934e-11ee-857e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 13, "chroma:document": "the sake of the example, execute the increments forFourWheels,Transports People,a n dBlue.T h e nnotWingsandnotYellowstay in place, signi\ufb01ed by the greyed-out arrows in the \ufb01gure.\u2022Conversely, the dotted arrows in the \ufb01gure visualize the random decre-ments for theFalseliterals. These arrows are visualized as beinglonger because of the higher Forget Value of 0.9. Again, you drawa random \ufb02oating-point value per pinpointed decrement. However,19"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a1-934e-11ee-800c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 14, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized\nMaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Forget Value 0.9Skipped Forget"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a2-934e-11ee-9265-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 14, "chroma:document": "Figure 1.7: Memory updating for rule \u2018ifFour WheelsandTransports PeopleandBluethenCar\u2019w h e nf a c i n gaYellowcar.now a value equal to or below 0.9e x e c u t e st h ed e c r e m e n t . F o rt h i sparticular example, assume that all of the decrements happen.The updated rule then becomes:ifFour WheelsandTransports PeopleandBluethenCar.Notice how the rule now is more similar to the object it observed.Updating of Memory \u2013 Example 2.The updated rule then faces ve-hicle #2 in Table1.1\u2013aYellowcar."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a3-934e-11ee-9f8b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 14, "chroma:document": "then faces ve-hicle #2 in Table1.1\u2013aYellowcar. Figure1.7shows how to update therule. This time, the rule\u2019s condition does not match the car\u2019s literals be-cause of the mismatching color. Accordingly, you use Erase Feedback. Asvisualized, Erase Feedback randomly decrements the memory position of allof the literals. Execute these decrements, except forFour WheelsandnotWingsto simulate the randomization (indicated by the greyed out arrows).The new rule becomes:ifFour WheelsthenCar.20"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a4-934e-11ee-abdd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 15, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized\nMaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Memorize Value 0.1Forget Value 0.9Skipped Memorize"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a5-934e-11ee-884e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 15, "chroma:document": "Figure 1.8: Memory updating for rule \u2018ifFour WheelsthenCar\u2019w h e nfacing aYellowcar.Updating of Memory \u2013 Example 3.The rule now faces vehicle #3,which also happens to beYellow.T h i s t i m e t h e r u l e \u2019 s c o n d i t i o n m a t c h e s t h evehicle\u2019s features becauseBlueis no longer included. Figure1.8depicts howthe resulting Recognize Feedback acts on the literals. First, it incrementstheTrueliteralsFour Wheels,Transports People,notWings,Yellow,a n dnotBlue. Again, the increments are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a6-934e-11ee-b00f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 15, "chroma:document": "n dnotBlue. Again, the increments are random, based on the Memorize Value.To simulate randomization, execute only the increments forFour Wheels,Transports People,a n dYellow.T h e \ufb01 g u r e a l s o s h o w s t h e d e c r e m e n t i n gof theFalseliterals:notFour Wheels,notTransports People,Wings,notYellow,a n dBlue.T o s i m u l a t e r a n d o m i z a t i o n w i t h F o r g e t V a l u e 0.9,perform all of these decrements. The resulting rule after observing vehicle#3 becomes:ifFour"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a7-934e-11ee-a37c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 15, "chroma:document": "rule after observing vehicle#3 becomes:ifFour WheelsandTransports PeoplethenCar.Figure1.9shows the memory of the updated rule. Observe how the rulenow memorizesFour WheelsandTransports Peoplemore strongly, whilegetting ready to memorizenotWings. Also observe how the color literalsYellow,notYellow,Blue,a n dnotBlueare gradually forgotten. Varyingfrom car to car, the frequency of each color is too low to be memorized withMemorize Value 0.1a n dF o r g e tV a l u e0.9. Finally, notice how the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a8-934e-11ee-94e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 15, "chroma:document": "dF o r g e tV a l u e0.9. Finally, notice how the literals21"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36a9-934e-11ee-97f3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 16, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36aa-934e-11ee-a9f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 16, "chroma:document": "MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.9: Memory for rule \u2018ifFour WheelsandTransports PeoplethenCar\u2019a f t e ru p d a t i n g .Menopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 31.\u00b7\u2022 \u00b7\u00b7\u2022\u00b7\u00b7\u00b7 \u20223.\u00b7\u2022 \u00b7\u00b7\u00b7\u2022\u00b7\u00b7 \u20225.\u00b7\u00b7 \u2022\u2022\u00b7\u00b7\u00b7\u00b7 \u2022Table 1.5: ThreeRecurrencepatients with nine Boolean features. Eachentry takes the valueTrue(\u2022)o rFalse(\u00b7).Menopause Inv-nodes Deg-malig#\u00aclt40\u00acge40\u00acpremeno\u00ac0-2\u00ac3-5\u00ac6-8\u00ac1\u00ac2\u00ac31.\u2022\u00b7 \u2022\u2022\u00b7\u2022\u2022\u2022\u00b73.\u2022\u00b7 \u2022\u2022\u2022\u00b7\u2022\u2022\u00b75.\u2022\u2022 \u00b7\u00b7\u2022\u2022\u2022\u2022\u00b7Table 1.6:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ab-934e-11ee-b91e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 16, "chroma:document": "\u2022\u2022\u00b7\u2022\u2022\u2022\u00b73.\u2022\u00b7 \u2022\u2022\u2022\u00b7\u2022\u2022\u00b75.\u2022\u2022 \u00b7\u00b7\u2022\u2022\u2022\u2022\u00b7Table 1.6: ThreeRecurrencepatients with nine negated Boolean features,signi\ufb01ed with \u2018\u00ac\u2019. Each entry takes the valueTrue(\u2022)o rFalse(\u00b7).notFour Wheels,notTransports People,a n dWingsare quickly forgotten,not representing any car observed thus far.High-Frequency Breast Cancer Pattern.Let us return to the prog-nosis of breast cancer recurrence, examining the e\u21b5ects of Memorize Value22"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ac-934e-11ee-ac06-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 17, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo0.2a n dF o r g e tV a l u e0.8. Consider a rule for predictingRecurrence.T h erule learns its condition by traversing theRecurrencepatients in the data.There is no need for you to consider theNon-Recurrencepatients at thispoint because they do not in\ufb02uence Recognize and Erase Feedback. Table1.5compiles the features of theRecurrencepatients for ease of reference.The Tsetlin machine also uses the negated features to make rules."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ad-934e-11ee-b9e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 17, "chroma:document": "also uses the negated features to make rules. Table1.6contains the truth values of these. You get the complete set of literalsthat the Tsetlin machine uses by combining the two tables.With Memorize Value 0.2 and Forget Value 0.8,Trueliterals that matchall threeRecurrencepatients increment 3\u21e50.2=0.6t i m e so na v e r a g e .They never decrement because everyRecurrencepatient matches. In con-trast, consider the literals that areFalsefor one of the patients, such asMenopause ge40. On average,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ae-934e-11ee-b894-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 17, "chroma:document": "the patients, such asMenopause ge40. On average, this literal increments 2\u21e50.2=0.4t i m e s ,while it decrements 1\u21e50.8=0.8t i m e s . S o ,t h el i t e r a ld e c r e m e n t sm o r ethan it increments, moving towards beingMaximally Forgotten.T h e o n l ypossible rule then becomes:ifDeg-malig 3and notMenopause lt40and notDeg-malig 1and notDeg-malig 2thenRecurrence.Note that sinceDeg-malig 3impliesnotDeg-malig 1andnotDeg-malig 2,the rule simpli\ufb01es to:ifDeg-malig 3and notMenopause"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36af-934e-11ee-927d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 17, "chroma:document": "rule simpli\ufb01es to:ifDeg-malig 3and notMenopause lt40thenRecurrence.Mid-Frequency Breast Cancer Pattern.You capture less frequentpatterns by diminishing forgetting and boosting memorization. For in-stance, let us see how Memorize Value 0.5a n dF o r g e tV a l u e0.5a \u21b5 e c tl e a r n -ing ofMenopause ge40.O n a v e r a g e , t h i s l i t e r a l n o w i n c r e m e n t s 2\u21e50.5=1.0times and decrements 1\u21e50.5=0.5 times. Accordingly, the new Memorizeand Forget Values encourage rules that match two"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b0-934e-11ee-b69b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 17, "chroma:document": "Forget Values encourage rules that match two out of treeRecurrencepatients. Then you, for instance, could get the ruleifDeg-malig 3andMenopause ge40and notMenopause lt40and notMenopause premenoand notDeg-malig 1and notDeg-malig 2then23"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b1-934e-11ee-bef0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 18, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoRecurrence,which simpli\ufb01es toifDeg-malig 3andMenopause ge40thenRecurrence.You could also get another rule than the one above when other regularitiesoccur two out of three times. However, which one you get does not mat-ter. You will later use multiple rules that together capture many di\u21b5erentpatterns in the data.1.4 Increasing Discrimination Power withReject FeedbackFrequent patterns that describe what is typical can be powerful"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b2-934e-11ee-a940-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 18, "chroma:document": "that describe what is typical can be powerful in their ownright. They express how the world appears, for instance, that the worldcontains vehicles that have four wheels and transport people. However,sometimes you want to classify objects into prede\ufb01ned classes such asCarandPlane.T h e n i t i s c r u c i a l t o m a k e s u r e t h a t t h e f r e q u e n t p a t t e r n sdistinguish between them. For instance, the condition of ruleifFour WheelsandTransports PeoplethenCarcharacterizes both cars"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b3-934e-11ee-9692-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 18, "chroma:document": "PeoplethenCarcharacterizes both cars and planes, so it cannot see the di\u21b5erence betweenthe two.You make a rule more discriminative with the following third and \ufb01nalfeedback type.Algorithm \u2013 Increasing Discrimination Power.Ar u l ei n c r e a s e si t sdiscrimination power when it faces an object of a class di\u21b5erent from itsown. That would be the case when our example rule for recognizingCarobserves a vehicle labeledPlane.L e a r n i n gt h e ns k i p sR e c o g n i z ea n dE r a s eFeedback,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b4-934e-11ee-8e4b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 18, "chroma:document": "k i p sR e c o g n i z ea n dE r a s eFeedback, going directly to the fourth step:24"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b5-934e-11ee-8b4e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 19, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized\nMaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Memorize Value 1.0"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b6-934e-11ee-b768-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 19, "chroma:document": "Figure 1.10: Memory updating for rule \u2018ifFour WheelsandTransports PeoplethenCar\u2019w h e nf a c i n gaBlueplane withFour WheelsthatTransports People.4.Reject Feedback.Check if the condition part of the rule isTrueby assessing the object\u2019s literals. If the condition part isTrue,t h e nmemorize allForgottenliterals that areFalsefor the object. Again,you memorize a literal by incrementing its position in memory. How-ever, this time there is no randomization \u2013 the increment is alwaysperformed. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b7-934e-11ee-8ca4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 19, "chroma:document": "\u2013 the increment is alwaysperformed. In e\u21b5ect, the memorization pushes the literals that arein memory position 1 to 5, and at the same time areFalse,t o w a r d sbeingMemorized.The above learning step has the name Reject Feedback because it makesthe rule reject the observed object by memorizing theFalseliterals.Example with Vehicle Data.Figure1.10shows how our example rulefor predictingCarupdates when observing vehicle #4 in Table1.1\u2013aPlanewithFour WheelsthatTransports People,i"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b8-934e-11ee-9aab-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 19, "chroma:document": "WheelsthatTransports People,i sBlue(notYellow),and hasWings.L o o k a t h o w a l l t h e l i t e r a l s t h a t a r eFalseand on theForgottenside all move towards beingMemorized.T h e s e a r e :notFourWheels,notTransports People,notWings,Yellow,a n dnotBlue.I nparticular,notWingsgoes fromForgottentoMemorizedby switching frommemory position 5 to memory position 6. The other literals are allTrue,25"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36b9-934e-11ee-9885-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 20, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMaximallyMemorized"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ba-934e-11ee-907d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 20, "chroma:document": "MaximallyForgottenMemorizedForgotten10  9  8  7  6  5  4  3  2  1Figure 1.11: Memory for rule \u2018ifFour WheelsandTransports PeopleandnotWingsthenCar\u2019a f t e ru p d a t i n g .Menopause Inv-nodes Deg-malig#lt40 ge40 premeno0-2 3-5 6-812 32.\u2022\u00b7 \u00b7\u2022\u00b7\u00b7\u00b7\u00b7 \u20224.\u00b7\u2022 \u00b7\u2022\u00b7\u00b7\u00b7\u2022 \u00b76.\u00b7\u00b7 \u2022\u2022\u00b7\u00b7\u2022\u00b7 \u00b7Table 1.7: ThreeNon-Recurrencepatients with nine Boolean features. Eachentry takes the valueTrue(\u2022)o rFalse(\u00b7).hence Reject Feedback leaves them alone. Figure1.11depicts the updatedmemory. The rule then becomesifFour"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36bb-934e-11ee-96f3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 20, "chroma:document": "the updatedmemory. The rule then becomesifFour WheelsandTransports Peopleand notWingsthenCar.At this point, the condition isFalsefor the observed plane because thecondition now contains the literalnotWings.T h en e x tt i m et h er u l ef a c e sap l a n e ,i tn ol o n g e rn e e d st ou p d a t eb e c a u s ei t sc o n d i t i o nw i l lt h e nb eFalse.I tc a nn o wd i s t i n g u i s hb e t w e e nCarandPlane.Example with Breast Cancer Data.Consider a rule that votes forRecurrenceif the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36bc-934e-11ee-abe1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 20, "chroma:document": "a rule that votes forRecurrenceif the literalDeg-malig 3isTrue:26"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36bd-934e-11ee-ac5e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 21, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoMenopause Inv-nodes Deg-malig#\u00aclt40\u00acge40\u00acpremeno\u00ac0-2\u00ac3-5\u00ac6-8\u00ac1\u00ac2\u00ac32.\u00b7\u2022 \u2022\u00b7\u2022\u2022\u2022\u2022\u00b74.\u2022\u00b7 \u2022\u00b7\u2022\u2022\u2022\u00b7\u20226.\u2022\u2022 \u00b7\u00b7\u2022\u2022\u00b7\u2022\u2022Table 1.8: ThreeNon-Recurrencepatients with nine negated Boolean fea-tures, signi\ufb01ed with \u2018\u00ac\u2019. Each entry takes the valueTrue(\u2022)o rFalse(\u00b7).ifDeg-malig 3thenRecurrence.Now, take a look at the literals of theNon-Recurrencepatients in Table1.7and Table1.8.Deg-malig 3isTruefor patient #2, despite being aNon-Recurrencepatient. When"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36be-934e-11ee-bb89-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 21, "chroma:document": "#2, despite being aNon-Recurrencepatient. When the above rule faces patient #2, it thus votesfor the wrong class, i.e.,Recurrence.R e j e c tF e e d b a c kc o r r e c t st h i se r r o rb ymemorizing theForgottenliterals that areFalse.F o r p a t i e n t # 2 ,t h e s e a r e :notMenopause lt40,Menopause ge40,Menopause premeno,notInv-nodes 0-2,Inv-nodes 3-5,Inv-nodes 6-8,Deg-malig 1,Deg-malig 2,a n dnotDeg-malig 3.Facing patient #2 several times, Reject Feedback eventually pushes one"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36bf-934e-11ee-84df-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 21, "chroma:document": "times, Reject Feedback eventually pushes one oftheForgotten Falseliterals to memory position 6, making itMemorized.From all of the literals listed above, Recognize and Erase Feedback makesure that the rule eventually memorizes literalnotMenopause lt40.T h eresulting rule becomesifDeg-malign 3and notMenopause lt40thenRecurrence.From this point, Reject Feedback no longer applies, and Recognize andErase Feedback operate alone.27"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c0-934e-11ee-ae58-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 22, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo\nObservationRule MatchingVotes\nFeaturesMajority VoteOutputPlane"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c1-934e-11ee-a1e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 22, "chroma:document": "Figure 1.12: A majority vote among the rules decides the output. Here,Planeobtains more votes thanCar(1 vs. 0 votes) and is thus output bythe Tsetlin machine.1.5 Putting the Pieces Together \u2013 HowSeveral Rules CoordinateTsetlin machines use multiple rules that interact. You will now explore howthis interaction takes place by dissecting the toy data on vehicles and thereal-life data on breast cancer.Classi\ufb01cation Procedure.The Tsetlin machine classi\ufb01es input by vot-ing. That is, a single rule"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c2-934e-11ee-8515-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 22, "chroma:document": "input by vot-ing. That is, a single rule does not get the \ufb01nal say on the class by itself.Instead, it casts a vote for its class. To classify, you simply count how manyrules vote for each class. The Tsetlin machine then outputs the class withthe most votes.In other words, a majority vote among the rules decidesthe output.28"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c3-934e-11ee-b520-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 23, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo#Menop.Inv-nodesDeg-maligRecur.R1R2R3v1.ge403-53yes\u2022\u2022\u00b7+22.lt400-23no\u00b7\u00b7\u2022\u000013.ge406-83yes\u2022\u2022\u00b7+24.ge400-22no\u00b7\u00b7\u2022\u000015.premeno0-23yes\u2022\u2022\u2022+16.premeno0-21no\u00b7\u00b7\u2022\u00001Table 1.9: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.C o l u m n sR1,R2,a n dR3list the truthvalue of each rule while columnvcontains the resulting vote sums.Demonstration of Vehicle Classi\ufb01cation.Figure1.12illustrates theclassi\ufb01cation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c4-934e-11ee-877e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 23, "chroma:document": "theclassi\ufb01cation of vehicle #4 in Table1.1\u2013aBlue PlanethatTransportsPeopleand hasFour WheelsandWings.T h e d e p i c t e d T s e t l i n m a c h i n eemploys two rules:R1ifFour WheelsandTransports Peopleand notWingsthenCar.R2ifWingsthenPlane.Rule R1 does not match the vehicle because it speci\ufb01esnotWings. Hence,Cargets no votes. Rule R2, on the other hand, matches. So,Planegets avote and wins the majority vote. The Tsetlin machine then outputsPlane.Demonstration of Breast Cancer Recurrence"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c5-934e-11ee-a175-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 23, "chroma:document": "of Breast Cancer Recurrence Prediction.Recallthe breast cancer dataset from Table1.4. Here are three rules that demon-strate the di\u21b5erent ways rules can cooperate:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3and notMenopause lt40thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.Notice that rules R1 and R2 for predictingRecurrenceare duplicates. Byvoting together, R1 and R2 can win majority votes that otherwise wouldbe tied, demonstrated below.29"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c6-934e-11ee-af46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 24, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoColumnsR1,R2,a n dR3in Table1.9contain the truth value of eachrule\u2019s condition, per patient from Table1.4. Again, \u2018\u2022\u2019m e a n sTrueand \u2018\u00b7\u2019meansFalse.F o ry o u rc o n v e n i e n c e ,c o l u m nvcontains the outcome of themajority vote. Each value in the column is simply the sum of matchingrules that votes forRecurrenceminus the sum of matching rules that voteforNon-Recurrence. Note that to \ufb01t all the information in one table,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c7-934e-11ee-8bab-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 24, "chroma:document": "Note that to \ufb01t all the information in one table, thetable only listsTrueliterals. All non-listed literals areFalse.Breast Cancer Classi\ufb01cation Example 1.Now consider patient #1in Table1.9.R u l e sR 1a n dR 2m a t c ht h i sp a t i e n t ,w h i l er u l eR 3d o e sn o t .This means that classRecurrencegets two votes for patient #1. The classNon-Recurrencegets zero votes. Accordingly, the vote sum becomes +2andRecurrencewins the majority vote. This is the class that the Tsetlinmachine predicts"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c8-934e-11ee-855f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 24, "chroma:document": "is the class that the Tsetlinmachine predicts for patient #1. The prediction matches the patient\u2019s labelin columnRecur.Breast Cancer Classi\ufb01cation Example 2.Sometimes a rule is not suf-\ufb01ciently precise by itself. Then the majority vote arbitrates disagreement.Patient #5 matches all three rules. While the correct class isRecurrence,rule R3 still votes forNon-Recurrence.T h e r e a s o n i s t h a t p a t i e n t # 5 h a sTrue Inv-nodes 0-2.B e c a u s eMenopause lt40isFalse(not listed)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36c9-934e-11ee-8ab9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 24, "chroma:document": "e c a u s eMenopause lt40isFalse(not listed) andDeg-malig 3isTrue, rules R1 and R2 matches as well. As a result, the votingsum becomes +1.Recurrencethen wins the majority vote.Remark.Note how the last example uses detailed rules with several lit-erals. At the same time, the output is decided by summation. One cansay that the Tsetlin machine integrates rule-based and summation-baseddecision making. Sometimes it is useful to describe objects in detail withrules, and sometimes it is useful to add"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ca-934e-11ee-849b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 24, "chroma:document": "withrules, and sometimes it is useful to add up evidence from multiple rulesusing summation. For comparison, so-calleddecision treesare rule-based,while so-calledlogistic regressionis summation-based.1.6 Learning to CoordinateStarting Up.The Tsetlin machine uses multiple rules. You start up byassembling how many rules you like. Then you assign one class to each ofthem. In the breast cancer example, you used twoRecurrencerules and one30"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36cb-934e-11ee-b8e9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 25, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoNon-Recurrencerule, for instance. The Tsetlin machine then initialises eachrule by placing all the literals in memory position 5, i.e., barelyForgotten.From there on, each rule learns its condition from observing ob jects one byone, together with each object\u2019s class. You already know how a single rulelearns. You will now discover how several rules coordinate the learning,\ufb01lling di\u21b5erent roles in the classi\ufb01cation of objects.Vote"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36cc-934e-11ee-8a4b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 25, "chroma:document": "roles in the classi\ufb01cation of objects.Vote Margin.The Tsetlin machine coordinates learning of multiple ruleswith aVote Margin.A V o t e M a r g i n i s a n i n t e g e r n u m b e r t h a t y o u u s e t ocreate a margin between the winning and the losing class. If you set theVote Margin to 2, you tell the Tsetlin machine that the winning class musthaveexactlytwo more votes than the losing class. It must then attempt toful\ufb01ll this requirement for every object it observes during learning. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36cd-934e-11ee-87be-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 25, "chroma:document": "for every object it observes during learning. In thismanner, the Tsetlin machine creates complementary rules. In the following,you can use a Vote Margin of 2 for illustration purposes.Complete Learning Algorithm.The Tsetlin machine learns comple-mentary rules as follows:1.Observe a new object and its class. The observation consists of theobject\u2019s literals (see Section1.1).2.Evaluate each rule\u2019s condition using the truth values of the literals(see Section1.2).3.Calculate the vote sum (see"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36ce-934e-11ee-bc36-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 25, "chroma:document": "Section1.2).3.Calculate the vote sum (see Section1.5):a)Identify the rules whose condition isTrue. Use these for voting.b)Add up the votes in favour of the object\u2019s class.c)Subtract the votes in favour of the other class.d)Refer to the summation outcome asv.e)Setvto the Vote Margin2if larger than2and to\u00002if smallerthan\u00002.4.Go through each rule and give it feedback ifRand()\uf8ff2\u0000v4, drawnrandomly per rule:31"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36cf-934e-11ee-a2db-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 26, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo#Menop.Inv-nodesDeg-maligRecur.R1R2R3v1.ge403-53yes\u2022\u2022\u00b7+22.lt400-23no\u00b7\u2022\u2022+03.ge406-83yes\u2022\u2022\u00b7+24.ge400-22no\u00b7\u00b7\u2022\u000015.premeno0-23yes\u2022\u2022\u2022+16.premeno0-21no\u00b7\u00b7\u2022\u00001Table 1.10: A table of six patients with information onMenopause,Inv-nodes,Deg-malig,a n dRecurrence.a)Give the rule Recognize or Erase Feedback if the rule belongs tothe object\u2019s class (see Section1.3).b)Give the rule Reject Feedback if it belongs to another class"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d0-934e-11ee-be56-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 26, "chroma:document": "Reject Feedback if it belongs to another class (seeSection1.4).5.Goto 1.Above,Rand() provides a random \ufb02oating-point value between 0.0a n d1.0. Observe how the random updating of rules according toRand()\uf8ff2\u0000v4is crucial for coordination. If you are far from achieving the Vote Margin forap a r t i c u l a ro b j e c t ,y o uu p d a t et h er u l e sm o r ea g g r e s s i v e l y . I nt h ee x t r e m e ,if the vote sum is\u00002o rs m a l l e r ,y o uu p d a t ea l lo ft h er u l e s . T h i si sb e c"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d1-934e-11ee-9099-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 26, "chroma:document": "p d a t ea l lo ft h er u l e s . T h i si sb e c a u s e2\u0000v4then becomes2\u0000(\u00002)4,w h i c hi se q u a lt o1.0.If the vote sum is zero, you randomly update each rule with probability0.5. This happens because2\u0000v4then becomes2\u000004,w h i c hi se q u a lt o0.5.If you are close to 2, updating calms down. Indeed, if the voting sum is2 or larger, you update none of the rules. This is because2\u0000v4becomes2\u000024,which is equal to 0.0.Because of the above random updating of the rules, they individuallyand"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d2-934e-11ee-94ea-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 26, "chroma:document": "updating of the rules, they individuallyand gradually assign themselves to classify the di\u21b5erent kinds of objectsthey face. When doing so, they prioritize objects that are further awayfrom the Vote Margin. In this manner, you achieve a resource allocatione\u21b5ect.32"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d3-934e-11ee-b4d0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 27, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoPrioritization Example.Consider a situation where you have the fol-lowing three rules:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.Table1.10shows how each rule votes per patient. For convenience, col-umnvagain includes the resulting vote sums. Observe how the vote sumsof patients #1, #3, #4, #5, and #6 give the correct classi\ufb01cation. How-ever, patients #4,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d4-934e-11ee-ac6b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 27, "chroma:document": "the correct classi\ufb01cation. How-ever, patients #4, #5, and #6 do not reach the Vote Margin of 2. As aresult, every time these patients appear you update each rule randomly ifRand()\uf8ff2\u000014, that is, one fourth of the time. Accordingly, patients #4,#5, and #6 are prioritized over patients #1 and #3. The vote sum ofpatient #2 gives the wrong classi\ufb01cation and is even further away fromclosing the margin. Every time this patient appears, you update each rulewhenRand() is less than2\u000004,w h i c hi sh a l"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d5-934e-11ee-b421-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 27, "chroma:document": "rulewhenRand() is less than2\u000004,w h i c hi sh a l fo ft h et i m e . S o ,p a t i e n t# 2i sprioritized over all the other patients.Coordination Example.Complementary dynamics drive the coordina-tion of rules. First of all, only rules that vote for the wrong class are a\u21b5ectedby Reject Feedback, shielding the rules that cast their votes correctly. Forinstance, rule R2 votes for the wrong class when facing patient #2, how-ever, rule R1 does not. Accordingly, only rule R2 receives the feedback."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d6-934e-11ee-9da0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 27, "chroma:document": "Accordingly, only rule R2 receives the feedback. Inbrief, Reject Feedback singles out exactly those rules that can reduce thevote sum deviance.Further, a Tsetlin machine memorizes precise and frequent rules quickerand more deeply. A rule memorizes the fastest when it matches all theobjects of its class and never casts a wrong vote. Rule R1 is such an idealrule. In e\u21b5ect, rule R1 receives the maximum amount of Recognize Feedbackand never experiences Erase or Reject Feedback. One can say that the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d7-934e-11ee-bd6e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 27, "chroma:document": "Erase or Reject Feedback. One can say that the moreuseful a rule is, the quicker and more deeply it is memorized. So, rule R1ends up deeply memorizingDeg-malign 3andnotMenopause lt40.33"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d8-934e-11ee-822d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 28, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er GranmoOn the other side of the spectrum, infrequently matching rules thatvote on the wrong class forget quicker and memorize slower. It is Recognizeand Erase Feedback that ensure forgetting of rare patterns. Recognize andErase Feedback also slow down memorization when the matching frequencydrops. Simultaneously, when voting on the wrong class, Reject FeedbackintroducesFalseliterals into the rule\u2019s condition, disrupting learning. RuleR2"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36d9-934e-11ee-a846-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 28, "chroma:document": "the rule\u2019s condition, disrupting learning. RuleR2 learns, for instance, slower than R1 because the Reject Feedback pushestheFalseliterals of patient #5 upwards in the rule\u2019s memory, interruptingthe memorization. Slower learning leads to more lightly memorized literals.This is helpful because a rule with lightly memorized literals can aptly shiftto more powerful patterns when found.Finally, observe that patient #5 will relentlessly trigger Recognize andErase Feedback because the voting sum never"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36da-934e-11ee-8829-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 28, "chroma:document": "andErase Feedback because the voting sum never becomes +2. However, thisfeedback only makes rule R1 more robust and helps rule R2 become moreprecise.Accordingly, the Tsetlin machine quickly settles in the goal state:R1ifDeg-malign 3and notMenopause lt40thenRecurrence,R2ifDeg-malign 3and notMenopause lt40thenRecurrence,R3ifInv-nodes 0-2thenNon-Recurrence.1.7 SummaryHere are the key points from this chapter:\u2022The Tsetlin machine recognizes objects by coordinating multiple if-then rules.\u2022Each rule"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36db-934e-11ee-8db5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 28, "chroma:document": "by coordinating multiple if-then rules.\u2022Each rule belongs to a class and learns by itself to recognize objectsof that class.\u2022The condition part of the rule consists ofand-a n dnot-operators.These form a Boolean expression that describes a pattern. The pat-tern is frequent and unique for the rule\u2019s class.34"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36dc-934e-11ee-a037-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 29, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo\u2022Am a j o r i t yv o t ea m o n gt h er u l e sd e c i d e st h eo u t p u t . T h eT s e t l i nm a -chine thus uni\ufb01es rule-based and summation-based decision making.\u2022There are three kinds of feedback that help produce rules that are bothfrequent and unique. Recognize and Erase Feedback produce frequentpatterns, while Reject Feedback makes the pattern more unique forthe class.\u2022A Vote Margin helps coordinate multiple rules. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36dd-934e-11ee-b77c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 29, "chroma:document": "Vote Margin helps coordinate multiple rules. In brief, each ruleacquires a suitable role by itself, supporting the other rules.\u2022Tsetlin machines thus unify three powerful strategies for learning pat-terns:1.Frequent pattern mining withRecognizeand Erase Feedback.2.Pattern discrimination withReject Feedback.3.Data dissection by means of theVote Margin.\u2022Being based on pureand-rules in Boolean algebra, Tsetlin machinerules are transparent and easy to comprehend by humans. This is incontrast to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36de-934e-11ee-a456-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 29, "chroma:document": "to comprehend by humans. This is incontrast to so-called black box machine learning models.\u2022And-rules in Boolean algebra support hardware-near bitwise opera-tors that dramatically increase inference speed. Further, the resultingsmall energy- and memory footprint is ideal for edge computing andInternet of Things (IoT).1.8 Exercises1.What is the structure of a Tsetlin machine rule?2.How are data features prepared as input to a Tsetlin machine?3.What are the three learning steps of a Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36df-934e-11ee-a349-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 29, "chroma:document": "are the three learning steps of a Tsetlin machine?4.When and why areFalseandForgottenliterals memorized?5.What part of Tsetlin machine learning coordinates the learning ofmultiple rules and how is the coordination done?35"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e0-934e-11ee-ad2c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 30, "chroma:document": "An Introduction to Tsetlin Machines (7/11/22) Ole-Christo\u21b5er Granmo6.When is Recognize Feedback triggered and how is this feedback re-lated to the Tsetlin machine learning steps?7.When is Reject Feedback triggered and how is this feedback relatedto the Tsetlin machine learning steps?8.What happens if we set the Forget Value closer to 0.0?9.Implement learning of a single rule with Recognize and Erase Feed-back, using the dataset in Table1.1.10.Implement learning of a single rule combining"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e1-934e-11ee-baeb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\tsetlin_book_chapter1.pdf", "page": 30, "chroma:document": "learning of a single rule combining Recognize, Erase, andReject Feedback, using the dataset in Table1.4.11.Implement learning of three rules using Vote Margin 1 \ufb01rst and thenVote Margin 2. Investigate any di\u21b5erences in the rules produced. Usethe dataset in Table1.4."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e3-934e-11ee-be23-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf", "page": 0, "chroma:document": "Received November 26, 2020, accepted December 18, 2020, date of publication January 6, 2021, date of current version January 14, 2021.\nDigital Object Identifier 10.1 109/ACCESS.2021.3049569\nExtending the Tsetlin Machine With\nInteger-Weighted Clauses for Increased\nInterpretability\nK. DARSHANA ABEYRATHNA\n , OLE-CHRISTOFFER GRANMO\n ,\nAND MORTEN GOODWIN\nCentre for Arti\ufb01cial Intelligence Research, University of Agder, 4879 Grimstad, Norway"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e4-934e-11ee-ad68-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf", "page": 0, "chroma:document": "Corresponding author: K. Darshana Abeyrathna (darshana.abeyrathna@uia.no)\nABSTRACT Building models that are both interpretable and accurate is an unresolved challenge for\nmany pattern recognition problems. In general, rule-based and linear models lack accuracy, while deep\nlearning interpretability is based on rough approximations of the underlying inference. However, recently, the\nrule-based Tsetlin Machines (TMs) have obtained competitive performance in terms of accuracy, memory"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e5-934e-11ee-b688-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf", "page": 0, "chroma:document": "footprint, and inference speed on diverse benchmarks (image classi\ufb01cation, regression, natural language\nunderstanding, and game-playing). TMs construct rules using human-interpretable conjunctive clauses in\npropositional logic. These, in turn, are combined linearly to solve complex pattern recognition tasks. This\npaper addresses the accuracy-interpretability challenge in machine learning by introducing a TM with integer"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e6-934e-11ee-8a54-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf", "page": 0, "chroma:document": "weighted clauses \u2013 the Integer Weighted TM (IWTM). The intent is to increase TM interpretability by\nreducing the number of clauses required for competitive performance. The IWTM achieves this by weighting\nthe clauses so that a single clause can replace multiple duplicates. Since each TM clause is formed adaptively\nby a Tsetlin Automata (TA) team, identifying effective weights becomes a challenging online learning"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152e36e7-934e-11ee-9ace-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Extending_the_Tsetlin_Machine_With_Integer-Weighted_Clauses_for_Increased_Interpretability.pdf", "page": 0, "chroma:document": "problem. We solve this problem by extending each team of TA with another kind of automaton: the stochastic\nsearching on the line (SSL) automaton. We evaluate the performance of the new scheme empirically using\n\ufb01ve datasets, along with a study of interpretability. On average, IWTM uses 6.5 times fewer literals than\nthe vanilla TM and 120 times fewer literals than a TM with real-valued weights. Furthermore, in terms of"}


