# embeddings_queue

created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab3d-934e-11ee-aba6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nDATASET TM INDEXED TM NON -INDEXED TM CUDA SPEED UP\nACC F1 ACC F1 ACC F1\nBBC S PORTS 85.08 \u00b11.75 85.58 \u00b11.69 87.36 \u00b11.91 88.02 \u00b11.47 84.64 \u00b12.20 86.13 \u00b12.24 38.9\u00d7\n20 N EWSGROUP 79.37 \u00b10.25 80.38 \u00b10.92 82.33 \u00b10.28 82.89 \u00b10.34 79.00 \u00b10.46 78.93 \u00b10.44 49.3\u00d7\nSEMEVAL 91.9 \u00b10.16 75.29 \u00b10.25 92.51 \u00b10.03 77.48 \u00b11.46 92.02 \u00b10.54 76.27 \u00b10.53 1.7\u00d7\nIMDB 88.42 \u00b12.05 88.39 \u00b12.16 88.2 \u00b13.14 88.13 \u00b13.44 89.92 \u00b10.23 88.90 \u00b10.24 34.6\u00d7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab3e-934e-11ee-851a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "JAVA 97.03 \u00b10.02 96.93 \u00b10.02 97.50 \u00b10.02 97.40 \u00b10.02 97.53 \u00b10.02 97.42 \u00b10.01 6.0\u00d7\nAPPLE 92.65 \u00b10.02 92.20 \u00b10.02 92.46 \u00b10.01 91.82 \u00b10.02 95.01 \u00b10.01 94.68 \u00b10.01 9.7\u00d7\nTable 3. Performance on multiple data sets. Mean and standard deviation are calculated over 5 independent runs. Speed up is calculated as\nhow many times faster is average execution time on CUDA implementation than on Indexed implementation.\nThunderSVM for all the investigated datasets. Note that"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab3f-934e-11ee-991a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "our parallelization solution leverages novel TM properties,\nallowing each TM clause to learn independently in its own\nthread.\n4.1. Regression\nWe \ufb01rst investigate performance with regression Tsetlin\nmachines ( RTM s) using two datasets: Bike Sharing and\nBlogFeedback.\nThe Bike Sharing dataset contains a 2-year usage log of the\nbike sharing system Captial Bike Sharing (CBS) at Wash-\nington, D.C., USA. The total number of rental bikes (casual"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab40-934e-11ee-9004-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "and registered), is predicted based on other relevant features\nsuch as time of day, day of week, season, and weather. In\ntotal, 16independent features are employed to predict the\nnumber of rental bikes. Overall, the dataset contains 17,389\nexamples. More details of the Bike Sharing dataset can be\nfound in (Fanaee-T & Gama, 2014).\nThe BlogFeedback dataset considers predicting the number\nof blog posts comments for the upcoming 24hours. The\ndata has been gathered from January, 2010 to March, 2012."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab41-934e-11ee-838a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "In total, BlogFeedback contains 60,021data samples, each\nwith281features. In this study, we employ the \ufb01rst 20,000\ndata samples. For both of the datasets, we use 80% of the\nsamples for training and the rest for evaluation.\nWe \ufb01rst study the impact of the number of clauses on pre-\ndiction error, measured by Mean Absolute Error (MAE). As\nillustrated in Fig. 3 for Bike Sharing, increasing the num-\nber of clauses (#clauses in x-axis) decreases the error by"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab42-934e-11ee-81b6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "allowing the RTM to capture more detailed sub-patterns.\nA larger number of clauses results in increased computation\ntime. Fig. 4 captures how execution time increases with the\nnumber of clauses for the three different implementations.\nFor the non-indexed RTM , each doubling of the number of\nclauses also doubles execution time. However, the indexed\nRTM is less affected, and is slightly faster than the CUDA\nimplementation when less than 300 clauses are utilized."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab43-934e-11ee-96ac-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "With more than 300 clauses, the CUDA implementation is\nsuperior, with no signi\ufb01cant increase in execution time as\nthe number of clauses increases. For instance, using 1280\n10 20 40 80 160 320 640 1280\nClauses20304050607080Error (MAE)Figure 3. MAE vs. #clauses on Bike Sharing.\n10 20 40 80 160 320 640 1280\nClauses0200040006000800010000Execution time [sec.]Non-Indexed\nIndexed\nCUDA\nFigure 4. Execution time vs. #clauses on Bike Sharing.\nclauses, the CUDA implementation is roughly 3.64times"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab44-934e-11ee-b5ef-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 5, "chroma:document": "faster than the indexed version. This can be explained by\nthe large number of threads available to the GPU and the\nasynchronous operation of the new architecture.\nLooking at how MAE and execution time vary over the\ntraining epochs for Bike Sharing (Fig. 5 and Fig. 6, respec-\ntively), we observe that MAE falls systematically across the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab45-934e-11ee-bfc8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nCUDA M ODEL BBC S PORTS 20 NG SEMEVAL IMDB JAVA(WSD) APPLE (WSD) REGR 1 REGR 2\nTHUNDER SVM 0.23 S 6.95 S 7.42 S 3.89 S 1.31 S 0.39 S 6.24 S 59.94 S\nTM 0.13 S 3.61 S 4.27 S 3.36 S 0.74 S 0.28 S 0.52 S 4.92 S\nTable 4. Comparison of training time between ThunderSVM and TM. The run time is an average of 5 experiments.\n0 20 40 60 80 100\nEpochs203040506070Error (MAE)Non-Indexed\nIndexed\nCUDA\nFigure 5. MAE over epochs on Bike Sharing."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab46-934e-11ee-a8e6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "CUDA\nFigure 5. MAE over epochs on Bike Sharing.\n0 20 40 60 80 100\nEpochs020406080100Execution Time [sec.]Non-Indexed\nIndexed\nCUDA\nFigure 6. Per Epoch Execution time over epochs on Bike Sharing.\nepochs, while the execution time remains stable (employing\nT= 1280 ,s= 1.5,n= 1280 ). Execution on BlogFeed-\nback exhibits similar behavior. Finally, the mean value of\nthe MAEs for each method are similar (Table 5) across 5\nindependent runs, indicating no signi\ufb01cant difference in\nlearning accuracy."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab47-934e-11ee-b9d7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "learning accuracy.\nComparing MAE with comparable contemporary methods\non Bike Sharing (Regr1), we observe that the TM is com-\npetitive \u2013 TM: 23.9; ANN: 27.4; and SVR: 22.6. Similarly,\nfor BlockFeedback (Regr2) we have \u2013 TM: 3.88; ANN: 7.2;\nand SVR: 6.0.4.2. Novelty Detection\nNovelty detection is another important machine learning\ntask. Most supervised classi\ufb01cation approaches assume a\nclosed world, counting on all classes being present in the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab48-934e-11ee-a982-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "data at training time. This assumption can lead to unpre-\ndictable behaviour during operation, whenever novel, previ-\nously unseen, classes appear. We here investigate TM-based\nnovelty detection, as proposed by Bhattarai et al. (2021c;b).\nTo this end, we use the two datasets: 20 Newsgroup and\nBBC Sports. In brief, we use the class voting sums (Section\n2) as features measuring novelty. We then employ a Mul-\ntilayer perceptron (MLP) for novelty detection that adopts"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab49-934e-11ee-ac53-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "the class voting sums as input.\nThe BBC sports dataset contains 737documents from the\nBBC sport website, organized in \ufb01ve sports article categories\nand collected from 2004 to 2005. Overall, the dataset en-\ncompasses a total of 4,613terms. For novelty classi\ufb01cation,\nwe designate the classes \u201cCricket\u201d and \u201cFootball\u201d as known\nand \u201cRugby\u201d as novel. We train on the known classes, which\nruns for 100epochs with 5,000clauses, margin Tof100,\nand speci\ufb01city sof15.0. The training times for both in-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4a-934e-11ee-9f39-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "dexed and non-indexed TMs are high compared with that of\nCUDA TM, which is around 39times faster. The 20 News-\ngroup dataset contains 18,828documents with 20classes.\nThe classes \u201ccomp.graphics\u201d and \u201ctalk.politics.guns\u201d are\ndesignated as known, and \u201crec.sport.baseball\u201d is considered\nnovel. We train the TMfor100epochs with a margin Tof\n500,10,000clauses and speci\ufb01city s= 25.0. The CUDA\nTMimplementation is here about 49times faster than the\nother versions."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4b-934e-11ee-af48-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "other versions.\nTo assess scalability, we record the execution time of both\nthe indexed and the CUDA TMwhile increasing the number\nof clauses (Fig. 7). For the indexed TM, the execution time\nincreases almost proportionally with the number of clauses,\nbut no such effect is noticeable for the CUDA TM.\nThe novelty scores generated by the TMare passed to a\nMultilayer Perceptron with hidden layer sizes ( 100,30) and\nRELU activation functions, trained using stochastic gradient"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4c-934e-11ee-b670-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 6, "chroma:document": "descent. As shown in Table 3, for both datasets, the non-\nindexed TMslightly outperforms the other TMversions,\nwhile the indexed and CUDA TMs have similar accuracy.\nThese differences can be explained by the random variation\nofTMlearning (i.e., the high standard deviations reported\nin Table 3)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4d-934e-11ee-9ad2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nDATASETSTM\nINDEXEDTM\nNON -INDEXEDTM\nCUDASPEED UP\nBIKESHARING 23.5\u00b10.04 22.5\u00b10.12 23.9\u00b10.08 156.4\u00d7\nBLOGFEEDBACK .3.91\u00b10.00 3.74\u00b10.03 3.88\u00b10.02 211.6\u00d7\nTable 5. MAE with con\ufb01dence interval, and Speed up on two regression datasets, calculated over 5 independent runs\nFigure 7. Execution time vs. #clauses on BBC Sports.\nBhattarai et al. (2021c) investigate the competitiveness\nof TMs on novelty detection. For BBC Sports they re-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4e-934e-11ee-ab0f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "port the following accuracies \u2013 TM: 89.47%; One-class\nSVM: 83.53%; and Average KNN: 55.54%. Likewise, for\n20Newsgroup they report \u2013 TM: 82.51%; One-class SVM:\n83.70%; and Average KNN: 81%.\n4.3. Sentiment and Semantic Analysis\nWe adopt the SemEval 2010 Semantic Relations (Hendrickx\net al., 2009) and the ACL Internet Movie Database (IMDb)\n(Maas et al., 2011) datasets to explore the performance of the\nTMimplementations for a large number of sparse features,\nas proposed by Saha et al. (2020)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab4f-934e-11ee-9303-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "as proposed by Saha et al. (2020).\nThe SEMEV AL dataset focuses on identifying semantic\nrelations in text. The dataset has 10,717examples, and we\nconsider each to be annotated to contain either the relation\nCause-Effect or not. The presence of an unambiguous causal\nconnective is indicative of a sentence being a causal sentence\n(Xuelan & Kennedy, 1992). For each TM, we use 40clauses\nper class to identify this characteristic of causal texts. The"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab50-934e-11ee-bbc8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "IMDb dataset contains 50,000highly polar movie reviews,\nwhich are either positive or negative. Due to the large variety\nand combination of possibly distinguishing features, we\nassign 7,000clauses to each class. For both datasets we use\nunigrams and bigrams as features.\nAs noted in Table 3, the accuracy obtained by the CPU (non-\nindexed) and the CUDA implementations are comparable\non the SEMEV AL dataset, while the indexed TMperforms\nFigure 8. Per Epoch Execution time over epochs on SEMEV AL."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab51-934e-11ee-80b3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "slightly poorer. However, the execution time is much lower\nfor the CUDA version than the other two (Fig. 8). This is\nfurther shown in Fig. 9. Clearly, the CPU-based TMwith\nindexing takes an increasing amount of time to execute as\nthe number of clauses grows, but no such effect is observed\nwith CUDA TM. More speci\ufb01cally, the execution time\nincreases only by 40% when the number of clauses goes\nfrom 20to2,560.\nWith the IMDB dataset, the CUDA version performs better"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab52-934e-11ee-a703-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "in terms of accuracy, with less variance compared to the\nCPU versions (Table 3). It exhibits similar behaviour as in\nthe SEMEV AL dataset with respect to execution time over\nincreasing number of epochs.\nThe following provides a comparison of accuracy, compiled\nfrom related literature. For Semantic Relation Classi\ufb01ca-\ntion (SemEval) TM accuracy is again competitive (TM:\n92.6%; RandomForest: 88.76%; NaiveBayes: 87.57%;\nSVM: 92.3%; and CNN-LSTM: 90.73%). The same"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab53-934e-11ee-b5a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 7, "chroma:document": "SVM: 92.3%; and CNN-LSTM: 90.73%). The same\ngoes for IMDB Sentiment Analysis (TM: 90.5%; NBOW:\n83.62%; CNN-BiLSTM: 89.54%; Tree-LSTM: 90.1%; Self-\nAT-LSTM: 90.34%; and DistilBERT: 92.82%).\n4.4. Word Sense Disambiguation\nWord Sense Disambiguation (WSD) is a vital task in\nNLP (Navigli, 2009) that consists of distinguishing the\nmeaning of homonyms \u2013 identically spelled words whose\nsense depends on the surrounding context words. We here\nperform a quantitative evaluation of the three TMimplemen-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab54-934e-11ee-ba9b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nFigure 9. Execution time vs. #clauses on SEMEV AL.\ntations using a recent WSD evaluation framework (Loureiro\net al., 2020) based on WordNet. We use a balanced dataset\nfor coarse-grained classi\ufb01cation, focusing on two speci\ufb01c\ndomains. The \ufb01rst dataset concerns the meaning of the word\n\u201cApple\u201d, which here has two senses: \u201capple inc.\u201d (company)\nand \u201capple apple\u201d (fruit). The other dataset covers the word"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab55-934e-11ee-ab4a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "\u201cJA V A\u201d, which has the two senses: \u201cjava java\u201d (geographi-\ncal location) and \u201cjava comp.\u201d (computer language). The\nApple dataset has 2,984samples split into training and test-\ning samples of 1,784and1,200, respectively. The JA V A\ndataset has 5,655samples split into 3,726and1,929sam-\nples, for training and testing. For preprocessing, we \ufb01lter\nthe stop words and stem the words using the Porter Stemmer\nto reduce the effect of spelling mistakes or non-important"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab56-934e-11ee-8c60-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "variations of the same word. To build a vocabulary (the fea-\nture space), we select the 3,000most frequent terms. The\nnumber of clauses n, margin T, and speci\ufb01city sused are\n300,50,5.0respectively, for both datasets.\nThe accuracy and F1 scores of the non-indexed and indexed\nTMs are quite similar for the Apple dataset (Table 3). How-\never, the CUDA TM outperforms both of them by a signi\ufb01-\ncant margin. In the case of JA V A dataset, the performance"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab57-934e-11ee-a10c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "is comparable for all three, and CUDA TMis slightly better.\nThe reader is requested to refer to (Yadav et al., 2021b) for\nfurther details on TM-based WSD.\nThe following is a comparison of accuracy with respect\nto related literature (Loureiro et al., 2020). For Apple we\nhave TM: 95.1%; ThunderSVM: 80.32%; FastTextBase-\n1NN: 96.3%; FastTextCommonCrawl-1NN: 97.8%; and\nBERTbase: 99.0%. Correspondingly, for JA V A we get\nTM:97.53%; ThunderSVM: 93.62%; FastTextBase-1NN:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab58-934e-11ee-b699-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "TM:97.53%; ThunderSVM: 93.62%; FastTextBase-1NN:\n98.7%; FastTextCommonCrawl-1NN: 99.5%; and BERT-\nbase: 99.6%. Note that the latter TM results are from\nour current paper. Another recent paper, however, shows\nthat better hyperparameters can make the TMoutperformFastTextBase-1NN, with the TM reaching 97.58% and\n99.38% for Apple and JA V A, respectively (Yadav et al.,\n2021b).\n4.5. Analysis of Experiment Results\nThe above results support our initial claims that the pro-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab59-934e-11ee-a5f2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "posed TMarchitecture provides learning speeds that are\nnearly constant for a reasonable number of clauses. Beyond\na suf\ufb01ciently large clause number, the computation time\nincreases at approximately the same rate as the increase in\nnumber of clauses. The number of clauses to use highly de-\npends on the dataset. In our case, using up to 7,000clauses\nprovides high accuracy for all the datasets, simultaneously\ndemonstrating almost constant training time overall. A fur-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5a-934e-11ee-ac28-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "ther increase does not signi\ufb01cantly improve accuracy. I.e.,\nthe number of clauses was decided empirically to provide a\ngeneral benchmark. Employing 7,000clauses also allows\nus evaluate the capability of leveraging the 5,120cores on\nthe GPU. Hence, considering both datasets and available\ncores, 7,000clauses are enough to demonstrate the almost\nconstant time scaling, until exhausting the cores. However,\nwe also extended the clauses to 15,000as shown in Fig. 7,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5b-934e-11ee-a351-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "and the execution time remained almost the same, thereby\nvalidating excellent exploitation of the available cores. To\nachieve this, our scheme adds 1bit per example per clause\nfor tallying, enabling desynchronization. TMs are further\nreported to use less memory than ANNs (Lei et al., 2020).\n5. Conclusions and Future Work\nIn this paper, we proposed a new approach to TMlearning,\nto open up for massively parallel processing. Rather than\nprocessing training examples one-by-one as in the original"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5c-934e-11ee-9aa2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "TM, the clauses access the training examples simultane-\nously, updating themselves and local voting tallies in paral-\nlel. The local voting tallies allow us to detach the processing\nof each clause from the rest of the clauses, supporting de-\ncentralized learning. There is no synchronization among the\nclause threads, apart from atomic adds to the local voting\ntallies. Operating asynchronously, each team of TAs most of\nthe time executes on partially calculated or outdated voting\ntallies."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5d-934e-11ee-85b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "tallies.\nBased on the numerical results, our main conclusion is\nthatTMlearning is very robust towards relatively severe\ndistortions of communication and coordination among the\nclauses. Our results are thus compatible with the \ufb01ndings\nof Sha\ufb01k et al. (2020), who shows that TM learning is\ninherently fault tolerant, completely masking stuck-at faults.\nIn our future work, we will investigate the robustness of TM\nlearning further, which includes developing mechanisms"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5e-934e-11ee-bb29-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 8, "chroma:document": "for heterogeneous architectures and more loosely coupled\nsystems, such as grid-computing."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab5f-934e-11ee-afdf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nReferences\nAbeyrathna, K. D., Granmo, O.-C., Sha\ufb01k, R., Yakovlev, A.,\nWheeldon, A., Lei, J., and Goodwin, M. A Novel Multi-\nStep Finite-State Automaton for Arbitrarily Deterministic\nTsetlin Machine Learning. In Proceedings of the 40th\nInternational Conference on Innovative Techniques and\nApplications of Arti\ufb01cial Intelligence (SGAI), Cambridge,\nUK. Springer International Publishing, 2020."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab60-934e-11ee-929f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "UK. Springer International Publishing, 2020.\nAbeyrathna, K. D., Granmo, O.-C., Zhang, X., Jiao, L.,\nand Goodwin, M. The Regression Tsetlin Machine - A\nNovel Approach to Interpretable Non-Linear Regression.\nPhilosophical Transactions of the Royal Society A , 378,\n2020.\nAbeyrathna, K. D., Granmo, O.-C., and Goodwin, M.\nExtending the Tsetlin Machine With Integer-Weighted\nClauses for Increased Interpretability. IEEE Access , 9:\n8233 \u2013 8248, 2021.\nBerge, G. T., Granmo, O.-C., Tveit, T., Goodwin, M.,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab61-934e-11ee-82c8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "Jiao, L., and Matheussen, B. Using the Tsetlin Machine\nto Learn Human-Interpretable Rules for High-Accuracy\nText Categorization With Medical Applications. IEEE\nAccess , 7:115134\u2013115146, 2019.\nBhattarai, B., Granmo, O.-C., and Jiao, L. Explainable\nTsetlin Machine framework for fake news detection with\ncredibility score assessment, 2021a.\nBhattarai, B., Granmo, O.-C., and Jiao, L. Word-level\nHuman Interpretable Scoring Mechanism for Novel Text\nDetection Using Tsetlin Machines, 2021b."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab62-934e-11ee-aeb8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "Detection Using Tsetlin Machines, 2021b.\nBhattarai, B., Jiao, L., and Granmo, O.-C. Measuring the\nNovelty of Natural Language Text Using the Conjunctive\nClauses of a Tsetlin Machine Text Classi\ufb01er. In 13th Inter-\nnational Conference on Agents and Arti\ufb01cial Intelligence\n(ICAART), Vienna , Austria . INSTICC, 2021c.\nBlakely, C. D. and Granmo, O.-C. Closed-Form Expressions\nfor Global and Local Interpretation of Tsetlin Machines\nwith Applications to Explaining High-Dimensional Data."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab63-934e-11ee-bd09-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "arXiv preprint arXiv:2007.13885 , 2020.\nFanaee-T, H. and Gama, J. Event labeling combining en-\nsemble detectors and background knowledge. Progress\nin Arti\ufb01cial Intelligence , 2(2-3):113\u2013127, 2014.\nGorji, S., Granmo, O. C., Glimsdal, S., Edwards, J., and\nGoodwin, M. Increasing the Inference and Learning\nSpeed of Tsetlin Machines with Clause Indexing. In\nInternational Conference on Industrial, Engineering\nand Other Applications of Applied Intelligent Systems"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab64-934e-11ee-b33d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "(IEA/AIE), kitakyushu, Japan . Springer, 2020.Gorji, S. R., Granmo, O.-C., Phoulady, A., and Goodwin, M.\nA Tsetlin Machine with Multigranular Clauses. In Pro-\nceedings of the Thirty-ninth International Conference on\nInnovative Techniques and Applications of Arti\ufb01cial Intel-\nligence (SGAI), Cambridge, UK , volume 11927. Springer\nInternational Publishing, 2019.\nGranmo, O.-C. The Tsetlin Machine - A Game The-\noretic Bandit Driven Approach to Optimal Pattern"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab65-934e-11ee-bfe4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "oretic Bandit Driven Approach to Optimal Pattern\nRecognition with Propositional Logic. arXiv preprint\narXiv:1804.01508 , 2018.\nGranmo, O.-C., Glimsdal, S., Jiao, L., Goodwin, M., Om-\nlin, C. W., and Berge, G. T. The Convolutional Tsetlin\nMachine. arXiv preprint arXiv:1905.09688 , 2019.\nHendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P.,\n\u00b4O S\u00b4eaghdha, D., Pad \u00b4o, S., Pennacchiotti, M., Romano,\nL., and Szpakowicz, S. Semeval-2010 task 8: Multi-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab66-934e-11ee-994b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "way classi\ufb01cation of semantic relations between pairs of\nnominals. In Proceedings of the Workshop on Semantic\nEvaluations: Recent Achievements and Future Directions,\nBoulder, Colorado , pp. 94\u201399. ACL, 2009.\nJiang, C. and Snir, M. Automatic tuning matrix multi-\nplication performance on graphics hardware. In 14th\nInternational Conference on Parallel Architectures and\nCompilation Techniques (PACT\u201905), Krasnoyarsk, Russia ,\npp. 185\u2013194. IEEE, 2005."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab67-934e-11ee-b240-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "pp. 185\u2013194. IEEE, 2005.\nJiao, L., Zhang, X., Granmo, O.-C., and Abeyrathna, K. D.\nOn the Convergence of Tsetlin Machines for the XOR\nOperator. arXiv preprint arXiv:2101.02547 , 2021.\nKarnaugh, M. The map method for synthesis of combi-\nnational logic circuits. Transactions of the American\nInstitute of Electrical Engineers, Part I: Communication\nand Electronics , 72(5):593\u2013599, 1953.\nLei, J., Wheeldon, A., Sha\ufb01k, R., Yakovlev, A., and Granmo,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab68-934e-11ee-aa63-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "O.-C. From Arithmetic to Logic based AI: A Comparative\nAnalysis of Neural Networks and Tsetlin Machine. In\n2020 27th IEEE International Conference on Electronics,\nCircuits and Systems (ICECS) , pp. 1\u20134. IEEE, 2020.\nLei, J., Rahman, T., Sha\ufb01k, R., Wheeldon, A., Yakovlev, A.,\nGranmo, O.-C., Kawsar, F., and Mathur, A. Low-Power\nAudio Keyword Spotting Using Tsetlin Machines. Jour-\nnal of Low Power Electronics and Applications , 11, 2021.\nURLhttps://www.mdpi.com/2079-9268/11/\n2/18 ."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab69-934e-11ee-9df3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 9, "chroma:document": "URLhttps://www.mdpi.com/2079-9268/11/\n2/18 .\nLoureiro, D., Rezaee, K., Pilehvar, M. T., and Camacho-\nCollados, J. Language models and word sense disam-\nbiguation: An overview and analysis, 2020."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6a-934e-11ee-9f0e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "Massively Parallel and Asynchronous Tsetlin Machine Architecture\nMaas, A., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y ., and\nPotts, C. Learning word vectors for sentiment analysis.\nInProceedings of the 49th annual meeting of the asso-\nciation for computational linguistics: Human language\ntechnologies, Portland, Oregon, USA , pp. 142\u2013150. ACL,\n2011.\nNavigli, R. Word sense disambiguation: A survey. ACM\nComputing Surveys , 41:10:1\u201310:69, 2009."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6b-934e-11ee-b104-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "Computing Surveys , 41:10:1\u201310:69, 2009.\nOommen, B. J. Stochastic searching on the line and its ap-\nplications to parameter learning in nonlinear optimization.\nIEEE Transactions on Systems, Man, and Cybernetics,\nPart B (Cybernetics) , 27(4):733\u2013739, 1997.\nOwens, J. D., Luebke, D., Govindaraju, N., Harris, M.,\nKr\u00a8uger, J., Lefohn, A. E., and Purcell, T. J. A survey of\ngeneral-purpose computation on graphics hardware. In\nComputer graphics forum , volume 26, pp. 80\u2013113. Wiley\nOnline Library, 2007."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6c-934e-11ee-bf2f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "Online Library, 2007.\nPhoulady, A., Granmo, O.-C., Gorji, S. R., and Phoulady,\nH. A. The Weighted Tsetlin Machine: Compressed Repre-\nsentations with Clause Weighting. In Proceedings of the\nNinth International Workshop on Statistical Relational\nAI (StarAI), New York, USA , 2020.\nSaha, R., Granmo, O.-C., and Goodwin, M. Mining In-\nterpretable Rules for Sentiment and Semantic Relation\nAnalysis using Tsetlin Machines. In Proceedings of the\n40th International Conference on Innovative Techniques"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6d-934e-11ee-bca8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "and Applications of Arti\ufb01cial Intelligence (SGAI), Cam-\nbridge, UK . Springer International Publishing, 2020.\nSatish, N., Harris, M., and Garland, M. Designing ef\ufb01-\ncient sorting algorithms for manycore GPUs. In 2009\nIEEE International Symposium on Parallel & Distributed\nProcessing, Rome, Italy , pp. 1\u201310. IEEE, 2009.\nSha\ufb01k, R., Wheeldon, A., and Yakovlev, A. Explainability\nand Dependability Analysis of Learning Automata based\nAI Hardware. In IEEE 26th International Symposium"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6e-934e-11ee-9d37-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "AI Hardware. In IEEE 26th International Symposium\non On-Line Testing and Robust System Design (IOLTS),\nNaples, Italy . IEEE, 2020.\nTsetlin, M. L. On behaviour of \ufb01nite automata in random\nmedium. Avtomat. i Telemekh , 22(10):1345\u20131354, 1961.\nValiant, L. G. A Theory of the Learnable. Communications\nof the ACM , 27(11):1134\u20131142, 1984.\nWen, Z., Shi, J., Li, Q., He, B., and Chen, J. ThunderSVM:\nA fast SVM library on GPUs and CPUs. The Journal of\nMachine Learning Research , 19(1):797\u2013801, 2018."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab6f-934e-11ee-b6a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "Machine Learning Research , 19(1):797\u2013801, 2018.\nWheeldon, A., Sha\ufb01k, R., Rahman, T., Lei, J., Yakovlev, A.,\nand Granmo, O.-C. Learning Automata based Energy-\nef\ufb01cient AI Hardware Design for IoT. Philosophical\nTransactions of the Royal Society A , 2020.Xuelan, F. and Kennedy, G. Expressing causation in written\nenglish. RELC Journal , 23(1):62\u201380, 1992.\nYadav, R. K., Jiao, L., Granmo, O.-C., and Goodwin, M.\nHuman-Level Interpretable Learning for Aspect-Based"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab70-934e-11ee-84d6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "Sentiment Analysis. In Proceedings of the AAAI Confer-\nence on Arti\ufb01cial Intelligence, Vancouver, Canada . AAAI,\n2021a.\nYadav, R. K., Jiao, L., Granmo, O.-C., and Goodwin, M.\nInterpretability in Word Sense Disambiguation using\nTsetlin Machine. In 13th International Conference on\nAgents and Arti\ufb01cial Intelligence (ICAART), Vienna, Aus-\ntria. INSTICC, 2021b.\nZhang, X., Jiao, L., Granmo, O.-C., and Goodwin, M. On\nthe Convergence of Tsetlin Machines for the IDENTITY-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab71-934e-11ee-97b1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2009.04861.pdf", "page": 10, "chroma:document": "and NOT Operators. IEEE Transactions on Pattern Anal-\nysis and Machine Intelligence , 2021."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab72-934e-11ee-8eee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "Drop Clause: Enhancing Performance, Robustness and Pattern Recognition\nCapabilities of the Tsetlin Machine\nJivitesh Sharma, Rohan Yadav, Ole-Christoffer Granmo, Lei Jiao\nCenter for Artificial Intelligence Research (CAIR)\nUniversity of Agder, Norway\n{jivitesh.sharma, rohan.k.yadav, ole.granmo, lei.jiao}@uia.no\nAbstract\nLogic-based machine learning has the crucial advantage of\ntransparency. However, despite significant recent progress,\nfurther research is needed to close the accuracy gap between"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab73-934e-11ee-ab11-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "logic-based architectures and deep neural network ones. This\npaper introduces a novel variant of the Tsetlin machine (TM)\nthat randomly drops clauses, the logical learning element of\nTMs. In effect, TM with Drop Clause ignores a random se-\nlection of the clauses in each epoch, selected according to a\npredefined probability. In this way, the TM learning phase be-\ncomes more diverse. To explore the effects that Drop Clause\nhas on accuracy, training time and robustness, we conduct"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab74-934e-11ee-b06b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "extensive experiments on nine benchmark datasets in natural\nlanguage processing (IMDb, R8, R52, MR, and TREC) and\nimage classification (MNIST, Fashion MNIST, CIFAR-10,\nand CIFAR-100). Our proposed model outperforms baseline\nmachine learning algorithms by a wide margin and achieves\ncompetitive performance compared with recent deep learn-\ning models, such as BERT-Large and AlexNet-DFA. In brief,\nwe observe up to 10% increase in accuracy and 2\u00d7to4\u00d7"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab75-934e-11ee-be35-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "faster in learning than those of the standard TM. We visu-\nalize the patterns learnt by Drop Clause TM in the form of\nheatmaps and show evidence of the ability of drop clause\nto learn more unique and discriminative patterns. We finally\nevaluate how Drop Clause affects learning robustness by in-\ntroducing corruptions and alterations in the image/language\ntest data, which exposes increased learning robustness.\nIntroduction\nResearchers across various fields are increasingly paying at-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab76-934e-11ee-97ae-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "tention to the interpretability of AI techniques. While inter-\npretability previously was inherent in most machine learn-\ning approaches, the state-of-the-art methods now increas-\ningly rely on black-box deep neural network-based models.\nNatively, these can neither be interpreted during the learn-\ning stage nor while producing outputs. For this reason, a\nsurge of techniques attempts to open the black box by visual\nexplanations and gradient-based interpretability (Simonyan,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab77-934e-11ee-868b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "Vedaldi, and Zisserman 2014; Zhang, Wu, and Zhu 2018;\nRadhakrishnan et al. 2018; Bau et al. 2017; Selvaraju et al.\n2017; Ribeiro, Singh, and Guestrin 2016).\nThe Tsetlin Machine (TM) is an interpretable rule-\nbased machine learning algorithm that produces logical\nCopyright \u00a9 2023, Association for the Advancement of Artificial\nIntelligence (www.aaai.org). All rights reserved.rules (Granmo 2018). In brief, it employs an ensemble of\nTsetlin Automata (TA) that learns propositional logic ex-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab78-934e-11ee-921c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "pressions from Boolean input features, and TM achieves in-\nterpretability by leveraging sparse disjunctive normal form.\nIndeed, humans are particularly good at understanding flat\nand short logical AND-rules, reflecting human reason-\ning (Noveck et al. 1991). In addition, due to its Boolean\nrepresentations and finite-state learning mechanisms, it pos-\nsesses a minimalistic memory footprint.\nSimilar to neural networks, the vanilla TM suffers overfit-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab79-934e-11ee-977e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "ting in the sense that its learning elements, i.e., clauses,\nare prone to capture noisy and redundant patterns. Initially,\nwe believed that this could be simply because of the large\nnumber of clauses. However, by decreasing the number of\nclauses, there was a proportionate decrease in performance.\nIn order to mitigate overfitting, inspired by Dropout (Srivas-\ntava et al. 2014), we propose a method named Drop Clause\n(DC) in this paper1. More specifically, DC method randomly"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7a-934e-11ee-9619-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "drops or switches off a set of clauses during training, which\nis similar to what dropout does in neural networks. Unlike\ndropout, DC induces stochasticity in TM learning, boosting\nperformance in terms of accuracy and learning speed. It fur-\nther improves the patterns captured by the clauses, making\nthem more robust towards input perturbations. As a direct\nconsequence of DC during training, the training time is re-\nduced proportionally and the heatmaps produced becomes\nmore distinct."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7b-934e-11ee-9b55-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "more distinct.\nPaper Contributions:\n\u2022 We propose to drop clauses randomly during each train-\ning iteration, which introduces additional stochasticity.\n\u2022 We demonstrate that DC makes the TM capture more\nunique patterns, improving its generalization perfor-\nmance.\n\u2022 DC TM leads to competitive results to large deep neural\nnetwork models on nine benchmark datasets for NLP and\nimage classification. Since TM is more akin to a stan-\ndard machine learning algorithm, like a decision tree,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7c-934e-11ee-b932-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 0, "chroma:document": "than a deep learning model, we compare DC TM with\nwell-known machine learning algorithms, documenting\nsuperior performance across all the datasets.\n1The code is available online on: https://github.com/\nAnonymous-2491/Drop-Clause-Interpretable-TM.\nThe Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23)\n13547"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7d-934e-11ee-80c6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "\u2022 Via heatmaps, we visualize that DC TM becomes more\nclear-cut and robust on image classification and language\nsentiment analysis.\nDrop Clause for Tsetlin Machines\nModern society needs reliable, unbiased, and trustworthy\nAI systems. Researchers have shown that neural networks\nare not fully mature for integration into society. For exam-\nple, neural network interpretability is fragile towards adver-\nsarial examples (Ghorbani, Abid, and Zou 2019). Gener-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7e-934e-11ee-aa49-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "ating adversarial perturbations that produce visually indis-\ntinguishable images for humans leads to dramatically dif-\nferent neural network interpretations, without changing the\nlabel. While methods such as GradCAM (Selvaraju et al.\n2017) and LIME (Ribeiro, Singh, and Guestrin 2016) can\ntake a peek inside black-box models, the explanations of-\nfered can be arbitrarily inaccurate, as argued by Rudin et al.\nThey pinpoint several reasons for choosing inherently inter-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab7f-934e-11ee-90a2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "pretable models over attempting to explain black-box ones\n(Rudin 2019). They further raise three algorithmic chal-\nlenges that the machine learning community faces to suc-\nceed with human-level interpretability. Although neural net-\nworks are inherently inadequate for these challenges, the\nTM addresses them natively:\n\u2022Challenge 1: Rule-based logical models. The TM learns\na rule-based model using logical operations. Learning\nis game-theoretic with Nash equilibria that correspond"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab80-934e-11ee-8014-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "to the optimal propositional logic expressions (Granmo\n2018; Zhang et al. 2022; Jiao et al. 2022).\n\u2022Challenge 2: Linear models with sparse scoring systems.\nThe TM is a linear model that produces sparse clauses\nand integer weighted scores (Abeyrathna, Granmo, and\nGoodwin 2021).\n\u2022Challenge 3: Domain-specific interpretable AI. The TM\nis inherently interpretable, used for producing inter-\npretable models across several domains (Abeyrathna\net al. 2020; Lei et al. 2021; Yadav et al. 2021, 2022)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab81-934e-11ee-983e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "In what follows, we first briefly present the basic TM and\nits convolutional version. Thereafter, we introduce the DC\ntechnique, which enhances the stochasticity of TM learning.\nTM and Convolutional TM\nA TM in its simplest form takes a feature vector x=\n[x1, x2, . . . , x o]\u2208 {0,1}oofoBoolean values as in-\nput, producing a Boolean output \u02c6y\u2208 {0, 1}. Pat-\nterns are expressed as conjunctive clauses Cj(AND-\nrules), built from literals L={l1, l2, . . . , l 2o}="}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab82-934e-11ee-a211-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "{x1, x2, . . . , x o,\u00acx1,\u00acx2, . . . ,\u00acxo}:\nCj=2o^\nk=1h\ng(aj\nk)\u21d2lki\n. (1)\nIn Eq. (1), aj\nkis the TA state that controls inclusion of literal\nlkin clause jandg(\u00b7)maps the TA state to action 0or1. The\nimply operator, in turn, implements the action in the clause.\nThat is, if g(aj\nk)is1, then the imply operator requires that\nthe literal is 1for the whole expression to be 1. Ifg(aj\nk)is0,on the other hand, the expression becomes 1regardless the\ntrue value of lk.\n\u02c6y=(\n1,Pn\u22121\nj=1,3,...Cj\u2212Pn"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab83-934e-11ee-b584-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "true value of lk.\n\u02c6y=(\n1,Pn\u22121\nj=1,3,...Cj\u2212Pn\nj=2,4,...Cj\u22650,\n0,Otherwise.(2)\nClauses are divided into even-indexed/odd-indexed clauses\nthat vote for output \u02c6y={0,1}. Odd-indexed clauses vote\nin favour of the class whereas even-indexed clauses vote\nagainst the class, as shown in Eq. (2). A positive vote count\nmeans majority of the clauses predict the input to belong to\nthe class and a negative vote count implies majority of the\nclauses believe the input does not belong to the class. A de-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab84-934e-11ee-a520-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "tailed explanation can be found in (Granmo 2018).\nThe Convolutional TM (CTM) is as an interpretable alter-\nnative to CNNs. Whereas the TM categorizes an image by\nemploying each clause once to the whole image, the CTM\nuses each clause as a convolution filter. That is, a clause is\nevaluated multiple times, once per image patch taking part\nin the convolution. The output of a convolution clause is ob-\ntained simply by ORing the outcome of evaluating the clause\non each patch:\n\u02c6y=(\n1,Pn\u22121\nj=1,3,...WB"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab85-934e-11ee-889c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "on each patch:\n\u02c6y=(\n1,Pn\u22121\nj=1,3,...WB\nb=1Cj\u2212Pn\nj=2,4,...WB\nb=1Cj\u22650,\n0,Otherwise.\nHere, brefers to one out of Bavailable image patches. A\ndetailed explanation can be found in (Granmo et al. 2019).\nPlease see the supplementary material for further details on\nTM and CTM.\nDrop Clause\nHere we propose a novel regularization method for the TM,\ni.e., DC. This technique is inspired by the dropout method\nfor neural networks. In DC, clauses are removed with a prob-\nability pin each training epoch:\n\u02c6y=("}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab86-934e-11ee-aa0e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "ability pin each training epoch:\n\u02c6y=(\n1,Pn\u22121\nj=1,3,...\u03c0jCj\u2212Pn\nj=2,4,...\u03c0jCj\u22650,\n0,Otherwise.(3)\nAbove, \u03c0j\u2208 {0, 1}for clause jis zero with probability p\nfor each complete epoch. Clearly, the vanilla TM and its DC\nvariant are equivalent if p= 0.\nDC TM is similar to dropout in neural networks, and the pur-\npose is to reduce the chance of learning redundant patterns.\nA vanilla TM seeks to minimize prediction error on the train-\ning data. Achieving this, there may still be unused patterns"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab87-934e-11ee-a69e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "available in the training data. These patterns can potentially\nbe useful when facing new data, such as test data. By ran-\ndomly dropping clauses for complete epochs, we mobilize\nother clauses to take over the role of the dropped clauses.\nDC can be thought of as having a pool of clauses and se-\nlecting a set of clauses with 1\u2212pprobability for training\nevery epoch. However, due to this stochastic learning, the\nmobilized clauses may solve the task in a different way each"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab88-934e-11ee-9cbd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 1, "chroma:document": "time. Hence, robustness increases overall when the complete\nset of clauses are turned on again. The resulting effects are\nevaluated experimentally in the next section.\nThe stochasticity induced by DC in the TM learning process\n13548"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab89-934e-11ee-b660-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "is similar to the stochasticity induced by stochastic gradi-\nent descent (SGD) (Robbins 2007). In SGD, a mini-batch of\ntraining samples is selected randomly from the training data.\nWhile, in DC, various sets of clauses are selected for train-\ning with probability 1\u2212p. The major difference is that, SGD\nis stochastic in terms of data sampling during the learning\nstage, whereas DC is stochastic in terms of sampling the el-\nements of the model structure (clauses are the elements that"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8a-934e-11ee-850d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "form the TM model). Another difference is that the stochas-\nticity of DC can be adjusted with the DC probability hyper-\nparameter p. Note that the Drop Clause TM still retains its\nconvergence properties. Please see supplementary material\nfor details.\nEnhanced Performance\nWe here investigate the effects that DC has on the perfor-\nmance of TMs. To assess the generality of our approach,\nwe test DC both on NLP and image classification. For NLP,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8b-934e-11ee-8cba-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "we use TM with weighted clauses, and for image classifi-\ncation, we use the CTM. For evaluation, we compare our\napproach with the comparable state-of-the-art deep learning\nmodels. Since TM is a rule-based machine learning method,\nwe also cover more standard machine learning algorithms.\nThe DC implementation is written in PyCUDA for paral-\nlel GPU computation. We train our models on 16 NVIDIA\nTesla V100 TensorCore GPUs for the image classification"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8c-934e-11ee-83a4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "task and on one NVIDIA RTX 3070 8GB GPU for the NLP\ntasks2.\nNatural Language Processing\nWe explore the performance of DC on NLP tasks first.\nThe first experiments assess how varying the DC proba-\nbility affects classification accuracy and then compare the\nbest found configuration with similar machine learning al-\ngorithms and state-of-the-art techniques. We adopt 5popular\nstandard datasets, summarized below:\n\u2022IMDb consists of 50,000 movie reviews for sentiment"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8d-934e-11ee-b294-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "analysis, split in half for training and testing. Here, we\nuse10,000clauses and set TM threshold Tto8,000and\nspecificity sto2.0.\n\u2022Reuters-21578 contains the text categorization datasets\nR8andR52. R8 is divided into 8categories, with 5,485\ntraining and 2,189testing samples, whereas R52 consists\nof52categories, divided into 6,532training and 2,568\ntesting samples. We here employ 3,000clauses with T=\n2,000ands= 7.0.\n\u2022MRis another movie review dataset for binary sentiment"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8e-934e-11ee-9d59-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "classification, consisting of 10,662samples. We use the\ntrain-test split as in (Tang, Qu, and Mei 2015) and use\n5,000clauses with T= 4,000ands= 6.0.\n\u2022TREC is a question classification dataset that encom-\npasses 6categories. There are a total of 6,000samples,\n5,500for training and 500for testing. Here, we employ\n5,000clauses with T= 4,000ands= 2.0.\n2Note that these are not hardware requirements. We use these\nGPUs just to train faster.TM p=\n0p=.1p=.25p=.5p=.75\nIM\nDB 87.2 88.3 89.6 90.4 91.27"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab8f-934e-11ee-8544-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "IM\nDB 87.2 88.3 89.6 90.4 91.27\nR8 96.16 97.6 98.1 98.5 98.94\nR52 89.14 89.5 90.8 91.5 92.75\nMR 75.14 77.25 77.9 78.2 78.67\nTREC 88.05 89.8 90.1 90.5 89.9\nTable 1: Effect of DC on NLP Datasets.\nCT\nM p= 0 p=.1p=.25p=.5p=.75\nMN\nIST 99.3 99.3 99.45 99.35 98.2\nF-MNIST 91.5 91.75 92.5 92.25 91.25\nCIFAR-10 69.3 70.5 73.2 75.1 72.6\nCIFAR-100 35.5 39.5 42.6 45.2 40.8\nTable 2: Effect of DC on Image Classification.\nOur first step is to compare the difference in perfor-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab90-934e-11ee-818d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "mance with respect to changing DC probability p, as\nshown in Table 1. The selected DC probabilities are p\u2208\n{0.1,0.25,0.5,0.75}. For IMDb, R8, R52 and MR, the best\nperformance is achieved with p= 0.75, which is equivalent\nof dropping 75% of the total clauses per class, per epoch.\nAs for TREC, p= 0.5works best. As seen, DC has sig-\nnificant effect on accuracy, with average accuracy going up\nby4.07% for IMDb, 2.78% for R8, 3.61% for R52, 3.53%\nfor MR and 2.45% for TREC. Additionally, we observe a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab91-934e-11ee-b81c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "substantial reduction in training time proportional to the DC\nratio. Inference times on these datasets are less than 2.3sper\n1000 samples on average on an NVIDIA GTX 1080 GPU.\nThe TM is a rule-based machine learning algorithm, so we\nalso compare our proposed TM model with a few traditional\nmachine learning techniques. Specifically, we compare our\nproposed TM with DC against Support Vector Machine\n(SVM), Random Forests (RF), K-Nearest Neighbours (K-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab92-934e-11ee-a80b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "NN) and XGBoost (XGB). Table 2 displays the comparison\nresults3. From Table 2, it is clear that DC TM outperforms\nother machine learning algorithms by a wide margin.\nWe also compare our model with deep learning methods,\nsome of which representing the state of the art. Tables 4\nand 5 show the results of the comparisons. Enhanced with\nDC, the accuracy of TM is comparable not only to CNNs\nand LSTMs but also to computationally complex and para-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab93-934e-11ee-9d4a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "metrically large state-of-the-art models like BERT-Large.\nOn all datasets in Table 4, inclusion of DC propels the\nperformance of the TM to outperform CNN (Kim 2014)\nand bidirectional LSTM with pretrained word embeddings.\nIn fact, on R8 and R52, DC TM is able to achieve better\naccuracy than BERT (Devlin et al. 2019) and comes close\nto its performance on MR. Also, comparing our model with\nthe state-of-the-art graph convolutional neural network,\nS2GC (Zhu and Koniusz 2020), DC TM achieves better"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab94-934e-11ee-ab32-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 2, "chroma:document": "performance on R8 and MR, and comparable performance\n3Note that the machine learning methods used here for compar-\nison have default parameters from sklearn.\n13549"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab95-934e-11ee-a6fa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "TM\nTM SVM RF K-NN XGB\n(DC)\nIM\nDB 87.2 91.27 83.2 78.5 72.5 84.4\nR8 96.16 98.94 84.7 82.4 77.3 85.1\nR52 89.14 92.75 71.0 72.5 58.2 75.4\nMR 75.14 78.67 52.2 51.7 39.1 56.3\nTREC 88.05 90.5 67.5 65.4 56.1 77.6\nTable 3: Comparison with ML methods.\nTM\nCNN B ILSTM BERT S2GC\n(DC) LARGE\nIM\nDB 91.27 87.9 88.9 95.4 -\nR8 98.94 95.71 96.31 96.02 97.4\nR52 92.75 87.59 90.54 89.66 94.5\nMR 78.67 77.75 77.68 79.24 76.7\nTable 4: Comparison on IMDb, R8, R52 and MR."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab96-934e-11ee-ad60-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "Table 4: Comparison on IMDb, R8, R52 and MR.\non R52. Similar results are obtained on the TREC dataset\nshown in Table 5. DC TM outperforms a baseline LSTM\nmodel as well as a vanilla Transformer (TF) and Trans-\nformer with feature projection (FP) (Qin, Hu, and Liu\n2020).\nEven though BERT-Large achieves better performance on\nTREC dataset, TM has major advantages over these models\nin terms of computational complexity and transparency,\nwhich are further enhanced by DC (See Section \u201cEnhanced"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab97-934e-11ee-b3af-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "Robustness\u201d). While it is disputed whether attention is\nexplainable (Jain and Wallace 2019), attention is much\nmore complex (quadratic complexity with floating-point op-\nerations) than our proposed model (linear complexity with\nBoolean operations) that achieves comparable performance.\nNote that our model only relies on simple bag-of-words\ntokens in the target datasets, without considering any\nadditional pretrained world knowledge (Yadav et al. 2021),"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab98-934e-11ee-9bb1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "like word2vec, Glove or BERT features. Yet, it achieves\ncompetitive performance compared to deep learning and\nsome state-of-the-art models.\nImage Classification\nWe now turn to image classification, again focusing on how\nDC affects performance of TM. To this end, we evaluate DC\non four benchmark image classification datasets: MNIST,\nFashion-MNIST, CIFAR-10 and CIFAR-100. We binarize\nthe datasets using an adaptive Gaussian thresholding pro-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab99-934e-11ee-a929-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "cedure as proposed in (Granmo et al. 2019). This binariza-\ntion results in images with only 1bit per pixel per channel,\nconsiderably reducing the memory overhead. For the task of\nimage classification, we employ the CTM.\n\u2022MNIST encompasses 70,000 28 \u00d728gray-scale im-\nages of hand written single digits, 60,000 for training\nand10,000for testing. Here, we use 8,000clauses with\nT= 6,400ands= 5.0.TM\n(DC) LSTM TF FP+TF BERT-L ARGE\nTR\nEC-6 90.5 87.19 87.33 89.5 95.6\nTable 5: Comparison on TREC-6."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9a-934e-11ee-adb4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "Table 5: Comparison on TREC-6.\n\u2022Fashion-MNIST contains 28\u00d728gray-scale images\nfrom the Zalando catalogue. For this dataset, we use the\nsame parameters as the MNIST dataset.\n\u2022CIFAR-10 and CIFAR-100 consist of 50,000 32 \u00d732\ncolor images of objects divided into 10categories for\nCIFAR-10 and 100for CIFAR-100. Another 10,000im-\nages are provided for testing. Here, we make use of\n60,000clauses with T= 48,000ands= 10.0.\nAgain, we explore the effects of four DC probability set-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9b-934e-11ee-8af4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "tings, p\u2208 {0. 1,0.25,0.5,0.75}, shown in Table 2. The best\nperformance is achieved with p= 0.25for the MNIST\nand Fashion-MNIST, while p= 0.5gives the best result\nfor the CIFAR datasets, which is equivalent to dropping a\nquarter (or half) of the clauses per training iteration. No-\ntice the considerable performance increase, especially on the\nCIFAR datasets, with DC of p= 0.5. Peak accuracy on\nMNIST is 99.45% (\u00b10.25%) , and on Fashion-MNIST it is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9c-934e-11ee-bd29-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "92.5% (\u00b10.25%) . CIFAR-10 and CIFAR-100 peak at 75.1%\n(\u00b10.4%) and45.2% (\u00b10.2%) respectively, averaged over\n100runs. Apart from the accuracy gain, DC with p= 0.5\nreduces training time by approximately 50%, for the CI-\nFAR datasets. For instance, training time per epoch drops\nfrom 61.83sto32.58sfor CIFAR-10. The inference time\nfor the DC CTM on CIFAR-10 is 1.5sper1000 images on\none NVIDIA Tesla V100 GPU, leveraging the computation\nbenefits of binary operations for TM inference."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9d-934e-11ee-bebf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "benefits of binary operations for TM inference.\nTable 6 shows the comparison of the DC CTM with other\npopular machine learning techniques on the image classifi-\ncation. As seen, CTM outperforms the traditional machine\nlearning techniques by a wide margin, further widened by\nthe introduction of DC4.\nTable 7 contains a comparison between the DC CTM with\ncertain related and state-of-the-art techniques. Binary neural\nnetworks can be represented exactly as a propositional logic"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9e-934e-11ee-a5a3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "expression (Narodytska et al. 2018). As such, they are partic-\nularly comparable to CTM and we therefore also include re-\nsults for EEV-BNN (Jia and Rinard 2020). EEV-BNN is a bi-\nnary neural network that has been verified with Boolean sat-\nisfiability. Employing EEV-BNN, we use the MNIST-MLP\nand Conv-Large BNN architectures from (Jia and Rinard\n2020) for comparisons. The DC CTM significantly outper-\nforms EEV-BNN on MNIST and CIFAR-10, as shown in"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cab9f-934e-11ee-8142-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "forms EEV-BNN on MNIST and CIFAR-10, as shown in\nTable 7. We also compare our method with a transformer\nbased model of reduced complexity to make it more compa-\nrable to our model. The Vision Nystromformer (ViN) (Jee-\nvan and Sethi 2021) is a vision transformer (ViT) based\nmodel that reduces the quadratic computational complex-\nity of ViT by using the Nystrom methods for approximat-\n4Note that the accuracy on CIFAR-100 is not displayed in Ta-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba0-934e-11ee-a9e1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 3, "chroma:document": "ble 6 for other machine learning algorithms as the accuracy is mi-\nnuscule.\n13550"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba1-934e-11ee-baab-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "CT\nM CTM SVM RF K-NN XGB\n(DC)\nMN\nIST 99.3 99.45 93.4 93.8 95.3 96.2\nF-MNIST 91.5 92.5 84.6 87.5 85.4 88.4\nCIFAR-10 69.3 75.1 37.5 48.7 33.9 47.8\nCIFAR-100 35.5 45.2 - - - -\nTable 6: Image Classification with ML methods.\nCT\nM EEV H YBRID ALEXNET MAXOUT\n(DC) -BNN -V IN -DFA\nMN\nIST 99.45 98.25 96.4 98.2 99.06\nF-MNIST 92.5 - - 91.66 -\nCIFAR-10 75.1 63.45 75.26 64.7 86.8\nCIFAR-100 45.2 - - 52.62 59.52\nTable 7: Image Classification with SOTA models."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba2-934e-11ee-9c30-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "Table 7: Image Classification with SOTA models.\ning self-attention. It achieves 65.06% accuracy on CIFAR-\n10 whereas the Hybrid-ViN, which incorporates rotary posi-\ntional embedding, obtains slightly better accuracy as shown\nin Table 7, comparable to our model.\nWe also contrast our model against AlexNet with Direct\nFeedback Alignment (DFA) (Webster, Choi, and changwook\nAhn 2021) for parallel backpropagation training, which\nmakes it comparable to the fast training of TM. The results"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba3-934e-11ee-8f85-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "in Table 7 show that AlexNet-DFA achieves similar perfor-\nmance on MNIST and Fashion-MNIST, whereas it is out-\nperformed by DC CTM on CIFAR-10. However, there is a\nconsiderable difference on the CIFAR-100 dataset due to the\nlarger label space. Finally, as DC is inspired from dropout,\nwe compare our model with the Maxout network (Goodfel-\nlow et al. 2013), which was a natural companion to dropout.\nMaxout achieves significantly better performance on the CI-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba4-934e-11ee-bad0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "FAR datasets as shown in Table 7. Note that on color images,\nbinarization can lead to loss of important information espe-\ncially when we use 1-bit per channel, compared with 8-bits\nper channel for lossless color information. Also, as previ-\nously stated, TM possesses computational advantages over\nthese methods along with transparency.\nAnother point to note is that, originally, dropout can increase\nthe performance of basic CNNs by about 3%and6%on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba5-934e-11ee-8ffc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "the performance of basic CNNs by about 3%and6%on\nCIFAR-10 and CIFAR-100, respectively (Srivastava et al.\n2014). The DC method shows promising results by improv-\ning the performance of vanilla CTM by about 6%on CIFAR-\n10 and 10% on CIFAR-100.\nEnhanced Pattern Recognition\nIn this section, we observe more distinct heatmaps produced\nby TM and CTM for NLP and image classification tasks as a\nresult of DC, thereby visually exemplifying its superior pat-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba6-934e-11ee-ab93-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "tern recognition capabilities. We show enhancements in the\nquality of heatmaps on the MR dataset at the word-level and\non the CIFAR-10 dataset at the pixel-level. The enhance-\nments encompass richer and more distinct word representa-\ntions for NLP tasks and more clear-cut pixel representations\nfor objects in images. More specifically, these enhancementsindicate that DC is able to capture more unique, relevant and\ndiscriminative patterns. These enhancements further mani-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba7-934e-11ee-acd2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "fest the stronger pattern recognition capabilities of DC5.\nNatural Language Sentiment Analysis\nWe investigate local word-level patterns captured by TM and\nDC TM in NLP using a randomly selected test example from\nMR: \u201cA waste of fearless purity in the acting craft\u201d. The ex-\nample was selected among the ones that correctly classified\nwith DC but incorrectly classified without it. The purpose is\nto contrast and exemplify how DC captures better patterns"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba8-934e-11ee-af15-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "and enhances the heatmaps. Note that for NLP, the major-\nity of the features appear in negated form. In order to have\nword-level understanding of the model, we use a frequency-\nbased interpretation, i.e., we highlight the features based on\nhow frequently they appear in clauses (Yadav et al. 2021).\nAn arguably unique property of TMs is that the patterns they\nproduce are both descriptive (frequent) and discriminative.\nIn other words, each clause captures a full description of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152caba9-934e-11ee-87f3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "target concept, not merely the discrimination boundary.\nFigure 1a highlights the top 100most frequent features in\nnegated form (darker color means high frequency), present-\ning in the clauses triggered by the given sample using vanilla\nTM without DC. Three example clauses C0,C2, and C4\nare shown below for the color-coded features. In this case,\nthe prediction is wrong. More importantly, the literals in the\nconjunctions do not make sense.\nThe corresponding result for DC TM is depicted in Fig-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabaa-934e-11ee-b484-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "ure 1b. Here, the 100most frequent features in the negated\nform seem intuitive for predicting negative sentiment. Fea-\ntures such as \u201cNOT witti (witty)\u201d, \u201cNOT grace (graceful)\u201d,\n\u201cNOT terrif (terrific)\u201d, \u201cNOT honest\u201d, \u201cNOT cool\u201d, or \u201cNOT\nintellig (intelligent)\u201d generally mean absence of positive\nsentiment, which in this case makes the model draw the cor-\nrect conclusion (negative sentiment). Some of the specific\npatterns that are responsible for predicting the correct out-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabab-934e-11ee-b052-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "put are C0, C2,andC4, as shown in the figure. Although\nrandomly chosen, this example is representative for how TM\nwith DC is able to capture a larger variety of correct patterns\nthan what the vanilla TM is capable of.\nImage Classification\nIn image classification, TM clauses form self-contained pat-\nterns by joining pixels into multi-pixel structures. That is,\nthe image pixels are sent directly to the clauses, following\npropositional AND-rules. Therefore, Boolean propositional"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabac-934e-11ee-a6a6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "expressions (clauses) capture patterns in an image that con-\ntains pixels as literals in the clauses, which are compara-\ntively easy for humans to comprehend (Valiant 1984).\nThe CTM forms clauses from the pixels of the square image\npatches obtained in the convolution. Accordingly, we extract\nthe top kweighted clauses per class in patch form. For vi-\nsualization, we represent the non-negated pixels of a clause\nas1, negated pixels as \u22121, and excluded pixels as 0. In ad-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabad-934e-11ee-b649-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 4, "chroma:document": "ditional to the image content, each clause also encodes po-\nsitions in the image where it is valid. If a certain position is\n5More examples of enhanced pattern recognition and inter-\npretability can be found in the supplementary material.\n13551"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabae-934e-11ee-9961-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 5, "chroma:document": "A waste of fearless purity in\nthe acting craft\npositive sentiment(a) Without drop clause.\nA waste of fearless purity in\nthe acting craft\nnegative sentiment\n(b) With drop clause (p= 0.75).\nFigure 1: Word-level patterns of a sample captured: (a) without and (b) with DC.\ninvalid, the clause literals corresponding to the pixel values\nare treated as 0to indicate no activation at that location. The\nresulting clause masks are applied to the image and their ac-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabaf-934e-11ee-b63f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 5, "chroma:document": "tivation maps are added up to produce a heatmap.\nIn Figure 2, we show the advantage of DC TM via an exam-\nple. As can be seen from the figure, DC is able to capture\nthe object with more precision by capturing more discrimi-\nnative patterns and including pixels relevant to classification.\nWe believe this is because the remaining clauses are forced\nto substitute the dropped clauses, learning to perform their\ntasks more independently. Due to the stochastic nature of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb0-934e-11ee-b5a2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 5, "chroma:document": "learning, they will, however, learn to perform the tasks dif-\nferently than the dropped clauses. As a result, DC reduces\nredundancy and induces diversity in the learning of the pat-\nterns. An example of how these heatmaps can be interpreted\nis shown in the supplementary material.\n(a) Image\n (b) TM (p= 0)\n (c) TM (p= 0.5)\nFigure 2: CIFAR-10 patterns captured: (b) without and (c)\nwith DC.\nEnhanced Robustness\nIn this section, we show empirically that introduction of DC"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb1-934e-11ee-925e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 5, "chroma:document": "makes TM more robust towards input perturbations during\ntesting. We evaluate robustness on both sentiment analysis\n13552"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb2-934e-11ee-a1c2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "TM TM (DC) BiLSTM\nMR 75.14 78.67 77.68\nMR (perturb) 73.12 77.12 75.85\nTable 8: Robustness Test on MR.\nCTM CTM (DC) BNN\nMNIST 99.3 99.45 98.25\nMNIST-C 88.12 91.75 85.5\nTable 9: Robustness Test on MNIST-C.\nand image classification6.\nNatural Language Sentiment Analysis\nWe select the MR dataset to test the robustness of TM on\nsentiment analysis. We train our models on the standard\ntraining data and perturb the testing data to evaluate ro-\nbustness towards out of distribution changes. We monitor"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb3-934e-11ee-bf52-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "the decrease in testing accuracy with and without perturba-\ntions. During testing, an input instance (sentence) is selected\nfor perturbation with probability 0.5. Then, a word is cho-\nsen from this sentence randomly with uniform probability.\nThe selected word is swapped with the most similar word\naccording to the cosine similarity between their respective\nGlove embedding vectors. Swapping with the most similar\nword results in changing the binary feature representation"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb4-934e-11ee-aa9e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "by a hamming distance of 2, while keeping the meaning\nand sentiment of the sentence unchanged. The following is\nan MR example demonstrating this strategy: \u201cMagnificent\ndrama well worth watching\u201d \u2192\u201cSplendid drama well worth\nwatching\u201d. The word Magnificent is changed to its synonym\nSplendid. Table 8 shows the comparison between TM and\nDC TM on MR with and without test data perturbations. The\ndifference between the test accuracy on the dataset with and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb5-934e-11ee-aaf2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "without perturbations is 2.02% and1.45% for the standard\nTM and DC TM respectively. In addition, DC makes TM\nmore robust compared with BiLSTM, where the accuracy\ndrops by 1.83%. The increase in TM robustness could be\ndue to the fact that since clauses are stochastically dropped,\nthey seem to be less prone to pick up irrelevant or noisy pat-\nterns.\nImage Classification\nWe compare the robustness of CTM and DC CTM using the\nMNIST-C dataset (Mu and Gilmer 2019), which is a cor-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb6-934e-11ee-b1ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "rupted version of MNIST. It consists of 15corruptions such\nas Gaussian blur, scale, rotate, shear, impulse noise, canny\nedges, fog etc. We train our models on the standard MNIST\ntraining data without any augmentations. During testing, we\nselect whether to use non-corrupted or corrupted test image\nwith probability 0.5. We then select one of the 15corrup-\ntions to apply on the test image with uniform probability.\nDC enhances the robustness of CTM, as can be seen from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb7-934e-11ee-9a5d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "6Note that we do not test for adversarial robustness. We only\ntest the robustness of our models with corrupted input.Table 9. There is a drop of 11.18% for the standard CTM\nwhereas the drop in accuracy is reduced to 7.7%with DC.\nAlso, from Table 9, we observe that both the TM versions are\nmore robust than binarized neural networks (BNN)7, whose\naccuracy drops by 12.75%.\nDiscussion\nTo summarize our results, DC improves the performance\nof the TM, enhancing the advantages that TMs have over"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb8-934e-11ee-9a5b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "neural networks and traditional machine learning techniques\nwhen it comes to computational complexity, memory and\nenergy consumption, training and inference time and robust-\nness (Abeyrathna et al. 2021; Lei et al. 2020). The compu-\ntational advantages make TMs suitable for federated learn-\ning and deployment on edge devices. On the other hand,\nthe TM still achieves lower accuracy on the datasets we\nuse here when compared with certain deep neural network"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabb9-934e-11ee-a204-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "models. However, TM is a machine learning method that\nachieves overall good performance while being inherently\ninterpretable unlike deep neural networks that are black-\nbox models. For certain NLP tasks, pretrained models, such\nas word2vec, GloVe, BERT, and GPT, have been dominat-\ning, leveraging unlabelled data. TMs require Boolean in-\nput, making it difficult to leverage existing high dimensional\nembeddings, impeding performance, with the exception of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabba-934e-11ee-9533-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "enhancing the input with GloVe-derived synonyms (Yadav\net al. 2021). However, there is huge difference in model\nsizes, where SOTA deep neural networks have parameters\nupwards of 10million floating-point parameters, easily go-\ning beyond 100 million (BERT-Large has 340 million pa-\nrameters). On the other hand, TM employs a few thousand\nBoolean clauses. Also, the pixel-level interpretability on im-\nage classification can be difficult for humans to interpret"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabbb-934e-11ee-acfa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "as the Boolean expressions for clauses capturing patterns\ncan be quite long, containing numerous pixels, especially\nfor large images. Nevertheless, there are several O(n)tech-\nniques that can easily simplify and reduce Boolean expres-\nsions (Meurer et al. 2017).\nConclusions\nIn this paper, we propose drop clause as a technique to im-\nprove the generalization ability of the TM. Drop clause en-\nhances stochasticity during training, to capture more diverse"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabbc-934e-11ee-bbe4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "patterns. As a result, the performance of TM is significantly\nboosted, which is empirically shown on a varied collection\nof datasets, both in terms of accuracy and training time. We\nfurther show how drop clause improves the pattern recogni-\ntion capabilities of TM, simultaneously improving the clar-\nity of the produced patterns. When comparing our model\nwith the state-of-the-art deep learning models, we observe\ncompetitive accuracy levels. Since TM is more akin to tradi-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabbd-934e-11ee-bf7c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 6, "chroma:document": "tional machine learning techniques, we also compare it with\na selection of those, reporting superior performance of the\nTM. We finally establish that drop clause improves the ro-\nbustness of the TM towards data corruptions and perturba-\ntions during testing.\n7The MNIST-MLP architecture from (Jia and Rinard 2020) is\nused here.\n13553"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabbe-934e-11ee-93c4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "References\nAbeyrathna, K. D.; Bhattarai, B.; Goodwin, M.; Gorji, S. R.;\nGranmo, O.-C.; Jiao, L.; Saha, R.; and Yadav, R. K. 2021.\nMassively Parallel and Asynchronous Tsetlin Machine Ar-\nchitecture Supporting Almost Constant-Time Scaling. In\nMeila, M.; and Zhang, T., eds., Proceedings of the 38th\nInternational Conference on Machine Learning, volume\n139 of Proceedings of Machine Learning Research, 10\u201320.\nPMLR.\nAbeyrathna, K. D.; Granmo, O.-C.; and Goodwin, M."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabbf-934e-11ee-b437-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "Abeyrathna, K. D.; Granmo, O.-C.; and Goodwin, M.\n2021. Extending the Tsetlin Machine With Integer-Weighted\nClauses for Increased Interpretability. IEEE Access, 9.\nAbeyrathna, K. D.; Pussewalage, H. S. G.; Ranasinghea,\nS. N.; Oleshchuk, V . A.; and Granmo, O.-C. 2020. Intru-\nsion Detection with Interpretable Rules Generated Using the\nTsetlin Machine. In 2020 IEEE Symposium Series on Com-\nputational Intelligence (SSCI). IEEE.\nBau, D.; Zhou, B.; Khosla, A.; Oliva, A.; and Torralba, A."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc0-934e-11ee-b3db-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "2017. Network Dissection: Quantifying Interpretability of\nDeep Visual Representations. CoRR, abs/1704.05796.\nDevlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.\nBERT: Pre-training of Deep Bidirectional Transformers for\nLanguage Understanding. In Proceedings of the 2019 Con-\nference of the North American Chapter of the Association\nfor Computational Linguistics: Human Language Technolo-\ngies, Volume 1 (Long and Short Papers), 4171\u20134186. Min-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc1-934e-11ee-962f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "neapolis, Minnesota: Association for Computational Lin-\nguistics.\nGhorbani, A.; Abid, A.; and Zou, J. 2019. Interpretation\nof Neural Networks Is Fragile. Proceedings of the AAAI\nConference on Artificial Intelligence, 33(01): 3681\u20133688.\nGoodfellow, I.; Warde-Farley, D.; Mirza, M.; Courville, A.;\nand Bengio, Y . 2013. Maxout Networks. In Dasgupta, S.;\nand McAllester, D., eds., Proceedings of the 30th Interna-\ntional Conference on Machine Learning, volume 28 of Pro-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc2-934e-11ee-855f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "ceedings of Machine Learning Research, 1319\u20131327. At-\nlanta, Georgia, USA: PMLR.\nGranmo, O.-C. 2018. The Tsetlin Machine - A Game\nTheoretic Bandit Driven Approach to Optimal Pattern\nRecognition with Propositional Logic. arXiv preprint\narXiv:1804.01508.\nGranmo, O.-C.; Glimsdal, S.; Jiao, L.; Goodwin, M.; Omlin,\nC. W.; and Berge, G. T. 2019. The Convolutional Tsetlin\nMachine. arXiv preprint arXiv:1905.09688.\nJain, S.; and Wallace, B. C. 2019. Attention is not Ex-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc3-934e-11ee-bfc7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "planation. In Proceedings of the 2019 Conference of the\nNorth American Chapter of the Association for Computa-\ntional Linguistics: Human Language Technologies, Volume\n1 (Long and Short Papers), 3543\u20133556. Minneapolis, Min-\nnesota: Association for Computational Linguistics.\nJeevan, P.; and Sethi, A. 2021. Vision Xformers: Efficient\nAttention for Image Classification. arXiv:2107.02239.\nJia, K.; and Rinard, M. 2020. Efficient Exact Verification of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc4-934e-11ee-8d4a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "Binarized Neural Networks. In Larochelle, H.; Ranzato, M.;\nHadsell, R.; Balcan, M. F.; and Lin, H., eds., Advances in\nNeural Information Processing Systems, volume 33, 1782\u2013\n1795. Curran Associates, Inc.Jiao, L.; Zhang, X.; Granmo, O.-C.; and Abeyrathna, K. D.\n2022. On the Convergence of Tsetlin Machines for the XOR\nOperator. IEEE Trans. Pattern Anal. Mach. Intell.\nKim, Y . 2014. Convolutional Neural Networks for Sen-\ntence Classification. In Proceedings of the 2014 Confer-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc5-934e-11ee-b0d3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "ence on Empirical Methods in Natural Language Processing\n(EMNLP), 1746\u20131751. Doha, Qatar: Association for Com-\nputational Linguistics.\nLei, J.; Rahman, T.; Shafik, R.; Wheeldon, A.; Yakovlev, A.;\nGranmo, O.-C.; Kawsar, F.; and Mathur, A. 2021. Low-\nPower Audio Keyword Spotting Using Tsetlin Machines.\nJournal of Low Power Electronics and Applications, 11.\nLei, J.; Wheeldon, A.; Shafik, R.; Yakovlev, A.; and\nGranmo, O.-C. 2020. From Arithmetic to Logic based AI: A"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc6-934e-11ee-ac72-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "Comparative Analysis of Neural Networks and Tsetlin Ma-\nchine. In 2020 27th IEEE International Conference on Elec-\ntronics, Circuits and Systems (ICECS), 1\u20134.\nMeurer, A.; Smith, C. P.; Paprocki, M.; \u02c7Cert\u00b4\u0131k, O.; Kir-\npichev, S. B.; Rocklin, M.; Kumar, A.; Ivanov, S.; Moore,\nJ. K.; Singh, S.; Rathnayake, T.; Vig, S.; Granger, B. E.;\nMuller, R. P.; Bonazzi, F.; Gupta, H.; Vats, S.; Johansson,\nF.; Pedregosa, F.; Curry, M. J.; Terrel, A. R.; Rou \u02c7cka, v.; Sa-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc7-934e-11ee-be04-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "boo, A.; Fernando, I.; Kulal, S.; Cimrman, R.; and Scopatz,\nA. 2017. SymPy: symbolic computing in Python. PeerJ\nComputer Science, 3: e103.\nMu, N.; and Gilmer, J. 2019. MNIST-C: A Robustness\nBenchmark for Computer Vision. CoRR, abs/1906.02337.\nNarodytska, N.; Kasiviswanathan, S.; Ryzhyk, L.; Sagiv, M.;\nand Walsh, T. 2018. Verifying properties of binarized deep\nneural networks. In Proceedings of the AAAI Conference on\nArtificial Intelligence, volume 32."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc8-934e-11ee-8326-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "Artificial Intelligence, volume 32.\nNoveck, I.; Lea, R. B.; Davidson, G. M.; and O\u2019brien, D.\n1991. HUMAN REASONING IS BOTH LOGICAL AND\nPRAGMATIC. Intellectica, 11: 81\u2013109.\nQin, Q.; Hu, W.; and Liu, B. 2020. Feature Projection for\nImproved Text Classification. In Proceedings of the 58th\nAnnual Meeting of the Association for Computational Lin-\nguistics, 8161\u20138171. Online: Association for Computational\nLinguistics.\nRadhakrishnan, A.; Durham, C.; Soylemezoglu, A.; and Uh-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabc9-934e-11ee-b76e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "ler, C. 2018. Patchnet: Interpretable Neural Networks for\nImage Classification. arXiv:1705.08078.\nRibeiro, M. T.; Singh, S.; and Guestrin, C. 2016. \u201dWhy\nShould I Trust You?\u201d: Explaining the Predictions of Any\nClassifier. In Proceedings of the 22nd ACM SIGKDD In-\nternational Conference on Knowledge Discovery and Data\nMining, KDD \u201916, 1135\u20131144. New York, NY , USA: Asso-\nciation for Computing Machinery. ISBN 9781450342322.\nRobbins, H. 2007. A Stochastic Approximation Method."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabca-934e-11ee-b1a9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 7, "chroma:document": "Annals of Mathematical Statistics, 22: 400\u2013407.\nRudin, C. 2019. Stop Explaining Black Box Machine Learn-\ning Models for High Stakes Decisions and Use Interpretable\nModels Instead. Nature Machine Intelligence, 1.\nSelvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.;\nParikh, D.; and Batra, D. 2017. Grad-CAM: Visual Explana-\ntions from Deep Networks via Gradient-Based Localization.\n13554"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabcb-934e-11ee-99d7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 8, "chroma:document": "In2017 IEEE International Conference on Computer Vision\n(ICCV), 618\u2013626.\nSimonyan, K.; Vedaldi, A.; and Zisserman, A. 2014. Deep\nInside Convolutional Networks: Visualising Image Classifi-\ncation Models and Saliency Maps. arXiv:1312.6034.\nSrivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; and\nSalakhutdinov, R. 2014. Dropout: A Simple Way to Pre-\nvent Neural Networks from Overfitting. Journal of Machine\nLearning Research, 15(56): 1929\u20131958."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabcc-934e-11ee-aa6a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 8, "chroma:document": "Learning Research, 15(56): 1929\u20131958.\nTang, J.; Qu, M.; and Mei, Q. 2015. PTE: Predictive Text\nEmbedding through Large-scale Heterogeneous Text Net-\nworks. CoRR, abs/1508.00200.\nValiant, L. G. 1984. A Theory of the Learnable. Communi-\ncations of the ACM, 27(11): 1134\u20131142.\nWebster, M. B.; Choi, J.; and changwook Ahn. 2021. Learn-\ning the Connections in Direct Feedback Alignment. In In-\nternational Conference on Learning Representations.\nYadav, R. K.; Jiao, L.; Granmo, O.-C.; and Goodwin, M."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabcd-934e-11ee-a4a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 8, "chroma:document": "2021. Enhancing Interpretable Clauses Semantically using\nPretrained Word Representation. In BLACKBOXNLP.\nYadav, R. K.; Jiao, L.; Granmo, O.-C.; and Goodwin, M.\n2021. Human-Level Interpretable Learning for Aspect-\nBased Sentiment Analysis. In The Thirty-Fifth AAAI Con-\nference on Artificial Intelligence (AAAI-21). AAAI.\nYadav, R. K.; Jiao, L.; Granmo, O.-C.; and Goodwin, M.\n2022. Robust Interpretable Text Classification against Spu-\nrious Correlations Using AND-rules with Negation. In Inter-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabce-934e-11ee-a41c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 8, "chroma:document": "national Joint Conference on Artificial Intelligence (IJCAI) .\nZhang, Q.; Wu, Y . N.; and Zhu, S.-C. 2018. Interpretable\nconvolutional neural networks. In Proceedings of the IEEE\nConference on Computer Vision and Pattern Recognition ,\n8827\u20138836.\nZhang, X.; Jiao, L.; Granmo, O.-C.; and Goodwin, M.\n2022. On the Convergence of Tsetlin Machines for the\nIDENTITY- and NOT Operators. IEEE Trans. Pattern Anal.\nMach. Intell., 44(10): 6345\u20136359.\nZhu, H.; and Koniusz, P. 2020. Simple spectral graph con-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabcf-934e-11ee-9eac-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\26588-Article Text-30651-1-2-20230626.pdf", "page": 8, "chroma:document": "volution. In International Conference on Learning Repre-\nsentations.\n13555"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd0-934e-11ee-8b7b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "electronics\nArticle\nAdaptive Sparse Representation of Continuous Input for\nTsetlin Machines Based on Stochastic Searching on the Line\nKuruge Darshana Abeyrathna *, Ole-Christoffer Granmo and Morten Goodwin\n/gid00030/gid00035/gid00032/gid00030/gid00038/gid00001/gid00033/gid00042/gid00045 /gid00001\n/gid00048/gid00043/gid00031/gid00028/gid00047/gid00032/gid00046\nCitation: Abeyrathna, K.D.; Granmo,\nO.-C.; Goodwin, M. Adaptive Sparse\nRepresentation of Continuous Input\nfor Tsetlin Machines Based on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd1-934e-11ee-993c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "for Tsetlin Machines Based on\nStochastic Searching on the Line.\nElectronics 2021 ,10, 2107. https://\ndoi.org/10.3390/electronics10172107\nAcademic Editor: Dimitris Apostolou\nReceived: 1 July 2021\nAccepted: 25 August 2021\nPublished: 30 August 2021\nPublisher\u2019s Note: MDPI stays neutral\nwith regard to jurisdictional claims in\npublished maps and institutional af\ufb01l-\niations.\nCopyright: \u00a9 2021 by the authors.\nLicensee MDPI, Basel, Switzerland.\nThis article is an open access article"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd2-934e-11ee-b382-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "This article is an open access article\ndistributed under the terms and\nconditions of the Creative Commons\nAttribution (CC BY) license (https://\ncreativecommons.org/licenses/by/\n4.0/).Centre for Arti\ufb01cial Intelligence Research, University of Agder, 4870 Grimstad, Norway;\nole.granmo@uia.no (O.-C.G.); morten.goodwin@uia.no (M.G.)\n*Correspondence: darshana.abeyrathna@uia.no\nAbstract: This paper introduces a novel approach to representing continuous inputs in Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd3-934e-11ee-b2ad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "Machines (TMs). Instead of using one Tsetlin Automaton (TA) for every unique threshold found\nwhen Booleanizing continuous input, we employ two Stochastic Searching on the Line (SSL) automata\nto learn discriminative lower and upper bounds. The two resulting Boolean features are adapted to\nthe rest of the clause by equipping each clause with its own team of SSLs, which update the bounds\nduring the learning process. Two standard TAs \ufb01nally decide whether to include the resulting"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd4-934e-11ee-bc5d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "features as part of the clause. In this way, only four automata altogether represent one continuous\nfeature (instead of potentially hundreds of them). We evaluate the performance of the new scheme\nempirically using \ufb01ve datasets, along with a study of interpretability. On average, TMs with SSL\nfeature representation use 4.3times fewer literals than the TM with static threshold-based features.\nFurthermore, in terms of average memory usage and F1-Score, our approach outperforms simple"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd5-934e-11ee-ad95-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "Multi-Layered Arti\ufb01cial Neural Networks, Decision Trees, Support Vector Machines, K-Nearest\nNeighbor, Random Forest, Gradient Boosted Trees (XGBoost), and Explainable Boosting Machines\n(EBMs), as well as the standard and real-value weighted TMs. Our approach further outperforms\nNeural Additive Models on Fraud Detection and StructureBoost on CA-58 in terms of the Area Under\nCurve while performing competitively on COMPAS."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd6-934e-11ee-b92a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "Curve while performing competitively on COMPAS.\nKeywords: Tsetlin Machine; Tsetlin automata; Stochastic Searching on the Line automaton; inter-\npretable AI; interpretable machine learning; XAI; rule-based learning; decision support system\n1. Introduction\nDeep learning (DL) has signi\ufb01cantly advanced state-of-the-art models in machine\nlearning (ML) over the last decade, attaining remarkable accuracy in many ML application"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd7-934e-11ee-94e3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "domains. One of the issues with DL, however, is that DL inference cannot easily be inter-\npreted [ 1]. This limits the applicability of DL in high-stakes domains such as medicine [ 2,3],\ncredit-scoring [ 4,5], churn prediction [ 6,7], bioinformatics [ 8,9], crisis analysis [ 10], and\ncriminal justice [ 11]. In this regard, the simpler and more interpretable ML algorithms,\nsuch as Decision Trees, Logistic Regression, Linear Regression, and Decision Rules, can"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd8-934e-11ee-8220-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "be particularly suitable. However, they are all hampered by low accuracy when facing\ncomplex problems [ 12]. This limitation has urged researchers to develop machine learning\nalgorithms that are capable of achieving a better trade-off between interpretability and\naccuracy.\nWhile some researchers focus on developing entirely new machine learning algo-\nrithms, as discussed above, other researchers try to render DL interpretable. A recent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabd9-934e-11ee-ad91-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "attempt to make DL interpretable is the work of Agarwal et al. [ 11]. They introduce Neural\nAdditive Models (NAMs), which treat each feature independently. The assumption of\nindependence makes NAMs interpretable but impedes accuracy compared with regular\nDL [ 11]. Another approach is to try to explain DL inference with surrogate models. Here,\none strives to attain local interpretability; i.e., explaining individual predictions [ 13]. Nev-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabda-934e-11ee-be00-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 0, "chroma:document": "Electronics 2021 ,10, 2107. https://doi.org/10.3390/electronics10172107 https://www.mdpi.com/journal/electronics"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabdb-934e-11ee-b1b4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "Electronics 2021 ,10, 2107 2 of 25\nertheless, these explanations are only approximate and cannot explain the complete DL\nmodel (global interpretability) [14].\nMany prominent interpretable ML approaches are based on natively interpretable\nrules, tracing back to some of the well-known learning models such as association rule\nlearning [ 15]. These have for instance been used to predict sequential events [ 16]. Other"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabdc-934e-11ee-93f4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "examples include the work of Feldman on the dif\ufb01culty of learning formulae in Disjunctive\nNormal Form (DNF) [ 17] and Probably Approximately Correct (PAC) learning, which has\nprovided fundamental insights into machine learning as well as a framework for learning\nformulae in DNF [ 18]. Approximate Bayesian techniques are another set of approaches\nfor the robust learning of rules [ 19,20]. Hybrid Logistic Circuits (HLC), introduced in [ 21],"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabdd-934e-11ee-9d5a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "are yet another novel approach to interpretable machine learning. Here, layered logical\noperators translate into a logistic regression function. HLC has demonstrated promising\naccuracy in image classi\ufb01cation. However, in general, rule-based machine learning scales\npoorly and is prone to noise. Indeed, for data-rich problems, in particular those involving\nnatural language and sensory inputs, rule-based machine learning is inferior to DL."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabde-934e-11ee-b5dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "Another recent interpretable approach to machine learning is Explainable Boosting\nMachines (EBMs) [ 22]. EBMs are highly intelligible and explainable, while their accuracy\nis comparable to state-of-the-art machine learning methods such as Random Forest and\nBoosted Trees [ 23]. Indeed, EBMs are recognized as state-of-the-art within Generalized\nAdditive Models (GAMs) [ 22,23]. The EBMs learn feature functions independently, using"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabdf-934e-11ee-a715-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "methods such as gradient boosting or bagging. This allows the user to determine how\nmuch each feature contributes to the model\u2019s prediction and is hence directly interpretable.\nDespite being rule-based, the recently introduced Tsetlin Machines (TMs) [24] have\nobtained competitive accuracy in a wide range of domains [ 25\u201329], while producing\nhuman-interpretable outputs. At the same time, TMs utilize comparably low computa-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe0-934e-11ee-8399-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "tional resources [ 30]. Employing a team of TAs [ 31], a TM learns a linear combination of\nconjunctive clauses in propositional logic, producing decision rules similar to the branches\nin a decision tree (e.g., ifXsatis\ufb01es condition A and not condition B then Y = 1) [ 26]. In\nother words, the TM can be said to unify logistic regression and rule-based learning in a\nway that boosts accuracy while maintaining interpretability."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe1-934e-11ee-860e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "Regarding recent progress on TMs, they have recently been adopted for various\napplication domains such as natural language understanding [ 32,33], image classi\ufb01ca-\ntion [ 34], and speech processing [ 35]. Simultaneously, the TM architecture and learning\nmechanism has been improved in terms of accuracy, computation speed, and energy usage.\nThe convolutional TM provides competitive performance on MNIST, Fashion-MNIST,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe2-934e-11ee-b1d0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "and Kuzushiji-MNIST, in comparison with CNNs, K-Nearest Neighbor, Support Vector\nMachines, Random Forests, Gradient Boosting, Binary Connect, Logistic Circuits, and\nResNet [ 36]. The regression TM [ 27] opens up for continuous output, achieving on par or\nbetter performance compared to Random Forest, Regression Trees, and Support Vector\nRegression. With the proposed Booleanization scheme in [ 37], the TM is also able to operate\nwith continuous features."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe3-934e-11ee-82e6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "with continuous features.\nFurthermore, clauses are enhanced with weights in [ 25]. The weights reduce the\nnumber of clauses required without any loss of accuracy. Later, integer weights replaced\nreal-valued weights to both reduce the number of clauses and to increase the interpretability\nof the TM [ 38]. On several benchmarks, the integer-weighted TM version outperformed\nsimple Multi-Layered Arti\ufb01cial Neural Networks, Support Vector Machines, Decision Trees,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe4-934e-11ee-8ef5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "Random Forest, K-Nearest Neighbor, Random Forest, Explainable Boosting Machines\n(EBMs), and Gradient Boosted Trees (XGBoost), as well as the standard TM. Further, the\nintroduced multi-granular clauses for TM in [ 28] eliminate the pattern speci\ufb01city parameter\nfrom the TM, consequently simplifying the hyper-parameter search. By indexing the\nclauses on the features that falsify them, up to an order of magnitude faster inference and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe5-934e-11ee-88d7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 1, "chroma:document": "learning has been reported [ 29]. Several researchers have further introduced techniques and\narchitectures that reduce memory footprint and energy usage [ 39], while other techniques\nimprove learning speed [39,40] and support explainability [41,42]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe6-934e-11ee-ae8a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "Electronics 2021 ,10, 2107 3 of 25\nRecent theoretical work proves the convergence to the correct operator for \u201cidentity\u201d\nand \u201cnot\u201d. It is further shown that arbitrarily rare patterns can be recognized using a\nquasi-stationary Markov chain-based analysis. The work \ufb01nally proves that when two\npatterns are incompatible, the most accurate pattern is selected [ 43]. Convergence for the\n\u201cXOR\u201d operator has also recently been proven by Jiao et al. [44]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe7-934e-11ee-a81e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "The approach proposed in [ 37] is currently the most effective way of representing\ncontinuous features through Booleanization. However, the approach requires a large\nnumber of TAs to represent the Booleanized continuous features. Indeed, one TA is needed\nper unique continuous value. Consequently, this increases the training time of the TM\nas it needs to update all the TAs in all of clauses for each training iteration. Further, this"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe8-934e-11ee-aa98-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "adds more post-processing work for generating interpretable rules out of TM outputs. To\novercome this challenge in TMs, we propose a novel approach to represent continuous\nfeatures in the TM, encompassing the following contributions:\n\u2022 Instead of representing each unique threshold found in the Booleanization process by\na TA, we use a Stochastic Searching on the Line (SSL) automaton [ 45] to learn the lower\nand upper limits of the continuous feature values. These limits decide the Boolean"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabe9-934e-11ee-9c36-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "representation of the continuous value inside the clause. Only two TAs then decide\nwhether to include these bounds in the clause or to exclude them from the clause. In\nthis way, one continuous feature can be represented by only four automata, instead of\nrepresenting it by hundreds of TAs (decided by the number of unique feature values\nwithin the feature).\n\u2022 A new approach to calculating the clause output is introduced to match with the\nabove Booleanization scheme."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabea-934e-11ee-a0a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "above Booleanization scheme.\n\u2022 We update the learning procedure of the TM accordingly, building upon Type I and\nType II feedback to learn the lower and upper bounds of the continuous input.\n\u2022 Empirically, we evaluate our new scheme using eight data sets: Bankruptcy, Balance\nScale, Breast Cancer, Liver Disorders, Heart Disease, Fraud Detection, COMPAS,\nand CA-58. With the \ufb01rst \ufb01ve datasets, we show how our novel approach affects"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabeb-934e-11ee-852b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "memory consumption, training time, and the number of literals included in clauses, in\ncomparison with the threshold-based scheme [ 46]. Furthermore, performances on all\nthese datasets are compared against recent state-of-the-art machine learning models.\nThis paper is organized as follows. In Section 2, we present the learning automata\nfoundation we build upon and discuss the SSL automaton in more detail. Then, in Section 3 ,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabec-934e-11ee-992e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "we introduce the TM and how it traditionally has dealt with continuous features. We then\npropose our new SSL-based scheme. We evaluate the performance of our new scheme\nempirically using \ufb01ve datasets in Section 5. In this section, we use the Bankruptcy dataset\nto demonstrate how rules are extracted from TM clause outputs. The prediction accuracy\nof the TM SSL-based continuous feature representation is then compared against several"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabed-934e-11ee-92d6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "competing techniques, including ANNs, SVMs, DTs, RF, KNN, EBMs (the current state-of-\nthe-art of Generalized Additive Models (GAMs) [ 22,23]), Gradient Boosted Trees (XGBoost),\nand the TM with regular continuous feature representation. Further, we contrast the\nperformance of the TM against reported results on recent state-of-the-art machine learning\nmodels, namely NAMs [ 11] and StructureBoost [ 47]. Finally, we conclude our paper in\nSection 6."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabee-934e-11ee-aec7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "Section 6.\n2. Learning Automata and the Stochastic Searching on the Line Automaton\nThe origins of Learning Automata (LA) [ 48] can be traced back to the work of M. L.\nTsetlin in the early 1960s [ 31]. In a stochastic environment, an automaton is capable of\nlearning the optimum action that has the lowest penalty probability through trial and error.\nThere are different types of automata; the choice of a speci\ufb01c type is decided by the nature\nof the application [49]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabef-934e-11ee-96fb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 2, "chroma:document": "of the application [49].\nInitially, the LA randomly perform an action from an available set of actions. This\naction is then evaluated by its attached environment. The environment randomly produces\nfeedback; i.e., a reward or a penalty as a response to the action selected by the LA. De-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf0-934e-11ee-a028-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "Electronics 2021 ,10, 2107 4 of 25\npending on the feedback, the state of the LA is adjusted. If the feedback is a reward, the\nstate changes towards the end state of the selected action, reinforcing the action. When\nthe feedback is a penalty, the state changes towards the center state of the selected action,\nweakening the action and eventually switching the action. The next action of the automaton\nis then decided by the new state. In this manner, an LA interacts with its environment"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf1-934e-11ee-8747-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "iteratively. With a suf\ufb01ciently large number of states and a reasonably large number of\ninteractions with the environment, an LA learns to choose the optimum action with a\nprobability arbitrarily close to 1.0 [48].\nDuring LA learning, the automaton can make deterministic or stochastic jumps as a\nresponse to the environment feedback. LA make stochastic jumps by randomly changing\nstates according to a given probability. If this probability is 1.0, the state jumps are deter-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf2-934e-11ee-a3f6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "ministic. Automata of this kind are called deterministic automata. If the transition graph of\nthe automaton is kept static, we refer to it as a \ufb01xed-structure automaton. The TM employs\nTAs to decide which literals to include in the clauses. A TA is deterministic and has a \ufb01xed\nstructure, formulated as a \ufb01nite-state automaton [ 31]. A TA with 2Nstates is depicted in\nFigure 1. States 1 to Nmap to Action 1 and states N+1 to 2 Nmap to Action 2."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf3-934e-11ee-bf8b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "1         2    .....\u2026   N-1      N                  N+1    N+2  \u2026\u2026.  2N -1    2N  Action 1          Action 2         \nReward  \nPenalty  \nFigure 1. Transition graph of a two-action Tsetlin Automaton with 2N memory states.\nThe Stochastic Searching on the Line (SSL) automaton pioneered by Oommen [ 45] is\nsomewhat different from regular automata. The SSL automaton is an optimization scheme"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf4-934e-11ee-b1cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "designed to \ufb01nd an unknown optimum location \u03bb\u2217on a line, seeking a value between 0to\n1,[0, 1].\nIn SSL learning, \u03bb\u2217can be one of the Npoints. In other words, the search space is\ndivided into Npoints, { 0, 1/N, 2/N,. . .,(N\u22121)/N, 1}, with Nbeing the discretization\nresolution. Depending on the possibly faulty feedback from the attached environment ( E),\n\u03bbmoves towards the left or right from its current state on the created discrete search space."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf5-934e-11ee-8b9d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "We consider the environment feedback 1, E=1, as an indication to move towards right (or\nto increase the value of \u03bb) by one step. The environment feedback 0, E=0, on the other\nhand, is considered as an indication to move towards the left (or to decrease the value of \u03bb)\nby one step. The next location of \u03bb,\u03bb(n+1)can thus be expressed as follows:\n\u03bb(n+1) ={\n\u03bb(n) +1/N, if E(n) =1 and 0 \u2a7d\u03bb(n)<1 ,\n\u03bb(n)\u22121/N, if E(n) =0 and 0 <\u03bb(n)\u2a7d1 .(1)\n\u03bb(n+1) ={\n\u03bb(n), if \u03bb(n) =1 and E(n) =1 ,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf6-934e-11ee-a5fb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "\u03bb(n+1) ={\n\u03bb(n), if \u03bb(n) =1 and E(n) =1 ,\n\u03bb(n), if \u03bb(n) =0 and E(n) =0 .(2)\nAsymptotically, the learning mechanism is able to \ufb01nd a value arbitrarily close to \u03bb\u2217\nwhen N\u2192\u221eand n\u2192\u221e.\n3. Tsetlin Machine (TM) for Continuous Features\nAs seen in Figure 2, conceptually, the TM decomposes into \ufb01ve layers to recognize\nsub-patterns in the data and categorize them into classes. In this section, we explain the\njob of each of these layers in the pattern recognition and learning phases of the TM. The"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf7-934e-11ee-8333-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 3, "chroma:document": "parameters and symbols used in this section are explained and summarized in Table 1."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf8-934e-11ee-9031-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 4, "chroma:document": "Electronics 2021 ,10, 2107 5 of 25\n \n \n \n \n \n \n \n \n \n \n      \n \n \n \n \n \n \n \n \n \n  \n \n \n \n \ud835\udc73=[\ud835\udc651,\ud835\udc652,\ud835\udc653\u2026\ud835\udc65\ud835\udc5c,\u00ac\ud835\udc651,\u00ac\ud835\udc652,\u00ac\ud835\udc653\u2026\u00ac\ud835\udc65\ud835\udc5c] \u2026\u2026\u2026.  Clause -1            \n           \n           \n           \n                                                                               (\ud835\udc5a\u00d72\ud835\udc5c) \ud835\udc58 \n\ud835\udc57 \ud835\udc4e\ud835\udc57,\ud835\udc58 \ud835\udc68= \ud835\udc4e\ud835\udc57,\ud835\udc58   \n\ud835\udc4e1,\ud835\udc58\u2208{1,\u20262\ud835\udc41}   Clause -2 \n\ud835\udc4e2,\ud835\udc58\u2208{1,\u20262\ud835\udc41}   Clause -m \n\ud835\udc4e\ud835\udc5a,\ud835\udc58\u2208{1,\u20262\ud835\udc41}   1/0 1/0 1/0 \u2026\u2026\u2026.  \ud835\udc63= \u2211\ud835\udc50\ud835\udc57+\u2212 \u2211\ud835\udc50\ud835\udc57\u2212\n\ud835\udc57 \ud835\udc57    \n- - + \nLayer 1  Layer 2 Layer 3 Layer 4 Layer 5 \nFigure 2. The TM structure."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabf9-934e-11ee-80a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 4, "chroma:document": "Figure 2. The TM structure.\nTable 1. Parameters and symbols used in Section 3.\nParameter/Symbol Description Parameter/Symbol Description\nXInput vector containing\nopropositional variablesLThe augmented feature set containing\nboth original and negated features\nxkkthpropositional\nvariablelk kthliteral\nm The number of clauses cj jthclause and stored in vector C\nIjThe index set of the included\nliterals in clause jNThe number of states per action\nin the TA\naj,kState of the kthliteral"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabfa-934e-11ee-8832-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 4, "chroma:document": "in the TA\naj,kState of the kthliteral\nin the jthclause, stored\nin matrix AI1\nXThe indexes of the literals of\nvalue 1\nvThe difference between positive\nand negative clause outputspjDecision on receiving Type I or Type II\nfeedback, stored in vector P\nT Feedback threshold s Learning sensitivity\nrj,kThe decision whether the kth\nTA of the jthclause is to receive\nType Ia feedback, stored in\nmatrix Rqj,kThe decision whether the kth\nTA of the jthclause is to receive\nType Ib feedback, stored in\nmatrix Q"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabfb-934e-11ee-8542-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 4, "chroma:document": "Type Ib feedback, stored in\nmatrix Q\nIIa Stores TA indexes selected for\nType Ia feedbackIIb Stores TA indexes selected for\nType Ib feedback\n\u2295Denotes adding 1 to the current\nstate value of the TA\u2296Denotes substracting 1 from the current\nstate value of the TA\nIII Stores TA indexes selected for\nType II feedback"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabfc-934e-11ee-9ccd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "Electronics 2021 ,10, 2107 6 of 25\nLayer 1: the input. In the input layer, the TM receives a vector of opropositional\nvariables: X,xk\u2208{0, 1}o. The objective here of the TM is to classify this feature vector\ninto one of the two classes, y\u2208{0, 1}. However, as shown in Figure 2, the input layer\nalso includes negations of the original features, \u00acxk, in the feature set to capture more\nsophisticated patterns. Collectively, the elements in the augmented feature set are called"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabfd-934e-11ee-b316-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "literals: L= [x1,x2, . . . , xo,\u00acx1,\u00acx2, . . . ,\u00acxo] = [ l1,l2, . . . , l2o].\nLayer 2: clause construction. The sub-patterns associated with class 1and class 0are\ncaptured by mconjunctive clauses. The value mis set by the user where more complex\nproblems might demand a large m. All clauses receive the same augmented feature set\nformulated at the input layer, L. However, to perform the conjunction, only a fraction of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabfe-934e-11ee-b97e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "literals is utilized. The TM employs two-action TAs in Figure 1 to decide which literals are\nincluded in which clauses. Since we found a number 2\u00d7oof literals in L, the same number\nof TAs\u2014one per literal k\u2014is needed by a clause to decide the included literals in the clause.\nWhen the index set of the included literals in clause jis given in Ij, the conjunction of the\nclause can be performed as follows:\ncj=\u22c0\nk\u2208Ijlk. (3)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cabff-934e-11ee-a4a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "cj=\u22c0\nk\u2208Ijlk. (3)\nNotice how the composition of a clause varies from another clause depending on\nthe indexes of the included literals in the set Ij\u2286{1,. . ., 2o}. For the special case of\nIj=\u2205\u2014i.e., an empty clause\u2014we have\ncj={1during learning\n0otherwise .(4)\nThat is, during learning, empty clauses output 1, and during classi\ufb01cation, they output 0.\nLayer 3: storing states of TAs of clauses in the memory. The TA states on the left-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac00-934e-11ee-a4f6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "hand side of the automaton (states from 1toN) ask to exclude the corresponding literal\nfrom the clause while the states on the right-hand side of the automaton (states from N+1\nto2N) ask to include the literal in the clause. The systematic storage of states of TAs in the\nmatrix, A:A= (aj,k)\u2208{1,. . ., 2N}m\u00d72o, with jreferring to the clause and kto the literal,\nallows us to \ufb01nd the index set of the included literals in clause j,IjasIj={k|aj,k>N,\n1\u2264k\u22642o}."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac01-934e-11ee-a5d4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "1\u2264k\u22642o}.\nLayer 4: clause output. Once the TA decisions are available, the clause output can be\neasily computed. Since the clauses are conjunctive, a single literal of value 0is enough to\nturn the clause output to 0if its corresponding TA has decided to include it in the clause.\nTo make the understanding easier, we introduce set I1\nX, which contains the indexes of the\nliterals of value 1. Then, the output of clause jcan be expressed as\ncj={\n1 if Ij\u2286I1\nX,\n0 otherwise.(5)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac02-934e-11ee-9de9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "cj={\n1 if Ij\u2286I1\nX,\n0 otherwise.(5)\nThe clause outputs, computed as above, are now stored in vector C, i.e., C= (cj)\u2208\n{0, 1}m.\nLayer 5: classi\ufb01cation: The TM structure given in Figure 2 is used to classify data into\ntwo classes. Hence, sub-patterns associated with each class have to be separately learned.\nFor this purpose, the clauses are divided into two groups, where one group learns the\nsub-pattern of class 1while the other learns the sub-patterns of class 0. For simplicity,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac03-934e-11ee-b090-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "clauses with an odd index are assigned a positive polarity ( c+\nj), and they are used to capture\nsub-patterns of output y=1. Clauses with an even index, on the other hand, are assigned\nnegative polarity ( c\u2212\nj) and they seek the sub-patterns of output y=0.\nThe clauses that recognize sub-patterns output 1. This makes the classi\ufb01cation process\neasier as we simply need to sum the clause outputs of each class and assign the sample"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac04-934e-11ee-8450-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 5, "chroma:document": "into the class which has the highest sum. A higher sum means that more sub-patterns are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac05-934e-11ee-91e4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "Electronics 2021 ,10, 2107 7 of 25\nidenti\ufb01ed from the designated class and that there is a higher chance of the sample being\nin that class. Hence, with vbeing the difference in clause output, v=\u2211jc+\nj\u2212\u2211jc\u2212\nj, the\noutput of the TM is decided as follows:\n\u02c6y={\n1 if v\u22650\n0 if v<0 .(6)\nA TM learns online, updating its internal parameters according to one training sample\n(X,y)at a time. As we discussed, a TA team decides the clause output, and collectively, the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac06-934e-11ee-882f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "output of all the clauses decides the TM\u2019s output. Hence, to maximize the accuracy of the\nTM\u2019s output, it is important to sensibly guide individual TAs in clauses. We achieve this\nwith two kinds of reinforcement: Type I and Type II feedback. Type I and Type II feedback\ndecide if the TAs in clauses receive a reward, a penalty, or inaction feedback, depending\non the context of their actions. How the type of feedback is decided and how the TAs are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac07-934e-11ee-bc46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "updated according to the selected feedback type is discussed below in more details.\nType I feedback: Type I feedback has been designed to reinforce the true positive\noutputs of the clauses and to combat against the false negative outputs of the clauses. To\nreinforce the true positive output of a clause (the clause output is 1 when it has to be 1),\ninclude actions of TAs whose corresponding literal value is 1 are strengthened. However,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac08-934e-11ee-800d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "more \ufb01ne-tuned patterns can be identi\ufb01ed by strengthening the exclude actions of TAs in the\nsame clause whose corresponding literal value is 0. To combat the false negative outputs\nof the clauses (the clause output is 0 when it has to be 1), we erase the identi\ufb01ed pattern\nby the clause and make it available for a new pattern. To do so, the exclude actions of TAs,\nregardless of their corresponding literal values, are strengthened. We now sub-divide the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac09-934e-11ee-97f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "Type I feedback into Type Ia and Ib, where Type Ia handles the reinforcing of exclude actions\nwhile Type Ib works on reinforcing exclude actions of TAs. Together, Type Ia and Type\nIb feedback force clauses to output 1. Hence, clauses with positive polarity need Type I\nfeedback when y=1and clauses with negative polarity need Type I feedback when y=0.\nTo diversify the clauses, they are targeted for Type I feedback stochastically as follows:\npj={\n1 with probabilityT\u2212max(\u2212T,min(T,v))\n2T,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0a-934e-11ee-8ed1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "pj={\n1 with probabilityT\u2212max(\u2212T,min(T,v))\n2T,\n0 otherwise.(7)\nAll clauses in each class should not learn the same sub-pattern, nor only a few subpatterns.\nHence, clauses should be smartly allocated among the sub-patterns. The user set target T\nin(7)does this while deciding the probability of receiving Type I feedback; i.e., Tnumber\nof clauses are available to learn each sub-pattern in each class. A higher Tincreases the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0b-934e-11ee-a55a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "robustness of learning by allocating more clauses to learn each sub-pattern. Now, Ttogether\nwith vdecides the probability of clause jreceiving Type I feedback, and accordingly, the\ndecision pjis made. The decisions for the complete set of clauses to receive Type I feedback\nare organized in the vector P= (pj)\u2208{0, 1}m.\nOnce the clauses to receive Type I feedback are singled out as per (7), the probability\nof updating individual TAs in selected clauses is calculated using the user-set parameter"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0c-934e-11ee-ba7f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 6, "chroma:document": "s(s\u22651), separately for Type Ia and Type Ib. According to the above probabilities, the\ndecision whether the kthTA of the jthclause is to receive Type Ia feedback, rj,kor Type Ib\nfeedback, qj,kis made stochastically as follows:\nrj,k={\n1 with probabilitys\u22121\ns,\n0 otherwise.(8)\nqj,k={\n1 with probability1\ns,\n0 otherwise.(9)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0d-934e-11ee-b5b9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "Electronics 2021 ,10, 2107 8 of 25\nThe above decisions are stored in the two matrices Rand Q, respectively; i.e., R=\n(rj,k)\u2208{0, 1}m\u00d72oand Q= (qj,k)\u2208{0, 1}m\u00d72o. Using the complete set of conditions, TA\nindexes selected for Type Ia are IIa={(j,k)|lk=1\u2227cj=1\u2227pj=1\u2227rj,k=1}.Similarly,\nTA indexes selected for Type Ib are IIb={\n(j,k)|(lk=0\u2228cj=0)\u2227pj,y=1\u2227qj,k=1}\n.\nThe states of the identi\ufb01ed TAs are now ready to be updated. Since Type Ia strengthens"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0e-934e-11ee-9e02-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "theinclude action of TAs, the current state should move more towards the include action\ndirection. We denote this as \u2295, and here\u2295adds 1 to the current state value of the TA. The\nType Ib feedback, on the other hand, moves the state of the selected TA towards the exclude\naction direction to strengthen the exclude action of TAs. We denote this by \u2296, and here,\u2296\nsubtracts 1 from the current state value of the TA. Accordingly, the states of TAs in Aare\nupdated as A\u2190(\nA\u2295IIa)\u2296IIb."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac0f-934e-11ee-961d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "updated as A\u2190(\nA\u2295IIa)\u2296IIb.\nType II feedback: Type II feedback has been designed to combat the false positive\noutput of clauses (the clause output is 1 when it has to be 0). To turn this clause output\nfrom 1to0, a literal value of 0can simply be included in the clause. Clauses with a positive\npolarity need Type II feedback when y=0and clauses with negative polarity need this\nwhen y=1as they do not want to vote for the opposite class. Again, using the user-set"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac10-934e-11ee-93be-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "target T, the decision for the jthclause is made as follows:\npj={\n1 with probabilityT+max(\u2212T,min(T,v))\n2T,\n0 otherwise.(10)\nThe states of the TAs whose corresponding literal of value 0in selected clauses\naccording to (10) are now moved towards the include action direction with a probability of 1.\nHence, the index set of this kind can be identi\ufb01ed as III={(j,k)|lk=0\u2227cj=1\u2227pj=1}.\nAccordingly, the states of TAs in Aare updated as A\u2190A\u2295III."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac11-934e-11ee-a935-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "When training has been completed, the \ufb01nal decisions of the TAs are recorded, and\nthe resulting clauses can be deployed for operation.\nBooleanization of Continuous Features: In the TM discussed so far, the input layer\naccepted only Boolean features; i.e., X= [x1,x2,x3,. . .,xo]with xk,k=1, 2, ..., o, being 0 or\n1. These features and their negations were directly fed into the clauses without any further\nmodi\ufb01cations. However, continuous features in machine learning applications are more"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac12-934e-11ee-8a5e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "common than values of simply 1or0. In one of our previous papers [ 37], we presented\na systematic procedure of transforming continuous features into Boolean features while\nmaintaining ranking relationships among the continuous feature values.\nWe here summarize the previous Booleanization scheme using the example presented\nin [27]. As seen in Table 2, we Booleanize the two continuous features listed in table\ncolumn 1 and column 2.\n1. First, for each feature, the unique values are identi\ufb01ed;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac13-934e-11ee-b419-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "2. The unique values are then sorted from smallest to largest;\n3. The sorted unique values are considered as thresholds. In the table, these values can\nbe seen in the \u201cThresholds\u201d row;\n4. The original feature values are then compared with identi\ufb01ed thresholds, only from\ntheir own feature value set. If the feature value is greater than the threshold, set the\ncorresponding Boolean variable to 0; otherwise, set it to 1;"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac14-934e-11ee-aa44-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "5. The above steps are repeated until all the features are converted into Boolean form.\nThe \ufb01rst feature in the \ufb01rst column of the table contains three unique values: 5.779,\n10.008, and 3.834 (step (i)). Once they are sorted as required in step ii, we obtain 3.834,\n5.779, and 10.008. Now, we consider them as thresholds: \u22643.834,\u22645.779, and\u226410.008 (step\n(iii)). Here, we \ufb01nd that each original feature in column 1 is going to represent 3 bit values."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac15-934e-11ee-8c7c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 7, "chroma:document": "According to step iv, we now compare the original values in the \ufb01rst feature against its\nthresholds. The \ufb01rst feature value of 5.779 is greater than 3.834 (resulting in 0), equal to\n5.779 (resulting in 1), and less than 10.008 (resulting in 1). Hence, we replace 5.779 with 011.\nSimilarly, 10.008 and 3.834 can be replaced with 001 and 111, respectively."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac16-934e-11ee-bcc1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "Electronics 2021 ,10, 2107 9 of 25\nThe conversion of the feature values for the second feature starts once all the feature\nvalues in the \ufb01rst feature are completed. This procedure is iterated until all the continuous\nvalues of all the continuous features have been converted to Boolean form (step (v)).\nTable 2. Preprocessing of two continuous features.\nRaw Feature Thresholds\n1 2 \u22643.834 \u22645.779 \u226410.008 \u226411.6 \u226425.7 \u226432.4 \u226456.1\n5.779 25.7 0 1 1 0 1 1 1\n10.008 56.1 0 0 1 0 0 0 1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac17-934e-11ee-8e4d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "10.008 56.1 0 0 1 0 0 0 1\n5.779 11.6 0 1 1 1 1 1 1\n3.834 32.4 1 1 1 0 0 1 1\nThis Boolean representation of continuous features is particularly powerful as it allows\nthe TM to reason about the ordering of the values, forming conjunctive clauses that specify\nrules based on thresholds, and with negated features, also rules based on intervals. This\ncan be explained again with the following example.\nThe threshold\u22643.834 in the \u201cThreshold\u201d row stands for the continuous value 3.834 of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac18-934e-11ee-a45d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "the \ufb01rst feature. Similarly, threshold \u22645.779 and\u226410.008 represent the continuous values\n5.779 and 10.008 , respectively. Consider a clause with threshold \u22645.779 included in the\nclause, which is the only threshold included in the clause. Then, for any input value less\nthan or equal to\u22645.779 from that feature, the clause outputs 1. Now, consider the case\nof having two thresholds, \u22645.779 and\u226410.008, included in the clause. The threshold"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac19-934e-11ee-a180-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "\u22645.779 still decides the clause output due to the fact that the AND of\u22645.779 and\u226410.008\nthreshold columns in Table 2 yields the threshold column \u22645.779. When a clause includes a\nnegated threshold it reverses the original threshold. Consider a clause that only includeds\nthe negation of the threshold \u22643.834. Now, the clause outputs 1 for all the values greater\nthan\u22643.834 from that feature, as the NOT of\u22643.834 is equivalent to 3.834 <."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1a-934e-11ee-ad33-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "The above explanation of the threshold selection reveals that the lowest original\nthreshold included in the clause and the highest negated threshold included in the clause\ndecide the upper and lower boundary of the feature values, and these thresholds are the\nonly important thresholds for calculating the clause output. Hence, this motivates us to\nrepresent the continuous features in clauses in a new way and train the clauses accordingly\nas follows."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1b-934e-11ee-9ec2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "as follows.\n4. Sparse Representation of Continuous Features\nHowever, the above representation of continuous values in clauses is costly as it needs\ntwo times the total number of unique values of TAs per clause. This is more severe when the\ndataset is large and when there is a large number of input features to be considered. Hence,\nwe introduce SSL automata to represent the upper and lower limits of the continuous"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1c-934e-11ee-9909-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 8, "chroma:document": "features. With the new representation, a continuous feature can be then represented by only\ntwo automata instead of having two times the number of unique values in the considered\ncontinuous feature. The new parameters and symbols used in this section are explained\nand summarized in Table 3."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1d-934e-11ee-be5c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "Electronics 2021 ,10, 2107 10 of 25\nTable 3. Parameters and symbols used in Section 4.\nParameter/Symbol Description Parameter/Symbol Description\nXc Input vector containing\nocontinuous featuresxc\nkkthcontinuous feature\nSSLl\nj,kLower limit of the continuous\nfeature kin clause jSSLu\nj,kUpper limit of the continuous\nfeature kin clause j\nEl\nj,kFeedback to the SSL automata\nwhich represent the lower limit\nof the feature kin clause jEu\nj,kFeedback to the SSL automata\nwhich represent the upper limit"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1e-934e-11ee-8d92-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "which represent the upper limit\nof the feature kin clause j\nTAl\nj,kTA which decides whether to\ninclude or exclude lower limit\nof the kthfeature in clause jTAu\nj,kTA which decides whether to\ninclude or exclude upper limit\nof the kthfeature in clause j\nrl\nj,kThe decision whether the TA of the\nlower limit of kthfeature in\nthejthclause is to receive\nType Ia feedbackru\nj,kThe decision whether the TA of the\nupper limit of kthfeature in\nthejthclause is to receive\nType Ia feedback\nql"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac1f-934e-11ee-9550-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "thejthclause is to receive\nType Ia feedback\nql\nj,kThe decision whether the TA of the\nlower limit of kthfeature in\nthejthclause is to receive\nType Ib feedbackqu\nj,kThe decision whether the TA of the\nupper limit of kthfeature in\nthejthclause is to receive\nType Ib feedback\nlj,kComputed literal value for the\nkthfeature in clause j\nInput Features. As discussed earlier, the TM takes opropositional variables as input,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac20-934e-11ee-886e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "X= [x1,x2,x3,. . .,xo]. In this section, we discuss how the TM maps Xcwhich contains o\ncontinuous features, Xc= [xc\n1,xc\n2,xc\n3, . . . , xc\no]into one of two classes y=1 or y=0.\nFeature Representation. Each continuous feature is assigned two SSLs to represent\nthe upper and lower limits of the continuous value in a clause; i.e., SSLl\n1,SSLu\n1,. . .,SSLl\nk,\nSSLu\nk,. . .,SSLl\no,SSLu\no. Here, SSLl\nkand SSLu\nkare lower and upper limit values of the kth"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac21-934e-11ee-ad30-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "kare lower and upper limit values of the kth\ncontinuous feature, respectively. However, step size Nwithin an SSL in this case is not\nconstant. When Ein (1) and (2) is 1, the SSL state moves to the higher neighboring unique\nvalue of the attached continuous feature of the SSL. Similarly, when Eis 0, SSL state moves\nto the lower neighboring unique value of the considered continuous feature.\nClauses. Each conjunctive clause in the TM receives Xcas an input. The inclusion"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac22-934e-11ee-8869-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "and exclusion decisions of the corresponding upper and lower bounds of xc\nkin the clause\nare made by TAs. Hence, each clause now needs 2oTAs, where half of them make the\ndecision related to the lower bound of the continuous features while the other half make\nthe decision related to the upper bound of the continuous features. The matrix Atherefore\nstill contains m\u00d72oelements: A= (aj,k)\u2208{1, . . . , 2 N}m\u00d72o.\nIn the phase of calculating clause outputs, both limit values given by SSLs and the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac23-934e-11ee-ac61-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 9, "chroma:document": "decisions of TAs on their corresponding SSLs are considered. The value of the kthliteral, lj,k,\nwhich represents the kthcontinuous feature inside the clause, j, to perform the conjunction\nis evaluated as follows:\n\u2022 Condition 1: Both TAl\nj,kand TAu\nj,kwhich respectively make the decision on SSLl\nj,kand\nSSLu\nj,kdecide to include them in the clause. Then,\nlj,k={\n1, if SSLl\nj,k<xc\nk\u2264SSLu\nj,k,\n0, if xc\nk\u2264SSLl\nj,k\u2228SSLu\nj,k<xc\nk.(11)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac24-934e-11ee-9148-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "Electronics 2021 ,10, 2107 11 of 25\n\u2022 Condition 2: The TAl\nj,kdecides to include SSLl\nj,kin the clause and TAu\nj,kdecides to\nexclude SSLu\nj,kfrom the clause. Then,\nlj,k={\n1, if SSLl\nj,k<xc\nk,\n0, if xc\nk\u2264SSLl\nj,k.(12)\n\u2022 Condition 3: The TAu\nj,kdecides to include SSLu\nj,kin the clause and TAl\nj,kdecides to\nexclude SSLl\nj,kfrom the clause. Then,\nlj,k={\n1, if xc\nk\u2264SSLu\nj,k,\n0, if SSLu\nj,k<xc\nk.(13)\n\u2022 Condition 4: Both TAl\nj,kand TAu\nj,kdecide to exclude their corresponding SSLs from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac25-934e-11ee-9d35-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "the clause, which consequently takes the lower limit to the lowest possible and the\nupper limit to the highest possible values. Hence, lj,kalways becomes 1 or can be\nexcluded when conjunction is performed.\nHence, when at least one of the TAs that represent the lower and upper limits decides\nto include its corresponding limit in the jthclause, the index of the feature is included in Ij,\nIj\u2286{1,. . .,o}. Then, depending on the literal value according to the above conditions, the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac26-934e-11ee-a5ee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "clause output is computed, cj=\u22c0\nk\u2208Ijlk,j=1, . . . , m.\nClassi\ufb01cation. Similar to the standard TM, the vote difference vis computed as\nv=\u2211jc+\nj(Xc)\u2212\u2211jc\u2212\nj(Xc). Once the vote difference is known, the output class is decided\nusing (6).\nLearning. In this new setup, the clauses still receive Type I and Type II feedback.\nHowever, both TAs and SSLs have to be updated as feedback is received. In other words,\nType I and Type II feedback should be able to guide SSLs to learn the optimal lower and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac27-934e-11ee-b999-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "upper limits of the continuous features in each clause and lead TAs to correctly decide\nwhich limits should be included or excluded in individual clauses.\nAs discussed earlier, Type Ia feedback reinforces the true positive outputs of clauses\nby rewarding the include action of TAs when the literal value is 1. In the new setting, Type\nIa feedback updates both SSLs and TAs when xc\nkis within the lower and upper boundaries,\nSSLl\nj,k<xc\nk\u2264SSLu"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac28-934e-11ee-a76f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "SSLl\nj,k<xc\nk\u2264SSLu\nj,kand when the clause output is 1 when it has to be 1 (positive clauses\nwhen y=1and negative clauses when y=0). Under these conditions, the decision\nregarding whether both the TAs of upper and lower bounds of kthfeature in the jthclause\nare to receive Type Ia feedback, rl\nj,kand ru\nj,k, is stochastically made as follows:\nrl\nj,k={\n1 with probabilitys\u22121\ns,\n0 otherwise.(14)\nru\nj,k={\n1 with probabilitys\u22121\ns,\n0 otherwise.(15)\nThe environment feedbacks El\nj,kto update SSLl"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac29-934e-11ee-a98a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "The environment feedbacks El\nj,kto update SSLl\nj,kand Eu\nj,kto update SSLu\nj,kare0and\n1, respectively. By doing so, we force SSLs to tighten up the boundary of the continuous\nfeature kand include them in the clause jby reinforcing the include action of TAs. Notice\nthat the above updates are made only if the condition in (7) is satis\ufb01ed.\nType Ib feedback activates if xc\nkis outside any of the upper or lower boundaries,\nxc\nk\u2264SSLl\nj,k\u2228SSLu\nj,k<xc"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2a-934e-11ee-be7b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 10, "chroma:document": "xc\nk\u2264SSLl\nj,k\u2228SSLu\nj,k<xc\nkor if the clause output is 0. For the case where xc\nk\u2264SSLl\nj,kor the\nclause output is 0 when it has to be 1, the decision on whether the TA of the lower bound of\nthekthfeature in the jthclause should receive Type Ib feedback, ql\nj,k, is stochastically made\nas follows:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2b-934e-11ee-8ecd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "Electronics 2021 ,10, 2107 12 of 25\nql\nj,k={\n1 with probability1\ns,\n0 otherwise.(16)\nSimilarly, when it violates the upper bound requirement\u2014i.e., SSLu\nj,k<xc\nkor if the\nclause output is 0 when it has to be 1\u2014the decision to receive Type Ib feedback on the TA\nwhich represents upper bound is made as follows:\nqu\nj,k={\n1 with probability1\ns,\n0 otherwise.(17)\nThe environment feedbacks for the SSLs when the Type Ib feedback is applied are 0\nand 1 for El\nj,kand Eu"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2c-934e-11ee-b406-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "and 1 for El\nj,kand Eu\nj,k, respectively. In this way, SSLs are forced to expand the boundary\nand TAs are discouraged from the inclusion of their respective SSLs in the clause.\nOnce the eligible clauses (positive clauses when y=0and negative clauses when\ny=1) to receive Type II feedback are stochastically selected using (10), the states of the\nindividual SSLs and TAs in them are updated. The original idea of Type II feedback is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2d-934e-11ee-b130-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "to combat the false positive output of the clause. In the new updating scheme, this is\nachieved by expanding the boundaries of the kthfeature if xc\nkis outside of the boundary\nand including them in the clause, which then turns the clause output to 0 eventually. Hence,\nifxc\nk\u2264SSLl\nj,k, the environment feedback on SSLl\nj,k,El\nj,kbecomes 0 and the state of the TA\nthat appears for SSLl\nj,kincreases by one step with probability 1. Likewise, if SSLu\nj,k<xc\nk,\nthe environment feedback on SSLu\nj,k,Eu"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2e-934e-11ee-bb97-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "j,k<xc\nk,\nthe environment feedback on SSLu\nj,k,Eu\nj,kbecomes 1 and the state of the TA that appears for\nSSLu\nj,kincreases by one step with probability 1.\nThe above decisions on receiving Type Ia, Type Ib, and Type II are stored in IIa,IIb,\nand III, respectively. The processing of the training example in the new scheme ends with\nthe state matrix Aof TAs being updated as A\u2190((\nA\u2295IIa)\u2296IIb)\u2295III, and the states\nof SSLs are updated according to (1) and (2) with the identi\ufb01ed environment feedback of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac2f-934e-11ee-843d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "individual SSLs, E.\n5. Empirical Evaluation\nIn this section, the impact of the new continuous input feature representation to the\nTM is empirically evaluated using \ufb01ve real-world datasets (the implementation of Tsetlin\nMachine with versions of relevant software libraries and frameworks can be found at:\nhttps://github.com/cair/TsetlinMachine , accessed on 24 August 2021). The datasets Liver\nDisorder dataset ,Breast Cancer dataset , and Heart Disease dataset are from the health sector. The"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac30-934e-11ee-ae9b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "Balance Scale and Corporate Bankruptcy datasets are the two other remaining datasets. The\nLiver Disorder dataset ,Breast Cancer dataset ,Heart Disease dataset , and Corporate Bankruptcy\ndatasets were selected as these applications in the health and \ufb01nance sectors demand\nboth interpretability and accuracy in predictions. The Balance Scale dataset was added\nto diversify the selected applications. As an example, we use the Corporate Bankruptcy"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac31-934e-11ee-be89-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 11, "chroma:document": "dataset to examine the interpretability of the TM using both the previous continuous feature\nrepresentation and the proposed representation. A summary of these datasets is presented\nin Table 4.\nTable 4. Binarizing categorical features in the Bankruptcy dataset.\nDataset Number of Instances Number of Attributes Interpretability Needed\nCorporate Bankruptcy 250 7 Yes\nBalance Scale 625 4 Not necessarily\nBreast Cancer 286 9 Yes\nLiver Disorders 345 7 Yes\nHeart Disease 270 13 Yes"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac32-934e-11ee-8444-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "Electronics 2021 ,10, 2107 13 of 25\nThe performance of the TM is also contrasted against several other standard machine\nlearning algorithms, namely Arti\ufb01cial Neural Networks (ANNs), Decision Trees (DTs),\nSupport Vector Machines (SVMs), Random Forest (RF), K-Nearest Neighbor (KNN), Ex-\nplainable Boosting Machines (EBMs), [ 22], and Gradient Boosted Trees (XGBoost) [ 50]\nalong with two recent state-of-the-art machine learning approaches: StructureBoost [ 47]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac33-934e-11ee-b797-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "and Neural Additive Models [ 11]. For comprehensiveness, three ANN architectures are\nused: ANN-1\u2014with one hidden layer of 5 neurons, ANN-2\u2014with two hidden layers\nof 20 and 50 neurons each, and ANN-3\u2014with three hidden layers and 20, 150, and 100\nneurons. The other hyperparameters of each of these models are decided using trial and\nerror. The reported results in this sections are the average measure over 50 independent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac34-934e-11ee-9836-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "experiment trials. The data are randomly divided into training (80%) and testing (20%) sets\nfor each experiment.\n5.1. Bankruptcy\nIn \ufb01nance, interpretable machine learning algorithms are preferred over black-box\nmethods to predict bankruptcy as bankruptcy-related decisions are sensitive. However,\nat the same time, the accuracy of the predictions is also important to mitigate \ufb01nancial\nlosses [51].\nThe Bankruptcy dataset (available from: https://archive.ics.uci.edu/ml/datasets/"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac35-934e-11ee-b085-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "qualitative_bankruptcy, accessed on 24 August 2021) that we consider in this experi-\nment contains historical records of 250companies. The output\u2014i.e., bankruptcy or non-\nbankruptcy\u2014is determined by six pertinent features: (1) industrial risk, (2) management\nrisk, (3) \ufb01nancial \ufb02exibility, (4) credibility, (5) competitiveness, and (6) operation Risk.\nThese are categorical features where each feature can be in one of three states: negative (N),\naverage (A), or positive (P)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac36-934e-11ee-b01b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "average (A), or positive (P).\nThe output \u201cbankruptcy\u201d is considered as class 0 while \u201cnon-bankruptcy\u201d is class\n1. The features are, however, ternary. Thus, the TM has to be used with the proposed\nSSL scheme to represent categorical features directly in clauses or features should be\nBooleanized using the Booleanization scheme before feeding them into the TM. If the\nfeatures are Booleanized beforehand, each feature value can be represented in three"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac37-934e-11ee-8d22-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "Boolean features as shown in Table 5. Thus, the complete Booleanized dataset contains 18\nBoolean features.\nTable 5. Binarizing categorical features in the Bankruptcy dataset.\nCategory Integer CodeThresholds\n\u22640 \u22641 \u22642\nA 0 1 1 1\nN 1 0 1 1\nP 2 0 0 1\nFirst, the behavior of the TM with 10 clauses is studied. The included literals in\nall these 10 clauses at the end of training are summarized in Table 6. In the TM with"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac38-934e-11ee-a33b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "Booleanized features, the TAs in clause 1 decide to include only the negation of feature 11,\n\u00acx11. Feature 11is the negative credibility , which we can \ufb01nd after binarizing all features.\nThe TAs in clauses 2, 4, 6, 8, and 10 decide to include the negation of average competitiveness\nand negative competitiveness , which are non-negated in clauses. The TAs in clauses 3, 5, and\n9, on the other hand, decide to include negated negative competitiveness . Clause 7 is \u201cempty\u201d"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac39-934e-11ee-a3f2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 12, "chroma:document": "as TAs in this clause decide not to include any literal in the clause."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3a-934e-11ee-8516-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "Electronics 2021 ,10, 2107 14 of 25\nTable 6. Clauses produced by TM with Booleanization and SSL schemes for m=10.\nClause ClassTM with\nBooleanized SSLs\n1 1 \u00acx11 -\n2 0 \u00acx13\u2227x14 1<xc\n5\u22642\n3 1 \u00acx14 2<xc\n5\n4 0 \u00acx13\u2227x14 1<xc\n5\u22642\n5 1 \u00acx14 -\n6 0 \u00acx13\u2227x14 1<xc\n5\u22642\n7 1 - -\n8 0 \u00acx13\u2227x14 1<xc\n5\u22642\n9 1 \u00acx14 -\n10 0 \u00acx13\u2227x14 1<xc\n5\u22642\nAccuracy (Training/Testing) 0.98/1.00 0.99/0.96\nTable 6 also contains the clauses learnt by the TM when the SSL continuous feature"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3b-934e-11ee-af25-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "approach is used. The clauses 2, 4, 6, 8, and 10 which vote for bankruptcy activate negative\ncompetitiveness . On the other hand, clause 3 recognizes the sub-patterns of class 1 outputs 1\nforpositive competitiveness . There are four free votes for class 1 from the \u201cempty\u201d clauses 1,\n5, 7, and 9, which are again ignored during classi\ufb01cation. Note also that, without loss of\naccuracy, the TM with the SSL approach simpli\ufb01es the set of rules by not including negative"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3c-934e-11ee-9310-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "credibility in any of the clauses. With the identi\ufb01ed thresholds for the continuous values\n(categorical in this application), the TM with the SSL approach ends up with the simple\nclassi\ufb01cation rule:\nOutcome ={\nBankruptcy ifNegative Competitiveness\nNon-bankruptcy otherwise .(18)\nBy asking the TMs to utilize only two clauses, we can obtain the above rule more\ndirectly, as shown in Table 7. As seen, again, the TM with both feature representations\nachieves similar accuracy."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3d-934e-11ee-91a1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "achieves similar accuracy.\nTable 7. Clauses produced by TM with Booleanization and SSL schemes for m=2.\nClause ClassTM with\nBooleanized SSLs\n1 1 \u00acx14 2<xc\n5\n2 0 \u00acx13\u2227x14 1<xc\n5\u22642\nAccuracy (Training/Testing) 0.99/0.96 0.96/0.98\nThe previous accuracy results represent the majority of experiment trials. Some experi-\nments fail to obtain this optimum accuracy. Instead of conducting the experiments multiple\ntimes to \ufb01nd the optimum clause con\ufb01guration in the TM, the number of clauses can be"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3e-934e-11ee-8446-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "increased to \ufb01nd more robust con\ufb01gurations of clauses. Even though this provides stable\nhigher accuracy for almost all the trials, a large number of clauses affects the interpretability.\nThis is where we have to consider achieving a balance between accuracy and interpretabil-\nity. For the Bankruptcy dataset, how robustness increases with clauses can be seen in\nTables 8 and 9 . The average performance (precision, recall, F1-Score, accuracy, speci\ufb01city)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac3f-934e-11ee-b488-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 13, "chroma:document": "is summarized in the tables for the TM with both feature arrangements, respectively."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac40-934e-11ee-b2a7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 14, "chroma:document": "Electronics 2021 ,10, 2107 15 of 25\nTable 8. Performance of TM with Booleanized continuous features on Bankruptcy dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.754 0.903 0.997 0.994 0.996 0.994\nRecall 1.000 1.000 1.000 0.998 1.000 1.000\nF1-Score 0.859 0.948 0.984 0.996 0.998 0.997\nAccuracy 0.807 0.939 0.998 0.996 0.998 0.996\nSpeci\ufb01city 0.533 0.860 0.995 0.993 0.996 0.990\nNo. of Lit. 19 88 222 832 3622 15,201\nTable 9. Performance of TM with SSL continuous feature scheme on Bankruptcy dataset."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac41-934e-11ee-bc4c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 14, "chroma:document": "m 2 10 100 500 2000 8000\nPrecision 0.622 0.777 0.975 0.995 0.994 0.996\nRecall 0.978 0.944 0.994 1.000 0.997 0.995\nF1-Score 0.756 0.843 0.984 0.997 0.995 0.995\nAccuracy 0.640 0.787 0.982 0.997 0.995 0.994\nSpeci\ufb01city 0.191 0.568 0.967 0.994 0.993 0.993\nNo. of Lit. 8 40 94 398 1534 7286\nTable 8 reports the results of the TM with a regular Booleanization scheme. The goal\nhere is to maximize the F1-Score, since accuracy can be misleading for imbalanced datasets."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac42-934e-11ee-bd80-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 14, "chroma:document": "As can be seen, the F1-Score increases with clauses and peaks at m=2000 . To obtain this\nperformance with the Booleanized features, the TM classi\ufb01er uses 3622 (rounded to nearest\ninteger) literals (include actions).\nDespite the slight reduction in the F1-Score, the TM with the proposed continuous\nfeature representation reaches its best F1-Score with only 398literals in the TM classi\ufb01er.\nThis represents a greater than nine-fold reduction of literals, which is more signi\ufb01cant"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac43-934e-11ee-97ad-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 14, "chroma:document": "compared to the reduction of the F1-Score ( 0.001 ). At this point, the number of clauses, m,\nequals 500.\nFigure 3 outlines how the number of literals varies with the increase of the number\nof clauses. The TM with the new continuous feature representation scheme consistently\nuses fewer literals in its classi\ufb01er than the TM with regular feature representation. The\ndifference between the number of literals in both approaches increases with the number of\nclauses.\n2 10 100 500 2000 8000"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac44-934e-11ee-b20d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 14, "chroma:document": "clauses.\n2 10 100 500 2000 8000\nNo. of Clauses0200040006000800010,00012,00014,000No. of LiteralsBinarized features\nSSLs scheme16,000\nFigure 3. The number of literals included in TM clauses to work with Bankruptcy dataset.\nThe performance of the TM with both continuous feature arrangements is compared\nagainst multiple standard machine learning models: ANN, KNN, XGBoost, DT, RF, SVM,\nand EBM. The performances of these techniques along with the best performance of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac45-934e-11ee-b6c4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "Electronics 2021 ,10, 2107 16 of 25\nTM setups are summarized in Table 10. The best F1-Score is obtained by the TM with\nregular Booleanized features. The second best F1-Score belongs to ANN-3 and the TM with\nthe SSL scheme. Memory wise, the TM with both input feature representations together\nwith DT needs close to zero memory at both training and testing, while ANN-3 requires a\ntraining memory of 28,862.65 KB and a testing memory of 1297.12 KB. More importantly,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac46-934e-11ee-a336-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "the training time per epoch and the number of literals in clauses are reduced with the SSL\nscheme for the TM compared to the Booleanization approach.\nTable 10. Performance comparison for Bankruptcy dataset.\nPrec. Reca. F1 Acc. Spec. No. of Lit.Memory RequiredTraining Time(Training/Testing)\nANN-1 0.990 1.000 0.995 0.994 0.985 - \u2248942.538 KB/\u224826.64 KB 0.227 s.\nANN-2 0.995 0.997 0.996 0.995 0.993 - \u22483476.76 KB/\u2248590.76 KB 0.226 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac47-934e-11ee-bb72-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "ANN-3 0.997 0.998 0.997 0.997 0.995 - \u224828,862.65 KB/\u22481297.12 KB 0.266 s.\nDT 0.988 1.000 0.993 0.993 0.985 - \u22480.00 KB/\u22480.00 KB 0.003 s.\nSVM 1.000 0.989 0.994 0.994 1.000 - \u224890.11 KB/\u22480.00 KB 0.001 s.\nKNN 0.998 0.991 0.995 0.994 0.998 - \u22480.00 KB/\u2248286.71 KB 0.001 s.\nRF 0.979 0.923 0.949 0.942 0.970 - \u2248180.22 KB/\u22480.00 KB 0.020 s.\nXGBoost 0.996 0.977 0.983 0.983 0.992 - \u22484964.35 KB/\u22480.00 KB 0.009 s.\nEBM 0.987 1.000 0.993 0.992 0.980 - \u22481425.40 KB/\u22480.00 KB 13.822 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac48-934e-11ee-b9f5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "TM (Booleanized) 0.996 1.000 0.998 0.998 0.996 3622 \u22480.00 KB/\u22480.00 KB 0.148 s.\nTM (SSLs) 0.995 1.000 0.997 0.997 0.994 398 \u22480.00 KB/\u22480.00 KB 0.119 s.\n5.2. Balance Scale\nWe then move to the Balance Scale dataset (available from http://archive.ics.uci.edu/\nml/datasets/balance+scale, accessed on 24 August 2021). The Balance Scale dataset consists\nof three classes: a balance scale that tips to the left, tips to the right, or that is in balance."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac49-934e-11ee-a257-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "The above class is decided collectively by four features: (1) the size of the weight on the\nleft-hand side, (2) distance from the center to the weight on the left, (3) size of the weight on\nthe right-hand side, and (4) distance from the center to the weight on the right. However,\nwe remove the third class\u2014i.e., \u201cbalanced\u201d\u2014and contract the output to a Boolean form.\nThe resulting dataset ends up with 576 data samples."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4a-934e-11ee-8a61-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "Tables 11 and 12 contain the results of the TM with two continuous feature representa-\ntions, with varying m. The F1-Score reaches its maximum of 0.945 atm=100for the TM\nwith the Boolean feature arrangement. The average number of literals used to achieve the\nabove performance is 790. The TM with the SSL scheme reaches its maximum performance\nwhen m=500. The number of literals used in the classi\ufb01er to achieve this performance\nis 668."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4b-934e-11ee-8c25-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "is 668.\nTable 11. Performance of TM with Booleanized continuous features on Balance Scale dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.647 0.820 0.966 0.949 0.926 0.871\nRecall 0.986 0.965 0.930 0.934 0.884 0.746\nF1-Score 0.781 0.886 0.945 0.933 0.880 0.749\nAccuracy 0.728 0.875 0.948 0.936 0.889 0.780\nSpeci\ufb01city 0.476 0.782 0.966 0.935 0.905 0.819\nNo. of Lit. 17 77 790 3406 15,454 60,310\nTable 12. Performance of TM with SSL continuous feature scheme on Balance Scale dataset."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4c-934e-11ee-8848-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 15, "chroma:document": "m 2 10 100 500 2000 8000\nPrecision 0.579 0.717 0.919 0.961 0.877 0.851\nRecall 0.957 0.947 0.972 0.938 0.867 0.794\nF1-Score 0.717 0.812 0.944 0.946 0.854 0.781\nAccuracy 0.612 0.777 0.943 0.948 0.854 0.795\nSpeci\ufb01city 0.254 0.598 0.916 0.959 0.840 0.795\nNo. of Lit. 4 17 140 668 2469 9563"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4d-934e-11ee-b4cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "Electronics 2021 ,10, 2107 17 of 25\nThe variation of the number of literals over different numbers of clauses in the TM\nwith these two continuous feature arrangements is graphed in Figure 4. The TM with the\nSSL scheme uses a smaller number of literals for all the considered number of clauses, with\nthe difference increasing with number of clauses.\n2 10 100 500 2000 8000\nNo. of Clauses010,00020,00030,00040,00050,00060,000No. of LiteralsBinarized features\nSSLs scheme"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4e-934e-11ee-8366-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "SSLs scheme\nFigure 4. The number of literals included in TM clauses to work with the Balance Scale dataset.\nFor the Balance Scale dataset, the performances of the other machine learning al-\ngorithms are also obtained. Along with the TM performance, the prediction accuracies\nof other models are presented in Table 13. The highest F1-Score from all the considered\nmodels is procured by EBM. Out of the two TM approaches, the TM with the SSL scheme"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac4f-934e-11ee-8bdf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "shows the best performance in terms of the F1-Score, while using less training time and\ntraining memory.\nTable 13. Performance comparison for Balance Scale dataset.\nPrec. Reca. F1 Acc. Spec. No. of Lit.Memory RequiredTraining Time(Training/Testing)\nANN-1 0.993 0.987 0.990 0.990 0.993 - \u2248966.57 KB/\u224824.56 KB 0.614 s.\nANN-2 0.995 0.995 0.995 0.995 0.994 - \u22483612.65 KB/\u2248589.82 KB 0.588 s.\nANN-3 0.995 0.995 0.995 0.995 0.995 - \u224833,712.82 KB/\u22481478.64 KB 0.678 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac50-934e-11ee-b647-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "DT 0.984 0.988 0.986 0.986 0.985 - \u2248131.07 KB/\u22480.00 KB 0.007 s.\nSVM 0.887 0.889 0.887 0.887 0.884 - \u224865.53 KB/\u2248241.59 KB 0.001 s.\nKNN 0.968 0.939 0.953 0.953 0.969 - \u2248249.77 KB/\u2248126.87 KB 0.001 s.\nRF 0.872 0.851 0.859 0.860 0.871 - \u22480.00 KB/\u22480.00 KB 0.021 s.\nXGBoost 0.942 0.921 0.931 0.931 0.942 - \u22481126.39 KB/\u22480.00 KB 0.030 s.\nEBM 1.000 1.000 1.000 1.000 1.000 - \u22481642.49 KB/\u22480.00 KB 15.658 s.\nTM (Booleanized) 0.966 0.930 0.945 0.948 0.966 790 \u224816.37 KB/\u22480.00 KB 0.011 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac51-934e-11ee-9231-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "TM (SSLs) 0.961 0.938 0.946 0.948 0.959 668 \u22489.43 KB/\u22480.00 KB 0.004 s.\n5.3. Breast Cancer\nThe nine features in the Breast Cancer dataset (available from: https://archive.ics.uci.\nedu/ml/datasets/Breast+Cancer, accessed on 24 August 2021) predict the recurrence of\nbreast cancer. The nine features in the dataset are age, menopause, tumor size, inverse\nnodes, node caps, degree of malignancy, side (left or right), the position of the breast,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac52-934e-11ee-a3db-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "and irradiation status. The numbers of samples in the \u201cnon-recurrence\u201d and \u201crecurrence\u201d\nclasses are 201 and 85, respectively. However, some of these samples are removed as they\ncontain missing values in their features.\nThe performances of the TMs with two feature arrangements and the number of\nliterals they included in their clauses to achieve this performance are summarized in\nTables 14 and 15 , respectively, for the Booleanized scheme and the SSL scheme. In contrast"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac53-934e-11ee-ac62-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 16, "chroma:document": "to the previous two datasets, the F1-Score for the TMs with both feature arrangements"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac54-934e-11ee-a21e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 17, "chroma:document": "Electronics 2021 ,10, 2107 18 of 25\npeaks at m=2. The performance then decreases with the increase of m. The numbers of\nliterals at this phase in TMs with Booleanized and SSL feature arrangements are 21 and 4,\nrespectively. Overall, the TM with the SSL scheme requires the smallest amount of literals,\nas can be seen in Figure 5.\nTable 14. Performance of TM with Booleanized continuous features on Breast Cancer dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.518 0.485 0.295 0.101 0.058 0.054"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac55-934e-11ee-b17d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 17, "chroma:document": "Precision 0.518 0.485 0.295 0.101 0.058 0.054\nRecall 0.583 0.380 0.416 0.205 0.200 0.250\nF1-Score 0.531 0.389 0.283 0.089 0.090 0.088\nAccuracy 0.703 0.737 0.644 0.633 0.649 0.581\nSpeci\ufb01city 0.742 0.864 0.731 0.800 0.800 0.750\nNo. of Lit. 21 73 70 407 1637 6674\nTable 15. Performance of TM with SSL continuous feature scheme on Breast Cancer dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.465 0.468 0.071 0.126 0.090 0.070\nRecall 0.759 0.575 0.233 0.467 0.333 0.233"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac56-934e-11ee-9513-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 17, "chroma:document": "Recall 0.759 0.575 0.233 0.467 0.333 0.233\nF1-Score 0.555 0.494 0.109 0.195 0.141 0.107\nAccuracy 0.645 0.701 0.630 0.525 0.589 0.628\nSpeci\ufb01city 0.599 0.753 0.778 0.551 0.682 0.775\nNo. of Lit. 4 16 101 321 997 4276\n2 10 100 500 2000 8000\nNo. of Clauses01000200030004000500060007000No. of LiteralsBinarized features\nSSLs scheme\nFigure 5. The number of literals included in TM clauses to work with Breast Cancer dataset."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac57-934e-11ee-93cd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 17, "chroma:document": "All the other algorithms obtain an F1-Score of less than 0.5. The performances of DT,\nRF, SVM, XGBoost, and EBM can be identi\ufb01ed as the worst of all models as summarized\ninTable 16 . The best F1-Score is obtained by the TM with the SSL feature representation\nprocedure, while the TM with Booleanized features obtain the second best F1-Score. The\nincrease of the F1-Score from 0.531 to 0.555 comes also with the advantage of having 19"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac58-934e-11ee-a30e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 17, "chroma:document": "less literals in clauses for the SSL approach. Both the training and testing memory usage\nof the TM with these two feature arrangements is negligible. The TM also has the lowest\ntraining time of all algorithms, amounting to 0.001 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac59-934e-11ee-b6fc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "Electronics 2021 ,10, 2107 19 of 25\nTable 16. Performance comparison for Breast Cancer dataset.\nPrec. Reca. F1 Acc. Spec. No. of Lit.Memory RequiredTraining Time(Training/Testing)\nANN-1 0.489 0.455 0.458 0.719 0.822 - \u22481001.97 KB/\u224835.74 KB 0.249 s.\nANN-2 0.430 0.398 0.403 0.683 0.792 - \u22483498.47 KB/\u2248608.71 KB 0.248 s.\nANN-3 0.469 0.406 0.422 0.685 0.808 - \u224838,645.07 KB/\u22481837.76 KB 0.288 s.\nDT 0.415 0.222 0.276 0.706 0.915 - \u2248102.39 KB/\u22480.00 KB 0.005 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5a-934e-11ee-8dee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "SVM 0.428 0.364 0.384 0.678 0.805 - \u2248241.66 KB/\u2248299.00 KB 0.001 s.\nKNN 0.535 0.423 0.458 0.755 0.871 - \u2248249.85 KB/\u224861.43 KB 0.001 s.\nRF 0.718 0.267 0.370 0.747 0.947 - \u2248139.26 KB/\u22480.00 KB 0.020 s.\nXGBoost 0.428 0.344 0.367 0.719 0.857 - \u22481327.10 KB/\u22480.00 KB 0.026 s.\nEBM 0.713 0.281 0.389 0.745 0.944 - \u22481724.41 KB/\u22480.00 KB 6. 007 s.\nTM (Booleanized) 0.518 0.583 0.531 0.703 0.742 21 \u22480.00 KB/\u22480.00 KB 0.001 s.\nTM (SSLs) 0.465 0.759 0.555 0.645 0.599 4 \u22480.00 KB/\u22480.00 KB 0.001 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5b-934e-11ee-a4f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "5.4. Liver Disorders\nThe Liver Disorders dataset (available from: https://archive.ics.uci.edu/ml/datasets/\nLiver+Disorders, accessed on 24 August 2021) was created in the 1980s by BUPA Medical\nResearch and Development Ltd. (hereafter \u201cBMRDL\u201d) as a part of a larger health-screening\ndatabase. The dataset contains data in seven columns: mean corpuscular volume, alkaline\nphosphotase, alamine aminotransferase, aspartate aminotransferase, gamma-glutamyl"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5c-934e-11ee-a262-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "transpeptidase, number of half-pint equivalents of alcoholic beverages (drunk per day),\nand selector. By taking the selector attribute as a class label, some researchers have used this\ndataset incorrectly [ 52]. However, in our experiments, the \u201cnumber of half-pint equivalents\nof alcoholic beverages\u201d is used as the dependent variable, Booleanized using the threshold\n\u22653. Further, only results of various blood tests are used as feature attributes; i.e., the\nselector attribute is discarded."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5d-934e-11ee-8f69-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "selector attribute is discarded.\nTables 17 and 18 summarize the performance of the TM with two feature arrangements.\nAs can be seen, the F1-Scores of the TM with Booleanized continuous features peak at\nm=2, while this value for the TM with the SSL scheme is m=10. With 10 clauses, the\nmethod of representing continuous features for the TM with theSSL scheme considers\nmerely 9 literals in clauses to acquire a better F1-Score. The increase of the number of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5e-934e-11ee-a8d5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "literals included in TM clauses with the increase of the number of clauses can be seen in\nFigure 6. Again, this con\ufb01rms that the TM with the SSL scheme uses a considerably smaller\nnumber of literals overall.\nTable 17. Performance of TM with Booleanized continuous features on Liver Disorders dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.566 0.540 0.506 0.455 0.442 0.417\nRecall 0.799 0.597 0.508 0.595 0.500 0.593\nF1-Score 0.648 0.550 0.389 0.450 0.375 0.437"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac5f-934e-11ee-ade0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "F1-Score 0.648 0.550 0.389 0.450 0.375 0.437\nAccuracy 0.533 0.540 0.516 0.522 0.526 0.504\nSpeci\ufb01city 0.204 0.436 0.497 0.395 0.500 0.396\nNo. of Lit. 27 51 117 509 2315 8771\nTable 18. Performance of TM with SSL continuous feature scheme on Liver Disorders dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.619 0.591 0.546 0.420 0.414 0.522\nRecall 0.905 0.924 0.605 0.700 0.700 0.407\nF1-Score 0.705 0.709 0.447 0.525 0.520 0.298\nAccuracy 0.587 0.574 0.526 0.546 0.543 0.461"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac60-934e-11ee-85f2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 18, "chroma:document": "Accuracy 0.587 0.574 0.526 0.546 0.543 0.461\nSpeci\ufb01city 0.101 0.098 0.400 0.300 0.300 0.600\nNo. of Lit. 2 9 89 452 1806 7229"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac61-934e-11ee-a58d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "Electronics 2021 ,10, 2107 20 of 25\n2 10 100 500 2000 8000\nNo. of Clauses02000400060008000No. of LiteralsBinarized features\nSSLs scheme\nFigure 6. The number of literals included in TM clauses to work with the Liver Disorders dataset.\nFrom the performance of the other machine learning models, summarized in Table 19,\nwe can observe that the highest F1-Score ( 0.729 ) is produced by RF. The performance of DT\nin terms of the F1-Score is comparable to the performance of RF. However, DT requires a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac62-934e-11ee-827f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "training memory of 49.15 KB, while RF uses a negligibly small memory both for training\nand testing to work with the Liver Disorders dataset. The TM, on the other hand, performs\nbetter with SSL continuous features representation than with the Booleanized continuous\nfeatures. This performance is the fourth best among all the other models. For training and\ntesting, the TMs with both feature representation approaches require an insigni\ufb01cantly"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac63-934e-11ee-81c1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "small amount of memory. However, the TM with SSL feature representation requires less\ntime for training.\nTable 19. Performance comparison for Liver Disorders dataset.\nPrec. Reca. F1 Acc. Spec. No. of Lit.Memory RequiredTraining Time(Training/Testing)\nANN-1 0.651 0.702 0.671 0.612 0.490 - \u2248985.13 KB/\u224818.53 KB 0.305 s.\nANN-2 0.648 0.664 0.652 0.594 0.505 - \u22483689.39 KB/\u2248598.26 KB 0.305 s.\nANN-3 0.650 0.670 0.656 0.602 0.508 - \u224838,365.46 KB/\u22481758.23 KB 0.356 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac64-934e-11ee-b09a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "DT 0.591 0.957 0.728 0.596 0.135 - \u224849.15 KB/\u22480.00 KB 0.025 s.\nSVM 0.630 0.624 0.622 0.571 0.500 - \u22481597.43 KB/\u22480.00 KB 0.005 s.\nKNN 0.629 0.651 0.638 0.566 0.440 - \u22480.00 KB/\u2248434.17 KB 0.001 s.\nRF 0.618 0.901 0.729 0.607 0.192 - \u22480.00 KB/\u22480.00 KB 0.017 s.\nXGBoost 0.641 0.677 0.656 0.635 0.568 - \u22483219.45 KB/\u22480.00 KB 0.081 s.\nEBM 0.641 0.804 0.710 0.629 0.406 - \u22487790.59 KB/\u22480.00 KB 10.772 s.\nTM (Booleanized) 0.566 0.799 0.648 0.533 0.204 27 \u22480.00 KB/\u22480.00 KB 0.003 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac65-934e-11ee-8124-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "TM (SSLs) 0.591 0.924 0.709 0.574 0.098 9 \u22480.00 KB/\u22480.00 KB 0.001 s.\n5.5. Heart Disease\nThe last dataset we use is the Heart Disease dataset (Available from: https://archive.ics.\nuci.edu/ml/datasets/Statlog+%28Heart%29, accessed on 24 August 2021). The goal of\nthis dataset is to predict future heart disease risk based on historical data. The complete\ndataset consists of 75 features. However, in this experiment, the updated version of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac66-934e-11ee-8639-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "dataset, containing 13 features, is used: one ordered, six real-valued, three nominal, and\nthree Boolean features.\nTables 20 and 21 summarize the performance of the TM with two feature arrangement\nschemes. For the TM with Boolean features, the best F1-Score occurs with m=10, achieved\nby using 346literals on average. The F1-Score of the TM with SSL continuous features\npeaks again at m=10with only 42literals. Even though the TM with Boolean features"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac67-934e-11ee-99ca-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 19, "chroma:document": "performs better in terms of accuracy, the TM with SSL feature representation outperforms\nthe Boolean representation of continuous features by obtaining a higher F1-Score."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac68-934e-11ee-b73e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "Electronics 2021 ,10, 2107 21 of 25\nTable 20. Performance of TM with Booleanized continuous features on Heart Disease dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.547 0.607 0.835 0.507 0.351 0.360\nRecall 0.938 0.815 0.626 0.408 0.646 0.486\nF1-Score 0.682 0.687 0.665 0.383 0.446 0.392\nAccuracy 0.593 0.672 0.749 0.619 0.533 0.584\nSpeci\ufb01city 0.306 0.566 0.848 0.803 0.460 0.665\nNo. of Lit. 118 346 810 1425 11,399 52,071"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac69-934e-11ee-acd6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "No. of Lit. 118 346 810 1425 11,399 52,071\nTable 21. Performance of TM with SSL continuous feature scheme on Heart Disease dataset.\nm 2 10 100 500 2000 8000\nPrecision 0.529 0.588 0.562 0.305 0.674 0.687\nRecall 0.971 0.915 0.504 0.431 0.660 0.667\nF1-Score 0.680 0.714 0.510 0.343 0.571 0.555\nAccuracy 0.591 0.674 0.709 0.630 0.633 0.581\nSpeci\ufb01city 0.272 0.471 0.853 0.701 0.582 0.512\nNo. of Lit. 10 42 151 783 3152 12,365"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6a-934e-11ee-8055-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "No. of Lit. 10 42 151 783 3152 12,365\nConsidering the number of literals used with an increasing number of clauses ( Figure 7 ),\nboth approaches behave almost similarly until m=500, and then the TM with Booleanized\nfeatures includes more literals in clauses than the proposed approach.\n2 10 100 500 2000 8000\nNo. of Clauses010,00020,00030,00040,00050,000No. of LiteralsBinarized features\nSSLs scheme\nFigure 7. The number of literals included in TM clauses to work with Heart Disease dataset."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6b-934e-11ee-a661-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "Out of the considered machine learning models, as summarized in Table 22, EBM\nobtains the best F1-Score. However, EBM needs the highest training time and uses the\nsecond largest amount of training memory, while both TMs use negligible memory during\nboth training and testing and consume much less training time than EBM.\nTable 22. Performance comparison for Heart Disease dataset.\nPrec. Reca. F1 Acc. Spec. No. of Lit.Memory RequiredTraining Time(Training/Testing)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6c-934e-11ee-8dde-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "ANN-1 0.764 0.724 0.738 0.772 0.811 - \u2248973.64 KB/\u224816.46 KB 0.297 s.\nANN-2 0.755 0.736 0.742 0.769 0.791 - \u22483659.59 KB/\u2248578.11 KB 0.266 s.\nANN-3 0.661 0.662 0.650 0.734 0.784 - \u224833,952.49 KB/\u22481513.41 KB 0.308 s.\nDT 0.827 0.664 0.729 0.781 0.884 - \u22480.00 KB/\u2248266.23 KB 0.016 s.\nSVM 0.693 0.674 0.679 0.710 0.740 - \u22481363.96 KB/\u2248262.14 KB 0.004 s.\nKNN 0.682 0.615 0.641 0.714 0.791 - \u22480.00 KB/\u2248319.48 KB 0.001 s.\nRF 0.810 0.648 0.713 0.774 0.879 - \u2248413.69 KB/\u22480.00 KB 0.017 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6d-934e-11ee-a8ab-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 20, "chroma:document": "XGBoost 0.712 0.696 0.701 0.788 0.863 - \u22483694.58 KB/\u22480.00 KB 0.057 s.\nEBM 0.827 0.747 0.783 0.824 0.885 - \u22484763.64 KB/\u22480.00 KB 11.657 s.\nTM (Booleanized) 0.607 0.815 0.687 0.672 0.566 346 \u22480.00 KB/\u22480.00 KB 0.014 s.\nTM (SSLs) 0.588 0.915 0.714 0.674 0.471 42 \u22480.00 KB/\u22480.00 KB 0.001 s."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6e-934e-11ee-9e32-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "Electronics 2021 ,10, 2107 22 of 25\n5.6. Summary of Empirical Evaluation\nTo compare the overall performance of the various techniques, we calculate the\naverage F1-Scores across the datasets. Furthermore, to evaluate the overall interpretability\nof TMs, we also report the average number of literals used overall.\nIn brief, the average F1-Scores of ANN-1, ANN-2, ANN-3, DT, SVM, KNN, RF, XG-\nBoost, EBM, TM (Booleanized), and TM (SSL) are 0.770 ,0.757 ,0.744 ,0.742 ,0.713 ,0.737 ,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac6f-934e-11ee-92c8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "0.724 ,0.728 ,0.775 ,0.762 , and 0.782 , respectively. Out of all the considered models, the\nTM with SSL continuous feature representation obtains the best average F1-Score, which\nis0.782 . It should also be noted that increasing ANN model complexity (from ANN-1 to\nANN-3) reduces the overall F1-Score, which can potentially be explained by the small size\nof the datasets.\nIndeed, the average numbers of literals employed are 961for the TM with Booleanized"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac70-934e-11ee-ab56-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "continuous features and 224for the TM with the SSL feature scheme. That is, the TM\nwith SSL feature representation uses 4.3times fewer literals than the TM with Booleanized\ncontinuous features.\nThe average combined memory requirements (training and testing) for the TM ap-\nproaches are 3.27 KB and 1.89 KB for Booleanized features and SSL features, respec-\ntively. The combined memory usage of the TM with SSL feature representation is signif-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac71-934e-11ee-b0f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "icantly less compared to the other models\u2014ANN-1: \u2248528 times, ANN-2: \u22482211 times,\nANN-3:\u224819,197 times, DT:\u224859times, SVM:\u2248441times, KNN:\u22481836 times, RF:\u224878\ntimes, XGBoost:\u22481517 times, and EBM: \u22482121 times.\nIt should also be noted that increasing the number of clauses stabilizes the precision,\nrecall, F1-score, accuracy, and speci\ufb01city measures, rendering variance insigni\ufb01cant. That\nis, variance becomes negligible for all the datasets and feature representations."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac72-934e-11ee-af82-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "5.7. Comparison against Recent State-of-the-Art Machine Learning Models\nIn this section, we compare the accuracy of the TM with reported results on recent state-\nof-the-art machine learning models. First, we perform experiments on Fraud Detection\nand COMPAS: Risk Prediction in Criminal Justice datasets to study the performance of\nthe TM in comparison with Neural Additive Models [ 11]. A Neural Additive Model is\na novel member of the so-called General Adaptive Models. In Neural Additive Models,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac73-934e-11ee-9f32-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "the signi\ufb01cance of each input feature towards the output is learned by a dedicated neural\nnetwork. During the training phase, the complete set of neural networks is jointly trained\nto learn complex interactions between inputs and outputs.\nTo compare the performance against StructureBoost [ 47], we use the CA weather\ndataset [ 53]. For simplicity, we use only the CA-58 subset of the dataset in this study.\nStructureBoost is based on gradient boosting and is capable of exploiting the structure of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cac74-934e-11ee-a5ec-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "categorical variables. StructureBoost outperforms established models such as CatBoost\nand LightBoost on multiple classi\ufb01cation tasks [47].\nSince the performance of both of the above techniques has been measured in terms of\nthe Area Under the ROC Curve (AUC), here, we use a soft TM output layer [ 54] to calculate\nthe AUC. The performance characteristics are summarized in Table 23.\nTable 23 shows that for the Fraud Detection dataset, the TM with the SSL continuous"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21a-934e-11ee-8bc1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "feature representation approach performs on par with XGBoost and outperforms NAMs\nand all the other techniques mentioned in [ 11]. For the COMPAS dataset, the TM with\nthe SSL feature arrangement exhibits competitive performance compared to NAMs, EBM,\nXGBoost, and DNNs. The TM with SSL feature representation shows, however, superior\nperformance compared to Logistic Regression and DT on COMPAS. The performance of\nthe TM on CA-20 is better in comparison to StructureBoost, LightBoost, and CatBoost"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21b-934e-11ee-a5eb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 21, "chroma:document": "models, as reported in [47]."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21c-934e-11ee-b1bd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "Electronics 2021 ,10, 2107 23 of 25\nTable 23. Performance (in AUC) comparison against recent state-of-the-art machine learning models.\nModel Fraud Detection COMPAS CA-58\nLogistic Regression 0.975 0.730 -\nDT 0.956 0.723 -\nNAMs 0.980 0.741 -\nEBM 0.976 0.740 -\nXGBoost 0.981 0.742 -\nDNNs 0.978 0.735 -\nLightBoost - - \u22480.760 \u2020\nCatBoost - - \u22480.760 \u2020\nStructureBoost - - \u22480.764 \u2020\nTM (SSLs) 0.981 0.732 0.770\n\u2020 These results were extracted from graphs in [47].\n6. Conclusions"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21d-934e-11ee-bb98-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "6. Conclusions\nIn this paper, we proposed a novel continuous feature representation approach for\nTsetlin Machines (TMs) using Stochastic Searching on the Line (SSL) automata. The SSLs\nlearn the lower and upper limits of the continuous feature values inside clauses. These\nlimits decide the Boolean representation of the continuous value inside the clauses. We\nhave provided empirical evidence to show that the novel way of representing continuous"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21e-934e-11ee-b67f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "features in the TMs can reduce the number of literals included in the learned TM clauses\nby 4.3 times compared to the Booleanization scheme without loss of performance. Further,\nthe new continuous feature representation is able to decrease the total training time from\n0.177 s to 0.126 s per epoch, and the combined total memory usage is reduced from 16.35 KB\nto 9.45 KB while exhibiting on par or better performance. In terms of the average F1-Score,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd21f-934e-11ee-bd51-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "the TM with the proposed feature representation also outperforms several state-of-the-art\nmachine learning algorithms.\nIn our future work, we intend to investigate the possibility of applying a similar\nfeature representation for multi-class and regression versions of the TM.\nAuthor Contributions: Conceptualization, K.D.A. and O.-C.G.; software, K.D.A. and O.-C.G.; vali-\ndation, K.D.A. and O.-C.G.; formal analysis, K.D.A. and O.-C.G.; investigation, K.D.A., O.-C.G. and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd220-934e-11ee-aa37-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "M.G.; writing\u2014original draft preparation, K.D.A.; writing\u2014review and editing, K.D.A., O.-C.G.\nand M.G.; supervision, O.-C.G. All authors have read and agreed to the published version of the\nmanuscript.\nFunding: This research received no external funding.\nCon\ufb02icts of Interest: The authors declare no con\ufb02ict of interest.\nReferences\n1. Miotto, R.; Wang, F.; Wang, S.; Jiang, X.; Dudley, J.T. Deep learning for healthcare: review, opportunities and challenges. Brief."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd221-934e-11ee-bc57-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "Bioinform. 2018 ,19, 1236\u20131246. [CrossRef] [PubMed]\n2. Bellazzi, R.; Zupan, B. Predictive data mining in clinical medicine: Current issues and guidelines. Int. J. Med. Inform. 2008 ,\n77, 81\u201397. [CrossRef]\n3. Pazzani, M.J.; Mani, S.; Shankle, W.R. Acceptance of rules generated by machine learning among medical experts. Methods Inf.\nMed. 2001 ,40, 380\u2013385.\n4. Baesens, B.; Mues, C.; De Backer, M.; Vanthienen, J.; Setiono, R. Building intelligent credit scoring systems using decision tables."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd222-934e-11ee-8280-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "InEnterprise Information Systems V ; Springer: Berlin/Heidelberg, Germany, 2004; pp. 131\u2013137.\n5. Huysmans, J.; Dejaeger, K.; Mues, C.; Vanthienen, J.; Baesens, B. An empirical evaluation of the comprehensibility of decision\ntable, tree and rule based predictive models. Decis. Support Syst. 2011 ,51, 141\u2013154. [CrossRef]\n6. Lima, E.; Mues, C.; Baesens, B. Domain knowledge integration in data mining using decision tables: Case studies in churn"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd223-934e-11ee-8b84-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 22, "chroma:document": "prediction. J. Oper. Res. Soc. 2009 ,60, 1096\u20131106. [CrossRef]\n7. Verbeke, W.; Martens, D.; Mues, C.; Baesens, B. Building comprehensible customer churn prediction models with advanced rule\ninduction techniques. Expert Syst. Appl. 2011 ,38, 2354\u20132364. [CrossRef]\n8. Freitas, A.A.; Wieser, D.C.; Apweiler, R. On the importance of comprehensible classi\ufb01cation models for protein function prediction.\nIEEE/ACM Trans. Comput. Biol. Bioinform. 2008 ,7, 172\u2013182. [CrossRef] [PubMed]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd224-934e-11ee-9bbd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Electronics 2021 ,10, 2107 24 of 25\n9. Szafron, D.; Lu, P .; Greiner, R.; Wishart, D.S.; Poulin, B.; Eisner, R.; Lu, Z.; Anvik, J.; Macdonell, C.; Fyshe, A.; et al. Proteome\nAnalyst: Custom predictions with explanations in a web-based tool for high-throughput proteome annotations. Nucleic Acids Res.\n2004 ,32, W365\u2013W371. [CrossRef]\n10. Lazreg, M.B.; Goodwin, M.; Granmo, O.C. Deep learning for social media analysis in crises situations. In Proceedings of the 29th"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd225-934e-11ee-a7d7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Annual Workshop of the Swedish Arti\ufb01cial Intelligence Society (SAIS), Malm\u00f6, Sweden, 2\u20133 June 2016; p. 31.\n11. Agarwal, R.; Frosst, N.; Zhang, X.; Caruana, R.; Hinton, G.E. Neural Additive Models: Interpretable Machine Learning with\nNeural Nets. arXiv 2020 , arXiv:2004.13912.\n12. Molnar, C. Interpretable Machine Learning ; Leanpub: Victoria, BC, Canada, 2019.\n13. Ribeiro, M.T.; Singh, S.; Guestrin, C. \u201cWhy should i trust you?\u201d Explaining the predictions of any classi\ufb01er. In Proceedings of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd226-934e-11ee-9b65-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13\u201317 August\n2016; pp. 1135\u20131144.\n14. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.\nLeanpubNat. Mach. Intell. 2019 ,1, 206\u2013215. [CrossRef]\n15. Agrawal, R.; Imieli \u00b4 nski, T.; Swami, A. Mining Association Rules Between Sets of Items in Large Databases. SIGMOD Rec. 1993 ,\n22, 207\u2013216. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd227-934e-11ee-a926-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "22, 207\u2013216. [CrossRef]\n16. McCormick, T.; Rudin, C.; Madigan, D. A Hierarchical Model for Association Rule Mining of Sequential Events: An Approach to\nAutomated Medical Symptom Prediction. Ann. Appl. Stat. 2011 . [CrossRef]\n17. Feldman, V . Hardness of Approximate Two-Level Logic Minimization and PAC Learning with Membership Queries. J. Comput.\nSyst. Sci. 2009 ,75, 13\u201326. [CrossRef]\n18. Valiant, L.G. A Theory of the Learnable. Commun. ACM 1984 ,27, 1134\u20131142. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd228-934e-11ee-9540-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "19. Wang, T.; Rudin, C.; Doshi-Velez, F.; Liu, Y.; Klamp\ufb02, E.; MacNeille, P . A Bayesian Framework for Learning Rule Sets for\nInterpretable Classi\ufb01cation. J. Mach. Learn. Res. 2017 ,18, 2357\u20132393.\n20. Hauser, J.R.; Toubia, O.; Evgeniou, T.; Befurt, R.; Dzyabura, D. Disjunctions of Conjunctions, Cognitive Simplicity, and\nConsideration Sets. J. Mark. Res. 2010 ,47, 485\u2013496. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd229-934e-11ee-936c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "21. Liang, Y.; Van den Broeck, G. Learning logistic circuits. In Proceedings of the 33rd AAAI Conference on Arti\ufb01cial Intelligence,\nHilton Hawaiian Village, Honolulu, HI, USA, 27 January\u20131 February 2019; Volume 33, pp. 4277\u20134286.\n22. Nori, H.; Jenkins, S.; Koch, P .; Caruana, R. InterpretML: A Uni\ufb01ed Framework for Machine Learning Interpretability. arXiv 2019 ,\narXiv:1909.09223."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22a-934e-11ee-9f67-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "arXiv:1909.09223.\n23. Lou, Y.; Caruana, R.; Gehrke, J. Intelligible models for classi\ufb01cation and regression. In Proceedings of the 18th ACM SIGKDD\nInternational Conference on Knowledge Discovery and Data Mining, Beijing, China, 12\u201316 August 2012; pp. 150\u2013158.\n24. Granmo, O.C. The Tsetlin Machine\u2014A game theoretic bandit driven approach to optimal pattern recognition with propositional\nlogic. arXiv 2018 , arXiv:1804.01508."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22b-934e-11ee-854e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "logic. arXiv 2018 , arXiv:1804.01508.\n25. Phoulady, A.; Granmo, O.C.; Gorji, S.R.; Phoulady, H.A. The Weighted Tsetlin Machine: Compressed Representations with\nClause Weighting. In Proceedings of the Ninth International Workshop on Statistical Relational AI (StarAI2020), New York, NY,\nUSA, 7 February 2020.\n26. Berge, G.T.; Granmo, O.C.; Tveit, T.O.; Goodwin, M.; Jiao, L.; Matheussen, B.V . Using the Tsetlin Machine to learn human-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22c-934e-11ee-a80a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "interpretable rules for high-accuracy text categorization with medical applications. IEEE Access 2019 ,7, 115134\u2013115146. [CrossRef]\n27. Abeyrathna, K.D.; Granmo, O.C.; Zhang, X.; Jiao, L.; Goodwin, M. The Regression Tsetlin Machine\u2014A Novel Approach to\nInterpretable Non-Linear Regression. Philos. Trans. R. Soc. A 2019 ,378, 20190165. [CrossRef]\n28. Gorji, S.R.; Granmo, O.C.; Phoulady, A.; Goodwin, M. A Tsetlin Machine with Multigranular Clauses. In Lecture Notes in Computer"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22d-934e-11ee-8176-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Science: Proceedings of the Thirty-ninth International Conference on Innovative Techniques and Applications of Arti\ufb01cial Intelligence\n(SGAI-2019) ; Springer: Berlin/Heidelberg, Germany, 2019; Volume 11927.\n29. Gorji, S.R.; Granmo, O.-G.; Glimsdal, S.; Edwards, J.; Goodwin, M. Increasing the Inference and Learning Speed of Tsetlin\nMachines with Clause Indexing. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22e-934e-11ee-926f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Systems ; Springer: Berlin/Heidelberg, Germany, 2020.\n30. Wheeldon, A.; Sha\ufb01k, R.; Yakovlev, A.; Edwards, J.; Haddadi, I.; Granmo, O.C. Tsetlin Machine: A New Paradigm for Pervasive\nAI. In Proceedings of the SCONA Workshop at Design, Automation and Test in Europe (DATE), Grenoble, France, 9\u201313\nMarch 2020.\n31. Tsetlin, M.L. On behaviour of \ufb01nite automata in random medium. Avtomat. Telemekh 1961 ,22, 1345\u20131354."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd22f-934e-11ee-a8a0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "32. Yadav, R.K.; Jiao, L.; Granmo, O.C.; Goodwin, M. Interpretability in Word Sense Disambiguation using Tsetlin Machine. In\nProceedings of the 13th International Conference on Agents and Arti\ufb01cial Intelligence (ICAART), Vienna, Austria, 4 February\n2021; INSTICC: Lisboa, Portugal, 2021.\n33. Bhattarai, B.; Jiao, L.; Granmo, O.C. Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd230-934e-11ee-9c33-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Machine Text Classi\ufb01er. In Proceedings of the 13th International Conference on Agents and Arti\ufb01cial Intelligence (ICAART),\nVienna, Austria, 4 February 2021; INSTICC: Lisboa, Portugal, 2021.\n34. Sharma, J.; Yadav, R.; Granmo, O.C.; Jiao, L. Human Interpretable AI: Enhancing Tsetlin Machine Stochasticity with Drop Clause.\narXiv 2021 , arXiv:2105.14506.\n35. Lei, J.; Rahman, T.; Sha\ufb01k, R.; Wheeldon, A.; Yakovlev, A.; Granmo, O.C.; Kawsar, F.; Mathur, A. Low-Power Audio Keyword"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd231-934e-11ee-b1ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 23, "chroma:document": "Spotting Using Tsetlin Machines. J. Low Power Electron. Appl. 2021 ,11, 18. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd232-934e-11ee-b5ef-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Electronics 2021 ,10, 2107 25 of 25\n36. Granmo, O.C.; Glimsdal, S.; Jiao, L.; Goodwin, M.; Omlin, C.W.; Berge, G.T. The Convolutional Tsetlin Machine. arXiv 2019 ,\narXiv:1905.09688.\n37. Abeyrathna, K.D.; Granmo, O.C.; Zhang, X.; Goodwin, M. A scheme for continuous input to the Tsetlin Machine with applications\nto forecasting disease outbreaks. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd233-934e-11ee-b296-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Systems ; Springer: Berlin/Heidelberg, Germany, 2019; pp. 564\u2013578.\n38. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased\nInterpretability. IEEE Access 2021 ,9, 8233\u20138248. [CrossRef]\n39. Wheeldon, A.; Sha\ufb01k, R.; Rahman, T.; Lei, J.; Yakovlev, A.; Granmo, O.C. Learning Automata based Energy-ef\ufb01cient AI Hardware\nDesign for IoT. Philos. Trans. R. Soc. A 2020 ,378, 20190593. [CrossRef] [PubMed]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd234-934e-11ee-ad1d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "40. Abeyrathna, K.D.; Bhattarai, B.; Goodwin, M.; Gorji, S.; Granmo, O.C.; Jiao, L.; Saha, R.; Yadav, R.K. Massively Parallel and\nAsynchronous Tsetlin Machine Architecture Supporting Almost Constant-Time Scaling. In Proceedings of the Thirty-eighth\nInternational Conference on Machine Learning (ICML2021), Virtual, 18\u201324 July 2021.\n41. Sha\ufb01k, R.; Wheeldon, A.; Yakovlev, A. Explainability and Dependability Analysis of Learning Automata based AI Hardware. In"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd235-934e-11ee-86f8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Proceedings of the IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS), Napoli, Italy,\n13\u201315 July 2020.\n42. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern\nRepresentation. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems ; Springer:\nBerlin/Heidelberg, Germany, 2020."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd236-934e-11ee-95e1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Berlin/Heidelberg, Germany, 2020.\n43. Zhang, X.; Jiao, L.; Granmo, O.C.; Goodwin, M. On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators.\nIEEE Trans. Pattern Anal. Mach. Intell. 2021 , 1. [CrossRef]\n44. Jiao, L.; Zhang, X.; Granmo, O.C.; Abeyrathna, K.D. On the Convergence of Tsetlin Machines for the XOR Operator. arXiv 2021 ,\narXiv:2101.02547."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd237-934e-11ee-bde5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "arXiv:2101.02547.\n45. Oommen, B.J. Stochastic searching on the line and its applications to parameter learning in nonlinear optimization. IEEE Trans.\nSyst. Man Cybern. Part B (Cybern.) 1997 ,27, 733\u2013739. [CrossRef]\n46. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. Adaptive Sparse Representation of Continuous Input for Tsetlin Machines Based\non Stochastic Searching on the Line. 2020. In Preparation."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd238-934e-11ee-9753-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "47. Lucena, B. StructureBoost: Ef\ufb01cient Gradient Boosting for Structured Categorical Variables. arXiv 2020 , arXiv:2007.04446.\n48. Narendra, K.S.; Thathachar, M.A. Learning Automata: An Introduction ; Courier Corporation: Chelmsford, MA, USA, 2012.\n49. Thathachar, M.A.L.; Sastry, P .S. Networks of Learning Automata: Techniques for Online Stochastic Optimization ; Kluwer Academic\nPublishers: London, UK, 2004."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd239-934e-11ee-b2de-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Publishers: London, UK, 2004.\n50. Chen, T.; Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd Acm Sigkdd International Conference\non Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13\u201317 August 2016; pp. 785\u2013794.\n51. Kim, M.J.; Han, I. The discovery of experts\u2019 decision rules from qualitative bankruptcy data using genetic algorithms. Expert Syst.\nAppl. 2003 ,25, 637\u2013646. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd23a-934e-11ee-96bd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Appl. 2003 ,25, 637\u2013646. [CrossRef]\n52. McDermott, J.; Forsyth, R.S. Diagnosing a disorder in a classi\ufb01cation benchmark. Pattern Recognit. Lett. 2016 ,73, 41\u201343. [CrossRef]\n53. Lucena, B. Exploiting Categorical Structure Using Tree-Based Methods. arXiv 2020 , arXiv:2004.07383.\n54. Abeyrathna, K.D.; Granmo, O.C.; Goodwin, M. On Obtaining Classi\ufb01cation Con\ufb01dence, Ranked Predictions and AUC with"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd23b-934e-11ee-bc2b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\electronics-10-02107-v2.pdf", "page": 24, "chroma:document": "Tsetlin Machines. In Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, ACT,\nAustralia, 1\u20134 December 2020; pp. 662\u2013669. [CrossRef]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd27c-934e-11ee-91a4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "Self-timed Reinforcement Learning\nusing Tsetlin Machine\nAdrian Wheeldon, Alex Y akovlev and Rishad Sha\ufb01k\nMicrosystems Group, Newcastle University, UK\nEmail: {a.r.wheeldon2, alex.yakovlev, rishad.sha\ufb01k}@ncl.ac.uk\nAbstract \u2014We present a hardware design for the learning\ndatapath of the Tsetlin machine algorithm, along with a latency\nanalysis of the inference datapath. In order to generate a\nlow energy hardware which is suitable for pervasive arti\ufb01cial"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd27d-934e-11ee-ab66-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "intelligence applications, we use a mixture of asynchronous design\ntechniques\u2014including Petri nets, signal transition graphs, dual-\nrail and bundled-data. The work builds on previous design of\nthe inference hardware, and includes an in-depth breakdown\nof the automaton feedback, probability generation and Tsetlin\nautomata. Results illustrate the advantages of asynchronous de-\nsign in applications such as personalized healthcare and battery-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd27e-934e-11ee-bdd3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "powered internet of things devices, where energy is limited and\nlatency is an important \ufb01gure of merit. Challenges of static timing\nanalysis in asynchronous circuits are also addressed.\nI. I NTRODUCTION\nWe present a comprehensive design and analysis for an\nasynchronous learning datapath based on the Tsetlin machine\n(TM) algorithm. When coupled with asynchronous inference\nhardware [1], a complete asynchronous TM capable of online\nlearning is formed. We also analyze the latency of the infer-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd27f-934e-11ee-8a43-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "ence datapath, showing its evolution during training.\nThe TM [2] algorithm is effective in many large classi\ufb01ca-\ntion problems [3, 4, 5]. In addition, the TM\u2019s reinforcement\nlearning and logic-based inference make it a good candidate\nfor energy ef\ufb01cient hardware [6]. We design an energy-frugal\nTM hardware with a view of use cases in pervasive arti\ufb01cial\nintelligence; e.g. in personal healthcare, accessibility, environ-\nmental monitoring and predictive maintenance. We give a brief"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd280-934e-11ee-8aee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "introduction to the TM algorithm in Section II.\nWe wish to implement the TM using asynchronous circuits\nas they can be bene\ufb01cial for low energy sensing systems [7],\nwhen tightly coupled with analog blocks [8], and when power\ndelivery is unstable or unpredictable [9]. These features are\noften present in the aforementioned applications [10].\nThe architecture of the hardware is designed using a hy-\nbrid Petri net model in Section III. The model incorporates\ninference and learning components."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd281-934e-11ee-92cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "inference and learning components.\nTMs use groups of reinforcement automata, called Tsetlin\nautomata ( TA s), to create an ensemble learning effect. We\ndecompose the automata reinforcement feedback into three\nstages to aid the hardware design in Section IV.\nThe reinforcement process of the TM involves random\nchoice with de\ufb01ned probabilities\u2014some \ufb01xed and some vary-\ning at runtime. These probabilistic choices enable diversity\nof learning in the TA s. We design a quasi delay insensitive"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd282-934e-11ee-bd0c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "(QDI ) pseudorandom bit generator ( PRBG ) for this use in theClauses Automaton T eams\nTA TA\nMajority VotingClause Resultsexclude\nClass Con\ufb01denceFeedback\nClassi\ufb01cationInput Feature Vector ( f) PRBG\nLearning\nFigure 1. Overview of the Tsetlin machine ( TM) architecture.\nTM in Section V, and motivate its use in other low power\napplications.\nConsidering the implementation of the state-holding TA s,w e\ncompare QDI and bundled-data ( BD) design styles suitable for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd283-934e-11ee-9ad9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "the low energy applications in Section VI. The comparison\nwill show the limitations of the two asynchronous design\nmethods in terms of area, energy and scalability. We use the\nWORKCRAFT framework [11] for speci\ufb01cation and synthesis\nof the QDI implementation using signal transition graphs\n(STGs ), as well as veri\ufb01cation of the STG properties required for\na functional and robust design. For BD, we are inspired by the\ndesynchronization work\ufb02ow of Cortadella et al. [12] in using"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd284-934e-11ee-b52c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "a commercial synthesis tool designed for synchronous design.\nBD circuits are readily-implementable with industrial-quality\ntools, in contrast with QDI circuits.\nMain contributions of this work:\n\u2022Visualization of an asynchronous TM hardware architec-\nture using Petri nets (Section III).\n\u2022Latency analysis of the asynchronous TM inference datap-\nath from [1] and its evolution during training (Section III).\n\u2022A detailed decomposition of TA reinforcement feedback\ninto three stages (Section IV)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd285-934e-11ee-b639-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "into three stages (Section IV).\n\u2022Design of a QDI PRBG for probabilistic choice in the TM\nand other low power applications (Section V).\n\u2022Comparison of QDI and BD implementations of the TAfor\nlow energy hardware (Section VI).\nII. T SETLIN MACHINES IN BRIEF\nTMs learn patterns in binary data using propositional logic.\nThe main inference component of the TM is the clause that\ncomposes an AND expression of the input features and their"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd286-934e-11ee-8eba-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 0, "chroma:document": "402021 27th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC)\n978-1-7281-4132-9/21/$31.00 \u00a92021 IEEE\nDOI 10.1109/ASYNC48570.2021.000142021 27th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC) | 978-1-7281-4132-9/21/$31.00 \u00a92021 IEEE | DOI: 10.1109/ASYNC48570.2021.00014\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd287-934e-11ee-b4cc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "1 2 n\u00b7\u00b7\u00b7 n+1 n+2 2n\u00b7\u00b7\u00b7Action 1 Exclude Action 2 Include\nPenalty Reward\nFigure 2. Tsetlin automaton state graph.\ncomplements. The TM comprises many clauses, each produc-\ning a vote. The composition of each clause is controlled by a\nvector of exclude bits (see Figure 1). These bits are parameters\nthat are learned by teams of TA s.\nEach clause can produce a vote for its class. The algorithm\nstates that half of the clauses vote positively, while the other"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd288-934e-11ee-aa72-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "half of the clauses vote negatively (we will denote this by\nthe boolean c neg). The inclusion of inhibition in the voting\nsystem enables non-linear decision boundaries in the infer-\nence process. A majority vote gives an indication of class\ncon\ufb01dence. This con\ufb01dence is used to classify the input data\nand in\ufb02uence future decisions of the automata through the\nfeedback mechanism [2]. In this work we consider only the\nsingle class TM for simplicity."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd289-934e-11ee-9a38-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "single class TM for simplicity.\nThe TA is a class of \ufb01nite reinforcement automaton [13].\nIt produces an exclude output for states below the midpoint,\nand include for states above the midpoint as illustrated in\nFigure 2. The TA receives a penalty or reward from the\nfeedback mechanism based on the current state of the TM.\nContinued rewards in the end states cause the TAto saturate.\nA penalty in one of the midstates ( norn+1 ) causes the\nTA to transition across the decision boundary\u2014inverting its"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28a-934e-11ee-8a83-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "output from exclude to include, or vice versa. The feedback\nmechanism is described in more detail in Section IV.\nIII. A SYNCHRONOUS TSETLIN MACHINE\nFigure 3 is an architectural diagram for the asynchronous\nimplementation of the TM shown in Figure 1, demonstrating\nthe scalability of the design. The diagram can be composed\nwith the Petri-net-like tiles in Figure 4 to form a visualization\nof the complete system. In these diagrams, rectangles represent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28b-934e-11ee-8fd9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "transitions or computations, and circles represent places as in\nclassical Petri nets. When abutting these tiles, adjoining places\nare reduced to a single place. This representation of the system\nis not a formalism.\nThe main structure is a 3\u00d73grid constructed with feature\ninputs in rows and clauses in columns. The small size facil-\nitates the explanation of the system, but the architecture can\nbe easily extended by appending clause columns to the right\nside, or feature rows below f3."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28c-934e-11ee-8c5f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "side, or feature rows below f3.\nThe clause tile is detailed in Figure 4a and comprises a\npartial clause ( PC) computation for the feature input and its\ncomplement (as introduced in Section II); a partial feedback\ncalculation for stage three feedback ( FB3 ); and two TA s. The\nfeedback stages will be discussed later in Section V.\nThe clause sum is computed in the sum tile at the bottom\nof each clause column. Figure 4b details the sum tile. StageClause\nTileClause\nTileClause\nTile\nClause\nTileClause"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28d-934e-11ee-a2c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "TileClause\nTileClause\nTile\nClause\nTileClause\nTileClause\nTile\nClause\nTileClause\nTileClause\nTile\nSum\nTileSum\nTileSum\nTilef1\nf2\nf3\nFB1 calcyexp\nlearnClassi\ufb01cationConcurrency Opportunity\nFigure 3. Architectural diagram for a Tsetlin machine with three feature\ninputs (rows) and three clauses (columns).\n(a) Clause tile. (b) Sum tile.\nFigure 4. Petri-net-like visualizations for the Tsetlin machine system: (a) the\nclause tile including partial clause ( PC) and two Tsetlin automata ( TA s ); and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28e-934e-11ee-9885-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "(b) the sum tile.\ntwo feedback ( FB2 ) for the clause is computed at this stage\nusing the clause sum.\nThe diagrams capture the potential for both concurrency and\nserialization in the design. As presented, the system operates\nin a serialized manner where the feature inputs are passed\nthrough each clause in series. By contracting the places in\nthe highlighted regions of Figure 4a, tokens are passed to the\nright without waiting for PCcalc and FB3 calc , this allowing"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd28f-934e-11ee-885b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "clauses to compute in parallel.\nThe majority of the system is implemented in a dual-\nrail ( DR) style using a reduced completion detection ( RCD )\nscheme [7]. However, the TA s are implemented in a single-rail\n(SR) style (as indicated in Figure 4a) which will be motivated\nand discussed in Section VI.\nThe RCD scheme is a simpli\ufb01ed scheme which completion\ndetects only the outputs of a block. This brings bene\ufb01ts of\nearly propagation and drastically reduced area and power"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd290-934e-11ee-ba27-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 1, "chroma:document": "overheads versus traditional full completion detection. The\ntrade-off is an added timing assumption on the return-to-idle\nphase of the DR signals. Full details of the RCD scheme are\n41\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd291-934e-11ee-9951-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "fpep\n0fnep\n1\npcnfnen\n0fpen\n1\npcp\n(a) Partial clause ( PC).\ncpcn... ...\n(b) Partial clause combiner.HA0 HA1 HA2 HA3\nHA4 HA5 HA6 HA7\nHA8 FA0 FA1a0a1 a2a3 a4a5 a6a7\ns cs c\nscs\nc\nspinvc\ny0 y1 y2spinv\ny3\n(c) Population count with implicit\ndual-rail wires and blocks.\n\u0001\u0002\u0003\u0004\u0004\u0005\u0006\u0001\u0002\u0003\u0004\u0004\u0005\u0007\n(d) Magnitude comparator.\nFigure 5. Dual-rail circuits for inference. Spinv = spacer inverter.\navailable in [7].\nA. Inference Circuits\nThe inference datapath comprises the clauses, majority"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd292-934e-11ee-95de-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "voting and classi\ufb01cation introduced in Figure 1. Clauses are\nseparated into partial clause calculations as discussed earlier\nin this section and can be implemented by the schematics in\nFigures 5a and 5b. Feature inputs occur in DR encoding along\nwith exclude signals e0ande1. The outputs of the partial clause\nare combined using a DR AND tree.\nMajority voting calculated the sum of the clause votes ( csum)\nand is implemented using DR population count. The schematic"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd293-934e-11ee-8d72-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "in Figure 5c shows the implementation based on DR half-\nand full-adders. The OR gates and wires in this circuit are\nalso implicitly DR encoded. Two spacer inverters (spinv) are\nrequired to ensure uniform spacer polarity at the outputs.\nFor this single class TM example, the output is classi\ufb01ed\nusing a threshold function in the form of a magnitude com-\nparator. The magnitude comparator lends itself to low energy\nimplementation in DR and saves energy by evaluating single-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd294-934e-11ee-ac8b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "bit comparisons from most signi\ufb01cant bit to least signi\ufb01cant\nbit only if needed, all within the period of one cycle [1].\nTo expand the example to two classes or more, an argmax\nfunction would take the place of thresholding.\nIn the following sections we will explore the inference\ncircuit delays and analyze the inference latency of the asyn-chronous datapath with differing input operands. For synthesis\nwe use a commercial, off-the-shelf technology library: UMC"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd295-934e-11ee-912a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "65 nm LOW-LEAKAGE , herein referred to as UMC 65LL.\nB. Delay Analysis\nSince we use an RCD scheme which enables early propaga-\ntion [1], the delays of the circuits can vary based on the input\noperands. We analyze the delay distributions of the inference\ncircuits using uniformly-distributed random inputs to gain an\nunderstanding of this phenomenon. We will use this to explain\nthe evolution of latency during training in the next section.\nClause: In the worst-case delay no literals are excluded"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd296-934e-11ee-a697-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "(e=0 ), all feature inputs are logic-1 ( f=1 ); therefore the\ndelay is bounded by the slowest input and the delay arc is\nthrough the positive rail. In the best-case delay, all but one\nliteral is excluded and the literal is logic-0; therefore the delay\nis bounded by signal propagation through the OR tree in the\nnegative rail. Since the clause is a large AND tree, there is a\nhigh probability of c=0 , so the delay distribution is mostly\nclustered at the lower end as Figure 6 shows. A much smaller"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd297-934e-11ee-b310-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "portion is seen towards the circuit\u2019s maximum delay and is\nassociated with c=1 .\nPopulation Count: The delay of the circuit is bounded\nby the carry chain formed by HA8,FA0and FA1. In the best\ncase there are no carries and the circuit latency is governed by\nthe longest logic path which comprises the \ufb01rst two layers of\nHAs (which have roughly equal paths), through the OR gates\nand \ufb01nally FA0to the y1output. In the worst case, output y3\nmust wait for the aforementioned best-case path as well as"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd298-934e-11ee-9fd9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "the carry through FA1. This leads to the log-normal-like delay\ndistribution seen in Figure 6.\nMagnitude Comparator: For uniform random inputs and\na normalized worst-case delay of 1, the comparator achieves\na mean delay of 0.05. This is attributed to early propagation\nand the evaluation of single-bit comparisons from MSB toLSB .\nThe delay of the circuit increases as the absolute difference\nbetween operands decreases, because more of the MSBs are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd299-934e-11ee-b782-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "equal. This circuit is therefore fastest during inference and\nthe later stages of training. The operand space halves for each\nsingle-bit comparison leading to the negative exponential delay\ndistribution in Figure 6.\nC. Operand-dependent Inference Latency\nWe investigate the evolution of inference latency during\ntraining by plotting the spacer \u2192codeword latency at different\nstages. We use a binarized version of the I RIS Flower dataset1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29a-934e-11ee-8500-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "to provide feature inputs ( f). Exclude inputs ( e) are retrieved\nfrom a TM pre-trained on the same dataset after the speci\ufb01ed\nnumber of epochs. Figure 7 shows the distributions for the\nuntrained TM, and the trained TM after 4 epochs and 50 epochs.\nThe plots show the trend of the mean towards greater circuit\ndelay as the TM is trained and therefore more clauses become\nactivated during the computation of each datapoint ( c=1 ).\nIn this case we see more examples from the right side of"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29b-934e-11ee-9a5f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 2, "chroma:document": "1Available: https://github.com/cair/TsetlinMachine\n42\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29c-934e-11ee-8854-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "Meanc=0 c=1\n00.20.4Clause\nMean00.20.4Population Count\nMean\nDecreasing Operand Delta\n00.20.4Magnitude Comparator\nMean\n00 .20 .40 .60 .8100.20.4\nNormalized DelayInference DatapathProbability\nFigure 6. Latency distributions of the Tsetlin machine inference datapath and\nits components for uniformly-distributed random inputs. Post-synthesis results\nusing the UMC 65 nm LOW-LEAKAGE library.\nMean\n00.20.4Untrained\nMean\n00.20.4Trained 4 epochs ( 81 % Accuracy)\nMean\n140 160 180 200 22000.20.4"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29d-934e-11ee-931b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "Mean\n140 160 180 200 22000.20.4\nLatency ( ps)Trained 50 epochs ( 96 % Accuracy)Probability\nFigure 7. Latency distribution of the Tsetlin machine inference datapath for\nthe I RIS dataset.\nthe clause delay distribution in Figure 6. This is the main\ncontributor to the increase in mean latency as the system\nbecomes more trained. The magnitude comparator offers a\nnetdecrease in mean latency as training progresses since the\ndifference between input operands will increase, meaning we"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29e-934e-11ee-b6b6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "shift towards the left of the distribution in Figure 7. However\nthis shift is somewhat smaller than that of the clauses, and\nit therefore has a lesser effect on the inference datapath. The\ndistribution of the population count is much more uniform\nthan that of the other circuits, and therefore has a negligible\neffect during training.IV . D ECOMPOSITION OF FEEDBACK\nDuring learning, feedback is given to each TAto in\ufb02uence\nits future action, and therefore control pattern detection in the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd29f-934e-11ee-9fb9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "TM as a whole. The feedback is based on a carefully-designed\npayoff matrix [2, p. 10] and is split into two categories: Type\nI and Type II. Type I feedback reinforces good decisions and\npenalizes false negative outputs. Type II feedback combats\nfalse positives by penalizing TA s that exclude when c=1 .\nWe decompose the feedback logic into three stages based on\nscope: FB1 atTM level, FB2 at clause level, and FB3 atTAlevel.\nFB1 and FB2 are assigned types: none, T1or T2; these are based"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a0-934e-11ee-a7fb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "on the Type I and Type II concepts introduced previously. FB3\nassigns an action to a single TA: penalty, reward or inaction;\nand is based on the outcomes of the payoff matrix2. Each\nfeedback stage\u2019s output is input to the next, until \ufb01nally the\noutput of FB3 is input to the corresponding TA. The following\nsections discuss each feedback stage in detail. Finally we will\ndiscuss the synthesis of the feedback circuits in the DR style\nin Section IV -D.\nA. FB1:TM-level Feedback\nInputs: learn ,yexp"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a1-934e-11ee-be86-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "A. FB1:TM-level Feedback\nInputs: learn ,yexp\nOutput: Feedback type (none, T1or T2)\nThe \ufb01rst stage broadly enables feedback for the entire TM if\nlearn is asserted. The feedback type is chosen according to\nyexp \u2014the expected class as provided by the labeled data\u2014as\nfollows:\nFB1=\u23a7\n\u23aa\u23a8\n\u23aa\u23a9none ,forlearn =0\nT1, forlearn =1,yexp =1\nT2, forlearn =1,yexp =0\nB. FB2: Clause-level Feedback\nInputs: FB1 ,c neg,pT\nOutput: Feedback type (none, T1or T2)\nThe second stage swaps the feedback type from T1 to T2,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a2-934e-11ee-aed8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "or vice versa if the associated clause is negated (c neg=1 ,\ndiscussed in Section II). Additionally, feedback is stopped\n(inaction given) if the number of activated clauses (those\nproducing a logic-1 at the output) meets the threshold, T. This\nis determined by probabilistic choice, pT, which is discussed\nat the end of this section. If there is no feedback from the\nprevious stage ( FB1 =none), then none will be generated\n(FB2=none). Parameter c neg is determined at design time,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a3-934e-11ee-83eb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "therefore the logic for FB1 can be separated and depends only\non FB1 and pT. This will be demonstrated in Section IV -D.\nClause-level feedback can be summarized as follows:\nFB2=\u23a7\n\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23a8\n\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23aa\u23a9none ,for FB1=none\nnone ,for FB1=T1,pT=0\nnone ,for FB1=T2,pT=1\nT1, for c neg=1,FB1=T2,pT=0\nT2, for c neg=1,FB1=T1,pT=1\nT1, for c neg=0,FB1=T1,pT=1\nT2, for c neg=0,FB1=T2,pT=0\n2We do not consider boosting of true positive feedback, which is an option\nin the original algorithm.\n43"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a4-934e-11ee-8a90-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 3, "chroma:document": "in the original algorithm.\n43\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a5-934e-11ee-9071-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "Table I\nTRUTH T ABLE FOR FB3 .\u00d7= DON \u2019TC A R E .\nFB2 inc c x p s FB3\nnone\u00d7 \u00d7\u00d7\u00d7 inaction\nT1 10\u00d70 penalty\nT1 10\u00d71 inaction\nT1 11\u00d70 inaction\nT1 11\u00d71 reward\nT1 00\u00d70 reward\nT1 00\u00d71 inactionFB2inc c x p s FB3\nT1 01 0 0 inaction\nT1 01 0 1 reward\nT1 01 1 0 inaction\nT1 01 1 1 penalty\nT2 1\u00d7\u00d7\u00d7 inaction\nT2 01 0 \u00d7 penalty\nT2 00\u00d7\u00d7 inaction\nThe TM algorithm describes probabilities of generating T1\nand T2[2, p. 10]. In this work we refer to them as P(T1)and\nP(T2)which are based on csum andTas follows:"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a6-934e-11ee-baa9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "P(T2)which are based on csum andTas follows:\nP(T1)=T\u2212clamp (csum)\n2T,P(T2)=T+clamp (csum)\n2T,\nwhere the clamp function restricts its argument to [\u2212T,T]such\nthat the probabilities lie in the range [0,1]. As the number\nof activated clauses (those producing c=1 ) approaches T,\nthe probability of the TA s in that clause receiving feedback\ndecreases. In order to generate a randomized choice based on\nthese probabilities, we could assign booleans as follows:\nq1=rand <P(T1),q2=rand <P(T2)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a7-934e-11ee-b19a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "q1=rand <P(T1),q2=rand <P(T2)\nwhere rand is a real number in the range [0,1]. Since P(T1)\nandP(T2)are complementary in the range [0,1], q1and q2\nare also complementary in the binary domain and therefore\nq2=q1. We therefore introduce a single new signal for FB2 ,\npT, which takes the place of the preceding probabilities. The\ngeneration of this signal will be discussed in Section V.\nC. FB3:TA-level Feedback\nInputs: FB2 ,inc,c,x,ps\nOutput: TAaction (inaction, penalty or reward)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a8-934e-11ee-9fa7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "Output: TAaction (inaction, penalty or reward)\nThe third and \ufb01nal feedback stage translates the feedback\ntype from FB2 into a TAaction for the speci\ufb01ed TAbased on:\nthe TA\u2019s current action, encoded as include in signal inc; the\ncurrent clause output, c; the feature of complemented feature\n(forf) associated with the TA,x; and a probabilistic choice\ncomponent, ps.\nThe FB3 output is chosen according to Table I, where\nps=1 denotes that the higher probability (s\u22121)/soption is"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2a9-934e-11ee-86b6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "chosen from the payoff matrix. ps=0 denotes that the lower\nprobability 1/sis chosen. The generation of signal pswill be\ndiscussed in Section V.\nD. Synthesis of Feedback Circuits\nUsing one-hot encoding in the feedback circuits main-\ntains direct compatibility with the inference datapath and\nensures speed independence, which is important for hazard-\nfreeness. Signals inc,c,xare already one-hot encoded (dual-\nrail) for the inference circuits in Section III, so these en-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2aa-934e-11ee-a938-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "codings are reused here in the learning circuits. Further-\nmore, we encode FB1 as follows, and FB2 similarly: FB1=\n{FB1T2,FB1T1,FB1none}.FB3 describes TAactions and is encoded\nas: FB3 ={FB3r,FB3p,FB3i}, for reward, penalty, inactionrespectively. Note that rail orders do not matter as we refer to\nthe rails by name. In this way, to indicate T2feedback on FB1 ,\nwe would set FB1T2=1,FB1T1=0,FB1none=0 .\nUsing the previously de\ufb01ned logic, we can generate circuits"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ab-934e-11ee-95ac-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "for the feedback. The implementation of stage two feedback\ndepends on the clause polarity\u2014it being either negated (c neg=\n1) or non-negated (c neg=0 ). Note that the stage one circuit is\ninstantiated once per TM, the stage two circuit once per clause ,\nand the stage three circuit once per TA.\nIn stage three, the FB2 signal is shared throughout the TA s\nwithin the same clause. For this reason, we carefully design the\nlogic so that FB2 signals are injected as close to the outputs as"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ac-934e-11ee-9e03-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "possible, making the propagation path the shortest. Therefore\nwhen FB2=0 , the computation on the corresponding FB3 rails\nwill conclude rapidly, and for all TA s in the clause.\nV. G ENERA TION OF RANDOM BITSpT,ps\nWe introduced two distinct probabilistic choice mechanisms\nin Section IV. In Section IV -B we introduced the signal pT:\na probabilistic boolean chosen at the clause level which may\nforce FB2=none. This boolean is dependent on Tand csum."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ad-934e-11ee-ab43-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "The probability of pT=1 varies at runtime according to csum.\nIn Section IV -C we introduced the signal ps, which is a random\nboolean generated for each TA. It is required under certain\ncircumstances to choose between two TAactions. This boolean\nis dependent on sand is \ufb01xed during runtime.\nFrom these requirements, we must be able to generate\nbiased random bits: bits where P(1)\u0338=0.5(unbiased random\nbits have P( 1 )=0 .5, such as those generated by a linear"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ae-934e-11ee-badb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "feedback shift register ( LFSR )). In addition, we must be able\nto vary P(1) at runtime for pT. Probabilistic choices are used\nin the TM to ensure diversity of learning. A pseudorandom\ngenerator is suf\ufb01cient to satisfy these needs, and bene\ufb01ts from\nhigher energy ef\ufb01ciency than a true random generator.\nDue to the requirement of PRBGs for each TA, and addition-\nally for each clause, their area and energy consumption are of\nutmost importance. We chose to implement a PRBG based on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2af-934e-11ee-8285-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "the principles of irregular sampling of a regular waveform, as\nthis allows us to minimize the overheads and take advantage\nof the asynchronous nature of the rest of the system.\nA. Asynchronous Sampling of Clock with V ariable Duty Cycle\nWe take advantage of the asynchronous inference logic by\nusing asynchronous handshakes to sample a regular clock\nwaveform. A clock with 50 % duty cycle will generate un-\nbiased random bits. A clock can be generated using a ring"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b0-934e-11ee-a035-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "oscillator ( RO). To ensure random sampling, the clock and\nhandshake must be uncorrelated .ROs can be gated by adding a\nNAND or NOR gate into the ring. However this technique will\ndisadvantage us in this case as the RO will always start up in\nthe same phase, therefore the clock and sampling signals may\nbecome correlated.\nWe take advantage of the properties of a purely inverter-\nbased RO. The RO can be power gated using header/footer\ntransistor when entropy generation is not required. Such an RO"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b1-934e-11ee-a30b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 4, "chroma:document": "44\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b2-934e-11ee-96f8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "will start up in a non-deterministic phase according to thermal\nnoise and other effects in the inverters.\nTo generate biased bits for pTand ps, we need to vary the\nduty cycle of the clock. Agustin and Lopez-V allejo [14] show\nhow to construct an RO such that each tap has a unique duty\ncycle. This is achieved by using inverters with asymmetric rise/\nfall times. For example, odd inverters have fast rise and slow\nfall times, and even inverters vice versa. This can be done via"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b3-934e-11ee-aca0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "transistor sizing (ie. in the silicon library), or altering supply\nvoltage to the inverters (ie. at implementation stage). Using\nthis ROsetup, we can multiplex between taps to obtain different\nclock duty cycles, and therefore alter our PRBG probability at\nruntime.\nWe have introduced a clock into our asynchronous circuit,\nand this may seem counterintuitive, however the load on this\nclock (and therefore its energy consumption) will be low\ncompared with fully synchronous designs where the clock"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b4-934e-11ee-9085-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "drives large numbers of \ufb02ip-\ufb02ops. In Section V -B we will\ninvestigate the optimal number of PRBGs required. By taking\nthis distributed PRBG approach, we can vastly reduce the circuit\narea compared with a na\u00efve approach using one LFSR per TA.\nFigure 8 shows the circuit used to select the correct duty\ncycle clock and sample it with the asynchronous req input. The\nmutex ensures ackpand acknoutputs are mutually exclusive.\nThese outputs represent the random output bit with a dual-rail"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b5-934e-11ee-92cc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "encoding. That is for {ackp,ackn}:{0,1}represents logic-0,\nand{1,0}logic-1.{0,0}is the spacer ornull state used to\nseparate valid output values temporally.\nWhen the clk and req signals are both low, the output will\nbe in the spacer state. If a request is made (by asserting req)\nduring the low period of the clock, the output of the set-\ndominant latch is reset, and req wins the mutex, resulting in\nacknasserting. If clk rises now, the output of the latch rises,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b6-934e-11ee-aacf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "however the mutex is still held by req. A following deassertion\nofreq will result in clk gaining the mutex, however ackpwill\nbe masked by the AND gate.\nIf a request is made during the high period of clk, the output\nof the latch will already be high and clk will have won the\nmutex. On the rise of req the AND gate will unmask ackp.I f\nclk falls now, the output of the latch remains high since the R\ninput is inactive, and therefore ackpalso remains high."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b7-934e-11ee-ae60-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "The circuit is synthesized for the UMC 65LL cell library\nand the results are summarized in Table II. The asynchronous\nhandshake shows almost 10\u00d7saving in area compared to an 8-\nbit LFSR . Power and energy are also drastically decreased. The\nasynchronous handshake is well suited to energy-conscious,\npervasive applications.\nB. Optimal Number of PRBGs\nThe original TM algorithm takes a probabilistic choice for\nevery TAupdate involving s. However Abeyrathna et al. [15]"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b8-934e-11ee-81ea-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "show that randomizing every dth TA update can save energy\nwith a minimal drop in test accuracy. With d= 1000 ,a n\naccuracy within 2% ofd=1 was maintained for three of the\nfour datasets tested.MutexS\u2022\nRQclk\ndc\u2192\nfrom\nRO\nreqackpackn\nFigure 8. Handshake circuit for the asynchronous pseudorandom bit generator\nwith dual-rail output.\nTable II\nPRBG SSYNTHESIZED FOR UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .\nImplementation Area Cycle Time [ ns] Power [\u03bc W] Energy [ fJ]\nLFSR8 92.20 .38 6.41 2 .44"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2b9-934e-11ee-91dd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "LFSR8 92.20 .38 6.41 2 .44\nHandshake 9.72 0 .23 0.621 0 .143\nFollowing on, we could hypothesize an optimal number\nof PRBGs to ful\ufb01ll the needs of a given TM. If every 100 th\nTA update is randomized, we will need to produce at most\nNTA s/100 probabilistic choices per learning phase.\nThis could result in a 100\u00d7reduction in PRBGs for an\nLFSR -based design, in exchange for arbitration overhead to\nshare one PRBG between multiple TA s. Similar could be said"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ba-934e-11ee-bb46-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "for the asynchronous handshake design, although the already\nsmall size of the design means that arbitration overhead would\nalmost certainly result in a larger area than instantiating an\nasynchronous handshake PRBG for every TA.\nVI. A SYNCHRONOUS TSETLIN AUTOMA TON\nIn a TM, the TA s consume most of the hardware re-\nsources [6]. The number of TA s is given by NTA s =\n2NClasses NClauses NInputs . Therefore the goal is to create a\nlightweight TA implementation with a focus on area and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2bb-934e-11ee-836e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "energy. The TA hardware must implement the \ufb01nite state\nautomaton described by the state diagram in Figure 2.\nThis section explores two methods for the design of a TA.\nFirstly a QDI approach starting from an STG speci\ufb01cation; and\nsecondly a BDapproach starting from a traditional synchronous\nspeci\ufb01cation, using matched delays to time the design. The\ndesigns will be compared with a synchronous implementation.\nA. Quasi Delay Insensitive Approach"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2bc-934e-11ee-9702-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "A. Quasi Delay Insensitive Approach\nThe STG in Figure 9 represents a one-hot \ufb01nite state machine\n(FSM ) for a two action, six-state TA(for the de\ufb01nition of STGs ,\nsee [16]). The inputs pand rcome from the FB3 rails FB3p\nand FB3rwhich were introduced in Section IV -D. The FB3irail\ndenotes inaction and is only used for completion detection.\nState bits are encoded in the x\u2217\u2217_0and x\u2217\u2217_1places at the\ntop of the graph. States x1\u2217(x2\u2217) are the action 1 (action 2)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2bd-934e-11ee-8206-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "states\u2014in these states the automaton is indicating the action\nto exclude (include) the feature or feature complement from\nthe clause composition. The x\u22171(x\u22173) states are the closest to\n(furthest from) the action decision boundary. As an example,\nwhen x13_1holds a token, the automaton is indicating action\n1 (exclude) strongly.\nA token at p0denotes the idle state for the automaton. It\noffers a choice between penalty and reward represented by\n45"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2be-934e-11ee-b685-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 5, "chroma:document": "45\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2bf-934e-11ee-b17d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "signal transitions p+and r+respectively. Each subsequent\nbranch from p+,r+, enables a transition in one of the action\noutput signals a1,a2. Depending on the current action and the\nprevious state, a state transition may also occur. The correct\nbranch is chosen based on read arcs from the x\u2217\u2217_1places. For\nexample in the 2nd left-most branch of r+, we transition from\nstate x12tox13. This happens by \ufb01rstly giving the token from\nplace x13_0tox13_1, and secondly giving token x12_1tox12_0."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c0-934e-11ee-85d6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "After state transitions and action output have occurred, the\nacknowledge output, ack , makes a positive transition. Finally,\nthe previously given input and action signals return low,\nfollowed by ack\u2212and the token returns to p0. The automaton\nis now ready for the next penalty or reward input.\nThe internal transitions x\u2217\u2217encode some information about\nthe states in the STG . However, this is not enough to satisfy\ncomplete state coding ( CSC ) required for synthesis of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c1-934e-11ee-9861-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "STG to logic gates [16]. In the previous example where we\ntransition from x12tox13, there is a point where tokens are\nheld by both x13_1and x12_1. The STG reaches this same state\nwhen transitioning in the opposite direction, from x12tox13.\nIn order to achieve CSC , we introduce internal signals to\nencode the direction of travel of the state: x\u2217L\u2217,x\u2217R\u2217; for left\nand right. We insert these signals into every branch to maintain\nuniformity in the STG . Although not all these internal signals"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c2-934e-11ee-9d47-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "are strictly required, they ease scalability of the STG and help\nthe synthesis tool to minimize and share logic ef\ufb01ciently.\nThe STG in Figure 9 passes all veri\ufb01cation tasks required\nfor synthesis of a correct and hazard-free circuit in the\nWORKCRAFT framework: consistency, deadlock freeness, in-\nput properness, and output persistency. The circuit was syn-\nthesized and mapped to the target technology by the MPS AT\nbackend (the P ETRIFY backend did not produce a result in a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c3-934e-11ee-bc76-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "reasonable amount of time).\nIn order to read the state of the TAwithout giving penalty or\nreward (during inference for example), we add a latch to the\noutput to store one of the porrsignals. The latch is controlled\nby the ack handshake.\nThe STG state space expands exponentially as more signal\ntransitions are added. This makes it dif\ufb01cult for the synthesis\ntool to handle TA s with more than three action states. Addition-\nally there is a high possibility for human error when copying"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c4-934e-11ee-bb3f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "and editing branches of the STG . And as the number of TA\naction states increases, the graph becomes large and unwieldy\nto navigate. For these reasons we explore a BD approach to\nasynchronous TAdesign.\nB. Bundled-Data Approach\nA synchronous one-hot FSM can be constructed using \ufb02ip-\n\ufb02ops as storage elements. The next state logic can be derived\nusing the adjacent states and p,rinputs as follows:\nx13=x13\u00b7r+x12\u00b7r\nx12=x11\u00b7r+x13\u00b7p\nx11=x12\u00b7p+x21\u00b7px21=x22\u00b7p+x11\u00b7p\nx22=x21\u00b7r+x23\u00b7p\nx23=x23\u00b7r+x22\u00b7r"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c5-934e-11ee-81b3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "x22=x21\u00b7r+x23\u00b7p\nx23=x23\u00b7r+x22\u00b7r\nThis synchronous circuit is converted into a self-timed one\nthrough the process of desynchronization [12], by decom-posing the \ufb02ip-\ufb02ops into master/slave latches and replacing\nthe clock with handshake controllers and delays matched to\nthe combinational logic paths. The timing of the resultant BD\ndesign is not as robust as the QDI approach in Section VI-A\nsince the matched delays are \ufb01xed at design time and are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c6-934e-11ee-94c2-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "subject to process, variation and temperature variations.\nTo reduce the overhead of the handshake controllers, we\ngroup the TA s into an array and use one handshake controller\nfor many latches. Depending on the overall size of TM for\nthe target application, the TA s could be grouped in different\nways. A monolithic array will minimize controller overhead,\nhowever there comes a limit where buffers must be introduced\ninto the latch enable signals to maintain integrity. At this point,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c7-934e-11ee-9a84-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "it may become desirable to group the TA s by clause.\nThe speci\ufb01cation of the BD design is easily scaled due to\nthe use of parameterizable V erilog code. The work\ufb02ow takes\nadvantage of mature synchronous tools for synthesis.\nC. Synthesis Results\nTiming and power analyses of asynchronous circuits need\nspecial consideration. In conventional synchronous static tim-\ning analysis, delay through logic gates is calculated using the\ninput slew and output load capacitance. Power estimations also"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c8-934e-11ee-aa1b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "rely on these parameters. Synchronous tools cut combinational\nloops arbitrarily, therefore may be optimistic about slews,\ngate delays and internal power in asynchronous circuits which\ncontain such loops.\nTo obtain results for the QDI design we use a specialized\nasynchronous static timing analysis tool [17] which takes into\naccount the combinational loops in the timing calculation.\nThe tool operates on the gate-level netlist and propagates"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2c9-934e-11ee-8072-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "slews iteratively through circuits with loops, therefore avoiding\noptimism in the slews and gate delays. In the case there are\nmultiple paths in the circuit which begin and end at the same\ninput/output combination, the tool uses the worst-case path to\ndetermine the cycle time.\nSince the UMC 65LL library does not contain C-elements,\nthe QDI circuit is combinational with feedback loops. Table III\nshows a results summary of the TAimplementations. All results"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ca-934e-11ee-802b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "are synthesis estimations and no layout is performed. In this\ncase, the QDI circuit trades increased cycle time and energy\nfor increased robustness to process, variation and temperature\nvariations. The BD design strikes a good trade-off between\nperformance, area and elastic timing to integrate with the rest\nof the system. The layout process will introduce more power\nTable III\nRESUL TS FOR THE SIX -ST A TE TSETLIN AUTOMA TON SYNTHESIZED FOR\nTHE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY ."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2cb-934e-11ee-a5a8-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 6, "chroma:document": "THE UMC 65 nm LOW-LEAKAGE SILICON LIBRARY .\nImplementation AreaCycle Time\n[ns]Leakage\n[nW ]Power\n[\u03bcW]Energy\n[fJ]\nSynchronous 87.80 .35 3 .08 4 .58 1 .60\nBundled-data386.00 .41 2 .89 5 .19 2 .13\nQDI 123 3 .35 3 .42 1 .58 5 .29\n4Averaged over the array.\n46\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2cc-934e-11ee-8727-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "\u0001\u0002\u0003\u0004\u0005\u0004\u0006\u0007\b\t\u0006\n\u0007\u000b\f\r\u000e\b\u000f\r\f\u0007\u0005\u0004\u0010\u0004\u0006\u0007\u0005 \u0011\u0002\u0012\f\r\u000b\u0005\b\u0004\u0007\b\u0013\u0007\u000b\u0014\u0005\u0010\f\u0010\u0002\u0005\u0015\u0007\u0002\u0014\u0016\u0006\u0010\b\u0013\u0007\u0003\u0006\u000b\u0002\u000b\b\u0017\u0010\f\u0010\u0002\u0005\n\u0018\u0003\u0010\u0004\u0006\u0007\b\u0019\n\u001a\u0002\u001b\u0003\u001c\n\u000b\u0002\u001d\n\u0017\u0010\f\u0010\u0002\u0005\u0018\u0003\u0010\u0004\u0006\u0007\b\u001e\n\u001a\u0004\u0007\u0003\u001c\n\u000b\u0002\u001d\n\u0017\u0010\f\u0010\u0002\u0005\n\u0001\u0002\u0003\u0004\u0005\u0004\u0006\u0007\b\t\u0006\n\u0007\u000b\f\r\u000e\nFigure 9. Signal transition graph for a Tsetlin automaton with two actions and six states. Inputs: p,r. Outputs: a1,a2,ack . Other signals are internal.\nand latency into the synchronous and BD designs due to their\nclock trees, which the QDI design will not suffer from.\nVII. C ONCLUSION\nOur Petri net visualization of the TM architecture enables"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2cd-934e-11ee-9e0b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "\ufb02exible hardware implementations by encapsulating both con-\ncurrency and serialization. By decoupling the TA storage\nelements from the clauses, e.g. in a standalone array, the TM\ncould be serialized further by reusing clause and sum hardware\nacross multiple cycles, trading off throughput for reduced logic\narea (and leakage power).\nEarly propagation enabled by the RCD scheme used in the DR\ncircuits can be leveraged to trigger \ufb01ne-grained power gating"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ce-934e-11ee-a1bb-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "in the system to further reduce energy consumption.\nDistributed generation of probabilistic choices maximizes\nconcurrency and decreases energy overheads compared to\na centralized approach. A simple PRBG implementation is\nsuf\ufb01cient and most suitable for our applications. Features\nof a clock source undesirable in synchronous circuits (jitter,\netc.) can be leveraged for stochasticity. Further study of PRBG\nproperties and their effects on TM learning are a subject for\nfuture work.\nACKNOWLEDGMENT"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2cf-934e-11ee-94f0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "future work.\nACKNOWLEDGMENT\nThe authors thank Christos Sotiriou and colleagues for help\nin asynchronous ST A and for access and support for their tool.\nWe are also grateful to colleague Thomas Bunnam for fruitful\ndiscussions on ROs and RNG techniques. The authors would\nlike to acknowledge the funding support from EPSRC IAA\ngrant: Whisperable.\nREFERENCES\n[1] A. Wheeldon et al. , \u201cLow-Latency Asynchronous Logic Design for\nInference at the Edge,\u201d in Proc. Conf. Des. Autom. Test Eur. , Feb. 2021,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d0-934e-11ee-9211-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "pp. 370\u2013373.[2] O.-C. Granmo, \u201cThe Tsetlin Machine \u2013 A Game Theoretic Bandit Driven\nApproach to Optimal Pattern Recognition with Propositional Logic,\u201d\narXiv:1804.01508v15 [cs.AI] , Apr. 2018.\n[3] G. T. Berge et al. , \u201cUsing the Tsetlin Machine to Learn Human-\nInterpretable Rules for High-Accuracy Text Categorization with Medical\nApplications,\u201d arXiv:1809.04547 [cs, stat] , Sep. 2018.\n[4] C. D. Blakely and O.-C. Granmo, \u201cClosed-Form Expressions for Global"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d1-934e-11ee-86fa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "and Local Interpretation of Tsetlin Machines with Applications to\nExplaining High-Dimensional Data,\u201d arXiv:2007.13885 [cs, stat] , Jul.\n2020.\n[5] B. Bhattarai, O.-C. Granmo, and L. Jiao, \u201cMeasuring the Novelty of\nNatural Language Text Using the Conjunctive Clauses of a Tsetlin\nMachine Text Classi\ufb01er,\u201d arXiv:2011.08755 [cs] , Nov. 2020.\n[6] A. Wheeldon et al. , \u201cLearning automata based energy-ef\ufb01cient AI\nhardware design for IoT applications,\u201d Philos Trans R Soc Math Phys"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d2-934e-11ee-bbfa-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "Eng Sci , vol. 378, no. 2182, Oct. 2020.\n[7] \u2014\u2014, \u201cSelf-timed, minimum latency circuits for the internet of things,\u201d\nIntegration , vol. 69, pp. 138\u2013146, Nov. 2019.\n[8] D. Sokolov et al. , \u201cAutomating the Design of Asynchronous Logic\nControl for AMS Electronics,\u201d IEEE Trans. Comput.-Aided Design\nIntegr . Circuits Syst. , vol. 39, no. 5, pp. 952\u2013965, 2020.\n[9] X. Zhang et al. , \u201cA Novel Power Delivery Method for Asynchronous\nLoads in Energy Harvesting Systems,\u201d J. Emerg. Technol. Comput. Syst. ,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d3-934e-11ee-8e41-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "vol. 7, no. 4, pp. 1\u201322, Dec. 2011.\n[10] D. Balsamo et al. , \u201cWearable and autonomous computing for future\nsmart cities: Open challenges,\u201d in 2017 25th Int. Conf. on Software\nTelecommun. Comput. Netw. (SoftCOM) , 2017, pp. 1\u20135.\n[11] \u201cThe workcraft framework.\u201d [Online]. Available: https://workcraft.org/\n[12] J. Cortadella et al. , \u201cDesynchronization: Synthesis of asynchronous\ncircuits from synchronous speci\ufb01cations,\u201d IEEE Trans Comput Integr"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d4-934e-11ee-8d26-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "Circuits Syst , vol. 25, no. 10, pp. 1904\u20131921, Oct. 2006.\n[13] K. S. Narendra and M. A. L. Thathachar, Learning Automata: An\nIntroduction . Englewood Cliffs, N.J: Prentice Hall, 1989.\n[14] J. Agustin and M. Lopez-V allejo, \u201cAn in-depth analysis of ring oscilla-\ntors: Exploiting their con\ufb01gurable duty-cycle,\u201d IEEE Trans Circuits Syst\nRegul Pap , vol. 62, no. 10, pp. 2485\u20132494, Oct. 2015.\n[15] K. D. Abeyrathna et al. , \u201cA Novel Multi-Step Finite-State Automaton"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d5-934e-11ee-b1e0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "for Arbitrarily Deterministic Tsetlin Machine Learning,\u201d in Proc. 40th\nSGAI Int. Conf. Artif. Intell. , vol. 7, Jul. 2020.\n[16] J. Cortadella et al. ,Logic synthesis of asynchronous controllers and\ninterfaces . Springer-V erlag Berlin Heidelberg, 2002.\n[17] S. Simoglou et al. , \u201cGraph-based ST A for asynchronous controllers,\u201d\nIntegration , vol. 75, pp. 91\u2013101, Nov. 2020.\n47"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d6-934e-11ee-aa31-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\Self-timed_Reinforcement_Learning_using_Tsetlin_Machine.pdf", "page": 7, "chroma:document": "Integration , vol. 75, pp. 91\u2013101, Nov. 2020.\n47\nAuthorized licensed use limited to: UNIVERSITY OF AGDER. Downloaded on December 05,2023 at 09:06:51 UTC from IEEE Xplore.  Restrictions apply."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d7-934e-11ee-8618-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "Measuring the Novelty of Natural Language Text Using the\nConjunctive Clauses of a Tsetlin Machine Text Classi\ufb01er\nBimal Bhattarai\nUniversity of Agder\nGrimstad, Norway\nbimal.bhattarai@uia.noOle-Christoffer Granmo\nUniversity of Agder\nGrimstad, Norway\nole.granmo@uia.noLei Jiao\nUniversity of Agder\nGrimstad, Norway\nlei.jiao@uia.no\nAbstract\nMost supervised text classi\ufb01cation approaches assume a closed world, counting on all classes"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d8-934e-11ee-a894-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "being present in the data at training time. This assumption can lead to unpredictable behaviour\nduring operation, whenever novel, previously unseen, classes appear. Although deep learning-\nbased methods have recently been used for novelty detection, they are challenging to interpret\ndue to their black-box nature. This paper addresses interpretable open-world text classi\ufb01cation,\nwhere the trained classi\ufb01er must deal with novel classes during operation. To this end, we extend"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2d9-934e-11ee-a3fe-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "the recently introduced Tsetlin machine (TM) with a novelty scoring mechanism. The mecha-\nnism uses the conjunctive clauses of the TM to measure to what degree a text matches the classes\ncovered by the training data. We demonstrate that the clauses provide a succinct interpretable\ndescription of known topics, and that our scoring mechanism makes it possible to discern novel\ntopics from the known ones. Empirically, our TM-based approach outperforms seven other nov-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2da-934e-11ee-9625-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "elty detection schemes on three out of \ufb01ve datasets, and performs second and third best on the\nremaining, with the added bene\ufb01t of an interpretable propositional logic-based representation.\n1 Introduction\nIn recent years, deep learning-based techniques have achieved superior performance on many text clas-\nsi\ufb01cation tasks. Most of the classi\ufb01ers use supervised learning, assuming a closed-world environment"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2db-934e-11ee-825d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "(Scheirer et al., 2013). That is, the classes present in the test data (or during operation) are also assumed\nto be present in the training data. However, when facing an open-world environment, new classes may\nappear after training (Bendale and Boult, 2016). In such cases, assuming a closed world can lead to\nunpredictable behaviour. For example, a chatbot interacting with a human user will regularly face new"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2dc-934e-11ee-9d45-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "user intents that it has not been trained to recognize. A chatbot for banking services may, for instance,\nhave been trained to recognize the intent of applying for a loan. However, it will provide meaningless\nresponses if it fails to recognize that asking for a lower interest rate is a new and different user intent. The\nproblem with neural network-based supervised classi\ufb01ers that use the typical softmax layer is that they"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2dd-934e-11ee-a9c0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "erroneously force novel input into one of the previously seen classes, by normalizing the class output\nscores to produce a distribution that sums to 1.0. Instead, a robust classi\ufb01er should be able to \ufb02ag input\nas novel, rejecting to label it according to the presently known classes. Recently, many important appli-\ncation areas make use of novelty detection such as medical applications, fraud detection (Veeramreddy"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2de-934e-11ee-bd3f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "et al., 2011), sensor networks (Zhang et al., 2010) and text analysis (Basu et al., 2004). For a further\nstudy of these classes of techniques, the reader is referred to (Pimentel et al., 2014).\nFigure 1 illustrates the problem of novelty detection, i.e., recognizing when the data fed to a classi\ufb01er\nis novel and somehow differs from the data that was available during training. In brief, after training on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2df-934e-11ee-b676-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "data with known classes (blue data points to the right in Figure 1), the classi\ufb01er should be able to detect\nnovel data arising from new, previously unseen, classes (yellow data points to the right in the \ufb01gure). We\nwill refer to the known data points as positive examples and the novel data points as negative examples .\nNote that the problem of novelty detection is closely related to so-called outlier detection (Chandola et"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e0-934e-11ee-b419-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 0, "chroma:document": "al., 2009)(Pincus, 1995). However, as exempli\ufb01ed to the left in Figure 1, the latter problem involves\n\ufb02agging data points that are part of an already known class, yet deviating from the other data points, e.g.,\ndue to measurement errors or anomalies (red point).arXiv:2011.08755v1  [cs.CL]  17 Nov 2020"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e1-934e-11ee-b423-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "Outlier decision boundaryOutlier detection Novelty detection\ntrained example\ndistant\nknown classnovel exampleFigure 1: Visualization of outlier detection and novelty detection.\nProblem De\ufb01nition: Generally, in multiclass classi\ufb01cation, we have a set of example data points\nX= (xi,yi),xi\u2208Rs,wherei= 1,2,3,...,N indexes the positive examples, sis the dimensionality\nof the data point xi, andyiis the label of xi, assigning it a class. For any data point xi, also referred"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e2-934e-11ee-a4ce-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "to as a feature vector, a classi\ufb01er function \u02c6y=f(x;X)is to assign the data point a predicted class \u02c6y,\nafter the function has been \ufb01tted to the training data X. Additionally, a novelty scoring function z(x;X),\nalso \ufb01tted on X, calculates a novelty score, so that a threshold \u03c3can be used to recognize novel input.\nIn other words, the classi\ufb01er is to return the correct class label while simultaneously being capable of\nrejecting novel examples."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e3-934e-11ee-b1b5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "rejecting novel examples.\nUnder the above circumstances, standard supervised learning would fail, particularly for methods\nbased on building a discriminant function, such as neural networks. As shown to the left of Figure 1, a\ndiscriminant function captures the discriminating \u201cboundaries\u201d between the known classes and cannot\nreadily be used to discern novel classes. Still, a traditional way of implementing novelty detection is to"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e4-934e-11ee-9b51-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "threshold the entropy of the class probability distribution (Hendrycks and Gimpel, 2016). Such methods\nare not actually measuring novelty, but closeness to the decision boundary. Such an approach thus leads\nto undetected novel data points when the data points are located far from the decision boundary. Another,\nand perhaps more robust, approach is to take advantage of the class likelihood function, which estimates\nthe probability of the data given the class."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e5-934e-11ee-b1cf-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "the probability of the data given the class.\nPaper Contributions: Unlike traditional methods, in this paper we leverage the conjunctive clauses\nin propositional logic that a TM builds. TM clauses represent frequent patterns in the data, and our\nhypothesis is that these frequent patterns characterize the known classes succinctly and comprehensively.\nWe thus establish a novelty scoring mechanism that is based simply on counting how many clauses match"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e6-934e-11ee-9f1c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "the input. This score can, in turn, be thresholded manually to \ufb02ag novel input. However, for more robust\nnovelty detection, we train several standard machine learning techniques to \ufb01nd accurate thresholds. The\nmain contributions of our work can thus be summarized as follows:\n\u2022 We devise the \ufb01rst TM-based approach for novelty detection, leveraging the clauses of the TM\narchitecture.\n\u2022 We illustrate how the new technique can be used to detect novel topics in text, and compare the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e7-934e-11ee-bea0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "technique against widely used approaches on \ufb01ve different datasets.\n2 Related Work\nThe perhaps most common approach to novelty detection is distance-based methods (Hautamaki et al.,\n2004), which assumes that the known or seen data are clustered together while novel data have a high\ndistance to the clusters. The major drawback of these methods is computational complexity when per-\nforming clustering or nearest neighbor search in a large dataset. Early work on novelty detection also"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e8-934e-11ee-966b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 1, "chroma:document": "includes one-class SVMs (Sch \u00a8olkopf et al., 2001), which are only capable of using the positive training\nexamples to maximize the class margin. This shortcoming is overcome by the Center-Based Similarity"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2e9-934e-11ee-a712-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "(CBS) space learning method (Fei and Liu, 2015), which uses binary classi\ufb01ers over vector similarities\nof training examples transformed into the center of the class. To build a classi\ufb01er for detecting novel\nclass distributions, Chow et al. proposed a con\ufb01dence score-based method that suggests an optimum\ninput rejection rule (Chow, 1970). The method is relatively accurate, but it does not scale well to high-\ndimensional datasets."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ea-934e-11ee-a655-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "dimensional datasets.\nThe novelty detection method OpenMax (Bendale and Boult, 2016) is more recent, and estimates the\nprobability of the input belonging to a novel class. To achieve this, the method employs an extra layer,\nconnected to the penultimate layer of the original network. The computational complexity of the method\nis however high, and the underlying inference cannot easily be interpreted for quality assurance. Lately,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2eb-934e-11ee-94c6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "Yu et al. (Yu et al., 2017) adopted the Adversarial Sample Generation (ASG) framework (Hautamaki\net al., 2004) to generate positive and negative examples in an unsupervised manner. Then, based on\nthose examples, they trained an SVM classi\ufb01er for novelty detection. Furthermore, in computer vision,\nScheirer et al. introduced the concept of open space risk to recognize novel image content (Scheirer\net al., 2013). They proposed a \u201c1-vs-set machine\u201d that creates a decision space using a binary SVM"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ec-934e-11ee-af5f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "classi\ufb01er, with two parallel hyperplanes bounding the non-novel regions. In Section 4, we compare\nthe performance of our new TM-based approach with the most widely used approaches among those\nmentioned above.\n3 Tsetlin Machine-based Novelty Detection\nIn this section, we propose our approach to novelty detection based on the TM. First, we explain the\narchitecture of the TM. Then, we show how novelty scores can be obtained from the TM clauses. Finally,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ed-934e-11ee-a317-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "we integrate the TM with a rule-based classi\ufb01er for novel text classi\ufb01cation.\n3.1 Tsetlin Machine (TM) Architecture\nThe TM is a recent approach to pattern classi\ufb01cation (Granmo, 2018) and regression (Dar-\nshana Abeyrathna et al., 2020). It builds on a classic learning mechanism called a Tsetlin automaton\n(TA), developed by M. L. Tsetlin in the early 1960s (Tsetlin, 1961). In all brevity, multiple teams of TA"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ee-934e-11ee-b681-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "combine to form the TM. Each team is responsible for capturing a frequent pattern of high precision,\nby composing a conjunctive clause. In-built resource allocation principles guide the teams to distribute\nthemselves across the underlying sup-patterns of the problem. Recently, the TM has performed compet-\nitively with the state-of-the-art techniques, including deep neural networks for text classi\ufb01cation (Berge"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ef-934e-11ee-940d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "et al., 2018). Further, the convergence of TM learning has been studied in (Zhang et al., 2020). In\nthe following, we propose a new scheme that extends the TM with the capability of recognizing novel\npatterns.\nAs explored in the following, Figure 2 capture the building blocks of a TM. As seen, a vanilla TM\ntakes a vector X= (x1,...,x o)of binary features as input (Figure 3). We binarize text by using binary"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f0-934e-11ee-a80a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "features that capture the presence/absence of terms in a vocabulary, akin to a bag of words, as done in\n(Berge et al., 2018). However, as opposed to a vanilla TM, our scheme does not output the predicted\nclass. Instead, it calculates a novelty score per class.\nTogether with their negated counterparts, \u00afxk=\u00acxk= 1\u2212xk, the features form a literal set L=\n{x1,...,x o,\u00afx1,..., \u00afxo}. A TM pattern is formulated as a conjunctive clause Cj, formed by ANDing a\nsubsetLj\u2286Lof the literal set:\nCj(X) =\u22c0"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f1-934e-11ee-8910-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "subsetLj\u2286Lof the literal set:\nCj(X) =\u22c0\nlk\u2208Ljlk=\u220f\nlk\u2208Ljlk. (1)\nFor example, the clause Cj(X) =x1x2consists of the literals Lj={x1,x2}and outputs 1ifx1=\nx2= 1. The number of clauses employed is a user set parameter m. Half of the clauses are assigned a\npositive polarity and the other half are assigned a negative polarity. The output of a conjunctive clause\nis determined by evaluating it on the input literals. When a clause outputs 1, this means that it has"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f2-934e-11ee-b783-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 2, "chroma:document": "recognized a pattern in the input. Conversely, the clause outputs 0when no pattern is recognized. The\nclause outputs, in turn, are combined into a classi\ufb01cation decision through summation and thresholding"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f3-934e-11ee-b896-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 3, "chroma:document": "using the unit step function u:\n\u02c6y=u\uf8eb\n\uf8edm/2\u2211\nj=1C+\nj(X)\u2212m/2\u2211\nj=1C\u2212\nj(X)\uf8f6\n\uf8f8. (2)\nThat is, the classi\ufb01cation is performed based on a majority vote, with the positive clauses voting for\ny= 1and the negative for y= 0. The classi\ufb01er \u02c6y=u(x1\u00afx2+ \u00afx1x2\u2212x1x2\u2212\u00afx1\u00afx2), e.g., captures the\nXOR-relation.\n \n \n \nNovelty Score \n \n  \n \n \nyes NoType I/II feedback\nTraining clauses\nFigure 2: Optimization of clauses in TM for generating novelty score."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f4-934e-11ee-b24a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 3, "chroma:document": "A clause is composed by a team of TA, each TA deciding to Exclude orInclude a speci\ufb01c literal in the\nclause. The TA learns which literals to include based on reinforcement: Type I feedback is designed to\nproduce frequent patterns, while Type II feedback increases the discriminating power of the patterns (see\n(Granmo, 2018) for details).\n3.2 Novelty Detection Architecture\nFor novelty detection, however, we here propose to treat all clause output as positive, disregarding clause"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f5-934e-11ee-9e3a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 3, "chroma:document": "polarity. This is because both positive and negative clauses capture patterns in the training data, and thus\ncan be used to detect novel input. We use this sum of absolute clause outputs as a novelty score, which\ndenotes the resemblance of the input to the patterns formed by clauses during training. The resulting\nmodi\ufb01ed TM architecture is captured by Figure 3, showing how four different outputs are produced by\nthe TM, two per class. These outputs form the basis for novelty detection."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f6-934e-11ee-b30b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 3, "chroma:document": "The overall novelty detection architecture is shown in Figure 4. Each TM (one per class) produces\ntwo novelty score for its respective class, one based on the positive polarity clauses and one based on the\nnegative polarity ones. The novelty scores are normalized and then given to a classi\ufb01er, such as decision"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f7-934e-11ee-901a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 4, "chroma:document": "Text Input\nBinary encoded input\nClass 1 Class 2\nPositive\nNovelty Score\nPositive\nNovelty ScoreInput\nClauses\nOutput\nNegative\nNovelty Score\nNegative\nNovelty ScoreFigure 3: Multiclass Tsetlin Machine (MTM) framework to produce the novelty score for each class.\nSeen Class\n Novel Class\nNovelty Score\n Novelty Score\nClassifier\nOutput (0 or 1)\nFigure 4: Novelty detection architecture.\ntree (DT), K nearest neighbor (KNN), Support vector machine (SVM) and logistic regression (LR). The"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f8-934e-11ee-86fc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 4, "chroma:document": "output from these classi\ufb01ers are \u201dNovel\u201d or \u201dNot Novel\u201d, i.e. 0or1in the \ufb01gure.\nFor illustration purposes, instead of using a machine learning algorithm to decide upon novelty, one\ncan instead use a simple rule-based classi\ufb01er, introducing a classi\ufb01cation threshold T. Then the novelty\nscore for input sentence can be compared with Tto detect whether a sentence is novel or not. That is,\nTdecides how many clauses must match to qualify the input as non-novel. The classi\ufb01cation function"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2f9-934e-11ee-b953-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 4, "chroma:document": "F(X)for a single class can accordingly be given as:\nF(X) =\uf8f1\n\uf8f4\uf8f2\n\uf8f4\uf8f31,if\u2211m/2\nj=1C+\nj(X)>T,\n1,if\u2211m/2\nj=1C\u2212\nj(X)<\u2212T,\n0,otherwise.\n4 Experiments and Results\nIn this section, we present the experimental evaluation of our proposed TM model and compare it with\nother baseline models across \ufb01ve datasets."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2fa-934e-11ee-b17c-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "4.1 Datasets\nWe used the following datasets for evaluation.\n\u2022 20-newsgroup dataset: The dataset contains 20 classes with a total of 18,828 documents. In our ex-\nperiment, we consider the two classes \u201ccomp.graphics\u201d and\u201ctalk.politics.guns\u201d as known classes\nand the class \u201crec.sport.baseball\u201d as novel. We take 1000 samples from both known and novel\nclasses for novelty classi\ufb01cation.\n\u2022 CMU Movie Summary corpus: The dataset contains 42,306 movie plot summaries extracted from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2fb-934e-11ee-9a52-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "Wikipedia and metadata extracted from Freebase. In our experiment, we consider the two movie\ncategories \u201caction\u201d and\u201chorror\u201d as known classes and \u201cfantasy\u201d as novel.\n\u2022 Spooky Author Identi\ufb01cation dataset: The dataset contains 3,000 public domain books from the\nfollowing horror \ufb01ction authors: Edgar Allan Poe (EAP), HP Lovecraft (HPL) , and Mary Shelley\n(MS) . We train on written texts from EAP andHPL while treating texts from MSas novel."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2fc-934e-11ee-9067-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "\u2022 Web of Science dataset (Kowsari et al., 2017): This dataset contains 5,736 published papers, with\neleven categories organized under three main categories. We use two of the main categories as\nknown classes, and the third as a novel class.\n\u2022 BBC sports dataset: This dataset contains 737 documents from the BBC Sport website, organized\nin \ufb01ve sports article categorie and collected from 2004 to 2005. In our work, we use \u201ccricket\u201d and\n\u201cfootball\u201d as the known classes and \u201crugby\u201d as novel."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2fd-934e-11ee-8bbd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "4.2 A Case Study\nTo cast light on the interpretability of our scheme, we here use substrings from the 20 Newsgroup dataset,\ndemonstrating novelty detection on a few simple cases. First, we form an indexed vocabulary set V\nincluding all literals from the dataset. The input text is binarized based upon the index of the literals in\nV. For example, if a word in the text substring has been assigned index 5inV, the5thposition of the"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2fe-934e-11ee-a9c9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "input vector is set to 1. If a word is absent from the substring, its corresponding feature is set to 0. Let\nus consider substrings from the two known classes and the novel class from the 20 Newsgroup dataset.\n\u2022Class : comp.graphics (known)\nText: Presentations are solicited on all aspects of Navy-related scienti\ufb01c visualization and virtual\nreality.\nLiterals :\u201cPresentations\u201d ,\u201csolicited\u201d ,\u201caspects\u201d ,\u201cNavy\u201d ,\u201crelated\u201d ,\u201cscienti\ufb01c\u201d ,\u201cvisualiza-\ntion\u201d ,\u201cvirtual\u201d ,\u201creality\u201d ."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd2ff-934e-11ee-a20a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "tion\u201d ,\u201cvirtual\u201d ,\u201creality\u201d .\n\u2022Class : talk.politics.guns (known)\nText: Last year the US suffered almost 10,000 wrongful or accidental deaths by handguns alone. In\nthe same year, the UK suffered 35 such deaths.\nLiterals :\u201cLast \u201d ,\u201cyear\u201d ,\u201cUS\u201d ,\u201csuffered\u201d ,\u201cwrongful\u201d ,\u201caccidental\u201d ,\u201cdeaths\u201d ,\u201chandguns\u201d ,\n\u201cUK\u201d ,\u201csuffered\u201d .\n\u2022Class : rec.sport.baseball (Novel)\nText: The top 4 are the only true contenders in my mind. One of these 4 will de\ufb01nitely win the\ndivision unless it snows in Maryland."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd300-934e-11ee-9006-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 5, "chroma:document": "division unless it snows in Maryland.\nLiterals :\u201ctop \u201d ,\u201conly\u201d ,\u201ctrue\u201d ,\u201ccontenders\u201d ,\u201cmind\u201d ,\u201cwin\u201d ,\u201cdivision\u201d ,\u201cunless\u201d ,\u201csnows\u201d ,\n\u201cMaryland\u201d\nAfter training, the two known classes form conjunctive clauses that capture literal patterns re\ufb02ecting the\ntextual content. For the above example, we get the following clauses:\n\u2022C+\n1=\u201cPresentations\u201d\u2227\u201caspects\u201d\u2227\u201cNavy\u201d\u2227\u201cscienti\ufb01c\u201d\u2227\u201cvirtual\u201d,\u201creality\u201d\u2227\u201cyear\u201d\u2227\u201cUS\u201d\n\u2227\u201cmind\u201d\u2227\u201cdivision\u201d"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd301-934e-11ee-b794-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "\u2022C\u2212\n1=\u00ac(\u201csuffered\u201d\u2227\u201caccidental\u201d\u2227\u201cunless\u201d\u2227\u201csnows\u201d )\n\u2022C+\n2=\u201clast\u201d\u2227\u201cyear\u201d\u2227\u201cUS\u201d\u2227\u201cwrongful\u201d\u2227\u201cdeaths\u201d,\u201caccidental\u201d\u2227\u201chandguns\u201d\u2227\u201cNavy\u201d\n\u2227\u201csnows\u201d\u2227\u201cMaryland\u201d\u2227\u201cdivisions\u201d\n\u2022C\u2212\n2=\u00ac(\u201cpresentations\u201d\u2227\u201csolicited\u201d\u2227\u201cvirtual\u201d\u2227\u201ctop\u201d\u2227\u201cwin\u201d )\nHere, the clauses from each class captures the frequent patterns from the class. However, it may also\ncontain certain literals from other classes. The positive polarity clauses provide evidence on the presence"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd302-934e-11ee-bbfc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "of a class, while negative polarity clauses provide evidence on the absence of the class. The novelty\nscore for each class is calculated based on the propositional formula formed by the clauses. In general,\nfor input from known classes, the novelty scores are higher. This is because the clauses have been trained\nto vote for or against input from the known class. For example, when we pass a known class to our model,\nthe clauses might produce scores as in Table 1 (for illustration purposes)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd303-934e-11ee-916f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "ClassC+\n1C\u2212\n1C+\n2C\u2212\n2\nKnown +6 -3 +3 -5\nNovel +2 -1 +1 -2\nTable 1: Novelty score example when known text sentence is passed to the model.\nThe scores are then used as features to prepare a dataset for employing machine learning classi\ufb01ers to\nenhance novelty detection. As exampli\ufb01ed in the table, novelty scores for known classes are relatively\nhigher than those of novel classes. This allows the \ufb01nal classi\ufb01er to robustly recognize novel input, as\nexplored empirically below."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd304-934e-11ee-80e6-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "explored empirically below.\n4.3 Empirical Evaluation\nWe divided the task into two experiments i.e., 1) Novelty score calculation 2) Novelty/Known Class\nclassi\ufb01cation. In the \ufb01rst experiment, we employed the known classes to train the TM. The TM runs\nfor 100 epochs with hyperparameter setting of 5000 clauses, a threshold Tof 25, and a sensitivity s\nof 15.0. Then, we use the clauses formed by the trained TM model to calculate the novelty scores for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd305-934e-11ee-b569-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "both the known and novel classes. We adopt an equal number of examples to gather the novelty score\nfrom both known and novel classes. In the second experiment, the novelty score generated from the \ufb01rst\nexperiment is forwarded as input to standard machine learning classi\ufb01ers, such as DT, KNN, SVM, LR,\nNB, and MLP to classify whether a text is novel.\nDataset DT KNN SVM LR NB MLP\n20 Newsgroup 72.5 % 82 % 78.0 % 72.75 % 69.0 % 82.50 %"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd306-934e-11ee-811b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "Spooky action author 53.42 % 57.89 % 63.15 % 52.63 % 58.68 % 63.15 %\nCMU movie 61.05 % 64.73 % 62.10 % 55.00 % 58.94 % 68.68 %\nBBC sports 84.21 % 85.96 % 75.43% 70.17 % 73.68 % 89.47 %\nWeb of Science 64.70 % 67.97 % 69.93 % 67.10 % 62.09 % 70.37 %\nTable 2: Accuracy of different machine learning classi\ufb01ers to detect novel class in various dataset.\nThe experimental results for all datasets are shown in Table 2. As seen, multilayer perceptron (MLP)"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd307-934e-11ee-a06f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "(hidden layer sizes 100,30 and relu activation with stochastic gradient descent) is superior for all of the\ndatasets. In our experiments, the 20 Newsgroups and BBC sports datasets yielded better results than the\nthree other data sets, arguably because of the sharp distinctiveness of examples in the known and novel\nclasses. We further believe that the novelty scores are clustered based on known and novel classes, thus,"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd308-934e-11ee-b05f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 6, "chroma:document": "distance-based methods seem effective in classi\ufb01cation. We plotted the novelty score of thousand text\nsamples from known and novel classes using our framework, which shows how scores differ signi\ufb01cantly\nfor each sample as visualized in Figure 3. Moreover, we can see from the graph that when the texts from"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd309-934e-11ee-977b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 7, "chroma:document": "/s50/s48/s48 /s52/s48/s48 /s54/s48/s48 /s56/s48/s48 /s49/s48/s48/s48/s48/s49/s48/s48/s50/s48/s48/s51/s48/s48/s52/s48/s48/s53/s48/s48/s83/s99/s111/s114/s101\n/s83/s97/s109/s112/s108/s101/s32/s83/s101/s101/s110/s32/s99/s108/s97/s115/s115\n/s32/s78/s111/s118/s101/s108/s32/s99/s108/s97/s115/s115Figure 5: Visualization of differences in novelty score for known and novel classes\nthe novel and known classes are discriminative, the TM produces distinct novelty scores, thus improving"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30a-934e-11ee-a6a1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 7, "chroma:document": "the ability of machine learning classi\ufb01ers to detect novel texts. In all brevity, we believe that the clauses\nof a TM capture frequent patterns in the training data, and thus novel samples will reveal themselves by\nnot \ufb01tting with a suf\ufb01ciently large number of clauses. The number of triggered clauses can therefore be\nutilized to measure novelty score. Also, the clauses formed by trained TM models should only to a small\ndegree trigger on novel classes, producing distinctively low scores."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30b-934e-11ee-bc4e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 7, "chroma:document": "Algorithms 20 Newsgroup Spooky action author CMU movie BBC sports WOS\nLOF 52.51 % 50.66 % 48.84 % 47.97 % 55.61 %\nFeature Bagging 67.60 % 62.70 % 64.73 % 54.38 % 69.64 %\nHBOS 55.03 % 48.55 % 48.57 % 49.53 % 55.09 %\nIsolation Forest 52.01 % 48.66% 49.10 % 49.35% 54.70 %\nAverage KNN 76.35 % 57.76 % 56.21 % 55.54 % 79.22 %\nK-Means clustering 81.00 % 61.30 % 49.20 % 47.70 % 41.31 %\nOne-class SVM 83.70 % 43.56 % 51.94 % 83.53 % 36.32 %\nTM framework 82.50 % 63.15% 68.15 % 89.47 % 70.37 %"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30c-934e-11ee-994b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 7, "chroma:document": "Table 3: Performance comparison of proposed TM framework with cluster and outlier based novelty\ndetection algorithms.\nWe compared the performance of our TM framework with different clustering and outlier detection\nalgorithms such as Cluster-based Local Outlier Factor (CBLOF), Feature Bagging ( neighbors,\nn= 35 ), Histogram-base Outlier Detection (HBOS), Isolation Forest, Average KNN, K-Means clus-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30d-934e-11ee-8738-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 7, "chroma:document": "tering, and One-class SVM. The evaluation was performed on the same preprocessed datasets for a fair\ncomparison. To make comparision more robust, we preprocessed the data for the baseline algorithms\nusing count vectorizer, term frequency-inverse document frequency (TFIDF) and Principle component\nanalysis (PCA). Additionally, we utilized maximum possible outlier fraction (i.e., 0.5) for these methods."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30e-934e-11ee-8cd1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "The performance comparison is given in Table 3, which shows that our framework surpasses the other al-\ngorithms on three of the datasets and performs competitively in the remaining two. However, in datasets\nlikeWOS , where there are many similar words shared between known and novel classes, our method is\nsurprisingly surpassed by the distance-based algorithm. One-class SVM closely follows the performance"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd30f-934e-11ee-bfdc-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "of our TM framework, which may be due to its linear structure that prevents over\ufb01tting on imbalanced\nand small datasets. In our paper, we are not just trying to build a state-of-art novelty detection algorithm.\n5 Conclusions\nIn this paper, we studied a problem of novelty detection in multiclass text classi\ufb01cation. We proposed a\nscore-based TM framework for novel class detection. We \ufb01rst used the clauses of the TM to produce a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd310-934e-11ee-86f7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "novelty score, distinguishing between known and novel classes. Then, a machine learning classi\ufb01er is\nadopted for novelty classi\ufb01cation using the novelty scores provided by the TM. The experimental results\non various datasets demonstrate the effectiveness of our proposed framework. Our future work includes\nusing a large text corpus with multiple classes for experimentation and studying the properties of the\nnovelty score theoretically.\nReferences"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd311-934e-11ee-a281-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "novelty score theoretically.\nReferences\nSugato Basu, Mikhail Bilenko, and Raymond J. Mooney. 2004. A probabilistic framework for semi-supervised\nclustering. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and\nData Mining , KDD \u201904, page 59\u201368, New York, NY , USA. Association for Computing Machinery.\nAbhijit Bendale and Terrance E. Boult. 2016. Towards open set deep networks. In The IEEE Conference on\nComputer Vision and Pattern Recognition (CVPR) , June."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd312-934e-11ee-8989-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "Geir Thore Berge, Ole-Christoffer Granmo, Tor Oddbj\u00f8rn Tveit, Morten Goodwin, Lei Jiao, and Bernt Viggo\nMatheussen. 2018. Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text\nCategorization with Medical Applications. CoRR , abs/1809.04547.\nVarun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM Comput. Surv. ,\n41(3), July."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd313-934e-11ee-b82d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "41(3), July.\nC. K. Chow. 1970. On optimum recognition error and reject tradeoff. IEEE Trans. Information Theory , 16:41\u201346.\nK. Darshana Abeyrathna, Ole-Christoffer Granmo, Xuan Zhang, Lei Jiao, and Morten Goodwin. 2020. The\nregression Tsetlin machine: a novel approach to interpretable nonlinear regression. Philosophical Transactions\nof the Royal Society A: Mathematical, Physical and Engineering Sciences , 378(2164):20190165."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd314-934e-11ee-bbd7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "Geli Fei and Bing Liu. 2015. Social media text classi\ufb01cation under negative covariate shift. In Proceedings of the\n2015 Conference on Empirical Methods in Natural Language Processing , pages 2347\u20132356, Lisbon, Portugal,\nSeptember. Association for Computational Linguistics.\nOle-Christoffer Granmo. 2018. The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal\nPattern Recognition with Propositional Logic. CoRR , abs/1804.01508."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd315-934e-11ee-a88b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "V . Hautamaki, I. Karkkainen, and P. Franti. 2004. Outlier detection using k-nearest neighbour graph. In Proceed-\nings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. , volume 3, pages 430\u2013433\nV ol.3.\nDan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassi\ufb01ed and out-of-distribution examples\nin neural networks. CoRR , abs/1610.02136.\nKamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd316-934e-11ee-b036-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 8, "chroma:document": "Laura E. Barnes. 2017. Hdltex: Hierarchical deep learning for text classi\ufb01cation. CoRR , abs/1709.08267.\nMarco A. F. Pimentel, David A. Clifton, Lei Clifton, and Lionel Tarassenko. 2014. Review: A review of novelty\ndetection. Signal Process. , 99:215\u2013249, June.\nR. Pincus. 1995. Barnett, v., and lewis t.: Outliers in statistical data. 3rd edition. j. wiley & sons 1994, xvii. 582\npp., \u00a349.95. Biometrical Journal , 37(2):256\u2013256."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd317-934e-11ee-a327-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 9, "chroma:document": "W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult. 2013. Toward open set recognition. IEEE\nTransactions on Pattern Analysis and Machine Intelligence , 35(7):1757\u20131772, July.\nBernhard Sch \u00a8olkopf, John Platt, John Shawe-Taylor, Alexander Smola, and Robert Williamson. 2001. Estimating\nsupport of a high-dimensional distribution. Neural Computation , 13:1443\u20131471, 07.\nM. L. Tsetlin. 1961. On the behavior of \ufb01nite automata in random media. Avtomatika i Telemekhanika , 22:1345\u2013\n1354."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd318-934e-11ee-b8f9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 9, "chroma:document": "1354.\nJyothsna Veeramreddy, V . Prasad, and Koneti Prasad. 2011. A review of anomaly based intrusion detection\nsystems. International Journal of Computer Applications , 28:26\u201335, 08.\nYang Yu, Wei-Yang Qu, Nan Li, and Zimin Guo. 2017. Open-category classi\ufb01cation by adversarial sample\ngeneration. CoRR , abs/1705.08722.\nY . Zhang, N. Meratnia, and P. Havinga. 2010. Outlier detection techniques for wireless sensor networks: A survey.\nIEEE Communications Surveys Tutorials , 12(2):159\u2013170, Second."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd319-934e-11ee-83ee-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2011.08755.pdf", "page": 9, "chroma:document": "Xuan Zhang, Lei Jiao, Ole-Christoffer Granmo, and Morten Goodwin. 2020. On the convergence of tsetlin\nmachines for the identity-and not operators. arXiv preprint arXiv:2007.14268 ."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31a-934e-11ee-9613-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "Self-timed Reinforcement Learning\nusing Tsetlin Machine\nAdrian Wheeldon, Alex Yakovlev and Rishad Sha\ufb01k\nMicrosystems Group, Newcastle University, UK\nEmail: {a.r.wheeldon2, alex.yakovlev, rishad.sha\ufb01k}@ncl.ac.uk\nAbstract \u2014We present a hardware design for the learning\ndatapath of the Tsetlin machine algorithm, along with a latency\nanalysis of the inference datapath. In order to generate a\nlow energy hardware which is suitable for pervasive arti\ufb01cial"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31b-934e-11ee-83b0-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "intelligence applications, we use a mixture of asynchronous design\ntechniques\u2014including Petri nets, signal transition graphs, dual-\nrail and bundled-data. The work builds on previous design of\nthe inference hardware, and includes an in-depth breakdown\nof the automaton feedback, probability generation and Tsetlin\nautomata. Results illustrate the advantages of asynchronous de-\nsign in applications such as personalized healthcare and battery-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31c-934e-11ee-bddd-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "powered internet of things devices, where energy is limited and\nlatency is an important \ufb01gure of merit. Challenges of static timing\nanalysis in asynchronous circuits are also addressed.\nI. I NTRODUCTION\nWe present a comprehensive design and analysis for an\nasynchronous learning datapath based on the Tsetlin machine\n(TM) algorithm. When coupled with asynchronous inference\nhardware [1], a complete asynchronous TMcapable of online\nlearning is formed. We also analyze the latency of the infer-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31d-934e-11ee-99c1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "ence datapath, showing its evolution during training.\nThe TM[2] algorithm is effective in many large classi\ufb01ca-\ntion problems [3, 4, 5]. In addition, the TM\u2019s reinforcement\nlearning and logic-based inference make it a good candidate\nfor energy ef\ufb01cient hardware [6]. We design an energy-frugal\nTMhardware with a view of use cases in pervasive arti\ufb01cial\nintelligence; e.g. in personal healthcare, accessibility, environ-\nmental monitoring and predictive maintenance. We give a brief"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31e-934e-11ee-a71a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "introduction to the TMalgorithm in Section II.\nWe wish to implement the TMusing asynchronous circuits\nas they can be bene\ufb01cial for low energy sensing systems [7],\nwhen tightly coupled with analog blocks [8], and when power\ndelivery is unstable or unpredictable [9]. These features are\noften present in the aforementioned applications [10].\nThe architecture of the hardware is designed using a hy-\nbrid Petri net model in Section III. The model incorporates\ninference and learning components."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd31f-934e-11ee-8ef9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "inference and learning components.\nTMs use groups of reinforcement automata, called Tsetlin\nautomata ( TAs), to create an ensemble learning effect. We\ndecompose the automata reinforcement feedback into three\nstages to aid the hardware design in Section IV.\nThe reinforcement process of the TMinvolves random\nchoice with de\ufb01ned probabilities\u2014some \ufb01xed and some vary-\ning at runtime. These probabilistic choices enable diversity\nof learning in the TAs. We design a quasi delay insensitive"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd320-934e-11ee-b0be-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "(QDI) pseudorandom bit generator ( PRBG ) for this use in the\nClauses Automaton Teams\nTA TA\nMajority VotingClause Resultsexclude\nClass Con\ufb01denceFeedback\nClassi\ufb01cationInput Feature Vector ( f) PRBG\nLearningFigure 1. Overview of the Tsetlin machine ( TM) architecture.\nTMin Section V, and motivate its use in other low power\napplications.\nConsidering the implementation of the state-holding TAs, we\ncompare QDIand bundled-data ( BD) design styles suitable for"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd321-934e-11ee-bd28-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "the low energy applications in Section VI. The comparison\nwill show the limitations of the two asynchronous design\nmethods in terms of area, energy and scalability. We use the\nWORKCRAFT framework [11] for speci\ufb01cation and synthesis\nof the QDI implementation using signal transition graphs\n(STGs), as well as veri\ufb01cation of the STGproperties required for\na functional and robust design. For BD, we are inspired by the\ndesynchronization work\ufb02ow of Cortadella et al. [12] in using"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd322-934e-11ee-ae74-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "a commercial synthesis tool designed for synchronous design.\nBDcircuits are readily-implementable with industrial-quality\ntools, in contrast with QDIcircuits.\nMain contributions of this work:\n\u2022Visualization of an asynchronous TMhardware architec-\nture using Petri nets (Section III).\n\u2022Latency analysis of the asynchronous TMinference datap-\nath from [1] and its evolution during training (Section III).\n\u2022A detailed decomposition of TAreinforcement feedback\ninto three stages (Section IV)."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd323-934e-11ee-b7c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 0, "chroma:document": "into three stages (Section IV).\n\u2022Design of a QDI PRBG for probabilistic choice in the TM\nand other low power applications (Section V).\n\u2022Comparison of QDIand BDimplementations of the TAfor\nlow energy hardware (Section VI).\nII. T SETLIN MACHINES IN BRIEF\nTMslearn patterns in binary data using propositional logic.\nThe main inference component of the TMis the clause that\ncomposes an AND expression of the input features and theirarXiv:2109.00846v1  [eess.SP]  2 Sep 2021"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd324-934e-11ee-ad26-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "1 2 n \u00b7 \u00b7 \u00b7 n+ 1 n+ 2 2n \u00b7 \u00b7 \u00b7Action 1 Exclude Action 2 Include\nPenalty RewardFigure 2. Tsetlin automaton state graph.\ncomplements. The TMcomprises many clauses, each produc-\ning a vote. The composition of each clause is controlled by a\nvector of exclude bits (see Figure 1). These bits are parameters\nthat are learned by teams of TAs.\nEach clause can produce a vote for its class. The algorithm\nstates that half of the clauses vote positively, while the other"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd325-934e-11ee-8f12-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "half of the clauses vote negatively (we will denote this by\nthe boolean c neg). The inclusion of inhibition in the voting\nsystem enables non-linear decision boundaries in the infer-\nence process. A majority vote gives an indication of class\ncon\ufb01dence. This con\ufb01dence is used to classify the input data\nand in\ufb02uence future decisions of the automata through the\nfeedback mechanism [2]. In this work we consider only the\nsingle class TMfor simplicity."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd326-934e-11ee-9228-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "single class TMfor simplicity.\nThe TAis a class of \ufb01nite reinforcement automaton [13].\nIt produces an exclude output for states below the midpoint,\nand include for states above the midpoint as illustrated in\nFigure 2. The TAreceives a penalty or reward from the\nfeedback mechanism based on the current state of the TM.\nContinued rewards in the end states cause the TAto saturate.\nA penalty in one of the midstates ( norn+ 1) causes the\nTAto transition across the decision boundary\u2014inverting its"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd327-934e-11ee-ac64-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "output from exclude to include, or vice versa. The feedback\nmechanism is described in more detail in Section IV.\nIII. A SYNCHRONOUS TSETLIN MACHINE\nFigure 3 is an architectural diagram for the asynchronous\nimplementation of the TMshown in Figure 1, demonstrating\nthe scalability of the design. The diagram can be composed\nwith the Petri-net-like tiles in Figure 4 to form a visualization\nof the complete system. In these diagrams, rectangles represent"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd328-934e-11ee-a8f1-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "transitions or computations, and circles represent places as in\nclassical Petri nets. When abutting these tiles, adjoining places\nare reduced to a single place. This representation of the system\nis not a formalism.\nThe main structure is a 3\u00d73grid constructed with feature\ninputs in rows and clauses in columns. The small size facil-\nitates the explanation of the system, but the architecture can\nbe easily extended by appending clause columns to the right\nside, or feature rows below f3."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd329-934e-11ee-92ec-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "side, or feature rows below f3.\nThe clause tile is detailed in Figure 4a and comprises a\npartial clause ( PC) computation for the feature input and its\ncomplement (as introduced in Section II); a partial feedback\ncalculation for stage three feedback ( FB3); and two TAs. The\nfeedback stages will be discussed later in Section V.\nThe clause sum is computed in the sum tile at the bottom\nof each clause column. Figure 4b details the sum tile. Stage\nClause\nTileClause\nTileClause\nTile\nClause\nTileClause"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32a-934e-11ee-80c9-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "TileClause\nTileClause\nTile\nClause\nTileClause\nTileClause\nTile\nClause\nTileClause\nTileClause\nTile\nSum\nTileSum\nTileSum\nTilef1\nf2\nf3\nFB1calcyexp\nlearnClassi\ufb01cationConcurrency OpportunityFigure 3. Architectural diagram for a Tsetlin machine with three feature\ninputs (rows) and three clauses (columns).\n(a) Clause tile.\n (b) Sum tile.\nFigure 4. Petri-net-like visualizations for the Tsetlin machine system: (a) the\nclause tile including partial clause ( PC) and two Tsetlin automata ( TAs); and"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32b-934e-11ee-8262-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "(b) the sum tile.\ntwo feedback ( FB2) for the clause is computed at this stage\nusing the clause sum.\nThe diagrams capture the potential for both concurrency and\nserialization in the design. As presented, the system operates\nin a serialized manner where the feature inputs are passed\nthrough each clause in series. By contracting the places in\nthe highlighted regions of Figure 4a, tokens are passed to the\nright without waiting for PCcalc and FB3calc, this allowing\nclauses to compute in parallel."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32c-934e-11ee-8969-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "clauses to compute in parallel.\nThe majority of the system is implemented in a dual-\nrail ( DR) style using a reduced completion detection ( RCD)\nscheme [7]. However, the TAsare implemented in a single-rail\n(SR) style (as indicated in Figure 4a) which will be motivated\nand discussed in Section VI.\nThe RCD scheme is a simpli\ufb01ed scheme which completion\ndetects only the outputs of a block. This brings bene\ufb01ts of\nearly propagation and drastically reduced area and power"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32d-934e-11ee-857f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 1, "chroma:document": "overheads versus traditional full completion detection. The\ntrade-off is an added timing assumption on the return-to-idle\nphase of the DRsignals. Full details of the RCD scheme are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32e-934e-11ee-8214-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "fpep\n0fnep\n1\npcnfnen\n0fpen\n1\npcp(a) Partial clause\n(PC).\ncpcn. . . . . .\n(b) Partial clause com-\nbiner.\nHA0 HA1 HA2 HA3\nHA4 HA5 HA6 HA7\nHA8 FA0 FA1a0a1 a2a3a4a5 a6a7\ns cs c\nscs\nc\nspinvc\ny0 y1 y2spinv\ny3(c) Population count with implicit\ndual-rail wires and blocks.\nclass 0class 1\n(d) Magnitude comparator.\nFigure 5. Dual-rail circuits for inference. Spinv = spacer inverter.\navailable in [7].\nA. Inference Circuits\nThe inference datapath comprises the clauses, majority"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd32f-934e-11ee-832b-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "voting and classi\ufb01cation introduced in Figure 1. Clauses are\nseparated into partial clause calculations as discussed earlier\nin this section and can be implemented by the schematics in\nFigures 5a and 5b. Feature inputs occur in DRencoding along\nwith exclude signals e0ande1. The outputs of the partial clause\nare combined using a DR AND tree.\nMajority voting calculated the sum of the clause votes ( csum)\nand is implemented using DRpopulation count. The schematic"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd330-934e-11ee-859f-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "in Figure 5c shows the implementation based on DRhalf-\nand full-adders. The ORgates and wires in this circuit are\nalso implicitly DRencoded. Two spacer inverters (spinv) are\nrequired to ensure uniform spacer polarity at the outputs.\nFor this single class TMexample, the output is classi\ufb01ed\nusing a threshold function in the form of a magnitude com-\nparator. The magnitude comparator lends itself to low energy\nimplementation in DRand saves energy by evaluating single-"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd331-934e-11ee-8755-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "bit comparisons from most signi\ufb01cant bit to least signi\ufb01cant\nbit only if needed, all within the period of one cycle [1].\nTo expand the example to two classes or more, an argmax\nfunction would take the place of thresholding.In the following sections we will explore the inference\ncircuit delays and analyze the inference latency of the asyn-\nchronous datapath with differing input operands. For synthesis\nwe use a commercial, off-the-shelf technology library: UMC"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd332-934e-11ee-a779-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "65 nm LOW-LEAKAGE , herein referred to as UMC 65LL.\nB. Delay Analysis\nSince we use an RCD scheme which enables early propaga-\ntion [1], the delays of the circuits can vary based on the input\noperands. We analyze the delay distributions of the inference\ncircuits using uniformly-distributed random inputs to gain an\nunderstanding of this phenomenon. We will use this to explain\nthe evolution of latency during training in the next section.\nClause: In the worst-case delay no literals are excluded"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd333-934e-11ee-a316-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "(e= 0), all feature inputs are logic-1 ( f= 1); therefore the\ndelay is bounded by the slowest input and the delay arc is\nthrough the positive rail. In the best-case delay, all but one\nliteral is excluded and the literal is logic-0; therefore the delay\nis bounded by signal propagation through the ORtree in the\nnegative rail. Since the clause is a large AND tree, there is a\nhigh probability of c= 0, so the delay distribution is mostly\nclustered at the lower end as Figure 6 shows. A much smaller"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd334-934e-11ee-9f47-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "portion is seen towards the circuit\u2019s maximum delay and is\nassociated with c= 1.\nPopulation Count: The delay of the circuit is bounded\nby the carry chain formed by HA8,FA0and FA1. In the best\ncase there are no carries and the circuit latency is governed by\nthe longest logic path which comprises the \ufb01rst two layers of\nHAs (which have roughly equal paths), through the ORgates\nand \ufb01nally FA0to the y1output. In the worst case, output y3\nmust wait for the aforementioned best-case path as well as"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd335-934e-11ee-b4d4-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "the carry through FA1. This leads to the log-normal-like delay\ndistribution seen in Figure 6.\nMagnitude Comparator: For uniform random inputs and\na normalized worst-case delay of 1, the comparator achieves\na mean delay of 0.05. This is attributed to early propagation\nand the evaluation of single-bit comparisons from MSB toLSB.\nThe delay of the circuit increases as the absolute difference\nbetween operands decreases, because more of the MSBs are"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd336-934e-11ee-be8e-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "equal. This circuit is therefore fastest during inference and\nthe later stages of training. The operand space halves for each\nsingle-bit comparison leading to the negative exponential delay\ndistribution in Figure 6.\nC. Operand-dependent Inference Latency\nWe investigate the evolution of inference latency during\ntraining by plotting the spacer \u2192codeword latency at different\nstages. We use a binarized version of the I RISFlower dataset1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd337-934e-11ee-9409-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 2, "chroma:document": "to provide feature inputs ( f). Exclude inputs ( e) are retrieved\nfrom a TMpre-trained on the same dataset after the speci\ufb01ed\nnumber of epochs. Figure 7 shows the distributions for the\nuntrained TM, and the trained TMafter 4 epochs and 50 epochs.\nThe plots show the trend of the mean towards greater circuit\ndelay as the TMis trained and therefore more clauses become\n1Available: https://github.com/cair/TsetlinMachine"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd338-934e-11ee-97e5-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "Meanc=0 c=1\n00.20.4Clause\nMean00.20.4Population Count\nMean\nDecreasing Operand Delta\n00.20.4Magnitude Comparator\nMean\n0 0.2 0.4 0.6 0.8 100.20.4\nNormalized DelayInference DatapathProbabilityFigure 6. Latency distributions of the Tsetlin machine inference datapath and\nits components for uniformly-distributed random inputs. Post-synthesis results\nusing the UMC 65 nm LOW-LEAKAGE library.\nMean\n00.20.4Untrained\nMean\n00.20.4Trained 4 epochs ( 81 % Accuracy)\nMean\n140 160 180 200 22000.20.4"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd339-934e-11ee-adda-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "Mean\n140 160 180 200 22000.20.4\nLatency ( ps)Trained 50 epochs ( 96 % Accuracy)Probability\nFigure 7. Latency distribution of the Tsetlin machine inference datapath for\nthe I RISdataset.\nactivated during the computation of each datapoint ( c= 1).\nIn this case we see more examples from the right side of\nthe clause delay distribution in Figure 6. This is the main\ncontributor to the increase in mean latency as the system\nbecomes more trained. The magnitude comparator offers a"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33a-934e-11ee-8225-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "netdecrease in mean latency as training progresses since the\ndifference between input operands will increase, meaning we\nshift towards the left of the distribution in Figure 7. However\nthis shift is somewhat smaller than that of the clauses, and\nit therefore has a lesser effect on the inference datapath. The\ndistribution of the population count is much more uniform\nthan that of the other circuits, and therefore has a negligible\neffect during training.IV. D ECOMPOSITION OF FEEDBACK"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33b-934e-11ee-8832-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "During learning, feedback is given to each TAto in\ufb02uence\nits future action, and therefore control pattern detection in the\nTMas a whole. The feedback is based on a carefully-designed\npayoff matrix [2, p. 10] and is split into two categories: Type\nI and Type II. Type I feedback reinforces good decisions and\npenalizes false negative outputs. Type II feedback combats\nfalse positives by penalizing TAsthat exclude when c= 1.\nWe decompose the feedback logic into three stages based on"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33c-934e-11ee-8384-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "scope: FB1atTMlevel, FB2at clause level, and FB3atTAlevel.\nFB1and FB2are assigned types: none, T1orT2; these are based\non the Type I and Type II concepts introduced previously. FB3\nassigns an action to a single TA: penalty, reward or inaction;\nand is based on the outcomes of the payoff matrix2. Each\nfeedback stage\u2019s output is input to the next, until \ufb01nally the\noutput of FB3is input to the corresponding TA. The following\nsections discuss each feedback stage in detail. Finally we will"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33d-934e-11ee-899d-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "discuss the synthesis of the feedback circuits in the DRstyle\nin Section IV-D.\nA.FB1:TM-level Feedback\nInputs: learn ,yexp\nOutput: Feedback type (none, T1orT2)\nThe \ufb01rst stage broadly enables feedback for the entire TMif\nlearn is asserted. The feedback type is chosen according to\nyexp \u2014the expected class as provided by the labeled data\u2014as\nfollows:\nFB1=\uf8f1\n\uf8f4\uf8f2\n\uf8f4\uf8f3none,forlearn = 0\nT1, forlearn = 1,yexp = 1\nT2, forlearn = 1,yexp = 0\nB.FB2: Clause-level Feedback\nInputs: FB1, cneg,pT"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33e-934e-11ee-b4c7-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "B.FB2: Clause-level Feedback\nInputs: FB1, cneg,pT\nOutput: Feedback type (none, T1orT2)\nThe second stage swaps the feedback type from T1toT2,\nor vice versa if the associated clause is negated (c neg= 1,\ndiscussed in Section II). Additionally, feedback is stopped\n(inaction given) if the number of activated clauses (those\nproducing a logic-1 at the output) meets the threshold, T. This\nis determined by probabilistic choice, pT, which is discussed"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd33f-934e-11ee-b00a-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "at the end of this section. If there is no feedback from the\nprevious stage ( FB1=none), then none will be generated\n(FB2=none). Parameter c negis determined at design time,\ntherefore the logic for FB1can be separated and depends only\nonFB1and pT. This will be demonstrated in Section IV-D.\nClause-level feedback can be summarized as follows:\nFB2=\uf8f1\n\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2\n\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3none,for FB1=none\nnone,for FB1=T1,pT= 0\nnone,for FB1=T2,pT= 1\nT1, for c neg= 1,FB1=T2,pT= 0\nT2, for c neg= 1,FB1=T1,pT= 1"}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd340-934e-11ee-81c3-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 3, "chroma:document": "T2, for c neg= 1,FB1=T1,pT= 1\nT1, for c neg= 0,FB1=T1,pT= 1\nT2, for c neg= 0,FB1=T2,pT= 0\n2We do not consider boosting of true positive feedback, which is an option\nin the original algorithm."}


created_at: 2023-12-05 09:10:18
topic: persistent://default/default/a0c119a8-dc6b-46d3-8158-5f41a08fd57f
id: 152cd341-934e-11ee-a185-ac198e74a6f7
encoding: FLOAT32
metadata:
{"source": "source_documents\\2109.00846.pdf", "page": 4, "chroma:document": "Table I\nTRUTH TABLE FOR FB3.\u00d7=DON \u2019T CARE .\nFB2 inc c x p s FB3\nnone\u00d7 \u00d7\u00d7\u00d7 inaction\nT1 1 0\u00d70 penalty\nT1 1 0\u00d71 inaction\nT1 1 1\u00d70 inaction\nT1 1 1\u00d71 reward\nT1 0 0\u00d70 reward\nT1 0 0\u00d71 inactionFB2inc c x p s FB3\nT10 1 0 0 inaction\nT10 1 0 1 reward\nT10 1 1 0 inaction\nT10 1 1 1 penalty\nT21\u00d7\u00d7\u00d7 inaction\nT20 1 0 \u00d7 penalty\nT20 0\u00d7\u00d7 inaction\nThe TMalgorithm describes probabilities of generating T1\nand T2[2, p. 10]. In this work we refer to them as P(T1)and\nP(T2)which are based on csumandTas follows:"}


