objects, at the user’s discretion.) The overwhelming visual advantage of
Tumbug is not seen in such a static icon, however: Tumbug can literally show
realistic motion of the objects, like a simulation, whereas CD theory has no
mechanism at all to allow even one moving object to be displayed.

Some similarities between CD theory and Tumbug:

  * •

Both use around 1-2 dozen building blocks.

  * •

Some of the building blocks are identical, and those that are not identical
are similar.

  * •

All slots and Primitive Acts in CD theory can be represented by Tumbug.

  * •

Both use aggregation.

  * •

Both use abstract objects, and treat them slightly differently from concrete
objects.

Some differences between CD theory and Tumbug:

  * •

Tumbug is based largely on physics and math; CD theory is based on human-
oriented actions.

  * •

Tumbug is almost completely visual; CD theory is text-based.

  * •

Tumbug can display images, even moving images; CD theory cannot display images
at all.

  * •

Tumbug can represent phrases; CD theory needs a complete sentence with actor
and action.

  * •

CD theory Primitive Acts cannot cover every possibility; Tumbug Building
Blocks so far appear to be able to do so, at least for WS150 problems.

  * •

CD theory requires separate text to describe TO versus FROM; Tumbug needs no
such distinction, other than arrow placement.

  * •

CD theory distinguishes static objects (PP) from Primitive Acts (ACT) on those
objects, though it is possible in Tumbug to define an object so that it
changes state or behavior via an internal Correlation Box without dictating
those changes explicitly. (This Tumbug capability is not described in this
document, however.)

  * •

Tumbug’s Building Blocks generalize to a small set of (five) Basic Building
Blocks, but CD theory does not note any such generalizations for its Primitive
Acts (ACT).

To a great extent, Tumbug appears to be what CD theory was attempting, but
Roger Schank used non-universal building blocks instead of considering
physics. Similarly, although not discussed in this document, Tumbug appears to
be what formal language theory was attempting, but Noam Chomsky used
traditional text instead of images. Tumbug potentially infuses new life into
these earlier milestone ideas, so perhaps Tumbug also has a chance of becoming
a milestone.

2\. Nearness to a one-to-one mapping to Tumbug concepts

As seen in Figure 210, CD theory concepts are defined in a way that typically
are represented by only one Tumbug concept. This is mostly the result of
relatively clear-cut definitions of CD theory’s Primitive Acts, not
necessarily due to a similar foundation of building blocks between the two
systems. 8/11 = 73% of CD theory Primitive Acts map one-to-one to a Tumbug
concept.

Figure 210: 73% of CD theory Primitive Acts map one-to-one to an equivalent
Tumbug concept.

###  14.2 English grammar representation

####  14.2.1 Kolln sentence patterns

Martha Kolln listed 10 basic English sentence patterns that she claims
constitute the vast majority of sentence patterns that appear in English
(Kolln and Funk 2006). Kolln’s 10 patterns are shown in Figure 211. Kolln
numbered these by Roman numerals, whereas this documented numbered these by
Arabic numerals.

Figure 211: Kolln’s sentence patterns with the order and abbreviations used by
Kolln.

Each of these patterns is described with some sample sentences converted to
Tumbug in the following sections.

It must be understood that Kolln’s statement applies to (1) Only basic
sentence patterns, since sentences of different types could always be combined
via conjunctions to create a much larger set of types through combinations of
the basic types, (2) Only statement versions of the sentence patterns, since
question versions of these patterns may alter the word order; (3) Non-
interjections, since otherwise an interjection consisting of a single word
would qualify as having correct grammar, which it does not.

The standard sentence diagram that was widely taught in the 1960s and 1970s,
formally called the Kellogg-Reed system, is much better known than the Kolln
patterns. However, the formal documentation of the Kellogg-Reed diagrams with
respect to possible combinations of values within the slots for subject
complement (SC), direct object (DO), indirect object (IO), and object
complement (OC) is so weak that the Kellogg-Reed system is ignored here, at
least for comparison purposes.

1\. NP be ADV/TP

Both cases are of ADV/TP are shown next, one where the adverbial is of time
(ADV/T), one where the adverbial is of place (ADV/P).

1.1. With Time Arrow (ADV/T)

See Figure 212.

Figure 212: [Kolln Pattern #1] Tumbug for ”The last performance was yesterday
(ADV/T).”

1.2. With Location Box (ADV/P)

See Figure 213.

Figure 213: [Kolln Pattern #1] Tumbug for ”The students are upstairs (ADV/P).”

2\. NP be ADJ

This Kolln pattern equates only to an object (O) having an attribute (A) with
a value (V), which Tumbug renders in OAV fashion as shown in Figure 214.

Figure 214: [Kolln Pattern #2] Tumbug for ”The students are diligent.”

3\. NP1 be NP2

3.1. As superset

This Kolln pattern demonstrates a principle mentioned by Roger Schank
regarding the need for a canonical form of a sentence when sentence meaning is
represented (Schank 1976, p. 172). In this case, there are two simple ways to
express the same meaning of the sentence ”The students are diligent.”:

  * •

”The students are scholars.” - Involves two noun phrases (NP1 = students, NP2
= scholars). This sentence is represented in Tumbug as shown in Figure 215.

  * •

”The students are scholarly.” - Involves one noun phrase (NP1). and one
attribute value (scholarly = true). This sentence is represented in Tumbug as
shown in Figure 216.

Figure 215: [Kolln Pattern #3] Tumbug for ”The students are scholars.” Figure
216: [Kolln Pattern #3] Tumbug for ”The students are scholarly.”

One of the main goals of representation here is to use only one representation
for the concept of ”scholars/scholarly,” which suggests that only one of these
two diagrams should be declared the canonical form. Figure 217 shows more
clearly how the two previous diagrams are related. (The last two diagrams are
essentially the same diagram, but one has the common attribute ”scholarly =
true” duplicated, the other does not.)

Figure 217: [Kolln Pattern #3] Two Tumbug ways to represent the concept that
”The students are scholarly.”

Currently the Tumbug convention used is to use the superset representation. A
similar sentence they has two representation options is diagrammed in Figure
218.

Figure 218: [Kolln Pattern #3] Two Tumbug ways to represent the concept that
”The tournament was an exciting event.” Top diagram: One attribute applies to
both the set and superset. Bottom diagram: No superset/subset structure, but
note that the same attribute appears in both sets.

Note: The use of the value ”true” in the aforementioned expression ”scholarly
= true” is a quick way to conform to OAV notational convention without need
for additional thought, because the attribute-value form is required to be
”$\langle$attribute$\rangle$ = $\langle$value$\rangle$”, which the expression
”scholarly” alone would not fulfill, and to think of which attribute would
allow ”$\langle$attribute$\rangle$ = scholarly” to make sense (e.g.,
”learnedness = scholarly”, or ”academic character = scholarly”) is more
difficult than merely writing ”scholarly = true.”

3.2. With removed possessive adjective

Another example said to be Kolln Pattern #3 is ”Professor Mendez is my math
teacher” (Kolln and Funk 2006, p. 31). This is another example of the
inaccuracies of the English language, since the verb ”to be” (conjugated here
as ”is”) has a different meaning than before. Here it would be wasted effort
to create a set called ”my math teachers” and to put ”Professor Mendez” as a
single instance of that set, since usually a student has only one math
teacher. Therefore ”is” does not mean ”is a subset of” here. In this case the
underlying problem is use of a possessive adjective, which should be removed
from a sentence before converting the sentence to Tumbug. There exist multiple
ways to represent this sentence without the possessive adjective, one of which
is shown in Figure 219. This is an indication of weak organizational criteria
used in defining the Kolln patterns.

Figure 219: [Kolln Pattern #3] Tumbug for ”Professor Mendez is my math
teacher.” = ”Professor Mendez IS-A math teacher OF me.”

4\. NP linking verb ADJ

4.1. With propositional attitude

This is another Kolln pattern that incorporates very different Tumbug
structures if the verb changes slightly. Many Kolln Pattern #4 verbs involve
general, average impressions, such as ”to seem” and ”to look,” others involve
specific sensory perceptions such as ”to taste,” ”to smell,” ”to sound,” and
others use neither, such as ”to grow” (meaning ”to become”), which has a very
different meaning: a single attribute that changes over time, which has
nothing to do with impressions or sensory modalities. Therefore this Kolln
category is not taken very seriously with respect to Tumbug, other than as a
supply of possibly unusual examples to test against Tumbug representation.

Figure 220 shows an example of Kolln Pattern #4 when the linking verb is a
propositional attitude, in this case ”seem.” English allows an ambiguity here,
regarding the entity (or group) who is interpreting the diligence of the
student. The only logical way to represent this situation is to show an Object
Circle that is unlabeled, as shown in the figure. To show a single person icon
would assume that only one person is doing the interpreting, to show an
”average person” icon would assume that the interpreter is average, to show a
person icon at all would assume that the interpreter is human, and to show a
”DK” (= Don’t Know) wildcard would assume that the reader cares, which might
entail extra, unnecessary, ongoing work in labeling, especially on a very
large network of connected Tumbug icons that no one is likely to care about.

Figure 220: [Kolln Pattern #4] Tumbug for ”The students seem diligent.”

The term ”propositional attitude” is from Davis 1990 (ch. 8), and the need for
this concept occurs in three of the Kolln patterns (Pattern #4, Pattern #9,
Pattern #10). It is a very useful concept that unifies several modalities of
knowledge. For details, see the section on propositional attitudes in this
document.

4.2. With attribute changing over time

An example of a Pattern #4 example that has a very different meaning and
Tumbug diagram is in Figure 221. In this case the only thing happening is that
an attribute value changes in time, which has nothing to do with sensory input
or overall impressions.

Figure 221: [Kolln Pattern #4] Tumbug for ”I grew sleepy.” = ”I transitioned
into the sleepy state.”

4.3. With sensory perceptions

Figure 222 illustrates another example of a propositional attitude, in this
case gustatory taste, which generalizes to any type of sensory perception.

Figure 222: [Kolln Pattern #4] Tumbug for ”The soup tastes salty.”

5\. NP1 linking verb NP1

5.1. As superset

See Figure 223.

Figure 223: [Kolln Pattern #5] Tumbug for ”The students became scholars.”

5.2. With propositional attitude

See Figure 224.

Figure 224: [Kolln Pattern #5] Tumbug for ”That seemed a good idea.”

5.3. With object changing over time

Kolln does not document the possibility of an object changing into another
object over time. The sentence ”The caterpillar became a butterfly” uses the
same verb and tense as before (viz., ”became”) in the sentence ”The students
became scholars” but in this case ”became” implies the object itself has
changed, not just its classification. This is an example of how the Kolln
patterns are based too much on specific verb categories rather than on the
meanings of those words in different contexts.

In contrast, Tumbug can easily represent an unmoving object changing into
another unmoving object in the same location by placing the two objects above
each other along the timeline, with the older object at an earlier time than
the newer object. See Figure 225. Since two solid objects cannot occupy the
same space in our physical world, such representation is sufficient, pro
tempore.

Figure 225: [Kolln Pattern #5] Tumbug for ”The caterpillar became a
butterfly.”

This simplicity comes at a cost of increased risk when creating diagrams,
however: Care must be taken not to place two different objects above each
other in a timeline unless one has moved out of the way completely. See Figure
226.

Figure 226: In the real world of physical objects, Object A may occupy Object
B’s space later in time only if Object B has moved out of the way.

6\. NP intransitive verb

Intransitive verbs are another category of verb that is very poor for
capturing a general meaning of such a verb, as evidenced by the two examples
below that use this same sentence pattern and verb class but have two very
different meanings and Tumbug diagrams.

6.1. With states

Note that in the Kolln example ”The students rested” (Kolln and Funk 2006, p.
35) the verb ”rested” is ambiguous as to whether the rest was physical or
cognitive. Fortunately, a general state diagram such as one for ”exertion
versus rest” (as shown in Figure 227) can subsume both possibilities at once.
Note also that the sentence given does not necessarily state that the state of
the students before the rest state was the exertion state: the students could
have just come from an earlier break, then decided to take a later break.
Therefore only the later specified state (”rest”) is marked with an asterisk
as known; the state diagram exists in the earlier depiction, but no state is
marked. This detail is very important to note when developing a matching
algorithm for Tumbug.

Figure 227: [Kolln Pattern #5] Tumbug for ”The students rested.” = ”The
students were in rest state.”

6.2. With motion

Intransitive verbs can also refer to motion, as in the verb ”to arrive” as
shown in Figure 228.

Figure 228: [Kolln Pattern #6] Tumbug for ”The visitors from El Paso arrived.”

7\. NP1 transitive verb NP2

Even Martha Kolln herself makes a heavy admission about the nature of
transitive verbs when discussing her sentence Pattern #7 (= Pattern VII) of
hers (Kolln and Funk 2006, p. 39):

”Traditionally, we think of the transitive verb as an action word: Its subject
is considered the doer and its object the receiver of the action. In many
Pattern VII sentences this meaning-based definition applies fairly accurately.
In our Pattern VII sample sentences, for instance, we can think of their
assignment as the receiver of the action studied and a home run as a receiver
of the action hit. But sometimes the idea of receiver of the action doesn’t
apply at all:

Our team won the game.
We enjoyed the game.

It hardly seems accurate to say that game ”receives the action.””

This equates to at a deep insight about verbs in all natural languages: Verbs
do not generalize to a single concept (e.g., not motion, action, or
relationship), which suggests the modern concept of a ”verb” is flawed. The
term ”verb” is overloaded with the attempt to include references to attribute
values, supersets, actions upon a target object, sensory input to the actor,
states, cause and effect, and possibly more. The Tumbug viewpoint explains
this problem as poor generalizations of language, leading to a poor knowledge
representation system for language (such as the Kellogg-Reed system or modern
tree diagrams of sentences).

7.1. With data transfer

Sometimes Pattern #7 applies to data transfer and/or perception, such as in
Figure 229.

Figure 229: [Kolln Pattern #7] Tumbug for ”The students studied their
assignments.”

7.2. With motion and contact

The most obvious applicability of Pattern #7 is physical contact of the
subject with the direct object, as in Figure 230.

Figure 230: [Kolln Pattern #7] Tumbug for ”The boy touched the shark.”

7.3. With states

See Figure 231.

Figure 231: [Kolln Pattern #5] Tumbug for ”Our team won the game” = ”Our team
caused the game outcome to enter into the win state.”

7.4. With emotions

See Figure 232. This figure combines what is essentially a State Diagram,
namely the Robinson Icon, which in this case is in the positive version of the
state of Event Related: Joy, elation, triumph, jubilation, which is a
cathected emotion with respect to the direct object, namely the game.

Figure 232: [Kolln Pattern #7] Tumbug for ”We enjoyed the game.” = ”The game
caused our emotion to enter into the joy state.”

8\. NP1 transitive verb NP2 NP3

It appears that this Kolln pattern always maps only to the same Tumbug diagram
of an object being transferred from subject to indirect object, as shown in
Figure 233.

Figure 233: [Kolln Pattern #8] Tumbug for ”The students gave their professor
their homework,” no shorthand diagram used.

9\. NP1 transitive verb NP2 ADJ

This Kolln sentence pattern maps to at least two different types of Tumbug
diagram.

9.1. With propositional attitude

Having a propositional attitude about an object or person does not inherently
involve a change of attribute in time. See the example in Figure 234.

Figure 234: [Kolln Pattern #9] Tumbug for ”The students consider the teacher
intelligent.”

9.2. With attribute changing over time

An object or person can cause a change of attribute in time. See the example
in Figure 235.

Figure 235: [Kolln Pattern #9] Tumbug for ”The teacher made the test easy.”

10\. NP1 transitive verb NP2 NP2

This Kolln sentence pattern maps to at least two different types of Tumbug
diagram.

10.1. With propositional attitude

See Figure 236.

Figure 236: [Kolln Pattern #10] Tumbug for ”The students consider the course a
challenge.”

10.2. With causation

See Figure 237.

Figure 237: [Kolln Pattern #10] Tumbug for ”Barrie named his pug Jill.”

11\. Nearness to a one-to-one mapping to Tumbug concepts

Kolln’s 10 sentence patterns fail to produce any reasonable approximation of
1-to-1 mapping with Tumbug concepts, as shown in Figure 238. Kolln’s examples
are useful, however, as a fairly comprehensive source of sample sentences to
test against Tumbug’s representation ability. If Kolln’s 10 sentence patterns
cover 95% of encountered English sentences, and if Tumbug can represent those
same patterns, then Tumbug should also have at least 95% ability to represent
English sentences. 2/10 = 20% of Kolln patterns map one-to-one to Tumbug
concepts.

Figure 238: Only 20% of Kolln patterns map one-to-one to an equivalent Tumbug
concept.

12\. The five simplest sentence patterns

Note that the various examples of Pattern #6–”The students rested” versus ”The
visitors from El Paso arrived”–produce diagrams that are very different from
one another. The second example describes the arrival of people in motion
whereas the first example is very unclear what it describes, even as to
whether any motion was involved before the action of ”rested.”

This is not the only Kolln grammatical pattern that is highly unpredictable.
Pattern #7–”The students studied their assignments” versus ”The boy touched
the stingray”–has the same general problem. The second example has a direct
object that receives the action whereas the first example has more the nature
that the assignment was doing to action to the students.

These anomalies lead to a profound insight into all human language: All human
language is structured around the highly inconsistent nature of verbs, rather
than being extremely well-structured by basing all grammatical patterns on
objects and object motions. The author suggests this is the primary reason why
natural language has always been so difficult to describe with grammatical
rules, and consequently why the field of AI has always had extra difficulty
with natural language. Even a seemingly exotic language such as Japanese,
which has particles, has the same two meanings of the verb ”to be” as English
does, and goes through the same distortions as English in order to force-fit
such verbs into inefficient, inconsistent data structures.

Side conjecture: The Proto-Human language, if such a language ever existed,
may have had a needlessly difficult grammar that failed to focus only on
objects and motions, so all natural languages since then, no matter how
seemingly different they are, may have carried the same flaw that forced
descriptions of attribute and superset relationships to use unneeded verbs
only because action verb sentences used a structure that required a verb.

This fundamental problem of human grammar does suggest one way to slightly
improve foreign language learning, however: Teach grammar first with only
moving objects, up to three in number, then only later move into more abstract
verbs that are difficult to represent with moving objects, or with cause-and-
effect substituted for motion. Notice that grammar nicely falls into only
about five basic patterns if only the common patterns of motion are used.

Consider Pattern #6, Pattern #7, and Pattern #8. From a general standpoint,
these few sentence patterns fall into a small, logical set of possible motions
of objects:

Pattern #2: Subject does not move, has an attribute and value: 1 object + 0
motions
Pattern #3: Subject does not move, is part of a superset: 1 object + 0 motions

Pattern #6: Subject moves itself: 1 object + 1 motion
Pattern #7: Subject moves and makes contact with Direct Object: 2 objects + 1
motion
Pattern #8: Subject transfers Direct Object to Indirect Object: 3 objects + 1
motion

Pattern #1 can be chosen to be ignored, since it deals only with an attribute
of space or time.
Pattern #4 can be chosen to be ignored, since it deals with a propositional
attitude, which is indirect.
Pattern #5 can be chosen to be ignored, since it deals with a linking verb
over time: complicated.
Pattern #9 can be chosen to be ignored, since it deals with a propositional
attitude, which is indirect.
Pattern #10 can be chosen to be ignored, since it deals with a propositional
attitude, which is indirect.

The summary of these observations is shown in Figure 239.

Figure 239: These five sentence patterns are the simplest, most basic, and by
far the most common in English. These five patterns are the only patterns
discussed in depth in this document.

The above heuristics were a lot of generalization for the sole purpose of
simplifying human grammar, but this practice could be justified if it
sufficiently improves the understanding of grammar, and/or if the simpler
patterns are more frequently encountered. The names of these most basic visual
sentence patterns are given by the author in Figure 240.

Figure 240: A summary of Tumbug patterns, with regard to Parts of Speech and
motion.

An interesting and little-known fact is that Tumbug’s Transfer pattern does
not exist in Spanish in the same way that it exists in English. For example,
the English sentence ”The students gave their professor their homework” would
need to be grammatically rearranged to a Spanish grammatical pattern with an
extra preposition, such as ”The students gave their homework to their
professor” (”Los estudiantes le dieron sus tareas a su profesor”). This fact
illustrates that slight variations in the number of sentence pattern types
occur between major languages, but that Tumbug representations remains
unchanged between such syntactical rearrangements.

Three of the most basic Tumbug patterns (E, C, T) match three of the English
verb valency patterns listed by D.J. Allerton (Allerton 2006, pp. 170-171),
which demonstrates that Tumbug’s natural grammatical patterns have already
been discovered earlier in a different context, therefore Tumbug is already
mostly supported by existing grammar theory. These matches are shown in Figure
241.

Figure 241: Three of Allerton’s valency categories exactly match three of the
four basic sentence patterns of Tumbug. Allerton’s tetra-valent category of
sentence patterns involves four objects, which is beyond the number of Tumbug
patterns discussed in detail in this document.

Side conjecture: Human parsing of a sentence, whether that input is written or
spoken, may involve in part the operation of matching the grammatical pattern
of the input to a small set of known, allowable grammatical patterns. This
matching would be a very fast operation because of the small number of
possible grammatical patterns involved, each of which has only small
structures involved.

Various problems with the Kolln criteria for classifying English sentences
often become evident, and are sometimes noticed by English teachers, therefore
further detailed comparison between Tumbug patterns versus Kolln patterns is
probably not worthwhile.

####  14.2.2 Allerton sentence patterns

Since Allerton mentioned a valency category that is not part of Kolln’s
patterns or Tumbug’s five mentioned patterns, namely the tetra-valent
category, this section briefly considers that category and its relationship to
Tumbug. The tetra-valent categories are shown as text in Figure 242.

Figure 242: Allerton’s tetra-valent categories with examples and abbreviation
key, all described with text. (Source: Allerton 2006, p. 171.)

Allerton lists five examples of the tetra-valent category. Of those five
examples, only one involves a fourth true object. Incidentally, Tumbug calls
this sentence pattern ”Swap” since it generalizes to any type of trade due to
two people + two exchanged objects. The other Allerton examples show either
two locations, which are not true objects, or phrasal verbs, each of which can
be reduced to one verb. The first two examples in this category are shown in
Figure 243.

Figure 243: The first two tetra-valent examples of Allerton, all described
with Tumbug images.

The tetra-valent (= Swap) patterns suggest that there does not exist any upper
limit to the number of sentence patterns that may exist. This is correct, and
this is partly why Tumbug was developed: Tumbug is model-based, which means
that any number of objects and actions can be combined in any way, which
results in a system that can be arbitrarily complicated but will still
describable by a visual model. Some examples of higher valency are:

  * •

swap (4-valent): 2 people + 2 objects

  * •

escrow (4-valent): 3 parties + 1 property

  * •

nuclear fusion (4-valent): 2 incoming atoms + 2 outgoing atoms

  * •

attacked pinned piece in chess (4-valent): 4 pieces

  * •

discovered check whose uncovering piece wins a piece in chess (4-valent): 4
pieces

  * •

swap of two properties through one escrow (5-valent): 2 properties + 3 parties

###  14.3 Object-attribute-value (= OAV) triples

Object-attribute-value (OAV) representation is relatively well-known in
computer science, and is usually called an ”OAV triple,” or sometimes ”OAV
triplet.” The traditional OAV way of diagramming OAV triples is shown in
Figure 244, and the equivalent Tumbug way is shown in Figure 245.

Figure 244: The traditional way to diagram an OAV triple. Figure 245: The
Tumbug way to diagram an OAV triple.

Tumbug uses OAV terminology and its meaning verbatim, although Tumbug
sometimes extends this terminology and the associated diagrams. For example,
Tumbug sometimes adds a novel extension to OAV called ”system-object-
attribute-value” (SOAV), shown in Figure 246, and Tumbug sometimes splits the
OAV concept into the ”subject-object” (SO) concept, an example of which is
shown in Figure 247.

Figure 246: [78] Tumbug for ”In 1765, Thompson went to Cooper’s grave.” This
is an SOAV diagram because objects (O, e.g. ”Thompson”), attributes (A, e.g.,
”status”), and values (V, e.g., ”alive”) are integrated into a larger system
(S), where the larger system is the entire Tumbug diagram shown above for this
English sentence. Figure 247: [3] Tumbug for ”Per social norms, if a helper
helps a helpee, then the helpee will thank the helper.” This is an SO diagram
because no attributes (A) or values (V) are shown in this system (S) that
contains only objects (O) without Attribute Lines.

###  14.4 Vector graphics editors

There exists a basic similarity between Tumbug and vector graphics editors in
that both systems are typically used to represent real-world systems with
diagrams. Both involve images in motion: Tumbug has arrows in its diagrams
that imply motion, and vector graphics editors allow users to actually drag
such diagrams across the computer screen in real time via a mouse button. Both
systems manipulate images with respect to their shapes (circles, squares,
rectangles, lines, arrows, etc.), motions (translation, rotation, scaling,
reflection), shape distortions (shearing, etc.), and their attributes (color,
texture, border, etc.). All the aforementioned listed motions are called
affine transformations, which in 2D using homogeneous coordinates can be done
via a single multiplication of a 3x3 transformation matrix with a 3x1 vector
of the object’s coordinates. Figure 248 shows an example of one vector
graphics editor, OpenOffice Draw, where the user in the process of rotating an
icon.

Figure 248: An OpenOffice Draw menu after an image has been selected. Note the
affine transformations of Rotate and Flip (= reflection) at the start of the
Modify menu.

It should not be surprising to realize that the operations done by vector
graphics editors are the same operations that humans see performed in the real
world on physical objects, since moved diagrams are usually used to represent
some motion aspect of the real world in some useful manner to humans. This is
summarized in Figure 249, assuming that the occurrence is happening with
respect to a given viewer.

Figure 249: A summary of affine transformations and how they relate to the
real world.

The sole visual operations of a vector graphics editor that tend not to
correspond to anything in the real world are the abilities to create and
destroy objects instantly. (Spontaneous creation of virtual particles of the
real world is a possibility in theoretical physics, however.) A vector
graphics editor might have an ”Insert $\mid$ Picture” menu option to implement
the creation operation of an object (see Figure 250, and the deletion
operation might be done by selecting the object and hitting the ”delete” key
on the keyboard. It might be said that the human brain uses an unknown piece
of software that is a combination of vector graphics editor + simulator +
video recorder.

Figure 250: OpenOffice Draw implements the object creation operation by
”Insert $\mid$ Picture.”

Creation of objects out of nothing can be represented easily by Tumbug, as
well, since when the history of the object is viewed with help from a
timeline, suddenly the object is question ceases to exist before a certain
point in its history. If a Tumbug diagrammer desires to be more explicit, the
”Does Not Exist” value could be placed at the specific time and specific place
that an object appeared, as shown in Figure 251. Conclusion: Tumbug can
represent any primary operation that can be performed by any typical vector
graphics editor.

Figure 251: Creation of objects out of nothing can be represented in Tumbug by
showing no earlier object history.

The same statements are true for object deletion. See Figure 252.

Figure 252: Deletion of objects can be represented in Tumbug by showing no
later object history.

###  14.5 Semantic networks

Semantic networks are a fairly well-known KRM in computer science, one that
uses nodes and links, and therefore bears a superficial resemblance to Tumbug.
A well-known article by Woods (1975) discusses a large number of shortcomings
of semantic networks. Tumbug addresses most or all these shortcomings. Full
coverage of a comparison and semantic networks would be too lengthy for this
document, however, so only a sampling of Woods’ criticisms are given here,
with each criticism followed by the Tumbug solution.

####  14.5.1 Kinship relations

With regard to data bases and semantic networks, the sentence ”Harry is John’s
uncle” is problematic because:

  1. 1.

The term ”uncle” is partially ambiguous because it could refer to either the
mother’s side of the family or the father’s side of a family.

  2. 2.

Even if the ambiguity is removed, the uncle kinship relation involves
comparing nodes that are not directly connected in a kinship tree.

  3. 3.

Even if the applicable nodes are explicitly connected with an ”uncle” link, it
is too impractical to store all possible uncle relationships explicitly, due
to combinatorial explosion of all possible matches.

See Figure 253 for a standard kinship tree to see the nodes and links that are
involved in this problem. If a specific ego (= the person under consideration
when describing that person’s kinship relationships) is chosen, then the ego
and the two choices of uncle are marked with a 0D Marker as in Figure 254.
Tumbug solves these issues by the following solutions, respectively:

  1. 1.

Either use an ”OR” to disjunct the mother’s side of the family with the
father’s side of a family, or combine both sides of the family into a more
generic group called ”parents’ siblings” and use the male members of that
group.

  2. 2.

Use a 0D Marker to flag the ego and uncle group, then use a relocatable
template that highlights any two nodes, based on the ego at one end and the
uncle distance-and-position at the other end as in Figure 255 and Figure 256.
(Again, one of the main goals of Tumbug is to represent everything visually,
as much as practical.)

  3. 3.

Store a specific ego-uncle relationship only when that specific relationship
has been needed to be considered by the system when answering a query.

Figure 253: A standard (lineal) kinship diagram. (Source: “Eskimo Kinship
Chart” by Fred the Oyster/Wikimedia Commons, CC0.) Figure 254: A kinship
diagram that flags the possible uncles of the shown ego. Figure 255: The ego-
uncle relationship can be thought of a spatial template that can be moved to
start at any ego node. Figure 256: The ego-uncle relationship template moved
to a different ego node (off the left-hand edge of the chart).

Further development anticipated: The uncle kinship relationship template shown
is rigid and specific to the tree layout shown, not flexible with respect to
arbitrary tree layout or unpredictable counts or unpredictable branches. The
conversion of such a rigid to a flexible template was not attempted.

####  14.5.2 Specific objects versus generic objects

Woods wrote (Woods 1975, p. 22): ”For example, if I create a node and
establish two links from it, one labeled SUPERC and pointing to the ”concept”
TELEPHONE and another labeled MOD and pointing to the ”concept” BLACK, what do
I mean this node to represent? Do I intend it to stand for the ”concept” of a
black telephone, or perhaps I mean it to assert a relationship between the
concepts of telephone and blackness–i.e., that telephones are black (all
telephones?, some telephones?). When one devises a semantic network notation,
it is necessary not only to specify the types of nodes and links that can be
used and the rules for their possible combinations (the syntax of the network
notation) but also to specify the import of the various types of links and
structures–what is meant by them (the semantics of the network notation).”

First, presumably SUPERC = superclass, which would be clearer in modern times
due to the modern popularity of object-oriented programming. Woods’ described
semantic network diagram is shown in Figure 257.

Figure 257: The SUPERC semantic network example described by Woods.

The three interpretations that Wood mentioned are shown in Figure 258. Tumbug
represents a superclass using the Venn diagram convention of drawing a region
around the region in question, where the larger region represents the
superclass. Tumbug represents each of these three interpretations differently,
by virtue of making attribute-value pairs distinctly different from Object
Circle labels and Location Box labels, and by virtue of making objects and
classes distinctly different via Object Circles and Aggregation Boxes, and by
virtue of allowing Aggregation Boxes to span arbitrary regions, where each
Aggregation Box may have its own unique label. Woods is correct about the
drawbacks of semantic networks, but clearly Tumbug has much more
representative power than semantic networks. Even object-oriented programming
clearly has more representative power than semantic networks. Semantic
networks are clearly becoming old-fashioned due to their inherent limitations.

Figure 258: Woods’ three interpretations of the SUPERC example, as represented
in Tumbug. Top: ”The particular telephone in question (= telephone #1) is
black.” Middle: ”All telephones are black.” Bottom: ”Some telephones are
black.”

####  14.5.3 Intensions versus extensions

Woods wrote (1975, pp. 22-23): ”Basically a predicate such as the English word
”red” has associated with it two possible conceptual things which could be
related to its meaning in the intuitive sense. One of these is the set of all
red things–this is called the extension of the predicate. The other concept is
an abstract entity which in some sense characterizes what it means to be red,
it is the notion of redness which may or may not be true of a given object;
this is called the intension of the predicate.”

This confusion was already addressed in this document by the insight that a
concept can arbitrarily change its Part of Speech merely by moving the node
that holds that concept to a different part of the OAVC diagram of Figure 330.
Therefore ”red” as an adjective differs from ”red” as a noun in a
diagrammatically simple way in Tumbug. Therefore ”extension” is ”red” as an
adjective, ”intension” is ”red” as a noun. Such Parts of Speech differences
were shown in Figure 259, and are shown again here in Figure 259 with only
words for colors.

Figure 259: The concept of color, like most concepts, can appear as different
Parts of Speech.

##  15 Applications of Tumbug

###  15.1 Natural language translation

Below is the translation of a single English sentence to two different foreign
languages, where each foreign language is from a different branch of the Indo-
European languages: French, from the Romance languages, and German, from the
Germanic languages. Three steps are shown in the translation to Tumbug: (1)
translation, (2) analysis, (3) Tumbug representation, then a general
discussion of the process and its nuances.

####  15.1.1 Conversion from a natural language to Tumbug

1\. Translation

See Figure 260.

Figure 260: An English sentence translated into French and German.

2\. Analysis

See Figure 261.

Figure 261: Analysis of the sentence in the three translations.

3\. Tumbug representation

The following abbreviations appear in the diagrams:

DC = Don’t Care
DNE = Does Not Exist

For the English sentence, see Figure 262.

Figure 262: Tumbug for ”I threw the ball” in English.

For the French translation, see Figure 263.

Figure 263: Tumbug for ”I threw the ball” in French.

For the German translation, see Figure 264.

Figure 264: Tumbug for ”I threw the ball” in German.

####  15.1.2 Discussion of language translation

1\. Small mismatches

Evidence that Tumbug is nearly a universal representation between these three
languages is that the iconic Tumbug structures for the same sentence are
absolutely identical, as seen in Figure 265.

Figure 265: Comparison of Tumbug representations of the same sentence across
three languages.

There are a few subtle mismatches, however, all in the attribute values, and
all due to some languages being more specific than others. These mismatches of
attribute values are marked in Figure 266.

Figure 266: Emphasized differences between Tumbug representations of the same
sentence across three languages.

Pairs of minor mismatches across languages in Tumbug of this example are
marked with arcs.

The mismatches are the following:

  1. 1.

Regarding aim: French and German care about the exact meaning of ”to throw”
since each of those languages would use a different verb if the meaning were
slightly different. For example, French would use ”jeter” if no aim or
detached object were involved, and German would use ”abgeben” (or ”ausgeben”)
for the same reason. English makes no such distinction, so English shows ”DC”
(= don’t care) for the ”aim present” attribute.

  2. 2.

Regarding attachment: French and German care whether the object being thrown
is an attached object or not, such as ”to throw a switch” versus ”to throw a
ball.” A different verb would be used, depending on the attachment status.

  3. 3.

Regarding gender: French and German have different grammatical genders for
their respective words for ”ball.” English has no grammatical genders so it
shows ”DNE” (= Does Not Exist). In this case, grammatical gender is a human-
attributed quality that does not exist in inanimate objects, so in this
example, object gender is irrelevant. If a human were in the place of direct
object or indirect object, however, then gender could be relevant, possibly
because of outfit, marriage status, voice range, or any other influences of
biological gender.

Other subtle mismatches could occur easily with different sentences or with
different languages. For example ”you” may have different levels of formality,
such as in German and Dutch, some languages do not have grammatical genders,
such as in Japanese, and some languages do not have articles, such as in
Japanese and Latin.

All such mismatches are due to differing levels of details across the
languages, however, none whatsoever with the structure and relationships
between the top-level entities or actions after the exact word meaning has
been determined. Of course a sentence could be completely ambiguous even in
its own language, such as ”The pot is in the pen,” but that is a deficiency on
the part of a speaker or the language, not a deficiency in Tumbug. Tumbug does
have some potential weaknesses, but these are subtle and very unlikely to be
encountered. These weaknesses are discussed later in this document.

2\. Corrections of the small mismatches

There exist only two obvious ways to correct the aforementioned minor
mismatches in the translation example: (1) require the writer/speaker to be
more specific, as specific as the most specific languages that might be
encountered in later translation, (2) use context to determine the most likely
meaning. The first option is usually not within the reader’s control, so
usually context must be used.

Consider how context of a sentence can be implied by following sentences:

  1. 1.

Textual passage: ”I saw the player trying to steal third base and he wasn’t
too far away. I threw the ball.”
Suggested context: baseball, where a ball is a critical piece of equipment
that must be thrown accurately
Implied aim present: TRUE
Implied detached object: TRUE

  2. 2.

Textual passage: ”I was going through the trash, pocketing anything of value
and tossing aside anything else. First I found an earring, so I pocketed it.
Then I found a disgustingly moldy ball of cheese. I threw the ball.”
Suggested context: trash, where the ball described is something the holder
would toss aside quickly, to anywhere
Implied aim present: FALSE
Implied detached object: TRUE

The contextual information in this case clarifies the two main ambiguities
from before: whether there was an intended target, and whether the thrown item
was attached. The ambiguity of which gender is involved depends only on
language-specific grammatical rules, and not the meaning of the word after the
proper word has been clarified. The result of this context determination from
the English (or other language) sentence is the first step for translation,
and results in essentially a word vector of TRUE-FALSE values. The second step
of translation matches this word vector against a database of (typically)
foreign words, as shown in Figure 267.

Figure 267: The one determined context (here C1) from the source language is
matched against each word in a data base of candidate words in the target
language, with one such candidate word (here W2) used in the example. Note
that a ”DON’T CARE” value will match either a ”TRUE” or ”FALSE” value.

The third step would be to calculate which context-word combinations had the
closest match. For example, in the context of baseball, a context that was
determined by analysis of the the sentence just described whose source
language was English, each context vector of the source language would be
matched against each context vector of the target language. For two contexts
from English and two candidate words from French the result would be (2x2 =)
four vectors, each cell of which contains an indication as to whether there
was a match or mismatch. For example, TRUE matched with FALSE = mismatch, and
FALSE matched with DON’T CARE = match. Figure 268 shows the final results. One
simple metric for determining degree of match would be to simply add the
number of matches. In the context of baseball (C1), the highest matching
candidate French word was ”lancer,” which had two matches versus only one
match with ”jeter.” Figure 268 shows this entire matching process as
described.

Figure 268: Each given context (here C1 and C2) is matched against each
candidate word in the target language (here W1 and W2), and the count of
matches across each vector’s elements is shown at the right of that vector. In
the context of baseball (C1) being matched against French words, the French
word with the highest count is assumed to be the French word that has the
closest match. The closest matching French word is therefore assumed to be the
word ”lancer” (W2), which has two matches, as compared to the next highest
matching word, ”jeter” (W1), which has only one match. Therefore in the
context of baseball, the French word with the closest match to the English
word ”to throw” is ”lancer.”

###  15.2 Mathematics: Arithmetic

Arithmetic in Tumbug uses a Causation Arrow to represent the operator (e.g.,
”+” or ”-”) or function (e.g., 3x, x2), and uses Data C Object Circles to
represent the numbers. Numbers are not real-world objects so numbers cannot
use Physical C Object Circles. This is in contrast to the way the OOP language
Smalltalk handles numbers: in Smalltalk, numbers are objects, and operators
are methods. For example, in Smalltalk to add two numbers, such as 1 and 2,
Smalltalk would send the method ”+ 2” to the number ”1” (Winston 1998, p. 16).

In Tumbug, ”1 + 2” would be rendered as in Figure 269, which translates to the
verbal description ”Messages 1 and 2 together are inputted to the virtual
adder, which causes the number 3 to be outputted.” A ”virtual adder” could
mean anything like a person’s mind or an imaginary computer adder. If a real
world computer or real-world calculator were involved then the circle around
the Causation Arrow would be solid: a Physical C Object Circle. Numbers will
always be virtual, so numbers are always represented by Data C Object Circles.
Although computer memory can erase and overwrite numbers at will, human memory
cannot: the concept of ”1” and ”2” are practically permanent concepts in human
memory after being learned, as is their sum ”3,” therefore the diagram shows
the ”3” concept in a different location than directly below the ”1” or ”2” so
that neither the ”1” nor ”2” are overwritten by Tumbug conventions. Whether
the machine performing the adding is mechanical (e.g., a cash register) or
electrical (e.g., a computer), any calculation it performs takes time,
therefore a timeline (Time Arrow) should always be present in any Tumbug
diagram that represents calculation.

Figure 269: To add the numbers 1 and 2, Tumbug sends them as two pieces of
data to a virtual operator called ”+”.

###  15.3 Logic: First Order Predicate Calculus

####  15.3.1 Syllogisms with Venn diagrams

Venn diagrams are not only closed geometrical figures, each of which may
contain points in arbitrary locations within any given region, but can also be
considered diagrams of physical regions that may contain objects within any
given region, like an aerial view of a horse corral that may contain horses.
Therefore a Venn diagram can be considered only a cleaned up and abstracted
aerial photograph, which conveniently bridges the gap between reality and
geometrical diagrams. Therefore there is no need for Tumbug to include a
separate Building Block for a Venn diagram.

One type of formal logic is First Order Predicate Calculus (FOPC). FOPC often
uses syllogisms represented as Venn diagrams, which are used to represent
sets. As noted earlier in this document, sets are very conveniently
represented in Tumbug with any type of Aggregation Boxes, since order (or 2D
position) is not important in a set. Therefore three intersecting sets in
Tumbug would be represented as three intersecting C Aggregation Boxes, as
shown in Figure 270.

There exist only 24 valid types of syllogisms, and each type has been given a
person’s name. In this document the Socrates example has the Barbara form, the
reptiles example has the Celarent form, and the rabbits example has the Darii
form.

Figure 270: A Venn diagram in Tumbug uses C Aggregation Boxes instead of the
familiar circles of Venn diagrams.

####  15.3.2 Barbara form (AAA-1)

The Socrates syllogism example in this section is the well-known syllogism:

All men are mortal.
Socrates is a man.
Therefore, Socrates is mortal.

In this syllogism the two sets involved–assuming that the information is
regarded as sets whenever possible–happen to be nested and non-intersecting,
as shown in Figure 272. In the literature, usually even Socrates is
represented as a third set, for some reason, but technically Socrates is an
element of a set, not a set per se, so is represented an object here.

Figure 271: One way to represent the Socrates syllogism in Tumbug.

The way a person would likely construct a mental picture such as in the
aforementioned syllogism would be incrementally as each of the three sentences
is read: a single mental picture is used across all three sentences, and after
each sentence is read a new piece is added to the composite picture as shown
in Figure 272. Note that the final composite picture does not use the
”mortals” concept as a set, but rather as an attribute-value pair.

Figure 272: The three diagram creation steps that correspond to the three
sentences of the syllogism.

The clue that a set is involved is the verbal form ”all X,” such as in ”all
men” or ”all birds” or ”all even numbers.” In logical calculus or math the
term ”for all” would be the analogous clue. The English verb ”is” (or ”are” or
”am”) when used in descriptions of the form ”X is a Y” (or ”X are a Y” or ”I
am a Y”) means ”X is in set Y,” as in ”Socrates is man” or ”Tweety is a bird”
or ”6 is an even number.”

Using these rules the aforementioned sequence of three steps results
mechanically, with little or no thought, while building up the associated
image on the side: (1) ”All men are mortal” = ”The set of men has attribute
mortal” $\Rightarrow$ create set ”men” and attach attribute-value ”mortality =
mortal.” (2) ”Socrates is a man” = ”The object Socrates is in the set of men”
$\Rightarrow$ create object ”Socrates” and place it inside set ”men.” (3)
”Socrates is mortal” = ”The object Socrates has attribute mortal”
$\Rightarrow$ realize that since set ”men” has attribute-value ”mortality =
mortal” then every object in that set must have that same attribute-value, by
convention.

The last (3rd) step as represented in Figure 273 may seem somewhat trivial and
redundant. However, if this syllogism had been represented with two sets, one
set for ”mortals” as is usually done, the insight of the third step could be
made visible as an arrow connecting Socrates and the set ”mortals.” Such an
insight more closely describes of people would typically think of such a new
association, often as a ”that’s true, though I never quite thought of it that
way before” reaction. This type of spatial fusion evidently happens frequently
when mentally piecing together maps to plan a route. For example, if a person
knows the layout of the interior of a building well, and knows the relative
placement of the surrounding buildings but has always entered and exited
through the same entrance of the first building, then if challenged with
finding a new exit that is closer to the destination building, would likely
mentally need to fuse the two separate mental maps in order to decide which
exit would be closer to the destination from a global viewpoint–a shortcut
that could be always be used thereafter. This same ”insight arrow,” called a
Relationship Marker in Tumbug, is explicitly shown in all the syllogisms in
this document. Note that extra connections such as these are likely
responsible for increasing a person’s ”understanding” of a topic (Atkins
2019).

Figure 273: The insight that Socrates is mortal as represented by a
Relationship Marker that directly connects the two concepts.

Note that the order of the three sentences of the Socrates syllogism is not
critical. For example, if the 1st and 2nd steps were swapped, the conclusion
(and final diagram) would be the same, but would have been built up in a
different order, as shown in the Figure 274.

Socrates is a man.
All men are mortal.
Therefore, Socrates is mortal.

Figure 274: The three diagram creation steps that correspond to the rearranged
three sentences of the syllogism.

Insight: The described process of the brain building up composite images that
represent meaning across multiple sentences may provide major clues about the
brain automatically providing context for disambiguation of pronouns in
subsequent sentences. In contrast, if a sentence containing an ambiguous
pronoun was a sentence that was removed from its surrounding sentences then
the sentence would likely lose critical information that could disambiguate
that pronoun if the sentence were given to a computer to analyze.

####  15.3.3 Celarent form (EAE-1)

The reptile syllogism example of this section is of the Celarent form, and is
shown in Figure 275.

No reptile has fur.
All snakes are reptiles.
Therefore, no snake has fur.

Figure 275: One way to represent the reptile syllogism in Tumbug.

The clue that a set is involved is the verbal form ”no X,” such as in ”no
reptile” or ”no man” or ”no negative number.” Implied is that there exists a
set called ”X” and no element in that set can have the properties described
afterward. In logical calculus or math the term ”there does not exist” would
be the analogous clue.

A complication arises from this example: How to represent ”no” (or ”not”) with
respect to an allowed range in Venn diagrams. Two ways are shown in Figure 276
and Figure 277. Representation #1: The X-ed line between the two sets means
that an element from one set cannot exist in the set on the other side of the
X-ed line.

Figure 276: An element of the left set with attribute A cannot appear in the
right set.

Representation #2: The black dot represents an element in the set on the left,
and the arrows emanating from it suggest the range in which it is allowed to
appear in the Venn diagram. The dot can appear in any hatched area, i.e., the
dot can appear in any area except the white (unhatched) region. The arrows are
shown for clarity here, but are not needed in practice since the shading
already spans every region where the element may go. See Figure 277 and Figure
278.

Figure 277: An element of the left set with attribute A can appear in any set
or region except the white ( A) region. Figure 278: An element of the left set
with attribute A can appear in any set or region except the white (A) region.
This is the diagram that applies to the reptiles example, where A = fur, A =
NOT fur.

For Tumbug the hatched region representation would probably be clearest, since
Tumbug has many lines in the diagram already, and using an X-ed line does not
make it clear where the element in the left set can go, only where it cannot
go.

Using the same conventions as in the Barbara example, the following steps
would be involved to create the appropriate Venn diagram, as shown in Figure
279. (1) ”No reptile has fur” = ”The set of reptiles and the set of furry
animals cannot intersect” $\Rightarrow$ create set ”reptiles” and set ”furry
animals” so that they are non-intersecting. (Alternative: Attach attribute-
value ”skin covering = NOT fur” to the reptiles set, and attach attribute-
value ”skin covering = fur” to the furry animals set.) (2) ”All snakes are
reptiles” = ”The set of snakes is a subset of the set of reptiles”
$\Rightarrow$ create set ”snakes” and place it inside set ”reptiles.” (3) ”No
snake has fur” = ”No element in the snakes set can exist in the furry animals
set.” Realize that since any snake must exist only in the ”snakes” set, which
is mutually exclusive with the ”furry animals” set, this is already implied,
by the shading convention.

Figure 279: The three diagram creation steps that correspond to the three
sentences of the syllogism.

####  15.3.4 Darii form (AII-1)

The rabbits syllogism example of this section is of the Darii form. Figure 280
shows this syllogism in square Venn diagram form.

All rabbits have fur.
Some pets are rabbits.
Therefore, some pets have fur.

Figure 280: One way to represent the rabbits syllogism in Tumbug.

The clue that a set is involved is the verbal form ”all X,” here ”all
rabbits.”

Using the same conventions as in the Barbara example, the following steps
would be involved to create the appropriate Venn diagram, as shown in Figure
281. (1) ”All rabbits have fur” = ”The set of rabbits is a subset of the set
of furry animals” $\Rightarrow$ create set ”rabbits” and set ”furry animals”
so that ”rabbits” is a subset of ”furry animals.” (Alternative: Attach
attribute-value ”skin covering = fur” to the rabbits set.) (2) ”Some pets are
rabbits” = ”Some of the set of pets is inside the set of rabbits, but not
necessarily inside the set of furry animals” $\Rightarrow$ create set ”pets”
and make it overlap with the set ”furry animals” but in a way that it does not
overlap into the purely ”furry animals” region. (3) ”Some pets have fur” =
”Some part of the pets set is inside the furry animals set.” Realize that part
of the ”pets” set exists in the ”furry animals” set.

Figure 281: The three diagram creation steps that correspond to the three
sentences of the syllogism.

###  15.4 Computer language representation

Algorithms can be represented in Tumbug rather easily, especially from
flowcharts, but also from the code itself.

Sequential steps of an algorithm are shown in Tumbug as occurring sequentially
in time, along a Timeline Arrow. The rectangles along the timeline are not
flowchart boxes, but rather C Aggregation Boxes. The contents of each C
Aggregation Box on the left is the Tumbug representation of the corresponding
statement ${s_{i}}$ in the code to the right, so each C Aggregation Box will
typically hold connected Tumbug icons such as C Object Circles, Motion Arrows,
attribute-value pairs, etc. Each red dot shown in the code samples here
indicates the statement within the program that is executing, and there is a
corresponding red dot beside the corresponding part of the Tumbug
representation.

Caveat: Tumbug’s ordinary level of representation deals with human life and
human-sized objects, so using Tumbug for computer code would likely be
extravagantly wasteful of space. The same is true of mathematical operations.
Tumbug is similar to a very high order programming language, especially an
object-oriented simulation language, so Tumbug would be most appropriate for
important human-level events that lend themselves to visual simulation.

####  15.4.1 Sequential code

Sequential code as a flowchart in Tumbug would look like Figure 282. Figure
283 of a Tumbug representation of sequential flowchart run being traced
suggests why Tumbug is more applicable to machine understanding of a
situation: At each step the entire piece of code is recopied, with the only
change being where the current program statement is, which is indicated by the
red dot. This results in the ability to see not only the current focus (i.e.,
the current program statement) but also its context in all the code around it
(i.e., the program).

Figure 282: Sequential code in Tumbug is represented by one path composed
largely of Pathway Tubes. Figure 283: Trace of sequential code in Tumbug, with
flowchart copied at each step on right: (S1 S2 S3 S4).

####  15.4.2 Looped code

A loop in the code would cause the collection of steps inside the loop to
occur repeatedly, with each collection occurring sequentially, and the steps
within each loop occurring sequentially, as well. This is shown in Figure 284
and Figure 285.

Figure 284: An IF-THEN in Tumbug is represented by two paths, the choice of
which depends on one state. Figure 285: Trace of looped code in Tumbug, with
flowchart copied at each step on right: (S1 S2 S3 S2 S3 S4).

####  15.4.3 Branching code

A conditional statement (such as IF-THEN, IF-THEN-ELSE, or CASE) causes only
one thread of execution to occur, chosen appropriately from the multiple
alternatives given in the program. Tumbug uses a Split Time Arrow to represent
this splitting of possibilities. The result looks very similar to a flowchart,
as shown in Figure 286 and Figure 287.

Figure 286: An IF-THEN-ELSE in Tumbug is represented by two paths, the choice
of which depends of one state.

.

Figure 287: Trace of branching code in Tumbug, with flowchart copied at each
step on right: (S1 S3 S4).

###  15.5 Natural language representation

This section discusses the more difficult grammatical topics within English
that might cause translation from English into Tumbug to be difficult. Almost
all these topics are about verbs. The first section covers several of the
simpler topics in a single section; each subsequent section covers one
complicated topic per section.

####  15.5.1 Categories of verbs

1\. Active verbs versus stative verbs

Active verbs are also called ”dynamic verbs.” Active verbs are verbs that
describe obvious actions (e.g., ”to throw,” ”to run,” ”to hit,” ”to jump,” ”to
sing”), whereas stative verbs describe state of being (e.g., ”to know,” ”to
earn,” ”to love,” ”to want,” ”to own”).

Sometimes a third category is included, called ”event verbs.” This is said to
include sentences such as ”Four people died in the crash” and ”It’s raining
again,” but these examples fit well as stative verbs since ”to die” is a
change of state from living to dead, and ”to rain” is a change of state of
weather to wet precipitation.

The differences between these categories is clearly represented in Tumbug
because actions must either involve motion, whereupon a Motion Arrow and C
Object Circle is used, or be caused or entered via a state diagram, whereupon
a Causation Arrow and State Circle is used. If color coding is used then the
difference in diagrams is even more obvious since State Circles have no color
whereas C Object Circles do. See Figure 288.

Figure 288: In Tumbug, active verbs use very different arrows than stative
verbs.

2\. Transitive verbs versus intransitive verbs

A transitive verb is one that requires a recipient object for the sentence to
make sense (e.g., ”I pushed the cart,” ”Please bring coffee,” ”Juan threw the
ball”), in contrast to the sentence making without a recipient object (e.g.,
”They jumped,” ”The dog ran,” ”She arrived”). Some verbs can be either,
depending on context (e.g., ”Her husband shaved” versus ”The barber shaved the
customer”). The difference is extremely obvious in Tumbug: if only the
subject’s C Object Circle is present then the verb is transitive, but if the
direct object’s C Object Circle is present as well as the subject’s C Object
Circle then the verb is intransitive. See Figure 289.

Figure 289: In Tumbug, transitive verbs use two Object Circles whereas
intransitive verbs use one Object Circle.

3\. Linking verbs

Linking verbs do not describe actions or causations of states, but rather
describe an object with either a predicate adjective or a predicate noun. In
Tumbug, a predicate adjective is represented by attribute-value pair that
hangs off an Object Circle, and a predicate noun (also called a ”predicate
nominative”) is represented as a superset that is represented as a C
Aggregation Box that surrounds the predicate noun. The main example of a
linking verb in English is ”to be,” which in Spanish has the two forms ”ser”
and ”estar.” Any verb that describes sensory input (e.g., ”to feel
$\langle$adjective$\rangle$,” ”to smell $\langle$adjective$\rangle$),” ”to
taste $\langle$adjective$\rangle$”) or any verb that describes impressions
(e.g., ”to seem $\langle$adjective$\rangle$,” ”to look
$\langle$adjective$\rangle$,” ”to become $\langle$adjective$\rangle$”) is a
linking verb. Tumbug inherently does not consider linking verbs to be verbs in
the normal sense since Tumbug does not represent linking verbs with any type
of arrow, only with Attribute Lines. See Figure 290.

Figure 290: There exist two types of linking verbs, and neither involves an
action or a causation.

4\. Helping verbs

”Helping verbs” are also called ”auxiliary verbs.” A helping verb is an
additional verb used with the main verb to convey some type of additional
information. The types of information conveyed can be:

  * •

time tense

    * –

HELPING VERB EXAMPLES: to be, to have, to do

    * –

SENTENCE EXAMPLES:

      * *

I am going to the zoo.

        * ·

helping verb = ”to be”

        * ·

conveyed info = present continuous tense

      * *

I have waited a long time for this.

        * ·

helping verb = ”to be”

        * ·

conveyed info = present continuous tense

      * *

I did not want to go home.

        * ·

helping verb = ”to do”

        * ·

conveyed info = simple past tense

  * •

modality

    * –

HELPING VERB EXAMPLES: can, could, may, might, will, would

    * –

SENTENCE EXAMPLES:

      * *

It might rain.

        * ·

helping verb = ”might”

        * ·

conveyed info = possibility

      * *

Jorgen can skate backwards.

        * ·

helping verb = ”can”

        * ·

conveyed info = (general) ability

      * *

I may delete this later.

        * ·

helping verb = ”may”

        * ·

conveyed info = possibility

  * •

emphasis

    * –

HELPING VERB EXAMPLES: to do

    * –

SENTENCE EXAMPLES:

      * *

I do like the book!

        * ·

helping verb = ”to do”

        * ·

conveyed info = emphasis

From a Tumbug perspective, ”helping verb” is an ambiguous concept based on
syntax, not grammar, therefore most helping verb categories can be ignored in
this study. See Figure 291. However, time tenses and modal verbs are deeper
grammatical topics that are discussed in detail elsewhere in this document,
each in its own dedicated section.

Figure 291: ”Helping verbs” is a diverse class of verbs with different uses
and representations in Tumbug.

5\. Regular verbs versus irregular verbs

The issue of whether a verb is regular or irregular is a syntactical issue,
not a semantic issue. Since Tumbug deals with semantics, Tumbug per se does
not consider whether a verb is regular or irregular, which means for example
that there will be no written distinction between the concepts of ”see” and
”saw”: the only difference is implied by the time shown at that event on the
timeline (Time Arrow) as shown in Figure 292. The closest that Tumbug might
approach this distinction is if: (1) software of some type were translating
from Tumbug representation of a sentence in one language to a different
language; (2) Tumbug were being used to store conjugation rules.

Figure 292: Tumbug ordinarily distinguishes between past and present via
timeline, therefore Tumbug is not usually concerned with textual descriptions
or their spelling differences since such text is redundant. Therefore the two
diagrams are identical except for the positioning of times on the timeline.

6\. Phrasal verbs

A phrasal verb is a verb that includes a preposition (e.g., for, with, to, up,
across) and/or adverb to the verb, which together can create a new meaning
that is completely different from the meaning of the original verb alone.
Phrasal verbs are irrelevant in Tumbug, for the same reason that conjugations
are irrelevant in Tumbug: these are spelling, phrasal, or syntactical
differences that are independent of meaning. Some examples are listed in
Figure 293.

Figure 293: Examples of phrasal verbs. Tumbug is concerned only with unique
concepts, not with new meanings that have been attributed to groups of
existing words by humans, therefore phrasal verbs have little inherent meaning
to Tumbug.

A Tumbug diagram of the original verb will usually be very different than a
Tumbug diagram of the same verb made into a phrasal verb.

7\. Infinitives

As noted elsewhere, conjugations of verbs by time, number, and person are
essentially meaningless in Tumbug since Tumbug fundamentally uses no text
anyway; such conjugation information can only be inferred by the diagram or
its attribute values, which also fundamentally use no text. Since the
infinitive form of a verb is characterized by lack of time, number, and
person, a generic Tumbug diagram that somehow depicts a given action alone is
how an infinitive would appear in Tumbug, as shown in Figure 294.

Figure 294: In Tumbug the infinitive form of a verb will show no tense (i.e.,
no Time Arrow), no quantity (i.e., a Nonquan is shown rather than an Object
Circle or Location Box), and no person (i.e., no relative location of the
agent is implied by the diagram). Here the arrows suggest an appendage moving
upwards, which suggests the infinitive ”to wave.”

8\. Reflexive verbs

One definition of a ”reflexive verb” is a verb whose direct object is the same
as its subject. For example, a verb used with a reflexive pronoun in a
sentence of the form ”I $\langle$verb$\rangle$ myself” or ”You
$\langle$verb$\rangle$ yourself” are necessarily reflexive verbs, such as the
sentence ”I wash myself” as shown in Figure 295. Verbs that describe actions
that are ordinarily done to oneself, such as ”to wash” or ”to shave,” do not
require a reflexive pronoun. In Tumbug, reflexive verbs are immediately
visually recognizable because the action immediately circles back to the
subject. This is a benefit since no confusion about the meaning of the
sentence can occur, unlike in written English.

Figure 295: Top: ”To wash” as a transitive verb. Bottom: ”To wash oneself” or
”to wash” as a reflexive verb.

####  15.5.2 Active tense versus passive tense

Marvin Minsky mentioned active tense versus active tense (Minsky 1974, p. 33)
via the three example sentences in Figure 296, Figure 297, Figure 298.

Figure 296: Tumbug for ”He kicked the ball.” (active tense) Figure 297: Tumbug
for ”The ball was kicked.” (passive tense) Figure 298: Tumbug for ”There was
some kicking today.” (passive tense)

Tumbug works as beautifully on passive tense as it does on active tense. If
one of the objects is not identified, a completely unlabeled Object Circle can
be used, or a question mark ”?” can either be placed inside an Object Circle,
a lone question mark can be used, or an icon can be labeled with a lone
question mark can be used. The representation method that is most consistent
with the spirit of Tumbug is that of showing a completely unlabeled Object
Circle for a sentence’s Subject because some Subject must have caused the
action: a missing Subject would imply that the Subject either did not exist or
that action arose out of a void, or both, which is logically impossible. In
addition, the placing of an unlabeled Object Circle in the diagram makes it
less likely that such passive tense wording can be used to misdirect the
reader, which is often the intent of passive tense. With this Tumbug
convention any unaccounted-for object must automatically be brought into the
diagram with an obvious presence that is difficult to ignore. For an example
of this justification, in the sentence ”The ball was kicked,” the Merriam-
Webster definition of ”to kick” is ”to strike out with the foot or feet,”
which means that a visual representation of ”to kick” absolutely must contain
some reference to a foot, which in turn implies the existence of some entity
that had that foot as part of its body, an entity that was capable of striking
out with that foot. This undefined entity is exactly the meaning of an
unlabeled Physical Object Circle, therefore such a Physical Object Circle with
a foot should be used, which has the automatic side benefit of making any
implied unknowns blatantly obvious instead of being hidden via language.

####  15.5.3 Grammatical time aspects

The basic time ranges in which events are described in natural languages are,
of course: (1) past, (2) present, and (3) future. Most natural languages add
an additional complication to timelines called ”aspect,” however. The most
common aspects are: (1) simple, (2) progressive, (3) perfect, (4) present
progressive. ”Aspect” describes whether a given event happened at a specific
point in time, versus whether a given event happened over a period of time.
Each of these cases is listed in Figure 299 with its overall description and
associated Tumbug diagram. Note that these particular meanings of types of
aspect are not always fixed–natural language is imperfect–but for any specific
interpretation Tumbug will render them faithfully. For example, although
”simple” aspect usually means at a single point in time, it can also refer to
a cluster of points in time.

In this section the time spans that are of interest are emphasized using a
thickened Time Arrow in those regions. This practice is not a fixed part of
Tumbug: braces or shaded regions can accomplish the same task of indicating
any specific times spans of interest.

Figure 299: Aspect is largely only a matter of how long the interval of time
is, over which an action happens.

1\. Simple aspect

The ”simple” aspect means a single point in time, or sometimes a cluster of
such points, shown here by the thick line that ranges over narrow period of
time. See Figure 300, Figure 301, Figure 302.

Figure 300: (in the past:) ”Ken called (via an information line) yesterday.”
Figure 301: (in the present:) ”Ken is calling (via an information line) now.”
Figure 302: (in the future:) ”Ken will call (via an information line)
tomorrow.”

2\. Progressive aspect

The ”progressive” aspect means the described action occurred over a range of
time. This situation is shown here by the thick line that ranges over an
extended period of time longer than a single moment. See Figure 303, Figure
304, Figure 305.

Figure 303: (in the past:) ”Terry was walking his dog yesterday.” Figure 304:
(in the present:) ”Terry is walking his dog now.” Figure 305: (in the future:)
”Terry will be walking his dog tomorrow.”

3\. Perfect aspect

”Perfect” in grammatical context means ”complete,” meaning the described
action has stopped. The ”perfect” aspect means an ongoing action in the past
was completed by a certain reference time.

See Figure 306, Figure 307, Figure 308.

Figure 306: (perfect in the past / past perfect:) ”The dog had eaten bones (by
the time we got home).” Figure 307: (perfect in the present / present
perfect:) ”The dog has eaten bones (by now).” Figure 308: (perfect in the
future / future perfect:) ”The dog will have eaten bones (by the time we get
home).

Note that in the present perfect the reference time coincides with ”now.” This
differs from the other two perfect tenses (past perfect and future perfect),
where the reference time is always assumed to be after the time of the
described action, and with both the described action and reference time on the
same side of ”now.”

4\. Perfect-progressive aspect

”Perfect” in grammatical context means ”complete,” meaning the described
action has stopped. The ”progressive” aspect means the described action
occurred over a range of time. The ”perfect-progressive” aspect means an
action over a length of time was completed. See Figure 309, Figure 310, Figure
311.

Note that English is ambiguous in this aspect since this aspect does not
necessarily imply that the described action will happen after the reference
time. Some English instructors say the described action is ”expected to
continue,” others say the described action ”may continue.” English obviously
has some flaws for accurate scientific usage. The figures show that the thick
line of event duration continues (via arrow head), which implies the ”expected
to continue” interpretation was intended. Figure 309, Figure 310, and Figure
311 are identical to the preceding figures except the event duration stops,
which implies the ”may continue” interpretation was intended.

Figure 309: (in the past / past perfect progressive:) ”Denny had been driving
a forklift (and might have continued to do so).” Figure 310: (in the present /
present perfect progressive:) ”Denny has been driving a forklift (and might
still be doing so).” Figure 311: (in the future / future perfect progressive:)
”Denny will have been driving a forklift (and might continue to do so).”

The difference between perfect aspect and progressive-perfect aspect is
summarized in Figure 312, Figure 313, and Figure 314.

Figure 312: In unfortunate English language ambiguity, perfect-progressive
aspect can mean either that the described action continued or that it stopped.
Figure 313: A Split Time Arrow can be used to include both possibilities of
”perfect-progressive aspect.” Figure 314: A comparison of the ”perfect aspect”
diagram with the Split Time Arrow diagram of ”perfect-progressive aspect.”

5\. Discussion of grammar

The preceding sections demonstrate that Tumbug can accurately represent all
the time aspects of English grammar, so well in fact that certain ambiguities
in English time aspects become very clear.

Note that Tumbug representations can be made much more thoroughly visual than
they were in the aforementioned aspect diagrams. For example, instead of
writing ”yesterday,” ”now,” or ”tomorrow” on the timelines, any type of visual
representation could be used instead, such as a clock, a calendar, numbers, or
lengths of a cylinder that represents some multiple of time. Similarly, icons
of humans can be used instead of C Object Circles. Also, details can always be
added, such as a human holding a leash that is around the dog’s neck, which
would presumably represent an additional force (represented by a Force Arrow)
that constrains the dog’s movement. Tumbug is ultimately intended to be
entirely visual, and it has very nearly that capability already if the human
diagrammer has the time to fill in the visual details.

####  15.5.4 Modal verbs

1\. Lists

1.1. Simple lists

The following list is likely a complete list of the modal verbs of American
English and British English, in alphabetical order:

  1. 1.

be able to

  2. 2.

can

  3. 3.

could

  4. 4.

had best (uncommon, spoken only)

  5. 5.

had better (uncommon)

  6. 6.

have got to (uncommon)

  7. 7.

have to

  8. 8.

may

  9. 9.

might

  10. 10.

must

  11. 11.

needn’t (uncommon)

  12. 12.

ought to

  13. 13.

shall

  14. 14.

should

  15. 15.

will

  16. 16.

would

These modal verbs precede regular verbs, and convey the following modal
concepts that modify the context of the regular verbs that follow them:

  1. 1.

ability

  2. 2.

advice

  3. 3.

conditional - decomposes

  4. 4.

formal directive

  5. 5.

future

  6. 6.

habit in past tense

  7. 7.

habit, present tense

  8. 8.

intention

  9. 9.

likelihood/certainty

  10. 10.

not necessary

  11. 11.

obligation/correctness/rightness

  12. 12.

offer/invitation/persuasion

  13. 13.

permission

  14. 14.

(polite) request

  15. 15.

prediction

  16. 16.

suggestion

  17. 17.

willpower/intent (after ”I” or ”we” in formal British)

Unfortunately the modal verbs listed above are highly overloaded, meaning that
one modal verb can have multiple meanings from the above list. The above list
is a conglomeration of the words used to describe of concept meanings given by
different authors on the Internet. Below is the nearly the same list of modal
verbs, with most of their possible conveyed concepts added, each concept taken
from the above list.

1.2. Modal verb $\Rightarrow$ modal concept

  1. 1.

be able to

    1. (a)

Mike is able to solve complicated math equations. (ability)

    2. (b)

Will you be able to walk the dog this afternoon? ((polite) request)

  2. 2.

can

    1. (a)

She can swim underwater. (ability)

    2. (b)

You can’t learn a new language if you don’t practice. (conditional -
decomposes)

    3. (c)

I can be with you at 9 a.m. (likelihood/certainty)

    4. (d)

Can I give you a hand? (offer/invitation/persuasion)

    5. (e)

Yes, you can have an ice-cream. (permission)

    6. (f)

Can you change this for a different color, please? ((polite) request)

  3. 3.

could

    1. (a)

He could run the mile in 4 minutes when he was younger. (ability)

    2. (b)

If I could be President for one day, I would change the world. (conditional -
decomposes)

    3. (c)

I could make friends easily when I was younger. (habit in past tense)

    4. (d)

Mary could have been better than her sister at ballet. (likelihood/certainty)

    5. (e)

I could go for you? (offer/invitation/persuasion)

    6. (f)

Could you pass me the salt please? ((polite) request)

    7. (g)

Could I leave early today, please? (permission)

  4. 4.

had best

    1. (a)

You’d best leave it till Monday. (advice, informal)

  5. 5.

had better

    1. (a)

I had better not buy that coat. (advice, formal)

  6. 6.

have got to

    1. (a)

Drivers have got to get a license to drive a car in the US. (necessity)

    2. (b)

I have got to be at work by 8:30 AM. (obligation/correctness/rightness)

  7. 7.

have to

    1. (a)

We have to wear a uniform at work. (obligation/correctness/rightness)

  8. 8.

may

    1. (a)

She may come later. (likelihood/certainty)

    2. (b)

You may go now. (permission)

    3. (c)

May I speak to Mr. Jones please? ((polite) request)

  9. 9.

might

    1. (a)

The train might be delayed. (likelihood/certainty)

    2. (b)

Might I have a glass of water please? ((polite) request)

    3. (c)

She might like a jewelry box for her birthday. (suggestion)

  10. 10.

must

    1. (a)

It’s snowing, so it must be very cold outside. (likelihood/certainty)

    2. (b)

Children must do their homework. (obligation/correctness/rightness)

  11. 11.

needn’t

    1. (a)

You needn’t do your homework. (not necessary)

  12. 12.

ought to

    1. (a)

You ought to have your car serviced before the winter. (advice)

  13. 13.

shall

    1. (a)

Pupils shall not use the main entrance. (formal directive)

    2. (b)

This time next week I shall be in Scotland. (future)

    3. (c)

Shall we tell him? (offer/invitation/persuasion)

    4. (d)

We shall not be moved! (willpower/intent)

  14. 14.

should

    1. (a)

You should stop smoking. (advice)

    2. (b)

He should be here in 5 minutes. (future)

    3. (c)

The government should reduce the sales tax. (ideal/preferred)

    4. (d)

Should I turn the heating on? (offer/invitation/persuasion)

    5. (e)

Should we invite Sarah and David? (possibility)

  15. 15.

will

    1. (a)

This year Christmas will fall on a Monday. (future)

    2. (b)

John will always be late! (habit, present tense)

    3. (c)

She will come tomorrow. (intention)

    4. (d)

You will need to show your boarding pass at the gate.
(obligation/correctness/rightness)

    5. (e)

Short hair will suit you. (prediction)

    6. (f)

Will you take the dog for a walk please? ((polite) request)

  16. 16.

would

    1. (a)

Tom would do something like that, wouldn’t he? (habit, present tense)

    2. (b)

When I lived in Italy, we would often eat in the restaurant next to my flat.
(habit in past tense)

In particular, requests such as ”Could you pass me the salt please?” are at
the polite end of a spectrum of imperatives, the other end of which would be a
stark imperative such as ”Pass me the salt.” Prediction also involves a
spectrum of likelihood between certainty and non-certainty, such as ”The total
will be 10” versus ”Short hair will suit you.” Note that the ”permission”
concept of some modal verbs is analogous to having a pass, whereas the ”formal
directive” concept (in the negative) is the opposite, and analogous to not
having a pass, therefore these two concepts are related.

There are some interesting or important things to know about these modal
verbs: (1) Consensus among linguists is that Future Tense always implies
possibility, at least if the agents are people. (2) ”To be able” seems at
first impression like a synonym for ”can” but a little-known English grammar
rule renders this first impression untrue since the generality of the event
being discussed dictates which of those two modal verbs must be used. However,
some people maintain that either ”to be able” or ”can” can be used for a
(polite) request, even though politeness usage is rarely mentioned in
conjunction with ”to be able.” The table in Figure 315 includes these extra
meanings for ”to be able,” and includes the implication of possibility with
each use of Future Tense, though with these extra meanings put in parentheses.

In summary, the following concept meanings were excluded for the following
reasons:

  * •

Conditional: irrelevant since ”if-then” has no effect on the concept included

  * •

Desire: often seems implied, but not listed by anybody as implied, as in fact
is not logically implied

Modal verbs may be redundant, especially in tense, due to prior coverage by
Tumbug. For example, one meaning of ”shall” is mere future tense: In British
English ”I shall” is exactly equivalent to ”I will” in American English, yet
Tumbug already is capable of showing future tense with a timeline so Tumbug
does not need a modal verb to express future tense another way.

In this way the original list of modal concepts is reduced to the following
shorter list of modal concepts:

  1. 1.

Ability

  2. 2.

Advice

  3. 3.

Formal Directive

  4. 4.

Formality (Formal, Informal)

  5. 5.

Habit (Past Tense, Present Tense)

  6. 6.

Ideal

  7. 7.

Intention

  8. 8.

Likelihood

  9. 9.

Obligation

  10. 10.

Offer

  11. 11.

Permission

  12. 12.

Possibility

  13. 13.

Prediction

  14. 14.

(polite) Request

  15. 15.

Suggestion

  16. 16.

Tense

  17. 17.

Willpower

2\. Table of modal verbs versus meanings

Under the assumption that the above list is definitive, a vector that
contained all of these attributes would therefore capture all the nuances of
every modal verb in English by inclusion or exclusion of selected modal
concepts within the vector’s components. For example, if the vector’s
components were defined as…

[Ability, Advice, Formal Directive, Formality, Habit, Intention, Likelihood,
Obligation, Offer, Permission, Request, Possibility, Prediction, Suggestion,
Will]

…then the modal verbs of American English and British English would be defined
as in Figure 315.

Figure 315: All the modal verbs of English and their meanings. A cell with
parenthesized contents means this extra implication should exist.

3\. Tumbug implementation

All of the above lists and summary table of modal verbs converted the
ambiguities of modal verbs into discrete cases arranged as mathematical
vectors (= each of the lists) and a matrix (= the table). Tumbug eventually
must be able to visually represent any given modal verb, however, which is
discussed next.

The table in Figure 315 can be regarded as a crossbar switch, where each row
is a specific modal value, and each column is a modal concept. As each
specific modal verb-with-meaning (row) is activated, one or more associated
modal concepts (columns) become activated. If each row is implemented as an
artificial neuron and each column is implemented as an artificial neuron, then
the firing of one row causes the firing of one more columns as dictated by the
connections in crossbar switch. The Figure 316 shows the specific modal verb-
with-meaning ”can (permission)” neuron activating the modal concepts ”Per” (=
Permission) and ”Req” (= Request) neurons. Note that a device other than an
artificial neuron could be used instead, such as a light-emitting diode (LED)
as a signaling device.

Figure 316: Tumbug implementation of modal verb-with-meaning ”can
(permission).”

Similar Figure 317 shows the specific modal verb-with-meaning ”able to
(ability)” neuron activating the modal concept ”Abi” (= Ability) neuron and
the modal concept ”Req” (= Request) neuron.

Figure 317: Tumbug implementation of modal verb-with-meaning ”be able to
(ability).”

When only the column header and its choices are considered, this combination
of slots appears as a pegboard-like pattern, shown in Figure 318. Activation
of a neuron within one of the vertically protruding banks of neurons could be
implemented in various ways, such as: (1) using the vertical, perpendicular
bank of neurons as shown in the figure; (2) omitting the vertical bank of
neurons, and using a neuron firing at a different intensity in the horizontal
bank of neurons, such as firing weakly to represent Informal versus firing
strongly to represent Formal.

Figure 318: The skeletal template for any modal verb, which will represent a
single modal verb when appropriate cells are filled in.

When this shape is enhanced with activated neurons in unique locations, each
activated location shown here in white, a uniquely colored variation of the
pegboard pattern results. Each of these unique visual patterns can be
considered a visual representation of a different modal verb.

Although such a visual pattern might seem to bear no resemblance to any image
a human might associate with a given modal verb, this does not matter. The
reason it does not matter is that the brain routinely uses maps that do not
intuitively correspond to any map in the real world. For example, a spatial
map exists across the somatosensory cortex that represents its person’s body,
roughly in the same shape as the body in one region (see Figure 319), and does
(or at least could) contain a representation every part of the body that
contains sensors, whether internal or external. Via this map, stimulation of a
given set of neurons is inherently understood by the brain to represent
signals from that corresponding part of the body, not understood as signals
from neurons themselves. In fact, the phenomenon of ”phantom limbs” has been
hypothesized by Dr. Ronald Melzack to be caused by a ”neuromatrix” that is an
intermediate biological neural network that sometimes errs in the mapped
locations that it forwards to the brain (Melzack and Wall 1965). There is no
reason to believe that collections of neurons cannot represent nonexistent or
abstract concepts such as modal verb meanings in the same way. Swirly arrays
bridge the gap between smooth, irregular, biological structures, and regular,
grid-like, computer data structures.

Figure 319: Left: A ”body map” of a human as represented by a human’s
somatosensory cortex, with labeled regions. (Source: Course Hero, Inc.) Right:
A Swirly Array whose cells spatially correspond to each labeled region on the
left.

4\. WS150 example: #31 (garden)

”[31] There is a gap in the wall. You can see the garden through it. You can
see the garden through what? POSSIBLE ANSWERS: {the gap, the wall}”

In Figure 320, the entire diagram for ”can (ability)” is used in place of
text, which renders that modal verb-with-meaning a pure image. Similarly, an
eye icon is used in place of the word ”vision” after ”sensory modality = ”.

Figure 320: [31] Tumbug for ”You can see the garden through it.”

Figure 321 shows which of the modal concepts use multiple choices for their
attribute values, and shows additional attributes that could optionally be
used in a choice-like or spectrum-like manner.

Figure 321: All modal concepts currently covered by Tumbug, with emphasis on
those that may involve a spectrum of values.

####  15.5.5 Propositional attitudes

1\. Lists

A propositional attitude is a mental state held by an agent about a
proposition. The presence of a propositional attitude in a sentence is
signaled by a verb of attitude followed immediately by a clause that begins
with ”that,” a clause that the verb governs. Some examples from WS150 are:
”[41] I’m sure that my map will show this building…” has the verb ”to be sure
that” that signals belief. ”[133] Dr. Adams informed Kate that she had
cancer…” has the verb ”to inform that” that signals knowledge. ”[109] Bill
thinks that calling attention to himself was rude to Bert…” has the verb ”to
think that…” that signals belief.

The propositional attitudes listed by Davis are (Davis 1990, ch. 8):

  1. 1.

belief/believe/believes

  2. 2.

knowledge

  3. 3.

knowing whether and what

  4. 4.

perception/perceive/perceives

Stuart J. Russell and Peter Norvig list some of these and more (Russell and
Norvig 2010, p. 450):

  1. 1.

belief/believe/believes

  2. 2.

intends

  3. 3.

informs

  4. 4.

knows

  5. 5.

want/wants

Various online sources list these:

  1. 1.

assert

  2. 2.

belief/believe/believes

  3. 3.

command

  4. 4.

consider

  5. 5.

deny

  6. 6.

desire

  7. 7.

doubt

  8. 8.

fear

  9. 9.

hope

  10. 10.

imagine

  11. 11.

intend

  12. 12.

judge

  13. 13.

know

  14. 14.

perception/perceive/perceives

  15. 15.

want/wants

  16. 16.

wish

2\. Tumbug implementation

Iconic representation of propositional attitudes is difficult because of their
artificial nature and because of their quantity. Unlike emotions and modal
verbs, each of which constitute a relatively small quantity of basic
categories that are often grouped by similarity, there exist at least dozens
of propositional attitudes of extremely varied meanings. As a result, a
slightly different way of representing propositional attitudes should be used
for Tumbug. This document does not recommend any specific solution in the form
of a master icon that shows all propositional attitudes (similar to the
Robinson Icon), though in general it is recommended that the methods of
organization used earlier in this document for emotions and modal verbs be
used, namely (1) generalization of meaning, (2) alphabetization, and (3)
dedication of one neuron per concept. For example, the following categories
appear promising, and cover all the aforementioned listed propositional
attitudes:

  * •

emotional motivation: fear, hope

  * •

general motivation: desire, intend, want, wish

  * •

cognitive: believe, consider, deny, doubt, imagine, judge, know, perceive

  * •

communication: assert, inform

  * •

grammatical: command

Pro tem, in this document propositional attitudes are represented as text,
namely as Label Strings, placed at the top of a C Aggregation Box to avoid the
need to label a typically short, tilted line stemming from the agent. Tumbug
uses a C Aggregation Box to hold a proposition, and the C Aggregation Box is
given an attribute-value pair in text that contains the propositional
attitude, such as ”belief = true” or ”desirability = false.” Note that
propositional attitudes need not necessarily have discrete true or false
values. For example, ”believe” and ”desire” can span a wide range of
applicability strength, and ”imagine” and ”perceive” are not binary at all.

Note that proposition attitudes are typically based on a multitude of incoming
data from multiple senses, therefore the specific sensory modalities involved
are hidden as an irrelevant detail, and similarly Motion Arrows representing
the flow of objects (such as soup, for taste) or data (such as light, for
vision) are omitted when propositional attitudes are used. Instead, single
Attribute Line connects a C Aggregation Box that summarizes the impression,
perception, belief, knowledge, etc. that the subject has about what is
happening inside the C Aggregation Box.

Also note that a propositional attitude always involves an observer who is
interpreting the situation depicted in the C Aggregation Box. This is
analogous to a cathected emotion, as in the Robinson Icon, meaning that the
observer is investing energy in an object, idea, or person, which in the case
of propositional attitudes is a situation. Therefore a propositional attitude
situation diagrammed in Tumbug must always contain: (1) the observer (Object
Circle, though often unlabeled), (2) the situation (C Aggregation Box), (3)
the attitude (label over the C Aggregation Box), (4) the link (Attribute Line)
between the observer and the situation.

Figure 322 and Figure 323 demonstrate ”belief” and ”desire,” two of the most
common propositional attitudes in the English language. Memory is shown as a
Data C Object Circle since memory is a virtual phenomenon. (If desired, memory
could be shown housed inside of a skull that is represented by a Physical C
Object Circle.) Memory is shown attached to the human icon with an Attribute
Line, since memory is an attribute of humans. (If desired, a zoom ability
could be used to position this memory inside the head of the human icon, where
the skull is physically located, but the resulting diagram would be difficult
to understand at a glance because of the very small sub-diagram.) Propositions
are also virtual phenomena, so those are also shown as Data C Object Circles.
Two causations are shown via Causation Arrows, one caused by a person’s memory
automatically storing a new proposition, another caused by a person’s mental
models automatically assigning a belief level to the newly stored proposition.
Note that nearly the same situation occurs in the human memory (”memory with
mental models and tastes”), regardless of which of these propositional
attitudes is involved.

Figure 322: Belief: New proposition automatically enters memory, and is
automatically assigned a belief level. Figure 323: Desire: New proposition
automatically enters memory, and is automatically assigned a desirability
level.

3\. WS150 example: #87 (dishwasher)

Figure 324 shows Tumbug representation of a part of WS150 question #87, which
involves the propositional attitude ”want.” Note that the result resembles the
method used to depict a personal fantasy that is used in Hollywood films,
where the person doing the visualization is visualizing themself at an
approximate time period while doing what they want to do. Often films use a
wavy fluctuation of the fantasy enactment to clue the viewer that the scene is
fantasy, often done as a smaller frame within the main frame, whereas in
Tumbug the fantasy scenario is contained in an Aggregation Box.

Figure 324: [87] ”Fred wanted to watch TV.”

###  15.6 Tumbug as software

Tumbug was intended to be a diagrammatic KRM that can be drawn in 2D, but
Tumbug would greatly benefit if Tumbug were implemented as a software tool.
Some desirable effects of software implementation of Tumbug would be:

  * •

Correlation Boxes could keep attribute values in sync, automatically, in the
background.

  * •

Motion Arrows could be eliminated because each intended moving object would
actually move.

  * •

Timelines could be eliminated because flow of time would already be present
and obvious.

  * •

Every intermediate position or state between two endpoints in time could be
explicitly seen.

  * •

Automatic storage of new experiences into memory could be implemented.

  * •

Automatic assignment of propositional attitudes as attributes in memory could
be implemented.

  * •

Any static zoomed diagrams could be omitted because zooming would be available
on demand.

  * •

Icon libraries could be extensive and readily available, which would enhance
understandability.

  * •

Libraries of scenarios of common actions (e.g., walking, talking, buying)
could save writing time.

  * •

3D displays would be much more practical.

  * •

Visual aids could be automatically placed on the diagram, such as to show
alignment of objects, if desired.

  * •

All diagrams created by a visual editor that are in a document could be easily
updated in parallel, mostly automatically, to reflect a change in one diagram
or in the text describing that diagram.

Tumbug is only a KRM and intended to be only a foundation for subsequent AGI
architectures, but a software version of Tumbug that converted textual
sentences into spatiotemporal icons might already have a few uses, such as the
following:

  * •

A written story could be immediately simulated, especially when writing a
movie script.

  * •

Contradictions, especially in legal cases, might be detected via conflicting
attribute values.

  * •

An aid to search engines, by allowing objects and actions to be combined to
narrow the search, or to search for a specific situation using general terms
that would ordinarily return too many matches.

  * •

An aid to foreign language translation programs.

  * •

Children learning to read could immediately relate words to visual objects and
visual motions.

##  16 A few deep implications of Tumbug

###  16.1 SCOVA: an extension to OAV triples

It is remarkable that all of Tumbug’s Building Blocks generalize to only five
Basic Building Blocks (BBBs):

  * •

(O) Object-like concepts. (objects and aggregations of objects)

  * •

(A) Attribute-like concepts. (adjectives and adverbs)

  * •

(V) Value-like concepts. (values, ranges of values, and wildcards)

  * •

(C) Change-like concepts. (time, motion, causation, state changes, and
functions).

  * •

(S) System-like concepts. (systems composed of any mixed combination of the
above types)

Since three of these concepts (O, A, V) are already addressed by standard OAV
triples, this suggests that the OAV triple form of knowledge representation
could be augmented if the other two concepts (C, S) of this document were
added. With letters rearranged, this combination of concepts {O, A, V, C, S}
can be spelled ”SCOVA,” which could be the name of a new KRM. The main
difference between SCOVA and OOP would be that whereas change in OOP is often
done by message passing between objects, change in SCOVA would be an
additional component that would reduce message passing and would eliminate
functions (”methods”) defined inside of OOP software objects.

Some of the simplest, most common, most useful, and most meaningful
combinations of these give SCOVA components mentioned in this document are
shown in Figure 325. The names used here attempt to keep the more familiar
abbreviation ”OAV” at the beginning of each longer abbreviation. ”S” is not
often used until Phase 2.

Figure 325: Some commonly mentioned, simple combinations of the SCOVA
components. From left-to-right, starting on the top row, these are named in
this document: OC, AVC, OAV, OAVC.

Side conjecture: Since the number of Basic Building Blocks of Tumbug, namely
SCOVA, are five in number, and since the layers of the neocortex are six in
number (I. molecular layer, II. external granular layer, III. external
pyramidal layer, IV. internal granular layer, V. Internal pyramidal layer, and
VI. multiform layer), it is conceivable that each layer of the neocortex is
assigned one of the five components of SCOVA. If true, this would confirm the
suspicion of Jeff Hawkins that the existence of those six layers is a major
clue to intelligence (Hawkins 2004, p. 51, 69), since the implication would
then be that the neocortex has somehow managed to implement what appear to be
the five naturally arising building blocks of representation.

Side conjecture: Since the number of Basic Building Blocks of Tumbug, namely
SCOVA, are five in number, and since the number of main types of neocortical
neurons are six in number (1. pyramidal cells, 2. fusiform cells, 3. stellate
cells, 4. basket cells, 5. cells of Cajal-Retzius, and 6. cells of
Martinotti), it is conceivable that each layer of the neocortex is assigned
one of the five components of SCOVA.

###  16.2 Mathematical implications

The main mathematical implication found during this study is that mathematics
is a disembodied form of Tumbug.

The Correlation Boxes of Tumbug make clear the relationship between Tumbug and
mathematics: mathematics is concerned only about the numerical values of
attributes of objects, whereas Tumbug puts those numbers into context and
gives them meaning with respect to the entire diagrammed system. This is why a
single number in mathematics has no relationship to the real world unless it
is associated with a variable that is defined, especially if units of
measurement are supplied (e.g., centimeters, joules, years, radians, mean,
etc.). This is also why mathematics is considered abstract: it strips away all
meaning and structure from the real world except for numerical values. This in
turn suggests that mathematics is the wrong tool for implementing AGI, since
AGI must simulate the real world with all of its structures, complexities,
fuzziness, and vagueness. This, in fact, is said to be exactly the reason that
microworld research (Partridge 1991, p. 95) of early AI failed, which included
famous systems such as the SHRDLU program (which operated in the virtual
blocks world), Shakey the robot (which operated inside a special, uncluttered
building), FREDDY the stationary robot (which operated on a steerable
platform), the SAM program (which operated in the virtual world of textual
stories), and the FRUMP program (which operated in the virtual world of wire-
service news reports).

As an example, Figure 132 is the earlier Tumbug representation of a bottle
that poured water into a cup, and Figure 326 shows the Tumbug equivalent of
the same system with everything stripped away except for the mathematical
components.

Figure 326: [24] A Tumbug diagram with a Correlation Box that has everything
except the math stripped away.

Note that any sense of time is removed (i.e., the timeline is removed), any
sense of the meaning of the variables is removed (i.e., the objects and their
inner partitions are removed), and the physical contact between the objects is
removed (i.e., the stream of water is removed). All the mathematical system
knows is the values for the variables, how to transform one variable’s value
into the other variable’s value (via the function f), and maybe the variable
names (which in the computer are ultimately only numerical memory addresses,
not even text, and certainly the computer does not understand how the text
relates to anything in the real world). These observations lend credence to
the various claims that mathematics is the wrong tool for describing the real
world, and therefore the wrong tool for AGI, e.g., (Devlin 1997, p. 283),
(Wolfram 2002, p. 821).

##  17 Insights from Tumbug

###  17.1 CD theory and Parts of Speech

The core of CD theory, described earlier in this document, is Roger Schank’s
insight that oftentimes several similar words refer to the same concept, but
the words differ only because of the Part of Speech for which they are used.
(In this document on this topic, the only Parts of Speech of interest are
noun, adjective, verb, and adverb.) For example, ”quick” is the word form when
used as an adjective, but ”quickly” is the word form when used as an adverb,
”quickness” is the word form when used as a noun, and ”quicken” is the word
form when used as a verb. Obviously the same concept and same root word are
referenced, viz. ”quick,” but the word becomes distorted due only to English
grammatical rules, a situation that is neither logical nor efficient when
representing meaning in a diagrammatic form. (In speech, however, where a
sentence diagram must be converted to a 1D temporal chain for audio
communication, this is very efficient.) Some English words that function this
way in at least three Parts of Speech are listed in the table in Figure 327.

Figure 327: The rules of English often require a concept to change its concept
name slightly, according to which Part of Speech it is adopting at any given
moment.

It is understandable that a natural language would need to modify the root
word to succinctly flag its use in the sentence, otherwise a grammatically
correct sentence such as ”Quick quickness quickly quickened” (see Figure 328
would be rendered as ”Quick quick quick quick.” However, in a diagram that
represents meaning, this is inefficient practice since essentially the same
word appears in quadruplicate. This suggests an interesting insight about
language: objects, actions, and attributes can be described by the same
concept. This means that a single concept, say represented by variable Q, can
appear as a description of any object, action, or attribute, as illustrated in
Figure 329.

Figure 328: Tumbug for ”Quick quickness quickly quickened,” which is
grammatically correct. Parts of Speech rules of English dictate that the
concept’s basic word must change form in many cases, however. Figure 329: In a
very general view, any single concept represented by variable ”Q” can occupy
any slot for any structure or attribute, regardless of Parts of Speech rules
of English, and regardless of whether a word formally exists for every Part of
Speech variation of that concept.

The reverse situation generates another implication: The Part of Speech usage
of concept Q depends only on the type of structure that Q is identifying. See
Figure 330. In particular: (1) If Q is identifying a given C Object Circle
then Q must be rendered as a noun in spoken or written language. (2) If Q is
identifying a given Motion Arrow, then Q must be rendered as a verb in spoken
or written language. (3) If Q is identifying a given Attribute Line then Q
must be rendered as an adjective or adverb, depending only on whether the
Attribute Line is attached to an C Object Circle or Motion Arrow,
respectively. Otherwise, Q is one of the unlabeled core concepts of which
Schank described as the foundation of CD theory.

Figure 330: The Part of Speech associated with core concept Q is dependent
only on the structure at which is appears.

There exists at least one caveat to the above observation, however: Since
Motion Arrows are 2D rather than 1D, it must be specified whether the action
described by a Motion Arrow is moving toward versus away from the state Q
specified. In English the de facto convention is clear: Motion toward state Q
is always implied, never in the reverse direction. For example, the verb ”to
embolden” means to become bold, not to stray from being bold. Similarly, the
verb ”to beautify” means to make something beautiful, not to cause something
to be no longer beautiful.

##  18 Pros and cons of Tumbug

###  18.1 The main strengths of Tumbug

  * •

Tumbug is likely the first purely visual representation ever used in computer
science.

  * •

Tumbug initiates a totally new, unexplored category of WS and CSR approaches.

  * •

Tumbug clearly distinguishes between active tense and passive tense.

  * •

Tumbug clearly distinguishes between simple aspect and progressive aspect.

  * •

Tumbug clearly distinguishes between prefect and perfect-progressive aspect.

  * •

Tumbug has better representational power than CD theory.

  * •

Tumbug appears to implement the ”universal language representation” needed in
natural language processing (NLP).

  * •

Tumbug closely approaches the ability of a visual simulator, therefore
probably also closely approaches the way that humans think about events.

###  18.2 Possible weaknesses of Tumbug

  1. 1.

It is not clear in a Tumbug diagram with a timeline which phases of the event
should be documented, especially in how much detail. For example, an eating
(INGEST) action for a human usually involves details such as chewing and
swallowing, but these details are probably not important enough to document.
As another example, reaching the top of the stair for a human usually involves
traversing each step along the way, which is likely an irrelevant detail for
most purposes.

  2. 2.

The functions currently allowed in the Correlation Boxes are not required to
be invertible functions. For example, the allowed function y = $x^{2}$ over
the real numbers is not invertible because then x = $\pm\sqrt{y}$. This
allowance is currently assumed to be necessary since general intelligence
would need to be able to access the total picture of how a function (or
system) behaves, and would presumably be able to compensate for the many-to-
one maps, or to decide on its own which values to accept from multiple
possibilities. However, some drawbacks of allowing arbitrary functions are:
(1) increased complexity of the mapping, (2) lack of clear-cut direction of
correlation (viz., positive or negative), (3) possible inability of the system
to make rapid, accurate guesses about the type of correlation involved between
newly encountered variables because of the extra caution the system must
exercise in order to remain general, (4) possible inability of the system to
fill in missing functions by unsupervised examination of its own newly stored
knowledge.

  3. 3.

There can probably never be a canonical form of every sentence that is to be
rendered in Tumbug. Even if the shortcut notations, such as for motion or
neglect of a C Aggregation Box, are ignored, there still may not be a clear-
cut standard form for some reason. There may be, but this is not clear. A
convincing argument that canonical forms are difficult for any KRM to produce
is given by William A. Woods (Woods 1975, pp. 16-21).

  4. 4.

Several problematic situations occur when attempting to represent situations
spatially. For example, how would one diagram ”a small cluster of points in
time” along a timeline without showing an exact number of points in the
representation, and without showing their exact temporal order? Or how would
one diagram a list of unsorted numbers in a generic way, without showing a
specific number of numbers, or any of their values? How would one show in a
generic way that a given set of jigsaw puzzle pieces needed to be placed
together tightly but without forcing, and without showing the number of puzzle
pieces or their shapes? The author has solved a number of such
representational problems, therefore Tumbug can represent at least some such
complications, but that topic is out of scope for this document. More
generally, such problems qualify as representation of mathematical concepts,
which has been barely addressed in this study.

  5. 5.

Some Tumbug icons are difficult to generalize easily from 1D to 2D. For
example, a 1D sound broadcasting line is difficult to draw as fanning out into
2D space from a point. Similarly, a moving appendage such as an arm is
difficult to describe with straight arrows, since most likely the appendage
will swing on joints, which implies usage of arrows swinging through an angle.
Most such problems could be resolved sufficiently with a software
implementation of Tumbug without too much difficulty, but such software does
not exist yet.

  6. 6.

The differences and overlap of action verbs and stative verbs is not
completely clear. For example, the verb ”to eat” would probably be most
accurately classified as an action verb since eating involves food moving to
the mouth, the motions of chewing and swallowing, and the food moving to the
stomach. However, from a wider perspective the detailed motions of eating are
not very important. For example, some animals such as starfish eat by egesting
their stomachs around their victims instead of using their mouths to hold and
chew food, so overall the act of eating is to change the state of the eater
and food so that the food becomes located inside the eater for nutrition
purposes. Also, a phrase such as ”the photocopier ate my document” is
immediately understandable, despite not involving any living organism, mouth,
or stomach. The author’s prediction is that Marvin Minsky may have been
correct: it is very possible the brain uses several types of KRMs, including
motion diagrams, state diagrams, and verbatim images, and that the brain links
these representations in a coordinated way through learning.

###  18.3 Programs writing programs

A KRM like Tumbug is probably the most promising approach to provide a future
means for machines to write computer programs or self-modifying code because
Tumbug’s ability to store spatiotemporal images verbatim reduces information
lost in the transition from the real world to the virtual world. Currently,
the real world consists of spatiotemporal images that need to be converted to
a numerical-textual format that the virtual world of digital computers can
store and manipulate easily, which typically requires time to be discretized,
images to be discretized, details to be omitted that initially seemed
irrelevant to the programmer (but that may be deemed important later), shape
information to be lost, and cause-effect information (that only a human would
understand well) to be lost, as shown in Figure 331. Currently, computers
cannot truly understand the real world, even when outfitted with video
cameras, radar, and lidar. Even with multisensory input, computers cannot make
commonsense conclusions about occluded objects or about interactions between
physical objects. Computers also suffer additional problems from multisensory
integration.

Figure 331: Programs struggle to write other programs partly due to their lost
spatiotemporal information. As a result, programs need a programmer to cause a
program to exist in the virtual world, based on the programmer’s ability to
understand the real world well enough to abstract it and to represent that
abstraction.

In contrast, a machine that could store information in the same spatiotemporal
organization as the real world would automatically eliminate the programmer,
at least as a data type translator, as shown in Figure 332. That would free
the programmer to focus on writing behavioral programs instead of also needing
to convert between data types. The language of the real world obviously
consists of spatiotemporal images across various sensory modalities, not
abstractions like numbers, text, and algorithms. An immediate consequence of
having such an internal representation that matched the representation of the
real world is that presumably a program could then write other programs as
shown in Figure 333, which could be considered the ultimate goal of computer
programming.

Figure 332: Programs whose representation matched that of the real world would
not need a programmer. Figure 333: A virtual world that matched the real world
could host a program to write other programs.

##  19 Discussion of Tumbug

The two main categories of approaches to ANI are symbolic AI and subsymbolic
AI. Symbolic methods refer to text and are typically implemented with rule-
based expert systems, logic programming, and semantic nets. Subsymbolic
methods refer to non-text are usually implemented with neural networks. Tumbug
is closer to a symbolic approach, but uses icons instead of symbols. Since
Tumbug’s icons have great freedom of placement, unlike text, Tumbug is more
flexible than text, and therefore more powerful than text for representation.
Thus Tumbug represents a new category of approach that has not been pursued,
which in itself seems promising.

Was the design of Tumbug intended to imply that the brain is literally moving
around circular icons? No. The circular icons are merely the simplest
pictorial concept of a generic ”thing.” In practice all that would be needed
to implement a circular icon would be a fuzzy cluster of one to several
neurons whose summarized attributes include coordinates of the thing they
represent, combined with a possibly crude method of distinguishing one
location (or state) from another. In this sense, Tumbug circular icons could
be considered part of a fuzzy system, as in fuzzy logic, where the exact size,
shape, boundaries, and curvature of the icons are irrelevant.

Tumbug can be considered a fuzzy system, but the fuzziness is created through
simpler diagrams, not membership functions. In a sense, $\langle$percentage of
fuzziness$\rangle$ = 1 - $\langle$percentage of detail present in a
diagram$\rangle$).

Is not the design of Tumbug’s representation of modal verbs and emotions
spatially arbitrary? Yes, but so are biological systems. The same patch of
skin with pressure sensors can be placed anywhere on the surface of the body,
but in one position it could represent pressure on a foot whereas in another
position it could represent pressure on the neck. In biological systems the
only difference in meaning is based on that patch’s location on the body map
that is stored in the cortex. In a computer, labels substitute for locations
in such a body map. For virtual concepts such as modal verbs, such a design
suggests that humans can physically ”feel” the difference between two
different modal verb meanings using the same hardware that humans use to feel
the difference between two physical body locations.

The late Marvin Minsky may or may not have agreed with the approach being used
by Tumbug. Although Minsky strongly advocated emphasis on KRMs as the
foundation of AGI, his recommended approach was to combine existing KRMs and
to focus on the management system that would coordinate those KRMs (GBH
Archives 1990), whereas the Tumbug approach has so far been to find a single,
universal KRM that could represent every possible concept, and to eschew the
unnatural, nonvisual KRMs that have been developed so far for use only in
digital computers. In Pei Wang’s terminology, Minsky’s approach may have
tended toward the hybrid approach whereas the Tumbug approach definitely takes
the unified approach. However, Minsky’s opinion still retains its merit: note
that Tumbug has naturally gravitated toward appending specific computer
science KRMs such as SCOVA + state diagram as in Figure 203, and state diagram
+ image as in Figure 138. If such lower level KRMs are what Minsky envisioned,
then Tumbug is already tending toward Minsky’s envisioned ”missing link” of
AGI.

Conjecture: Humans have a part of the brain that allows humans to feel virtual
concepts in the same way that a part of the brain feels physical sensations.
This conjecture fits Jeff Hawkins’ mentioned observation that the same
cortical structure is used throughout the brain, regardless of sensory
modality. Tumbug suggests that this observation can be generalized to state
that the same cortical structure might be used for virtual concepts as is used
for sensory modality, i.e., that understanding a virtual concept is
architecturally equivalent to understanding a sensory modality.

##  20 Summary

The main premise of this project is that if the proper KRM is used then the
most difficult parts of AGI system design are automatically and greatly
simplified. The proposed KRM for that purpose is a visual, iconic system
called Tumbug that was designed by the author. Tumbug is the research topic of
this Phase 1, which is part of a larger research project that aims at
producing full AGI. The full envisioned research project has five phases that
are collectively called The Visualizer Project. Older but unpublished results
from initial excursions into Phase 2 and Phase 3 algorithms appear to confirm
the premise that Tumbug is a promising foundation for AGI. Tumbug consists of
about 30 Building Blocks (icons) that generalize to only five Basic Building
Blocks, which turn out to be essentially the three components of OAV triples,
namely Object (O), Attribute (A), and Value (V), plus two additional
components called Change (C) and System (S). Change is a general concept that
includes concepts such as time, motion, force, and cause-and-effect, and
System is any legally assembled combination of Tumbug’s Building Blocks.

By using a single pictorial KRM that consists of only about 30 Building
Blocks, Tumbug can represent an extremely wide span of topics in mathematics,
logic, algorithms, physics, human motivations, and human activities. Tumbug is
a radically different type of KRM in that Tumbug can be made fully visual
without use of text or numbers, which sets Tumbug apart from all other AI
approaches such as expert systems, neural networks (whose units require
labeling with text), cellular automata (whose units require labeling with
text), and any variations of these. In short, Tumbug can probably visually
represent any topic or concept that the average person is likely to encounter.
This unified representation method could allow various computer systems,
whether based on natural language, data base records, numbers, computer code,
knowledge graphs, or images, to be integrated without the need for conversion
at their interfaces. Several examples taken from sentences of the Winograd
Schema are shown. For additional practical applications, Tumbug seems to be
particularly well-suited for natural language translation, and possibly also
for universal computer code representation in the style of .NET.

Tumbug appears to be an improvement on the Conceptual Dependency KRM of Roger
Shank in that Tumbug’s Building Blocks are far more universal and general,
such as objects, motions, and attributes rather than detailed and largely
human-based actions such as ingest, expel, and grasp. Tumbug could be the
basis of ”The Language of Thought” that Jerry Fodor conjectured in 1975, a
hypothesized language often called ”Mentalese.”

##  21 Future Research Plan

###  21.1 The Visualizer Project

A ”visualizer” is defined here as a type of processor whose native KRM is
imageoids, in contrast to a ”computer,” whose native KRM is numbers. Therefore
computers and visualizers are different processing architectures that each
have their own unique niche of problems that they can solve particularly
competently.

This article completes and documents Phase 1 of a 5-phase research project
called the Visualizer Project that aims to produce AGI in the form of a design
for a visualizer. Tumbug is the foundation of all the phases that are expected
to follow. The author believes that the design of a bona fide AGI system will
be produced by the end of the fifth phase, though probably not yet coded at
that time. Figure 334 shows all the anticipated phases of the Visualizer
Project.

Figure 334: The research plan. This document completes Phase 1. The developed
KRM from this phase (viz., Tumbug) will be used in all expected future phases
of this research.

One inherent limitation of Phase 1 is it is difficult to state when a
collection of Building Blocks is ”complete” because there is no theoretical
limit to the number of Tumbug Building Blocks, only a practical limit, usually
based on the frequency of usage within a given domain, and also there has not
been a need in Phase 1 to fully explore some recently introduced Building
Blocks. A few Tumbug Building Blocks were introduced in this document largely
because they are expected to be needed sometime in future, especially for
spatial reasoning (especially Correlation Boxes and 1D Markers), for reducing
textual descriptions from Tumbug (especially Value Bars), for coding
implementation (especially 0D Markers and Zoom Boxes), for a learning
algorithm (especially Data Set Boxes), or for mathematical theorems
(especially C-A Aggregation Boxes), but so far there has not been sufficient
need to justify the time required to thoroughly explore those constructs.

It is expected that the need for several new Building Blocks will arise from