# Logical Information Cells I

Jean-Claude Belfiore  Huawei Wireless Technology Lab. Paris Research Center
Daniel Bennequin  Huawei Wireless Technology Lab. Paris Research Center
University of Paris Diderot, Faculty of Mathematics  Xavier Giraud  Huawei
Wireless Technology Lab. Paris Research Center

###### Abstract

In this study we explore the spontaneous apparition of visible intelligible
reasoning in simple artificial networks, and we connect this experimental
observation with a notion of semantic information.
We start with the reproduction of a DNN model of natural neurons in monkeys,
studied by Neromyliotis and Moschovakis in 2017 and 2018, to explaini how
”motor equivalent neurons”, coding only for the action of pointing, are
supplemented by other neurons for specifying the actor of the action, the eye
$\displaystyle E$, the hand $\displaystyle H$, or the eye and the hand
together $\displaystyle EH$. There appear inner neurons performing a logical
work, making intermediary proposition, for instance $\displaystyle E\vee EH$.
Then, we remarked that adding a second hidden layer and choosing a symmetric
metric for learning, the activities of the neurons become almost quantized and
more informative. Using the work of Carnap and Bar-Hillel 1952, we define a
measure of the _logical value_ for collections of such cells. The logical
score growths with the depth of the layer, i.e. the information on the output
decision increases, which confirms a kind of bottleneck principle.
Then we study a bit more complex tasks, _a priori_ involving predicate logic,
using the operators $\displaystyle\forall$ and $\displaystyle\exists$. In
these experiments, bars of different lengths and colors ($\displaystyle
R$,$\displaystyle G$,$\displaystyle B$,…) are presented and the network has to
decide if some pairs are disjoint $\displaystyle D$, intersect only
$\displaystyle IO$ or are related by inclusions $\displaystyle II$. Also for
this task, even for two bars, the logical value increases considerably with
the total depth: three hidden layers being sufficient and necessary.
Amazingly, with less layers the network performs well, but it uses other
strategies, like Fourier analysis; then a bifurcation occurs with more hidden
layers.
Also amazing is the fact that the logical population takes almost no account
of the statistics in the data; for instance $\displaystyle D$ and
$\displaystyle IO$ are the most frequent inputs, but most of the neurons
eliminate $\displaystyle D$ or $\displaystyle II$.
With a richer learning, for instance varying the lengths of two bars, and
exchanging the long and the small ones, the network develops a richer and more
inventive logic; for instance it becomes able to treat directly with
$\displaystyle IO$.
We compare the logic and the measured weights. This shows, for groups of
neurons, a neat correlation between the logical score and the size of the
weights. It exhibits a form of sparsity between the layers. The most
spectacular result concerns the triples which can conclude for all conditions:
when applying their weight matrices to their logical matrix, we recover the
classification. This shows that weights precisely perform the proofs.

###### Contents

  1. 1 Towards Reasoning Networks  (Réseaux Raisonnants)
    1. 1.1 Generalities
    2. 1.2 The experiments
      1. 1.2.1 Motor equivalence. Propositional Logic.
      2. 1.2.2 Topology of colored segments. Predicate Logic.
      3. 1.2.3 Measuring Logics and Semantics
      4. 1.2.4 Neural network description
  2. 2 The simplest model. Propositional calculus.
    1. 2.1 Introduction
    2. 2.2 Experimental settings
      1. 2.2.1 Input layer description
      2. 2.2.2 The network
      3. 2.2.3 The output layer and the loss function
      4. 2.2.4 Displaying the activity of a neuron
    3. 2.3 Theoretical deduction of the movement from the first layer
    4. 2.4 Characteristics of the hidden layer
    5. 2.5 More and more layers
    6. 2.6 Logical values
  3. 3 Simple networks for doing predicate calculus
    1. 3.1 A first experiment in two forms
      1. 3.1.1 The experimental setting
      2. 3.1.2 Theory, predicative cells, conclusive or not
      3. 3.1.3 Tests of generalization
    2. 3.2 A second experiment with a richer learning
    3. 3.3 A third experiment, the blue object incoming
    4. 3.4 The important role of symmetries
    5. 3.5 The important role of redundancy
  4. 4 How do weights perform logic and deduction?
  5. 5 Conclusion and discussion: Questions and Perspectives

##  1 Towards Reasoning Networks
(Réseaux Raisonnants)

In this study we explore the spontaneous apparition of reasoning in simple
artificial networks, and we interpret this experimental observation in terms
of semantic information.

We prefer to write Réseaux raisonnants than Réseaux pensants, which had
resonated better with the Roseau pensant of Blaise Pascal (”L’homme n’est
qu’un roseau, le plus faible de la nature; mais c’est un roseau pensant”, in
his Pensées), because human thinking cannot be summarized by logical grouping
and deduction.

###  1.1 Generalities

The general hypothesis is as follows: when confronted to a supervised (or
reinforced) classification task, or to the determination of structures in the
data which can be formulated in logical terms, a simple deep neural network
($\displaystyle DNN$) develops by itself a coherent way of reasoning, which
proceeds by decomposing the decision into elementary steps, and recomposing
them. This is the opposite of the current hypothesis made today, that DNNs
have difficulties with compositional structures, privileging global
correlations. Now our hypothesis must be made more precise, and limited. In
particular, we confirm here the known fact, that, in order to work efficiently
and reasonably, the network requires specially good metrics (loss functions)
for the output, convenient non-linearities and a sufficiently large number of
layers, as soon as the task becomes more complex.
The kind of reasoning which appears here experimentally, shares several
essential properties with human reasoning, inference and deduction; in
particular it relies on attentive preferences for certain characteristics of
the data, and a kind of discretization of the message that are related to
them, as words do, for instance. The cumulative experience here is short in
time and uses a large set of data, nothing is done by long time evolution,
natural wisdom or preliminary memories. However, as we will report, remarkable
inventions are made by the network itself. It makes us expect that introducing
other aspects, like _a priori_ knowledge, or long and short term memories,
will considerably increase the capacity of discrimination, imagination and
reasoning of these networks.

In a theoretical companion paper, _Topos and Stacks of Deep Neural Networks_
[BB21], we have presented a notion of _semantic functioning_ of DNNs, based on
topos theory, stacks geometry and logics. This notion generalizes what we use
in the present experimental study. These two studies were conducted in
parallel, in order to confront top down principles of Topological Information
Theory with a bottom up view of the activity of artificial neurons.

In another companion study, _Logico-probalisitic Information_ , Belfiore,
Bennequin and Giraud [BBG20], to appear soon, we study the relation between
the logical information which appears here and the Shannon probabilistic
information, applied to subjective probabilities in Bayesian networks of
(logical) variables. Probabilities are certainly necessary for more complex
tasks, for the acquisition of uncertain knowledge, and their flexible use.
Even in the present study of very simple networks, Bayesian rules had
interest, however, for the essential part, logic only was sufficient. This
probabilistic study connects the present work with the today current
interpretation of semantic information, as it is defined and used for instance
by Bao et al. 2011 [BBD+11], 2014 [BBDH14] and Xie et al. 2020 [XQLJ20], for
semantic communication, generalizing Carnap and Bar-Hillel 1952 [CBH52]. For
other related directions, we can cite Barwise and Seligman 1997 [BS97],
Floridi 2004 [Flo04], D’Alfonso 2011 [D’A11].

###  1.2 The experiments

In this contribution, we consider two sets of experiments with Deep Neural
Networks (DNNs). One set to highlight the capacity of the network of doing
propositional calculus and the second one to highlight the ability of the
network of doing predicate calculus.

####  1.2.1 Motor equivalence. Propositional Logic.

In the first experiment an action is registered as an entry, which is made by
several possible actors $\displaystyle a,b,c$ in $\displaystyle A$ and the
network must decide whom was the actor. In the first layer, each neuron reacts
to a given action according to three functions
$\displaystyle\varphi_{a},\varphi_{b},\varphi_{c}$ of the physical parameters
of the action, say $\displaystyle s\in S$ (concretely an angle). First, only
one hidden layer was introduced; the $\displaystyle DNN$ learned and
discriminated. To make apparent a logical functioning here, we examined the
activity of the neurons of the hidden layer as a function of the actor and of
the parameter. We observed that some cells, for some of the actors, develop,
by learning, an almost saturated answer at a common value, which then, does
not depend anymore on $\displaystyle s\in S$. This allows to exclude these
actors when the activity departs from this value. This can be seen as a
preliminary knowledge about the future decision, which can be deduced
logically. This experiment reproduces the results of a network imagined by two
neurophysiologists, E.Neromyliotis and A.K.Moschovakis, in $\displaystyle
2017$ and $\displaystyle 2018$ [NM17, NM18] to study the ambiguity of motor
equivalent cells, in the spirit of the famous mirror cells, coding for an
action independently of the fact that it is you or someone else which achieves
this action.
To explore the logical potential of the network, we added another hidden
layer. And in this case we saw an impressive diversity of logical analysis,
made by cells which saturated at different values; many cells made Boolean
statements in $\displaystyle a,b,c$ for different intervals of spiking
activity. However, in general no individual cell contained sufficient
information to conclude alone, but collectively were able to conclude in a
logical manner.
The fact that the answer logically deducible from the cells, is correct or
not, is also important, and we always verified if this was the case or not;
remarkably this was always the case. Another important aspect is the capacity
of generalization of the network; we verified that it was very good and
induced a nice adaptation of the logical cells.
More details are given in the corresponding section below.

In order to understand this experiment and more general situations, we have
introduced a (preliminary) notion of logical information.

We are working with a collection of input data $\displaystyle\Xi$, and a
neural network $\displaystyle X_{w}$ which has learned its weights
$\displaystyle w\in W$ in order to answer a well posed final question about
data (which corresponds, in our example, to a classification). We look at all
sets of activities in a given inner layer $\displaystyle L_{k}$, conditioned
by some intermediate statements $\displaystyle P$ about the data (named
questions), the goal being the final classification. A proposition
$\displaystyle P$ corresponds to a sub-collection $\displaystyle\Xi_{P}$ of
$\displaystyle\Xi$. The corresponding sets of activities constitute a
receptive field. And we suppose that some neurons develop quantized activities
$\displaystyle\varepsilon$ for some propositions. We introduce then the set
$\displaystyle D_{k}$ made of those $\displaystyle\varepsilon$’s and their
complements. A definition of the semantic information in $\displaystyle
L_{k}$, with respect to the final questions at the output layer, is the
collection $\displaystyle I_{k}^{\varepsilon},\varepsilon\in D_{k}$ of the
propositions that can be decided (i.e. proved or disproved) from each
collection of activations states $\displaystyle\varepsilon\in D_{k}$.
Therefore this information depends both on the known individual receptive
fields, attached to semantics propositions, and on specific sets of
activations in the whole layer. It is important to remind that we are forced,
by the experimental setting, to consider sets of sets of propositions like
$\displaystyle\varepsilon_{a}\models P_{1},...,P_{m}$ for $\displaystyle a\in
L_{k}$. The question becomes: can we deduce logically the final questions, the
classification, by considering only $\displaystyle L_{k}$, knowing the
statistics of some responses, and using the sets $\displaystyle
I_{k}^{\varepsilon(\xi)}$, associated to a possible input data
$\displaystyle\xi\in\Xi$? And if not, which part of the classification can be
decided? Then we interpret the collection of sets $\displaystyle
I^{\varepsilon}_{k},\varepsilon\in D_{k}$, as a model in $\displaystyle L_{k}$
of the global problem posed to the network. In other terms, each collection
$\displaystyle I^{\varepsilon}_{k}$, for a given $\displaystyle\varepsilon\in
D_{k}$, is viewed as axioms for a theory, and we ask if the final questions
are decidable or not in this theory [LK16].

######  Remark.

Nothing, a priori, forbids to collect information from several layers and
compare them, asking what a layer knows about another one. This leads to a
notion of shared information in the network111We interpret these shared
information in terms of categories, functors and natural transformations in a
Grothendieck topos.

######  Remark.

The semantic information in a layer does not purely describe objective
operations of a network which has learned because the necessary saturation,
for having logical cells, depends on three almost independent factors:

  1. 1.

the data collection $\displaystyle\Xi$,

  2. 2.

the network $\displaystyle X$ and the metric used for learning,

  3. 3.

our own choice of the set of intermediate semantic propositions
$\displaystyle\\{P\\}$, for generating saturation over the corresponding
subsets $\displaystyle\Xi_{P}$ 222Theoretically it would be possible to cancel
point $\displaystyle 3$, by considering all propositions, but practically the
number of choices is too large for that. As it happens in Physics, the result
of the experiment depends on the theory and on the experimental design, in
particular, what is measured. We will present, in appendix, the exact
parameters we have used in the experiments..

####  1.2.2 Topology of colored segments. Predicate Logic.

With the second example we tested the ability of a simple DNN to manage
predicates logics. This example is inspired by image analysis or speech
analysis, but it is also extremely simple, considering one dimensional images
of two or three colored segments, or the superposition of two of three voices
in time, and asking if they intersect or not, and if one is included in
another, or superpose with it. The main interest was the passage from usual
_propositional calculus_ to _predicate calculus_ , involving quantifiers,
existential and universal. Here also everything worked well, at the condition
of increasing the number of layers to at least three.
One of the amazing inventions of the network that we observed, was the
comparison of the lengths of the objects (respectively the sentences) in the
absence of any questions about these lengths: the network understood by itself
that the inclusion is possible only in one sense, without forgetting the
colors (resp. the timbers). This allowed it to generalize fairly well when the
colors were exchanged.

With one or two hidden layers, the logical behavior was obscure. Very
interestingly, with two hidden layers a kind of Fourier analysis is developed
by the network. But with three hidden layers, we observed a wonderful set of
quantized logical cells. Importantly, these logical cells were only interested
by propositions which are consequences of the output questions, and together
they can answer these questions after two layers. In some sense this tells us
that the propositional calculus coming from the objectives dominates. However,
also importantly, the propositions which are more complex than the others from
the point of view of predicative calculus, posed difficult problem to the
cells, and were accessible to them only indirectly, by complementing the
direct decisions.

Thus we get a kind of dynamics of information from layers to deeper layers.
Then we have a version of semantic bottleneck. In a companion paper [BBG20],
considering the link with probabilistic inference, we discuss the relation of
this experimental discovery with the Bottleneck principle of Tishby, Pereira
and Bialek 2000 [TPB01], Tishby and Zaslavsly 2015 [TZ15].
It is important to say that in the two above experiments, the minimum of
error, around $\displaystyle 1/100$ is achieved with one hidden layer, and
maintained with two and three hidden layers. However the logical functioning
progresses with the number of added hidden layers, showing that the semantic
information increases with the depth of the network, then the minimization
principle induces a maximization of information. It could be that the form of
the back-propagation algorithm, which looks like a belief propagation
algorithm, is responsible of this shift to semantic and logic.

More complex tasks, for instance the complete description of the topology of
three colored segments, provoke the appearance of probabilistic estimations:
the cells behave as Bayesian estimators, the quantization is not so good, but
the collective decisions are good. The understanding of their information
content needs a threshold, but fundamentally the principles are unchanged. See
_Logico-probabilistic Information_ [BBG20].

Note that, during the work which is reported here, we had the impressions of a
new kind of Physics, with biological flavors, in interaction with humans
problems and some aspects of human behaviors.

####  1.2.3 Measuring Logics and Semantics

For understanding these experiments and, we hope, also more general
situations, we have introduced a schematic notion of logical information.
Remind that we are working with a collection of input data $\displaystyle\Xi$,
and a neural network $\displaystyle X_{w}$ which has learned its weights
$\displaystyle w\in W$ in order to answer a well posed final question about
data (which corresponds to a classification). Then we look at the whole sets
of activities in a given inner layer $\displaystyle L_{k}$, conditioned by
some intermediary statements $\displaystyle P$ about the data; we name them
questions, in direction of the final classification. A proposition
$\displaystyle P$ corresponds to a sub-collection $\displaystyle\Xi_{P}$ of
$\displaystyle\Xi$. The corresponding sets of activities constitute a
receptive field. And we suppose that some neurons develop discretized
activities $\displaystyle\varepsilon$ for some propositions. We introduce the
set $\displaystyle D_{k}$ made by these $\displaystyle\varepsilon$ and their
complements. Then, by definition, the semantic information in $\displaystyle
L_{k}$, with respect to the final questions, at the output, is the collection
$\displaystyle I_{k}^{\varepsilon};\varepsilon\in D_{k}$ of the propositions
that can be decided (i.e. proved or disproved) from each collection of
activations states $\displaystyle\varepsilon\in D_{k}$.
Therefore this information depends both on the known individual receptive
fields, attached to semantic propositions, and on specific sets of activations
in the whole layer. It is important to remind that we are forced, by the
experimental setting, to consider sets of sets of propositions like
$\displaystyle\varepsilon_{a}\models P_{1},...,P_{m}$ for $\displaystyle a\in
L_{k}$. The question becomes: can we deduce logically the final questions, the
classification, by considering only $\displaystyle L_{k}$, knowing the
statistics of certain responses, and using the sets $\displaystyle
I_{k}^{\varepsilon(\xi)}$, associated to a possible input data
$\displaystyle\xi\in\Xi$? And if not what part of the classification can be
decided? Then we interpret the collection of sets $\displaystyle
I^{\varepsilon}_{k};\varepsilon\in d_{k}$, as a model in $\displaystyle L_{k}$
of the global problem posed to the network. In other terms, each collection
$\displaystyle I^{\varepsilon}_{k}$, for a given $\displaystyle\varepsilon\in
D_{k}$, is viewed as axioms for a theory, and we ask if the final questions
are decidable or not in this theory. We also propose numerical measures of
logical values.

######  Remark.

In the more theoretical study, _Topos and Stacks of DNNs_ [BB21], we define a
more general notion of semantic information, which allows to compare the
theories expressed by several layers about what happens in a given layer.

######  Remark.

The semantic information in a layer do not describes purely objective
operations of a network which has learned, because the necessary saturation,
for having logical cells, depends on three almost independent factors:

  1. 1)

the collection of data $\displaystyle\Xi$,

  2. 2)

the network $\displaystyle X$ and the metric used for learning,

  3. 3)

our own choice of intermediate semantic propositions $\displaystyle P$, for
generating saturation over the corresponding subsets $\displaystyle\Xi_{P}$.

Theoretically it would be possible to forget point $\displaystyle 3$, by
considering all propositions, but practically, the number of choices is too
large for that. As it happens in Physics, the result of the experiment depends
on the theory and on the experimental design, in particular what is measured.

####  1.2.4 Neural network description

In all experiments described in this text, we have used the fully connected
network represented in figure 1.

Figure 1: The networks under test

We use the following notations:

Object | Notation
---|---
$\displaystyle k$-th layer | $\displaystyle L_{k}$
Number of cells in the $\displaystyle k$-layer | $\displaystyle N_{k}$

Implementations are performed with PYTORCH with the following options:

  * $\displaystyle\looparrowright$

Biases are forced to zero.

  * $\displaystyle\looparrowright$

Non linear activation functions are the same on each layer, namely
$\displaystyle x\mapsto\tanh(ax)$ where $\displaystyle a$ is a positive
constant used to improve discretization and to speed up convergence. The
chosen values have been selected trough simulation.

  * $\displaystyle\looparrowright$

Either Mean Squared Error (MSE) or CrossEntropyLoss (CE) is used as the loss
function

  * $\displaystyle\looparrowright$

Adam optimizer has been selected.

All simulations are run on a computer equipped with an intel core i7-8565U
CPU.

##  2 The simplest model. Propositional calculus.

###  2.1 Introduction

Many neuroscientists have made the observation that the motor system must
necessarily involve cognitive operations, cf. Georgopoulos 2000 [Geo00]. Even
the simplest animals, like the worm c-elegans or the ascidian larva, ciona
intestinalis, possess a repertoire of voluntary actions, and have to select at
the right time the most convenient one, then select in what order they must
execute the sequence of actions, using memory, anticipation and evaluation.
Then it is natural to expect a sort of reasoning in every animals (in fact
every living entity, including plants). The small animals we just mentioned
have brains, containing few hundreds of neurons, interconnected by thousands
of synapses, assembled in areas and organized in moduli, dedicated to several
functions. Cf. Kato et al. 2015 [KKS+15], Ryan et al. 2018 [RLM18].
In higher mammals like primates, the brain is much more complex, but still
organized in areas, moduli and networks of sub-systems, and in many cases, the
individual neurons have personal receptive fields, something of interest in
the world or in the functioning. It is not to say that assemblies are not
important, to the contrary, they are the more important ingredient for every
perception, memory and decision (cf Hebb’s book [Heb49]), however these
assemblies rely on the personalities of the individual cells.
In their two papers [NM17, NM18], E. Neromyliotis and A.K. Moschovakis (N&M)
studied specific neurons in the pre-motor cortex of monkeys (more precisely in
a small region, named arcuate sulcus (AS), and in periarcuate cortex, both
concerned by the movements of the eye and of the fore-limbs. They found two
different sub-populations:

  1. 1)

Meq cells (Movement equivalent), which fire during preparation and execution
of directed movements of the eye and of the arm, without preference for the
conditions eye alone (E), hand alone (H) and both eye and hand together (EH),
but with preference for a goal in space depending of each condition;

  2. 2)

S-cells which manifest a sort of indifference for one or two of the above
conditions but continue to prefer some directions, some of them we will call
Logical Information Cells, as alluded in [NM17], because they announce a
partial choice of condition.

Taking into account anatomy and timing, the authors suggested that Meq
activity precedes S-cells activity, in order to prepare decision and execution
in the primary motor cortex and the spinal chord.
N&M said that all these kinds of cells were already found by Fujii et al. 2002
[FMT02], in other close areas, the supplementary eye field SEF and the
supplementary motor area SMA, specially pre-SMA, the more rostral part of SMA.
This region pre-SMA is a crucial node for our discussion, because it is
involved in most of the abstract cognitive processes happening in the brain.
For instance Houdé et al. 2000 [HZM+00], using functional imaging, have shown
that, when shifting from a more perceptual task to a more deductive logical
task, there is a shift of brain’s activity from a more posterior network
(ventral and dorsal) to a left-prefrontal network, mainly constituted by the
middle frontal gyrus MFG, the Broca’s area, the anterior insula AI (sic) and
the pre-SMA. For the authors this corresponds to a network supporting logical
thinking in general. Further studies have confirmed this view; for instance
Johnston and Leek 2004 [JLA+04], on mental computations, Tremblay and Small
2010 [TS10], on language comprehension tasks, either with words either with
body gestures. However, we must mention the interesting discussion about the
necessary link of pre-SMA with a motor action, see Nachev et al. 2008 [NKH08],
Johnston and Leek 2009 [JL09].
Of course, thinking and even reasoning, is not limited to pure logical
reasoning, for instance the brain conducts probabilistic estimation and
inference, as formalized by Bayes for example (cf Pearl [Pea90], Johnson-Laird
et al. 2015 [JLKG15]), and neuronal networks in the prefrontal cortex well
correspond to this aspect of thinking (Koechlin et al. 2003 [KOK03]). A large
network, named the Default Mode Network (DMN), which corresponds to the highly
complex activity at rest, is also known to support spontaneous thinking; it
involves several cortical areas, in particular PFC, and sub-cortical regions,
like the basal ganglia BG, the thalamus T, the region around the Hippocampus,
and also the Amygdala, known for its expression of all the emotions. The
medial temporal cortex MTL is involved in most of the Long Term Memory
operations, in particular episodic and semantic memories, and in MTL the
perirhinal cortex PRC is specially concerned by concepts formations and the
understanding of their meaning. Thus the brain uses a network of many networks
for reasoning and performing semantic operations. However, the pre-SMA and its
cells surely have a wider role than preparing saccades or reaching with the
arms, in reasoning in general, even if it is hard to separate from some
movement operations. This is a good reason for starting with these cells.

###  2.2 Experimental settings

####  2.2.1 Input layer description

The input layer emulate MEQ neuronal responses. It is inspired by biological
data though we do not aim at replicating the true biological situation. We
have built a layer that is likely to produce meaningful results. The input
layer is a set of $\displaystyle N_{0}$ cells corresponding to the MEQ
neurons. Given an activator $\displaystyle a\in\mathcal{A}_{3}=\\{E,H,EH\\}$
corresponding to $\displaystyle\mathsf{Eye}$, $\displaystyle\mathsf{Hand}$,
$\displaystyle\mathsf{Eye+Hand}$, the neuron $\displaystyle x$ in the input
layer gives rise to an activation signal
$\displaystyle\varphi^{x}_{a}(\theta)\in\mathbb{R}$ defined as

$\displaystyle\varphi^{x}_{a}(\theta):=\frac{\exp\left(\kappa(x,a)\cos(\theta-\mu(x,a))\right)}{2\pi
I_{0}(\kappa(x,a))}$

$\displaystyle a$ represent the root cause of the signal,
$\displaystyle\mu(x,a)$ is the preferred angle of the neuron $\displaystyle x$
for a given $\displaystyle a\in A_{3}=\\{E,H,EH\\}$ and
$\displaystyle\kappa(x,a)$ is related to the inverse of the spread around the
mean as shown on the figure below

Figure 2: Von Mises distribution

A setting or a batch is a pair $\displaystyle s=(a,\theta)$ and $\displaystyle
S_{a}^{1}$ is the circle $\displaystyle\\{(a,\theta),\;\theta\in\mathbb{R}\\}$
and the set of the settings is $\displaystyle G_{3}=S_{E}^{1}\vee
S_{H}^{1}\vee S_{EH}^{1}$. It gives rise to an activation vector

$\displaystyle(\varphi^{1}_{a}(\theta),\ldots,\varphi^{N_{0}}_{a}(\theta))$

Additional conditions are required to build the input layer: preferred angle
distribution and activation spread can be chosen. The following rules are
implemented:

  * $\displaystyle\looparrowright$

$\displaystyle]-\pi,\pi]$ is equipartitionned in $\displaystyle N_{0}$
subintervals with centers $\displaystyle\theta_{1},\ldots,\theta_{N_{0}}$
gathered in a set $\displaystyle C$.

  * $\displaystyle\looparrowright$

$\displaystyle\tau_{a}:x\mapsto\mu(x,a)$ is a permutation of $\displaystyle C$
for all $\displaystyle a\in A_{3}$.

  * $\displaystyle\looparrowright$

Relative distributions $\displaystyle x\mapsto\tau_{a}(x)-\tau_{E}(x)$ where
$\displaystyle a\in\\{H,EH\\}$ are Gaussian like with well separated maximum.
Relative distributions are not that important as long as they are
significantly different.

  * $\displaystyle\looparrowright$

For a given $\displaystyle a\in A_{3}$, all neurons have the same
$\displaystyle\kappa(x,a)$ value, i.e. $\displaystyle x\mapsto\kappa(x,a)$ is
constant. We use $\displaystyle\kappa("E")=1.0$,
$\displaystyle\kappa("H")=2.0$ and $\displaystyle\kappa("EH")=1.5$.

####  2.2.2 The network

We carry out experiment on three networks described as follows:

$\displaystyle\bullet\quad$ Number of cells in the $\displaystyle k$-layer

$\displaystyle p$ | $\displaystyle N_{0}$ | $\displaystyle N_{1}$ | $\displaystyle N_{2}$ | $\displaystyle N_{3}$ | $\displaystyle N_{4}$
---|---|---|---|---|---
3 | 55 | 50 | 4 |  |
4 | 55 | 50 | 25 | 4 |
5 | 55 | 55 | 50 | 25 | 4

$\displaystyle\bullet\quad$ Non linearity: $\displaystyle x\mapsto\tanh(4x)$.

####  2.2.3 The output layer and the loss function

The three activators $\displaystyle a\in{\cal A}_{3}$ are represented by the
three roots of unity $\displaystyle 1,\omega,\omega^{2}$ in order to preserve
symetry. The complex number corresponding to $\displaystyle a$ is denoted
$\displaystyle z_{a}$. Let us assume that a setting $\displaystyle
s=(a_{0},\theta_{0})$ has been selected where $\displaystyle a_{0}\in{\cal
A}_{3}$ and $\displaystyle\theta_{0}\in]-\pi,\pi[$. The output layer has four
neurones:

  * -

the first pair provides a complex number $\displaystyle z=x+iy$ and the
decision is made towards the activator $\displaystyle a$ minimizing
$\displaystyle|z-z_{a}|$.

  * -

the second pair identifies $\displaystyle\theta$ by means of a pair
$\displaystyle(u,v)$ which provides an estimate of $\displaystyle\cos(\theta)$
and $\displaystyle\sin(\theta)$.

We denote $\displaystyle w$ the set of all weights in the neural network and
$\displaystyle f_{w}$ the map applying a setting $\displaystyle s=(a,\theta)$
on the network output $\displaystyle(x,y,u,v)$. The set of all weights
$\displaystyle w$ minimizes the euclidean distance (MSE criterion)

$\displaystyle
d^{2}(s,f_{w}(s))=|z-z_{a}|^{2}+\left(u-\cos(\theta)\right)^{2}+\left(v-\sin(\theta)\right)^{2}$

where $\displaystyle z$ and $\displaystyle z_{a}$ are defined in the previous
subsection.

####  2.2.4 Displaying the activity of a neuron

Given a setting $\displaystyle s=(a,\theta)$ where $\displaystyle
a\in\mathcal{A}_{3}$ and $\displaystyle\theta\in]-\pi,\pi[$, we denote the
output of the last hidden layer as $\displaystyle\varphi^{y}_{a}(\theta)$
where $\displaystyle y\in[1,N_{p-1}]$. In order to visualize the response of a
neuron for all settings $\displaystyle s=(a,\theta)$, we have associated the
discrete valued parameter $\displaystyle a$ with a color and we have
represented the excursion of
$\displaystyle\theta\mapsto\varphi^{y}_{a}(\theta)$ by means of the 2D polar
curve $\displaystyle\mathcal{C}_{j}$ which plots

$\displaystyle\theta\mapsto\varphi^{y}_{a}(\theta)(\cos(\theta),\sin(\theta))$

using the color corresponding to $\displaystyle a$. Continuous lines
correspond to positive values of $\displaystyle\varphi^{y}_{a}(\theta)$ while
dashed lines correspond to negative values.

From the examples of figure 3, we can observe that when $\displaystyle
a\in\\{E,EH\\}$, $\displaystyle\theta\mapsto\varphi^{25}_{a}$ is negative and
almost constant. In a similar way, we can observe that when $\displaystyle
a\in\\{H,EH\\}$, $\displaystyle\theta\mapsto\varphi^{28}_{a}$ is positive and
almost constant.

Figure 3: First examples of neuron activities

This follows the model explained in [NM17, NM18]:

  1. 1)

a first layer $\displaystyle L_{1}$ contains $\displaystyle 55$ neurons of
type $\displaystyle\mathsf{Meq}$; each one, say $\displaystyle x$, is
represented by three $\displaystyle 2\pi$-periodic functions
$\displaystyle\varphi^{x}_{a}$ for $\displaystyle a=E,H$ or $\displaystyle
EH$, with values in $\displaystyle[0,1]$. The value
$\displaystyle\varphi^{x}_{a}(\theta)$ for $\displaystyle\theta\in[0,2\pi]$
represents the activity of the neuron $\displaystyle x$ when the movement is
made in direction $\displaystyle\theta$ and for the condition $\displaystyle
a$;

  2. 2)

a hidden layer $\displaystyle L_{2}$ is made by $\displaystyle 50$ neurons
with activity in $\displaystyle[-1,1]$, computed by a $\displaystyle L-NL$
transformation of the vector measuring the activity in $\displaystyle L_{1}$:

$\psi^{y}=\tanh(\Sigma_{x}w^{y}_{x}\varphi^{x}).$ (1)

  3. 3)

a third and last layer is made by four neurons, $\displaystyle
z_{1},z_{2},z_{3},z_{4}$, with activity in $\displaystyle[-1,1]$, the two
first ones correspond to the condition, the two other ones correspond to the
angle.

The coordinates $\displaystyle z_{1},z_{2}$ of the conditions $\displaystyle
E$, $\displaystyle H$, $\displaystyle EH$ respectively correspond to the three
vertices of an equilateral triangle in the square $\displaystyle[-1,1]^{2}$:
$\displaystyle 1=(1,0)$, $\displaystyle\omega=(-1/2,\sqrt{3}/2)$,
$\displaystyle\omega^{2}=(-1/2,-\sqrt{3}/2)$. The coordinates $\displaystyle
z_{3},z_{4}$ correspond respectively to the cosine and sine of the angle
$\displaystyle\theta$. In the functioning feed-forward network they are
computed by the fully connected equation 1 from the activity in $\displaystyle
L_{2}$.

The correspondence to be learned by the $\displaystyle NN$ is the natural one:
in $\displaystyle L_{3}$ the description of an individual movement by
$\displaystyle(a,\cos\theta,\sin\theta)$, in entry $\displaystyle L_{1}$ the
corresponding vector $\displaystyle X_{1}(a,\theta)$.

######  Remark 1.

The choice of the four cells in $\displaystyle L_{3}$ is made for respecting
at most the symmetries of the experiment. We have also tested a model which
doesn’t respect the symmetry between the three conditions $\displaystyle a$,
replacing $\displaystyle z_{1},z_{2}$ by only one neuron $\displaystyle
z^{\prime}_{1}$, taking its values in $\displaystyle[-1,1]$, with
$\displaystyle-1$ for $\displaystyle E$, $\displaystyle 0$ for $\displaystyle
H$ and $\displaystyle 1$ for $\displaystyle EH$. We will compare the results
of this model $\displaystyle z^{\prime}$ with the model $\displaystyle z$ (see
figure 8) in the following sections.

######  Remark.

Each input is an angle $\displaystyle\theta$ and a condition $\displaystyle
a$, but the neurons $\displaystyle x$ in the first layer don’t register this
pair, they react to it according to their receptive field, by taking a unique
real value $\displaystyle\varphi_{a}^{x}(\theta)$. This is not so far from
primary sensory reactions of schematized retinal cells to a colored flash,
$\displaystyle\theta$ being the place in the visual plane where the flash
appears, $\displaystyle a$ being the color ($\displaystyle L$ for long, red,
$\displaystyle M$ for medium, green, $\displaystyle S$ for short, blue),
modulating the reaction of the cell. At this stage, the two components (place
and color) are intermingled, and the network has to detect (extract) the color
only. Thus, even if it was not the original motivation of our experiment, this
is not very far from the usual exploitation of artificial neural networks. (In
the visual system of primate, things are a bit different : one layer after the
retina, in the thalamus, most color neurons have a preference for three
algebraic combinations of the pigments, $\displaystyle L+M+S$, $\displaystyle
L-M$, $\displaystyle L+M-S$).

The functions $\displaystyle\varphi^{x}_{a}$ are Von-Mises distributions
densities (see Figure 2). The sampling for the $\displaystyle 55$ cells is
uniform in $\displaystyle\theta$, and contains four sub-populations,

  1. 1.

similar preferred angle for $\displaystyle E,H$ and $\displaystyle EH$,

  2. 2.

orthogonal angles for $\displaystyle E$ and $\displaystyle H$ divided in
$\displaystyle 2.A$, resp. $\displaystyle 2.B$ resp. $\displaystyle 2.C$,
where the preference of $\displaystyle EH$ is almost the same as
$\displaystyle E$, resp. $\displaystyle H$, resp. another one.

For comparison of the feed-forward element $\displaystyle F_{w}(X_{1})$ with
the truly expected $\displaystyle X_{3}$, we take the Euclidian distance, or
the Euclidian distance after dilatation of $\displaystyle z_{1},z_{2}$ (resp.
$\displaystyle z^{\prime}_{1}$ in the asymmetric model mentioned in remark 1.

###  2.3 Theoretical deduction of the movement from the first layer

Note that the natural map $\displaystyle\Phi$ is from the elements
$\displaystyle X_{3}$ in the disjoint of three circles $\displaystyle
G_{3}=\vee_{a}S_{a}^{1}$ to the elements $\displaystyle X_{1}$ in the
hypercube $\displaystyle K_{1}=[0,1]^{55}$, then the image $\displaystyle
K_{3}\subset K_{1}$ of this map contains the set for training, testing and
generalizing together. The network has to compute an inverse
$\displaystyle\Phi^{-1}$ of the map $\displaystyle\Phi$ from $\displaystyle
K_{3}$ to $\displaystyle G_{3}$.
We will write $\displaystyle A_{3}={E,H,EH}$ for the set of conditions.
Of course, when functioning, the result of the feed-forward starting with a
point $\displaystyle X_{1}=\Phi(a,\theta)$ does’nt give exactly a point in
$\displaystyle G_{3}$, it gives a point in the cube $\displaystyle
I_{4}=[-1,1]^{4}$ (or $\displaystyle I_{3}$ for $\displaystyle z^{\prime}$).
Experiment show that the error is small: this point in $\displaystyle I_{4}$
(resp. $\displaystyle I_{3}$) is very close from the point
$\displaystyle(a,\cos\theta,\sin\theta)$.

######  Proposition 1.

The map $\displaystyle\Phi$ is injective.

###### Proof.

let $\displaystyle X_{3}=(a,\theta)$ be given in $\displaystyle G_{3}$, the
$\displaystyle 55$ components of vector $\displaystyle\Phi(X_{3})$ are the
numbers $\displaystyle\varphi^{x}_{a}(\theta),1\leq x\leq 55$. The form of
each function $\displaystyle\varphi^{x}_{a}$ implies that each of its value
determines $\displaystyle\theta$ up to the symmetry with respect to the angle
$\displaystyle\theta^{x}_{a}$ giving the maximum of
$\displaystyle\varphi^{x}_{a}$. Consequently, as soon as we consider two
neurons which have different values of $\displaystyle\theta^{x}_{a}$, the
ambiguity is suppressed. We now turn to the condition $\displaystyle a$, and
consider a different condition $\displaystyle b$; the last of the four
families of neurons, i. e. $\displaystyle 2.C$, implies that the two vectors
$\displaystyle\varphi^{x}_{a}(\theta),x\in L_{1}(2.C)$ and
$\displaystyle\varphi^{x}_{b}(\theta),x\in L_{1}(2.C)$ are different. ∎

This proposition doesn’t give a very practical algorithm for computing the
inverse. We develop now such an algorithm.

A direct observation of the neurons in layer $\displaystyle L_{1}$ explains
why they are able to construct $\displaystyle\Phi^{-1}$, at least within a
good approximation.
The main observation is the following one: when an angle $\displaystyle\theta$
is given, the population of neurons generates a correspondence between
activation value, say $\displaystyle 1$ for $\displaystyle\varphi>1/2$ versus
$\displaystyle 0$ for $\displaystyle\varphi<1/2$ and a subset of
$\displaystyle A_{3}={E,H,EH}$ and its complement in $\displaystyle A_{3}$. We
call such a subset a _simple proposition_. We will meet more elaborate
propositions in section 3.

Figure 4:
$\displaystyle\theta\mapsto\varphi^{j}_{a}(\cos(\theta),\sin(\theta))$ for
$\displaystyle j=17,18,19,20$ and $\displaystyle a\in A_{3}$

For instance, looking at cell $\displaystyle 17$ for
$\displaystyle\theta=\pi/2$, we see that the cell is active in conditions
$\displaystyle E,H$ but not $\displaystyle EH$, then proposition
$\displaystyle P$ is $\displaystyle E\vee H$, versus its complement
$\displaystyle\neg P=EH$.
If an ambiguous situation happens, for instance near the value $\displaystyle
1/2$ of $\displaystyle\varphi_{a}^{x}$, we can forget this cell $\displaystyle
x$. However experiment shows that this scarcely happens.
For each $\displaystyle\theta$, we check that the set of propositions
$\displaystyle P(x)$ is sufficiently rich to deduce the condition from the
vector $\displaystyle\varphi^{x}_{a}(\theta),x\in L_{1}(2)$.
Note that with the population of type $\displaystyle 1$ only, this would not
have happened, the three conditions being non-separated. But the population
$\displaystyle L_{1}(1)$ is sufficiently rich to determine the angle
$\displaystyle\theta$ with a good approximation.
These two assertions have to be verified, but they correspond to our choice of
distributions of densities $\displaystyle\varphi^{x}_{a}$.

Now a possible logical algorithm works as follows:

  1. 1)

determine $\displaystyle\theta$ from a particular vector $\displaystyle
X_{1}(17)$ of activity in $\displaystyle L_{1}(17)$, for instance by linear
voting [GSK86]: take the sum over $\displaystyle x$ of the cosine (resp. sine)
of its preferred angle (here, in $\displaystyle L_{1}(17)$, it is the same for
the three conditions), weighted (i.e. multiplied) by the observed activity
$\displaystyle\varphi^{x}$, then take the arccosine (resp. the arcsine).

  2. 2)

From this approximate value, deduce the condition, as explained before, by
logical computations, either the simplest one, either another vote: the number
of times $\displaystyle a$ appears in the list of propositions corresponding
to the vector $\displaystyle X_{1}(17)$.

  3. 3)

From this condition, use the full population of curves
$\displaystyle\varphi^{x}_{a}$, to get a more accurate value of the angle
$\displaystyle\theta$.

  4. 4)

Check that this gives the same condition as in step $\displaystyle 2$.

This mixture of usual decoding and logics can be seen as a sort of _logical
conditioning_ , the conditioning being done here on a continuous parameter
like $\displaystyle\theta$.

Of course it is not the way this simple DNN has worked. But we will show now
that he is probably right, because the hidden layer contains much more
interesting Logical Cells than the first layer, as we will show now.

###  2.4 Characteristics of the hidden layer

In the hidden layer we compute the activity of each cell, a real number
between $\displaystyle 0$ and $\displaystyle 1$, again denoted
$\displaystyle\varphi_{a}^{x}$, corresponding to the movement directed to
$\displaystyle x$ and effected by $\displaystyle a$.

(a) Hidden Layer Cell $\displaystyle 26$

(b) Hidden Layer Cell $\displaystyle 30$

(c) Hidden Layer Cell $\displaystyle 11$

(d) Hidden Layer Cell $\displaystyle 16$

Figure 5:
$\displaystyle\theta\mapsto\varphi^{j}_{a}(\cos(\theta),\sin(\theta))$ for
$\displaystyle j\in\\{9,10,11,16\\}$

The main observation coming from what we see in layer $\displaystyle L_{2}$,
after supervised learning, is that, in many cells, two of the curves
$\displaystyle\varphi_{a}^{x}$ are almost saturated in $\displaystyle+1$ or
$\displaystyle-1$, the same value for both of them, but the third curve shows
positive and negative values. Precisely $\displaystyle 35$ cells out of
$\displaystyle 50$ have this property, that two of the graphs stay positive or
negative, the same for both, and the third one no (cf. Figure 5(a)). In all of
these cells, the last graph had exactly two zeros, defining two segments of
the circle.
Five other cells had only one graph which doesn’t change of sign. (Figure
5(b))
Five other cells had the three graphs of the same sign. (Figure 5(c))
And five cells had the three graphs which changed of signs. (Figure 5(d))
The enumeration of the $\displaystyle 35$ cells gave $\displaystyle 10$ cells
of type $\displaystyle E$ (i.e. the transgressing curve is
$\displaystyle\varphi_{E}$), $\displaystyle 12$ of type $\displaystyle H$, and
$\displaystyle 13$ cells of type $\displaystyle EH$.
The enumeration of the $\displaystyle 5$ cells gave four of type
$\displaystyle E$, (i.e. the only graph which doesn’t change of sign is
$\displaystyle\varphi_{E}$), and one of type $\displaystyle H$.
The fundamentally new fact, with respect to $\displaystyle L_{1}$, is the
possibility to use the discretization by the sign, to get logical
propositions. For instance, take one of the $\displaystyle 35$, say of type
$\displaystyle E$, with the two graphs $\displaystyle\varphi_{H}$ and
$\displaystyle\varphi_{EH}$ positive, then, if the cell fire negatively, it
tells ”$\displaystyle E$ is true”.
For a cell with only one positive graph, say $\displaystyle\varphi_{E}$, if
the cell fire negatively, it tells: $\displaystyle E$ is false, i.e.
$\displaystyle H\vee EH$ is true.
When the three graphs have the same sign, the cell gives no information at
all, its firing seems to be independent of the condition $\displaystyle E$,
$\displaystyle H$ or $\displaystyle EH$.
And for another reason, when all the graphs change their sign, the cell gives
also no information, because we are not able to extract an information from
the activity of the cells, knowing its receptive field.

However, from that, the hidden layer is able to recover the condition from its
activity and the above logical formulas. Why? This is because, for each
condition, the segments in the circle that are informative cover nicely the
circle. This was checked by inspection, see Figure 5, for $\displaystyle H$.
By definition, a segment is informative for a cell $\displaystyle i$ of type
$\displaystyle a$, if it corresponds to the sign which gives the assertion
”$\displaystyle a$ is true”.
Then, assume for instance that the condition is $\displaystyle H$, and the
movement at the input is directed to a point $\displaystyle x$, belonging to
the informative segment of a cell $\displaystyle i$ of type $\displaystyle a$,
with a sign $\displaystyle+$, then the cell fires positively, consequently the
cell tells ”$\displaystyle a$ is true”.
We see no contradiction in the reconstruction, which is not surprising, by
construction of the curves $\displaystyle\varphi_{a}^{x}$, and the preceding
definitions.

The existence of curves which almost saturate independently of the angle,
giving logical propositions, confirmed the observations of Nemyriolitis and
Moschovakis [NM18].

The appearance of this quantization, and the easy deduction of the
classification that it allows, gives us hope that with more layers, more
logical functioning could appear. And this is true.

###  2.5 More and more layers

We added between $\displaystyle L_{2}$ and the output a second hidden layer
$\displaystyle L_{3}$, fully connected with $\displaystyle L_{2}$, containing
$\displaystyle 25$ cells.
The inputs were the same. The network learned very well by back-propagation,
in fact it learned much better and easier than with only one hidden layer.
Now the main observation coming from the contemplation of the last hidden
layer $\displaystyle L_{3}$ was the appearance of neurons where one or two (or
three) of the graphs almost saturated at positive or negative value, but now,
contrarily to what happened in $\displaystyle L_{2}$ before, these values were
sometimes different; which changes radically the things as we will see.

(a) Layer $\displaystyle L_{3}$ Cell $\displaystyle 5$

(b) Layer $\displaystyle L_{3}$ Cell $\displaystyle 3$

(c) Layer $\displaystyle L_{3}$ Cell $\displaystyle 21$

(d) Layer $\displaystyle L_{3}$ Cell $\displaystyle 1$

Figure 6: Multi-Layer $\displaystyle L_{3}$ with
$\displaystyle\theta\mapsto\varphi^{j}_{a}(\cos(\theta),\sin(\theta))$ for
$\displaystyle j\in\\{5,3,21,1\\}$

All the (apparently) stupid cells with three times the same sign disappeared.
Eight cells had two saturated curves with opposite signs (see Figure 6(a)).
Seven cells had two saturated curves with the same sign, as in the
$\displaystyle L_{2}$ before (see Figure 6(b)). One cell had three almost
saturated curves, not all of the same sign (see Figure 6(c)).

Seven cells had only one saturated curve (see Figure 6(d)). Eventually, two
cells had behaviors difficult to interpret, crossing all the curves.

Another noticeable thing with respect to the preceding simpler network, was
that five out of seven cells with two saturations of the same sign had an
informative segment covering half of the circle. This allowed these
$\displaystyle 7$ cells to do almost as well as the preceding $\displaystyle
35$ informative cells, in applying the argument of reconstruction by
coverings. (More precisely, seven of the $\displaystyle 25$, versus
$\displaystyle 35$ cells among the $\displaystyle 50$ in $\displaystyle L_{2}$
before.)

All that gives $\displaystyle 23$ logical cells, that are cells whose activity
can be translated in a logical proposition.
Each of the above type of cells gives a different structure of implication:

Examples. ($\displaystyle E$ is blue, $\displaystyle H$ is red and
$\displaystyle EH$ is green)

  * •

Cell 1: $\displaystyle H\Rightarrow-1$, or equivalently
$\displaystyle+1\Rightarrow E\vee EH$. The information is: independently of
the angle, if the activity is positive, the condition is either $\displaystyle
E$ either $\displaystyle EH$. (Figure 6(d))

  * •

Cell 3: $\displaystyle E$ and $\displaystyle EH$ stay strongly positive, but
$\displaystyle H$ presents two signs, then negative implies the condition
$\displaystyle H$, which we note $\displaystyle-1\Rightarrow H$. The
information of this cell is: independently of the angle, if the activity is
negative, the condition is $\displaystyle H$ (Figure 6(b)).

  * •

Cell 5: $\displaystyle E\Rightarrow 1$, and $\displaystyle H\Rightarrow-1$,
therefore $\displaystyle<0\Rightarrow H\vee EH$ and
$\displaystyle>0\Rightarrow E\vee EH$ (Figure 6(a)).

  * •

Cell 21: $\displaystyle H\Rightarrow+1$, $\displaystyle E\Rightarrow-1$ and
$\displaystyle EH\Rightarrow-1$, then $\displaystyle+1\Rightarrow H$,
$\displaystyle-1\Rightarrow E\vee EH$ (Figure 6(c)).

######  Remark.

In this analysis, as in the following arguments, we use the fact
that$\displaystyle P\Rightarrow Q$ is equivalent to $\displaystyle\lnot
Q\Rightarrow\lnot P$, and we will use many times that $\displaystyle E,H$ and
$\displaystyle EH$ are exclusive. It is legitimate to question these
assumptions : how can the network be aware of the Boolean axioms in logic? The
answer is that it has learned these elements during the learning process,
because the asked output is $\displaystyle E,H$ and $\displaystyle EH$ and the
metric sanctions any mixture of the conditions.

Let us give a first example of possible fully conclusive reasoning:

######  Proposition 2.

Suppose we have three cells of the type of cell 5 of figure 6(a), involving
symmetrically all possible pairs of arguments, for instance,
cell $\displaystyle I$: $\displaystyle 1\Rightarrow E\vee H,-1\Rightarrow
E\vee EH$;
cell $\displaystyle II$: $\displaystyle 1\Rightarrow E\vee EH,-1\Rightarrow
H\vee EH$;
cell $\displaystyle III$: $\displaystyle 1\Rightarrow E\vee H,-1\Rightarrow
H\vee EH$.
Then the condition follows from the three activities, as soon as they are non-
contradictory.

###### Proof.

the following implications are easily verified:

$(1,1,1)\Rightarrow E,\quad(1,1,-1)\Rightarrow\bot,\quad(1,-1,1)\Rightarrow
H,\\\ (1,-1,-1)\Rightarrow H,\quad(-1,1,1)\Rightarrow
E,\quad(-1,1,-1)\Rightarrow EH,\\\
(-1,-1,1)\Rightarrow\bot,\quad(-1,-1,-1)\Rightarrow EH.$ (2)

∎

######  Proposition 3.

For an input corresponding to the discrete condition $\displaystyle a$, the
above three cells together can reconstruct the answer $\displaystyle a$ by
pure logical deduction.

###### Proof.

From the symmetry under $\displaystyle\mathfrak{S}_{3}$ (group of permutations
of $\displaystyle 3$ objects), we can assume that the condition is
$\displaystyle E$, then from the table, by contraposition, the cell
$\displaystyle II$ fires at $\displaystyle+1$, and the cell $\displaystyle
III$ also. From that, reading the table in the written direction, we get
$\displaystyle\vdash E\vee EH$ and $\displaystyle\vdash E\vee H$, then
$\displaystyle E$ is asserted to be true. ∎

Remarkably, in $\displaystyle L_{3}$ there existed cells for each of the three
types $\displaystyle I,II,III$: $\displaystyle 23$ and $\displaystyle 25$ are
of type $\displaystyle I$, $\displaystyle 5$,$\displaystyle 6$,$\displaystyle
10$ of type $\displaystyle II$ and $\displaystyle 14$, $\displaystyle 16$,
$\displaystyle 18$ of type $\displaystyle III$.
Thus the third layer can decide very easily and by pure logical reasoning what
is the true condition.

We say that a set of cells which satisfies the result of proposition 2 is
_complete_. And we say that a set of cells which satisfies proposition 3 is
_good enough_.

######  Remark.

It can happen that a triple has one of these properties without having the
other one. Examples are given below.

A set which is both complete and good enough is said to be _efficient_.
To be _conclusive_ for a group of logical cells _a priori_ depends on its
possible activities, and then, on the input it receives. Efficient always
means conclusive for a given set of inputs, and reconstructing the right
answer for the required objectives.
These notions are more useful when every set of cells containing a complete
(resp. good enough, resp. efficient) set has the same property. This
requirement is equivalent to the absence of contradiction in the propositions
coming from real data in input. A fundamental experimental fact that we
observed in this study, is this absence of contradiction in real data. However
a contradiction could _a priori_ could happen in ”generalization” data, if the
new data generate saturated answers violating the logic, but what we observed
in all the unadapted data for generalization, was more the vanishing of
apparent logical structures, for instance no saturation at all.

Other types of cell could be like $\displaystyle 21$:
cell $\displaystyle IV$: $\displaystyle 1\Rightarrow E,-1\Rightarrow H\vee
EH$;
cell $\displaystyle V$; $\displaystyle 1\Rightarrow EH,-1\Rightarrow E\vee H$.

######  Proposition 4.

The set of cells $\displaystyle\\{I,IV,V\\}$ and the set of cells
$\displaystyle\\{I,II,V\\}$ are complete sets, but they are not good enough.
However, taken all together, $\displaystyle\\{I,II,IV,V\\}$ form a set which
is good enough, then is efficient.

###### Proof.

For the first triple:

$(1,1,1)\Rightarrow\bot,\quad(1,1,-1)\Rightarrow
E,\quad(1,-1,1)\Rightarrow\bot,\\\ (1,-1,-1)\Rightarrow
H,\quad(-1,1,1)\Rightarrow\bot,\quad(-1,1,-1)\Rightarrow E,\\\
(-1,-1,1)\Rightarrow EH,\quad(-1,-1,-1)\Rightarrow\bot.$ (3)

For the second one:

$(1,1,1)\Rightarrow EH,\quad(1,1,-1)\Rightarrow
E,\quad(1,-1,1)\Rightarrow\bot,\\\ (1,-1,-1)\Rightarrow
H,\quad(-1,1,1)\Rightarrow EH,\quad(-1,1,-1)\Rightarrow E,\\\
(-1,-1,1)\Rightarrow EH,\quad(-1,-1,-1)\Rightarrow\bot.$ (4)

Now consider an input of type $\displaystyle H$, nobody can tell what will be
the predicted activity in $\displaystyle IV$ or $\displaystyle V$. But in
$\displaystyle I$, we know that it will be $\displaystyle+1$, and in
$\displaystyle II$, we know it will be $\displaystyle-1$, then in the first
group we can only conclude $\displaystyle\vdash H\vee E$ and in the second
group $\displaystyle\vdash H\vee EH$.
To verify the efficiency of the union of the groups, we have to look at
$\displaystyle E$ and $\displaystyle EH$.
In the case of $\displaystyle E$, cells $\displaystyle II$ and $\displaystyle
IV$ express $\displaystyle+1$, then they assert respectively the truth of
$\displaystyle E\vee EH$ and $\displaystyle E$, which gives $\displaystyle E$.
In the case of $\displaystyle EH$, the cell $\displaystyle I$ tells
$\displaystyle-1$, and the cell $\displaystyle V$ tells $\displaystyle+1$,
then they respectively conjecture $\displaystyle E\vee EH$ and $\displaystyle
EH$, thus together they tell that $\displaystyle EH$ is true. ∎

From the population of receptive fields in the layer $\displaystyle L_{3}$, we
see that the network hesitates between two strategies, one is a mixture of
geography and logic, like the hidden layer $\displaystyle L_{2}$ before, with
conditioning by angular regions, and one purely logical with efficient groups
of cells, that do not look at angles anymore. Of course this could be helpful
to develop two possibly cooperating strategies, but we asked us if a growing
complexity will induce a choice or maintain the two ways of reasoning.

The result is very instructive: if we increase the number of neurons in
$\displaystyle L_{3}$ (say $\displaystyle 50$ instead of $\displaystyle 25$),
the network regresses to the non-purely logical strategy it adopted with only
one hidden layer, but if we increase the number of hidden layers (we introduce
a third deeper hidden layer $\displaystyle L_{4}$ with $\displaystyle 25$
neurons), the network totally forgets the primitive (or initial) strategy, and
develops further the logic, it continues using the efficient groups of cells
just described above, and it invents new efficient triples, more directly
conclusive, that it was apparently not able to form before, when it had only
two hidden layers.

With three hidden layers, one $\displaystyle L_{2}$ with $\displaystyle 50$
neurons, and the two deeper ones $\displaystyle L_{3}$, $\displaystyle L_{4}$
with $\displaystyle 25$ neurons, we found the following innovative
composition: $\displaystyle 11$ cells of crossed type, like
$\displaystyle+1\rightarrow E\vee H,-1\rightarrow E\vee EH$ (in fact,
$\displaystyle 1$ as this one, i.e. like the above cell $\displaystyle I$,
$\displaystyle 5$ like the cell $\displaystyle II$, preferring $\displaystyle
EH$, $\displaystyle 5$ like the cell $\displaystyle III$, preferring
$\displaystyle H$, which is not totally optimal, but complete and good
enough); $\displaystyle 7$ cells of the fully saturated type (without any
stupid one with all saturations on the same sign), and here with optimal
distribution ($\displaystyle 2$ of the type $\displaystyle E$ (i.e.
$\displaystyle+\pm 1\rightarrow E,\mp 1\rightarrow H\vee EH$), $\displaystyle
3$ of the type $\displaystyle H$ (i.e. $\displaystyle\pm 1\rightarrow H,\mp
1\rightarrow E\vee EH$) and $\displaystyle 2$ of the type $\displaystyle EH$
(i.e. $\displaystyle\pm 1\rightarrow EH,\mp 1\rightarrow H\vee E$). Six cells
were less informative, like $\displaystyle+1\rightarrow E\vee EH$, and a last
one was obscure. (Cf. Figure 7, for a sample in $\displaystyle L_{3}$.)
Remarkably, no cell corresponds to the primitive strategy, mixing geography
and logic.

(a) Layer $\displaystyle L_{4}$ Cell $\displaystyle 3$

(b) Layer $\displaystyle L_{4}$ Cell $\displaystyle 6$

(c) Layer $\displaystyle L_{4}$ Cell $\displaystyle 8$

(d) Layer $\displaystyle L_{4}$ Cell $\displaystyle 12$

(e) Layer $\displaystyle L_{4}$ Cell $\displaystyle 14$

(f) Layer $\displaystyle L_{4}$ Cell $\displaystyle 22$

Figure 7: Multi-Layer $\displaystyle L_{4}$ with
$\displaystyle\theta\mapsto\varphi^{j}_{a}(\cos(\theta),\sin(\theta))$ for
$\displaystyle j\in\\{3,6,8,12,14,22\\}$

######  Proposition 5.

Consider three maximally saturated cells involving symmetrically all the
different pairs of arguments, for instance
cell $\displaystyle VI$: $\displaystyle 1\Rightarrow E,-1\Rightarrow H\vee
EH$;
cell $\displaystyle VII$: $\displaystyle 1\Rightarrow H,-1\Rightarrow E\vee
EH$;
cell $\displaystyle VIII$: $\displaystyle 1\Rightarrow EH,-1\Rightarrow H\vee
E$.
Then the set $\displaystyle\\{VI,VII,VIII\\}$ is efficient.

###### Proof.

Let us begin by checking that a condition follows from the three activities,
as soon as they are non-contradictory:

$(1,1,1)\Rightarrow\bot,\quad(1,1,-1)\Rightarrow\bot,\quad(1,-1,1)\Rightarrow\bot,\\\
(-1,1,1)\Rightarrow\bot,\quad,(1,-1,-1)\Rightarrow E,\quad(-1,1,-1)\Rightarrow
H,\\\ (-1,-1,1)\Rightarrow EH,\quad(-1,-1,-1)\Rightarrow\bot.$ (5)

Now suppose that the condition is $\displaystyle E$, then $\displaystyle VI$
fires at $\displaystyle+1$, $\displaystyle VII$ at $\displaystyle-1$ (because
$\displaystyle E\vee EH\rightarrow-1$ by contraposition) and $\displaystyle
VIII$ fires at $\displaystyle-1$ (because $\displaystyle E\vee H\rightarrow-1$
by contraposition); then, by the preceding assertion, with
$\displaystyle(1,-1,-1)$, the cells together assert that $\displaystyle E$ is
true. ∎

In the preceding layer $\displaystyle L_{3}$ we had only one cell of this
maximally saturated type, then the situation of triples was described by
propositions 2,3,4.
This new possibility, invented in $\displaystyle L_{4}$, makes $\displaystyle
18$ cells out of $\displaystyle 25$ all having the appearance of a wise
reasoning assembly.

######  Remark.

A noticeable difference between the triples in proposition 5 and proposition
2, is that in 5 it establishes a one to one correspondence between the
coherent activations (i.e. without contradiction) and the three conditions,
but in 2 it gives a two to one map, i.e. two different activities correspond
to the same condition.

Now let us look at the effect of doubling the number of cells in
$\displaystyle L_{3}$. The logical functioning collapses; the population
becomes even less logical than in $\displaystyle L_{2}$ for the network with
only one hidden layer. In this $\displaystyle L_{2}$, $\displaystyle 35$ cells
over $\displaystyle 50$ joined their efforts to reconstruct a condition by
using coverings by informative intervals in a symmetric and uniform manner;
here in the new $\displaystyle L_{3}$, only $\displaystyle 17$ cells over
$\displaystyle 50$ do that, with seven cells for $\displaystyle E$, seven for
$\displaystyle EH$ and three for $\displaystyle H$. Moreover, the majority of
the cells, $\displaystyle 26$ exactly, are concerned by the (less informative)
unions, five for $\displaystyle E\vee EH$, six for $\displaystyle E\vee H$ and
fifteen for $\displaystyle H\vee EH$. The exception which saves the honor
(from the logical point of view), is realized by three crossed cells (not
forming a complete triple, because two repeat the same message), plus two
cells having three saturations not all of the same sign. There exists another
good point for this population: it doesn’t contain a cell with three
saturations of the same sign, that $\displaystyle L_{2}$ contained. (But is it
a good point to exclude all fantasy?)
The explanation of this disaster seems to be related to the well known danger
of over-fitting. This is certainly part of the truth, but this is not all the
truth, because the number of weights to learn is $\displaystyle 55\times
50+50\times 50+4\times 50=50\times 109$, and in the network with three hidden
layers the number of weights to learn equals $\displaystyle 55\times
50+50\times 25+25\times 25+25\times 4=50\times 94,5$, which is not far. Of
course the three layers imply more non-linearity, but how to count that? We
can just certify:

######  Experimental Result 1.

With a comparable number of parameters to adapt, the addition of a layer
considerably increases the logical functioning, at the level of individual
cells and of collective behaviors, and the addition of cells in one layer has
the opposite effect, also at the levels of individual cells and of collective
behavior.

######  Remark.

Please, no deduction about the necessity of a large number of layers in an
administration. One can also contest our preference for logic, and logical
invention, versus fantasy, and leisure, for a comparable result. Do not forget
that both networks are successful for the task they have to accomplish. We
have not yet compared their powers of generalization. The point of view we
adopt here, is more our own intelligibility of the network’s functioning. We
do not contest that for some more complex tasks, the addition of many cells in
a layer could be preferable, or that logic could come later from another road
in more complex networks.

Another remark to temperate the difference: both $\displaystyle L_{4}$ and the
bigger $\displaystyle L_{3}$ introduced the largest variety of types of
logical cells, even if $\displaystyle L_{4}$ did that in a much more
equilibrate and efficient manner.

For the asymmetric model $\displaystyle z^{\prime}$ (see figure 8), we
conducted analog experiments, with one hidden layer of $\displaystyle 50$
cells, and two hidden layers of respectively $\displaystyle 50$ and
$\displaystyle 25$ cells. The main result was the failure to develop logic
coherently in both cases. The network learned well and performed very well the
classification, but doesn’t develop sufficiently many purely logical cells to
be efficient.

Figure 8: $\displaystyle z$ or $\displaystyle z^{\prime}$ encoding

With one hidden layer $\displaystyle L_{2}$, most of the cells have two
saturations of the same sign, they are $\displaystyle 10$ of type
$\displaystyle E$ and $\displaystyle 8$ of type $\displaystyle EH$, no one of
type $\displaystyle H$. They do the job of the analog cells in $\displaystyle
L_{2}$ for the symmetric model $\displaystyle z$, at least for $\displaystyle
E$ and $\displaystyle EH$.
Ten of the cells have one saturation, two for $\displaystyle E\vee H$, eight
for $\displaystyle H\vee EH$, no one for $\displaystyle E\vee EH$. Half of the
resting cells have three saturating graphs of the same sign, the other half
develop no saturation.
As we see, the condition $\displaystyle H$ encounters difficulties, and it was
hard to reconstruct it directly or indirectly from the activity in the layer
in intelligible manner. However, the network is successful with the
classification, including $\displaystyle H$, then this is a case where the
functioning is not explained by what we see in the layers.

The symmetry which is respected in the model $\displaystyle z$ and not in the
model $\displaystyle z^{\prime}$ is the group $\displaystyle\mathfrak{S}_{3}$
of permutations of $\displaystyle A_{3}$; the fact that in the model
$\displaystyle z^{\prime}$, $\displaystyle H$ is coded by a point inside the
interval $\displaystyle[-1,1]$ and $\displaystyle E$, $\displaystyle EH$ by
the boundaries, destroys the symmetry, and we constat that it also destroys an
important part of the logic inside the network. Cf. Figure 8.
Then we get the important conclusion:

######  Experimental Result 2.

The topology of encoding must respect the symmetries, for the emergence of a
logic in the inner layer, but not necessarily for the success of the neural
network.

As we will see in the discussion below, this is reminiscent of the appearance
of Fourier analysis or Color analysis in the first hidden layers of the CNNs,
and CNNs are known to be more successful than simple DNNs for image analysis.

With two hidden layers, this difficulty persisted. We observed superficially
the same kind of progress we saw in the model $\displaystyle z$: it appeared
one graph with three saturations not all of the same sign, and three crossed
cells. However, both of them were of the type $\displaystyle\pm 1\rightarrow
H\vee E,\mp 1\rightarrow H\vee EH$. Eight cells had two saturations of the
same sign, but no one of the type $\displaystyle H$, and eleven with one
saturation, of the type $\displaystyle E\vee H$ or $\displaystyle H\vee EH$,
not $\displaystyle E\vee EH$. Thus the penalty to $\displaystyle H$ persisted.
This implied that it was not possible to logically deduce or reconstruct the
condition from the activity. Even taken all together, the neurons of
$\displaystyle L_{3}$ didn’t form a good enough set.

There remain natural questions:

  * •

what are the weights of the logical cells for going to the output layer?

  * •

Do they reflect the logical preferences?

  * •

Same question for significative subsets of the last hidden layer.

  * •

Do we see a correlation between synergies of weights and synergies in
reasoning?

###  2.6 Logical values

We start with a tentative definition of the individual value of a neuron from
the logical point of view, in our simple example:

######  Definition 1.

Let’s give the value $\displaystyle 1/2$ to every assertion of the form
$\displaystyle 1$ implies $\displaystyle E$ (here $\displaystyle 1$ can be
either $\displaystyle+1$ or $\displaystyle-1$ and independently $\displaystyle
E$ can be replaced by $\displaystyle H$ or $\displaystyle EH$), the value
$\displaystyle 1/4$ for an assertion of the form $\displaystyle+1$ (resp.
$\displaystyle-1$) implies $\displaystyle E\vee H$ (resp. $\displaystyle E\vee
EH$, resp. $\displaystyle H\vee EH$). Then, by convention, the individual
information value of the cell $\displaystyle y$ is the sum of the assertions
it gives.

######  Remark 2.

The individual value of a cell can be $\displaystyle 0,1/4,1/2,3/4$. No cell
can get the score $\displaystyle 1$ because of the construction of the
assertions by contraposition, from at most three implications of the form
$\displaystyle a\Rightarrow\varepsilon$, where $\displaystyle a$ is a
condition and $\displaystyle\varepsilon=\pm 1$.

######  Remark 3.

The above propositions indicate that the value of a group is not the sum of
the values of the components. For the efficiency it even works in the wrong
sense.

######  Remark 4.

In the above definition, we have decided that an atom, like $\displaystyle E$
is more precious than a union like $\displaystyle\neg E=H\vee EH$. This is
apparently justified because we want to know the exact condition; however,
proposition LABEL:prop-triple shows that it is more difficult to justify when
thinking in terms of the neuronal assembly and of the collection of
propositions regarding $\displaystyle E$, $\displaystyle H$ and $\displaystyle
EH$. In fact, to find an algorithm which is able to decide the truth or not of
every proposition about the three conditions is equivalent to an algorithm
which can prove any of the atom when it is true, but it is also equivalent to
an algorithm which can prove that they are false when it is the case. For
instance $\displaystyle\neg E$ and $\displaystyle\neg H$ imply $\displaystyle
EH$. That is because we are working in a Boolean logic, which gives us an
important _a priori_ knowledge. In the Boolean setting there exists a duality
between prove and disprove.

######  Remark 5.

Note that the above duality doesn’t totally disappear in intuitionist logic.
Suppose we can prove $\displaystyle E\vee H$ and $\displaystyle E\vee EH$, in
a context where we know that $\displaystyle EH$ and $\displaystyle H$ are
contradictory (i.e. $\displaystyle H\wedge EH\Rightarrow\bot$), then we have a
proof of $\displaystyle E$ or a proof of $\displaystyle H$, and separately a
proof of $\displaystyle E$ or a proof of $\displaystyle EH$, then we have at
least one proof of $\displaystyle E$. On the other side, if we know that
$\displaystyle H$ is true, what can _a priori_ exclude that $\displaystyle EH$
is true if we don’t have assumed they are contradictory.

In a general finite Heyting algebra, there exists a dissymmetry between the
number of truth values of propositions and of truth values of propositions of
the form $\displaystyle\neg P$. Also, negative propositions are scarcer.

######  Remark 6.

The experiments we will report in the next sections, invited us, in particular
with respect to a problem of classification, to measure the logical value of a
cell or a group of cells, by the set of elementary propositions that a given
activity exclude, i.e. conjecture to be false. The above definition 1 accords
with that, giving $\displaystyle 1/4$ for each exclusion.
This corresponds nicely with the notion of _content_ , that Carnap and Bar-
Hillel studied in 1952. For a more complete discussion see the companion paper
on probabilities and the forthcoming theoretical paper on semantic
information. However, looking at more complex experiments, we will need a less
rigid notion of information.

The preceding remarks justify that we orient ourselves on the search of a
definition of logical value which is more collective, i.e. concerns the whole
layer, and which moreover, gives an equal value to the truth and falsity, i.e.
which concerns decidalibity.

In what follows, we consider a network which has learned, i.e. the weights are
fixed, and collections of vectors are given in the input layers, one for
learning, one for testing, one for generalizing. Without contraindication, the
collection which is considered is the collection which was used for learning.

The intuition: the more the hidden layer can easily deduce the condition
$\displaystyle a$ in the set $\displaystyle A$ from its activity, the higher
its logical information quantity is.
Note this is the point of view of an observer on the layer, knowing the
receptive field of each neuron, i.e. the manner this neuron reacts to every
input during the learning, or testing or generalizing. This implies no
obligation for the network by itself, which is working as it prefers. Thus
this measure of information has to be completed by the analysis of the
weights, to pass from this inner layer to the following ones, and the manner
these weights take care of the logical content, in order to achieve the role
assigned to the networks, here in the final layers.

######  Remark 7.

In the next section, with a little more complex experiment, we will compare
statistically the weights and the logic, and we will see that they perfectly
agree, showing that the network elaborate proofs through the weights.

In more general contexts, the above three conditions are replaced by some
discrete variable of interest, in a given set, as in ordinary classification.
We describe them by the truth or no of some statement in a formal language
$\displaystyle\mathbb{L}$, i.e. under some declaration of types $\displaystyle
X,Y,...$ and variables $\displaystyle z_{X},z_{Y},...$, a proposition
$\displaystyle P$, and so on. The objective is to decide if yes or no a subset
of propositions is true or false.
In the simple example below, the propositions described the Boolean calculus
over $\displaystyle A_{3}$, with three elements.
The set of propositions of interest is supposed finite and closed by opposite,
it is written $\displaystyle\mathcal{P}$ and we want to know if they are true
or not, i.e. prove or disprove.

######  Remark.

This can be embedded in an intuitionist framework, because we are not forced
to ask that necessarily, $\displaystyle P\vee\neg P$ is true.

In the simplest example we considered below, $\displaystyle\mathcal{P}$ is the
whole algebra of subsets of $\displaystyle A_{3}$, except the empty set.

######  Remark.

If we include the conditioning by angular intervals, as does the model
$\displaystyle z$ with $\displaystyle L_{1}$, $\displaystyle L_{2}$ and even a
part of $\displaystyle L_{3}$, the information is always maximal. Then
something finer has to be taken in account, which is the economy of the
theory. Here, the restriction to the Boolean algebra plays this selective
role.

First we saw the important role played by a _quantization_ of the receptive
fields of the neurons. This is a non-trivial point, because discrete or
continuous is also a matter of observation, or level of description. Presently
we don’t enter into this difficulty, and we assume that for all the cells in
the layer, it is possible to decide if they have a good quantization or not,
in function of the concerned properties to be proved or disproved.
And for simplicity we assume the quantization is binary, $\displaystyle-1$ or
$\displaystyle 1$, as in the example. It is not a big difficulty to extend the
discussion to several disjoint intervals in $\displaystyle[-1,+1]$.
The decision of the cell is given by the number $\displaystyle+1$ or
$\displaystyle-1$. For the cells which are uncertain, we could add a
$\displaystyle 0$, but we will see in one minute how to give them an
information zero.
Letter $\displaystyle C$ denotes a set with elements $\displaystyle+1$ and
$\displaystyle-1$ and perhaps $\displaystyle 0$. *It will be integrated in the
formal language $\displaystyle\mathbb{L}$ by adding a type $\displaystyle C$.*
The quantized activities of the layer $\displaystyle L_{k}$ are represented by
a subset of the product $\displaystyle C^{n}$ when the layer contains
$\displaystyle n$ neurons.

Then we attribute to the layer $\displaystyle L_{k}$ a vector of propositions
$\displaystyle I_{k}$ which is made as follows:
for the neuron $\displaystyle y$, two propositions of the form $\displaystyle
1\Rightarrow P_{y}$ and $\displaystyle-1\Rightarrow Q_{y}$, in the language
$\displaystyle\mathbb{L}$, which could be noted $\displaystyle P^{+}_{y}$ and
$\displaystyle P^{-}_{y}$ respectively.
If the neuron is uncertain (i.e. telling uh or $\displaystyle 0$) we adopt the
convention that its $\displaystyle P_{y}=Q_{y}=\top$.
Each possible quantized activity $\displaystyle\varepsilon$ is described by a
vector of coordinates $\displaystyle\varepsilon_{y}$ in the set $\displaystyle
C={-1,0,1}$. Then it defines a family of propositions $\displaystyle
P^{\varepsilon}_{y},y\in L_{k}$. These propositions constitute $\displaystyle
I_{k}^{\varepsilon}$; they can be understood as the axioms of a theory
$\displaystyle\mathbb{T}^{\varepsilon}$.

At least three collections of theories are interesting to consider:

  1. 0)

the full family $\displaystyle\mathcal{T}^{0}_{k}$, corresponding to all the
vectors $\displaystyle\varepsilon$, without exception,

  2. 1)

the sub-family $\displaystyle\mathcal{T}^{1}_{k}$ made by the _consistent_
theories only (i.e. without contradiction),

  3. 2)

the sub-family $\displaystyle\mathcal{T}^{2}_{k}$ of the preceding
corresponding to the vector that can really happen in the layer, given the set
of vectors in the input layers (learning, testing or generalizing).

The second one is more convenient than the first, because inconsistence is not
comfortable, but is it preferable to the third one? The sets of inputs is
difficult to describe, however its properties are determinant in all the
applications, and the role of the network is to define (or extract) a
structure from the data, which allows it to generalize the efficient
functioning to other data. Moreover, we saw before that efficiency depends on
a set of data. Then it is probably much better to work with the third species
of sets of axioms.333if ”everything works as it should be”, the collection of
these families $\displaystyle\mathcal{T}_{k}$ for all the layers forms an
object in the topos of the network.

######  Definition 2.

The minimal logical information of the working layer $\displaystyle L_{k}$ is
the minimum of the ratio between the number of propositions that can be
logically deduced from the axioms of any theory
$\displaystyle\mathbb{T}^{\varepsilon}_{k}$ belonging to
$\displaystyle\mathcal{T}^{2}_{k}$, and the cardinal of the consequences of
the wanted propositions.

In our simplest example, we can compare the input layer with the inner layer,
and also compare the different models, $\displaystyle z$ versus $\displaystyle
z^{\prime}$, then two hidden layers versus one, then three layers versus two.
From the above discussions, they are evidently disposed in a growing order of
logical information. (The only ambiguous case is $\displaystyle z$ with a too
fat $\displaystyle L_{3}$, which is difficult to compare with the model
$\displaystyle z^{\prime}$, because they don’t have the same defects.)

As the above propositions $\displaystyle 2,3,4,5$ show, it would be nice to
have a more localized notion of information, involving the set of subsets of
$\displaystyle L_{k}$. (We will introduce such a definition below, involving
together logic and probability.)
In reality, the practical information value is not only given by the whole
collection of theories, for instance the set
$\displaystyle\mathcal{T}^{2}_{k}$, it must also take in account the
collection of demonstrations of the propositions of interest or their
opposite, starting from the concrete axioms $\displaystyle
P^{\varepsilon}_{y},y\in L_{k}$, and measure their difficulty, number of
branches of trees in a proof, and number of useful initial propositions, then
here, the number of cells which are involved comes into the play.
A kind of Galois theory could exist in this context, describing how these sets
and proofs are changed by adding (or deleting) a certain set of logical cells.
This has to do with the stability of the deduction, which has also its
importance, practical and theoretical. Two aspects of stability appear
naturally:

  1. 1)

the deletion or dysfunction of few cells can destroy the information value or
not;

  2. 2)

the function can be easily recovered or not by a few re-learning or training.

These two aspects being probably not independent.

##  3 Simple networks for doing predicate calculus

In this second experiment, we test the hypothesis that a simple network of few
layers is also able to decide between propositions involving existential and
universal quantifiers, $\displaystyle\exists$, $\displaystyle\forall$, and
when doing that, spontaneously constructs cells that perform logical analysis
in the inner layers, having the same kinds of properties than the logical
cells of section 2: they introduce discrete responses, in such a manner that
spiking or not spiking implies propositions, from which it is possible to
answer the final question by logical deductions.

The goal of the DNN in this case is to recognize if two different objects are
disjoint, intersecting or in relation of inclusion. It can be images with two
colored rigid objects in a space, or two different voices pronouncing
sentences.
In the spatial case, the input data are collections of dots in two possible
colors, in the temporal one they are collections of sounds in two possible
timbers.
A first layer $\displaystyle HL_{0}$ is made of neurons detecting only one
color, or only one timber, a kind of transducer.

Our experiment shows that a fairly successful network exists with only one
hidden layer, but for observing logical cells, and for the aptitude of
generalization, the network must have at least two or three layers, depending
on the nature of the space or time interval.

###  3.1 A first experiment in two forms

####  3.1.1 The experimental setting

For simplicity we speak of images and colors, named Red and Green, and we
consider only homogeneous one dimensional spaces $\displaystyle D$: one is a
circle, one is a segment; both have a discretization of the order of
$\displaystyle 100$ dots or unit segments (See Figures 9(a) and 9(b)).

(a) Experiment on a line

(b) Experiment on a circle

Figure 9: Input Layer for $\displaystyle 3-$bar experiments

Rigidity of the objects $\displaystyle R$ and $\displaystyle G$ means that,
for each image and each color the dots form a segment of constant length.
These lengths are of the order of $\displaystyle 3$ or $\displaystyle 5$ out
of $\displaystyle 18$ in the linear case, as in the circular case.

Important: in this first experiment, we do not change the length of the
objects, then one of them, say the red, can never be included into the other,
say the green, but the green object can be strictly included into the red one.

Figure 10: Architecture of the network for the predicate experiments

As it is illustrated in Figure 10, a first layer $\displaystyle HL_{0}$
contains two families of neurons, each one having $\displaystyle 55$ elements
(a number coming from the first experiment) for detecting one of the two
colors. By convention, we say they detect red or green dots. This layer is not
considered as a hidden layer, it performs a transduction, as the cones do in
the retina.
We considered two types of receptors, the simple one is Gaussian or wrapped
Gaussian (theta distribution), the complex one is made by a difference of two
Gaussian curves or wrapped Gaussian curves, introducing a negative answer when
the color disappears of the receptive field, then detecting the contrast. The
type of receptor influences the learning and the generalization, but it
appeared that the simple one has better performance.

######  Remark.

The addition of a layer which locally combines the activities of
$\displaystyle HL_{0}$ has a negative effect as well.

The first truly hidden layer $\displaystyle HL_{1}$ has $\displaystyle 55$
cells (to have the possibility of comparing with section 2).
When we add a second hidden layer $\displaystyle HL_{2}$, it will have
$\displaystyle 50$ neurons, and when we add a third one $\displaystyle HL_{3}$
it will have $\displaystyle 25$ neurons, except with some mentioned
exception). See Figure 10 for an illustration.

The last layer has three neurons, named $\displaystyle D$ for disjunction,
$\displaystyle IO$ for intersection without inclusion (intersection only) and
$\displaystyle II$ for inclusion.

The mapping (to be learned) from a layer to the next one, is of the type
$\displaystyle\tanh(C\sum x)$, with $\displaystyle C$ equals $\displaystyle 1$
or $\displaystyle 2$. In all cases $\displaystyle C=2$ gives better results.
Between the last hidden layer and the output layer, we choose a linear
mapping, without using the $\displaystyle\tanh$ activation function, and to
normalize the result as a probability, adapted to the cross entropy. The main
reason is that this gives better results.

A metric was chosen on the last layer, for measuring the accuracy of the
answer, then for the training phase. The choice of this metric has a strong
influence on the results.
We already saw this point in the first experiment, but in this case, to ontain
a good performance, either in accuracy either for the logical behavior, it is
not sufficient to respect the symmetry between the three points, for instance
by using a two dimensional coding. In fact all the results below need the use
of the cross-entropy, i.e. the Kullback-Leibler distance between the feedback
normalized beliefs (given by the network) in the three options and the right
one (non random). Thus the quantity to minimize is $\displaystyle-\log p_{i}$
for the condition $\displaystyle i$; $\displaystyle i=1,2,3$ for
$\displaystyle D,IO,II$ respectively.
This metric is known to improve most classification problems [GKS17]; it is
particularly adapted to connecting semantic information and statistical
information.

In both the linear and circular cases, a dramatic improvement from a
quantitative point of view appeared with two hidden layers instead of one. IT
is the case for both the minimal loss function after training and the number
of residual errors. Note that this last number becomes stable with two hidden
layers (around $\displaystyle 1/100$ for the circular case and for the linear
case). We will discuss this limit later, however it obviously represents the
limited precision of the receptive fields, which makes them unable in many
case to distinguish the intersection or the inclusion versus intersection
only, when the boundaries of the objects are close. But it was not the goal of
this study to improve the performance in this direction.

####  3.1.2 Theory, predicative cells, conclusive or not

In order to describe logical cells, we constructed for each neuron
$\displaystyle i$, a receptive field to an intermediate proposition
$\displaystyle P$.

  1. 1.

For local propositions : for every point $\displaystyle a\in D$, the
distribution of the responses of the cell $\displaystyle i$, when Green and
Red appear together at the position $\displaystyle a$, noted $\displaystyle
R(a)\wedge G(a)$, the same for $\displaystyle R$ but not $\displaystyle G$
appearing in $\displaystyle a$, denoted $\displaystyle R(a)\wedge\neg G(a)$,
the same for $\displaystyle G$ but not $\displaystyle R$ appearing in
$\displaystyle a$, noted $\displaystyle G(a)\wedge\neg R(a)$, and finally the
same for $\displaystyle\neg R(a)\wedge\neg G(a)$.

  2. 2.

We also consider global propositions, describing the reaction of the cell
$\displaystyle i$ when the presented objects are disjoint (condition
$\displaystyle D$), when they intersect without inclusion (condition
$\displaystyle IO$, intersection only) and when the green object is included
in the red one (condition $\displaystyle II$).
Proposition $\displaystyle D$ is $\displaystyle\forall x\in D,\neg(R(x)\wedge
G(x))$.
Proposition $\displaystyle II$ is $\displaystyle\forall x\in D,G(x)\Rightarrow
R(x)$.
And proposition $\displaystyle IO$ is $\displaystyle(\exists x\in D,G(x)\wedge
R(x))\wedge(\exists y\in D,G(y)\wedge\neg R(y))$.

All these distributions were represented by a color code (rectangles for the
local questions, segments for the global ones), blue for $\displaystyle-1$,
red for $\displaystyle+1$, and barycenters of the colors for activity in
between $\displaystyle-1$ and $\displaystyle+1$ (see Figure 11).

(a) $\displaystyle HL_{3}$ Cell 25 - Activity Distribution

(b) $\displaystyle HL_{3}$ Cell 25 - View

Figure 11: Cell $\displaystyle 25$ in layer $\displaystyle HL_{3}$

In addition, we took advantage from the fact that the input image can be fully
described by two bounded real parameters, which are the positions of the two
centers of the intervals. In the circular case they are two angles, giving a
point in a flat torus; in the linear case, this gives a point in a square.
Therefore the complete activity of the cell as a function of the input, can be
represented by a colored square, with the above color code. We called this
representation the _raw activity_ of the individual cell (see Figure 12 for
the linear case).

Figure 12: Linear case - $\displaystyle HL_{3}$ Cell 2 - Raw Activity

It appears that the most readable representation is by far the raw activity,
but the other representations give a finer idea of the variability and allow
to confirm what appears on the raw activity.

######  Remark.

All results are very noisy with only one hidden layer $\displaystyle HL_{1}$,
but become very intelligible with two hidden layers.

In the circular case, we can see a Fourier analysis on the torus in
$\displaystyle HL_{1}$, as it is illustrated in Figures 13(a) and 13(b), which
is pursued in part in $\displaystyle HL_{2}$, where it also appears almost
discretized cells for the three propositions $\displaystyle D$, $\displaystyle
II$ and $\displaystyle IO$. In $\displaystyle HL_{1}$ the Fourier analysis is
made separately on the two middle angles $\displaystyle\theta_{R},\theta_{G}$,
but in $\displaystyle HL_{2}$ the analysis is done with respect to the phase
difference $\displaystyle\theta_{R}-\theta_{G}$, which is an evident
progression with respect to the logic, allowing the individual cells to
represent the characteristic functions of the objectives.
For the linear segment case, we got the same behavior, with a kind of partial
Fourier analysis corresponding to the action of
$\displaystyle\mathbb{Z}/2\mathbb{Z}$ by symmetry around the middle of the
full segment, but we observed an important difference with respect to the
circular network, because, with two hidden layers, no cell corresponds to
$\displaystyle IO$.

(a) Fourier analysis of cell 15

(b) Fourier analysis of cell 37

Figure 13: Fourier analysis of $\displaystyle HL_{1}$ in the circular case

We concluded that the $\displaystyle IO$ cells in $\displaystyle HL_{2}$ for
the circular case with two hidden layers is almost certainly a consequence of
the good approximation of the characteristic function of this proposition by
an harmonic of degree $\displaystyle 3$.

What is amazing, is that the Fourier analysis completely vanishes when we
introduce a third hidden layer $\displaystyle HL_{3}$, a consequence is the
total disappearance of cells interested in $\displaystyle IO$, intersection
only. This is true in both the circular and linear case.
We conclude that, with three hidden layers, something in the training appears,
which allows neurons to get access to higher frequencies, for representing
step functions, that where too difficult to represent with the first harmonics
of the Fourier analysis.
With three hidden layers, the cells in $\displaystyle HL_{2}$ are still
affected by noise but fully intelligible for the two conditions $\displaystyle
D$ and $\displaystyle II$. In $\displaystyle HL_{3}$ the representation is
perfect, fully quantized at $\displaystyle-1$ and $\displaystyle+1$, without
any cell $\displaystyle IO$. We say that a cell is of type $\displaystyle D$,
resp. $\displaystyle II$, resp. $\displaystyle IO$, if conditioned by an input
of the respective type it tells $\displaystyle\pm 1$ and conditioned by an
input of the two other types, it tells $\displaystyle\mp 1$.
A consequence is a nice predicative logic: with one $\displaystyle D$ cell and
one $\displaystyle II$ cell, the conclusion of the output is accessible.
Suppose for simplicity that both cells prefer $\displaystyle+1$ for their
respective type, then the pairs of possible activities $\displaystyle(+1,-1)$,
$\displaystyle(-1,-1)$, $\displaystyle(-1,-1)$ correspond respectively to a
prediction $\displaystyle D$, $\displaystyle II$, $\displaystyle IO$.

The fact that the cells quantize at two opposite values has the consequences
that the receptive fields for the propositions $\displaystyle G(a)\wedge R(a)$
and $\displaystyle\neg R(a)\wedge G(a)$ saturate at $\displaystyle-1$ or
$\displaystyle+1$, the first one corresponding to the condition $\displaystyle
D$, the second one to the condition $\displaystyle II$.

In numbers:

  1. 1.

in the circular case, with two layers, in $\displaystyle HL_{1}$ it is
difficult to detect a logical functioning, but in $\displaystyle HL_{2}$, with
$\displaystyle 50$ neurons, the Fourier analysis in
$\displaystyle\theta_{R}-\theta_{G}$ reconstructs fairly good raw activities,
giving $\displaystyle 20$ cells of type $\displaystyle D$, $\displaystyle 15$
cells of type $\displaystyle II$ and $\displaystyle 13$ cells of type
$\displaystyle IO$, and two strange cells. Cf. Figures …
We tried also with a second layer of $\displaystyle 25$ neurons and got the
same kind of spectrum: $\displaystyle 6$ cells $\displaystyle D$,
$\displaystyle 8$ cells $\displaystyle II$ and $\displaystyle 11$ cells
$\displaystyle IO$.

  2. 2.

With three hidden layers, $\displaystyle HL_{2}$ seems at first sight to
resemble the preceding, but it contains no cell of type $\displaystyle IO$,
$\displaystyle 26$ cells $\displaystyle D$ and $\displaystyle 24$ cells
$\displaystyle II$. The layer $\displaystyle HL_{3}$ develops an impressive
quantization, giving $\displaystyle 15$ cells $\displaystyle D$ and
$\displaystyle 10$ cells $\displaystyle II$.

  3. 3.

In the linear case, things are less imaginative, certainly because a
continuous Fourier analysis is missing.
With two layers, in $\displaystyle HL_{1}$ we recognize $\displaystyle 18$
cells corresponding to an asymmetric representation of
$\displaystyle\mathbb{Z}/2\mathbb{Z}$ (Figure 14), and $\displaystyle 32$
cells to a symmetric one (Figure 15). In hidden layer $\displaystyle HL_{2}$,
$\displaystyle 21$ $\displaystyle D$ cells, $\displaystyle 22$ cells
$\displaystyle II$, $\displaystyle 4$ cells Fourier symmetric, $\displaystyle
1$ asymmetric, and $\displaystyle 1$ not interpretable for us.

  4. 4.

With three hidden layers, in $\displaystyle HL_{2}$ we found $\displaystyle
17$ $\displaystyle D$ cells, $\displaystyle 22$ $\displaystyle II$ cells,
$\displaystyle 5$ symmetric Fourier cells, $\displaystyle 6$ asymmetric ones,
and in $\displaystyle HL_{3}$, no Fourier cell, $\displaystyle 15$ cells of
type $\displaystyle D$ (Figure 16(a)) and $\displaystyle 11$ of type
$\displaystyle II$ (Figure 16(a)).

Figure 14: Fourier analysis on $\displaystyle\mathbb{Z}/2\mathbb{Z}$: the
asymmetric case

Figure 15: Fourier analysis on $\displaystyle\mathbb{Z}/2\mathbb{Z}$: the
symmetric case

(a) $\displaystyle HL_{3}$ \- Cell of type $\displaystyle II$

(b) $\displaystyle HL_{3}$ \- Cell of type $\displaystyle D$

Figure 16: $\displaystyle 3$ hidden layers - $\displaystyle HL_{3}$

Important remark: the distribution of preference does not reflect at all the
statistics of the imputs, which are apparent on the raw graphics. In fact, the
number of $\displaystyle D-$cases is larger than the number of $\displaystyle
IO-$cases, which is much lager than the number of $\displaystyle II-$cases.
There is a tendency to the equilibrium between $\displaystyle II$ and
$\displaystyle D$. We plan to study the variability of these populations in a
further study.

We saw, in this experiment, which was expected to test predicate logic, the
same kind of results we saw in the first experiment, with ordinary
propositional calculus. Clearly, the Boolean logic at the output dominates.
And we could conjecture that this will happen in any classification problem.
However, this is not the full picture. This is clear for the condition
$\displaystyle IO$, because its ”mature” treatment relies on an indirect
logical reasoning, the proposition never being directly accessible. This
reminds the hidden predicate calculus, where the expression of this
proposition is twice more complex than the other ones, $\displaystyle D$ and
$\displaystyle II$.
Of course, we must be conscious that the above networks learn by minimization
of a certain functional $\displaystyle F$. They learn to approximate the
desired responses, and all that is fully supervised. The logic, at least,
results from the analytical properties of $\displaystyle F$ and the nature of
the images. Therefore we can interpret the difficulty of accessing
$\displaystyle IO$ by the difficulty to represent a characteristic function
which is more complex than the other ones; this complexity is precisely the
complexity of the logical formula in predicate calculus. However, we will see
below that a bit more solicited network can success in representing sometimes
this function for $\displaystyle IO$, and we never see a mixture between it
and the characteristic functions of $\displaystyle II$ or $\displaystyle D$.
It agrees with the quantization, and perhaps depends on it. The combination is
made by logics, not by interpolation. Thus we can at least suggest that the
minimum of the functional $\displaystyle F$ with $\displaystyle 3$ layers and
more, has a logical flavor, and that this logic is influenced both by the
output classification and by the predicative formulas with respect to the
localized input, i.e. the composite nature of the objects in the scene, and
the composite nature of their reciprocal relations.

####  3.1.3 Tests of generalization

The ability to generalize was tested on three different sets of data: changing
the lengths from $\displaystyle 3$ and $\displaystyle 5$ to $\displaystyle 4$
and $\displaystyle 6$ out of $\displaystyle 18$ with and without an invertion
of the colors, and a last one, just exchanging the colors without changing the
lengths.
The results were not so bad: $\displaystyle 12\%$ of errors for the change of
lengths respecting the colors, and the same for the exchange of colors without
changing the lengths, but surprisingly only $\displaystyle 8\%$ of error for
both the change of length and the exchange of colors. This is as if the
network were more perturbed when correlations that it had established by
itself are violated in the new data.

Important: in all cases the raw activities are similar to the raw activities
without generalization, only slightly deformed. However, very nicely, when the
small and large bars changed their color in the generalization test with
respect to the learning condition, the cells that were attached to saturation
of $\displaystyle R\wedge\neg G$ (resp. $\displaystyle G\wedge\neg R$) became
almost saturated for the other, $\displaystyle G\wedge\neg R$ (resp.
$\displaystyle R\wedge\neg G$). In fact these cells had no notion of color,
they were interested by the comparison of lengths, and the local property
$\displaystyle S(a)\wedge\neg L(a)$, $\displaystyle S$ for small and
$\displaystyle L$ for large.

Therefore, the network gave the impression that it had understood by itself
that what is important, to decide about the inclusion or not, was to
distinguish the short object from the large one!

This was a reason to modify the problem, with data of several lengths and
inverting the colors.

###  3.2 A second experiment with a richer learning

In the above experiment, the length of the objects did not vary, and we saw a
not too bad but limited capacity of generalization. We decide, then, to
consider a larger collection of images (or conversations between two persons)
where the objects in red or green (or the sentences) can change their lengths.
The conjecture is that the network will also succeed, constructing by itself
in $\displaystyle HL_{3}$ predicative cells of different types, able to
conclude by logical proofs.
In this set of experiments as well, we consider colored objects in a one
dimensional space. And we present the results for the linear segment, not the
circle.

The experimental setting is the same as before (see Figure 10), a layer
$\displaystyle HL_{0}$ with two populations of small Gaussian sensors, one for
green and the second one for red; then a layer $\displaystyle HL_{1}$ with
$\displaystyle 55$ cells, a layer $\displaystyle HL_{2}$ with $\displaystyle
50$ cells, a layer $\displaystyle HL_{3}$ with $\displaystyle 25$ cells. The
activation function is $\displaystyle\tanh(C\sum X)$, with $\displaystyle C=1$
or $\displaystyle C=2$, with the mentioned exception of the mapping from
$\displaystyle HL_{3}$ to the output $\displaystyle HL_{4}$, which is linear
and normalized as a probality. The overall metric is the cross-entropy. In the
back-propagation algorithms, we vary the sizes of batches and the number of
iterations.
However, the input images now contain bars of several lengths, for instance
$\displaystyle 2,4,6,8$ or $\displaystyle 3,5,7$ of both colors out of
$\displaystyle 23$ units for the diameter of the space. Discretization is
still $\displaystyle 100$ for the total space.

######  Remark.

A priori, coincidence of the objects in space can happen. However, we observed
that the network was unable to detect this case, when we included it in the
objectives. We supposed that this was due to the few data where coincidence
occurs. We then repeat the coincidences in such a manner that it happens as
often as other situations. The result is good for the accuracy, but mostly
destroys the logical behavior. Therefore we decided to avoid coincidences. We
could have decided to include them without asking about them, but in this
case, the questions about the inclusion of red in green or the converse would
have conflicted.

We asked four final questions: $\displaystyle D$ disjunction, $\displaystyle
IO\equiv(G\cap R\neq\emptyset)\wedge(G\cap R^{c}\neq\emptyset)$,
$\displaystyle II_{R}\equiv R\subsetneq G$, $\displaystyle II_{G}\equiv
G\subsetneq R$.
We did that with four lengths, $\displaystyle 2,4,6,8$ for the two objects,
and tested for $\displaystyle 3,5,7$ for both (out of a total length
$\displaystyle 23$), with the same discretization as before, $\displaystyle
100$.
The network in this case gave the best results we got from the beginning, with
a testing as good as the training, around $\displaystyle 1\%$ accuracy.
Moreover, the predicative cells were excellent, showing a low variability in
$\displaystyle HL_{2}$ and being very well quantized in $\displaystyle
HL_{3}$, corresponding to some partitions of the following set of ten
propositions $\displaystyle D\wedge(L=R)$, $\displaystyle II\wedge(L=R)$,
$\displaystyle(R\subset G)\wedge(L=R)$, $\displaystyle(G\subset
R)\wedge(L=R)$, $\displaystyle D\wedge(L=G)$, $\displaystyle II\wedge(L=G)$,
$\displaystyle(R\subset G)\wedge(L=G)$, $\displaystyle(G\subset
R)\wedge(L=G)$, and surprisingly $\displaystyle IO\wedge(L=R)$ and
$\displaystyle IO\wedge(L=G)$.
This means that the network has understood the existence of red and green
objects, not only the fact that one is small and the other one is large.

However, and it is a fundamental result: all the propositions that were
decided (proved or disproved) by the individual cells in this experiment (as
in the other one) belong to the algebra generated by the four propositions
forming the objectives: $\displaystyle D$, $\displaystyle IO$, $\displaystyle
R\subset G$, $\displaystyle G\subset R$.

In numbers: In $\displaystyle HL_{2}$, containing $\displaystyle 50$ cells, we
observed $\displaystyle 20$ cells of type $\displaystyle D$, $\displaystyle 6$
of type $\displaystyle II=(G\subset R)\vee(R\subset G)$, $\displaystyle 7$ of
type $\displaystyle R\subset G$, $\displaystyle 13$ of type $\displaystyle
G\subset R$, $\displaystyle 1$ of type $\displaystyle IO$, (but vaguely) and
three cells difficult to interpret, being saturated at one value or perhaps
doing Fourier analysis on $\displaystyle\mathbb{Z}/2\mathbb{Z}$.
In $\displaystyle HL_{3}$, we observed $\displaystyle 9$ $\displaystyle D$
cells, $\displaystyle 5$ $\displaystyle II$ cells, $\displaystyle 0$
$\displaystyle IO$ cells, $\displaystyle 3$ cells $\displaystyle R\subset G$,
$\displaystyle 6$ cells $\displaystyle G\subset R$, no bizarre cells, but a
wonderful pair of cells, number $\displaystyle 1$ and $\displaystyle 25$ (See
Figure 17):
cell $\displaystyle 1$ tells $\displaystyle+1$ when $\displaystyle L=R$ and
$\displaystyle\neg D$ are true or when $\displaystyle L=G$ and $\displaystyle
IO$ (here very net) are true, it tells $\displaystyle-1$ when $\displaystyle
L=R$ and $\displaystyle D$ are true or when $\displaystyle L=G$ and
$\displaystyle\neg IO$ are true, then when it spikes at $\displaystyle+1$ we
know $\displaystyle(IO\wedge(L=G))\vee(L=R)\wedge((IO)\vee(G\subset R))$ which
is equivalent to $\displaystyle IO\vee(G\subset R)$, and when it spikes at
$\displaystyle-1$, we know
$\displaystyle(D\wedge(L=R))\vee(L=G)\wedge(D\vee(R\subset G))$, which is
equivalent to $\displaystyle D\vee(R\subset G)$; the cell $\displaystyle 25$
does the analog, but exchanging $\displaystyle+1$ with $\displaystyle-1$
(which has no importance) and $\displaystyle R$ and $\displaystyle G$, which
has an importance, because the conclusion of its ”reasoning” is as follows:
$\displaystyle+1$ implies $\displaystyle D\vee(G\subset R)$, $\displaystyle-1$
implies $\displaystyle IO\vee(R\subset G)$.
Thus, considering the pair of neurons $\displaystyle 1$, $\displaystyle 25$,
we have $\displaystyle(+1,+1)$ implies $\displaystyle G\subset R$,
$\displaystyle(+1,-1)$ implies $\displaystyle IO$, $\displaystyle(-1,+1)$
implies $\displaystyle D$ and $\displaystyle(-1,-1)$ implies $\displaystyle
R\subset G$. This pair completely solves the classification problem.

(a) $\displaystyle HL_{3}$ \- Cell $\displaystyle 1$

(b) $\displaystyle HL_{3}$ \- Cell $\displaystyle 25$

Figure 17: Raw activities of the exceptional pair of cells in $\displaystyle
HL_{3}$ Figure 18: $\displaystyle HL_{2}$ \- Cell $\displaystyle 30$

We also analyzed two less complex problems:

  1. 1.

two bars of lengths $\displaystyle 3,5$ exchanging the colors $\displaystyle
R$ and $\displaystyle G$ one time over two, during learning. This learned
well, showed logic, but nothing remarkable: in $\displaystyle HL_{2}$, not too
noisy, $\displaystyle 18$ cells $\displaystyle D$, $\displaystyle 18$ cells
$\displaystyle II$, no cell $\displaystyle IO$, $\displaystyle 8$ cells
$\displaystyle II_{R}$, $\displaystyle 4$ cells $\displaystyle II_{G}$, and
$\displaystyle 2$ bizarre cells. In $\displaystyle HL_{3}$, $\displaystyle 13$
cells $\displaystyle D$, $\displaystyle 7$ cells $\displaystyle II$,
$\displaystyle 2$ cells $\displaystyle II_{R}$, $\displaystyle 3$ cells
$\displaystyle II_{G}$, nothing for $\displaystyle IO$.

  2. 2.

two bars of varying lengths $\displaystyle 2,4,6,8$ during learning, and
$\displaystyle 3,5,7$ for testing, but the red bar being always longer than
the green one. Again the result was good an logic, but without surprise in
$\displaystyle HL_{3}$: in $\displaystyle HL_{2}$, $\displaystyle 19$ cells
$\displaystyle D$, $\displaystyle 18$ cells $\displaystyle II$, $\displaystyle
12$ cells doing a sort of Fourier analysis ($\displaystyle 9$ symmetric,
$\displaystyle 3$ asymmetric), plus one interesting amazing cell, number
$\displaystyle 30$ telling something about $\displaystyle IO$, but partly
localized (see Figure 18): $\displaystyle-1$ tells $\displaystyle\neg II$,
$\displaystyle+1$ tells $\displaystyle\neg IO$, however the message is noisy.
In $\displaystyle HL_{3}$, $\displaystyle 15$ cells $\displaystyle D$ and
$\displaystyle 10$ cells $\displaystyle II$, well quantized.

###  3.3 A third experiment, the blue object incoming

We decided to progress towards future experiments where the network could
generalize in a much wider sense, not behaving like an interpolator, but
showing an understanding of what a new object (or a third voice) is and
analyzing its properties by itself, by analogy with the preceding questions
and the generated internal cells. We think that this would probably require a
change of architecture, but we started studying what kind of cells may appear
in our simple architecture in presence of a new object.

Two objects are presented within the line of length $\displaystyle 23$, a red
one of length $\displaystyle 6$ and a green one of length $\displaystyle 5$.
However, in half of the cases, a blue object of length $\displaystyle 3$
appears. It is detected by $\displaystyle 55$ cells reacting to the blue color
somewhere.
Again three layers, with six questions at output: $\displaystyle D$,
$\displaystyle II$ and $\displaystyle IO$ regarding $\displaystyle R$ and
$\displaystyle G$, conditioned by $\displaystyle B$ or $\displaystyle\neg B$.
We note $\displaystyle G=S$ (for small) and $\displaystyle R=L$ (for large).
The questions asked inside are as before, except they are doubled by the
conditioning $\displaystyle B$ or $\displaystyle\neg B$.
The results are excellent, both for the quality of learning and the very few
errors made.

(a) $\displaystyle HL_{2}$ \- Cell $\displaystyle 41$

(b) $\displaystyle HL_{2}$ \- Cell $\displaystyle 44$

Figure 19: Log activities of the exceptional cells in $\displaystyle HL_{2}$

(a) $\displaystyle HL_{3}$ \- Cell $\displaystyle 6$

(b) $\displaystyle HL_{3}$ \- Cell $\displaystyle 11$

Figure 20: Log activities of saturated cells in $\displaystyle HL_{3}$

In $\displaystyle HL_{2}$, almost all cells saturate at two values and all of
these ones express logical proposition in the Boolean algebra of the
objectives, the most numerous population reacts at the presence or absence of
$\displaystyle B$ (they are $\displaystyle 15$, then very redundant), after
that comes the population of $\displaystyle D$ cells conditioned by
$\displaystyle B$ (they are $\displaystyle 9$ and $\displaystyle D$ cells
without conditioning ($\displaystyle 7$ cells, with the same activity for
$\displaystyle B$ and $\displaystyle\neg B$, they could have exchanged, but
doing that they were not in the ”final algebra”!), then the $\displaystyle D$
cells conditioned by $\displaystyle\neg B$ ($\displaystyle 6$ cells), and the
$\displaystyle II$ cells without conditioning (also $\displaystyle 6$ cells,
same sign for $\displaystyle B$ and $\displaystyle\neg B$), then only one
$\displaystyle II$ cell conditioned by $\displaystyle B$ (why so few? it is
mysterious), two apparently uninformative cells saturated at one value, and
two exceptional cell, number $\displaystyle 41$ and $\displaystyle 44$ (see
Figure 19), which are of type $\displaystyle II$ for $\displaystyle\neg B$ and
of type $\displaystyle D$ for $\displaystyle B$. Each of these cells tells
$\displaystyle+1$ when $\displaystyle\neg B\wedge II$ or $\displaystyle
B\wedge D$, and tells $\displaystyle-1$ when $\displaystyle\neg B\wedge\neg
II$ or $\displaystyle B\wedge\neg D$. Therefore it gives the following axioms

$\displaystyle\displaystyle P_{+}$ $\displaystyle\displaystyle=(\neg
B,II)\vee(B,D);$ $\displaystyle\displaystyle P_{-}$
$\displaystyle\displaystyle=(\neg B,IO)\vee(\neg
B,D)\vee(B,IO)\vee(B,II)=IO\vee(\neg B,D)\vee(B,II).$

Remarkably, two such cells with exactly the same message, but a much better
saturation, appears in $\displaystyle HL_{3}$, the ones numbered
$\displaystyle 6$ and $\displaystyle 11$ (See Figure 20). The above preferred
propositions, $\displaystyle P_{-}$ being the closest to $\displaystyle IO$ in
this experiment, also belong to the final output algebra.
In addition, in $\displaystyle HL_{3}$ we found, $\displaystyle 5$ color cells
(telling if $\displaystyle B$ is here or not), $\displaystyle 5$ pure
$\displaystyle D$ cells, $\displaystyle 1$ pure $\displaystyle II$ cell,
$\displaystyle 4$ $\displaystyle D$ cells conditioned by $\displaystyle B$,
$\displaystyle 2$ $\displaystyle D$ cells conditioned by $\displaystyle\neg
B$, $\displaystyle 4$ $\displaystyle II$ cells conditioned by $\displaystyle
B$, $\displaystyle 2$ $\displaystyle II$ cells conditioned by
$\displaystyle\neg B$, and last but not least the two above mentioned original
cells.

Exercise: analyze the possibility of proofs starting with the axioms of the
most economical groups of neurons in these populations.

All the cells we observed in all the experiments quantize only for unions of
propositions belonging to the objectives. Only the individual choices and the
possibility of proofs they offer manifest inventions.
For instance, the network with the richest learning, with the four objectives
$\displaystyle D$, $\displaystyle IO$, $\displaystyle II_{R}$, $\displaystyle
II_{G}$ could have developed cells of type $\displaystyle L=R$ versus
$\displaystyle L=G$. This proposition does not belong to the output algebra,
but without any doubt, it contains useful information in order to conclude;
for instance, with a $\displaystyle II$ cell, we immediately conclude if it is
$\displaystyle II_{R}$ or $\displaystyle II_{G}$ which is true.
Therefore to go further, we have to consider ”objectives propositions” that do
not form necessarily a classification task.

###  3.4 The important role of symmetries

All periodic translations (i.e. rotations) in the circular case, plus the
reflections in any pair of antipodal points form the group of isometries of
the circle, which can easily be quantized and can be identified with a
dihedral group $\displaystyle D_{N}$ (we chose $\displaystyle N=100$) which
approximates the group of the isometries of the circle $\displaystyle
O_{2}(\mathbb{R})$. The irreducible linear representations are of dimension
$\displaystyle 2$ and can be identified with the vector space $\displaystyle
V_{n}$ of linear combinations of $\displaystyle\cos nx$ and $\displaystyle\sin
nx$, for $\displaystyle n\in\mathbb{N}$, named harmonics of degree
$\displaystyle n$.
Remarkably, the first network of two layers made an pertinent Fourier
analysis: in order to discriminate the green from the blue, it possesses six
cells of degree $\displaystyle n=1$. Their phases follow a uniform
distribution. Moreover, most of the blue cells correspond to second degree
harmonics, and several cells to degree $\displaystyle 3$ (See Figure 21).

(a) $\displaystyle HL_{2}$ \- Cell $\displaystyle 4$

(b) $\displaystyle HL_{2}$ \- Cell $\displaystyle 5$

(c) $\displaystyle HL_{2}$ \- Cell $\displaystyle 7$

(d) $\displaystyle HL_{2}$ \- Cell $\displaystyle 27$

(e) $\displaystyle HL_{2}$ \- Cell $\displaystyle 35$

(f) $\displaystyle HL_{2}$ \- Cell $\displaystyle 47$

Figure 21: $\displaystyle HL_{2}$ \- $\displaystyle R\wedge G$: Blue;
$\displaystyle R\wedge\neg G$: Red; $\displaystyle\neg R\wedge G$: Green;
$\displaystyle\neg R\wedge\neg G$: Black

Moreover many curves are symmetric with respect to an axis, expressing an
interest for the reflections.
With three hidden layers, this spontaneous use of Fourier analysis becomes
less evident in $\displaystyle HL_{2}$, and is replaced by invariants under
$\displaystyle D_{100}$ in $\displaystyle HL_{3}$.

In the linear case, where $\displaystyle D$ is a finite segment, there is only
one non-trivial symmetry, the reflection with respect to the middle, or mirror
symmetry. This implies that, with one hidden layer, most of the propositional
curves are anti-symmetric with respect to zero, the center of the segment (See
Figure 22).

(a) $\displaystyle HL_{2}$ \- Cell $\displaystyle 3$ \- Linear

(b) $\displaystyle HL_{2}$ \- Cell $\displaystyle 4$ \- Linear

(c) $\displaystyle HL_{2}$ \- Cell $\displaystyle 5$ \- Linear

(d) $\displaystyle HL_{2}$ \- Cell $\displaystyle 11$ \- Linear

(e) $\displaystyle HL_{2}$ \- Cell $\displaystyle 22$ \- Linear

(f) $\displaystyle HL_{2}$ \- Cell $\displaystyle 31$ \- Linear

Figure 22: $\displaystyle HL_{2}$ \- $\displaystyle R\wedge G$: Blue;
$\displaystyle R\wedge\neg G$: Red; $\displaystyle\neg R\wedge G$: Green;
$\displaystyle\neg R\wedge\neg G$: Black

For two hidden layers, in $\displaystyle HL_{2}$, most of these curves are
symmetric with respect to the vertical axis. This corresponds to the two
simplest representations of the cyclic group $\displaystyle C_{2}$. With three
layers all the cells, with two exceptions, are predicative, with almost all
curves horizontal, thus in the trivial representation of $\displaystyle
C_{2}$.

With only two layers, the network was a little bit more successful in the
linear case; maybe in this case, Fourier analysis on the circle is more
disturbing than helpful.

Then we notice that the network takes great care of the symmetries in the
data. Moreover it is remarkable that by itself with so few cells, it does a
generalized Fourier analysis, using the simplest irreducible linear
representations of the group of symmetries.

Also, remind that in the first experiment, the simple fact that the metric at
the end was changed in order to respect the symmetries in the logic of the
questions asked, was determinant for the success of the network.

##### Toposic interpretation (not necessary to read)

In toposic terms, this corresponds to the importance of the groupoid attached
to each layer. This groupoid takes in account the symmetries in the data and
the semantic of the questions. The logic in the topos of pre-sheaves over a
groupoid is Boolean. This agrees with the Boolean character of the semantic in
each individual layer. Then the apparition of generalized Fourier analysis in
this context, is not surprising from the point of view of topos, because the
linear representations (over any commutative ring) are nothing else that the
Abelian objects in the topos in fiber of each layer. The proper logic of the
topos of the site of the network gives the opposite: a non-Boolean progress in
the logic when going deeper in the layers.
The groupoid $\displaystyle G_{3}$ of the first model was made of three
objects $\displaystyle a,b,c$, non related, and one circle of morphisms for
each object. Each layer corresponds to a representation of $\displaystyle
G_{3}$. The logic is made by the vertices, edges and face of the triangle
$\displaystyle\Omega_{3}$, plus the empty set.
The external symmetries of $\displaystyle G_{3}$ itself are described by the
group of permutations $\displaystyle\mathfrak{S}_{3}$.

######  Remark.

According to the questions asked in our second series of experiments, we even
could have expected the intervention of a sort of quantization of the group of
all homeomorphisms of the space (circular or linear); perhaps when augmenting
the complexity of the data and the number of neurons, this would appear.

###  3.5 The important role of redundancy

Most cells in the last hidden layer are logical and, before the intervention
of the blue object, they are repeated several times almost identically. Even
with the blue one, the tendency to repeat is obvious. As this does not appear
in the preceding layers, this shows the existence of a standard way of giving
less weight to less logical cells in order to produce logical behaviors.

There is an analogy with statistics: the advantage to use independent
identically distributed variables. But also this can be linked to learning,
because the minimum of the metric can have highly specific properties.

In this second set of experiments, we saw that the probabilities are necessary
if we want to conclude with logical arguments from the last hidden layer. For
instance, each time a cell saturates at $\displaystyle+1$ for the proposition
$\displaystyle R(a)\wedge G(a)$, if its activity for a given image
$\displaystyle\xi$ is not $\displaystyle+1$ (sufficiently clearly) we directly
conclude $\displaystyle D$, but if it is $\displaystyle+1$ we cannot, except
if we add information coming from the other curve; if the distributions of
activity associated to the three other local propositions do not clearly
saturate at $\displaystyle+1$, we are allowed (and probably also the network)
to deduce from the activity $\displaystyle+1$ (or almost $\displaystyle+1$),
that there exists a point $\displaystyle a$ where $\displaystyle G(a)$ and
$\displaystyle V(a)$ is true. Consequently, in a statistical sense, if many
cells repeat this message it has more chance to be true, and we get a
proposition not far from a certitude. This will be an important ingredient of
a discussion about semantic information.

There remains an important experimental question: is it visible in the weights
chosen by learning that the network uses the logical deduction? And how the
statistical argument is taken into account?
The existence of the logical cells and the evidence of a role of probabilistic
inference give no great doubt that both logic and statistics are used, but it
clearly needs to be made more precise.

##  4 How do weights perform logic and deduction?

First, we developed an automatic detection and analysis of the logical
behaviors of cells, adaptable to all the above reported experiments.
First step has been to record the reactions of individual cells $\displaystyle
a\in L_{k};k=2,3$, to the input data $\displaystyle\xi$, conditioned by a
known answer $\displaystyle\sigma$ to one of the questions that were asked
inside, and frequently reflected the objectives of the network. For instance,
we recorded the distribution activity of $\displaystyle a$ for the inputs
where we know that the green object is included in the red one.
Then its activity, a real number $\displaystyle x_{a}(\xi|\sigma)$ between
$\displaystyle-1$ and $\displaystyle 1$, is partitioned in three sets labelled
by $\displaystyle-1$, $\displaystyle 0$ and $\displaystyle+1$, according to
the values smaller than $\displaystyle-1/3$, between $\displaystyle-1/3$ and
$\displaystyle 1/3$ and greater than $\displaystyle+1/3$. The cell is
considered to be _logical_ for the condition $\displaystyle\sigma$ if more
than $\displaystyle 80\%$ of the conditioned activities belong to one of the
interval of the partition. It appeared in all the above experiments that when
a cell is logical, the chosen segments contain an extremity, thus attributing
the number $\displaystyle-1$ or $\displaystyle+1$ to $\displaystyle a$ for the
condition $\displaystyle\sigma$. (Remark we have now, in more complex
experiments, cells which also choose the interior $\displaystyle]-1/3,1/3[$
under some condition.) Remark that, in most cases with three hidden layers or
more, most cells are logical for at least two conditions, and have
distributions much more concentrated than $\displaystyle 80\%$, around
$\displaystyle 95\%$ (see table 1).

Table 1: Logical cells behavior

Next step is to decode the activity: when the logical cell fires in the
opposite interval $\displaystyle\mp 1$ of its preferred interval for
$\displaystyle\sigma$, we consider that it excludes $\displaystyle\sigma$, and
we put a $\displaystyle 0$ in the table for this proposition
$\displaystyle\sigma$, meaning $\displaystyle\neg\sigma$ is asserted by
$\displaystyle a$ at this value $\displaystyle\mp 1$. If nothing can be
decided, we put $\displaystyle 1$. This gives a matrix, whose columns
represent the conditions and its lines the pairs
$\displaystyle(a,\varepsilon)$, where $\displaystyle a$ is a cell and
$\displaystyle\varepsilon$ is $\displaystyle+1$ or $\displaystyle-1$. The
logical score is defined by the number of $\displaystyle 0$ in the line of
$\displaystyle a,\varepsilon$.
Starting with these data, we computed the predictions of the pairs
$\displaystyle(a,b)$ and of the triples $\displaystyle(a,b,c)$ of cells, at
given values $\displaystyle\varepsilon_{a},\varepsilon_{b},...$ in
$\displaystyle\\{\pm 1\\}$. This gave matrices which four, resp. eight, lines
for each pair, resp. triple. The core of the matrix in general possesses less
rows, because not all vector of signs can be realized given the possible
inputs. For instance, if the objectives are $\displaystyle D_{RG}$,
$\displaystyle II_{RG}$, $\displaystyle IO_{RG}$, at most three vectors are
accessible.
It appeared that no pair can reconstruct (except one exception) all the
conditions, but a non-negligible subset of triples can do (around
$\displaystyle 5\%$ of the possible triples). To each triple, we gave a score
$\displaystyle N(a,b,c)$, which is the number of conditions that it can
reconstruct. The efficient ones are named here _conclusive_ , cf. section 3.
For instance in the case of three colors, $\displaystyle R,G,B$ where
$\displaystyle B$ is sometimes present sometimes not, and the objectives are
$\displaystyle D_{RG}B$, $\displaystyle II_{RG}B$, $\displaystyle IO_{RG}B$,
$\displaystyle D_{RG}\neg B$, $\displaystyle II_{RG}\neg B$, $\displaystyle
IO_{RG}\neg B$, the conclusive triples have a score of $\displaystyle 6$. The