Train a competitive layer on nonnormalized vectors using LVQ technique How can we train a competitive layer on non-normalized vectors using LVQ technique ?

an example is given below from Neural Network Design (2nd Edition) book

The net input expression for LVQ networks calculates the distance between the input
and each weight vector directly, instead of using the inner product. The result is that the
LVQ network does not require normalized input vectors. This technique can also be
used to allow a competitive layer to classify non-normalized vectors. Such a network is
shown in figure below.

Use this technique to train a two-neuron competitive layer on the (non-normalized)
vectors below, using a learning rate $\alpha=0.5$

$p_1=\begin{bmatrix}
1 \\
1
\end{bmatrix}, p_2=\begin{bmatrix}
-1 \\
2
\end{bmatrix}, p_3=\begin{bmatrix}
-2 \\
-2
\end{bmatrix}$

Present the vectors in the following order : $p_1, p_2, p_3, p_2, p_3, p_1$

Initial weights : $W_1=\begin{bmatrix}
0 \\
1
\end{bmatrix}, W_2=\begin{bmatrix}
1 \\
o
\end{bmatrix}$

We have 2 classes , 1 subclass for each class

\begin{equation}
    W^2=\begin{vmatrix}
1 & 0\\
0 & 1\\
\end{vmatrix}
\end{equation}

$p_1$:

\begin{equation}
    \alpha^1=compet(n^1)=compet\begin{vmatrix}
||w_1-p_1||\\
||w_2-p_1||\\
\end{vmatrix} = compet\begin{vmatrix}
||\begin{vmatrix}
0 & 1\\
\end{vmatrix}^T-\begin{vmatrix}
1 & 1\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
1 & 0\\
\end{vmatrix}^T-\begin{vmatrix}
1 & 1\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
1\\
1\\
\end{vmatrix}) = \begin{vmatrix}
1\\
0\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix}
1 & 0\\
0 & 1\\
\end{vmatrix}\begin{vmatrix}
1\\
0\\
\end{vmatrix}=\begin{vmatrix}
1\\
0\\
\end{vmatrix}
\end{equation}

\begin{equation}
    W_1(1) = W_1(0) + \alpha\cdot(p_1-W_1(0))=\begin{vmatrix}
0\\
1\\
\end{vmatrix}+0.5\cdot(\begin{vmatrix}
1\\
1\\
\end{vmatrix}-\begin{vmatrix}
0\\
1\\
\end{vmatrix})=\begin{vmatrix}
0\\
1\\
\end{vmatrix}+\begin{vmatrix}
0.5\\
0\\
\end{vmatrix}=\begin{vmatrix}
0.5\\
1\\
\end{vmatrix}
\end{equation}

$p_2$ :

\begin{equation}
    \alpha^1=compet(n^1)=compet\begin{vmatrix}
||w_1-p_2||\\
||w_2-p_2||\\
\end{vmatrix} = compet\begin{vmatrix}
||\begin{vmatrix}
0.5 & 1\\
\end{vmatrix}^T-\begin{vmatrix}
1 & 2\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
1 & 0\\
\end{vmatrix}^T-\begin{vmatrix}
1 & 2\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
1.8027756377\\
2.8284271247\\
\end{vmatrix}) = \begin{vmatrix}
0\\
1\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix}
1 & 0\\
0 & 1\\
\end{vmatrix}\begin{vmatrix}
0\\
1\\
\end{vmatrix}=\begin{vmatrix}
0\\
1\\
\end{vmatrix}
\end{equation}

wrong class

\begin{equation}
    W_2(1) = W_2(0) - \alpha\cdot(p_2-W_2(0))=\begin{vmatrix}
1\\
0\\
\end{vmatrix}-0.5\cdot(\begin{vmatrix}
-2\\
2\\
\end{vmatrix}=\begin{vmatrix}
2\\
-1\\
\end{vmatrix}
\end{equation}

$p_3$ :

\begin{equation}
    \alpha^1= compet\begin{vmatrix}
||\begin{vmatrix}
0.5 & 1\\
\end{vmatrix}^T-\begin{vmatrix}
-2 & 2\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
2 & -1\
\end{vmatrix}^T-\begin{vmatrix}
-2 & 2\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
70\\
5\\
\end{vmatrix}) = \begin{vmatrix}
1\\
0\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}
1\\
0\\
\end{vmatrix}
\end{equation}

wrong class

\begin{equation}
    W_1(2) = W_1(1) - \alpha\cdot(p_3-W_1(1))=\begin{vmatrix}
0.5\\
0\\
\end{vmatrix}-0.5\cdot(\begin{vmatrix}
-2.5\\
1\\
\end{vmatrix}=\begin{vmatrix}
1.75\\
0.5\\
\end{vmatrix}
\end{equation}

$p_2$ :

\begin{equation}
    \alpha^1=compet(n^1)= compet\begin{vmatrix}
||\begin{vmatrix}
1.75 & 0.5\\
\end{vmatrix}^T-\begin{vmatrix}
-1 & 2\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
2 & -1\
\end{vmatrix}^T-\begin{vmatrix}
-1 & 2\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
3.13\\
4.24\\
\end{vmatrix}) = \begin{vmatrix}
1\\
0\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}
1\\
0\\
\end{vmatrix}
\end{equation}

\begin{equation}
    W_1(3) = W_1(2) - \alpha\cdot(p_2-W_1(2))=\begin{vmatrix}
1.75\\
0.5\\
\end{vmatrix}+0.5\cdot(\begin{vmatrix}
-2.75\\
1.5\\
\end{vmatrix}=\begin{vmatrix}
0.375\\
1.25\\
\end{vmatrix}
\end{equation}

$p_3$ :

\begin{equation}
    \alpha^1=compet(n^1)= compet\begin{vmatrix}
||\begin{vmatrix}
0.375 & 1.25\\
\end{vmatrix}^T-\begin{vmatrix}
-2 & 2\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
2 & -1\
\end{vmatrix}^T-\begin{vmatrix}
-2 & 2\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
2.95\\
5\\
\end{vmatrix}) = \begin{vmatrix}
1\\
0\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}
1\\
0\\
\end{vmatrix}
\end{equation}

wrong class

\begin{equation}
    W_1(4) = W_1(1) - \alpha\cdot(p_3-W_1(3))=\begin{vmatrix}
0.375\\
1.25\\
\end{vmatrix}-0.5\cdot(\begin{vmatrix}
-2.375\\
0.75\\
\end{vmatrix}=\begin{vmatrix}
1.5625\
0.875\\
\end{vmatrix}
\end{equation}

p_1 :

\begin{equation}
    \alpha^1=compet(n^1)= compet\begin{vmatrix}
||\begin{vmatrix}
1.5625 & 0.875\\
\end{vmatrix}^T-\begin{vmatrix}
1 & 1\\
\end{vmatrix}^T||\\
||\begin{vmatrix}
2 & -1\
\end{vmatrix}^T-\begin{vmatrix}
1 & 1\\
\end{vmatrix}^T||\\
\end{vmatrix}=
compet(\begin{vmatrix}
0.57\\
2.23\\
\end{vmatrix}) = \begin{vmatrix}
1\\
0\\
\end{vmatrix})
\end{equation}

\begin{equation}
   \alpha^2=W^2\cdot\alpha^1\begin{vmatrix}
1\\
0\\
\end{vmatrix}
\end{equation}

\begin{equation}
    W_1(5) = W_1(4) - \alpha\cdot(p_1-W_1(4))=\begin{vmatrix}
1.5625\\
0.875\\
\end{vmatrix}+(\begin{vmatrix}
-0.28125\\
0.0625\\
\end{vmatrix}=\begin{vmatrix}
1.28125\
0.9375\\
\end{vmatrix}
\end{equation}


Are innovation weights shared in the NEAT algorithm? I am trying to implement NEAT algorithm in Python from scratch. However, I am stuck. When a new innovation number is created it has two nodes which represents the connection. Also this innovation number has a weight.

However, I know that innovation numbers are global variables, in other words when a innovation number is created,

ex. Innovation ID:1 - Node:1 to Node:4 - weight: 0.5

it will have a ID which will be used by other connections to represent the connection between Node:1 to Node:4.

When this innovation is used by another neural network, will it also use the weight of the innovation 1, which is 0.5 in this example?

Quoting Evolving neural networks through augmenting topologies, p. 10 (emphasis mine):

  When crossing over, the genes in both genomes with the same innovation numbers are lined up. These genes are called matching genes. Genes that do not match are either disjoint or excess, depending on whether they occur within or outside the range of the other parent’s innovation numbers. They represent structure that is not present in the other genome. In composing the offspring, genes are randomly chosen from either parent at matching genes, whereas all excess or disjoint genes are always included from the more fit parent.

Innovation numbers are used to line up genes in different genomes so you can perform crossover on networks with different topologies. Each network can optimise the weights in matching genes in a different way, so the weights are not shared. If they were, crossover would have nothing to contribute towards diversifying the population.


Adding voices to voice synthesis corpuses If one uses one of the open source implementations of the WaveNet generative speech synthesis design, such as https://r9y9.github.io/wavenet_vocoder/, and trains using something like the CMU's arctic corpus, now can one add a voice that sounds younger, older, less professional, or in some other way distinctive.  Must the entire training begin from scratch, or is there a more resource and time friendly way?

The original wavenet and the implementation you linked to is globally conditioned on speaker embeddings, which means that the network was given a unique identifier for the person speaking each time it trained on an audio clip. This allows the network to learn to mimic the voice of each person in the training data, but it only learns their voices, not arbitrary people's voices.
You might be able to do what you're describing by globally conditioning on different features of the speakers, such as age etc, though this would require a lot a speakers to span those dimensions.
I think it would definitely be possible to use style transfer for this, similar to the work done on tweaking people's age in images [1]. Some similar work has been done for audio in order to change the accent of a voices between accents [2], so I wouldn't be surprised to see more examples of this in the near future.
[1] https://www.youtube.com/watch?v=2qMw8sOsNg0
[2] https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html


What are the state-of-the-art approaches for continual learning with neural networks? There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with neural network architectures.
By continual learning, I mean improving a model (while using it) with a stream of data coming in (maybe after a partial initial training with ordinary batches and epochs).
A lot of real-world distributions are likely to gradually change with time, so I believe that we should be able to train NNs in an online fashion.
Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?

What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning.

This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but changing data result in a change of the task. Say your goal is to classify different breeds of dogs. Your online data-set morphs into excluding "Great Danes". Your neural network after enough epochs would forget about "Great Danes". The task is still serving its purpose by classifying different breeds but the task still changed. It changed from recognizing "Great Danes" as a dog breed to not recognizing "Great Danes" as a dog breed. The weights changed to exclude them but the methods I linked tries and keep weights intact even though it was not intended for the purpose of online learning. Just set the hyper parameters to include these techniques to low as I believe data won't have an instant change but would change over time, and you should be fine.

The most obvious technique being storing information as you train. This is called pseudo-rehearsal. With this at least you would be able to use stochastic gradient decent but you need memory and resources as the data set grows.

Then there was an attempt to reduce impacts of weights on old tasks to keep some relevancy to them. Structural Regularization.

Later these guys implemented HAT which seems to keep some weights static while others adapt to new tasks.


Why does the discriminator minimize the cross-entropy while the generator maximize it? In his original GAN paper Goodfellow gives a game theoretic perspective for GANs:

\begin{equation}
\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right) =
\mathbb{E}_{x\sim\mathit{p}_{\textrm{data}}\left(x \right)} \left[\textrm{log}\, D \left(x \right) \right]
+ \mathbb{E}_{z\sim\mathit{p}_{\textrm{z}}\left(z \right)} \left[\textrm{log} \left(1 - D \left(G \left(z \right)\right)\right) \right]
\end{equation}

I think I understand this formula, at least it makes sense to me.
What I don't understand is that he writes in his NIPS tutorial:

  In the minimax game, the discriminator minimizes a cross-entropy, but the generator maximizes the same cross-entropy.

Why does he write that the discriminator minimizes the cross-entropy while the generator maximizes it? Shouldn't it be the other way around? At least that is how I understand $\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right)$.

I guess this shows that I have a fundamental error in my understanding. Could anyone clarify what I'm missing here?

Your mistake is that you think that the referenced $V(D,G)$ is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the $V(D,G)$. Hence, if you consider the minus behind the $V(D,G)$ ($-V(D,G)$) the sentence will be meaningful.


Choosing Data Augmentation smartly for different application I'm trying to understand the role of data augmentation and how it can affect the performance/accuracy of a deep model. My target application is fire detection (on video frames), with almost 15K positive and negative samples, and I was using the following data augmentation techniques. Does using ALL the followings always increase the performance? Or we have to choose them somehow smartly given our target application?

rotation_range=20, width_shift_range=0.2, height_shift_range=0.2,zoom_range=0.2, horizontal_flip=True

When I think a bit more, fire is always straight up, so I think rotation or shift might in fact worsen the results, given that it makes the image sides stretch like this, which is irrelevant to fires in video frames. Same with rotation. So I think maybe I should only keep zoom_range=0.2, horizontal_flip=True and remove the first three. Because I see some false positives when we have a scene transition effect in videos.

Is my argument correct? Should I keep them or remove them?

In fact, choosing smartly the values of the image augmentation can help the performance of your system. Where I work we developed an object detector for cars. We had the following image augmentation parameters:

Apect ratio distorsion: it changed the cars dimensions
Additive noise: it blurred the image
Change colorspace: change the cars colors
Saturation change: brightness
Rotation: rotate image axis
Flipping (horizontally): mirror effect
Zooming: zoom in/out of image

At first we just combined all the effects (within a certain range of change per effect) randomly in each augmented image. But then we did an study for our use case and we found that our model performed better with: no aspect ratio deformation and no change in color space. After giving it a bit of a thought it really made sense for our use case because it is a detector for a very specific type of car.

However, although the improvements accomplished by optimizing the parameters of image augmentation boosted the performance of the model, the improvements were not huge, so in the end it did not really worth the effort. To give some numbers it improved $5\%$ in our custom test metrics, and we were expecting something around $10\%$ so from that moment on we left the specific image augmentation parameters but stopped thinking too much about it.

So, in conclussion, does it helps? yes, how much? well, that would depend on the use case. For us, it turned out it did not improve as much as we wanted.

Hope this helps :)


Should the biases be zero or randomly initialised? I'm initialising DNN of shape [2 inputs, 2 hiddens, 1 output] with these weights and biases:

#hidden layer
weight1= tf.Variable(tf.random_uniform([2,2], -1, 1),
         name="layer1");
bias1 = tf.Variable(tf.zeros([2]), name="bias1");

#output layer
weight2 = tf.Variable(tf.random_uniform([2,1], -1, 1),
          name="layer2");
bias2 = tf.Variable(tf.zeros([1]), name="bias2");

That's what I followed some online article, however, I wonder what if I initialise bias values using tf.random_uniform instead of tf.zeros? Should I choose zero biases or random biases generically?

From the stanford CNN class (http://cs231n.github.io/neural-networks-2/):

  Blockquote
  Initializing the biases. It is possible and common to initialize the biases to be zero, since the asymmetry breaking is provided by the small random numbers in the weights. For ReLU non-linearities, some people like to use small constant value such as 0.01 for all biases because this ensures that all ReLU units fire in the beginning and therefore obtain and propagate some gradient. However, it is not clear if this provides a consistent improvement (in fact some results seem to indicate that this performs worse) and it is more common to simply use 0 bias initialization.


Are Neural Net architectures accidental discoveries? Recently, I have been learning about new neural networks, which are used for specialised purposes, like speech recognition, image recognition, etc. The more I discover the more I get amazed by the cleverness behind models such as RNN's and CNN's. Questions about the working, intuition, mathematics have been asked a lot in this community, all with vague answers and apparent understandings.
So, my question is: did the researchers come up with these specialised models accidentally or did they follow particular steps to get to the model (like in a mathematical framework)? And how did they look at a particular class of problem and think "Yeah, a better solution might exist"?
Since the understanding of NN's is so vague, these are 'high risk, high reward' scenarios, since you might be chasing only the mirage (illusion) of a solution.

Although there is a strong element of "try and see" that has driven successful architectures, the drivers for what to try are often inspired by underlying theory or knowledge from other disciplines.

Specifically for basic CNN, which led to AlexNet and many of the best image processing, the concept of using local receptive fields in layers was inspired by study of neurons in the cat visual system.

Modern RNNs also did not appear out of nowhere, there has long been an appreciation of the difference between a feed-forward network and a recurrently-connected one, and the different applications possible. The step change to LSTM was deliberate response to analysis of problems training simplest forms of RNN.

Like much of science, these things are also driven by success in the real world following the research. Many promising ideas have been tried and rejected. Some have been used for a while then superseded, e.g. using RBMs or stacked auto-encoders to pre-train deep networks before ReLUs and Xavier initialisation were discovered - although both RBMs and auto-encoders still have their niches.

Tweaks to architectures, such as variants of LSTM/GRU, may even be deliberately searched and assessed as part of research. That is done with the explicit knowledge that this part of finding a good design is best done as a search across possibilities.

Despite the evolution-like progress, presenting all such advances as completely random or pure GA-like search is ignoring the conscious effort and research that leads to the designs. If you search literature on any major successful design (such as the existence of RNNs or CNNs in the first place), and read the papers, you will often find that modern neural network architectures have deep roots in older research, plus have mathematical and/or scientific justifications for the choices made.


New face generator from Nvidia I have just found the paper and documentation about GAN 2.0, the new face creator from Nvidia.

On the website https://thispersondoesnotexist.com/ they have used this approach to create realistic faces. Unfortunately, the website does not exist anymore.

Is there another webpage demonstrating the new face creator from Nvidia?

You can find a decent sized collection on archive.org. Just browse through the snapshots and they'll contain a few images. It probably doesn't contain every single one, but it has quite a decent set to start from. :)

Here's one I found:


Figuring out mapping between two matrices Imagine I have a 2D matrix, A. I apply some transformation to it, for example:
B = A_shifted + A.

Would it be possible to train a CNN to learn back the mapping from B to A? Giving B as example and A as target?

Thanks!

Yes, with some limitations.

CNNs can be used to map images to related images, and that should include many simple matrix transformations. For instance, here is an example of de-blurring OCRed text using a CNN.

Basically, you would train your network with lots of A, B examples, with the input as B and desired output as A.

The limitation is that where you have transformations that are technically irreversible, then the CNN may learn to produce a best "mean" output. The symptom of this will be fuzzy images lacking high frequency components, and matrices which are not representative of the target distribution, but that do solve the transformation (within limits of training accuracy). If you want to improve on that, and produce a more realistic/precise original, then you will probably want to look at adding a generative component - a GAN, VAE/GAN or RBM etc. Note this would not accurately produce the original matrix, but would generate one that both transformed into your given transformed matrix (within some level of accuracy) and was sampled from your input distribution. That is, it could be more of a feasible original than one generated using a simpler CNN architecture.


What is the correct formula for updating the weights in a 1-single hidden layer neural network? I'm creating a neural network with 3 layers and no bias.
On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:
$$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$
where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.
But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of I'm taking $o_k$ and $o$ has length $\text{output nodes}$.

In simple terms, what is the right formula for updating these weights?

Also, what is the right formula for updating the weights between the input layer and the hidden layer?

The correct formula for updating the weights between the hidden layer and the output layer is:
$$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$
where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.
I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.


In OCR, how should I deal with the warped text on the sides of oval objects? Consider an image that contains one can (or bottle, or any similar oval object), which has texts all over it. In the image below, I have many bottles, but you can assume that each image only contains one such object.

As we can see, in each can, the text can flow from left to right, and any OCR system may miss the text on the left and right sides of the can, as they are not aligned with the camera angle.
So, is there any solution/s for this, like preprocessing in a certain way, so that we can read the text or make this round object into a straight one? (If there is any Python program that can solve this problem, could you please share it with me?)

There are many papers on this but the following is a good start:

How to unwrap wine labels programmatically.  The author includes source code in Python.

You mentioned you do not want to do a panoramic view but that has more than one meaning.  If I assume you mean you do not want to rotate the can while taking multiple photos, or you don't want to take multiple photos from different angles, you could try a pericentric lens.  This would require some image processing to do the unwrapping.  More resolution is needed as the wrapping is much more severe.  The advantage though is that you will have a single image of the full cylindrical surface and won't miss any features or text.


How does replacing states with latent representations help RL agents? I have seen many papers using autoencoders to replace images (states) with latent representations. Some of those methods have shown higher rewards using such techniques. However, I do not understand how this helps the RL agent learn better. Perhaps viewing latent representations allows the agent to generalize to novel states more quickly?
Here are 2 papers I have read -

Deep Spatial Autoencoders for Visuomotor Learning
DAQN: Deep Auto-encoder and Q-Network

In short, it is much easier for the agent to learn from a smaller dimensional state space. This is because the agent must also do representation learning; i.e. it must also infer what the state is telling it as part of the learning process. If you think of the architecture used in DQN to solve Atari, they had a CNN that outputted a vector which was then passed through some dense layers. Here the representation learning was done by the CNN and was trained using an end-to-end approach i.e. all updates to the network weights were done through the reinforcement learning objective; that is there is no supervised or unsupervised learning that takes place.
This can be particularly difficult when you combine images with sparse rewards as there is not a lot of feedback so the representation learning can take a long time. This paper gives a good description of the problem of decoupling representation learning from reinforcement learning with a nice solution.
The other main 'problem setting' I have seen images replaced with a latent state is when the authors are looking at planning. The problem with doing any kind of planning is that a model of the transition dynamics, $p(s' | s, a)$, is needed. For high dimensional state spaces such as images, this can be very difficult to predict and even relatively small errors will quickly compound so if you use the model to predict multiple time steps into the future the planner is useless because of these compounding errors. I think there is a discussion on this in this paper (certainly there will be references therein that point you in the right direction).


Can fully connected layers be used for feature detection? I need help in understanding something basic.
In this video, Andrew Ng says, essentially, that convolutional layers are better than fully connected (FC) layers because they use fewer parameters.
But I'm having trouble seeing when FC layers would/could ever be used for what convolutional layers are used for, specifically, feature detection.
I always read that FC layers are used in the final, classification stages of a CNN, but could they ever be used for the feature detection part?
In other words, is it even possible for a "feature" to be deciphered when the filter size is the same as the entire image?
If not, it's hard for me to understand Andrew Ng's comparison---there aren't any parameter reduction "savings" if we're not going to use an FC "filter" in place of a CNN layer in the first place.
A semi-related question: Can multi-layer perceptrons (which I understand to be fully connected neural networks) be used for feature detection? If so, how do image-sized "filters" make any sense?

First of all, (FC) Neural Networks (NN) are universal function approximators. This means, that, in theory, there must exist some NN of appropriate size that is capable of doing what a CNN can do as well. The only difference is that the standard NN would have to be considerably larger than a corresponding CNN with the same capabilities. Each input pixel would probably have to map to a few dozens to hundreds of input nodes that are trained each on detecting some part of a distinct feature in their respective input image location. Consecutive, hidden layers than have to combine the parts extracted by the input layer for any given location and finally act as the true feature detectors, detecting features as compositions of values in the first layer.
So, for each position in the input image separately, all the filters have to be learned separately. This absence of weight sharing (since filters are not shared across input locations, but have to be trained for each separate input location) makes using a standard NN for feature extraction so inefficient.
However, in theory, also a standard NN could be trained on solving an image classification task, thereby learning to extract features. Actually, I would not be surprised if someone had tested that already on the MNIST dataset.


Why can we use a network to estimate $Q_\pi(s, a)$ in Actor-Critic Method? According to deep Q learning, we want to learn $Q^*(s,a)$, which is the optimal action-value function. It does make sense because we assume there is only one optimal function so the algorithm will converge supposedly.
But when it comes to actor-critic method, we use critic network (also called  value network) to estimate $Q_\pi(s, a)$. This is what confused me. Since our policy $\pi$ will change through time, the target $Q_\pi(s, a)$ of value network will also change. What will happen for a network to learn a changing function?

The policy doesn't change over time. That is, the values will change, otherwise we would not be learning anything, but our rules for action selection don't. I.e. we always take action according to the distribution postulated to our current estimate of the policy $\pi_\theta(a|s)$, we don't suddenly start taking $\max_a \pi_\theta(a|s)$, which would be a true change in policy and would make learning both the actor and the critic unstable.
Because NN's are able to handle noisy target distributions, this is how they can deal with the changing data. If you think of how Actor-Critic methods work, you would initially start to shift your NN to some unfeasible values (due to random initialisation and the Actor-Critic not having any information about the environment), but as you start to interact with the environment you will start to update the agent towards the 'true' policy.
An analogy in supervised learning would be to have some noisy data which is incorrect and some true data. If you trained your network on the noisy data for a small number of epochs and then never showed it to the network again and trained it solely on the correct data, it would forget it has ever seen the noisy data and only represent the new, true data.


What are Hyper-heuristics? I wanted to know what the differences between hyper-heuristics and meta-heuristics are, and what their main applications are. Which problems are suited to be solved by Hyper-heuristics?

TL:DR: Hyper-heuristics are metaheuristics, suited for solving the same kind of optimization problems, but (in principle) affording a "rapid prototyping" approach for non-expert practitioners. In practice, there are issues with the prevailing approach, motivating an emerging perspective on 'whitebox' hyper-heuristics.

In more detail:

Metaheuristics are methods for searching an intractably large space of possible solutions in order to find a 'high quality' solution. Popular metaheuristics include Simulated Annealing, Tabu Search, Genetic Algorithms etc.

The essential difference between metaheuristics and hyper-heuristics is the addition of a level of search indirection: informally, hyper-heuristics can be described as 'heuristics for searching the space of heuristics'. One can therefore use any metaheuristic as a hyper-heuristic, providing the nature of the 'space of heuristics' to be searched is appropriately defined.

The application area for hyper-heuristics is therefore the same as metaheuristics. Their applicability (relative to metaheuristics) is as a 'rapid prototyping tool': the original motivation was to allow non-expert practitioners to apply metaheuristics to their specific optimization problem (e.g. "Travelling-Salesman (TSP) plus time-windows plus bin-packing") without requiring expertise in the highly-specific problem domain. The idea was that this could be done by:

Allowing practitioners to implement only very simple (effectively,
randomized) heuristics for transforming potential solutions. For
example, for the TSP: "swap two random cities" rather than (say) the more
complex Lin-Kernighan heuristic.
Achieve effective results (despite using these simple heuristics) by combining/sequencing them in an intelligent way, typically by employing some form of learning mechanism.

Hyper-heuristics can be described as 'selective' or 'generative' depending on whether the heuristics are  (respectively) sequenced or combined. Generative hyper-heuristics thus often use methods such as Genetic Programming to combine primitive heuristics and are therefore typically customized by the practitioner to solve a specific problem. For example, the original paper on generative hyper-heuristics used a Learning Classifier System to combine heuristics for bin-packing. Because generative approaches are problem-specific, the comments below do not apply to them.

In contrast, the original motivator for selective hyper-heuristics was that researchers would be able to create a hyper-heuristic solver that was then likely to work well in an unseen problem domain, using only simple randomized heuristics.

The way that this has traditionally been implemented was via the introduction of the 'hyper-heuristic domain barrier' (see figure, below), whereby generality across problem domains is claimed to be achievable by preventing the solver from having knowledge of the domain on which it is operating. Instead, it would solve the problem by operating only on opaque integer indices into a list of available heuristics (e.g. in the manner of the 'Multi-armed Bandit Problem').

In practice, this 'domain blind' approach has not resulted in solutions of sufficient quality. In order to achieve results anywhere comparable to problem-specific metaheuristics, hyper-heuristic researchers have had to implement complex problem-specific heuristics, thereby failing in the goal of rapid prototyping.

It is still possible in principle to create a selective hyper-heuristic solver which is capable of generalizing to new problem domains, but this has been made more difficult since the above notion of domain barrier means that only a very limited feature set is available for cross-domain learning (e.g. as exemplified by a popular selective hyper-heuristic framework).

A more recent research perspective towards 'whitebox' hyper-heuristics advocates a declarative, feature-rich approach to describing problem domains. This approach has a number of claimed advantages:

Practitioners now need no longer implement heuristics, but rather simply specify the problem domain.
It eliminates the domain-barrier, putting hyper-heuristics on the same 'informed' status about the problem as problem-specific metaheuristics.
With a whitebox problem description, the infamous 'No Free Lunch' theorem (which essentially states that, considered over the space of all black box problems, Simulated Annealing with an infinite annealing schedule is, on average, as good as any other approach) no longer applies.

DISCLAIMER: I work in this research area, and it is therefore impossible to remove all personal bias from the answer.


Is the optimal policy always stochastic if the environment is also stochastic? Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?

Intuitively, if the environment is deterministic (that is, if the agent is in a state $s$ and takes action $a$, then the next state $s'$ is always the same, no matter which time step), then the optimal policy should also be deterministic (that is, it should be a map from states to actions, and not to a probability distribution over actions).

Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?

No.

An optimal policy is generally deterministic unless:

Important state information is missing (a POMDP). For example, in a map where the agent is not allowed to know its exact location or remember previous states, and the state it is given is not enough to disambiguate between locations. If the goal is to get to a specific end location, the optimal policy may include some random moves in order to avoid becoming stuck. Note that the environment in this case could be deterministic (from the perspective of someone who can see the whole state), but still lead to requiring a stochastic policy to solve it.
There is some kind of minimax game theory scenario, where a deterministic policy can be punished by the environment or another agent. Think scissors/paper/stone or prisoner's dilemma.

  Intuitively, if the environment is deterministic (that is, if the agent is in a state  and takes action , then the next state ′ is always the same, not matter which time step), then the optimal policy should also be deterministic (that is, it should be a map from states to actions, and not to a probability distribution over actions).

That seems reasonable, but you can take that intuition further with any method based on a value function:

If you have found an optimal value function, then acting greedily with respect to it is the optimal policy.

The above statement is just a natural language re-statement of the Bellman optimality equation:

$$v^*(s) = \text{max}_a \sum_{r,s'}p(r,s'|s,a)(r+\gamma v^*(s'))$$

i.e. the optimal values are obtained when always choosing the action that maximises reward plus discounted value of next step. The $\text{max}_a$ operation is deterministic (if necessary you can break ties for max value deterministically with e.g. an ordered list of actions).

Therefore, any environment that can be modelled by a MDP and solved by a value-based method (e.g. value iteration, Q-learning) has an optimal  policy which is deterministic.

It is possible in such an environment that the optimal solution may not be stochastic at all (i.e. if you add any randomness to the deterministic optimal policy, the policy will become strictly worse). However, when there are ties for maximum value for one or more actions in one or more states then there are multiple equivalent optimal and deterministic policies. You may construct a stochastic policy that mixes these in any combination, and it will also be optimal.


Is a linear activation function (in the output layer) equivalent to an identity function? I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.
I have seen several codes where the choice of the activation function for the output layer is linear.
Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?

And following this point, aren't you just as well off with just using the identity function as your output activation function?

When someone declares that the output of a neural network layer is linear this is exactly what they mean. It can also be described as "no activation function".

Saying that a NN layer has linear activation is a kind of short-hand for saying "the whole layer is a linear function of its inputs", which is true without adding any activation function, just using the weights and bias.

There is usually no separate linear function applied, and libraries such as Keras include the term 'linear' only for completeness, or so that the choice can be made explicit in the code, as opposed to an unseen default.

Note that the link to Keras activation definition above says:

  Linear (i.e. identity) activation function.


Data augmentation for very small image datasets I am looking for techniques for augmenting very small image datasets. I have a classification problem with 3 classes. Each class consists of 20 different shapes. The shapes are similar between the classes, but the task is to identify which class the shapes belong to. Per shape, I have between 1 and 35 training examples. For two classes, I have 25 training examples per shape, but the number of examples per shape for the third classes is usually around 5. Now, what data augmentation schemes do you recommend? Geometric / affine transformations seem like a good place to start. However, I have also thought of applying Fast Fourier Transform (do the forward transform, add some noise, do the inverse transform). GANs seem infeasible, right? Not enough data, I suspect. In any case, I am grateful for your advice.

In my opinions, you have around $25\times 20 \times 2 + 5 \times 20 = 1100$ samples, so the list of problems is:

Lack of data
Imbalance between class 1,2 and class 3

With the simple task, the model with the low capacity (small number of parameters) is more suitable for your task and to avoid overfitting because of the lack of data. Since the augmentation is nothing but regularization, based on EfficientNetV2 paper, with the small model you don't need too much regularization, select some basic affine transformation and elastic transformation then apply Rand Augment with small magnitude mix with 0.1 dropout should be enough.
The second problem should be solved with the weight class for weight, which means set the scale factor for the third class, or duplicated the samples of the third class then apply the strong affine transformation to it.


How to evaluate an RL algorithm when used in a game? I'm planning to create a web-based RL board game, and I wondered how I would evaluate the performance of the RL agent. How would I be able to say, "Version X performed better than version Y, as we can see that Z is much better/higher/lower."
I understand that we can use convergence for some RL algorithms, but, if the RL is playing against a human in the game, how am I able to evaluate its performance properly?

When you want to compare Reinforcement Learning algorithms, you might want to compare the average rewards they generate and how fast and close they get to the optimal policy. However, in the case of comparing it to humans, you might want to compare the game results of all the games played.
Reward Comparison
Often Reinforcement Learning algorithms are compared by using the rewards (either direct, maximum or average in time/iteration). For example, in this page about RL a comparison of two algorithms is shown:

Or, when you know the optimal actions, you can plot the number of plays/iterations against the percentage of actions. See for example this RL comparison on the 10-armed testbed problem:

Henderson et al. 2017 have a whole section about the evaluation metrics of Reinforcement Learning algorithms. They also comment on the plotting of the average or maximum cumulative rewards, moreover they mention the sample bootstrap method to create confidence intervals for a better comparison. Lastly, they mention that the significance of the improvements of the algorithms should be calculated, using a statistical test, such the two sample t-Test. Note that you should take into account the distributions of the datasets to choose the right statistical test. An interesting article related to this is A Hitchhiker's Guide to Statistical Comparison of Reinforcement Learning Algorithms of Colas et al..
Comparison of plays against humans
To find out how well different algorithms play against humans you should do a large number of games and compare - what you consider - important parameters, for example: did the algorithm won, the time it took to win, number of points gained, etc. These values can then be compared statistically. Note that you have to think well about the setup of these experiments since you only should change the algorithms, the other parameters should stay equal. Therefore, you should - preferably - use a large number of subjects (of different ages, sexes, etc. to cover many types of people), and try to prevent any bias; think about the order, how many games are played, location, time etc.


Can an AI learn to suffer? I had first this question in mind "Can an AI suffer?". Suffering is important for human beings. Imagine that you are damaging your heel. Without pain, you will continue to harm it. Same for an AI. But then I told myself "Wait a second. It already exists. It is the errors and warnings that shows up". We can say it has the similar purpose as suffering. However, I felt something missing. We feel pain. The errors and bugs are just data. Let's say a robot can use machine learning and genetic programming to evolve.

Can an AI learn to suffer? And not just know it as mere information.

At a very high level, regarding evolutionary game theory and genetic algorithms, it is absolutely possible that AI could develop a state that is analogous with suffering, although, as you astutely point out, it would involve conditions that a computer cares about.  (For instance, it might develop a feeling analogous to "being aggrieved" over non-optimality in the algorithmic sense, or "frustration" at equations don't add up, or "dissatisfaction" over goals that have not been achieved.)
The robot tormented by small children at the mall can certainly be said to be "suffering" in that the children block the performance of the robot's function, but the robot is not conscious and suffering might be said to require awareness.  However, even without consciousness, this very simple robot can learn new behaviors through which it mitigates or avoids the "suffering" brought on by not being able to fulfill its function.
You definitely want to look into the concept of suffering in a philosophical context and Epicurus would be a very useful place to start.
Epicurus is directly relevant in an algorithmic sense because he uses the term "ataraxia" meaning calm, and is derived from the verb "tarasso" which means to agitate or disturb.
Ataraxia can be mathematically expressed as an equilibrium.  Tarasso can be mathematically expressed as disequilibrium.
This relates directly to Game Theory in that disequilibrium can be said to be the primary requirement of games, and to AI in that Game Theory can be said to be the root of all AI.
Ataraxia is also understood in the sense of "freedom from fear", which is temporal in that fear is a function of uncertainty as it relates to the future in a predictive sense, and involves current condition vs. possible, less optimal future conditions.
Thus fear, which is a form of suffering, is rooted in computational intractability, even where the "computer" is is a human brain.
Early philosophers such as Democritus are especially useful because they were exploring critical, fundamental concepts, many of which can now be expressed with modern mathematics.
To wit: you can't arrive at suffering until you first define "the Good" and "the Bad", which is a binary relationship in which neither term can be said to have meaning without the opposite.  (Mathematically, it can be expressed in its simplest form as a finite, one-dimensional graph.)  This understanding is quite ancient.

It is worth noting that the continuing value of the early philosophers is partly a factor of wisdom not being dependent on the volume of knowledge, demonstrated by Socrates in the idea that wisdom may be as simple as knowing you don't know something.
The ancient sages didn't have the benefit of powerful measurement tools, advanced mathematics, or scientific method, but they were very smart, and even more importantly, wise.


Recognising Noise in Simple Classification I have created a classifier for some simple gestures using an input layer, a hidden layer with tanh activation and an output softmax layer. I'm also using the Adam optimiser. The network classifies perfectly with validation data. However, I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. For example:

One gesture input looks like this and is correctly classified as gesture 'A':

However, when I pass this 'noise', which is clearly differentiable to the human eye, as input it still classifies it with 100% confidence that it is the same gesture 'A'.

I assume it's because the inputs are still very close to 0? My instinct is to scale up the inputs perhaps to increase the differentiation between the noise and the input. However, in real operation the noise will all be on a similar scale to the inputs and I won't know what is noise and what isn't so I will still have to apply the same scaling to that noise. Will I run into the same problem?

On a more general note is there a teaching approach to prevent misclassifications, particularly if we know what they might look like? For example, in this case I thought I could perhaps generate some noise and use it at training time to create an extra noise class, or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold? For example, if the network only produces 50% classifcation confidence for an input then I can discard it as noise. Any suggestions much appreciated!

The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently.

You need to train the network on noise well if you want it to be able to recognize it.

  or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?

You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better

See:

  Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning


What can I do with an autoencoder? I cannot find information in detail about autoencoder

What can I do with an autoencoder (and how can I do this), practically speaking?

What does the encoder (this part I think I understand) and a decoder (could not find much about this) part do? Can it for example show on an explainable way how patterns in the data are being represented?

I read some papers that say that it can be used to denoise the input, how does this work? (Am I changing the values of my input)

Is it true that an autoencoder can be also done with PCA (if we assume linearity)?

An autoencoder learns to compress data, and then to decompress it again, recovering the original data.

It does this by learning a mapping from the original feature space to a lower-dimensional space, and then another mapping back. This is indeed, like PCA. The same technique is used to compress JPEG images to transmit them over the web.

Autoencoders can denoise an image because the model will not be able to easily compress or decompress random noise, and so learns to ignore it.

Autoencoders can also be used to find embeddings of data that are likely to have semantic meaning. For example, you can use an autoencoder to compress English text, and a different one to compress French text. If you add some special constraints, then the models can learn to embed in the same lower-dimensional space. This technique underlies many of the recent advances in machine translation.


Why isn't medical imaging improving faster with AI? Researcher here. I just read this piece about medical imaging ai with object recognition and it left me wondering why there are still 100,000+ deaths a year in the US due to misdiagnosis - anyone out there working on these problems? Vinod Khosla famously said that he'd rather get surgery from AI than from a human - so where are we at with that?

AI in healthcare is already playing a big role in diagnosing the diseases, assisting patients or helping the medical staff in supplying various things or performing actions.
The medical imaging helps AI to diagnosis the diseases without help radiologist. And to develop the AI model using medical images to diagnosis various disease need training datasets for machine learning algorithm helping to detect with accuracy.
But just like AI, medical imaging is not improving faster making sometimes difficult or more time taking to diagnosis the diseases. Actually, while developing such highly sensitive models, high-quality training data is required and there is lack of such annotated data to train the models.
The more training data is feed into the model, it will learn the detection process with more accuracy resulting predicting the faster diagnosis. And to improve the quality and quantity of data, more dedicated and fast labeling process is required.
Hence, data annotation companies now use the AI-assisted labeling process to annotate the medical images at faster speed with better accuracy.  Compare to manual annotation, AI companies can get multiple times faster annotated data for machine learning algorithms.
Once the training data will start available in large quantity for medical imaging analysis process through AI will also improve. The AI-assisted data annotation process can only help to produce the medical imaging datasets in large quantity for better predictions in healthcare sector.


Is DDPG just for deterministic environments? I want to develop an AI for continuous space. I reached to DDPG algorithm that takes actions deterministically.

If DDPG takes actions deterministically, should the environment also be deterministic? I want non-deterministic, continuous real-world environments. Is DDPG the algorithm I am looking for? Is there any other algorithm for my need?

I am not an expert in this area. But I believe that the word "Deterministic" is for "Policy" in the "Deterministic Policy" Gradient. It does not mean  deterministic environment.

Stochastic policy: Probabilistic(random) action choice for a given state.
Deterministic policy: one action is chosen for a given state.

Deterministic Policy Gradient algorithm still can handle a stochastic (and continuous, of cause) environment, but the policy will be deterministic.

Reference

"in DDPG, the Actor directly maps states to actions (the output of the network directly the output) instead of outputting the probability distribution across a discrete action space"  -towards Data Science

Deterministic Policy Gradient Algorithms by Silver et al. PDF


How's the action represented in MuZero for Atari? MuZero seems to use two different methods to encode actions into planes for Atari games:

For the input action to the representation function, MuZero encodes historical actions as simple bias planes, scaled as $a/18$, where $18$ is the total number of valid actions in Atari.(from the appendix E of the paper)
For the input action to the dynamics function, Muzero encode an action as a one-hot vector, which is tiled appropriately into planes(from the appendix F of the paper)

I'm not so sure about how to make of the term "bias plane".

About the second, my understanding is that, as an example, for action $4$, we first apply one-hot encoding, which gives us a zero vector of length $18$ with one in the $5$-th position(as there are $18$ actions). Then we tile it and get a zero vector of length $36$, with ones in the $5$-th and $23$-rd positions. At last, this vector is reshaped into a $6\times 6$ plane as follows:

$$
0, 0, 0, 0, 1, 0\\ 0, 0, 0, 0, 0, 0\\ 0, 0, 0, 0, 0, 0\\
0, 0, 0, 0, 1, 0\\ 0, 0, 0, 0, 0, 0\\ 0, 0, 0, 0, 0, 0
$$

The bias plane is a layer everywhere equal to the constant $a/18$ where $a$ is the action. So each of the 32 frames has three frames for RGB and a fourth frame which is the bias plane making for 128 input layers. This is explained in the Network Architecture section where it mentions that the actions are "broadcast" to the planes.

For this I do not have conclusive evidence, but I think tiling a vector means arranging parallel copies of it into e.g. a grid shape. In other words, the input is 6x6x18, and action 1 is represented as all ones in the first plane with all zeros in the remaining planes. One problem with the way you've depicted is that the input is subject to convolutions, but there is no inherent reason why the fifth position and eleventh actions (which are next to each other vertically) should be included in the same 3x3 filter application, but the fifth and seventh (for example) actions should not.


What is the most common practice to apply batch normalization? For a deep NN, should I generally apply batch normalization after each convolution layer? Or only after some of them?  Which? Every 2nd, every 3rd, lowest, highest, etc.?

In the literature, it differs. You will see models do it after or before pooling only, and sometimes you see it after every single convolution.

Batch normalization's assistance to neural networks wasn't really understood for the longest time, initially it was thought to assist with internal covariate shift (hypothesized by the initial paper: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift) but lately has been tied to the optimization process (How Does Batch Normalization Help Optimization?).

This means, from an architectural perspective, it is difficult to correctly assume how it should be utilized, unless you really understand its impact on the loss landscape and how your optimization process will traverse it given some initialization (Note, by the way, a recent paper by Google showed that you can alleviate a lot of the benefits of batch normalization sheerly by understanding what issues it's resolving and attempting to mitigate them in the initialization process: Fixup Initialization).

So I would recommend 3 things until it is more understood how to utilize it generally:

Play around, get frisky and experiment. Use what works best.
Use block featurizers that are known to work well like residual blocks. Proven in practice and will probably work for you too.
Do the research and investigate it more, if you find the answer, you'll be helping a lot of people :)


What are the top contributions from neuroscience to AI and viceversa? How much progress has been made from the interaction between these two fields? What are the top contributions from neuroscience to artificial intelligence and vice versa?

How much progress has been made from the interaction between these two fields?

Is there a formal field dedicated to the research of topics in the intersection between AI and neuroscience, with papers being published and maybe conferences being held periodically?

Understanding a human brain fully, how hippocampus and neocortex works would surely help AI enthusiasts to make better AI algorithms with superior memory, ability to learn and finding what precisely intelligence, feelings and conciousness are by itself. The AI looks at psysiological and anatomical data as a source of suggestions regarding possible mechanisms and processes that could be used. Still AI doesn't deal that much from neuroscience.

From brain to AI : edge-detection using zero-crossing at multiple scales is a decent example where physiological data are used to suggest a solution to a fundamental problem in computer vision. In computational vision, a problem is the initial organization and representation of the input image. The goal is to transform the measurements of light-intensity level made by photoreceptors eyes' have, into a representation that is more concise. One of the most successful approaches to the problem had its origin in physiological studies of the processing that takes place at the level of the retinal ganglion cells which can be thought of as filtering the image with center-surround receptive fields whose cross-section can be approximated analytically by the second derivative of a gaussian. There had been proven research that a powerful edge detection scheme could be based upon the operation of such cells.

From AI to brain : Integration of motion perception measurement is a worth-mentioning example. It originates in studies of problem of measuring the motion of edges and contours in the image. Such motion can not be determined on the basis of purely local measurements. The underlying reason is 'aperture problem'. Selective visual attention and long-range motion analytics are another two fields where use of computational cosiderations are illustrated in the design and interpretation of physiological studies.


Better to learn the same small set for multiple epochs then go to the next or learn from each one time repeatedly for multiple times? I don't know if I worded the title correctly.

I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.

Now, I have a neural network (Using keras with tf). My question is, is it better to train each file individually for X epochs (File 1 for 5 epochs, then File 2 for 5 epochs, etc.), or should I do an epoch for each, repeatedly (File 1 for an epoch, File 2 for an epoch, etc., and repeat for 5 times).

I've used the first and I get an accuracy of about 88%. Whould I get an improvement by doing the latter?

Small set for multiple epochs causes overfit, so expose entire data in one epoch, then augment (change a littel bit), and train again. Use small amount of dropout also.

I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.

Do not store augmented data, just do augment with function right after next epoch and overwrite the dataset array


Why does C++ seem less widely used than Python in AI? I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.

You don't need a powerful language for programming AI. Most of the developers are using libraries like Keras, Torch, Caffe, Watson, TensorFlow, etc. Those low level libraries are highly optimized and handle all the tough work. They are built with high-performance languages, like C, C++. Python is just there for high level task like describing the neural network layers, load data, launch the processing, and display results.

Using C++ for high level task instead of Python would give barely any performance improvement, but it would be harder for non-developers as it requires to care for memory management. Also, several AI people may not have a very solid programming or computer science background.

Another similar example would be game development, where the engine is coded in C/C++, and, often, all the game logic scripted in a higher-level language.


Why does the BERT NSP head linear layer have two outputs? Here's the code in question.

https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L491

class BertOnlyNSPHead(nn.Module):
    def __init__(self, config):
        super().__init__()
        self.seq_relationship = nn.Linear(config.hidden_size, 2)

    def forward(self, pooled_output):
        seq_relationship_score = self.seq_relationship(pooled_output)
        return seq_relationship_score

I think it was just ranking how likely one sentence would follow another? Wouldn't it be one score?

This seems to be inherited from the original Google implementation, which also uses 2 outputs (https://github.com/google-research/bert/blob/master/run_pretraining.py#L293). I can see two possible reasons that the original implementation uses 2 outputs:

They are using the cross entropy loss, which typically works with log probabilities. To get probabilities they use softmax activation, which requires an output for each class. It is possible, of course, to compute cross entropy from sigmoid activations (which would correspond to a 1-output architecture), but there seems to be some confusion as to whether the output of the sigmoid function should be used as a probability.

Using 2 outputs can simplify the computation of the binary cross entropy loss, which, in typical Google fashion, is computed using low-level tensorflow ops rather than with tf.nn.softmax_cross_entropy_with_logits. Specifically,

-tf.reduce_sum(one_hot_labels * log_probs, axis=-1)

where one_hot_labels and log_probs are $\mathbb{R}^{N \times 2}$, is much easier to read than
-tf.reduce_sum(binary_labels * tf.math.log(probs) + (1 - binary_labels) + tf.math.log(1 - probs))

where binary_labels and probs are $\mathbb{R}^N$.


How to estimate a behavior policy for off-policy learning based on data? I have a dataset which includes states, actions, and reward. The dataset includes information on the transition, i.e., $p(r,s' \mid s,a)$.

Is there a way to estimate a behavior policy from this dataset so that it can be used in an off-policy learning algorithm?

You can simply train a policy from the inputs to predict the actions in your dataset. You can use the cross entropy loss for this, i.e. maximize the the log probability that the policy assigns to the actions in the data set when given the corresponding inputs. This is called behavioral cloning.

The result is an approximation of the behavioral policy that lets you compute probability densities of actions. It is an approximation because the dataset is finite, and even more so when you restrict the learned policy to a class of distributions, e.g. Gaussians.


What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN? What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional Neural Network to solve it? Are there any examples i.e. papers can show by replacing ordinary CNN with Graph Convolutional Neural Network, an accuracy increasement or a quality improvement or a performance gain is achieved? Can anyone introduce some examples as image classification, image recognition especially in medical imaging, bioinfomatics or biomedical areas?

Generally speaking a graph CNN is applied to data represented by graphs, not images.

a graph is a collection of nodes and edges connecting them.
an image is a 2D or 3D matrix, in which each element denotes a pixel in space

If your data are just images, or something similar (e.g. some fMRI data), you usually cannot benefit from graph CNN compared with usual CNN.

Sometimes, the class labels of your images may be organized in a graph-like  (or tree-like) structure. In that case, you may have a chance to benefit from graph CNN.


Mining repeated subsequences in a given sequence Given an alphabet $I=\left\{i_1,i_2,\dots,i_n\right\}$ and a sequence $S=[e_1,e_2,\dots,e_m]$, where items $e_j \in I$, I am interested in finding every single pattern (subsequence of $S$) that appears in $S$ more than $N$ times, and that has a length $L$ between two limits: $m<L<M$. In case of overlapping between patterns, only the longest pattern should be considered.
For example, given $N=3$, $m=3$ and $M=6$, an alphabet $I=\left\{A,B,C,D,E,F\right\}$, and the following sequence $S$ (the asterisks simply mark the positions of the patterns in this example):
$$
S=[A,A,*F,F,A,A*,B,D,E,F,E,D,*F,F,A,A*,F,C,*C,A,B,D,D*,C,C,*C,A,B,D,D*,C,A,C,B,E,A,B,C,*F,F,A,A*,E,A,B,C,A,D,E,*F,F,A,A*,B,C,D,A,E,A,B,*C,A,B,D,D*,]
$$
The sought algorithm should be able to return the following patterns:
$$
[C,A,B,D,D], [F,F,A,A]
$$
together with their respective positions within $S$.
An available Python implementation would be very desirable (and a parallel implementation, even more so).
I was reading about BIDE algorithms, but I think this is not the correct approach here. Any ideas? Thanks in advance!

You can do this similar to the BIDE approach. It can be done like this:
class TreeNode:
    def __init__(self, element, depth, count=0, parent=None):
        self.count= count
        self.element= element
        self.depth= depth
        self.subnodes= dict()
        self.parent= parent

    def __repr__(self):
        return f'{self.__class__.__name__}({self.element}, {self.depth}, {self.count})'

    def get_subnode(self, element, create=True):
        if create is True:
            return self.subnodes.setdefault(element, TreeNode(
                element,
                self.depth+1,
                count=0,
                parent=self
            )
        )
        else:
            return self.subnodes.get(element)

    def get_subnode_increasing_count(self, element):
        subnode= self.get_subnode(element)
        subnode.count+= 1
        return subnode

    def harvest(self, N, m, M, prefix=[], append_result_to=None):
        # get all sequences which occurs more than N times
        # and have a length (depth) of m < depth < M
        prefix= prefix.copy()
        prefix_len= len(prefix)
        if append_result_to is None:
            result= list()
        else:
            result= append_result_to
        if self.element is not None:
            prefix.append(self.element)
            prefix_len+= 1
        if N >= self.count or prefix_len >= M:
            return result
        if N < self.count and m < prefix_len and prefix_len < M:
            # the subsequence matches the constraints
            result.append(prefix)
        for subnode in self.subnodes.values():
            subnode.harvest(
                N,
                m,
                M,
                prefix=prefix,
                append_result_to=result
            )
        return result

    def get_prefix(self):
        if self.parent is None:
            prefix= []
        else:
            prefix= self.parent.get_prefix()
        if self.element is not None:
            prefix.append(self.element)
        return prefix

    def print_tree_counts(self, leaves_only=False, min_count=0):
        if leaves_only is False or not self.subnodes:
            if self.count >= min_count:
                print(self.get_prefix(), self.count)
        for subnode in self.subnodes.values():
            subnode.print_tree_counts(leaves_only=leaves_only, min_count=min_count)

sequential approach
def find_patterns(S, N, m, M):
    root= TreeNode(None, 0, 0)
    active_nodes= []
    for el in S:
        root.count+=1
        # append the root node
        active_nodes.append(root)
        # now replace all nodes in active nodes by the node
        # that is reached one level deeper following el
        active_nodes= [node.get_subnode_increasing_count(el) for node in active_nodes if node.depth < M]

    # now harvest the tree after applying the restrictions
    return root, root.harvest(N, m, M)

S='AAFFAABDEFEDFFAAFCCABDDCCCABDDCACBEABCFFAAEABCADEFFAABCDAEABCABDD'
root, patterns= find_patterns(S, 3, 3, 6)
patterns

The result is:
[['F', 'F', 'A', 'A']]

Your second pattern occurs exactly 3 times and so doesn't fullfill the requirement of > 3 occurances ([C,A,B,D,D]).
modifications for parallel processing
To make it processable in parallel you can do a slide modification. Just create another method in TreeNode, that allows to merge nodes. Like this:
def merge_nodes(self, other_nodes):
    # merge other_nodes into this node
    # including all subnodes
    if len(other_nodes) > 0:
        elements= set()
        for other_node in other_nodes:
            self.count+= other_node.count
            elements.update(other_node.subnodes.keys())
        # elements now contains the set of the next elements
        # with which the sequence continues across the
        # other nodes
        for element in elements:
            # get the node of the resulting tree that represents
            # the sequnce continuing with element, if there is
            # no such subnode, create one, since there is at least
            # one other node that counted sequence seq + element
            my_subnode= self.get_subnode(element, create=True)
            other_subnodes= list()
            for other_node in other_nodes:
                # now get the subnode for each other node, that
                # represents the same sequence (seq + element)
                other_subnode= other_node.get_subnode(element, create=False)
                if other_subnode is not None:
                    other_subnodes.append(other_subnode)
            # merge the subnode the same way
            my_subnode.merge_nodes(other_subnodes)

Now you can call find_patterns on subtrees, but you need to take into account, that you may not split the input sequence normally. You need to define some overlapping sequence part, so that patterns, that begin at the end of one sequence-fragment can be completed but that this overlapping sequences need to be counted differently (otherwise you get double-counts, which lead to wrong inputs). So you have to make sure, that only patterns, which began in the sequence fragment are continued with the overlap part, but that no new patterns are started in the overlap part, because you count them elsewere:
def find_patterns(S, N, m, M, overlap=[]):
    root= TreeNode(None, 0, count=0, parent=None)
    active_nodes= []
    def process_element(active_nodes, element, M):
        return [
            node.get_subnode_increasing_count(element)
                for node in active_nodes
                    if node.depth < M
        ]
    for el in S:
        root.count+=1
        # append the root node
        active_nodes.append(root)
        # now replace all nodes in active nodes by the node
        # that is reached one level deeper following el
        active_nodes= process_element(active_nodes, el, M)
    # complete the already started sequences with the
    # overlapping sequence (the sequence that may be
    # processed in another process)
    for el in overlap:
        active_nodes= process_element(active_nodes, el, M)
    # now harvest the tree after applying the restrictions
    return root, root.harvest(N, m, M)

Now you can do:
split_point= len(S) // 2
S1= S[split_point:]
S2= S[:split_point]
S1_overlap= S2[:10]
# in fact you could have just used :5 above,
# but that doesn't affect the result
r1, _= find_patterns(S1, 3, 3, 6, overlap=S1_overlap)
# Note: you could safely start the comand in the next
# line in parallel to the previous call of find_patterns
# and of course, you could split your sequence in as
# many fragments, as you like, as long as you maintain
# the overlap correctly and call merge_nodes afterwards
r2, _= find_patterns(S2, 3, 3, 6)
r1.merge_nodes([r2])
patterns= r1.harvest(3, 3, 6)

This yields the same result as above. To make it a bit clearer, what I mean with the overlap part:
S1 is 'CBEABCFFAAEABCADEFFAABCDAEABCABDD'
S2 is 'AAFFAABDEFEDFFAAFCCABDDCCCABDDCA'
S1_overlap is 'AAFFAABDEF'

With find_patterns(S1, 3, 3, 6) you would only search for repeated patterns in S1, so it would not consider patterns which begin with the part BDD (starting at the end of S1) and are continued within S2. With find_patterns(S1, 3, 3, 6, overlap=S1_overlap) I consider these patterns as well.


How do weak learners become strong in boosting? Boosting refers to a family of algorithms which converts weak learners to strong learners. How does it happen?

As @desertnaut mentioned in the comment

No weak learner becomes strong; it is the ensemble of the weak learners that turns out to be strong

Boosting is an ensemble method that integrates multiple models(called as weak learners) to produce a supermodel (Strong learner).

Basically boosting is to train weak learners sequentially,  each trying to correct its predecessor. For boosting, we need to specify a weak model (e.g. regression, shallow decision trees, etc.), and then we try to improve each weak learner to learn something from the data.
AdaBoost is a boosting algorithm where a decision tree with a single split is used as a weak learner. Also, we have gradient boosting and XG boosting.


OpenAI spinning up convolutional networks with PPO I am using pytorch version of PPO and I have image input that I need to process with convolutional neural networks, are there any examples on how to set up the network? I know that stable baselines  support this to some extend, but I had better performance with spinning up so I would prefer to keep using these.

I have written a tutorial on using OpenAI Spinning Up in a image-based PyBullet + Gym environment here

In order to be able to use spinup for an image-based environment I had to fork it here and add CNN to PPO's core.py


Is there a document with a list of conjectures or research problems regarding reinforcement learning (like the Millennium Prize Problems)? Is there a document with a list of conjectures or research problems regarding reinforcement learning like the Millennium Prize Problems?

It's not really an exhaustive list, but Hutter maintains a small list of problems (click on the bullet point "Universal AI Book" here) related to AIXI (a reinforcement learning agent), some of which have already been solved. The money awards are in the range of 50-500 euros, so they are not as financially important as the Millennium Prize Problems.


Is artificial intelligence and, in particular, neural networks being used in real-world critical applications? Is artificial intelligence and, in particular, neural networks being used in real-world critical applications and devices?

I had a discussion with my colleague who states that nobody would use artificial intelligence, especially neural nets, for critical stuff, like technical devices or sensors.

I'm only aware of the problem of neural nets being so-called black-boxes, but, nevertheless, I think it is possible to make an NN robust so that it matches the demands of daily processes, also in sensitive fields like health care, energy market, self-driving cars, and so on. Yet I cannot underline this.

Does somebody have more insights or other information, opinions and so on? I appreciate any meaningful answer.

This is as much an ethical concern as a practical one.

AI systems are already reaching or exceeding human performance in many critical areas. Consider the detection of common cancers, where AI systems match or exceed humans. Another good example is Tesla's Autopilot, which is actually safer than human drivers, but gets a lot of bad press when it makes a mistake. Both of these systems are likely using Neural Networks, possibly alongside other heuristic or rule-drive approaches.

The issue isn't whether these systems can be "safe enough" for everyday use. They are safe enough in a societal sense already. The concern is that the people who die when these systems make mistakes are randomly selected, whereas the people who die when a human performs the work die because a human makes a mistake (usually). This is difficult to accept for the same reason that some people are scared of flying, even though it is many times safer than driving the same distance: there is a loss of control, and "good" people may die through no fault of their own, or perhaps through events that are no one's fault.

Whether we use this systems will thus probably depend on the application. In Medicine it's easy to see a case: we can do more tests than before for the same price. People who can afford to have a doctor review the machine's decisions are probably no worse off. People who couldn't afford this already are better off (they get a diagnosis with some positive predictive power value now, instead of none at all). In driving, it's more complicated, and will probably require further development. No one knows for sure how good self-driving cars can get, but they'll probably get somewhat better over time. Maybe they'll get good enough that they more or less never kill people, or maybe they won't.

I actually think your friend is way off about the use cases he thinks AI is not suitable for though. Check out this article in Military Embedded Systems. In applications where decisions are made on a pure cost/benefit basis (and not based on people's gut feelings about morality), AI systems are actually easier to adopt, and are already often better than human operators. This trend seems likely to increase in the future, so "technical devices and sensors", which are often black-boxes to the lay public anyway, seem like they are among the first things to go.


What would be the most effective self-learning algorithm for a 7 player social deduction game? There's this 7 player social deduction game called Secret Hitler, and I have been trying to find a self-learning AI algorithm to learn how to play this game for a while.  Basically, four players are given a liberal role, two players are given a fascist role, and 1 player is given a hitler role.  The liberals and hitler do not know any other roles, and the fascists know everyone's roles.  During a turn, a president elects a chancellor based on a yes/no vote and then the government passes a policy (either liberal or fascist) that is drawn from a randomized deck. At certain points in the game, different special abilities come into play, like executing a player or investigating their role. To win the game, the liberals must either enact 5 liberal policies or kill hitler; the fascists must enact 6 fascist policies or get hitler enacted as the chancellor after 3 fascist policies have been enacted.

Now, there are other details that are irrelevant that I didn't mention, but those are the general rules. It seems simple enough to build a visual implementation in a language like Java, but there are so many moving pieces that I would have to account for.  I doubt that simply making random moves at first and learning off of the bad/good moves would work, because I need a way for agents to make moves based on which roles they know.

Unfortunately, AlphaZero wouldn't work here, and I'm struggling to find any algorithm that would work for this (or any other social deduction game).  Do I have to write my own algorithm?  I'm slightly confident that this is a case of supervised learning where I can give weight to the nodes in a neural network that correspond to wins, but please correct me if I'm incorrect.

It is very likely that you want an algorithm like Counterfactual Minimax Regret. This algorithm has several variants, but they differ mostly in their efficiency.

CFR is the algorithm that was used to solve 2-Player Poker, although the solution comes from one of its more advanced versions. The algorithm is highly applicable to other games of incomplete information and games with roles, like Secret Hitler. Essentially, it learns how to act so that the actions it takes leak as little information to the other players as possible about its role (or hand of cards), while also learning about the roles of other players from their actions, and maximizing its chances of winning the game.


How to predict the "word" based on the meaning in a document? What I mean to say is

For example, if I give the meaning of Apple from the dictionary as input to the program, it should give output as Apple.
Or I say My day to day job involves monitoring and managing the resources - the output should be Project management.

The meaning and the word could be a dictionary or it could be custom. I am looking for ideas and tools to go further on this.
Thanks for the help.

While your question has some ambiguities, I try to answer.
From my understanding you want your model to predict “topic” of a sentence or a description. It’s just a classification problem with huge possible number of output classes.
The first initial issue is very short length of documents (sentences). Most of topic modelling algorithms such as LDA have statistical approach and do not work very well with very short documents (less than 50 words could be a good definition of very short document).
The second issue is how do you want to collect enough data to train your model that supposed to predict target out of extremely large number of output classes? Dictionaries are not enough because they offer a single definition for each word. Examples of words in dictionaries don’t help much and they will probably affect your model adversely. How can your model be generalised by a single (or few) example(s) for each class?
So, it’s not possible, but maybe having some innovations can help.
Here is definition of “apple” in oxford dictionary: ”a round fruit with shiny red or green skin that is fairly hard and white inside”. There are just two nouns in the definition: "fruit" and "Skin", if we just read the definition without considering these two words, even we, as human, struggle to guess.
Consider nouns in input data and use them to build up a natural graph. You just consider main classes such as "fruit". If you’re getting some good results, consider other words, adj, adv, ...


What is a non-starving policy in reinforcement learning? In the paper, Eligibility Traces for off-Policy Policy Evaluation (2010), by Doina Precup et al., mentioned the term "non-starving" many times. The specific use of the term was like "non-starving policy" in the context of off-policy learning.

A specific mention of the term

  we consider a method that requires nothing of the behavior
  policy other than that it be non-starving, i.e., that it
  never reaches a time when some state-action pair is never
  visited again.

What does the thing look like intuitively? Why is it required?

A non-starving policy is a (behavior) policy that is theoretically guaranteed to visit each state and take all possible actions from each state an infinite number of times, so that to always update $Q(s, a)$, $\forall s, \forall a$, an infinite number of times. In the context of off-policy prediction, this criterion implies that any trajectory will have no zero probability under a behavior policy. As a consequence, the experience from the behavior policy sufficiently covers the possibilities of any target policy.

An example of a non-starving policy is the $\epsilon$-greedy policy, which, with $0 < \epsilon \leq 1$ (which is usually a small number between $0$ and $1$) probability, takes a random action from a given state, and, with $1-\epsilon$ probability, takes the current best action, that is, the action with the highest value from a given state, according to the current value function.


How are filters weights updated for a CNN? I've been trying to learn backpropagation for CNNs. I read several articles like this one and this one. They all say that to compute the gradients for the filters, you just do a convolution with the input volume as input and the error matrix as the kernel. After that, you just subtract the filter weights by the gradients(multiplied by the learning rate). I implemented this process but it's not working.

Here's a simple example that I tried:

Input volume (randomised)

 1 -1  0
 0  1  0
 0 -1  1

In this case, we want the filter to only pick up the top left 4 elements. So the target output will be:

 1  0(supposed to be -1, but ReLU is applied)
 0  1

We know that the desired filter is:

 1  0
 0  0

But we pretend that we don't know this.

We first randomise a filter:

 1 -1
 1  1

The output right now is:

 3  0
-2  1

Apply ReLU:

 3  0
 0  1

Error (target - output):

-2  0
 0  0

Use error as kernel to compute gradients:

-2  2
 0 -2

Say the learning rate is 0.5, then the new filter is:

2 -2
1  2

This is still wrong! It's not improving at all. If this process is repeated, it won't learn the desired filter. So I must have understood the math wrong. So what's the problem here?

You have made many wrong assumptions in this question. First theoretically speaking,

Filters do not work in the way to 'pick up elements' (they work on principle of edge detection).
You have assumed only a single combination of filter weights will give the desired output (assuming continuous weights not binary). This is especially in prominence in the problem of Regularization where we want to choose a set of weights without over-fitting data.
The error you used looks very similar to Perceptron update rule (squared error gives the same derivative, but make sure you are not confusing the two).
Backpropagation through 'dead ReLu's' is not possible (see this answer for more details).

Now, let us check mathematically:

  Input Volume:

$$
    \begin{matrix}
    1 & -1 & 0 \\
    0 & 1 & 0 \\
    0 & -1 & 1 \\
    \end{matrix}
$$

  Desired Output:

$$
    \begin{matrix}
    1 & -1 \\
    0 & 1  \\
    \end{matrix}
$$

Note, in this step you desire an output which is negative (element (0,-1)), but you are forward propagating through a ReLu which is cutting off the negative part, thus the gradients have no way to communicate or update the required negative.
Basically,

$ wx \rightarrow ReLu \rightarrow y$ is happening and if 'x' is a negative number then $y$ is always $0$ thus $(target-y)$ is always $target$ and hence whatever the value of $x$ error remains constant, and if we want to backpropagate (assuming squared error) then:

$\frac{d}{dw} (target - y)^2 = 2*(target - y)*\frac{d}{dw}y = -2*(target - y)*0 = 0$ (Remember from the ReLu output graph slope is $0$ in the negative region).

  Now, you randomise a filter:

$$
    \begin{matrix}
    1 & -1 \\
    1 & 1  \\
    \end{matrix}
$$

apply ReLu and get the following:

$$
    \begin{matrix}
    3 & 0 \\
    0 & 1  \\
    \end{matrix}
$$

again you have chosen you target to have a negative number which is not possible in the case of ReLu activation.

But continuing you get the error as:

$$
    \begin{matrix}
    -2 & -1 \\
    0 & 0  \\
    \end{matrix}
$$

use error to compute gradients (which again you have calculated wrong,by Backpropagating through ReLu's with ) values and also missed the minus sign associated with the output, but you have compensated it by adding it to the $w$ whereas the convention is to subtract):

$$
    \begin{matrix}
    1 & 2 \\
    1 & -2  \\
    \end{matrix}
$$

And get the new filter as:

$$
    \begin{matrix}
    0.5 & 0 \\
    0.5 & 0  \\
    \end{matrix}
$$

This is a pretty good approximation of the desired filter (even though the previous steps have wrong assumptions, but it does not matter much, since what you essentially did was use a linear activation function which will work, if you go through enough iterations). So basically you are using a Linear Filters and the details are too hodge podge for me to go into, so I will suggest some resources for you to see ReLu backpropagation:

Deep Neural Network - Backpropogation with ReLU


How to identify the face of a certain customer in a grocery store? How would you solve the problem of identifying certain customer in a grocery store? Suppose our client is already signed-up on our website with an unique ID given to him. To come in to the store, firstly he must place a phone with a QR code in front of the reader, so that our server is informed on a specific client entering the shop. Then after successful authentication he heads towards the shelf with goods and pick some item. Identifying product name turned out not to be a challenge, as opposed to person.

From my perspective, the solution is to make a several photo of face from a different angle while person coming in. Quickly train CNN and feed it with face images when customer picks goods in order to choose the corresponding one.

So, what are your thoughts on this issue? What approach would you take to work it out?

You might consider pre-training a CNN on a large dataset. The CNN should be structured such that you input 2 different images and the CNN predicts whether or not they are the same person. Your dataset should include images from multiple angles, with and without occlusions like sunglasses, and with changes in hair. (One dataset useful for this is the AR Face Database.) Then you can just check if the face matches any of the pictures you took as they entered the store.


Random value generator using a single neuron or DNN AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be.

So 'generating random value' is also a task included in the scope that AI should be able to do

I'm trying to generate random value using a single neuron but the outcome isn't much good.
Any suggestions?

PS.
Random weight initialisation is allowed coz weights are constants at start.
Using 'random' function is forbidden anywhere else.

AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be.

Technically you would need AGI (Artifical General Intelligence) to do anything a human can do. This is not a technology that exists, but a goal of some AI research to perform more and more general tasks.

  So 'generating random value' is also a task included in the scope that AI should be able to do

Humans are actually very bad at generating random values directly. In fact it is possible to identify a person by getting them to generate random numbers from their mind. Indirectly, a human could pick up a die, roll it, and read the number - this is also something that is within capabilities of modern AI, but not something you could implement with a small neural network.

Computer software cannot generate random numbers directly, only pseudo-random numbers that are deterministic but follow a statistical pattern very similar to theoretical randomness. When combined with frequently changing seed data from the environment, this becomes very close to an ideal random source, but it does require some hardware input. Modern computer chips include an internal source of randomness to provide this seed data, but it is important to note that it is a piece of specialised hardware, not something that can be coded or simulated using a simple neural network.

  I'm trying to generate random value using a single neuron but the outcome isn't much good. Any suggestions?

This is going to be much harder than you assumed. Random number generators are always more complex than a single function, and require at least a little bit of architecture. The simplest architecture would be to add a state value to use as the next input to the generator, and which is somehow affected by the previous output. A simple feed-forward neural network does not have any architecture like this, you need to add it.

My first suggestion would be to learn how some simple software pseudo-random number generators (PRNGs) work. A good place to start might be linear congruential generators which are considered very poor quality PRNGs nowadays, but are the sort found in early computer systems.

It should be possible to create a rough approximation to a PRNG from a single neuron. Picking the weight, bias and non-linear functions carefully should allow you to produce something similar to the linear congruential generator, where it generates the next pseudo-random number when you input the previous one. A slightly more sophisticated approach would be to make this a simple recurrent neural network (RNN) so that the neural network has an internal state - such a network could still have a single neuron.

An example neuron that works like this might have a weight of 3, bias of 1 (to generate a logit from input $x$ of $z= 3x+1$) and have activation function $y = 1000\text{sin}(z) \text{ mod } 1$. This is not a normal activation function that you would use to train a neural network - in fact this neural network probably could not be trained at all. It has been specially designed to achieve your goal of generating (pseudo-)random numbers using a single neuron. It is a valid neural network though.

I created the above example neuron, started it with a seed of 0, and got the sequence 0.4710, 0.8543, 0.1536, 0.9479, 0.1142, 0.0683, 0.7741, 0.2985, 0.7315, 0.0777, 0.5078 by feeding back previous values - you can see it is working, although it is probably a very bad random number generator even compared to linear congruential generators. Possibly better than a human though.