Effect of batch size and number of GPUs on model accuracy I have a data set which was split using a fixed random seed and I am going to use 80% of data for training and rest on validation. Here are my GPU and batch size configurations

use 64 batch size with one GTX 1080Ti
use 128 batch size with two GTX 1080Ti
use 256 batch size with four GTX 1080Ti

All other hyper-parameters such as lr, opt, loss, etc., are fixed. Notice the linearity between the batch size and the number of GPUs.

Will I get the same accuracy for those three experiments? Why and why not?

This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a batch size too small(for example 1) will make the model hard to generalize and also slower to converge.

Source
This slide is a great demonstration of how batch size affects training. As you can see from the diagram, when you have a small batch size the route to convergence will be ragged and not direct. This is because the model may train on an outlier and have it's performance decrease before fitting again. Of course this is an edge case and you would never train a model with 1 batch size. However on the other hand with a batch size too large, your model will take too long per iteration. With at least a decent batch size (like 16+) the amount of iterations needed to train the model is similar, so a larger batch size is not going to help a lot. The performance is not going to vary a lot.
In your case, the accuracy will make a difference but only minimally. Whilst writing this answer, I have ran a few test on batch size effect on performance and time, and here is the results. (Results to be added for 1 batch size)
Batch size 256 Time required 98.50849771499634s : 0.9414
Batch size 128 Time required 108.53689193725586s : 0.9668
Batch size 64 Time required 129.92272853851318s : 0.9776
Batch size 32 Time required 162.13709354400635s : 0.9844
Batch size 16 Time required 224.82269191741943s : 0.9854
Batch size 8 Time required 351.2729814052582s : 0.9861
Batch size 4 Time required 514.2667407989502s : 0.9862
Batch size 2 Time required 829.1623721122742s : 0.9869

You can test out yourself here.

As you can see, accuracy increase with the batch size decreasing and this is because a higher batch size means it will be trained on less iterations. 2x batch size = half the iterations so this is expected. The time required have risen exponentially, but batch size of 32 or below don't seems to make a large difference in time taken. The accuracy seems to be normal as half the iterations are trained with double the batch size.
In your case I would actually recommend you stick with 64 batch size even for 4 GPU. In the case of multi-gpu, the rule of thumb will be using at least 16 or so batch size per GPU, as if you are using 8 or 4 batch size the GPU cannot be completely utilized to train the model.
For multiple GPU accuracy impact, there is virtually no impact on accuracy as far as I know. Multiple GPU just take it's own slice of batches and computes it and feed the output to one GPU to calculate the loss, and the operations done across the GPUs are the same operations done on 1 GPU, so it shouldn't matter.
EDIT:
For multiple GPU, there might me a slight difference due to percision error as told by @Posi2 . Please see here. Thanks @Posi2
Conclusion
Batch size don't matter to performance too much as long as you set a reasonable batch size (16+) and keep the iterations not epochs the same. However training time will be affected and for multi-gpu you should use the minimum batch size for each GPU that will utilize 100% of the GPU to train. 16 per GPU is quite good.


Understanding proof of lemma 1 (policy improvement bound) of the "Trust Region Policy Optimization" paper In the Trust Region Policy Optimization paper, in Lemma 1 of Appendix A, I did not quite understand the transition from (21) from (20). In going from (20) to (21), $A^\pi(s_t, a_t)$ is substituted with its value. The value of $A^\pi(s_t, a_t)$ is given as $\mathbb{E}_{s'∼P(s'|s,a)}[r(s) + \gamma V_\pi(s') − V_\pi(s)]$ at the very begining of the proof. But when $A^\pi(s_t, a_t)$ gets substituted, I don't see the expectation (over $s'∼P(s'|s,a)$) appearing anywhere. It will be of great help if somebody lends some light on this.

Let's assume $\gamma = 1$ to simplify things
\begin{align}
\mathbb E_{\tau|\pi} [\sum_{t = 0}^{\infty} A_\pi(s_t, a_t)] &= \mathbb E_{\tau|\pi}[A_\pi(s_0, a_0) + \ldots A_\pi(s_i, a_i) + \ldots]\\
&= \mathbb E_{a_0 \sim \pi,s_1 \sim P(s_1|s_0, a_0)}[A_\pi(s_0, a_0)] + \ldots + \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] + \ldots
\end{align}
if we observe only $i$-th timestep
\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a')]) \pi(a'|s_i)\\
&= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]]) \pi(a'|s_i)
\end{align}

\begin{equation}
\mathbb E[\mathbb E[f]] = \mathbb E[f]
\end{equation}

\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]) \pi(a'|s_i)\\
&= \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]
\end{align}

now for all timesteps sum everything up.


Key Point Extraction the best method? I have been researching about determining some key points on an image, in this case I'm gonna use cloth (top side of human body) pictures. I want to detect some corner points on those.

Example:

I have two solutions on my mind. One CNN with transpose layers resulting in heatmap where I can get points. The second is to get 24 number as output from the model meaning 12(x,y) point. I don't know which one will be better.

In face point detection, they use the second method. In human pose estimation, they use method one. So what do you suggest me to use? or do you have any new ideas? Thanks

The 2nd method would make sense only if your object is at the same position in all test images. You would have such situation if you operated on crops located by a separate object detection algorithm. This happens to be the case in facial key-point detection.

The 1st method would be much more robust to various object poses since it is translation covariant by design. If a keypoint is detected at location A, it will be equally well detected at any other position with the same set of parameters.


Is my understanding of how the convolution with stride 2 works in this example correct? I'm currently reading this explanation of convolutional neural networks and there's a part around strides that I don't quite understand. I'm just starting with this, so I apologize if this is a really basic question. But I'm trying to develop an understanding and some of these images have thrown me off.
Specifically, in this image

The stride has been increased to 2 and it's using a 3x3 filter (represented by the red, green and blue outline squares in the first picture)
Why is the blue square below the red one and not shifted to the side of the green one ending at the edge of the 7x7 volume? Should it not move left to right then down 2 squares when it reaches the next line?
I'm not sure if the author is just trying to show the stride moving down as it goes, but I think my confusion stems from the fact that the 1 stride image example is only moving in the horizontal direction (as seen below).

Is there something fundamental I haven't grasped here?

Yes, you're right, after the green one, it should also move two steps (because stride = 2) to the right once more. Note that in the $3 \times 3$ output volume picture, there's also still a white cell in the top right corner. That cell would get filled with whatever colour you choose to move to the right after the green one.
The blue one would then follow after what I described above, as the fourth square. I guess the author simply didn't feel like drawing a third square to the right of the green one, because the red + green squares already illustrate how the pattern works. The blue one was probably additionally drawn to illustrate that stride works the same way vertically as it does horizontally, e.g. blue also moves down two rows (because stride = 2).
See the bottom of the picture in the second answer here.


What is a fully convolution network? I was surveying some literature related to Fully Convolutional Networks and came across the following phrase,

  A fully convolutional network is achieved by replacing the parameter-rich fully connected layers in standard CNN architectures by convolutional layers with $1 \times 1$ kernels.

I have two questions.

What is meant by parameter-rich? Is it called parameter rich because the fully connected layers pass on parameters without any kind of "spatial" reduction?
Also, how do $1 \times 1$ kernels work? Doesn't $1 \times 1$ kernel simply mean that one is sliding a single pixel over the image? I am confused about this.

Fully convolution networks
A fully convolution network (FCN) is a neural network that only performs convolution (and subsampling or upsampling) operations. Equivalently, an FCN is a CNN without fully connected layers.
Convolution neural networks
The typical convolution neural network (CNN) is not fully convolutional because it often contains fully connected layers too (which do not perform the convolution operation), which are parameter-rich, in the sense that they have many parameters (compared to their equivalent convolution layers), although the fully connected layers can also be viewed as convolutions with kernels that cover the entire input regions, which is the main idea behind converting a CNN to an FCN. See this video by Andrew Ng that explains how to convert a fully connected layer to a convolutional layer.
An example of an FCN
An example of a fully convolutional network is the U-net (called in this way because of its U shape, which you can see from the illustration below), which is a famous network that is used for semantic segmentation, i.e. classify pixels of an image so that pixels that belong to the same class (e.g. a person) are associated with the same label (i.e. person), aka pixel-wise (or dense) classification.

Semantic segmentation
So, in semantic segmentation, you want to associate a label with each pixel (or small patch of pixels) of the input image. Here's a more suggestive illustration of a neural network that performs semantic segmentation.

Instance segmentation
There's also instance segmentation, where you also want to differentiate different instances of the same class (e.g. you want to distinguish two people in the same image by labeling them differently). An example of a neural network that is used for instance segmentation is mask R-CNN. The blog post Segmentation: U-Net, Mask R-CNN, and Medical Applications (2020) by Rachel Draelos describes these two problems and networks very well.
Here's an example of an image where instances of the same class (i.e. person) have been labeled differently (orange and blue).

Both semantic and instance segmentations are dense classification tasks (specifically, they fall into the category of image segmentation), that is, you want to classify each pixel or many small patches of pixels of an image.
$1 \times 1$ convolutions
In the U-net diagram above, you can see that there are only convolutions, copy and crop, max-pooling, and upsampling operations. There are no fully connected layers.
So, how do we associate a label to each pixel (or a small patch of pixels) of the input? How do we perform the classification of each pixel (or patch) without a final fully connected layer?
That's where the $1 \times 1$ convolution and upsampling operations are useful!
In the case of the U-net diagram above (specifically, the top-right part of the diagram, which is illustrated below for clarity), two $1 \times 1 \times 64$ kernels are applied to the input volume (not the images!) to produce two feature maps of size $388 \times 388$. They used two $1 \times 1$ kernels because there were two classes in their experiments (cell and not-cell). The mentioned blog post also gives you the intuition behind this, so you should read it.

If you have tried to analyze the U-net diagram carefully, you will notice that the output maps have different spatial (height and weight) dimensions than the input images, which have dimensions $572 \times 572 \times 1$.
That's fine because our general goal is to perform dense classification (i.e. classify patches of the image, where the patches can contain only one pixel), although I said that we would have performed pixel-wise classification, so maybe you were expecting the outputs to have the same exact spatial dimensions of the inputs. However, note that, in practice, you could also have the output maps to have the same spatial dimension as the inputs: you would just need to perform a different upsampling (deconvolution) operation.
How $1\times 1$ convolutions work?
A $1 \times 1$ convolution is just the typical 2d convolution but with a $1\times1$ kernel.
As you probably already know (and if you didn't know this, now you know it), if you have a $g \times g$ kernel that is applied to an input of size $h \times w \times d$, where $d$ is the depth of the input volume (which, for example, in the case of grayscale images, it is $1$), the kernel actually has the shape $g \times g \times d$, i.e. the third dimension of the kernel is equal to the third dimension of the input that it is applied to. This is always the case, except for 3d convolutions, but we are now talking about the typical 2d convolutions! See this answer for more info.
So, in the case we want to apply a $1\times 1$ convolution to an input of shape $388 \times 388 \times 64$, where $64$ is the depth of the input, then the actual $1\times 1$ kernels that we will need to use have shape  $1\times 1 \times 64$ (as I said above for the U-net). The way you reduce the depth of the input with $1\times 1$ is determined by the number of $1\times 1$ kernels that you want to use. This is exactly the same thing as for any 2d convolution operation with different kernels (e.g. $3 \times 3$).
In the case of the U-net, the spatial dimensions of the input are reduced in the same way that the spatial dimensions of any input to a CNN are reduced (i.e. 2d convolution followed by downsampling operations). The main difference (apart from not using fully connected layers) between the U-net and other CNNs is that the U-net performs upsampling operations, so it can be viewed as an encoder (left part) followed by a decoder (right part).


Can neural networks modify their own weights without back-propagation and gradient descent? Can neural networks modify their own weights without back-propagation and gradient descent?

Artificial neural networks (ANNs) do not modify their own weights! Humans create algorithms that modify the weights or architecture of ANNs.

Having said that, you can change the weights of ANNs using other methods other than gradient descent combined with back-propagation, such as Hebbian learning, evolutionary algorithms or reinforcement learning.


Can a technological singularity only occur with superintelligence? In Chapter 26 of the book Artificial Intelligence: A Modern Approach (3rd edition), the textbook discusses "technological singularity". It quotes I.J. Good, who wrote in 1965:

  Let an ultra-intelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra-intelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind. Thus the first ultra-intelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

Later on in the textbook, you have this question:

  26.7 - I. J. Good claims that intelligence is the most important quality, and that building ultr- intelligent machines will change everything. A sentient cheetah counters that "Actually speed is more important; if we could build ultrafast machines, that would change everything" and a sentient elephant claims "You're both wrong; what we need is ultrastrong machines," What do you think of these arguments?

It seems that the textbook question is an implicit argument against I.J. Good. Good may be treating intelligence as valuable, simply because man's strengths lies in that trait called "intelligence". But other traits could be equally valued instead (speed or strength) and sentient beings may speculate wildly about their preferred traits being "maximized" by some machine or another.

This makes me wonder whether a singularity could occur if we had built machines that were not maximizing intelligence, but instead maximizing some other trait (a machine that is always increasing its strength, or a machine that is always increasing its speed). These types of machines can be just as transformative - ultrafast machines may solve problems quickly due to "brute force", and ultrastrong machines can use its raw power for a variety of physical tasks. Perhaps a ultra-X machine can't build another ultra-X machine (as I.J. Good treated the design of machines as an intellectual activity), but a continually self-improving machine would still leave its creators far behind and force its creators to be dependent on it.

Are technological singularities limited to ultra-intelligences? Or technological singularities be caused by machines that are not "strong AI" but are still "ultra"-optimizers?

That would be a no for speed or strength, if you have a super strong entity but it cannot research new materials, it will be quickly limited, same thing for speed, Basically, you need something out of their field to improve them, which makes a runaway improvement impossible.

Though, we already have super strong and super fast machines, those are cars, trucks, hydraulic presses, industrial exoskeletons etc... But, even though we can build better ones through the use of the old ones, we still need to research stuff that can't be improved by old ones.

What we need for a singularity is a field where an improvement in it makes further improvement easier. And I don't know a field where this doesn't involve intelligence.
If there is one, that may be possible to have a non intelligence driven singularity there.


What is the “Hello World” problem of Unsupervised Learning? As a followup to this question, I'm interested in what the typical "Hello World" problem (first easy example problem) is for unsupervised learning.
A quick Google search didn't find any obvious answers for me.

I disagree with the context that MNIST is the "hello world" of supervised learning. It is definitely, though, the "hello world" of image classification, which is a very specific sub-field of supervised learning.
I'd consider the Iris dataset a better candidate for the "hello world" of supervised learning, with other close candidates such as the Wine, Wisconsin breast cancer or Pima indians datasets. However, as an even simpler and more alternative choice, a lot of people prefer generating their own 2-dimensional datasets so that can more intuitively understand what the different algorithms are doing. An example of this is TensorFlow playground.
Equivalently, in unsupervised learning there are a lot of different tasks. I personally think that clustering is probably the task that is easier for people to understand and as such the most common intro to unsupervised learning. Here there are, as well, two options:

Using an already established dataset, e.g. Iris (without the labels).
Generating your own synthetic 2-dimensional data, to better understand how the algorithms work. An example is this.


Tagging parts of speech when proper noun is a composite By means of parts of speech tagging, words of a given sentence can be assumed to be noun/verb etc, but if the sentence is for instance:

"My favourite book is harry potter and the prizoner of azkaban"

note that the inputs I receive would be from a chat interface so having a fixed format for the data can't be expected. Is there a way to identify "harry potter and the prizoner of azkaban" as a proper noun from such messages?

Currently this query tags as:

My|PRP$
favourite|JJ
book|NN
is|VBZ
harry|JJ
potter|NN
and|CC
the|DT
prizoner|NN
of|IN
azkaban|NN

I would like to know if this can be handled some way, or if there is another algorithm that can handle this?

I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:

My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).

now you can tokenize your text with identifying different NER tags, and join "B-" and "I-" prefixes with the same NER tag. e.g. you have "harry potter and ..." as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the "harry potter ..." as a single token, and it must tag it with "NN"

Another solution for this problem is called "chunking". It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive NNs as a single NP (harry potter): <NN>+ or a DT followed by a string of NNs (the brown fox): <DT>?<NN>+. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see nltk: information extraction)


Tensorflow: tf.contrib.DNNClassifier(..) or tf.estimator.DNNClassifier(..) Can someone explain the differance between tf.contrib.DNNClassifier (learn) and tf.estimator.DNNClassifier?

tf.contrib.DNNClassifier (learn) works but gives warnings:

  WARNING:tensorflow:From C:\Anaconda3\lib.. scalar_summary ...is
  deprecated and will be removed after 2016-11-30. Please switch to
  tf.summary.scalar.

But I can load the layers names and values with get_variable_names and get_variable_value.

tf.estimator works fine but how do I get the layers name and values?
See code below with a switch (if True/False) for the two versions

Instalation in Windows 10
3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)]
Tensorflow is installed in the root with:
pip install --upgrade tensorflow --ignore-installed (This is the only combination that works for me)
pip list gives
tensorflow (1.3.0)
tensorflow-tensorboard (0.1.4)

#Example of DNNClassifier for Iris plant dataset.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numpy as np
from sklearn import datasets
from sklearn import metrics
from sklearn import model_selection
import os
import tensorflow as tf
from tensorflow.contrib import learn
os.environ['TF_CPP_MIN_LOG_LEVEL']='3'  # get rid oc tf_jenkins WARNING

def main():
    # Load dataset.
    iris = datasets.load_iris()
    x_train, x_test, y_train, y_test = model_selection.train_test_split(
        iris.data, iris.target, test_size=0.2, random_state=42)

    # Define the training inputs and train
    get_train_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={'x':x_train}, y=y_train, num_epochs=None, shuffle=True)
    #Predict
    get_test_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={'x':x_test}, y=y_test, num_epochs=1, shuffle=True)

    # Build 3 layer DNN with 10, 20, 10 units respectively.
    feature_columns = [tf.feature_column.numeric_column('x', shape=np.array(x_train).shape[1:])]
    #Select code
    if True: # tf.estimator...
        classifier = tf.estimator.DNNClassifier(feature_columns=feature_columns,
            hidden_units=[10, 20, 10], n_classes=3)
        #Train
        classifier.train(input_fn=get_train_input_fn, steps=200)

        scores = classifier.evaluate(input_fn=get_test_input_fn)
        print('Accuracy (tf.estimator): {0:f}'.format(scores['accuracy']))
        # get_variable_names, get_variable_value. How ?????

    else: # learn (tf.contrib)
        classifier = learn.DNNClassifier(feature_columns=feature_columns,
            hidden_units=[10, 20, 10], n_classes=3)
        #Train
        classifier.fit(input_fn=get_train_input_fn, steps=200)

        scores = classifier.evaluate(input_fn=get_test_input_fn,steps=1)
        print("\nTest Accuracy (learn): {0:f}\n".format(scores["accuracy"]))

        # Get data
        names=classifier.get_variable_names()
        data={}
        for name in names:
            data[name]=classifier.get_variable_value(name)

if __name__ == "__main__":
    main()

Programming questions are a bit out of scope of this site, but it's a more generic one. For some reason the developers didn't specify it in code, but everything in contrib is experimental. From this discussion:

The distinction between core and contrib is really in core things don't change. Things are backward compatible until release 2.0, and nobody's thinking about that right now.
If you have something in core, it's stable, you should use it. If you have something in contrib, the API may change and depending on your needs you may or may not want to use it.

When a particular class or function is only in contrib, there's basically no choice. But for classes that have "graduated" to the core, contrib version gets deprecated automatically. So you want to use tf.estimator.DNNClassifier.


Why was Go a harder game for an AI to master than Chess? AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this feat required heavy machine learning.

My question is why was/is Go a harder game for AIs to master than Chess? I assume it has to do with Go's enormous branching factor; on a 13x13 board it is 169, while on a 19x19 board it is 361. Meanwhile, Chess typically has a branching factor of around 30.

The branching factor is important, as it limits the effectiveness of search.

However, the branching factor in chess is already too high to effectively search without techniques that reduce the size of the search space. Even with millions of tests per second, a computer can only check a small fraction of the possible future games in order to find results in its favour.

One key factor is heuristics - approximate measures of the value of each game state. A good heuristic can guide and improve search by orders of magnitude. There are some effective heuristics possible in chess, from weighted values of the pieces in play, scores for areas of the board that a position controls etc.

Heuristics for Go are much harder to find. Here's a sample paper from a few years ago that makes an attempt, there are several similar ones available online. Although plenty of options have been tried, and many were partially successful, none managed to bring the quality of computer play up to the standard of best human players.

One of the major achievements of AlphaGo was training a neural network that had good position evaluation - the "value network". The technology that made generating this approximate function of board game positions possible was deep learning, which has been developed very strongly since about 2010.

It is still possible a more analytical heuristic approach could be found that challenges deep learning models driven by self-play reinforcement learning on more raw board data. However, in some regards the reverse has been shown, with AlphaZero taking the same learning technique into chess and demonstrating its effectiveness against "old school" tuned expert heuristics.


What is the concept of training a neural network? I was trying to build an OCR system and heard about ANNs. I am weak at mathematics and statistics and couldn't stick up to reading those massive mathematical documents (research papers or ANN related books). But I kind of figured out that ANN training is all about balancing of weights and biases. Am I right? And please also point me to some docs where I can get help understanding ANNs to use in my OCR system.

Sorry, this is a very broad area. Proper understanding of neural networks requires advanced mathematics. It's not sufficient to say "balancing of weights and biases" because most ML algorithms have weights. You seriously need to grab a book.

OCR system itself is also very broad, it includes various object recognition techniques. You haven't even mentioned what you want to detect.

If you want to study, try:

  https://github.com/Elucidation/tensorflow_chessbot

This is a well-documented OCR example for chess pieces. The project uses both regression and convolutional neural network.


What is the current artificial general intelligence technology valuation? That is, if AGI were an existing technology, how much would it be valued to?
Obviously it would depend on its efficiency, if it requires more than all the existing hardware to run it, it would be impossible to market.
This question is more about getting a general picture of the economy surrounding this technology.
Assuming a specific definition of AGI and that we implemented that AGI, what is its potential economical value?
Current investments in this research field are also useful data.

I will try to give some sense to this question.

Artificial general intelligence (AGI) is the hypothetical[1] intelligence of a machine that has the capacity to understand or learn any intellectual task that a human being can. It is a primary goal of some artificial intelligence research and a common topic in science fiction and futures studies. AGI can also be referred to as strong AI,[2][3][4] full AI,[5] or general intelligent action.[6] Some academic sources reserve the term "strong AI" for machines that can experience consciousness.

These are the first sentences on AGI on wikipedia (link), and the softest limit there is

[learn] any intellectual task that a human being can.

Even taking only this, it would mean that any AGI has infinite economic value. As soon as there is something that can learn any human task and has the speed of current GPUs/CPUs it could potentially immediatly replace every human in every task. There are certainly enough computers with CPUs and GPUs out there.
This question is still a little flawed because you not only have to constrain the definition of AGI but also how it would actually be implemented.


is it possible to train several Neural Networks on different types of data and combine them? I want to create a NHL game predictor and have already trained one neural network on game data.

What I would like to do is train another model on player seasonal/game data and combine the two models to archive better accuracy.

Is this approach feasible? If it is, how do I go about doing it?

EDIT:

I have currently trained a neural network to classify the probability of the home team winning a game on a dataset that looks like this:

h_Won/Lost  h_metric2 h_metric3 h_metric4 a_metric2 a_metric3 a_metric4 h_team1 h_team2 h_team3 h_team4 a_team1 a_team2 a_team3 a_team4
 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0
 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0
 1            10       10         10        10         10        10      1       0        0      0         0      1        0      0

and so on.

I am preparing a dataset of player-data for each game that will have the shape of this:

Player     PlayerID    Won/Lost     team      opponent     metric1     metric2
 Henke         1           1          NY          CAP         10          10

Hopefully, this new dataset will have some accuracy on if team is going to have some predictive features that are good and recognised.

Now, say I have these two trained Nural Networks and they both have an accuracy of 70% by them self. But I want to combine them both in the hopes to achieve better predictability. How is this archived? How will the test-dataset be structured?

The term you need  is “model ensembles”, that’s the way models are combined.  Pretty hard to be more specific since you don’t give a language or any other details.


Corner detection algorithm gives very high value for slanted edges? I have tried implementing a basic version of shi-tomasi corner detection algorithm. The algorithm works fine for corners but I came across a strange issue that the algorithm also gives high values for slanted(titled) edges.

Here's what i did

Took gray scale image
computer dx, and dy of the image by convolving it with sobel_x and sobel_y
Took a 3 size window and moved it across the image to compute the sum of the elements in the window.
computed sum of the window elements from the dy image and sum of window elements from the dx image and saved it in sum_xx and sum_yy.
created a new image (call it result) where that pixel for which the window sum was computed was replaced with min(sum_xx, sum_yy) as shi-tomasi algorithm requires.

I expected it to give maximum value for corners where dx and dy both are high, but i found it giving high values even for titled edges.

Here are the some outputs of the image i received:

Result:

so far so good, corners have high values.

Another Image:

Result:

Here's where the problem lies. edges have high values which is not expected by the algorithm. I can't fathom how can edges have high values for both x and y gradients (sobel being close approximation of gradient).

I would like to ask your help, if you can help me fix this issue for edges. I am open to any suggestions and ideas .

Here's my code (if it helps):

def shi_tomasi(image, w_size):
    ans = image.copy()
    dy, dx = sy(image), sx(image)

    ofset = int(w_size/2)
    for y in range(ofset, image.shape[0]-ofset):
        for x in range(ofset, image.shape[1]-ofset):

            s_y = y - ofset
            e_y = y + ofset + 1

            s_x = x - ofset
            e_x = x + ofset + 1

            w_Ixx = dx[s_y: e_y, s_x: e_x]
            w_Iyy = dy[s_y: e_y, s_x: e_x]

            sum_xx = w_Ixx.sum()
            sum_yy = w_Iyy.sum()

            ans[y][x] = min(sum_xx, sum_yy)
    return ans

def sy(img):
    t = cv2.Sobel(img,cv2.CV_8U,0,1,ksize=3)
    return t
def sx(img):
    t = cv2.Sobel(img,cv2.CV_8U,1,0,ksize=3)
    return t

I can't go into details of the algorithm but here's some intuition about what's apparently going wrong:

The Sobel transformation identifies mostly-vertical and mostly-horizontal edges. For slanted edges, it also shows a response, just a bit weaker.

By using a window and taking the minimum of vertical and horizontal response, you identify points where you have both a vertical and a horizontal response. However, this is also the case at the slanted edges.

Given the structure of this simplistic corner detection algorithm, you can only expect to get meaningful results for strictly horizontal and vertical lines/edges and their intersections/corners. It is just not suitable for slanted edges.


Is there a way to train an RL agent without any environment? Following Deep Q-learning from Demonstrations, I'd like to avoid potentially unsafe behavior during early learning by making use of supervised learning with demonstration data. However, the implementation I'm following still uses an environment. Can I train my agent without an environment at all?

There are many techniques for training an RL agent without explicitly interacting with an environment, some of which are cited in the paper you linked. Heck, even using experience replay like in the foundational DQN paper is a way of doing this. However, while many models utilize some sort of pre-training for the sake of safety or speed, there are a couple of reasons why an environment is also used whenever possible.

Eventually, your RL agent will be placed in an environment to take its own actions. This is why we train RL agents. I'm assuming that, per your question, learning does not happen during this phase.

Maybe your agent encounters a novel situation
Hopefully, the experience your agent learns from is extensive enough to include every possible state-action pair $(s,a)$ that your agent will ever encounter. If it isn't, your agent won't have learned about these situations, and it will always perform suboptimally in them. This lack of coverage over the state-action space could be caused by stochasticity or nonstationarity in the environment.

Maybe the teacher isn't perfect
If you don't allow your agent to learn from its own experience, it will only ever perform as well as the agent that collected the demonstration data. That's an upper bound on performance that we have no reason to set for ourselves.


How does transformer leverage GPU which trains faster than RNN? How does transformer leverage GPU which trains faster than RNN? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multi-gpu and accelerates its training?

The issue with Recurrent models is that they don't parallelization during training.
Sequential models performs better with more memory but faces problem in learning long-term memory dependencies.

On the other hand Transformers take into account of self attention which boosts the speed of how fast the model can translate from one sequence to another and establishes dependencies b/w input and output and focus on relevant parts of the input sequence, which in turn eliminates recurrence and convolution unlike RNNs where sequential computation inhibits parallelization.


Java - A good place to begin if over all goal is ML and Ai? I am currently studying Java (Se && EE). I am wondering if it is a good platform for developing ML algorithms for AI.
Areas of interest: facial rec - Speech Rec - understanding conversation in group conversations.
Financial Institutions: Risk assessment ML, etc.

If goal is to develop ML algorithms then focus on Maths concepts linear algebra, probability and statistics. Try out CS problem solving basic data structure and algorithms.
Python has good ML libraries but if you know java then you can pick python easily.


Can neural networks evolve other neural networks? Can neural networks change or evolve other neural networks? Also, could evolutionary algorithms be applied to evolve neural networks?

For example, suppose that we have neural networks A and B. The neural network B changes the neural network A. If B "successfully" changed it, NN A will survive.

Yes, this is an active area of research as we speak. Both using classic algorithms (decision trees, random forests, Bayesian ensembles) as well as neural networks. This can also be done via evolutionary algorithms. I have personally used them for hyperparameter tuning in a few cases where squeezing out a couple of extra points of accuracy was key.

This is in fact what Google is doing with their AutoML system. They are using neural networks for architecture search.

Here is a Github repo with some interesting papers and links on the topic you are describing: https://github.com/hibayesian/awesome-automl-papers.


How to classify language as friendly or aggressive with AI? Just for the purpose of learning I'd like to classify the likeliness of a tweet being in aggressive language or not.

I was wondering how to approach the problem. I guess I need first train my neural network on a huge dataset of text what aggressive language is. This brings up the question where I would get this data in the first place?

It feels a bit like the chicken and egg problem to me so I wonder how would I approach the problem?

I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.

Second part is much easier at start but would be pain to optimize (both performance and computational cost). I would suggest you to start with Naive Bayes classifier for this job. That is the preferred classifier for spam detection. ANN would probably not work for this case because the data would be a huge sparse vector. Estimated number of words in English is over a million, which means the input layer of your ANN should be able to scale up to that number. Search for sparse vector classification for additional classifier that can be used in these cases.


Is true random number generation an AI concept? As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required.

On the other hand, humans can easily generate any random number independently of other factors.

Does this suggest that absolute random number generation an AI concept?

As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required.

This is true. In fact, it is impossible to solve using software. No software-only technique can generate randomness without an initial random seed or support from hardware.

This is also true for AI software. No AI design that uses deterministic software can do this - e.g. any Turing machine without a magic unexplained "random" function can be shown to remain deterministic, no matter how complex. That's because any combination of deterministic functions is deterministic. It may not be predictable without following the process exactly, it may be "chaotic" and depend critically on initial conditions, but it is 100% deterministic and repeatable.

  On the other hand, humans can easily generate any random number independently of other factors.

Typically not high quality randomness. It is not clear how we make random decisions, but entirely possible that internally we rely effectively on noisiness from the environment or our own internal "hardware" doing something simple such as racing decisions between neurons (timing will vary due to speed of electrical impulses, diffusion time of neuro-transmitters across synapses etc)

  Does this suggest that absolute random number generation an AI cocept?

I think it is orthogonal issue. We can already produce very high quality artificial randomness - better than human quality used by humans for conscious decisions (such as "choose a random number between 1 and 10"). These artificial random number generating systems are part of modern cryptography and are tested thoroughly.

Essentially "true" artificial randomness is a solved problem using hardware, and does not involve anything that has traditionally been called AI.

In reverse, AI systems often rely on stochastic functions in order to break symmetry, break ties, regularise models etc. So it does look like some kind of RNG is necessary within an artificial agent. However, even  pseudo-random number generators (PRNGs) seem to be fine for this purpose. Mersenne Twister is a very common choice for generating random numbers inside neural networks for weight initialisation, dataset shuffling, dropout regularisation, or when simulating environments for RL, or taking exploratory actions. Despite the fact that it is not "true" random, a PRNG will work just fine for these purposes.

Working definition of "true" randomness that I use: Even if you know the state of a system as accurately as modern physics allows, the output cannot be predicted with better accuracy than a fixed guess.

Human randomness already fails this test. If you ask someone to choose a number randomly between 1 and 9, you will generally have a better than 1 in 9 chance of guessing the correct value, based on statistical analysis. If we were able to take good state measurements of brains, it might be possible to predict with high accuracy - although this is unknown and not possible with current technology.


What is the most effective way to learn natural language processing online? There are many books, courses, etc. out there, but not sure which path to take.

So what would be the most effective way (shortest) to learn natural language processing online?

p.s. I mean learning fundamentals, not how to use existing libraries or services.

I think you could learn NLP ,by taking this course https://see.stanford.edu/Course/CS224N
Natural Language Processing - Stanford University | Coursera and try your best with the homework.

To further study, you could learn some deep learning algorithms like RNN. And of course, it’s better that you will be able to learn more a lot from university professors with minimal time period.

Hope it can help you.


What regulations are already in place regarding artificial general intelligences? What regulations are already in place regarding artificial general intelligences? What reports or recommendations prepared by official government authorities were already published?

So far, I know of Sir David King's report done for UK government.

I don't know that it has yielded any actual reports or regulations yet, but in the USA, the White House has been running a series of interagency workshops / working groups dedicated to Preparing for the Future of Artificial Intelligence.

Some of those sessions have been dedicated to legal / governance issues.


How long should the state-dependent baseline for policy gradient methods be trained at each iteration? How long should the state-dependent baseline be trained at each iteration? Or what baseline loss should we target at each iteration for use with policy gradient methods?

I'm using this equation to compute the policy gradient:

$$
\nabla_{\theta} J\left(\pi_{\theta}\right)=\underset{\tau \sim \pi_{\theta}}{\mathrm{E}}\left[\sum_{t=0}^{T} \nabla_{\theta} \log \pi_{\theta}\left(a_{t} | s_{t}\right)\left(\sum_{t^{\prime}=t}^{T} R\left(s_{t^{\prime}}, a_{t^{\prime}}, s_{t^{\prime}+1}\right)-b\left(s_{t}\right)\right)\right]
$$

Here is mentioned to use one or more gradient steps, so is it a hyper-parameter to be found using random search?

Is there some way we can use an adaptive method to find out when to stop?

In an experiment to train Cartpole-v2 using a policy gradient with baseline, I found the results are better when applying 5 updates than when only a single update was applied.

Note: I am referring to the number of updates to take on a single batch of q values encountered across trajectories collected using current policy.

I went into the pytorch code for the spinning up implementation of vanilla policy gradient and from what I could understand, found that they use a learning rate of 1e-3 for training the baseline and do a gradient descent 80 times on the same dataset by default with no termination criteria.

Also it is usually impossible to fit the value function completely as we are using a function approximator. The main point is not to get too carried away to get the lowest loss as the main improvement in the agent's performance will happen by doing policy gradient descents rather than trying to aggressively get the accurate value function for the baseline.

Link for the implementation:
spinning up implementation of vpg


Snake game: snake converges to going in the same direction every time This is a q-learning snake using a neural network as a q function aproximator and I'm losing my mind here the current model it's worst than the initial one.

The current model uses a 32x32x32 MLPRegressor from scikit-learn using relu as activation function and the adam solver.

The reward function is like following:

death reward = -100.0
alive reward = -10.0
apple reward = 100.0

The features extracted from each state are the following:

what is in front of the snake's head(apple, empty, snake)
what is in the left of the snake's head
what is in the right of the snake's head
euclidian distance between head and apple
the direction from head to the apple measured in radians
length of the snake

One episode consists of the snake playing until it dies, I'm also using in training a probability epsilon that represent the probability that the snake will take a random action if this isn't satisfied the snake will take the action for which the neural network gives the biggest score, this epsilon probability gradually decrements after each iteration.

The episode is learned by the regressor in reverse order one statet-action at a time.

However the neural network fails too aproximate the q function, no matter how many iterations the snake takes the same action for any state.

Things I tried:

changing the structure of the neural network
changing the reward function
changing the features extracted, I even tried passing the whole map to the network

Code (python): https://pastebin.com/57qLbjQZ

There are two problems here.

The code you posted doesn't incrimentally train your multilayer perceptron. Instead, it effectively re-randomizes the weights, and then re-fits the model each time you call .fit() at lines 35 & 54. Using SKLearn's _fit() function with Incremental=true might solve this, or you can package up the data into a larger batch, and train on that offline instead after several episodes.
Your reward function makes it painful to be alive, and doesn't give enough benefits through the Apples to make up for this. There are 100 squares that could contain the apple. On average, the apple will spawn about 5 squares from the snake in each direction. Since the snake can't move diagonally, that's 10 moves (5 left/right, 5 up/down). That means that if the snake plays perfectly, then on average, it might be able to get zero reward total. In practice, the snake will not play perfectly. This means living gives negative expected reward.

In contrast, if the snake can kill itself, it will stop getting negative rewards. The reward function you've used is maximized by getting big enough to run into your own tail as fast as possible. The snake should be able to do this after eating 3 apples I think. There is some incentive to hunt for food well, but not much compared with hitting your own tail as soon as possible.

If you want the snake to learn to hunt for the food, reduce the penalty for being alive to -1, or even -0.1. The snake will be much more responsive to signals from the food.


Would YOLO be able to detect objects in "different" positions? I have the following question about You Only Look Once (YOLO) algorithm, for object detection.
I have to develop a neural network to recognize web components in web applications - for example, login forms, text boxes, and so on. In this context, I have to consider that the position of the objects on the page may vary, for example, when you scroll up or down.
The question is, would YOLO be able to detect objects in "different" positions? Would the changes affect the recognition precision? In other words, how to achieve translation invariance? Also, what about partial occlusions?
My guess is that it depends on the relevance of the examples in the dataset: if enough translated / partially occluded examples are present, it should work fine.
If possible, I would appreciate papers or references on this matter.
(PS: if anyone knows about a labeled dataset for this task, I would really be grateful if you let me know.)

As you said, a CNN would be able to detect objects in different positions if the dataset contains enough examples of such cases, though the network is able to generalize and should be able to detect objects in slightly changed positions and orientations.
The term "translation invariance" does not mean that translating an object in the image would yield the same output for this object, but that translating the whole image would yield the same result. So the relative position of object IS important, modern CNN's takes decisions on the whole image (with strong local cues, of course).
To maximize the ability of your CNN to detect multiple orientation, you can train with data augmentation that rotate the images.
the same reasoning can be applied to partial occlusions: if there are enough samples with occlusion in the training set the network should be able to detect those ones. The network ability to generalize should also help a little when occlusions are small, and still be able to detect the object.
Some papers tried different experiment to demonstrate the robustness to occlusion and translation, for instance by looking at the network activation when artificially occluding a portion of the image with a gray rectangle, though I do not have a paper name in mind.


Why does a neuron in a multi-layer network need several input connections? For example, if I have the following architecture:

Each neuron in the hidden layer has a connection from each one in the
input layer.
3 x 1 Input Matrix and a 4 x 3 weight matrix (for the backpropagation we have of course the transformed version 3 x 4)

But until now, I still don't understand what the point is that a neuron has 3 inputs (in the hidden layer of the example). It would work the same way, if I would only adjust one weight of the 3 connections.
But in the current case the information flows only distributed over several "channels", but what is the point?
With backpropagation, in some cases the weights are simply adjusted proportionally based on the error.
Or is it just done that way, because then you can better mathematically implement everything (with matrix multiplication and so on)?
Either my question is stupid or I have an error in my thinking and assume wrong ideas. Can someone please help me with the interpretation.
In tensorflow playground for example, I cut the connections (by setting the weight to 0), it just compansated it by changing the other still existing connection a bit more:

There's a few reasons I can think of, though I have not read an explicit description of why it is done this way. It's likely that people just started doing it this way because it's most logical, and people who have attempted to try your method of having reduced connections have seen a performance hit and so no change was made.
The first reason is that if you allow all nodes from one layer to connect to all others in the next, the network will optimise unnecessary connections out. Essentially, the weighting of these connections will become 0. This, however, does not mean you can trim these connections, as ignoring them in this local minima might be optimal, but later it might be really important these connections remain. As such, you can never truly know if a connection between one layer and the next is necessary, so it's just better to leave it in case it helps improve network performance.
The second reason is it's just simpler mathematically. Networks are implemented specifically so it's very easy to apply a series of matrix calculations to perform all computations. Trimming connections means either:

A matrix must contain 0 values, wasting computation time
A custom script must be written to calculate this networks structure, which in the real world can take a very long time as it must be implemented using something like CUDA (on a GPU level, making it very complicated)

Overall, it's just a lot simpler to have all nodes connected between layers, rather than on connection per node.


block of worlds with position aware goal stack planning Can someone suggest an AI approach to moving blocks, one at a time, assuming control of an robotic arm, to get from the initial state on the left to the final state on right, preferably using goal stack planning.

actions

Pickup() — to pick up a block from table only
Putdown — to putdown a block on table only
Unstack — unstack a block from another block
Stack — stack a block on another clear block only

property functions

On(x,y)
Above(x,y)
Table(x)
Clear(x)

Your best bet would be to formulate the problem in PDDL, which should be fairly easy, and then use a standard planner to generate a plan from that description.
In PDDL you describe the properties and the possible actions, the start state and the goal state, and the planner will then take this to produce a sequence of actions that leads from the start state to the goal state. There is a planner available on-line that you can use.


Why does CNN forward pass take longer compared to MLP forward pass? Let's take a 32 x 32 x 3 NumPy array and convolve with 10 filters of size 2 x 2 x 3 with stride 2 to produce feature maps of volume 16 x 16 x 10. The total number of operations - 16 * 16 * 10 * 2 * 2 * 2 * 3 = 61440 operations. Now, let's take an input array of length 3072 (flattening the 32 * 32 * 3 array) and dot it with a weight matrix of size 500 x 3072. The total number of operations - 500 * 3072 * 2 = 3072000 operations. The convolution takes 4-5 times longer than np.dot(w, x) even though number of operations is less.
Here's my code for the convolution operation:
for i in range(16):
    for j in range(16):
        for k in range(10):
            v[i, j, k] = np.sum(x[2 * i:2 * i + 2, 2 * j:2 * j + 2] * kernels[k])

Is np.dot(w, x) optimized or something? Or are my calculations wrong? Sorry if this is a silly question...

From the NumPy Linear algebra documentation:

The NumPy linear algebra functions rely on BLAS and LAPACK to provide
efficient low level implementations of standard linear algebra
algorithms. Those libraries may be provided by NumPy itself using C
versions of a subset of their reference implementations but, when
possible, highly optimized libraries that take advantage of
specialized processor functionality are preferred.

In your case, np.dot() is a matrix-vector multiplication which internally calls  such a highly optimized BLAS routine. Those libraries are implemented in low-level languages like Fortan or C, they exist for many years now and are still unbeaten in terms of speed.
Your naive implementation of the convolution operation in python on the other hand is not optimized for execution speed and will be executed by the python interpreter (which is much slower than the compiled C functions).
You could try to replace your manual convolution by the SciPy's equivalent scipy.signal.convolve2d(), to make advantage of optimized libraries as well and get a speed up.


Are there relatively new research papers that describe how to make back-propagation more efficient? I read Yann LeCun's paper Efficient BackProp, which was published in 2000. I looked for similar but more recent papers on Arxiv, but I have not yet found any.
Are there relatively new research papers that describe how to make back-propagation more efficient?
So, I am looking for papers similar to Efficient Backprop by LeCun but newer. The papers could describe why ReLU now "dominates" tanh or even sigmoid (but tanh was Yann's favorite, as explained in the paper). ReLU is just one thing I am interested in, but the paper could also analyze e.g. the inputs from a statistical standpoint.

Here is a paper that explains why ReLU rules.
What we want is to disentangle data of different classes. In order to do that, we need a discontinuous mapping for the data. ReLU easily allows for that. It is even better than LeakyReLU, sigmoid and tanh in that regard. Also, the reason any of the activations work is because of the floating point error, there is inadvertently a discontinuous mapping for the whole data. I have also explained it here.


My DQN is stuck and can't see where the problem is I'm trying to replicate the DeepMind paper results, so I implemented my own DQN. I left it training for more than 4 million frames (more than 2000 episodes) on SpaceInvaders-v4 (OpenAI-Gym) and it couldn't finish a full episode. I tried two different learning rates (0.0001 and 0.00125) and seems to work better with 0.0001, but the median score never raises above 200.
I'm using a double DQN.
Here is my code and some photos of the graphs I'm getting each session.
Between sessions I'm saving the network weights; I'm updating the target network every 1000 steps.
I can't see if I'm doing something wrong, so any help would be appreciated. I'm using the same CNN construction as the DQN paper.

Here's the action selection function; it uses a batch  of 4  80x80 processed experiences in grayscale to select the action (s_batch means for state batch):

    def action_selection(self, s_batch):
        action_values = self.parallel_model.predict(s_batch)
        best_action = np.argmax(action_values)
        best_action_value = action_values[0, best_action]
        random_value = np.random.random()

        if random_value < AI.epsilon:
            best_action = np.random.randint(0, AI.action_size)
        return best_action, best_action_value

Here is my training function. It uses the past experiences as training; I tried to implement that if it lose any life, it wouldn't get any extra rewards, so in theory, the agent would try to not die:

    def training(self, replay_s_batch, replay_ns_batch):
        Q_values = []
        batch_size = len(AI.replay_s_batch)
        Q_values = np.zeros((batch_size, AI.action_size))

        for m in range(batch_size):

            Q_values[m] = self.parallel_model.predict(AI.replay_s_batch[m].reshape(AI.batch_shape))
            new_Q = self.parallel_target_model.predict(AI.replay_ns_batch[m].reshape(AI.batch_shape))
            Q_values[m, [item[0] for item in AI.replay_a_batch][m]] = AI.replay_r_batch[m]

            if np.all(AI.replay_d_batch[m] == True):
                Q_values[m, [item[0] for item in AI.replay_a_batch][m]] = AI.gamma * np.max(new_Q)

        if lives == 0:
            loss = self.parallel_model.fit(np.asarray(AI.replay_s_batch).reshape(batch_size,80,80,4), Q_values, batch_size=batch_size, verbose=0)

        if AI.epsilon > AI.final_epsilon:
            AI.epsilon -= (AI.initial_epsilon-AI.final_epsilon)/AI.epsilon_decay

replay_s_batch it's a batch of (batch_size) experience replay states (packs of 4 experiences), and replay_ns_batch it's full of 4 next states. The batch size is 32.

And here are some results, after training:

In blue, the loss (I think it's correct; it's near-zero). Red dots are the different match scores (as you can see, it does sometimes really good matches). In green, the median (near 190 in this training, with learning rate = 0.0001)

Here is the last training, with lr = 0.00125; the results are worse (it's median it's about 160). Anyway the line it's almost straight, I don't see any variation in any case.
So anyone can point me to the right direction? I tried a similar approach with pendulum and it worked properly. I know that with Atari games it takes more time but a week or so I think it's enough, and it seems to be stuck.
In case someone need to see another part of my code just tell me.

Edit: With the suggestions provided, I modified the action_selection function. Here it is:

def action_selection(self, s_batch):
    if np.random.rand() < AI.epsilon:
        best_action = env.action_space.sample()
    else:
        action_values = self.parallel_model.predict(s_batch)
        best_action = np.argmax(action_values[0])
    return best_action

To clarify my last edit: with action_values you get the q values; with best_action you get the action which corresponds to the max q value. Should I return that or just the max q value?

After some research and reading this post, I see where my problem was: I was introducing a full consecutive batch of experiences, selected randomly, yes, but the experiences in the batch were consecutives.
After redoing my experience selection method, my DQN is actually working and has reached about +200 points after 400000 experiences (about 500 episodes; only 2-3 hours or training!). Before I couldn't reach that score after days of training.
I'll let it train to see if there are something I can improve. Thanks to everyone who tried to help me! I let this answer here just in case someone has the same problem as me.


Why doesn't my image classification network get better with training? I am attempting to train a network to do something I thought would be a relatively simple case to learn with: identify whether the back of a scanned vintage postcard has one of 'no postage stamp', a '1 cent stamp', or a '2 cent stamp.' The images are 250px by about 150px, RGB color, and there are about two thousand of them. Ballpark 75% of them are no-stamp, 20% 1-cent, and 10% 2-cent.

When I attempt to train the network it seems like it is starting at 70 +/- 1 % accurate and hovers in that range for 50 epochs, never improving. I'm not sure I'm reading the metrics correctly, though, as this doesn't seem quite right.

I set this up by following the tutorial on the Keras blog: https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

I haven't implemented the latter part of the tutorial, where a pre-trained network is used, because I haven't found one that seems like it would be a similar problem.

My training and validation sets are here: https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3

And my Google Colab Jupyter notebook is here: https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD

The answer (as detailed quite thoroughly here and here) is that specifying

model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])

causes keras to guess, incorrectly, that because I am using binary_crossentropy for the loss function, that I would want to use binary_accuracy as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the categorical_accuracy metrics if one, as I do, has more than two classes.


Are there any references of NLP/text mining techniques for identifying the theme of news headlines? I am looking to extract the central theme from a given news headline using NLP or text mining. Is there any reference that goes in this direction?
Here's an example. Let's say that I have the following news headline.

BRIEF-Dynasil Corporation Of America Reports Q2 EPS Of $0.08

Then the algorithm should produce

Reports

Here's another example. The input is

China's night-owl retail investors leverage up to dominate oil futures trade

And the output would e.g. be

oil futures

You could formulate the problem as a topic classification task, hence you need labeled data.
From an unsupervised point of view, you could represent sentences with some fixed feature vector (latent representation).

Generating Sentences from a Continuous Space.
Paragraph2Vec.

BRIEF-Dynasil Corporation Of America Reports Q2 EPS Of $0.08
China's night-owl retail investors leverage up to dominate oil futures trade

Self-attention models would be very useful to this kind of problems since you don't need to encode all the context in the last hidden cell of some RNN model to classify to which theme the headline belongs.


How to generate a dataset for question answering from books (like Facebook's bAbI)? I wanted to train a chatbot for answering questions from books. I am trying to use Dynamic Memory Networks to do so.
How can I generate a data set, as Facebook did in the case of bAbI tasks, so that it can tackle a variety of questions on the data set?

If you are talking about "generating" in the sense of generative models , it is pretty tough. since we are still far beyond understanding the actual structure of question-answering.

And even state of the art methods for question answering are also not able to score well on datasets like babi , mostly 16 out of 20 tasks can be solved.


What is the intuition behind importance sampling for off-policy value evaluation? The technique for off-policy value evaluation comes from importance sampling, which states that

$$E_{x \sim q}[f(x)] \approx \frac{1}{n}\sum_{i=1}^n f(x_i)\frac{q(x_i)}{p(x_i)},$$ where $x_i$ is sampled from $p$.

In the application of importance sampling to RL, is the expectation of the function $f$ equivalent to the value of the trajectories, which is represented by the trajectories $x$?

The distributions $p$ represent the probability of sampling trajectories from the behavior policy and the distribution $q$ represents the probability of sampling trajectories from the target policy $q$?

How would the trajectories from distribution $q$ be better than that of $p$? I know from the equation how it is better, but it is hard to understand intuitively why this could be so.

Recall that our goal is to be able to accurately estimate the true value of each state by computing a sample average over returns starting from that state:
$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} Return_i $$
where $Return_i$ is the return obtained from the $i^{th}$ trajectory.

The problem is that the $\approx $ does not hold, since in off-policy learning, we got those returns by following the behavior policy, $p$, and not the target policy, $q$.

To address that, we have to correct each return in the sample average by multiplying by the importance sampling ratio.

$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} \rho_i Return_i$$

where the importance sampling ratio is : $\rho=\frac{\mathbb{P}(\text { trajectory under } q)}{\mathbb{P}(\text { trajectory under } p)}$

What this multiplication does is that it increases the importance of returns that were more likely to be seen under the target policy $q$ and it decreases those that were less likely. So, at the end, in expectation, it would be as if the returns were averaged following $q$.

(A side note: To avoid the risks of mixing $p$ and $q$, it might be a good idea to denote/think of the behavior policy as $b$ and the target policy as $\pi$, following the convention in Sutton and Barto's RL book.)


Can neural network take decision about its own weights (update of weights)? Can neural network take decision about its own weights (update of weights) during training phase or during the phase of parallel training and inference? When one region of hierarchical NN takes decision about weights of other region is the special case of my question.

I am very keen to understand about self-awareness, self-learning, self-improvement capabilities of neural networks, because those exactly those self-* capabilities are the key path to the artificial general intelligence (e.g. Goedel machine). Neural networks are usually mentioned as examples of special, single-purpose intelligence but I can not see the reason for such limitation if NN essentially trys to mimic human brains, at least in purpose if not in mechanics.

Well - maybe this desired effect is already effectively achieved/emerges in the operation of recurrent ANNs as the effect of collective behavior?

Establishing Names and Terms
The mathematician's name is Kurt Gödel, an Austrian-American, and the machine concept named after him is the Gödel machine.  The idea of meta-learning (learning how to learn) can be extended recursively to learning how to learn how to learn how to learn and so on.
Taking a decision is when the decision is external to the system and the system either copes or executes that decision, whereas making a decision is when the decision is internal to the system and the system executes it, reports it, or delegates it.
Both of these have been tried in the laboratory or in real systems and various levels of recursion have achieved success, but not indefinite recursion.
A Few Points from AI History
Creating a learn how to learn recursion was the object of several early LISP efforts at MIT, and meta-learning is in active research in multiple universities and corporations as of this writing.  The first successful application of a single level of recursion were the expert systems that could acquire knowledge using meta-rules, which were rules about creating new rules.
Current Artificial Networks Regarding Meta-learning
It is correct that back propagation is not meta-learning.  It is a single level of learning in the above recursive learning concept.  Artificial networks have been applied to learn to adjust the hyper-parameters of another network, but these strategies require large quantities of data, and they don't cognate.
They don't know how to adjust hyper-parameters.  They don't build models of learning and then use those models to learn.  The adjustment mechanism is the application of a learned function, not a conception of learning.
Requirements for Meta-learning
To establish a meta-learning paradigm, we must consider the elements of learning.

Existence of function abstraction
Ability to guess a parameterized function
Ability to mutate that function
Ability to tune the parameters
A target functional behavior
A way to detect whether the function is approaching the target behavior
A strategy for deciding when to mutate and when to tune
A way of performing experiments in isolation
A way to recurse in a way that incorporates ALL of the above

Once this has been achieved, it may be described as self-improvement and may qualify as what most theorists think of as a Gödel machine.
Specifics in the Question

Can [an artificial] network [make] decision about its own weights (update of weights) during training phase or during the phase of parallel training and inference?

Yes.

[Can] one [level] of [a] hierarchical [artificial network] [make] decision[s] about weights of other [levels]?

Yes, but in a limited way as of the time of this writing.

I am very keen to understand about self-awareness ...

Self-awareness is the ability of the system to analyze and either utilize the result of that analysis, report it, or delegate based on the result IN COMBINATION WITH the ability to use itself as the object of analysis.
This can be as simple as a program that parses Java and produces statistics using its own code as the code to parse.  It can be as complex as an anthropologist studying humanity.  It can be as personal as a human looking in the mirror and wondering about the qualities of person they are.  It can be as deep as someone wondering their purpose in their current life.
In all cases, there are those two qualities.

The ability to perform a type of analysis that could be applied toward a class of objects of which itself is a member
The selection of itself for analysis

There is one other aspect of awareness beyond these two that are not required but usually associated with awareness: Some way of directing attention of the analysis capability to itself regularly.  Here are a few examples from human experience.

Practice of daily meditation
Reading a book on purpose
Seeing a cognitive behavioral therapist
Writing an autobiography
The first, fourth, and tenth Step of anonymous programs
Watching a candid movie of one's self
A family member that likes to step back and evaluate the family
Religious accountability groups

Self learning is like self-awareness except that learning replaces awareness.  Obviously, self-learning is dependent on some degree of self-awareness.  We now have a more activity based list because of the additional element of adjustment or inception of action.

Decide, on the basis of personal abilities and interests, after evaluating options for living, to relocate
Realizing that self cannot remember commitments to meetings to buy a technology device that notifies the user before such commitments and using the calendar app rigorously
Finding that repeated attempts to eat greens rather than junk food is not working and joining a community of those with eating disorders with solutions to that issue
Deciding that childhood views of future were good ones and abandoning current paths for ones that were bolder and will likely lead to fulfillment

Self-improvement is a super-set of self-learning, since the only point of learning is improvement, however there are forms of improvement that don't strictly involve learning.  A system (or person) can execute something already learned that results in an improvement.

Artificial general intelligence (e.g. Gödel machine)

There is no mathematical proof that a Gödel machine would exhibit general artificial intelligence.  More importantly, there is no proof and a considerably body of evidence against the proposition that humans are generally intelligent.
Lastly, there is no proof that general intelligence is achievable in a biological or artificial system.  Gödel's second incompleteness theorem is strong evidence that such ideas may be naive.

[Artificial] networks are usually mentioned as examples of special, single-purpose intelligence, but I can not see the reason for such limitation if [they] essentially [try] to mimic human brains, at least in purpose if not in mechanics.

Artificial networks emerged out of a desire to mimic human brains, but the perceptron design is based on an old view of neuron functionality.  Simulating a single neuron might require an entire rack of CPUs and GPUs.
Furthermore, we are not yet clear on what intelligence is.  Definitions proposed vary widely.  The fields of artificial networks, bioinformatics, ontology of ideas, semantics, and the relationships between stable adaptivity and algorithms are all in their early stages, both theoretically and experimentally.

Maybe this desired [capability] is already effectively achieved [or currently emerging] in the operation of recurrent ANNs as the effect of collective behavior?

No, but RNNs are a tiny step closer to biological neurons in that their layers maintain state beyond the attenuation matrix.
By attenuation matrix is meant the matrix of parameters used in vector-matrix multiplication to control the strength of signals from the activation functions of one layer to the activation functions of the next.  It's common name in machine learning literature is simply, "The parameters."  Learning occurs as they change to converge the network on an optimal state.
RNNs are also capable of being Turing complete, so they can theoretically realize arbitrary algorithms.
However, the Requirements for Meta-learning above are not an expected capability of any single instance of an artificial networks, either of the MLP (multilayer perceptron) type or the RNN type.  Whether balancing two networks in symbiotic arrangement as in GANs, or a comprehensive recursive algorithm, or the simulation of neural and synaptic plasticity in silicon will lead to Gödel machines is unknown.
Let Science be Science
Whether ideas such as general intelligence are possible is unknown.  Whether ideas about singularities in books and mass media are realistic is unknown.  Whether the prophetic warnings of screenwriters about the emergence of artificial entities that become the dominant species on earth is unknown.
Jaques Ellul is perhaps the most scientifically accurate prophetic voice.  In his Techological Sociey, he presents heaps evidence in support of the idea that humans are already serving an autonomous technology and have been since prior to industrialization.
A Swiss philosopher, Francis Schaeffer, once prophecied, "They will say all kinds of things in the name of science that have nothing to do with science."  We should be careful to keep conjecture from reaching the status of scientific fact in technical conversation.  If we have no carefully drawn theory or empirical evidence to support a conjecture, it should be stated as a proposal, not a conclusion.


Why not more TD() in actor-critic algorithms? Is there either an empirical or theoretical reason that actor-critic algorithms with eligibility traces have not been more fully explored? I was hoping to find a paper or implementation or both for continuous tasks (not episodic) in continuous state-action spaces.

This has been the only related question on SE-AI that I have been able to find Why are lambda returns so rarely used in policy gradients?.

Although I appreciated the dialog and found it useful, I was wondering if there was any further detail or reasoning that might help explain the void.

Theoretically, nothing precludes the use of $\lambda$-returns in actor-critic methods. The $\lambda$-return is an unbiased estimator of the Monte Carlo (MC) return, which means they are essentially interchangeable. In fact, as discussed in High-Dimensional Continuous Control Using Generalized Advantage Estimation, using the $\lambda$-return instead of the MC return can actually help reduce the variance of gradient updates.

The above is similar to my answer in the other question you linked, so let me try to answer your question more specifically. Even though we can use $\lambda$-returns, why are they not too common in practice? I suspect there might be a few reasons:

Empirically, faster credit assignment might be more desirable than lower variance. Sometimes the learning speed of your algorithm is constrained simply by how quickly you can learn about the consequences of certain actions. In this case, it is faster to use the MC return, even if it theoretically has higher variance than the $\lambda$-return.
When proposing a new algorithm, adding $\lambda$-returns to it might give it an "unfair" advantage if the other baseline methods do not use them (the reviewers would not like this), so researchers tend to favor simpler 1-step or MC returns for the sake of consistency. I would guess this is why you don't typically see $\lambda$-returns in papers that propose new actor-critic methods. In some sense, it is generally assumed that you could always add $\lambda$-returns to them later and probably get better performance.$^*$
A decent number of deep RL researchers don't know what $\lambda$-returns are. This is especially true if they come from a pure deep learning background; they may have never read Reinforcement Learning: An Introduction which is where most people are introduced to TD($\lambda$) and $\lambda$-returns.

$^*$Exceptions to this are papers like mine (Reconciling $\lambda$-Returns with Experience Replay) where the contribution is the use of $\lambda$-returns in methods that previously could not use them. But actor-critic methods are pretty straightforward to combine with $\lambda$-returns.


What is the formula used to calculate the loss in the FaceNet model? The FaceNet model returns the loss of the predictions and ground-truth classes. How is this loss calculated?

The loss function used is the triplet loss function.

Let me explain it part by part.

Notation

The $f^a_i$ means the anchor input image. The $f^p_i$ means the postive input image, which corresponds to the same people as the anchor image. The $f^n_i$ corresponds to the negative sample, which is a different person(input image) then the anchor image.

The formula explained step by step

The first part, $||f^a_i - f^p_i||^2_2$ basically calculates the distance between the anchor image output features and the postive image output features, which you want the distance to be as small as possible as the input is the same person. For the second part, $||f^a_i - f^n_i||^2_2$ , it calculates the distance of the output features of the anchor image and the negative image. You wnat the distance to be as large as possible as they are not the same person. Finally, the $\alpha$ term is a constant(hyperparameter) that adds to the loss to prevent negative loss.

How it works

The loss function optimizes for the largest distance between the anchor and negative sample and the smallest distance of the positive and anchor sample. It cleverly combines both metrics into one loss function. It can optimize for both case simultaneously in one loss function. If there is no negative sample, the model will not be able to differciate different person and vice versa.

Hope I can help you and have a nice day!


GPFlow: Gaussian Process Uncertainty Quantification I trained some Gaussian process model with the Python library GPFlow on a dataset consisting of $(X, Y)$, inputs and outputs, in a regression setting. This model gives me pretty good predictions in the sense that the relative error is small almost everywhere. I want to use the uncertainty as well, which is given in a GPFlow setting in the form of a standard deviation (STD) associated with every prediction. Here's my problem: I normalised both inputs and outputs before training (separately) using sklearn's StandardScaler (effectively making the data normally distributed with $0$ mean and unit STD). So the STD given by the model pertains to the scaled data. How do I "rescale" the uncertainty estimates of the GP to the actual data? Using the inverse_transform function of the output scaler makes little sense. This issue might be easier solvable if I scaled with a MinMaxScaler (squishing all data points into the unit interval) by dividing by the length of the range of the original output set (at least I think it works that way). But how about the case of the StandardScaler? Any insights will be appreciated!

Not sure about which function to call in GPFlow, but the principle is simple. If your $Y$ data was scaled:
$$Y_{scaled} = {(Y - mean(Y)) \over {sd(Y)}}$$
So now if you have any prediction (from the scaled model) $\hat{Y}_{scaled}$, with stand error $\sigma_{\hat {Y}_{scaled}}$, then it is a normal distribution ${\cal N}(\hat {Y}_{scaled}, \sigma^2_{\hat {Y}_{scaled}})$. Then you can very easily rescale it back to the original scale ${\cal N}(\hat {Y}, \sigma^2_{\hat {Y}})$, where:
$$\hat {Y} = \hat {Y}_{scaled}\cdot sd(Y) + mean(Y)$$
$$\sigma_{\hat {Y}} = \sigma_{\hat {Y}_{scaled}} \cdot sd(Y)$$


How to recognize with just name and last name if the person is a political exposed person First than all, I am not sure if this questions is more about Machine Learning, or if its Artificial Intelligence, if not, just let me know I will delete it.

At my company we need to create a solution for banks, where a client comes in and they want to to open a bank account.

They need to know if that person is a politician or political exposed person, maybe they work in the european comission, or they are family from a pep for example.

The business users has lots of data sources where to get these people, for example: http://www.europarl.europa.eu/meps/en/full-list/all

They want to train a Model (Machine Learning), where the end user can enter the name:  Bill Clinton for example, and then the system has to return the percentage of a person being political or not.

Obviosly some persons are 100% politicial and the percentage will be 100%.

But if they enter a name that is not in any of their data sources, how would I train a model to decide if its pep or not?

quite confused

thanks

But if they enter a name that is not in any of their data sources, how would I train a model to decide if its pep or not?

Based on just a person's name and nothing else, the accuracy of this model is going to be very low. Consider that most first name, surname combinations in Europe are going to be repeated across the population.

However, the accuracy might still be slightly better than guessing. Some families and social classes could be more likely to be involved in political work, and a statistical model would pick up on that.

To train the model, take your positive names, and combine with a random selection of names of people that are known to be "not political" or similar enough. It doesn't matter if some of the names are the same provided you are confident in your data. Probably you could just take a phone directory or the electoral register or some other list of general names. Provided your "political" people are a small fraction of all people, this will work well enough even if you have some of them in the negative class.

Ideally you mix those name groups in the rough proportion that the bank expects to see "political" and "non-political" customers, so that your data set is a good representation of the target population.

Then you train a classifier on the names. As this is text and sequence data, you will need a solution for that. Possibly LSTM would be suitable architecture, but so might some feature selection from the names in a more simple ML model.

Remember to hold back some data (both positive and negative cases) for cross-validating and testing the model.

Expect the accuracy of your model when testing to be low. Very low. I would not at all be surprised to find the end result unusable by itself.

If this is seriously to be part of some bank's account setup process, there needs to be additional data used later in the process. A gate for additional checks based purely on someone's name will perform very poorly in my opinion.


How can Transformers handle arbitrary length input? The transformer, introduced in the paper Attention Is All You Need, is a popular new neural network architecture that is commonly viewed as an alternative to recurrent neural networks, like LSTMs and GRUs.
However, having gone through the paper, as well as several online explanations, I still have trouble wrapping my head around how they work. How can a non-recurrent structure be able to deal with inputs of arbitrary length?

Actually, there is usually an upper bound for inputs of transformers, due to the inability of handling long-sequence. Usually, the value is set as 512 or 1024 at current stage.
However, if you are asking handling the various input size, adding padding token such as [PAD] in BERT model is a common solution. The position of [PAD] token could be masked in self-attention, therefore, causes no influence. Let's say we use a transformer model with 512 limit of sequence length, then we pass a input sequence of 103 tokens. We padded it to 512 tokens. In the attention layer, positions from 104 to 512 are all masked, that is, they are not attending or being attended.


How to normalize for perceptual loss when training neural net from scratch? Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) and I standardise each channel of all images based on that mean and standard deviation. My output will be another image.
I want to use for example VGG for perceptual loss (VGG's weights will be frozen). Perceptual loss is when you input your prediction to a pretrained network to extract features from it. Then you do the same for the ground truth and the L2 distance between the features from ground truth and features from prediction is called perceptual loss.
As far as I know, I am supposed to standardise my data based on the mean and standard deviation VGG was trained with (since I am using VGG for inference essentially), which is different than the mean and standard deviation of my dataset. What is the correct way to do this? Should I undo the standardization of my dataset by multiplying standard deviation and adding the original mean to the output of my network, and then restandardise using VGG's statistics to calculate the loss? Or should I continue without restandardising?

I am a bit confused with the part of "training a new neural network from scratch" and then the part of "VGG's weights will be frozen" because the answer changes since they are different cases. It can also happen, from what I read, that you are using VGG as backbone of another network for transfer learning but it is not specified either. Anyway, I will try my best.
The cases I see:

Training a network from scratch: no, you don't need to undo your normalization. In fact, when training from scratch you can normalize however you want.
Training a network from scratch starting from pretrained weights: no, you don't need to change your normalization. Although it would help to faster convergence because the weights are fitted to other kind of normalization. But as the network trains it will fit to your new normalization without any problem.
Training a network that uses VGG as backbone with frozen weights: no, you don't need to change your normalization although it would help. In spite of having your backbone fitted for a specific kind of normalization your network head is still an universal approximator that would fit to any other kind of normalization. However if you use the same normalization you are easing the network's job (that's why we normalize in the first place)

When you really need to force same normalization is in inference. When you want to infer you really need to do the same normalization in preprocessing or you won't have any meaningful results.


Why would an AI need to 'wipe out the human race'? I'm reading such nonsense about how an AI would turn the world into a supercomputer to solve a problem that it thought it needed to solve. That wouldn't be AI. That's procedural programming stuck in some loop nonsense. An AI would need to evolve and re-organise its neurons. It wouldn't be stuck to hardcode if it becomes intelligent by re-writing its code.

It's a possible side effect

Any goal-oriented agent might, well, simply do things that achieve its goals while disregarding side effects that don't matter for these goals.

If my goals include a tidy living space, I may transform my yard to a nice, flat lawn or pavement while wiping out the complex ecosystem of life that was there before, because I don't particulary care about that.

If the goals of a particular powerful AI happen to include doing anything on a large scale, and somehow don't particularly care about the current complex ecosystem, then that ecosystem might get wiped out in the process. It doesn't need to want or need to wipe out us. If we are simply not relevant to its goals, then we are made of materials and occupy space that it might want to use for something else.

We are a threat to most goals

Any goal-oriented agent might want to ensure that they can fulfill their goals. Any smart agent will try to anticipate the actions of other agents that may prevent them from achieving those goals, and take steps to ensure that they succeed anyway. In many cases it is simpler to eliminate those other agents rather than ensure that their efforts fail.

For example, my goals may include storing a bag of sugar in a country house so that I can make pancakes when visiting without bringing all ingredients every time. However, if I leave it there, it is likely to get eaten by rats during winter. I may take all kinds of precautions to store it better, but rats are smart and crafty, and there's clearly a nontrivial chance that they will still succeed in achieving their goal anyway, so an effective extra precaution is killing the rats before they get a chance to try.

If the goals of a particular powerful AI are to do X; it may come to an understanding that (some?) humans might actually not want X but Y instead. It can also easily deduce that some of those humans might actively do things that prevent X and/or try to turn off the AI. Doing things that ensure that the goal gets achieved is pretty much what a goal-seeking agent does; in this case if existence of humans isn't strictly necessary for goal X, then eliminating them becomes a solid risk reduction strategy. It's not strictly necessary and it may take all kinds of other precautions as well, but just like in my example of rats, humans are smart and crafty and there's clearly a nontrivial chance that they will still succeed in achieving their goals (so that X doesn't happen as AI intends) so an effective extra precaution could be killing them before they get a chance to try.


Should I remove the text overlaying some images in the dataset before training the CNN? If I am attempting to train a CNN on some image data to perform image classification, but some of the images have pieces of text overlaying them (for the purpose of description to humans), then is it better for the CNN to remove the text? And if so, then how do I remove the text? Furthermore, is it a good idea to use both the images with text overlaying them and the images with the removed text for training, since it might act as a form of data augmentation?

Removing the overlayed text might increase accuracy, but you'd need to train a different model to do this, and that is an entirely different task as it is no longer classification, but generation. There are easier ways to augment your data and probably get similar benefits to your accuracy. However, if you would still like to do this, there is a lot of examples you can find by simple searching "Watermark removal machine learning" in google. Here's an example I found.
Overall, a CNN will be able to look past the overlayed text without issue, and perform classification like it would without the overlayed text. There is the possibility that it actually learns relationships between the overlayed text and the expected output, but that depends on the data, and is likely a harder task then simply identifying features.
The only issue you might run into is if the real data this model will be used on is different to the data provided, as in the real word images do not contain overlayed text describing what the image is.


Are there emergent models of time in robots? I am interested in the emergence of properties in agents, and, more generally in robotics.

I was wondering if there is work on the emergence of time-related concepts, on the low-level representation of notions like before and after. I know, for example, that there is work on the emergence of spatial representation (similar to knn), or even communication* but time seems to be a tricky concept.

This has everything to do with the platform, i.e. the way that the representation would be coded in. We tend to favour ways that have some meaning or somehow mimic natural, well, yes, human structures, like the brain. I am not a neuroscientist and do not know that the sense of time looks like in humans, or if it is even present in other living beings.

Is there some work on the (emergence of the) representation of time in artificial agents?

*I remember watching a really cool... Actually creepy video from these robots but cannot find it anymore. Does anyone have the link at hand?

To my knowledge, this is very  much an open research issue.

Here is a paper by Prof Leslie Smith, an acknowledged expert on neuromorphic perceptual coding, which explains the importance of the notion of perceptual time for Artificial General Intelligence and sketches an architecture from which a notion of 'now' might emerge: Perceptual Time.


after reading some literature in the area I'd recommend the following:

Try using Convolution Neural Networks (CNNs) this paper outlines some really good points on why should you use CNNs.
Try a Combination of the different layers in the same model. Start with some Convolution Layers, then some LSTMs and then a couple of Dense Layers followed by Dropout