How to enforce covariance-matrix output as part of the last layer of a Policy Network? I have a continuous state space, and a continuous action space. The way I understand it, I can build a policy network which takes as input a continuous state vector and outputs both mean vector and covariance matrix of the action-distribution. To get a valid action I then sample from that distribution.

However, when trying to implement such a network, I get the error message that the parts of my output layer which I want to be the covariance matrix are singular/not positive-semi-definite. How can I fix this? I tried different activation-functions and initializations for the last layer, but once in a while I run into the same problem again.

How can I enforce that my network outputs a valid covariance matrix?

Usually it is assumed that there is no correlation between different actions, so the covariance matrix will be zero everywhere except on the main diagonal. Diagonal will represent variances of actions. Diagonal covariance matrix will be positive semidefinite if all values on diagonal are $\geq$ 0 so you need to insure that output of final layer is $\geq$ 0, which can be done with ReLU activation for example.


How can I train a neural network to detect subliminal messages? Is there a way to train a neural network to detect subliminal messages? Where can I find the dataset on which to train the neural network?

If I have to create the dataset, how would I go about it?

United Nations has defined subliminal messages as perceiving messages without being aware of them, it is unconscious perception, or perception without awareness. Like you may be aware of a message but cannot consciously perceive that message in the form of text, etc.

There are two many types of subliminal messages, one which can be made through visual means, another which can be made through audio.

In visual means, I'm referring to these types:

Messages which are flashed for very short while on the screen.
Messages whose opacity is changed to blend with the background.
Messages whose colors are varied slightly to blend with the background.

Example of 3rd type of subliminal messages: if there is a background of red, on this can be shown a message made up of slight variation of red, as a conscious mind can't distinguish between such close shades of red, people will take the entire thing to be red block, but a subconscious mind notice the slight variation in color, register the message, because humans can see millions of colors.

Setting aside the dubious nature of subliminal messages. You have to clearly define what subliminal is. In all your cases presented, a manual algorithmic approach would be better. An ANN would tend to average out the subliminal messages as noise, just as your brain would. In the case of #2, #3, it's a matter of contrast, in which case computer vision methods of bringing out low contrast details to the forefront combined with a manual algorithm (perhaps an OCR pass for content not expected to have text) would be the way to go. ANN are not magic bullets, and still need to be used appropriately.


Is Intelligence a naturally occurring function of Information Technology? Here intelligence is defined as any analytic or decision making process, regardless of strength (utility), and, potentially, any process of computation that produces output, regardless of the medium.

The idea that AI is part of an evolutionary process, with humans as merely a vehicle of the next dominant species, has been a staple of Science Fiction for many decades.  It informs our most persistent mythologies related to AI.  (Recent examples include Terminator/Skynet, Westworld, Ex Machina, and Alien:Covenant).

What I'm driving at here is that, although the concept of Neural Networks have been around since the 1940's, they have only recently demonstrated strong utility, so it's not an unreasonable assumption to identify Moore's Law as the limiting factor.  (i.e. it is only recently that we have had sufficient processing power and memory to achieve this utility.)

But the idea of AI is ingrained into information technology once automation becomes possible.  Babbage's Difference Engine led to the idea of an Analytic Engine, and the game of Tic-Tac-Toe was proposed as a means of demonstrating intelligence.

What I'm driving at here is that the idea of analysis and decision making are so fundamental in regard to information technology, that it is difficult to see functions that don't involve them.  And, if the strength of analysis and decision making is largely a function of computing power:

Can intelligence be understood as a naturally occurring function of information technology?

Maybe, but it depends to a very large degree on the choice of definition.

One of the biggest challenges for AI researchers, neuroscientists, philosophers, and psychologists, has been that the layperson's understanding of intelligence does not appear to correspond to a well-defined concept. This point was most famously exploited by John R. Searle in his paper Minds Brains and Programs.

Consider your definition again carefully, and notice that decision is unbound. Does a rock rolling down a hill make a decision when it 'decides' to roll to the left? If not, Searle and his ilk would argue that a computer isn't making a decision when it decides to execute the next instruction. It is simply obeying deterministic laws of physics.

To escape this problem and still claim that machines are intelligent, you can either chose to believe that rocks make decisions (a view called panpsychism), or that we should behave as though rocks make decisions because doing so gives us a lot of explanatory power (the intentional stance), or claim that humans are no different from the rock or the computer (eliminative materialism). The last option leaves you with the problem of explaining subjective experiences though.

An excellent book that covers the mainstream arguments in this vein is Haugeland's Mind Design II. All the arguments I've outlined here are covered in great detail within. Among modern AI researchers, I'd say there is a split between those who embrace something like the intentional stance and those who embrace eliminative materialism.


How to deal with padded inputs in a fully connected feed forward network? I have a fully connected network that takes in a variable length input padded with 0.

However the network doesn't seem to be learning and I am guessing that the high number of zeros in the input might have something to do with that.

Are there solution for dealing with padded input in fully connected layers or should I consider a different architecture?

UPDATE (to provide more details):

The goal of the network if to clean full file paths:
i.g.:

/My document/some folder/a file name.txt > a file name
/Hard drive/book/deeplearning/1.txt > deeplearning

The constraint is that the training data labels have been generated using a regex on the file name itself so it's not very accurate.

I am hopping that by treating every word equally (without sequential information) the network would be able to generalize as to which type of word is usually kept and which is usually discarded.

Then network takes in a sequence of word embedding trained on paths data and output a logits that corresponds to probabilities of each word to be kept or not.

If you use fully connected neural network, it will be hard to do what you want. There's no inductive bias to easily generalize from different positions of the strings which you need to extract.

I suggest you use Seq2Seq models here: http://cs224d.stanford.edu/papers/seq2seq.pdf


Are there examples of neural networks (used for control) implemented on a FPGA or on a neurochip? Greetings to all respected colleagues!

I want to consult on the use of FPGAs and neurochips. I plan to use it in my laboratory project for programming control systems on neural networks.

In my work, there are a lot of applications of neural networks, and I became interested in their programming on FPGAs and neurochips.
But I don’t know a single example of a really made and working laboratory prototype in which a neural network is implemented on an FPGA or on a neurochip and controls something. If someone shares the link, I would carefully study it.

Assumption

In this answer it is assumed that with "neurochips" you mean chips made (using neuromorphic engineering) for neuromphic computing.

Related example

From what I currently understand from this article neuromorphic chips, in particular the TrueNorth chip, are being used (or emulated) for embedded systems related signals processing.

Doubt

The signal processing performed by these (emulated) neuromorphic chips might be part of real-life applications involving control systems.


Is there a relationship between Computer Algebra and NLP? My intuition is that there is some overlap between understanding language and symbolic mathematics (e.g. algebra). The rules of algebra are somewhat like grammar, and the step-by-step arguments get you something like a narrative. If one buys this premise, it might be worth training an AI to do algebra (solve for x, derive this equation, etc).
Moreover, when variables represent "real" numbers (as seen in physics, for example) algebraic equations describe the real world in an abstracted, "linear," way somewhat similar to natural language.
Finally, there are exercises in algebra, like simplifying, deriving useful equations, etcetera which edge into the realm of the subjective, yet it is still much more structured and consistent than language. It seems like this could be a stepping stone towards the ambiguities of natural language.
Can anyone speak to whether this has either (1) been explored or (2) is a totally bogus idea?

There isn't, really. Natural language is way more complex and irregular than algebra, which is far more formalised and unambiguous.
So far, in NLP, most success/progress has been made in little toy domains, which exclude most of the complexities of real life, including many ambiguities.
When you say the rules of algebra are somewhat like grammar, then that is because it is essentially a formal language, for which we can specify a grammar. There is currently no complete grammar for any human language (and I doubt there ever will be), let alone a formal one that can be processed by computer.
This was one of the reasons why the first AI boom, where a lot of over-hyped promises where made about being able to translate Russian into English automatically, failed abysmally: natural languages are more than just formal grammars of lexical items.
Stochastic approaches have gone some way towards pragmatic solutions, but when it comes to understanding language they are basically a fudge. And don't get me started on deep learning approaches to NLP.
So the only relationship is that we use the term 'grammar' for the descriptive formalisms in both cases; a formal grammar of algebra would be very different from a grammar for a human language.
This doesn't mean, however, that approaches developed in the field of NLP cannot be applied to algebra: even those which failed in NLP because they were overly limiting. To find out more about this, look for Chomsky Hierarchy -- that describes the different expressive powers of formal languages.
But I would argue that human language is outside of that, because it is not a formal language.


Any library recommended for recognizing squares in an image? What do you recommend for recognizing squares in an image?

The data needed to know is:

Identify concrete squares.
Identify the pixel coordinates of (the edges of) the squares in the image.

A couple of examples:

20 squares identified.
For each square identify its pixel coordinates.

2 squares identified.
First square coordinates: (38, 124, 86, 187).
Second square coordinates: (88, 124, 136, 187).

You might want to look into building convolutional neural network (CNN) for object detection using Keras. With plain white squares, it should work pretty good.


How can I incorporate domain knowledge to choose actions in the case of large action spaces in multi-armed bandits? Suppose one is using a multi-armed bandit, and one has relatively few "pulls" (i.e. timesteps) relative to the action set. For example, maybe there are 200 timesteps and 100 possible actions.
However, you do have information on how similar actions are to each other. For example, I might want to rent a car, and know the model, year, and mileage of each car. (Specifically, I want to rent a car on a daily basis for each day in a 200 day period; on each day, I can either continue with the existing car or rent a new one. There are 100 possible cars.)
How can I exploit this information to choose actions that maximize my payoff?

You will want to look into Contextual Multi-Armed Bandits. These are MAB problems that additionally involve feature vectors in some way.

You'll sometimes see researchers considering problems where you get to see a single feature vector per timestep (like an "environment state" you're in) which may provide useful information. You'll also sometimes see researchers considering problems where you observe a single feature vector per arm/action (per timestep). That second case is pretty much what you're describing; every arm/action has a feature vector (model, year, mileage of car), and you can use those to make predictions and generalize across arms.

To give a little bit of flavour for these algorithms, the most common ones simply assume that the rewards you observe after pulling arms are given by a linear combination of features, plus some noise. Then, they simply try to find (through online learning) a parameter vector such that observed rewards can be accurately approximated by the dot product of the parameter vector and a feature vector. Of course, there are lots of different variants of algorithms, and not all are linear, but this gives an idea of what they generally look like.


Is there a "better" (signal-based) language for artificial intelligence I assume, there must be "signal-driven" and maybe also real-time programming language, which based on connectivy-data more than variables (int, string, etc).

I would like to have a language without equaton (x=4) but more like "x related to 4" or "cat related to animal" etc...

What You need are other ways of knowledge representation, such as semantic networks or conceptual graphs. there you can define any possible relation between your entities. the knowledge of "x related to 4" exactly fits into "frames" and "semantic networks".
Jaynes in his book,discusses thoroughly what "plausibility" means and why we need to take into account weak syllogisms and start using probability theory as a platform for developing a (general) AI. this might also help with your "reasoning" phase (after you've developed your knowledge base)


Who is working on explaining the knowledge encoded into machine learning models? The thing about machine learning (ML) that worries me is that "knowledge" acquired in ML is hidden: we usually can't explain the criteria or methods used by the machine to provide an answer when we ask it a question.
It's as if we asked an expert financial analyst for advice and he/she replied, "Invest in X"; then when we asked "Why?", the analyst answered, "Because I have a feeling that's the right thing for you to do."  It makes us dependent on the analyst.
Surely there are some researchers trying to find ways for ML systems to encapsulate and refine their "knowledge" into a form that can then be taught to a human or encoded into a much simpler machine.  Who, if any, are working on that?

I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.

The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.

Other than that, in opposite to our minds, it looks like this solution can produce these structures in readable for us forms.


What are good alternatives to the expression "Artificial Intelligence"? I read a really interesting article titled "Stop Calling it Artificial Intelligence" that made a compelling critique of the name "Artificial Intelligence".

The word intelligence is so broad that it's hard to say whether "Artificial Intelligence" is really intelligent. Artificial Intelligence, therefore, tends to be misinterpreted as replicating human intelligence, which isn't actually what Artificial Intelligence is.
Artificial Intelligence isn't really "artificial". Artificial implies a fake imitation of something, which isn't exactly what artificial intelligence is.

What are good alternatives to the expression "Artificial Intelligence"? (Good answers won't list names at random; they'll give a rationale for why their alternative name is a good one.)

Artificial is said to derive from the Latin word "artificium" which connotes ideas such as crafting.  Thus, artificial is a correct usage, and algorithms can be regarded as "artifacts" in the context of information as opposed to physical manifestation of information (i.e. matter).

However, I agree that the use of artificial is problematic in that, should strong Artificial General Intelligence ever be achieved, there is a stigma to "artificiality" that could have implications regarding personhood.

My personal feeling is that we should be using:

Algorithmic Intelligence

which this is functional definition, and therefore more meaningful than "artificial".  Additionally, "algorithmic" is a neutral term, and provides a very accurate description of what these systems are.

In terms of what is considered "intelligent", you may want to look at the concept of Bounded Rationality.  There is no hard definition of "intelligence", just degrees of optimality in regard to decision making in a condition of uncertainty.

Because this is subjective for any problem that is not solved, modifiers are utilized, and thus we refer to AI as "strong" or "weak". These terms are also used to describe the degree to which certain types of problems (for instance a non-chance, perfect information game like Checkers) has been solved. Complexity theory will shed more light on this concept.

For more insight on "artificial", you might find this question on the philosophical origin of the Turing Test interesting, because it partly involves the meaning of a "thing". (There were multiple words for this in Ancient Greek.)


Output volume proof for convolutional neural network As I've been dabbling into the sliding window concept, I stumbled on a question that asked me to find the number of windows needed on a 1D image of $W$ size, knowing the window size $K$ and the stride $S$.
As much as I tried, I couldn't find a formula by myself (the closest I got was this one : $N=\frac{W + x(K-S)}{K}$ where $x$ was the number of overlapping rectangle zones, which seemed to be $x=N-1$ but the reccurence wasn't what I was looking for and it could be wrong as I was reasoning through induction).
I find the right formula on Internet at last (this one : $N=\frac{W-K+2P}{S}+1$ with $P$ the padding but my problem didn't needed one) but I can't find the proof of it.
Is there any place where I could find the proof ?

You can think about the problem in the following way (without padding, as the padding case is a simple extension of base case with $\tilde{W}:=W + 2P$).
You want to know how many windows are necessary to cover an image of size $W$, given a window of size $K$ and stride $S$. So your image is a vector with indices $1, 2\dots, W$; as you put the first window on the image, the window will cover the indices from $1$ to $K$. As we apply stride (meaning that we translate the window), we will  get a sequence considering the last covered index. The first element of this sequence is $i_1=K$, where the $1$ indexing $i$ is not the number of times we apply stride but the number of covering windows we have. So in this first case applying no stride and we have 1 covering window. Applying stride once to our window, the window will cover the indices form $S$ to $K+S$, so $i_2=K+(2 -1 )S$. In general you get $i_n=K+(n-1)S$.
Now, if we can exactly cover $W$ then there is a number $N$ such that $i_N=W$. This means $$K+(N-1)S=W,$$ which rearranging gives $$N=\frac{W-K}{S} + 1.$$


What does the figure "Blackjack Value Function..." from Sutton represent? I came across this graph in David Silver's youtube lecture and Sutton's book on reinforcement learning.

Can anyone help me understand the graph?
From the graph, for 10000 episodes what i  see is that when we don't have a usable ace we always lose the game except if the sum is 20 or 21. But in case if we have a usable ace there are chances to win when our sum is below 20. I don't know how this is possible.

The left hand graphs are showing you the estimated value function from using Monte Carlo evaluation, after 10,000 episodes. They give a sense of what your value table will look like before convergence.

In the case of upper "usable ace" chart, the estimates are still showing a lot of inaccuracy due to variance in the data. This is for two main reasons:

The probability of getting a usable ace at the start is a fraction of all (around 15%), so the number of samples used to build the chart is lower.
There are more variations in play when there is a usable ace, due to the extra flexibility allowed by it, so the end result also varies more, requiring more samples to converge.

In addition, if you are looking at the bottom edge of the chart, this represents the player starting with two aces. If you pick on one of the high points (dealer shows 4), then that also reduces the probability of seeing that particular state. So you are looking at a sample size of typically 4-5, but maybe in this case just one sample or maybe two, which the player then happened to go on to win, even though the odds made it unlikely. There is always some chance of winning and dealer showing 4 is a bad start for the dealer, who has a good chance of going bust provided the player does not.

If it hadn't happened this time for the "two aces + dealer showing 4" state, it may have happened for "two aces + dealer showing 5" state. That's due to  the nature of random sampling - if you have hundreds of states to sample, then a few of them are going to behave like outliers purely by chance until you have taken enough samples.

In short, 10,000 randomly sampled games are nowhere near enough to reduce error bounds on the value estimates to reasonable numbers for special cases such as starting with two aces. However, you can see in the 10,000 samples charts the beginnings of convergence, especially elsewhere in the chart.

  From the graph, for 10000 episodes what i see is that when we don't have a usable ace we always lose the game except if the sum is 20 or 21.

Actually you don't see that, the expected result is not quite -1.0, but a little higher. So that means there is still a chance to win. Under this player policy, the worst chance is no usable ace and score 19, because the policy will be to "hit" and need an Ace or 2 card just to stay in the game. Even then , the value is not quite as low as -1.0, but more like -0.9


If you mean to binarize the vector such that all positive values become 1 and the rest becoming 0, then you can do this.

bin_arr = np.zeros_like(FV_R)
bin_arr[FV_R > 0] = 1.

As an example,

In [7]: arr
Out[7]:
array([[ 0.15, -0.52],
       [ 1.  , -0.43]])

In [8]: bin_arr = np.zeros_like(arr)

In [9]: bin_arr[arr > 0] = 1

In [10]: bin_arr
Out[10]:
array([[1., 0.],
       [1., 0.]])
```


What is "Word Sense Disambiguation"? I recently came across this article which cites a paper which apparently won outstanding paper in ACL 2019. The theme is that it solved a longstanding problem called Word Sense Disambiguation.

What does the expression Word Sense Disambiguation mean? How does it affect NLP? And also how does the proposed method solve this problem?

"Word Sense Disambiguation" refers to the idea that words can have different meanings in different contexts. Here are some examples

"I went to the river river bank" vs "I deposited my check at the bank"
"He's mad good at that game" vs "I am so mad at you"

How it effects NLP, comes down to the way we process text. This generally includes the steps of tokenization and embedding them into some form of vector space. These embeddings in many cases are trained either through some self supervised task on some corpora (examples include Word2Vec or Glove) or doing it from scratch under whichever task/data-set is being used.

Now regarding that paper it does not solve the problem but it does extend a new methodology that assists in helping achieve a better learned generalizable representation for this task. The way I interpreted it is that they don't just use a sense-label (which would be a one-hot encoding / what they call discrete) but instead use a continuous sense representation and do the comparison there. This difference allows for words with similar but different senses to not be equidistant from words with completely different senses.


Can the inputs and outputs of a neural network be a neural network? Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that "if some NN exist, then edit any NN".

I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence.

A neural network essentially is a function:

$$\mathbf{y} = f(\mathbf{x}, \mathbf{\theta})$$

Where $\mathbf{x}$ is a vector input, $\mathbf{\theta}$ are changeable or learnable parameters, and $\mathbf{y}$ is a vector output.

There are some variations of this in practice, as you can make special arrangements of $\mathbf{x}$ and $\mathbf{y}$, or use an internal state and feedback loops to allow either or both $\mathbf{x}$, $\mathbf{y}$ to be sequences. However, the above function is basically what a neural network is; how it works beyond that summary are details that you can study.

In supervised learning, you are interested in fixed sizes/shapes of $\mathbf{x}, \mathbf{y}, \mathbf{\theta}$ and trying to find a value of $\mathbf{\theta}$ such that

$$f(\mathbf{x}, \mathbf{\theta}) \approx g(\mathbf{x})$$

where $g(\mathbf{x})$ is some "true" function that you care about, and can find or generate examples of, but typically don't fully know. The value $\mathbf{\theta}$ is called the parameters of the neural network. In addition to these parameters, there are also hyper-parameters of the neural network, which include how many neurons there are in each layer, the valid connections between them, which non-linear function is applied after summing connections between them, etc.

There are learning algorithms used to find $\mathbf{\theta}$ - the many variations of gradient descent being the most popular. Some algorithms - mainly evolutionary approaches - can also vary hyper-parameters, although a more common approach is to repeatedly find $\mathbf{\theta}$ using gradient descent, and vary hyper-parameters in different learning trials, using some metric of performance using test data to pick the best one.

  Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that "if some NN exist, then edit any NN".

Yes - partially. This is a data representation issue. To use it as an input, you would need to express the state of a neural network as a vector - or sequence of vectors - for the input, and the output/edit would also need to be a vector. Probably the simplest way to do this would be to use one network directly output a fixed length vector for $\mathbf{\theta}$ of the target network given an existing value of $\mathbf{\theta}$. That would not allow you to change connections or layer sizes etc, but it would be a very straightforward way to express "one neural network altering another" (ignoring whether this was in any way useful for a task).

If the output was a full representation of the new network, then you would have a function that took as input the definition of one neural network, and output the definition of another network. It would be up to you to convert to/from implemented neural networks and the representations for input $\mathbf{x}$ and $\mathbf{y}$.

If the output $\mathbf{y}$ was an "edit" for a change $\mathbf{x} \rightarrow \mathbf{x}'$, then you would have to decide what edits are allowed, design the representation and write code that applied the edit (the NN would not actually make any changes to another NN by itself). There is no standard way to do this, although there are things you could base this on (such as NEAT).

The big unanswered question with both of the approaches though is what your "true function" $g(\mathbf{x})$ is supposed to be. Having a neural network that represents a neural network generator or edit function is only half of the problem. You also need a way to either generate "correct" outputs to learn from, or a way to assess outputs against a goal.

The goal cannot simply be "make a valid edit", as the number of valid edits that will do nothing useful vastly outnumbers the number of edits that have some specific purpose. This is a similar issue to the fact that there are roughly $2^{8000000}$ valid 1MB files, but only a small proportion of those will be valid image files, and a smaller proportion still will be valid images that represent a natural image that could be taken by a camera. Neural networks that generate natural images therefore must be trained using natural images as a reference, otherwise they will tend to produce meaningless static-like noise.

  I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence

This is very broadly compatible with an Articial Life approach to AI. Although there are two differences between what you are proposing and typical A-life approaches:

Evolutionary algorithms need some measure of fitness, in order to select the best performing individuals to take forward. A-life solves this by implementing a very open environment that makes no direct judgement on outputs of functions, but allows virtual creatures that collect enough resources (defined in the environment) to procreate.

Your suggestion contains no hint that you are thinking of any kind of measure of success or fitness for either the editing NN or target NN. You will need some measure of fitness at least for the target network (and maybe the editing network too) if you intend to use evolutionary algorithms

A-life typically does not treat the evolutionary algorithm itself (the editing or the NNs) as a learning goal. You will not see the results of a good or bad editor until many simulations have passed, so this "meta-search" is likely to be incredibly slow.

A-life simulations are typically already quite slow to reach behaviour which is interesting (because it has emerged without direction from the developer), but usually quite simple for the given environment such as predators chasing prey.

From what we know of the evolution of life, a simple feedback mechanism of RNA molecules editing other RNA molecules - the "RNA world" - is considered a likely pre-life step. This has some parallels with what you are suggesting - and this or something similar perhaps, has resulted in intelligent beings such as ourselves. So your idea perhaps has some merit in a theoretical sense. However, it took biology billions of years to go from such a basic stage to self-aware creatures, and this took the full processing power of large numbers of atoms interacting in ways that even all the computers in the world could not simulate in a fraction of real time.

To speed things up, and turn the idea into something feasible, you would need to look into viable environments and evaluations that would focus the learning towards direct measures of intelligence. Also, you would do well to compare your idea about NNs editing NNs with learning approaches that don't use such a feedback loop.

There is no theory that suggests a NN that can edit or output other NNs would offer any advantage for research into strong AI, compared to other search methods. The idea is basically "AI alchemy" - an idea of an experiment that could perhaps be done, but without any theory backing it as being better or worse than other ideas.

Personally, I would expect the search for a NN which is a good NN editor for a NN which has some other task, to be too slow to be useful when faced with very broad tasks such as exhibiting high level reasoning.


Why do we need multiple LSTM units in a layer? What is the point of having multiple LSTM units in a single layer?

Surely if we have a single unit it should be able to capture (remember) all the data anyway and using more units in the same layer would just make the other units learn exactly the same historical features?

I've even shown myself empirically that using multiple LSTMs in a single layer improves performance, but in my head it still doesn't make sense, because I don't see what is it that other units are learning that others aren't? Is this sort of similar to how we use multiple filters in a single CNN layer?

Let's write down Fibonacci!

  K = 0 1 1 2 3 5 ...

And another series that is derived from Fibo;

  X = 1 4 7 12 30 $X_{5}$

Guessing $X_{5}$ is our task and both series are available to you (Fibonacci as an additional feature).

One unit that you feed with X will try to capture the relation of $X_{t}$ and $X_{t-1}$ only;

$X_{t}$ = $X_{t-1}$ + $X_{t-2}$

However, an additional unit that you insert&feed with K will not only try to capture $K_{t}$ and $K_{t-1}$ relation but also the relation between $K_{t}$ and $X_{t}$.

$K_{t}$ = $K_{t-1}$ + $2*$$X_{t}$ + $1$

In the example above, there is a clear correlation between $K_{t}$ and $X_{t}$ (which is not always the case) and it will support the network to capture the sequential relation. Even the CORRELATION is not crystal clear, almost every additional feature data correlates with the other features and will support the network to grab the relation.


structure of neural network for classification problems with large amounts of null classifications I am building a Convolution neural network to predict certain categories based on images (the location of a pointer on a surface) . However in many cases there will be no pointer in the view or something that is not the pointer. Initially I was just going to train it with outputs of the different classifications including the null classification. However given that the null classification is far more common than the others (perhaps 1000 times more likely) would it be better to have a separate null classifier and then if this outputted non null then the second classifier would be used.

Any suggestions?

I see multiple reasons to take a different route:

While in "classical" pattern recognition you might have done things like feature engineering outside of your model, one idea of deep learning was to "insource" it into the model and let the model take care of "everything". Following that, I have seen a general tendency in deep net architectures to let your deep net handle all the work in one single model. So your idea kind of contradicts the architectural mainstream.
There are so many deep net architectures being published which are well engineered and tested for all kinds of tasks. Therefore, I would check if there is anything "ready off the shelf" for a task like yours in the very first place. If there is, then it will probably be better than your self-engineered model architecture. And if there is not, you might want to check related tasks and see how they solved this problem, e.g. medical applications where specific cells need to be detected with most examples being negative (and if there is a positive example the area in a picture might still be mostly negative as it is just a single positive cell among many others).
Your thought of feeding the "present/not present" binary classification to your task of locating an object is very closely related to what has been discussed as auxiliary tasks. I don't know if it has been applied to exactly your problem but there are similar applications. For example there is an application where "name recognition" is complemented by the auxiliary task of deciding whether any name is present in a given sentence or not. Which is somewhat similar to your case. The paper An Overview of Multi-Task Learning in Deep Neural Networks provides an overview. A famous example using auxiliary tasks would be GoogleNet. This also goes back to my first point of letting the deep net handle "everything" internally.

However, as a disclaimer, this is a theoretical perspective as I cannot speak from experience regarding your problem.


Automatic prediction of whether a customer will come into the shop or not So as my university project I am planning to make a prediction system as described in the title. My current idea is to use the age/gender classifier and run it on a video(taken in front of a shop) which outputs a csv file of the age/gender/Customer ID. In addition, I will use the existing data of the shop of who came in/who didn't come into the shop but passed by the shop and by running XGBoost on this csv data I can predict which customer will come into the shop or not.

Do you think this idea is possible? Is there any other way to implement this idea. It would also be great if we could implement this in such a way as to make the deep learning model learn the various features of those who come into the shop or not.

This should be possible, but it's not completely clear what you are trying to do.

If you're trying to predict customer age and gender from a video, then you've got a computer vision problem. Deep learning methods are the state of the art for this, and probably some sort of convolutional deep neural network is your friend.

If you're trying to predict whether the customer will enter the shop from their age and gender, then @DuttA suggestion to start with simple classification techniques is probably the best bet. Try out logistic regression as a starting point.

If you really want to do both of these things at once, you can again try out a deep convolutional neural network.

All that said, it's not completely clear that there is signal in the videos you want to collect: it might be that predicting whether they'll enter the shop is not really possible on the basis of a short video. The only way to know for sure is to give it a try however.


How do I handle large images when training a CNN? Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN.

How do I handle such large image sizes without downsampling?

Here are a few more specific questions.

Are there any techniques to handle such large images which are to be trained?
What batch size is reasonable to use?
Are there any precautions to take, or any increase and decrease in hardware resources that I can do?

Here are the system requirements

Ubuntu 16.04 64-bit
RAM 16 GB
GPU 8 GB
HDD 500 GB

How do I handle such large image sizes without downsampling?

I assume that by downsampling you mean scaling down the input before passing it into CNN. Convolutional layer allows to downsample the image within a network, by picking a large stride, which is going to save resources for the next layers. In fact, that's what it has to do, otherwise your model won't fit in GPU.

  Are there any techniques to handle such large images which are to be trained?

Commonly researches scale the images to a resonable size. But if that's not an option for you, you'll need to restrict your CNN. In addition to downsampling in early layers, I would recommend you to get rid of FC layer (which normally takes most of parameters) in favor of convolutional layer. Also you will have to stream your data in each epoch, because it won't fit into your GPU.

Note that none of this will prevent heavy computational load in the early layers, exactly because the input is so large: convolution is an expensive operation and the first layers will perform a lot of them in each forward and backward pass. In short, training will be slow.

  What batch size is reasonable to use?

Here's another problem. A single image takes 2400x2400x3x4 (3 channels and 4 bytes per pixel) which is ~70Mb, so you can hardly afford even a batch size 10. More realistically would be 5. Note that most of the memory will be taken by CNN parameters. I think in this case it makes sense reduce the size by using 16-bit values rather than 32-bit - this way you'll be able to double the batches.

  Are there any precautions to take, or any increase and decrease in hardware resources that I can do?

Your bottleneck is GPU memory. If you can afford another GPU, get it and split the network across them. Everything else is insignificant compared to GPU memory.


Is transistor the first artificial intelligence? Artificial Intelligence is any device that perceives its environment
  and takes actions that maximize its chance of success at some goal.

I got this definition from Wikipedia that cited "Russell and Norvig (2003), Artificial Intelligence: A Modern Approach".

A transistor is a device that amplifies or switches electronic signals when it received an input signal.

Could one say the transistor is the AI?

It is certainly a basic building block of every AI out there, but is it an AI itself, albeit the most basic one?

I'm looking at it from a technological and economic point of view, leaving philosophy aside. From an economic perspective it seems to be an AI because transistor does useful work that it took an intelligent human to perform less than a century ago. And it does it completely on its own.

I think it might come down to whether the transistor is making a decision.  If the transistor is being used as a switch, that would seem to qualify as a decision, even though it's an extremely rudimentary decision.

Intelligence, in reference to Artificial (or Algorithmic) Intelligence, is not restricted to high intelligence.  A brute force Tic-Tac-Toe AI has extremely low, narrow intelligence but still constitute AI.  An automatic switch, which makes the most simple decision possible, a binary choice, would seem to be the most basic form of intelligence.

Norvig's definition seems rooted in game theory, which is important in terms of utility of intelligence.  But in a condition of intractability one is only assuming one's decision is more optimal than other choices. Outcomes can be evaluated, and a determination made as to whether the algorithm was "smart" or "dumb", but these terms refer to relative positions on a spectrum.

It's worth noting this fundamental definition of AI opens up a can of worms in that pre-transistor, automated mechanical switches such as the Strowger switch would also probably qualify.

And automata do not have to be electrical.  The History of Automation wiki suggests the first feedback control system was for a water clock invented by Ctesibius.  This device dates to the 3rd Century BCE, and water clocks were said to be the most accurate time pieces until Huygens.

This type of intelligence I tend to think of as autonomic, in the sense of involuntary, and distinct from higher functions, which are more commonly associated with "intelligence".

Note: The characteristics of an autonomic system in a computing context are quite interesting and include self-optimization, self-learning, and self-awareness.

ADDENDUM: After much thought, in reference to @JT's answer, I can't see how decisions can be separated from goals—there needs to be an intent, or the decision is merely random. This might prompt the question "can a simple switch be said to have a goal?"


Are CNN, LSTM, GRU and transformer AGI or computational intelligence tools? Will CNN, LSTM, GRU and transformer be better classified as Computational Intelligence (CI) tools or Artificial General Intelligence (AGI) tools? The term CI arose back when some codes like neural networks, GA, PSO were considered doing magical stuff. These days CI tools do not appear very magical. Researchers want codes to exude AGI. Do the current state of art Deep Learning codes fall in the AGI category?

CNNs, LSTMs, GRUs and transformers are or use artificial neural networks. The expression computational intelligence (CI) is often used interchangeably with artificial intelligence (AI). CI can also refer to a subfield or superfield of AI where biology is often an inspiration. See What is Computational Intelligence and what could it become? by Włodzisław Duch.

RNNs are Turing complete and CNNs have been shown to be universal function approximators (they can approximate any continuous function to an arbitrary accuracy given a sufficiently deep architecture), but that doesn't mean we will be able to create AGI with them, unless you believe that AGI is just a bunch of algorithms, but, IMHO, that alone doesn't produce AGI. See also the computational theory of mind.

To conclude, CNNs, LSTMs, GRUs and transformers are deep learning tools (so they could also be considered CI tools, given some definitions of CI), which might be useful for the development of AGI.


What does "Wide" vs. "Deep" mean in the context of Neural Networks? From this article, I read that "to accurately classify data with neural networks, wide layers are sometimes necessary."

However, I have seen many implementations and discussions on deep-learning, such as this, mention the concept of depth.

What is the difference in the context of neural networks?  How does width vs depth impact a neural network's performance?

The width refers to the number of neurons in a layer. The depth refers to the number of layers.

Have a look at the following question regarding the impact of these hyper-parameters on the performance of the neural network: https://stats.stackexchange.com/q/214360/82135.


What is a bad local minimum in machine learning? What is "bad local minima"?

The following papers all mention this expression.

Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit
limination of All Bad Local Minima in Deep Learning
Adding One Neuron Can Eliminate All Bad Local Minima

The adjective bad isn't mathematically descriptive. A better term is sub-optimal, which implies the state of learning might appear optimal based on current information but the optimal solution from among all possibilities is not yet located.

Consider a graph representing a loss function, one of the names to measure disparity between the current learning state and the optimal. Some papers use the term error. In all learning and adaptive control cases, it is the quantification of disparity between current and optimal states. This is a 3D plot, so it only visualizes the loss as a function of two real number parameters. There can be thousands, but the two will suffice to visualize the meaning of local versus global minima. Some may recall this phenomenon from pre-calculus or analytic geometry.

If pure gradient descent is used with an initial value to the far left, the local minimum in the loss surface would be located first. Climbing the slope to test the loss value at the global minimum further to the right would not generally occur. The gradient will in most cases cause the next iteration in learning algorithms to travel down hill, thus the term gradient descent.

This is a simple case beyond just the visualization of two parameters, since there can be thousands of local minima in the loss surface.

There are many strategic approaches to improve the speed and reliability in the search for the global minimum loss. These are just a few.

Descent using gradient, or more exactly, the Jacobian
Possibly further terms of the multidimensional Taylor series, the next in line being related to curvature, the Hessian
Injection of noise, which is at an intuitive (and somewhat oversimplified) level is like shaking the graph so a ball representing the current learning state might bounce over a ridge or peak and land (by essentially trial and error) the global minimum — Simulated annealing is a materials science analogy and involves simulating the injection of Brownian (thermal) motion
Searches from more than one starting position
Parallelism, with or without an intelligent control strategy, to try multiple initial learning states and hyper-parameter settings
Models of the surface based on past learning or theoretically known principles so that the global minimum can be projected as trials are used to tune the parameters of the model

Interestingly, the only way to prove that the optimal state is found is to try every possibility by checking each one, which is not nearly feasible in most cases, or by relying on a model from which the global optimal may be determined. Most theoretical frameworks target a particular accuracy, reliability, speed, and minimal input information quantity as part of an AI project, so that no such exhaustive search or model perfection is required.

In practical terms, for example, an automated vehicle control system is adequate when the unit testing, alpha functional testing, and eventually beta functional testing all indicate a lower incidence of both injury and loss of property than when humans drive. It is a statistical quality assurance, as in the case of most service and manufacturing businesses.

The graph above was developed for another answer, which has additional information for those interested.

If digital values are mere estimates, why not return to analog for AI?


Using ConceptNet5 to find similar systems to solve specific problems? I installed a local running instance of the ConceptNet5 knowledgebase in an elasticsearch server. I used this data to implement the so-called "Analogietechnik" (a creativity technique to solve a problem from the perspective of another system) as an algorithm.

The technique works as follows:

Choose a Feature of a System
Find Systems who have this feature also
Solve the problem from the perspective of these other systems
Apply the found solutions to the issue

As an example is here the problem of marketing a shopping mall: A Shopping mall has many rooms and floors (1). A museum has also many rooms and floors (2). How are museums marketed? They present many pictures or sculptures (3). We could use our rooms and floors to decorate them with pictures and sculptures (4).

Of course the idea to implement that as an artifically intelligent algorithm was not far. However, I feel a little bit overwhelmed by the amount of methods that exist out there. Neural Networks, Bayesian Interference and so on... My current experience doesn't go further than simple machine learning like kMeans-Clustering for example. Do you think it would be very hard to find a solution for this problem?

I'm thinking of a console application, where you can enter a conceptualized problem like "methods for creative writing", for example, and it uses the above method to find possible solutions of the issue. Of course no solution with extensive depth, more something like basic ideas derived from the knowledge database I have.

Lets take as an example a console application where someone asks "how to write a novel":

It should find out first that the system all is about is in the term "novel". To find a feature of that system it just searches concepts containing that term: it finds out "Novel is a story" So thats a feature.
Which systems are also stories? A good concept it should find is e.g. "Plot is a story". (Of course only when I am selecting the search results manually)--> How to find best concepts of a list when not knowing which fits best?
It should then find out that a plot is written using a storyline: "storyline is a plot"
One possible answer of the AI would in this case be: "By writing a storyline"

Do you know some helpful libraries, algorithms or other resources that might help me? I know this is not an easy thing to program, but you might agree that its highly interesting.

Hierarchical Temporal Memory should help with this. You would have to encode the text data into SDRs. You would then have a coincidence detector. Could you get the right information back out in the way you're trying to? I think so. I'm not fully learned in HTMs yet but check out Nupic (Open source).


Why does the discount rate in the REINFORCE algorithm appear twice? I was reading the book Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (complete draft, November 5, 2017).

On page 271, the pseudo-code for the episodic Monte-Carlo Policy-Gradient Method is presented. Looking at this pseudo-code I can't understand why it seems that the discount rate appears 2 times, once in the update state and a second time inside the return. [See the figure below]

It seems that the return for the steps after step 1 are just a truncation of the return of the first step. Also, if you look just one page above in the book you find an equation with just 1 discount rate (the one inside the return.)

Why then does the pseudo-code seem to be different? My guess is that I am misunderstanding something:

$$
{\mathbf{\theta}}_{t+1} ~\dot{=}~\mathbf{\theta}_t + \alpha G_t \frac{{\nabla}_{\mathbf{\theta}} \pi \left(A_t \middle| S_t, \mathbf{\theta}_{t} \right)}{\pi \left(A_t \middle| S_t, \mathbf{\theta}_{t} \right)}.
\tag{13.6}
$$

The discount factor does appear twice, and this is correct.
This is because the function you are trying to maximise in REINFORCE for an episodic problem (by taking the gradient) is the expected return from a given (distribution of) start state:
$$J(\theta) = \mathbb{E}_{\pi(\theta)}[G_t|S_t = s_0, t=0]$$
Therefore, during the episode, when you sample the returns $G_1$, $G_2$ etc, these will be less relevant to the problem you are solving, reduced by the discount factor a second time as you noted. At the extreme with an episodic problem and $\gamma = 0$ then REINFORCE will only find an optimal policy for the first action.
In continuing problems, you would use different formulations for $J(\theta)$, and these do not lead to the extra factor of $\gamma^t$.


Have I understood the loss function from the original U-Net paper correctly? In the original U-Net paper, it is written

The energy function is computed by a pixel-wise soft-max over the final
feature map combined with the cross entropy loss function.
...
$$
E=\sum_{\mathbf{x} \in \Omega} w(\mathbf{x}) \log \left(p_{\ell(\mathbf{x})}(\mathbf{x})\right) \tag{1}\label{1}
$$

where $w(\mathbf{x})$ is a weight map (I'm not interested in that part right now), and $p_{k}(\mathbf{x})$ is
$$
p_{k}(\mathbf{x})=\exp \left(a_{k}(\mathbf{x})\right) /\left(\sum_{k^{\prime}=1}^{K} \exp \left(a_{k^{\prime}}(\mathbf{x})\right)\right)
$$
The pixel-wise softmax with $a_{k}(\mathbf{x})$ being the activation in feature channel $k$ at pixel position $\mathbf{x}$ and $K$ the number of classes. Then $\ell(\mathbf{x})$ from $p_{\ell(\mathbf{x})}$ is the true label of each pixel, i.e. if the pixel at position $\mathbf{x}$ is part of class $1$, then $p_{\ell(\mathbf{x})}$ is equal to $p_1(\mathbf{x})$.
As far as is understand $-E$ should be the cross-entropy function. Right? I've already done the math for the binary case (ignoring $w(\mathbf{x})$) and it seemed to be equal.

Yes, $E$ is the cross-entropy function and a direct generalization of the binary case.
For the binary case, probability to belong to the class $1$ is given by a sigmoid function $\sigma(x)$ of the output $x$, and the probability to belong to the class $0$ is $1 - \sigma(x)$.
Therefore the binary crossentropy will give:
$$
-\sum_i (l_i \log \sigma(x) + (1 - l_i)  \log (1 - \sigma(x))
$$
Where the sum is over all samples in the dataset. Because $l_i$ is a binary variable one of these two terms would be zero. The gradient descent forces the model to predict true label with more confidence.
For the multiclass case, now the output is a $K$-vector and the softmax function forces its elements to sum to one:
$$
\sum_{i = 1}^{K} p_i = 0
$$
The true label is one-hot encoded vector, with $1$ on the position of the true label, and $0$ elsewhere. The generalization of the binary case is:
$$
-\sum_i \sum_{j = 1}^{K}(l_{ij} \log \sigma_j(x) + (1 - l_{ij})  \log (1 - \sigma_j(x))
$$
In this case, there will be $K$ non-vanishing contributions.The minization of $E$ forces the classifier to predict the true label more confidently, and all other (to make $\log(1 - \sigma_j(x))$) as small  as possible.


What algorithms does stackoverflow use for classifying duplicate questions? Can I get details about the algorithms used for classifying questions in stackoverflow ("Questions that may already have your answer"). Most of the suggestions I get are nowhere related to the question I have intended to ask.

I do not believe that StackExchange has published precisely what algorithm(s) they use for that, so we can't tell for sure.

However, in this meta.stackexchange question, you can follow some of the efforts that were undertaken in collecting training data for training such a classifier. The post also links to the "CQADupStack: Gold or Silver?" paper which describes the analysis of such a dataset that comes directly from StackOverflow. You might be able to find interesting literature by browsing google scholars for papers that cite this paper.

There is also another meta.stackoverflow discussion on this topic, where answers link to various community-developed projects / bots for this purpose. Again, not necessarily what is actually used by the StackExchange sites, but likely similar.

Finally, there is definitely a lot of research on performing such classifications (one example found by a quick google search is "Duplicate Question Detection in Stack Overflow: A Reproducibility Study", where many other relevant publications can be found in the list of References). This again does not necessarily lead to be precisely the algorithm that StackExchange happens to use, but many relevant ones, one or more of which they might be using.


Why do we use the word "kernel" in the expression "Gaussian kernel"? I've heard the expression "Gaussian kernel" in several contexts (e.g. in the kernel trick used in SVM). A Gaussian kernel usually refers to a Gaussian function (that is, a function similar to the probability density function of a Gaussian distribution) that is used to measure the similarity between two vectors (or numbers).

Why is this Gaussian function called a "kernel"? Why not just calling it a Gaussian (similarity) function? Does it have something to do with the kernel of a linear transformation?

The usage of the word "kernel" in the context of support vector machines probably comes from its usage in the context of integral transforms.

See the article Kernel of an integral operator, and the questions What is the difference between a kernel and a function? and Why is the kernel of an integral transform called kernel?.

The word "kernel" has been used in many other contexts, such as in computer vision, to refer to a certain function with a special purpose. See e.g. the paper that introduced SIFT.


How can an AI system develop its domain knowledge? Is there more than just Machine Learning? So machine learning allows a system to be self-automated in the sense that it can predict the future state based on what it has learned so far. My question is: Are machine learning techniques the only way of making a system develop its domain knowledge?

Well, we are talking about a system (a machine) which develops knowledge (learns), so it is kind of difficult for such a technique to not fall within machine learning.

But you could argue that inference engines which work on a graph based knowledge database to derive new propositions or probabilities are not part of machine learning. Of course in that case part of the knowledge is not acquired at all, but rather entered by the developers.

I'm still reading up on this, but my impression is that these knowledge databases and inference engines became rather popular in the nineties and many AGI-researchers today still work in that direction.


Emulating human brain - with analogous NN chips Considering the answers of this question, emulating a human brain with the current computing capacity is currently impossible, but we aren't very far from it.

Note, 1 or 2 decades ago, similar calculations had similar results.

The clock frequency of the modern CPUs seem to be stopped, currently the miniaturization (-> mobile use), the RAM/cache improvement and the multi-core paralellization are the main lines of the development.

Ok, but what is the case with the analogous chips? In case of a NN, it is not a very big problem, if it is not very accurate, the NN would adapt to the minor manufacturing differences in its learning phase. And a single analogous wire can substitute a complex integer multiplication-division unit, while the whole surface of the analogous printed circuit could work parallel.

According to this post, "software rewirable" analogous circuits, essentially "analogous FPGAs" already exist. Although the capacity of the FPGAs is highly below the capacity of the ASICs with the same size, maybe analogous chips for neural networks could also exist.

I suspect, if it is correct, maybe even the real human brain model wouldn't be too far. It would still require a massively parallel system of costly analogous NN chips, but it seems to me not impossible.

Could this idea work? Maybe there is even active research/development into this direction?

I'm not sure about "emulating the brain" per-se, but in a more general sense there has been some thought given to using analog computing for AI/ML.  It seems clear that analog computers do have certain advantages over digital computers.  For one, they can (depending on the application) be faster, albeit at the cost of some loss of precision.  But that's OK, because I don't think anybody believes the human brain is calculating floating point math using digital computing techniques either.  The human brain appears, at least superficially, to be largely probabilistic and able to tolerate some "slop" numerically.

The downside to analog computers, as I understand it, is that they're not as flexible... you basically hardwire a circuit to do one specific "thing" and that's really all it can do.  To change the "programming" you have to literally solder in a new component! Or, I suppose, adjust a potentiometer or adjustable capacitor, etc.  Anyway, the point is that digital computers are supremely flexible, which is one big reason they came to dominate the world.  But I can see where there could be room for going analog for discrete functions that make up some or all of an intelligent system.

As for research in the area, you might look into whatever DARPA was / is doing. There was an article in Wired a while back, talking about some DARPA initiatives related to analog computing.


Is "dataset size" and "model size" same thing? I mean what is determine my model size, connection amount between layers and neurons, or size of my dataset?

Dataset and model refer to different things. Dataset means part of data available for training (training dataset) or validation (validation dataset). Model is the learning process goal, the state of the computer "brain" after it has been fully educated (or made its learning). Model size refers to size of the container which contains the model. In deep learning it can be measured by width and depth of the network used, I also found a site comparing different models by npy file size, that physically contains the generated model as computer code. In that case model contained a more complex structure which was documented and size in bytes was for comparison purposes.

So in short, it is roughly speaking the size of layers and neurons, if I have to take one of your options. Dataset is a different thing.

More precise explanation about what is model and what is dataset:

https://www.quora.com/What-are-different-models-in-machine-learning


How to use a NN for seq2seq tasks? I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and predict the output of any other input I give it.
I have 2 files - one with the inputs and another with all the corresponding outputs and would probably use a bunch of Dense Layers with word embeddings to vectorize it into higher dimensions. However, I cannot find any good resources out there for that.
Does anyone know how to accomplish such an NN? Which architectures are best for pattern matching? examples, links, and other resources would be very welcome. I was considering using RNN's but found them not very good in the pattern matching tasks so had ditched them. I would still consider them if someone can provide a plausible explanation...

I think you meant pattern recognition instead of pattern matching in you question. Because pattern matching has nothing to do with NNs as far as I know. So RNNs is the easiest architecture you could use for this task. Have a look at this post. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
It's long, but it's very well written and explains why RNNs work well with sequential data. In brief RNNs accumulate information about what was fed in them previously and store it in the hidden state vector. That information is a summary of a sequence which helps it to predict the output sequence. RNNs can be applied to variable length input and catch dependencies among different parts of the input (as opposed to dense layers that can be applied to a sequence only point-wise).
Here's how exactly you can apply reccurent layers (https://www.tensorflow.org/guide/keras/rnn, https://pytorch.org/docs/stable/generated/torch.nn.RNN.html).
There are more advanced architectures like LSTMs. Also you can try transformers (http://jalammar.github.io/illustrated-transformer/) which use attention mechanism (https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html).


What is the definition of a heuristic function in the BayesChess paper? I am reading BayesChess: A computer chess program based on Bayesian networks (Fernandez, Salmeron; 2008)
It is a chess-playing engine using Bayesian networks. The following is mentioned about the heuristic function in section 3.

Here the heuristic is defined in terms of 838 parameters.
There are 5 parameters indicating the value of each piece (pawn,
queen, rook, knight, and bishop -the king is not evaluated, as it must
always be on the board), 1 parameter for controlling whether the king
is under check, 64 parameters for evaluating the location of each
piece on each square on the board (i.e., a total of 786 parameters,
corresponding to 64 squares × 6 pieces each colour × 2 colours) and
finally 64 more parameters that are used to evaluate the position of
the king on the board during the endgame.

The above sentence contains the parameters used by the heuristic function. But I didn't find the actual definition. What is the actual definition of the heuristic function?

By far the most common form of heuristic evaluation functions for Chess-playing (or, really, any game-playing) agents are simple linear functions. At least when we're talking about handcrafted features that's the case, of course all the hype with Deep Neural Networks in more recent years is different. So, when it's not specified in a paper like this exactly what their heuristic evaluation function looks like, you can relatively safely assume it's just a linear function.

With linear function, I mean that you have vectors of features $\boldsymbol{\phi}(s)$ for your states $s$, and a vector of weights $\boldsymbol{\theta}$, and the evaluation $f(s)$ of a state $s$ is simply given by the dot product (summing up all the multiplications of feature values with their corresponding weights):

$$f(s) = \boldsymbol{\phi}(s)^{\top} \boldsymbol{\theta} = \sum_i \phi_i(s) \times \theta_i(s),$$

where the subscript $i$ indicates taking the $i^{th}$ element of a vector.


Is radial basis function network appropriate for small datasets? I'm a computer engineering student and I'm about to work on my master thesis. My professor gave me a small dataset with brain Computed Axial Tomography records. I would like to use deep learning to help doctors diagnose a certain disease (obviously, I've also got the data for doing supervised learning).

Since the dataset is small, is radial basis function network a good solution? What do you think?

Btw, if you have any type of tips in using the RBF network for this kind of project I would be really grateful.

Not familiar with RBF but when you have a small data set image augmentation can help. You can do this easily with the Keras ImageDataGenerator, documentation is here. Alternatively you can create image augmentation yourself using image processing models like PIL or CV2.


Would AlphaGo Zero become perfect with enough training time? Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor?

(By perfect, I mean it always wins the game if possible, even against another perfect opponent.)

Assuming you mean a mathematically perfect player, similar to what we can achieve trivially in Tic Tac Toe, then the answer is "maybe". The underlying reinforcement learning algorithms that it uses do have some convergence guarantees, but there are some caveats:

Theories of convergence that apply to value and policy functions learned by RL assume unrealistic timescales and decays of learning parameters. If you have actual infinite time and resources then it is possible to explore all board states and learn their values accurately. But then, if you have those resources at your disposal, a brute-force search would work too.
Using neural network approximations to true value functions can put bounds on how well value functions are learned, as they rely on generalisation, and are characterised by an error metric. The values that they calculate are in fact guaranteed to not be mathematically perfect, and in part that is by design (because you want learning from similar states to apply to new unseen states, and as part of that need to accept the compromise that most state values will be slightly incorrect). This is especially true of the fast policy network used to drive Monte Carlo Tree Search (MCTS).
Running longer MCTS during play improves performance at cost of spending more time per move decision. Given infinite resources, MCTS can play perfectly, even from very crude heuristics.

The difference between AlphaGo Zero and a more traditional game search algorithm is to do with optimal use of available computing resources, as set by available hardware, training time and decision time when playing. It is orders of magnitude more effective to use the RL self-play approach combined with the focused MCTS as in AlphaGo Zero, than any basic search. At least for similar puzzles and games, it is more efficient than any other game playing technique that has been explored. We are likely still more orders of magnitude of effort away from a perfect Go player, and there is no reason yet to suspect that this will ever become practical, and Go become "solved".


What should I do when the new generated state has bigger distance to the goal than the parent state? I have implemented the hill climbing algorithm, with side away steps, which can increase the rate of success, because, when you don't have new generated states, you can go back to previous level and choose from there another state.

What should I do when new generated states have bigger distance to the goal than the parent state?

Do I have to go back to the previous level and choose another state, from which I will continue? Or I continue with one of the states generated which has bigger distance?

If you don't go back to your previous state, the best you will do is get stuck in a local maxima. So, to answer your question, yes, backtrack if your hill climbing has failed. The number of times you want to do this depends on you and your implementation. Hill climbing does not guarantee to find the global maxima and, in many cases, you won't know if it is a local maxima or a possible global maxima, unless you try the hill climbing a few times.

If you haven't already seen its Wikipedia article, it has a cool graph showing what I mean.


Which areas of applied math are relevant to AI? My background is in electrical engineering. I have a good grasp of CS foundations (e.g. data structures, algorithms, operating systems, discrete math and software engineering).

I have option of enrolling in a MS program in applied math at a good school. However, I would like to study AI.

Which areas of applied math are relevant to AI (not specifically ML)?

EE Math Refresh

AI is an interdisciplinary field.  You can begin by ensuring you are fresh in the mathematics you've already taken. You may already have all the books from your BS and MS.

Infinite series
Logical proofs
Linear algebra and matrices
Analytic geometry, especially the distinction between local and global extremes (minima and maxima), saddle points, and points of inflection
Set theory
Probability, especially Bayes' Theorem and the common probability distributions
Statistics, especially variance and correlation coefficient
Time series
Infinite series
Convergence — Central to AI batch processing
Partial differentials
Jacobian and Hessian matrices
Multivariate math — Spaces beyond ℝ3 are common in AI.
Boundary regions
Discrete math
Criteria for system stability — Central to real time intelligent control
Bernhard Riemann's manifolds is important for more advanced AI theory

An EE would already understand circuit paths (Gustav Kirchhoff's work, signal transmission, feedback theory, and control theory. To see the connection between AI and EE, Norbert Wiener's seminal work, Cybernetics, 1948, MIT Press, will be a helpful read and give you a clearer picture that both fields have common roots.

Finite Math

Since finite math curricula differ between schools of engineering, this list can be used as a checklist for AI.

Predicate logic and rules engines
Graphs (as in vertices connected by edges) and algorithms for them — pioneered by Leonhard Euler
Game theory — John von Neumann and Oskar Morgenstern's Game Theory (published in 1944 but still fresh and pertinent)
Decision trees
Markov chains and the Markov property — Andrey Markov's work
Information theory — Claude Shannon's extensions of the thermodynamic concepts of entropy, especially uniqueness, redundancy, relative entropy, and cross entropy — A necessity to more fully understand feature abstraction and auto-encoding
Chaos theory, especially auto-correlation, for analysis of chaotic system behavior in phase space — Chaos Theory Tamed, Garnett P. Williams, 1997, provides an excellent overview
Difference between random and pseudo random number generation
Curve fitting and gradient descent, especially the Levenberg–Marquardt algorithm
Stateless versus stateful algorithms, AI system components, and services
Algorithmic theory, especially tail recursion and algorithms for generalization, abstraction, and object recognition
Gödel's uncertainty theorems
Turing completeness and how it does not overturn yet overcomes Gödel's uncertainty
Turing's Imitation Game as a test for conversational intelligence
Other test criteria for intelligence outside the conversation context (i.e. piloting, driving, business intelligence, et cetera)
Topology — pioneered by Henri Poincaré
Computational linguistics — pioneered by Richard Hook Richens — the foundation of much NLP (natural language processing)

Machine Learning

Since machine learning is trending and will probably remain strong even as other AI technologies catch up, there are a few

MLPs (multilayer perceptrons)
Convolution and the use of convolution layers and pooling layers in deep convolution networks
LSTM (long-short term memory) networks, their predecessor RNNs (recurrent neural networks), and some of the newer competitors in the stateful network space
Attention based networks and other gating strategies (largely found online using academic searches for papers)

Soft Sciences

Because most of what is occurring, has occurred, and will occur in AI for the foreseeable future is the replication of human brain capabilities, the EE can point her or his analytical skills at biology (especially neurology), genetics as it may apply to the emergence and control of human intelligence (bioinformatics), psychology (especially cognitive science), and sociology (such as.

Because of the emergence of GANs and other devices that work on the basis of balance and equilibrium, it is useful to recall chemical equilibria and biological stasis.  These concepts apply directly to AI and are likely to play a more major role as the gap between brain network research and computing narrows further.

Automated Vehicles

The obvious emerging AI technology that is nicely aligned with the EE background is the movement of road and air vehicles from human driving and piloting through the various phases of automation.  The goal that most corporations see as the ideal is to progress forward until the driver's seats and cockpits are no longer installed in cars and planes because human interaction other than to provide the transportation destination poses a public safety issue.

Obviously, computer vision, object recognition, sound recognition, and collision avoidance technique are essentials and highly dependent upon rigorous mathematical treatment.


Number of states in taxi environment (Dietterich 2000) Dietterich, who introduced the taxi environment (see p. 9), states the following: In total there “are 500 [distinct] possible states: 25 squares, 5 locations for the passenger (counting the four starting locations and the taxi), and 4 destinations” (Dietterich, 2000, p. 9).

However, in my opinion there are only 25 (grid) * 4 (locations) * 2 (passenger in car) = 200 different states, because for the agent it should be the same task to go to a certain point, regardless of whether it's on its way to pick up or to drop-off. Only the action at the destination is different which would be stored binary (passenger in car or not)

Why does Dietterich come up with 500 states?

This is more of a combinatorics than AI question but regradless, the full state information for the environment is:

$(taxi \space position, passenger \space position, destination \space position)$

There are 25 possible taxi positions, 5 passenger positions and 4 destination positions making it $25 \cdot 5 \cdot 4 = 500$, so the paper is correct.

You are also correct but you divided 1 objective into 2 objectives and you have 2 separate policies, a pickup policy and a dropoff policy. So your state information would be for each policy:

$(taxi \space position, destination \space position)$

There are 25 possible taxi positions and 4 possible destination positions making it $25 \cdot 4 = 100$. You have 2 policies so you have $200$ states.

EDIT

Actually in the second case, I think you could get away with only 1 policy where you would simply change the destination position once you pick up the passenger so you dont need 2 separate policies and you would have only $100$ states


Is overfitting always a bad thing? DNN can be used to recognize pictures. Great. For that usage, it's better if they are somewhat flexible so as to recognize as cats even cats that are not on the pictures on which they trained (i.e. avoid overfitting). Agreed.

But when one uses NN as a replacement for numerical tables in an Air Collision Avoidance System (ACAS), it is primarily to reduce the "required storage space by a factor of 1000". For this usage, what we want from the NN is to say "take a slight left turn" or "turn right hard" if another ship comes slightly close on the right or rapidly close on the left, respectively.

For this usage, where the answer is much simpler than recognizing a cat, isn't overfitting a good thing? What would overfitting "look like" in this case and why would it be bad ?

This question somewhat relates to this one, where a general idea seems to be "Machine Learning is used for intractable things, you don't need ML for tractable things". And while it is quite correct that ACAS can be implemented without NN, I wouldn't call NN "useless" for ACAS, because a factor 1000 reduction in required space will always come in handy.

Typically the ramification of overfitting is poor performance on unseen data. If you're confident that overfitting on your dataset will not cause problems for situations not described by the dataset, or the dataset contains every possible scenario then overfitting may be good for the performance of the NN.


Does the correlation between inputs affect the model performance? I'm currently working on a regression problem and I have 10 inputs/attributes.

What should I do if there are correlations between different features of the input data? Does the correlation between inputs affect the performance (e.g. accuracy) of the model?

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are dependent. If they are dependent, then one feature gives you information about the other and vice-versa: in a certain way, one of the two is, at least partially, redundant.

Unnecessary features might not affect the performance (e.g. the accuracy) of a model. However, if you reduce the number of features, the learning process might actually be faster.

You may want to try some dimensionality reduction technique, in order to reduce the number of features.


Is it possible to train a neural network with 3 inputs and 12 outputs? The selection of experimental data includes a set of vectors of different dimensions. The input is a 3-dimensional vector, and the output is a 12-dimensional vector. The sample size is 120 pairs of input 3-dimensional and output 12-dimensional vectors.

Is it possible to train such a neural network (in MATLAB)? Which structure of the neural network is best suited for this?

There is nothing stopping you, you can setup Dense Neural Networks to have any size inputs or outputs (simple proof is to imagine a single layer NN with no activation is just a linear transform and given input dim $n$ and output dim $m$, it's just a matrix of $n$ x $m$, trivially this works though with any number of hidden layers)

The better question is should you?. In all honesty, it depends on the data that you have, but, usually, with only 120 examples you'll either overfit completely or do relatively well if the true solution is a very simple function, but, in general, in the common situations where that isn't the case I find myself more likely or not using Bayesian approaches, so I can actually consider confidence (with little data, this is really nice)


How do we create a good agent that does not outperform humans? A lot of research has been done to create the optimal (or "smartest") RL agent, using methods such as A2C. An agent can now beat humans at playing Go, Chess, Poker, Atari Games, DOTA, etc. But I think these kind of agents will never be a friend of humans, because humans won't play with a agent that always beats them.

How could we create an agent that doesn't outperform humans, but it has the human level skill, so that when it plays agains a human, the human is still motivated to beat it?

You basically have to degrade the result, assuming that the machine always finds the best move. There are a number of possibilities:

restrict the depth of searching. In early chess programs I believe that was the main way of regulating the difficulty. You stop the evaluation of moves after a particular depth in your search tree has been reached. This would be equivalent to only looking ahead two moves instead of twenty.
set a time limit. This is somewhat similar the restricting the depth of the search, but more generally applicable. If your algorithm accumulates candidate moves, and the general tendency is to get to the better moves after first finding a number of weaker ones, then you can stop at a given point in time and return what you have found then.
distort available information. This might not be that applicable to games such a chess, but you could restrict the information the machine has available for evaluating moves. Something like the "Fog of War" often used in strategy games. With incomplete information it is harder to find a good move, though it is not impossible, which makes it more challenging than, say, restricting the depth of search too much.
sub-optimal evaluation function. If you have a function that evaluates the quality of a move, simply fudge that function to not return the best value. Perhaps add a random offset to the return value to make it less deterministic/predictable.

There are probably other methods as well; the tricky part is to tread the fine line between appearing to be a weaker (but consistent) player, and just being a random number generator.


Why does using a higher representation space lead to performance increase on the training data but not on the test data? I read the following from a book:

You can intuitively understand the dimensionality of your representation space as “how much freedom you’re allowing the model to have when learning internal representations.” Having more units (a higher-dimensional representation space) allows your model to learn more-complex representations, but it makes the model more computationally expensive and may lead to learning unwanted patterns (patterns that will improve performance on the training data but not on the test data).

Why does using a higher representation space lead to performance increase on the training data but not on the test data?
Surely the representations/patterns learnt in the training data will be found too in the test data.

The answer to your question is that the capacity of your model (i.e. the number and type of function that your model can compute) generally increases with the number of parameters. So, a bigger model can potentially approximate better the function represented by your training data, but, at the same time, it may not take into account the test data, a phenomenon known as over-fitting the training data (i.e. fitting "too much" the training data).
In theory, you want to fit the training data perfectly, so over-fitting should not make sense, right? The problem is that, if we just fit all the (training) data, there is no way of empirically checking that our model will perform well on unseen data, i.e. will it generalize to data not seen during training?
We split our data into training and test data because of this: we want to understand whether our model will perform well also on unseen data or not.
There are also some theoretical bounds that ensure you that, probabilistically and approximately, you can generalize: if you have more training data than a certain threshold, the probability that you perform badly is small. However, these theoretical bounds are often not taken into account in practice because, for example, we may not be able to collect more data to ensure that the bounds are satisfied.

Surely the representations/patterns learnt in the training data will be found too in the test data.

This is possibly the wrong assumption and the reason why you are confused. You may assume that both your training data and test data come from the same distribution $p(x, y)$, but it does not necessarily mean that they have the same patterns. For example, I can sample e.g. 13 numbers from a Gaussian $N(0, 1)$, the first 10 numbers could be very close to $0$ and the last $3$ could be close to $1$. If you split this data so that your training data contains different patterns than the test data, then it is not guaranteed that you will perform well also on the test data.
Finally, note that, in supervised learning, our ultimate goal when we fit models to labeled data is to learn a function (or a probability distribution over functions), where we often assume that both the training and test data are input-output pairs from our unknown target function, i.e. $y_i = f(x_i)$, where $(x_i, y_i) \in D$ (where $D$ is your labelled dataset), and $f$ is the unknown target function (i.e. the function we want to compute with our model), so, if our model performs well on the training data but not on the test data and we assume that both training and test data come from the same function $f$, there is no way that our model is computing our target function $f$ if it performs badly on the test data.


Create optimizer object using the tf.keras.optimizers.get function I am trying to have the type of optimizer as a variable in the hyperparameter tuning phase. For that reason I am trying to use the tf.keras.optimizer.get function defined in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/optimizers.py.

However, I am getting different kind of errors depending on how I am calling the function and I cannot figure out the problem. For example, when I run the following command:

opt = tf.keras.optimizers.get({"class_name": hyperparams['optimizer'],
                                      "learning_rate": hyperparams['learning_rate']})

I get an error saying "ValueError: Improper config format: {'class_name': 'adagrad', 'learning_rate': 0.01} ".

Does anybody know what is the proper way of calling this function with a configuration dictionary if I want to have a configurable optimizer type and learning rate?

In order to have a configurable optimizer and configurable hyperparameters for it you need to make the function call in the following way:

opt = tf.keras.optimizers.get({"class_name": hyperparams['optimizer'],
                               "config": {"learning_rate": hyperparams['learning_rate']}})

where class_name is the name of the optimizer you want to use and the config dictionary contains the hyperparameters and their values for the given optimizer.


An AI that asks questions? Typical AI these days are question-answering machines. For example, Siri, Alexa and Google Home. But it is always the human asking the questions and the AI answering.

Are there any good examples of an AI that is curious and asks questions of its own accord?

You are referring to 'proactive AI' as opposed to 'reactive AI' like Alexa, Cortana, Siri, Bixby, Google Assistant, and others.  There hasn't been much progress in this area of AI.  Google's recent demonstration of Duplex addresses this to some extent.  Some chatbots are proactive. Genesys provides such capability.  Check out their video

Azure's bot service has a page on how to implement proactivity and there is another video that walks through the whole process: Learn to build Proactive Bot in 30 Minutes.


Are there any pathfinding algorithms that take customized rules into account when determining the shortest path? I need a pathfinding algorithm that considers the history of visited nodes and varies its path depending on some rules (like already visited). Are there good approaches serving this purpose?

To be more specific: Let's say I have a graph representing a map. I want to find a Route from B to D: Once I am in D following B->C->D, I want to calculate a new path, let's say to A. The shortest path would be D->C->B->A. But I want a path with unvisited nodes even if it's longer than the shortest possible path. The new path should be the shortest among all possible paths according to the rules.
Another example is the game "snake". Seeing the grid as a graph I cannot visit already visited nodes as fas as the corpus (of the snake) is in that node (or I have visited the node t time steps ago)

Maybe the problem is too specific and I have to implement some basic pathfinding algorithm in a wider algorithm.

Yes, this is easily solved using the A* algorithm. Once your agent has visited a particular node, increase the cost of that node to infinity and recalculate the path.


What are examples of daily life applications that use simulated annealing? In AIMA, 3rd Edition on Page 125, Simulated Annealing is described as:

  Hill-climbing algorithm that never makes “downhill” moves toward states with lower value (or higher cost) is guaranteed to be incomplete, because it can get stuck on a local maximum. In contrast, a purely random walk—that is, moving to a successor chosen uniformly at random from the set of successors—is complete but extremely inefficient. Therefore, it seems reasonable to try to combine hill climbing with a random walk in some way that yields both efficiency and completeness. Simulated annealing is such an algorithm. In metallurgy, annealing is the process used to temper or harden metals and glass by heating them to a high temperature and then gradually cooling them, thus allowing the material to reach a lowenergy crystalline state. To explain simulated annealing, we switch our point of view from hill climbing to gradient descent (i.e., minimizing cost) and imagine the task of getting a
  ping-pong ball into the deepest crevice in a bumpy surface. If we just let the ball roll, it will come to rest at a local minimum. If we shake the surface, we can bounce the ball out of the local minimum. The trick is to shake just hard enough to bounce the ball out of local minima but not hard enough to dislodge it from the global minimum. The simulated-annealing solution is to start by shaking hard (i.e., at a high temperature) and then gradually reduce the intensity of the shaking (i.e., lower the temperature)

I know its about its example, but I just want more examples where Stimulated Annealing used in daily life

Examples of simulated annealing in the 2010s

These are a few examples.

Optimised simulated annealing for Ising spin glasses, 2015, S.V. Isakov et. al.
A parallel simulated annealing method for the vehicle routing problem with simultaneous pickup–delivery and time windows, 2014, Chao Wang et. al.
Feature-based tuning of simulated annealing applied to the curriculum-based course timetabling problem, 2015, Ruggero Bellio et. al.

Analysis of Quote in the Question

The quote in the question is from page 125 of Artificial Intelligence: A Modern Approach, Stuart Russell and Peter Norvig, 2010, 3rd Edition.

The quote begins with the statement, "A hill-climbing algorithm that never makes 'downhill' moves toward states with lower value (or higher cost) is guaranteed to be incomplete," which is not strictly true. The stated guarantee has at least two important exceptions.

The class of hills being climbed have only one peak
The search is executed using parallel computing architecture scaled such that search granularity includes no more than one peak per processing unit

Describing simulated annealing using the analogy of combining inverted hill climbing and random walk, as Stuart and Norvig explains it, is not as clear and technically accurate as the original analogy. It is often better to find original sources of ideas and read them than to read simplified explanations. The original ideas may be just as easy or easier to understand than the textbook paraphrases (which is why the best textbooks simply quote the originating authors).

Source of the Term

The original algorithm termed simulated annealing is introduced in Optimization by Simulated Annealing, Kirkpatrick et. al.1, which may not qualify as one one explicitly employed by AI researchers or practitioners on a daily basis. Simulating the atomic motion of metals during annealing is the first of several approaches to injecting pseudo-random perturbations into convergence mechanisms. An approximation of a ${\chi}^2$ distribution with $k$ degrees of freedom is a more progressive injection source.

These approaches share the common goal of avoiding the mistaking of a local minimum in loss surface as its global minimum. Such mistaken conclusion causes a failure to achieve an optimal learning state according to the criteria the loss function represents.

As can be seen in the below excerpts, the original article is a good read and sets a foundation of understanding. Such a foundation will assist in understanding all methods used avoid local minima using stochastic injection.

  Iterative improvement consists of a search in this coordinate space for
  rearrangement steps which lead downhill. Since this search usually gets stuck in a local but not a global optimum, it is customary to carry out the process several times, starting from different randomly generated configurations, and save the best result.

  ...

  Metropolis et al 2, in the earliest days of scientific computing, introduced a simple algorithm that can be used to provide an efficient simulation of a collection of atoms in equilibrium at a given temperature.

  ...

  Random numbers uniformly distributed in the interval (0,l) are a convenient means of implementing the random part of the algorithm.

  ...

  Using the cost function in place of the energy and defining configurations by a set of parameters {$x_i$}, it is straightforward with the Metropolis procedure to generate a population of configurations of a given optimization problem at some effective temperature. This temperature is simply a control parameter in the same units as the cost function. The simulated annealing process consists of first "melting" the system being optimized at a high effective temperature, then lowering the temperature by slow stages until the system "freezes" and no further changes occur.

  ...

  Simulated annealing with Z-moves improved the random routing by 57 percent, averaging results for both x and y links.

References

[1] Optimization by Simulated Annealing, Science S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Science New Series, Vol. 220, No. 4598, May 1983, pp. 671-680

[2] N. Metropolis, A. Rosenbluth, M. Rosenbluth., A. Teller. E. Teller, J. Chem. Phys. 21. 1087 110511