Evolving network in game So I wrote simple feed forward neural network that plays tic-tac-toe:

9 neurons in input layers: 1 - my sign, -1 - opponent's sign, 0 - empty;
9 neurons in hidden layer: value calculated using Relu;
9 neurons in output layer: value calculated using softmax;

I am using evolutionary approach: 100 individuals play against each other (all-play-all). Top 10 best are selected to mutate and reproduce into the next generation. The fitness score calculated: +1 for correct move (it's possible to place your sing on already occupied tile), +9 for victory, -9 for a defeat.

What I notice is that the network's fitness keeps climbing up and falling down again. It seems that my current approach only evolves certain patterns on placing signs on the board and once random mutation interrupts current pattern new one emerges. My network goes in circles without ever evolving actual strategy. I suspect solution for this would be to pit network against tic-tac-toe AI, but is there any way to evolve actual strategy just by making it to playing against itself?

What I notice is that the network's fitness keeps climbing up and falling down again. It seems that my current approach only evolves certain patterns on placing signs on the board and once random mutation interrupts current pattern new one emerges. My network goes in circles without ever evolving actual strategy. I suspect solution for this would be to pit network against tic-tac-toe AI, but is there any way to evolve actual strategy just by making it to playing against itself?

The likely cause of this phenomenon is that your fitness function involves evaluating the fitness of an agent by letting it play lots of other agents (the entire population), many of which are likely very poor agents.

Because Tic-Tac-Toe is such a simple game, we know that optimal play from both sides leads to a draw. Suppose we have a population of the following three strategies:

$\pi_1$: an optimal player
$\pi_2$: a sub-optimal player
$\pi_3$: a different sub-optimal player

There can easily be situations where the optimal player $\pi_1$ gets a draw against both of the sub-optimal players (if they're not extremely bad), because right from the get-go an optimal player will play "safe" enough such that it can still guarantee a draw against an optimal opponent, which may not be the fastest way to win against a sub-optimal player.

In the same situation, the sub-optimal player $\pi_2$ for example may be able to consistently win against a slightly worse sub-optimal player $\pi_3$. In this example situation, your fitness function ranks the agents $f(\pi_2) > f(\pi_1) > f(\pi_3)$, which is wrong.

As you already suggested yourself, the most straightforward way to address this problem would be to simply evaluate the fitness of strategies by having them play against an optimal minimax-agent, rather than a population of many strategies (including poor ones).

If you really want to use only evolving, no tree search, you'll have to find a way to fix the fitness function such that the problem described above can no longer occur. One way you could try to do that (not 100% sure it would work, but imagine that it might) would be to set up some larger tournament bracket where agents progress through the brackets if they're able to beat others which they were paired up with. Getting further in the tournament would then increase the fitness of an agent. Very bad sub-optimal players should not be able to progress far in the tournament if they get beaten by other (sub-)optimal agents, but sub-optimal agents which manage to advance still shouldn't be able to get more than a draw against an optimal agent. Some things to keep in mind with this idea:

You'll likely want to repeat the entire tournament many times with different, randomized initial pairings in the bracket, and compute fitness based on average (or maybe median or max) ranking across all repetitions of the tournament. This would be to filter out members of the population getting particularly lucky or unlucky with the opponents they encounter in the brackets.
You'll have to think about which actions the agents select in such a tournament. Do they deterministically play the action given by the $\arg\max$ of the softmax outputs (in which case you could also just use the linear outputs rather than softmax outputs, since the softmax function does not change ranking of outputs). Or do they nondeterministically sample actions according to the softmax distribution? Sampling actions according to the softmax distribution seems attractive because it leads to more variance in game state situations encountered, and it is important to be robust and be capable of handling many different game states. On the other hand, it does introduce noise and may make an optimal agent accidentally lose. I suppose I would lead towards deterministic play with the $\arg\max$ over the softmax outputs. In a large tournament, there will still be sufficient variety due to encountering many different opponents.
You'll have to put thought into handling the situation where two agents may infinitely keep drawing against each other. Which agent advances? If this happens, I think I would gradually shift from deterministcally playing $\arg\max$ actions to playing according to the softmax distribution. This guarantees that someone eventually loses. Averaging results over multiple different games like this, plus averaging over multiple different repetitions of the complete tournament, should yield accurate results.
You'll want to think about how to handle illegal actions. You describe including punishments for illegal actions in the fitness function. This introduces extra noise / variance in the fitness function which is already difficult enough to get right to begin with, so I wouldn't do this. I'd recommend having a manual post-processing step in which you manually set probabilities of illegal actions being played to $0$, and normalizing to make sure the probabilities over the remaining actions add up to $1$ again. This is also what's done (on a much larger scale) by DeepMind in their state-of-the-art Go/Chess/Shogi agents for example. If you really don't want to do this, if you really want evolved strategies to automatically learn not to ever have high outputs for illegal actions, I would recommend immediately making them lose the game if they do suggest an illegal action. That way you still have a "clean" fitness function based only on wins/draws/losses.


Which ANN structure to use? Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$  has $d$ features.
I want to design a ANN so that the cost function below is minimized (the sum of
square of pairwise differences between model outputs)  and the given constraint is satisfied, where $w$ is ANN model parameter vector.

Question: what kind of ANN is suitable for this purpose?

If I understand your query correctly, you want to create a latent space that groups similar objects. You should then probably look for Siamese networks. However, your loss function will need another term to increase dissimilarity between different labels. Otherwise, as pointed out by Mike NZ, the net would collapse(yes, it is possible). Perhaps this will give some insights.
Note that the above method is not completely unsupervised. There are, in fact, a few claims of unsupervised classifications via clustering, although your objective function would look very different. You could go through this paper(called SCAN) for more details.
Hope it helps.
[Edit]
If you want a (lower-dimensional)representation of objects themselves, browsing through this could help. For a complex problem, linear reductions like PCA, although helpful, aren't probably what you're looking for. Here you can try training autoencoders. The loss function would work, along with some regularization term.


What are most commons methods to measure improvement rate in a meta-heuristic? When I run a meta-heuristics, like a Genetic Algorithm or a Simulated Annealing, I want to have a termination criterion that stops the algorithms when there is not any significant fitness improvement.
What are good methods for that?
I tried something like
$$improvement=\frac{fit(Solution_{new})}{fit(Solution_{old})}$$
and
$$improvement={fit(Solution_{new})}-{fit(Solution_{old})}$$
Both options don't seem to be good, because as the old solutions get better and newer solutions even if they are good don't improve so much compare to the old.

You can use one of your suggested methods to calculate the relative improvement, but you need also to define a threshold value $\epsilon$ that determines when a relative improvement is negligible, and so you can terminate the algorithm. To be more concrete, you could terminate the genetic algorithm when, for example, the following condition is met
$$
|f(x_{t+1}) - f(x_{t})| < \epsilon, \tag{1}\label{1}
$$
where

$f$ is the fitness function
$t$ is the iteration number of your iterative algorithm
$x_t$ is a solution at iteration $t$
$|\cdot|$ is the absolute value
$\epsilon$ is the threshold value (a hyper-parameter of your algorithm), which is typically a small number (e.g. $10^{-6}$), but this also depends on the magnitude of the fitness of the solutions

This stopping criterion in \ref{1} (sometimes known as the absolute error) is not the only possible one. For example, you can stop the genetic algorithm when it has run for a certain maximum number of iterations. The stopping criterion (or criteria, i.e. you can use more than one stopping criteria) that you choose probably depends on what you want to achieve and this is not just specific to genetic algorithms, but, in general, this applies to any iterative numerical algorithm.
The MathWorks article How the Genetic Algorithm Works enumerates several stopping criteria for genetic algorithms.
If you want to know more about the topic, maybe take a look at this paper On Stopping Criteria for Genetic Algorithms (2004) by Martín Safe et al.


How to deal with different actions for different states of the environment? I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments, I came across the Blackjack-v0 environment, which is a basic implementation of the game where the state is the hand count of the player and the dealer and if the player has a useable ace. The actions are only hit or stand and the possible rewards 1 if the player wins, -1 if the player loses, and 0 when it is a draw.
So, that got me thinking what a more realistic environment/model for this game would look like, taking into account the current balance and other factors and has multiple actions like betting 1-10€ and hit or stand.
This brings me to my actual question:

As far as I understand, in neural networks (and I do not very well yet, I guess) the input will be the state and the output the possible actions and how good the network thinks they are/will be. But now there are two different action spaces, which apply to different states of the game (betting or playing), so some of the actions are useless. How would be the right way to approach this scenario?

I'm guessing one answer would be to give some kind of negative reward if the network guesses a useless action, but, in this case, I think the reward should be the actual stake (negative reward) and the actual win if any. Therefore, this would cause some bias in how the game proceeds as it should start with some amount of balance and end if the balance is 0 or after a specified amount of rounds.
Limiting timesteps wouldn't be an option either, I guess, because it should be limited to rounds, so, for example, it won't end after a betting step.
Therefore, for a useless step, the reward would be 0 and the state would stay the same, but, for the neural network, it doesn't matter how many useless steps it takes because it'll make no difference to the actual outcome.
Corollary question:

Should be split up into two neural networks? One for betting and one for playing?

There are so many reinforcement learning algorithms. I recommend you this articles:

Introduction to Various Reinforcement Learning Algorithms. Part
1
Introduction to Various Reinforcement Learning Algorithms.    Part
2

There are several approaches. A NN can get state and action as an input  and expected discounted reward as an output. But you can just not pick an action if this action isn't available and there isn't necessity to give it negative or zero reward just don't learn your net to change it behavior for these actions because they are neither good nor bad.


Isn't deep fake detection bound to fail? Deep fakes are a growing concern: the ability to credibly alter a video may have great (negative) impacts on our society. It is so much of a concern, that the biggest tech companies launched a specific challenge: https://deepfakedetectionchallenge.ai/.

However, from what I understand, most deep fake generation techniques rely on the use of adversarial models. One model generates a new image, while another model tries to detect if the image is doctored or not. Both models "learn" from being confronted with the other.

That being said, if a good deep fake detection model emerges (from the previous challenge, or not), wouldn't it be rendered useless almost instantly by learning from it in an adversarial setting?

Not necessarily it depends on the function of the problem space for both the GANs.

A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be a better pitcher because the threshold of reaction time.

We don't yet know if a maximum threshold on realistic fake image generation exists or if a threshold on detection exists.

As both reach near perfect accuracy it could be that the amount of nodes needed to detect a nearly perfect generated image from real image is more neurons than atoms in the universe, or conversely the amount of nodes needed to generated a nearly perfect image could reach impossible port potions we won't know until we continue to make better and better networks that close in on the boundary of generation and detection of real vs fake images from a neural network.

Edit:
Let's imagine this problem one adversary edits an image with a colored line of pixels the goal hide the line by editing the image, the student is responsible with finding the line after the adversary changes the image. The problem can become infinitely difficult change all pixels to be the color of the line. The line is impossible to find the adversary always wins, if it finds this solution meaning it is in its reachable problem space based on its hardware capabilities and its learning model.

Deep fake detection is not bound to fail because the limit on the effectiveness of a generative model may have a steeper limit than a discriminator at near optimal performance. I have not seen any paper about this specifically and in fact I believe the discriminator has a more difficult job in most cases I just disagree with deciding at this moment that the detectors are doomed. The combination of creating real images motion and sound perfectly in sink is not a trivial problem, in some scenarios it is basically impossible.


What are some applications where tree models perform better than neural networks? Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, Gradient Boosting are favorable over neural networks when working with low-dimensional data and easy interpretable features (usually simple tabular data with numeric, ordinal or categorical features).
Whereas when working with everything high-dimensional like images, text, time series or other data with non-trivial features, I would recommend neural networks.
Of course there might be exceptions and the future may prove me wrong.


Is there a strong argument that survival instinct is a prerequisite for creating an AGI? This question stems from quite a few "informal" sources. Movies like 2001, A Space Odyssey and Ex Machina; books like Destination Void (Frank Herbert), and others suggest that general intelligence wants to survive, and even learn the importance for it.

There may be several arguments for survival. What would be the most prominent?

Steve Omohudro wrote a paper called Basic AI Drives that steps through why we would expect an AI with narrow goals to find some basic, general concepts as instrumentally useful for their narrow goals.

For example, an AI designed to maximize stock market returns but whose design is silent on the importance of continuing to survive would realize that its continued survival is a key component of maximizing stock market returns, and thus take actions to keep itself operational.

In general, we should be skeptical of 'anthropomorphizing' AI and other code, but it seems like there are reasons to expect this beyond "well, humans behave this way, so it must be how all intelligence behaves."


Modifying LSTM to include forecast I am looking at LSTM example here

However, I am not sure how to modify the setup if I have forecast available (assuming perfect forecast) for TEMP: Temperature and PRES: Pressure at time t.

i.e. pollution_t = fn(TEMP_t, PRES_t, TEMP_t_1, PRES_t_1, ..., othervariables and lagged values)

_t represents the perfect forecast available

_t_1 represents the variable values at previous time steps

Basically I am looking for something of the following setup:

You need a bidirectional LSTM. The temporal direction on the original LSTM and on GRU type cell connectivity is forward in time. Fortunately, the bidirectional class is available in Keras.

from keras.models import Sequential
from keras.layers import LSTM
from keras.layers import TimeDistributed
from keras.layers import Bidirectional
...
model = Sequential()
model.add(Embedding(max_features, 128, input_length=maxlen))
model.add(Bidirectional(LSTM(64)))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

Now the network has causality from both history and forecast. This is only useful if the forecasts reduce entropy, which means they have to be known effective forecasts that when fed to earlier cells (in the time domain) will enhance accuracy and confidence metrics.

When the loss function (for its derivative) is provided, it must return the error at $t = t(i)$, where $i$ is the index of the last sample in the sequence before the values become forecasts. This is the main distinction when the sequence includes both history and forecast.


How to create a fully connected(matrix) layer with vector input I am trying to replace last fully connected layer of size 4096/2048 with a matrix of size 100x300 with previous fc layer output of 2048.

I've tried

2D convolution - to map from 2048 --> 100x300 (Which is not realizable)
Intermediate projections :
    2048 --> 100
    [100x1] X [1x300] --> [100x300] (possible but complicated)

I am looking for a simple and effective solution with least linear transformations.

You can use tf.reshape() method (tensorflow doc) to reshape (2048) dimensional tensor to (100,300). Here's one way to do this:

input1 = tf.reshape(input1, [100,300], name="reshaped_tensor")

If you're not using TensorFlow but using Numpy, here's an implementation:

input1 = np.array(input1)
input1 = np.reshape(input1, (100,300))

Note: You might want to follow up this layer with tf.nn.conv2d layers to "densify" the sparse matrix/values obtained from the above step.


What activation function does the human brain use? Does the human brain use a specific activation function? I've tried doing some research, and as it's a treshold for whether the signal is sent through a neuron or not, it sounds a lot like ReLU. However, I can't find a single article confirming this. Or is it more like a step function (it sends 1 if it's above the treshold, instead of the input value).

The thing you were reading about is known as the action potential. It is a mechanism that governs how information flows within a neuron.

It works like this: Neurons have an electrical potential, which is a voltage difference inside and outside the cell. They also have a default resting potential, and an activation potential. The neuron tends to move towards the resting potential if it is left alone, but incoming electric activations from dendrites can shift its electric potential.

If the neuron reaches a certain threshold in electric potential (the activation potential), the entire neuron and its connecting axons goes through a chain reaction of ionic exchange inside/outside the cell that results in a "wave of propagation" through the axon.

TL;DR: Once a neuron reaches a certain activation potential, it electrically discharges. But if the electric potential of the neuron doesn't reach that value then the neuron does not activate.

  Does the human brain use a specific activation function?

IIRC neurons in different parts of the brain behave a bit differently, and the way this question is phrased sounds as if you are asking if there is a specific implementation of neuronal activation (as opposed to us modelling it).

But in general behave relatively similar to each other (Neurons communicate with each other via neurochemicals, information propagates inside a neuron via a mechanism known as the action potential...) But the details and the differences they cause could be significant.

There are various biological neuron models, but the Hodgkin-Huxley Model is the most notable.

Also note that a general description of neurons don't give you a general description of neuronal dynamics a la cognition (understanding a tree doesn't give you complete understanding of a forest)

But, the method of which information propagates inside a neuron is in general quite well understood as sodium / potassium ionic exchange.

  It (activation potential) sounds a lot like ReLU...

It's only like ReLU in the sense that they require a threshold before anything happens. But ReLU can have variable output while neurons are all-or-nothing.

Also ReLU (and other activation functions in general) are differentiable with respect to input space. This is very important for backprop.

This is a ReLU function, with the X-axis being input value and Y-axis being output value.

And this is the action potential with the X-axis being time, and Y being output value.


Why is the target called "target" in Monte Carlo and TD learning if it is not the true target? I was going through Sutton's book and, using sample-based learning for estimating the expectations, we have this formula
$$
\text{new estimate} = \text{old estimate} + \alpha(\text{target} - \text{old estimate})
$$
What I don't quite understand is why it's called the target, because since it's the sample, it’s not the actual target value, so why are we moving towards a wrong value?

It is our "current" target. We assume that the value we get now is at least a closer approximation to the "true" target.
We're not so much moving towards a wrong value as we are moving away from a more wrong value.
Of course, it is all base on random trials, so saying anything definite (such as: "we are guaranteed to improve at each step") is hard to show without working probabilistically. The expectation of the error of the value function (as compared to the true value function) will decrease, that is all we can say.


How do GPUs faciliate the training of a Deep Learning Architecture? I would love to know in detail, how exactly GPUs help, in technical terms, in training the deep learning models.

To my understanding, GPUs help in performing independent tasks simultaneously to improve the speed. For example, in calculation of the output through CNN, all the additions are done simultaneously, and hence improves the speed.

But, what exactly happens in a basic neural network or in a LSTM type complex models in regard with GPU.

GPUs are able to execute a huge amount of similar and simple instructions (floating point operations like addition and multiplication) in parallel. In contrast to a CPU which is able to execute a few complex tasks sequentially very quick. Therefore GPUs are very good at doing vector & matrix operations.

If you look at the operations performed inside a single basic NN layer you will see that most of the operations are matrix-vector multiplications:

$$x_{i+1} = \sigma(W_ix_i + b_i)$$

where $x_i$ is the input vector, $W_i$ the matrix of weights, $b_i$ the bias vector in the $i^{th}$ layer, $x_{i+1}$ the output vector and $\sigma(\cdot)$ the elemtwise non-linear activation. The computational complexity here is governed by the matrix-vector multiplication. If you look at the architecture of a LSTM cell you will notice that inside of it are multiple such operations.

Being able to execute the matrix-vector operations quickly and efficiently in parallel will reduce execution time, this is where GPUs excel CPUs.


Can the addition of dropout in a non-overfitting neural network increase accuracy? According to Wikipedia

  Dropout is a regularization technique for reducing overfitting in neural networks

My neural network is simple enough and does not overfit.

Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?

Can the addition of dropout, in a non-overfitting neural network, increase accuracy?

Yes, maybe.

  Even if I increase the complexity of the neural network?

Yes, maybe.

As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.

However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as bagging. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.


How is the F1 score calculated in a question-answering system? I have an NLP model for answer-extraction. So, basically, I have a paragraph and a question as input, and my model extracts the span of the paragraph that corresponds to the answer to the question.
I need to know how to compute the F1 score for such models. It is the standard metric (along with Exact Match) used in the literature to evaluate question-answering systems.

In QA, it's computed over the individual words in the prediction against those in the True Answer. The number of shared words between the prediction and the truth is the basis of the F1 score: precision is the ratio of the number of shared words to the total number of words in the prediction, and recall is the ratio of the number of shared words to the total number of words in the ground truth
Source: https://qa.fastforwardlabs.com/no%20answer/null%20threshold/bert/distilbert/exact%20match/f1/robust%20predictions/2020/06/09/Evaluating_BERT_on_SQuAD.html#F1


What could be the cause of the drop in the reward in A3C? The mean episodic reward is generally increasing, but it has spontaneous drops, and I'm not sure of their cause.

The problem has a sparse reward, batch size=2000, entropy_coefficient=0.1, other hyper-parameters are pretty standard.

Has anyone seen this kind of behavior? What could be the cause these drops in the reward(not enough exploration, too sparse rewards, the state not expressive enough, etc.)?

I'll share my understanding so far.
This kind of behavior is actually normal when using on-policy algorithms with a sparse final reward.
Issue stems from the fact that once you get stuck in a behavior policy which does nothing (uses a "do nothing" action, for instance, until timeout), it's quite hard to get out of it, because you keep getting experiences that teach you nothing (no reward signal at all) and keeps you in the same policy.
Possible mitigations:

Encourage more exploration (in A3C, make the entropy loss coefficient bigger) in order to recover more quickly from this type of stationary behavior.
Use an off-policy algorithm with a big enough replay buffer, so that even if you start behaving this way you still use experiences from a "healthy" older policy.

If you stick to a totally on-policy algorithm, making the batch_size bigger might help a little.


Is there a graph neural network algorithm that can deal with a different number of input and output nodes? I am new to graph neural networks and their applications. I have an input graph $G = \{V, E\}$ and an output graph $G' = \{V', E'\}$ where the number of nodes $V$ and $V'$ are different. I am trying to learn the function where $f(G) = G'$ and $V > V'$, thus, the function is mapping many-to-one ($n$ number of nodes map to one). The Graph Convolution Network (GCN) seems to have the same number of nodes in input and output with the function being learnt. Could I utilize the GCN for my task?

I suggest you look into link prediction. I have had good luck with the StellarGraph library. They have several algorithms implemented, including GCN.
Link prediction is a binary classification problem. Given two nodes, $v_i$ and $v_j$, does there exist a link between them? Using a library like StellarGraph will also produce node embeddings while performing link prediction.
For you scenario I'm picturing a three step process:

Link prediction and node embeddings on $G$.
Link prediction and node embeddings on $G'$.
Link prediction reusing existing embeddings where each link is between the two graphs. So each link is a tuple of the form: $(v_i, v_j')$ where $v_i \in V$ and $v_j' \in V'$. If there were no links predicted from $v_i$ to $v_j'$ then that might suggest to remove $v_j'$.

In the link prediction tasks that I've outlined, you can use GCN with StellarGraph. So there should be no problems in terms of the number of nodes.


What's a good neural network for this problem? I am very new to the field of AI so please bear with me.
Say there is a dice with three sides, -1,0 and 1, and I want to predict which side it lands on (so only one output is needed I guess). The input variables are numerous but not that many, maybe 7-10.
These input variables are certain formulae that involve calculations to do with wind, time, angle, momentum etc, and each formula returns which side it thinks the dice will like roll. Let's say that intuitively, by looking at these variables, I can make a very good guess at which side the dice lands on. If for example 6 out of 7 input variables say it likely that the dice will land on 1 but the 7th input suggests that it will land on 0, I would guess it lands on 1. As a human, I'm essentially consulting these inputs as a kind of "brains trust", and I act as a judge to make the final decision based on the brains trust. Of course in that example, my logic as a judge was simply majority rules, but what if some other more complicated non-linear method of judging was needed?
I essentially want my neural network to take this role as a judge. I have read that feedforward nns have limitations regarding control flow and loops, so I'm not sure if that structure will be appropriate. I'm not sure if recurrent nn will be appropriate either as I don't care what the previous inputs were.
Thanks

A simple feed-forward neural network with at least one hidden layer would suffice in your problem, and can deal with arbitrary non-linear relationships between input and output. If you expect relationships to be highly non-linear then additional layers might be required, but from your description of the problem, I would be surprised if you needed more than few layers, and a relatively small network.
However, I note that:

The input variables are numerous but not that many, maybe 7-10.

This gives you $3^{10} = 59049$ possible inputs. That's not much in terms of amount of data needed for ML statistical models. Assuming that even the best predictions are still probabilistic, then you may only need a million or so examples to create a reasonably accurate lookup table, not needing a neural network at all.
The strength of a neural network is to be able to generalise well from less examples than that. Of course, this is not perfect, but it would be able to do things such as notice if inputs 1,2 and 3 agree then that is always the most likely answer. If that turns out to be true (and not an accident of having low numbers of samples), then the NN could learn that useful pattern using far less data than a table-based approach.

I have read that feedforward nns have limitations regarding control flow and loops, so I'm not sure if that structure will be appropriate.

This is true, but does not impact your situation, because there is no control flow or loops involved. You have described a simple function. Whilst you or I might inspect the data and look backwards and forwards across it before coming to a decision, a neural network approximating a function does not need to do that, and in simple cases there is usually no benefit to doing so - a statistical summary of the correct mapping from input to output is more than sufficient and likely the best that can be done.

I'm not sure if recurrent nn will be appropriate either as I don't care what the previous inputs were.

As all your inputs represent the same kind of thing, you could implement as a RNN with a single input, -1, 0 or +1, always feeding in the predictions by type in the same order. It might resemble how you are thinking about the problem as a human (at least a better analogy than the direct statistical match in a feed-forward network), especially if you implemented a variant of attention. However, I don't think there would be any benefit to that in improved accuracy, and it would be a significant challenge to build that if you are new to AI.


What are the advantages and disadvantages of using LISP for constraint satisfaction in 3D space We are currently working on developing a 3D modeling software that allows designers to set spatial constraints to models. The computer then should generate a 3D mesh conforming to these constraints.

Why should or shouldn't we use Lisp for the constraint satisfaction part? Will Prolog environment be any better? Or should we stick to C/C++ libraries?

One requirement we have is that we want to use the Unity Game Engine as it has a lot of 3D tools built in

This is actually a question which will only receive opinion based answer's. A question you should ask yourself is if the constraint part is really that complex that it is worth to use a different programming language. Unity itself offers a C# API [1] and I would therefore stick with that.

[1] https://unity3d.com/programming-in-unity


Are there any downsides of using a fixed seed for a neural network's weight initialization? For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower error
I'm specifically concerned about supervised learning on point cloud data, but curious about whether it matters in general whenever you use a neural network.

When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.


Can I export a trained machine learning model so that others case use it? From what I know, AI/ML uses a large amount of data to train an algorithm to solve problems. But since it’s an algorithm, I was wondering if it's possible to export it. If I trained an AI with R, could I export a mathematical algorithm that could be imported by other users to use in their application, whether it’s written in R or another language?

So it’s like I’ve discovered a secret message decoding method. I don’t need to share the whole program for others to decode it. I just need to tell them the steps (algorithm) to decode it, and they can implement it in whatever application they want.

If the 'algorithm' you're talking about is a neural network, then you can distribute the learned parameters/weights to anyone who wants to use it. This is how neural nets are normally 'exported': without all of the training data used to create them. Actually, this is done with many kinds of models (parameterized ones).

In order to 'decode' the model, users would only have to know its structure. In the case of a neural network, they'd need to know the size of each layer, what activation functions were used, etc.

This is not possible with every type of ML model, however. Specifically, non-parametric models and 'lazy' models make use of training data at inference time. They wouldn't be useful without their training data. Classifying an input by finding its k nearest neighbors, for example, would require having the training data.


Does gradient descent in deep learning assume a smooth fitness landscape? I've come across the concept of fitness landscape before and, in my understanding, a smooth fitness landscape is one where the algorithm can converge on the global optimum through incremental movements or iterations across the landscape.
My question is: Does deep learning assume that the fitness landscape on which the gradient descent occurs is a smooth one? If so, is it a valid assumption?
Most of the graphical representations I have seen of gradient descent show a smooth landscape.
This Wikipedia page describes the fitness landscape.

I'm going to take the fitness landscape to be the graph of the loss function, $\mathcal{G} = \{\left(\theta, L(\theta)\right) : \theta \in \mathbb{R}^n\}$, where $\theta$ parameterises the network (i.e. it is the weights and biases) and $L$ is a given loss function; in other words, the surface you would get by plotting the loss function against its parameters.
We always assume the loss function is differentiable in order to do backpropagation, which means at the very least the loss function is smooth enough to be continuous, but in principle it may not be infinitely differentiable1.
You talk about using gradient descent to find the global minimiser. In general this is not possible: many functions have local minimisers which are not global minimisers. For an example, you could plot $y = x^2 \sin(1/x^2)$: of course the situation is similar, if harder to visualise, in higher dimensions. A certain class of functions known as convex functions satisfy the property that any local minimiser is a global minimiser. Unfortunately, the loss function of a neural network is rarely convex.
For some interesting pictures, see Visualizing the Loss Landscape of Neural Nets by Li et al.

1 For a more detailed discussion on continuity and differentiability, any good text on mathematical analysis will do, for example Rudin's Principles of Mathematical Analysis. In general, any function $f$ that is differentiable on some interval is also continuous, but it need not be twice differentiable, i.e. $f''$ need not exist.


How to combine several chatbots into one? I'm in the middle of a project in which I want to generate a TV series script (characters answering to each other, scene by scene) using SOTA models, and I need some guidance to simplify my architecture.
My current intuition is as follows: for a given character C1, I have pairs of sentences from the original scripts where C1 answers other characters, for example, C2 (C2->C1). These are used to fine-tune a data-driven chatbot. At inference time, the different chatbots simply answer each other, and, hopefully, the conversation will have some sense.
This is however unpractical and will be kind of a mess with many characters, especially if I use heavy models.
Is there an architecture out there that could be used for conversational purposes, which could be trained only once with the whole dataset while separating the different characters?
I'm open to any ideas!

Define features that describe C1 and C2 (like their ids) and add these features to inputs of your model. Thus your only model will be able to generate the next line for any pair of (C1, C2).


Weighing connections between two users using machine learning Given this data set:

User 1: {'Artist': 1, 'Public Figure': 9, 'Film Director': 1, 'Education': 1, 'Musician/Band': 4, 'Musician': 1, 'Community': 1, 'Author': 4, 'Politician': 1, 'TV Show': 2, 'Entrepreneur': 1, 'Journalist': 4, 'Product/Service': 1, 'Defense Company': 1, 'Computer Company': 2, 'Nonprofit Organization': 3, 'Computers & Internet Website': 4, 'Media/News Company': 3, 'Podcast': 1, 'News & Media Website': 2, 'Charity Organization': 1, 'Government Organization': 1, 'Magazine': 1}
User 2: {'Nonprofit Organization': 1, 'Movie': 2, 'Musician/Band': 22, 'Public Figure': 2, 'Entrepreneur': 2, 'TV Show': 2, 'Medical Company': 1, 'American Restaurant': 1, 'Sports & Recreation': 1, 'Media/News Company': 2, 'Sports Team': 1, 'Artist': 3, 'Musician': 1, 'College & University': 1, 'Athlete': 1, 'Music': 1, 'App Page': 1, 'Comedian': 1, 'Interest': 2, 'Product/Service': 1, 'Book Series': 1

I believe I can draw conclusions that the strength of the connection is high or low based on the fact that they like or dislike the same categories of topics. What machine learning technique would be best to apply to these connection types, if I had, say, 500 users with categories and weights for each?

I would like to automatically apply varying weights, if most people have Musician/Band or like Movies, that should be less interesting than when people like "Defense Company" like in User 1.

The question might be improved so that the answer can be more specific by defining two things more rigorously.

Strength of the connection
What the automation must infer from that value

Learning, Correlation, Projection, and Maximizing Profitability

Guessing from the comments, it appears that the goal is to profile the user for business purposes.  Two common applications of user profiling are

To present buying options or
To suggest friends in a social network.

Although it may appear that a learning program would serve both of these applications, they are actually two very different problems.

One is to optimize e-commerce profitability through increased sales per session.  The other is to build a social network where there are some similarities and perhaps a few deliberate differences in the pairing of user profiles from which the friend suggestion comes.

The first of the two has to do with the projecting of the potential of a sale by matching the users and relying on the assumption that two people with like interests are more likely to mirror one another in purchasing patterns to some degree that could be projected with statistical analysis.

But approaching the project by immediately jumping to learning algorithms may not produce the best results after a month or a year of development.

One Reasonable Approach for Some Problems of This Type

Depending on the definitions of STRENGTH and AUTOMATICALLY in the question, it may be a simple application of a probability matrix or hypercube that will create a functional application of user profiling based on interests. To create a correlation between users via an ordered list of interests to place in one of the cells of the matrix or hypercube, the Spearman's Rank Order Correlation is a generalized yet effective method for such applications.

Learning Usually Begins With a Guess

First, you must determine what is (preferably in precise mathematical terms) what is a favorable product presentation or favorable potential friend introduction.  After that is determined, one can develop a probability formula to project what would be favorable based on the correlation matrix or hypercube.  Bayes's Theorem may be quite applicable in the development of a probability formula from the constraints and relationships of the application.

Learning Added

Once improved favorablility gained through the application of the correlations  can be demonstrated, it would then be wise to apply a learning algorithm.  The sequence is thus.

Results gained by random selection of options to present the user
Results gained by using the correlation matrix or hypercube
Results gained by iteratively converging on an optimal presentation
Tuning the iterative convergence track change and yet not oscillate

A multi-tier, back propagating Hopfield neural net could be one option for varying parameters in the use of the correlations (or the choice of correlations, since Spearman's is only one correlation model for ordered discrete interests).  Neural networks are only one type of convergent approach too.  Modelling with as a system of differential equations and using extensions of the Taylor Series to evaluate with a defined independent input vector is another.

To see how simple convergence can be and how primitive what appears to be a learning algorithm can become, think about the divide and average method of iteratively determining a square root.


Can most of the basic machine learning models be easily represented as simple neural network architectures? I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. In chapter 1.2.1 Single Computational Layer: The Perceptron, the author says the following:

Different choices of activation functions can be used to simulate different types of models used in machine learning, like least-squares regression with numeric targets, the support vector machine, or a logistic regression classifier. Most of the basic machine learning models can be easily represented as simple neural network architectures.

I remember reading something about it being mathematically proven that neural networks can approximate any function, and therefore any machine learning method, or something along these lines. Am I remembering this correctly? Would someone please clarify my thoughts?

I think the author refers to both different choices of activation function and loss. It is explained in more detail in chapter 2. In particular 2.3 is ilustrative of this point.
I don't think there is a relation between this argument and universal approximation theorems, which state that certain classes of neural networks can approximate any function in certain domains, rather than any learning algorithm.


Should we also shuffle the test dataset when training with SGD? When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the training data into batches/sets of different samples from different classes. Should we also shuffle the test dataset?

Short answer
Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you shuffle or not the test or validation data (unless you are computing some metric that depends on the order of the samples), given that you will not be computing any gradient, but just the loss or some metric/measure like the accuracy, which is not sensitive to the order or the samples you use to compute it. However, the specific samples that you use affects the computation of the loss and these quality metrics. So, how you split your original data into training, validation and test datasets affects the computation of the loss and metrics during validation and testing.
Long answer
Let me describe how gradient descent (GD) and stochastic gradient descent (SGD) are used to train machine learning models and, in particular, neural networks.
Gradient descent (GD)
When training ML models with GD, you have a loss (aka cost) function $L(\theta; D)$ (e.g. the cross-entropy or mean squared error) that you are trying to minimize, where $\theta \in \mathbb{R}^m$ is a vector of parameters of your model and $D$ is your labeled training dataset.
To minimize this function using GD, you compute the gradient of your loss function $L(\theta; D)$ with respect to the parameters of your model $\theta$ given the training samples. Let's denote this gradient by $\nabla_\theta L(\theta; D) \in \mathbb{R}^m$. Then we perform a step of gradient descent
$$
\theta \leftarrow \theta - \alpha \nabla_\theta L(\theta; D) \label{1}\tag{1}
$$
Stochastic gradient descent (SGD)
You can also minimize $L$ using stochastic gradient descent, i.e. you compute an approximate (or stochastic) version of $ \nabla_\theta L(\theta; D)$, which we can denote as $\tilde{\nabla}_\theta L(\theta; B) \approx \nabla_\theta L(\theta; D)$, which is typically computed with a subset of $B$ of your training dataset $D$, i.e. $B \subset D$ and $|B| < |D|$. The step of SGD is exactly the same as the step of GD, but we use $\tilde{\nabla}_\theta L(\theta; B)$
$$
\theta \leftarrow \theta - \alpha \tilde{\nabla}_\theta L(\theta; B) \label{2}\tag{2}
$$
If we split $D$ into $k$ subsets (or batches) $B_i$, for $i=1, \dots, k$ (and these subsets usually have the same size, i.e. $|B_i| = |B_j|, \forall i$, apart from one of them, which may contain fewer samples), then the SGD step needs to be performed $k$ times, in order to go through all training samples.
Sampling, shuffling, and convergence
Given that $\tilde{\nabla}_\theta L(\theta; B_i) \approx \nabla_\theta L(\theta; D), \forall i$, it should be clear that the way you split the samples into batches can affect learning (i.e. the updates of the parameters).
For instance, you could consider your dataset $D$ as an ordered sequence/list, and just split it into $k$ sub-sequences. Without shuffling this ordered sequence before splitting, you will always get the same batches, which means that, if there's some information associated with the specific ordering of this sequence, then it may bias the learning process. That's one of the reasons why you may want to shuffle the data.
So, you could uniformly choose samples from $D$ to create your batches $B_i$ (and this is a way of shuffling, in the sense that you will be uniformly building these batches at random), but you can also sample differently and you could also re-use the same samples in different batches (i.e. sampling with replacement). Of course, all these approaches can affect how learning proceeds.
Typically, when analyzing the convergence properties of SGD, you require that your samples are i.i.d. and that the learning rate $\alpha$ satisfies some conditions (the Robbins–Monro conditions). If that's not the case, then SGD may not converge to the correct answer. That's why sampling or shuffling can play an important role in SGD.
Testing and validation
During testing or validation, you are just computing the loss or some metric (like the accuracy) and not a stochastic gradient (i.e. you are not updating the parameters, by definition: you just do it during training). The way you compute the loss or accuracy should not be sensitive to the order of the samples, so shuffling should not affect the computation of the loss or accuracy. For instance, if you use the mean squared error, then you will need to compute
\begin{align}
L(\theta; D_\text{test})
&= \operatorname {MSE} \\
&= {\frac {1}{n}}\sum _{i=1}^{n}(f_\theta(x_i)-{\hat {y_{i}}})^{2}\\
&= {\frac {1}{n}}\sum _{i=1}^{n}(y_{i}-{\hat {y_{i}}})^{2}
\end{align}
where

$f_\theta$ is your ML model
$x_i$ is the $i$th input
$y_{i}$ is the true label for input $x_i$
$\hat {y_{i}}$ is the output of the model
$n$ is the number of samples you use to compute the MSE

This is an average, so it doesn't really matter whether you shuffle or not. Of course, it matters which samples you use though!
Further reading
Here you can find some informal answers to the question "Why do we shuffle the training data while training a neural network?". There are other papers that partially answer this and/or other related questions more formally, such as this or this.


Should illegal moves be excluded from loss calculation in DQN algorithm? I'm implementing DQN algorithm to train my agent to play a turn-based game. The action space for the game is small, but not all moves are available at all the states. Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest.
During training, when the agent is calculating the loss between policy and target networks, should the illegal actions be ignored (set to 0) so that they don't affect the calculations?

I've implemented this exact scenario before; your approach would most likely be successful, but I think it could be simplified.

Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest.

In DQN, the Q-values are used to find the best action. To determine the best action in a given state, it suffices to look at the Q-values of all valid actions and then take the valid action with highest Q-value. Setting Q-values of invalid actions to 0 is unnecessary once you have a list of valid actions. Note that you would need that set of valid actions to set invalid Q-values to 0 in the first place, so the approach I'm suggesting is more concise without worsening the performance.
Since the relative order of the Q-values is all that is required to find the best action, there is no need for normalization. Also, the original DQN paper uses $\epsilon$-greedy exploration. Keep in mind to only sample from valid actions in a given state when exploring this way.

During training, when the agent is calculating the loss between policy and target networks, should the illegal actions be ignored (set to 0) so that they don't affect the calculations?

As noted in one of your previous questions, we train on tuples of experiences $(s, a, r, s')$. The definition of the Q-learning update is as follows (taken from line 6.8 of Sutton and Barto):
$$Q(S_t, A_t) \leftarrow Q(S_t, A_t) + \alpha \left[R_{t+1} + \gamma\max\limits_aQ(S_{t+1}, a) - Q(S_t, A_t)\right].$$
The update requires taking a maximum over all valid actions in $s'$. Again, setting invalid Q-values to 0 is unnecessary extra work once you know the set of valid actions. Ignoring invalid actions is equivalent to leaving those actions out of the set of valid actions.


Could a CNN hear the difference between sound of a pet moving, and a person? "Deep Learning" neural networks are now successful at image-recognition tasks that I would not have expected say 10 years ago.  I wonder if the current state of the art in machine learning could generally tell the difference between the sound of a dog or cat moving around a house, and a person walking in the same area, taking as input only the sound captured by a microphone. I think I could generally tell the difference, but it is hard to explain exactly how.  But this is also true of some tasks that deep learning is now succeeding at.  So, I suspect it is possible but it's not clear how you would go about it.

I have found algorithms to detect human speech (wikipedia:"Voice activity detection") but separating animal and human footsteps seems more subtle.

It is an interesting application. It is possible. You can interpret sound as histogram (2D image) and apply same image processing techniques (CNN) to extract information. Alternatively, you can keep them as phase / intensity values and train a network on top of them (RNN). That is a great idea. Go for it!


Predict best price using neural network? Our customer runs a tour agency. He has an excel spreadsheet containing the following information for people that have contacted them:

Customer name, country, tour duration (requested by customer), tour date, number of people in the tour (usually from 1 to 3), price given to the customer, answer: accepted/rejected (indicates if customer accepted or rejected the price given by the tour agency).

My customer wants a predictor or tool that can let him enter the details given by future customers, e.g:

Number of participants,
Tour duration,
Country (not sure if necessary?)

And the system will return the best price to charge the customer (so he won't reject the proposal but pay the maximum possible).

Another option would be that the tour agency owner will enter the price and the system will answer "Customer will accept that price" or "Customer will reject that price".

Is this even possible? I think it may be done using neural networks trained with the previous answers from customers that the tour agency owner has in his excel spreadsheet?

This is possible.... but there's no reason to use a neural network! Your best bet on a problem like this is likely to use a logistical regression for the yes/no aspect of the question and a linear regression (or combination of linear regressions) to answer the pricing question - there are also ways of simply using linear regressions and setting up cutoffs to answer the yes/no question.

The reality is that the accuracy of such a model/series of models would depend entirely on the quality and quantity of the data, but it's unlikely that in this case a neural network would provide a better result than smart usage of simpler models.


What linear rectifier is better? What rectifier is better in general case of Convolutional Neural Network and how about empirical rules to use each type?

ReLU
PReLU
RReLU
ELU
Leacky ReLU

I've read all the papers about PReLU, LeakyReLU (...) and all the claims how it improves this and that but the little dirty secret is: most of the time it doesn't matter at all and you can't go much wrong with ReLU - empirically proven. I've personally tried all of them in many different problems (from training small networks from scratch through changing activations in large pretrained models) My guess is that gradient doesn't die much in any of them and the rest is pretty much irrelevant.


How do I classify measurements into only two classes? I am a member of a robotics team that is measuring the amount of reflected IR light to determine the lightness/darkness of a given material. We eventually hope to be able to use this to follow a line using a pre-set algorithm, but the first step is determining whether the material is one of the binary options: light or dark.

Given a large population of values between 0 and 1023, probably in two distinct groupings, how can I best go about classifying a given point as light or dark?

The way to categorize measurements into two separate populations is through what ML people currently term unsupervised learning. Such a process is part of the AI tool chest. Statistics is part, but not all, of the mathematics involved in the theory that leads to algorithms that learn without labeling the array of points light or dark in advance.

Reflected IR light can be specular or mirror in nature and significant levels of IR emission will also occurs, without raising the surface temperature of materials under most common circumstances. There is also refraction and transmission to be considered in the robotics space.

Materials will not exhibit lightness. The material composition, coating, and surface texture will exhibit various reflectivities, emission dependent on the absolute temperature raised to the fourth power1, and transmission of light entering the material from various directions, possibly diffused internally.

  We eventually hope to be able to use this to follow a line using a pre-set algorithm.

That is one of the primary objectives of computer vision and has been since the first digital signal processing in the mid twentieth century.

  The first step is determining whether the material is one of the binary options, light or dark.

That may be an experimental first step, but the loss of information from taking a number that discretely represents a cube in horizontal-vertical-frame space, each of which has $\tau$ channels of $\beta$ bits, is not a good computer vision strategy. The total number of bits of IR information is given by

$$ \tau \beta \; \text{,} $$

so the data loss will be, in percent,

$$ \dfrac {100 \, (\tau \beta - 1)} {\tau \beta} \; \text{.} $$

Object recognition and thus collision avoidance will be frustrated by that much data loss, even if only one IR spectral range is contained in the cube ($\tau = 1$).

  Given a large population of values between 0 and 1023, probably in two distinct groupings, how can I best go about classifying a given point as light or dark?

It appears that $\tau$ does equal one and $\beta = \log_{2} (1023 - 0 + 1) = 10$, thus the information in bits per cube (pixel in a frame) is 10. If the project, for some reason, requires that 9 bits be discarded in a way to conserve as much of the original 10 bits as possible (perhaps to maintain a particular throughput through a transmission or processing bottleneck), then the way to do it is through a one bit feature extraction, meaning that the output of the artificial network should be one bit. These are a few unsupervised learning options to investigate.

RBMs (restricted Boltzmann machine)
K-means clustering
Gaussian mixture models

Most AI and artificial network frameworks and libraries will have examples of these in the example directory. It is advisable to approach the project like this.

Find web pages or articles using the above as search terms until an explanation that is clear to the team is found.
Study all three for at least a few hours each as a team.
Find a library or framework that contains all three.
Install the necessary prerequisites for all three on the lab computer(s).
Produce a working example of the three types so that selection is not a function of which one is easiest to get running (something which would qualify as an research anti-pattern).
Make a choice.
Change the working code, one verifiable step at a time into the desired unsupervised learning program.
Eventually, when thoroughly ready, make the jump from the example data set to the robotic IR light frame set.

Footnotes

[1] Stefan-Boltzmann law


Could AIs self-develop in the future? Is it possible that, at some time in the future, AIs will be able to initiatively develop themselves, rather than passively being developed by humanity?

This is known as the intelligence explosion hypothesis or recursive self-improvement.


How to have zero value or a value between 200 and 400 in the output of a deep learning model? I want to implement a DDPG method and obviously, the action space will be continuous. I have three outputs. The first output should be zero or a value between 200 and 400, and the other outputs have similar conditions. I don't know how can I implement this condition in the layers and activation functions. Should I use a binary activation before the scaled sigmoid function? How can I scale the activation function for this example?

(a1 = 0) or (200 < a1 < 400)
(a2 = 0) or (100 < a2 < 500)
(a3 = 0) or (200 < a3 < 1000)

generally the approach is to have a separate head. For example, imagine you have latent vector $z_k$, you would output two values: $h(z_k)$ and $f(z_k)$ where $0 \leq h \leq 1$ and $b_0 \leq f \leq b_1$ where $b_0$ and $b_1$ are your bounds.
In thios setup, during inference you would check $h_k$ and if its greater than some threshold (usually .5), youd evaluate/output $f_k$.
In this case your loss would look something like $L_i = L_i^{(1)}(h_i, y_i)+y_i*L_i^{(2)}(f_i,v_i)$ where $(y,v)$ are your labels, and the $L$'s are your loss of choice.


How to classify data which is spiral in shape? I have been messing around in tensorflow playground. One of the input data sets is a spiral. No matter what input parameters I choose, no matter how wide and deep the neural network I make, I cannot fit the spiral. How do data scientists fit data of this shape?

There are many approaches to this kind of problem. The most obvious one is to create new features. The best features I can come up with is to transform the coordinates to spherical coordinates.

I have not found a way to do it in playground, so I just created a few features that should help with this (sin features). After 500 iterations it will saturate and will fluctuate at 0.1 score. This suggest that no further improvement will be done and most probably I should make the hidden layer wider or add another layer.

Not a surprise that after adding just one neuron to the hidden layer you easily get 0.013 after 300 iterations. Similar thing happens by adding a new layer (0.017, but after significantly longer 500 iterations. Also no surprise as it is harder to propagate the errors). Most probably you can play with a learning rate or do an adaptive learning to make it faster, but this is not the point here.


How can I process neural network with 25000 input nodes? I'm trying to build a neural network between protein sequence and its drug fingerprint. My input size is 20000. The output size is 881. The sample size is 610.

Can I process this huge neural network? But how? And in which tool?

Yes, it should be no problem.
When you decide to use a CNN, you have to make sure that this makes sense. Another answer mentioned using 3x3 convolutions -- which I would recommend against. For that to work, you would need to turn your vector into a rectangular array, and you would be implying a structure that isn't there.
Use one-dimensional convolutions instead.


Is consciousness necessary for any AI task? Consciousness is challenging to define, but for this question let's define it as "actually experiencing sensory input as opposed to just putting a bunch of data through an inanimate machine." Humans, of course, have minds; for normal computers, all the things they "see" are just more data. One could alternatively say that humans are sentient, while traditional computers are not.

Setting aside the question of whether it's possible to build a sentient machine, does it actually make a difference if an AI is sentient or not? In other words, are there are tasks that are made impossible - not just more difficult - by a lack of sentience?

No-one knows.

Why: because it's not possible to formally determine even whether your fellow human beings are actually conscious (they may instead be what is philosophically termed a 'Zombie'). No test known to modern physics suffices to decide. Hence it's possible that you are the only sentient being, and everyone else is a robot.

Consequently, we cannot determine which tasks require sentience.

Note that the ontological status of Zombies is controversial: some philosophers of AI (e.g. Daniel Dennett) claim that Zombies are logically impossible while others such as David Chalmers would claim that a Zombie would be compelled to assert that they experience qualia (i.e. are sentient) even though they do not. Here is a very readable paper by Flanagan and Polger that also explains why a stronger neurological version of the Turing test is insufficient to detect a Zombie.

EDIT: In response to the comment about whether an objective test for distinguishing sentience from non-sentience exists:

No-one knows. What we do believe is that this would require something in addition to what modern physics can currently tell us. David Chalmers has speculated that qualia should be introduced as a new form of physical unit, orthogonal to the others in the same way that electrical charge is orthogonal to distance.

In the absence of an objective test, we have to rely on Turing test variants, which no more guarantee consciousness in the subject than they do intelligence.


In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias? While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the gradient
$$\hat{G}_B = \mathbb{E}_B[\nabla_{\theta}T_{\theta}] - \frac{\mathbb{E_B}[\nabla_{\theta}T_{\theta}e^{T_{\theta}}]}{\mathbb{E}_B[e^{T_{\theta}}]},$$
where $\mathbb{E}_B$ denotes the expectation operation w.r.t. a minibatch $B$, and $T_{\theta}$ is a neural network parameterized by $\theta$. The authors claim that this gradient estimation is biased and that can be reduced by simply performing an exponential moving average filtering.
Can someone give me a hint to understand these two points:

Why is $\hat{G}_B$ biased, and
How does the exponential moving average reduce the bias?

The lower bound in MINE is as follows:
$$\widehat{I(X;Z)}_n = \sup_{\theta\in\Theta} \mathbb{E}_{\mathbb{P}_{XZ}^{(n)}}[T_\theta] - \log{\mathbb{E}_{\mathbb{P}_X^{(n)} \otimes \hat{\mathbb{P}}_Z^{(n)}}[e^{T_\theta}]}$$
Here $\mathbb{\hat{P}^{(n)}}$ denotes the empirical distribution that we get from n i.i.d samples of $\mathbb{P}.$
Note that in the above equation, the first term is calculated from the joint distribution while the second term from the marginals of $X$ and $Z$. In the implementation of MINE, these statistics are calculated over the data from a minibatch. The marginal distribution is obtained by shuffling the values of Z (or X) along the batch dimension. Hence, in this case, the gradient is as follows.
$$\hat{G}_B = \mathbb{E}_B[\nabla_{\theta}T_{\theta}] - \frac{\mathbb{E_B}[\nabla_{\theta}T_{\theta}e^{T_{\theta}}]}{\mathbb{E}_B[e^{T_{\theta}}]},$$

As mentioned, the expectation over the marginals is not calculated
over the true marginal distribution (i.e. over the entire dataset)
but from the shuffled samples in the minibatch. Hence, the above gradient $G_B$ is biased.
When we maintain an exponential moving average of
$\mathbb{E}_B[e^{T_{\theta}}]$, we kind of incorporate statistics from outside the current minibatch also (i.e. over the entire dataset). This is an attempt to get an approximation of the true marginal estimate. The denominator term in the gradient allows for such a computationally inexpensive bias reduction trick.


What are some resources regarding the complexity of training neural networks? In the paper "Provable bounds for learning some deep representations", an autoencoder like a model is constructed with discrete weights and several results are proven using some random-graph theory, but I never saw any papers similar to this. i.e bounds on neural networks using random graph assumptions.

What are some resources (e.g. books or papers) regarding the time and space complexity of training neural networks?

I'm particularly interested in convolutional neural networks.

There are a few technical papers and books on the topic

Computational Limitations on Learning from Examples (1988) by Leonard Pitt and Leslie G. Valiant, published in Journal of the ACM
Training a 3-node neural network is NP-complete (1992) by Avrim L. Blum and Ronald L. Rivest, published in Neural Networks
Computational complexity of neural networks: a survey (1994) by Pekka Orponen, published in Nordic Journal of Computing
On the relative time complexities of standard and conjugate gradient backpropagation (1994) by J. V. Stone and R. Lister, published in IEEE
Neural Networks - A Systematic Introduction (1996), specifically chapter 10 The Complexity of Learning, a book by Raul Rojas
Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network (2014) by Gursel Serpen and Zhenning Gao, published in Complex Adaptive Systems
On the Computational Efficiency of Training Neural Networks (2014) by Roi Livni, Shai Shalev-Shwartz and Ohad Shamir, published in NIPS 2014
Convolutional Neural Networks at Constrained Time Cost (2014) Kaiming He and  Jian Sun, published in IEEE Conference on Computer Vision and Pattern Recognition (2015)
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time (2018) by Surbhi Goel and Adam Klivans

However, note that gradient descent (and other optimization algorithms) and the back-propagation algorithm are numerical algorithms (that is, they deal with numerical errors), so the time complexity is not the only factor affecting the actual performance of these algorithms and models.


Does fitness proportionate selection select multiple individuals? Does fitness proportionate selection select multiple individuals?
So, I read on Wikipedia and on multiple Stack Exchange threads about fitness proportionate selection or rather roulette selection, but what I don't understand is, how do I not select multiple times the same individuals? Should I pop them from my array and recalculate the probabilities or after selection remove duplicates? Or is there some sort of purpose for having multiple of the same selected?

Yes, you select the same individual multiple times, according to the distribution of fitness values. Typically, you have at least as many offspring as parents, so if you didn't replace parents in the pool to be potentially selected again, you'd just have every parent being selected once, which defeats the purpose of selection. You want fitter parents to be selected more often so that you drive the average fitness higher across generations.

That said, there are no iron clad rules. There are algorithms that create selection pressure in other ways, and you can do whatever works for you. But in the canonical version of a GA, you do selection with replacement. Though, many people find other selection schemes like tournament selection perform better across a pretty wide range of problems than roulette wheel anyway.


Can we teach machine to tie shoe lace? Is it possible with any of machine learning methods to train machine to tie shoe lace? If possible how data should be interpreted for the training?
If we are using reinforcement learning, how will it learn to reach the best rewards?

You would not believe how difficult this task is, assuming you want a humanoid robot to tie laces with humanoid hands. It’s possible of course but compared to what the current state of the art in machine learning is, this task is very very complex because it’s a physical system with unknown variables (Eg. coefficient of friction on laces), physical limitations (Eg. dexterity of robotic hands), and occluded vision (Eg. hand in front of laces) to name a couple of the issues one faces not to mention robots are expensive but I digress.

Robots can perform amazing preprogrammed sequences but if you wanted to have a robot to be able to tie any shoe in any situation, machine learning is the right approach.

Training data should include all of the things that humans use when they approach this problem:

proprioception (the location of joints in space aka where are your fingers)

a sense of touch, a robot would need a quite a few touch sensors
vision, although not actually necessary (because blind humans are capable of learning to do this task), it may speed up learning

Using reinforcement learning as a framework, one needs to be able to give the RL agent a signal that it is closer to tying the lace. This could be done in a few ways and it’s unclear what would be the best.

One method would be to train a separate model using supervised learning and pictures of shoes laced up to look at the current state of the bow (or lack thereof) and report if it looks like a tight bow. This method requires a whole other network but it may be the most versatile in the end although the RL agent may just learn how to place laces so it looks like a tight bow.

Although simple for humans, this is an interesting problem and thinking about how it could be implemented helps one understand how to apply ML in other areas.


How is regression machine learning? In regression, in order to minimize an error function, a functional form of hypothesis $h$ must be decided upon, and it must be assumed (as far as I'm concerned) that $f$, the true mapping of instance space to target space, must have the same form as $h$ (if $h$ is linear, $f$ should be linear. If $h$ is sinusoidal, $f$ should be sinusoidal. Otherwise the choice of $h$ was poor).

However, doesn't this require a priori knowledge of datasets that we are wanting to let computers do on their own in the first place? I thought machine learning was letting machines do the work and have minimal input from the human. Are we not telling the machine what general form $f$ will take and letting the machine using such things as error minimization do the rest? That seems to me to forsake the whole point of machine learning. I thought we were supposed to have the machine work for us by analyzing data after providing a training set. But it seems we're doing a lot of the work for it, looking at the data too and saying "This will be linear. Find the coefficients $m, b$ that fit the data."

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy.

but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper models come into play: You allow a learn-able featurization and classification/regression. Think about it this way, the result of your regression is most likely linearly associated with some set of features, they just may not be the ones you are interested in (you can prove this actually with any infinitely wide network :: Universal approx Thm). Unfortunately we cant use an infinitely dimensional model, so we run with these giant over-parametrized models where we hope the a good function can be described by a sub-structure (only recently are we starting to pay attention, to how these sub-structures form -- look at this paper)

But the way you bring about thinking about it is a large pit fall for many trying to move forward. Alot of ML people now of days gain success by throwing a function without alot of parameters on a big data problem, but youll see the largest advancements in the field come from a theoretical understanding of the featurization and optimization.

I hope this helped


What would happen if someone jumped in the front of a Google car? The Situation:
A self-driving car is traveling at it's maximum speed, 25 mph (40 km/h), in the middle of an empty street with the ability to change lanes on both sides. There are two passengers, one in the front and another in the back.

Someone jumps from the side of the road directly into the path of the car. A collision would occur in 50 meters. Breaking distance at this speed is about 24m.

The Question: Is it known how the current implementation of the Google Car AI would react, or is it currently a matter of speculation? A step-by-step explanation of the AI's decisioning process would be preferred.

Possible Answers: The car could activate its brakes immediately, coming to a halt as quickly as possible. This would be sooner than a human could stop, as people require time to recognize the possibility of a collision, and then physically slam on the brake. (thinking distance).

Alternatively, the car could continue traveling forward, processing the situation. (Similar to a humans thinking distance). The person may continue to move, either out of the way, or still into danger of being hit. In this case, the car may decide to change lanes in an attempt to pass around the person.

Lastly and most unlikely, the car will not alter its course and proceed to drive forward.

Do not attempt to do it to check;)

Given this YouTube video which is being given by Sebastian Thrun who had a TED talk which had nowhere near the same level of detail but had similar conclusions, it looks like the lidar system used by google's automated car system has decent resolution out to at least 30m picking out mobile bodies in the static background and then identifying it. So it should have plenty of time to brake and stop long before there was any risk to the pedestrian attempting to cross the street.

Skip to about 6:40 in the video to see a visual representation of the detection system.


Which loss function is the brain optimizing in order to learn advanced visual skills without expert/human supervision? In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.

What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. what does the brain use as ground truth, what is the loss function the brain is optimizing)? Finally, can we apply these insights in computer vision?

Update: I posted a related question on Psychology & Neuroscience StackExchange, which I think complements the question I posted here: check it out

I think you are slightly confusing 2 problems. 1 being classification of meta visual elements and the other being the visual system itself.

Our visual system, when it comes to processing information, has had billions of years of iteration(training), so that at birth(and before), we are already tuned for the processing of visual stimuli, as well as have the mechanisms to decipher objects in our spatial field of view.

These two papers(L1, L2), have a great deal of information about the evolution of our visual system and its processing. The second speculates on the connection of said evolution and the construction of "seeing systems" very interesting.

For further inquiry on this in particular, check out David Marr. He was probably the most influential early computer vision mind. He still is mentioned in many top-down AGI and computer vision research projects to this day.


Are there any online competitions for Reinforcement Learning? Kaggle is limited to only supervised learning problems. There used to be www.rl-competition.org but they've stopped.

Is there anything else I can do other than locally trying out different algorithms for various RL problems?

AICrowd has numerous challenges in the domain, with some very interesting challenges running currently. Here is a short list:

https://www.aicrowd.com/challenges?&categories=reinforcement-learning (Complete list with RL tag)
https://www.aicrowd.com/challenges/neurips-2020-procgen-competition
https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge
https://www.aicrowd.com/challenges/neurips-2019-minerl-competition

Hope this helps!


Would it be ethical to allow an AI to make life-or-death medical decisions? Would it be ethical to allow an AI to make life-or-death medical decisions?

For instance, where there an insufficient number of ventilators during a respiratory pandemic, not every patient can have one.  It seems like a straight forward question, but before you answer, consider:

Human decision-making in this regard is a form of algorithm.

(For instance, the statistics and rules that determine who gets kidney transplants.)

Even if the basis for the decision is statistical, the ultimate decision making process could be heuristic, so at least the bias could be identified.

In other words, the goal of this process, specifically, is favoring one patient over another, but doing so in the way that has the greatest utility.

Statistical bias is a core problem of Machine Learning, but human decision making is also subject to this condition.

One of the arguments in favor might be that at least the algorithm would be impartial, here in relation to human bias.

Finally, where there is scarcity, utilitarianism becomes more imperative.  (Part of the trolley problem is you only have two tracks.)  But the trolley problem is also relevant because it can be a commentary on the burden of responsibility.

I will interpret the questions as being about triage. This is particularly important in crisis situations, where a lot of such life-or-death decisions have to be taken.

In the START system there are four different categories:

the deceased, who are beyond help
the injured who could be helped by immediate transportation
the injured with less severe injuries whose transport can be delayed
those with minor injuries not requiring urgent care.

According to the category assigned to a patient, you then decide what to do. Other systems might be more fine-grained, but the principle is the same: effectively the human decision is in the classification, which then guides the assignment of resources. In the above list, the second category would probably be the highest priority for treatment. But once the category has been decided, the course of action has been determined, though the actual treatment options will then be considered for each case.

The ethics are thus in the judgment of the survival chances: if a nurse decides patient X is too far gone to warrant treatment, that's it. So this is the hard part, the life-or-death decision.

There are then two aspects to consider:

The accuracy of the diagnosis
The predicted likelihood of treatment being effective

The diagnosis should be (I'm not a medic!) fairly neutral. There are of course mis-diagnoses, but there is no value judgment involved. There might be some 'leakage' between step 1 and step 2, in that a human being might be influenced by the prospects of treatment of a diagnosis when deciding what the problem is. As in: this is a nice person, I don't want them to die, so I (subconsciously) exclude the diagnosis X which would mean certain death.

In this case a computer system that had sufficient accuracy when making a diagnosis would IMHO actually be more ethical than a human being. Provided, of course, that there is no bias in the diagnosis. Which I think is a practical problem, but in theory could be dealt with.

Once the diagnosis has been determined, the estimation of treatment success is the next decision. This also can be subject to human error, and a computer system with access to statistical information (again, unbiased) of issues and their likelihood of survival could be more independent of other factors which a human being might use, but which are unrelated to the case.

So basically, I would say that a computer system could be more ethical than a human being in such a situation. Because you can defend the way it reached it's decision, and it has been without taking into account factors not related to the problem at hand.

However, it's not always that easy. There are plenty of cases where other factors influence the decision. As long as there are fixed rules, that might not be a problem, though. Some issues would be (in each case the patients would have the same diagnosis and projected survival chance):

One ventilator, but an 8-year-old child and a 85-year-old man
A 30-year-old pregnant woman and the Prime Minister of the country
A one-month-old baby and a 25-year-old student
A homeless person and a billionaire

As a given, the system would have to be agnostic of general features such as gender, race, religion, etc. But how do you take into account age, social status, and a whole host of other factors?

If there is no difference in the situation, the fairest way would be to toss a coin. That would surely upset a lot of people ("How can you justify leaving our president to die in this time of crisis!"), but if you had explicit rules ("If one patient is in a higher tax bracket, they get priority") you might upset even more people. The advantage of having such rules would be to make the bias explicit, and in a way protect a nurse or paramedic from having to decide between a rich senator and a homeless person — whose family is more likely to go after you if you decided against them? And if you have explicit, unambiguous rules, why not use them to guide an AI system?

Every human being has their own preferences, and I am glad that I don't have to make those kinds of decisions — it sounds like a horrific task to me. How can you even sleep in peace after basically having condemned one each of the above cases to death?

That is another factor about the ethics of AI: by relieving humans from having to make such decisions, it would also have a beneficial effect. If the final decision is the same that the human would come to, then it's a win/win situation. But that is probably unlikely, due to a whole range of subconscious biases.

The prime issue to me seems the lack of recourse in the case of "computer said no". When a human makes a decision (like a referee in a game), there will always be an argument if people are unhappy with it. But in this case there could be none. The oracle has spoken, your father will be left to die. Sorry, no other outcome possible. It would probably be the same with a human decision, but it wouldn't feel as 'cold': you can observe that the person making the decision has to struggle with it. And you might understand that it was not an easy choice. With a computer, that element is missing.

Anyway, to summarise: given various caveats,

sufficiently accurate and unbiased diagnosis,
unbiased prediction of treatment outcome,
transparent handling of non-medical factors,

I would say that an AI system would me more ethical, as it has a principled way of reaching a decision which does not disadvantage any group of patients and would always get the same decision for the same patient; furthermore it takes a heavy burden off triage staff who otherwise have to make such decisions.

Which does not mean that I would be happy with my loved-ones' survival being decided by such a system :)


How to train a model for 1 image class to detect anomaly? I want to train a model with python over the images, and these images are for a metal product.
my aim is to detect the defects, to notice if a product is a failure.
what kind of architecture do you suggest? should I train over the class? or should I use an autoencoder?

It sounds like you only have "normal" examples with which to train your model, so this makes the problem feel like an application for outlier detection algorithms. There are a variety of approaches here. You could indeed take an autoencoder approach and then use the reconstruction error to determine if a new image is normal or not, on the presumption that normal images will have lower error. You could also take the activations from the bottleneck layer and build an explicit outlier model using an algorithm like isolation forest. And you are not limited to just autoencoders if you take the latter approach -- other models pretrained for other tasks like image net classification could also provide good features for outlier detection.


How should dropout change with network depth? I know I've seen this somewhere before, but can't find it now.  Say we have a neural network with a handful of layers, and we're applying dropout to each layer.  As we move closer to the output, should dropout decrease, increase, or stay the same?

We can think of the dropout as of averaging over many networks. Not as a way to mute or denoise your data. Therefore the rule of thumb is the dropout should be applied to higher layers in your network. It is usually applied to fully connected layers (if at all!). It is a common practice to keep 0.5-0.75. neurons active. Personally I prefer L2 much more.


What do the words "coarse" and "fine" mean in the context of computer vision? I was reading the well know paper Fully Convolutional Networks for Semantic Segmentation, and, throughout the whole paper, they talk use the term fine and coarse. I was wondering what they mean. The first time they say it in the intro is:

  Convolutional networks are driving advances in recognition. Convnets are not only improving for whole-image classification, but also making progress on local tasks with structured output. These include advances in
  bounding box object detection, part and keypoint prediction, and local correspondence.

  The natural next step in the progression from coarse to fine inference is to make a prediction at every pixel.

It's also used in other parts of the paper

  We next explain how to convert classification nets into fully convolutional nets that produce coarse output maps.

What do "coarse" and "fine" mean in the context of this paper? And in the general context of computer vision?

In English, "coarse" means "rough or loose in texture or grain" , while "fine" means "involving great attention to detail" or "(chiefly of wood) having a fine or delicate arrangement of fibers", but these definitions do not elucidate the meaning of these words in the context of computer vision.

This question was also asked here.

tl;dr

  What does that mean in the context of this paper?

With "coarse segmentation" the author means a segmentation that doesn't have much detail. "Fine segmentation", on the other hand, refers to a segmentation with a high level of detail.

  But also more importantly [what does that mean in the context of] general computer vision?

The most common use in CV is to describe how general or specific a class in is classification. A "coarse class" is a very broad one, while a "fine class" is a very specific one.

Intended use

What the author refers to is the level of detail of the resulting segmentation.

A coarse segmentation would mean that we have large blobs covering each class without much detail. On the other hand, a fine segmentation would have a much higher level of detail which can even go down to pixel level (i.e. pixel-by-pixel correct segmentation).

To make this clear look at the following two examples. As we go from the left to right, the segmentation maps go from coarse to fine:

Note that in the rightmost images the result an almost pixel-perfect (i.e. fine details) segmentation map, while the ones on the left don't have much detail and can be considered coarse.

Alternative use

Because this isn't an established terminology, some of the times coarse and fine can refer to the nature of the classes in a classification task. Take the top image for example; the label for a coarse classification task could be a tree. For a fine classification task we would have labels like oak tree, pine tree, etc.

The most prominent example of this is the cifar dataset which has two versions: a coarse one that has 10 classes and a fine one that has 100 classes, which are all subclasses of the coarse classes. For example a coarse class is fish, while the fine ones are aquarium fish, flatfish, ray, shark, trout, etc.

For semantic segmentation an example could be the following: you want to make a street segmentation model. This a coarse segmentation would mean that it just splits the image into road, vehicle, etc. A fine segmentation, on the other hand, could also detect the type of vehicle, e.g. truck, car, etc.