Should Monte Carlo tree search be able to consistently beat me in the connect four game? I've implemented the Monte Carlo tree search (MCTS) algorithm for a connect four game I've built. The MCTS agent beats a random choice agent 90-100% of the time, but I’m still able to beat it pretty easily. It even misses obvious three in a row opportunities where it just needs to add one more token to win (but places it elsewhere instead).

Is this normal behavior, or should the MCTS agent be able to beat me consistently too? I'm allowing it to grow its tree for 2 seconds before getting it to return its chosen action - could it be that it needs longer to think?

You should not let the tree grow for only two seconds rather you should use the simulation number equal to 1000 or something like that. I use the simulation number equal to 10000 for making a single move in the tictactoe game and it was working fine for me. Also, after the agent has chosen the move you do not have to start the statistics(N = visit count, V = expected reward, U = UCT score) from the beginning, you can use the current statistics and replace the root node with the chosen node.


How to tinker with CNN architectures? I was thinking of creating a CNN. Now it is known CNN takes long times to train so it is advisable to stick to known architectures and hyper-parameters.

My question is: I want to tinker with the CNN architecture (since it is a specialised task). One approach would be to create a CNN and check on small data-sets, but then I would have no way of knowing whether the Fully Connected layer at the end is over-fitting the data while the convolutional layers do nothing (since large FC layers can easily over-fit data). Cross Validation is a good way to check it, but it might not be satisfactory (since my opinion is that a CNN can be replaced with a Fully Connected NN if the data-set is small enough and there is little variation in the future data-sets).

So what are some ways to tinker with CNN and get a good estimate for future data-sets in a reasonable training time? Am I wrong in my previous assumptions? A detailed answer would be nice!

There is no easy way to play around with the hyper-parameters (number of layers, layer configuration, number of outputs per layer) of a CNN and get an accurate view of how these will affect the resulting performance of your model. However there are a few things that you can do to avoid wasting too much time training and re-training.

Why?

When training a CNN we aim to minimize a loss function, thus a better CNN model is defined as one which converges to a set of model parameters with a lower loss function. Identifying the minimum of the loss function for a given CNN is already very difficult, and there is no guarantee that the true minimum will every be reached by fault of gradient descent.

Each variation of the CNN resulting from the different hyper-parameters will still result in a very large number of model parameters. There is no way to know how the loss function will look in this hyper-dimensional space and even harder to estimate its minimum.

What to do?

First, you should try and understand how each layer in the network affects its inputs. You should know what kind of layers to use for what kind of data. You should also know what kinds of activation functions to use for different data distributions. You should also know how many model parameters per layer to try in order to sufficiently compress your data whilst not losing significant information.

You can get a lot of this intuition by reading papers which have found successful models for specific tasks.

In addition, you can train with smaller amounts of data and estimate the potential minimum loss function by seeing how fast the loss function moves towards its minimal value (momentum). Usually a lower minimum is achieved when the loss function decreases faster. However, this is in no way always true. A loss function can converge slowly at first and then speed up later. This is entirely possible. But, you can get some sense of the potential of your model in this way.


Can an AI learn how to play chess without instructions? Can an AI learn to play chess if you give it nothing but "the goal is to win" as starting criteria? If not, what is the minimum information the AI would need to be "seeded" with in order to learn to play chess?  What techniques could be used to create an AI that learns to play chess independently?

It's possible for an AI to learn chess without even knowing how to move the pieces. Google's AlphaZero didn't do that as their programmers coded the chess rules, but it's possible.

One can learn the rules from human played chess games. Once the rules are known, we could use reinforcement learning to improve playing strength (and other board games).


What is the best XOR neural network configuration out there in terms of low error? I'm trying to understand what would be the best neural network for implementing an XOR gate. I'm considering a neural network to be good if it can produce all the expected outcomes with the lowest possible error.
It looks like my initial choice of random weights has a big impact on my end result after training. The accuracy (i.e. error) of my neural net is varying a lot depending on my initial choice of random weights.
I'm starting with a 2 x 2 x 1 neural net, with a bias in the input and hidden layers, using the sigmoid activation function, with a learning rate of 0.5. Below my initial setup, with weights chosen randomly:

The initial performance is bad, as one would expect:
Input | Output | Expected | Error
(0,0)   0.8845      0       39.117%
(1,1)   0.1134      0       0.643%
(1,0)   0.7057      1       4.3306%
(0,1)   0.1757      1       33.9735%

Then I proceed to train my network through backpropagation, feeding the XOR training set 100,000 times. After training is complete, my new weights are:

And the performance improved to:
Input | Output | Expected | Error
(0,0)   0.0103      0       0.0053%
(1,1)   0.0151      0       0.0114%
(1,0)   0.9838      1       0.0131%
(0,1)   0.9899      1       0.0051%

So my questions are:

Has anyone figured out the best weights for a XOR neural network with that configuration (i.e. 2 x 2 x 1 with bias) ?

Why my initial choice of random weights make a big difference to my end result? I was lucky on the example above but depending on my initial choice of random weights I get, after training, errors as big as 50%, which is very bad.

Am I doing anything wrong or making any wrong assumptions?

So below is an example of weights I cannot train, for some unknown reason. I think I might be doing my backpropagation training incorrectly. I'm not using batches and I'm updating my weights on each data point solved from my training set.
Weights: ((-9.2782, -.4981, -9.4674, 4.4052, 2.8539, 3.395), (1.2108, -7.934, -2.7631))

The initialization of the weights has a big impact on the results. I'm not sure specifically for the XOR gate, but the error can have a local minimum that the network can get "stuck" in during training. Using stochastic gradient descent can help give some randomness that gets the error out of these pits. Also, for the sigmoid function, weights should be initialized so that the input to the activation is close to the part with the highest derivative so that training is better.


Parameters to calculate affluence in localities of Metro city I have to calculate the affluence in localities of Metro city. To calculate affluence, I am considering a parameter per capita income.

Where I can get a dataset of it? What are other parameters I should consider for the problem?

Any guidance will be fruitful for me.

Affluence could encompass several parameters:
Income;
Wealth (property ownership);
Life expectancy;
Access to services such as education and health;
Access to clean natural resources;
Low levels of criminality.

Property prices in each locality might be easy to obtain from real estate agent sources
Ratings for schools or medical facilities in each area might be published

Generally, where public statistics are collected on a locality, they will be related in one way or another to affluence. A useful strategy might be to collect as many of these diverse data sets as possible, and to learn a composite affluence score from the data. It is very likely that all of these parameters will be correlated to a greater or lesser degree, and so you could accurately learn about affluence from a small number of these parameters.


Facial Recognition + Database + Compare & Identify - is it complicated? The last week I've been looking for freelancers who are able to do this project for me but they weren't that experienced in it, so I would like to know whether my idea is complicated or is it their lake of experience.
Scenario:
1. The facial recognition system will be installed on a vertical screen where a camera would be attached to it and it will be assigned on the entrance of the room.
2. Once a visitor come to the entrance, and looks at the screen, a text on the screen would say: " Welcome! It seems like it's your first visit! Please enter your name then a keyboard should popup"
3. The visitor would enter his first name into the keyboard, and it would be saved alongside his face in the database, and it would say thank you, {name}.
4. If the same visitor visits again, the system should say: " Welcome back {name}, happy to see you again"!

Depending on the level of accuracy/speed you want your system to have, you could try using an existing python API. First thing I found on google, the README looks adapted to your project. Otherwise ask your question with more accurate wanted specifications perhaps.


When do mutations in NEAT occur? I read through the Evolving Neural Networks through Augmenting Topologies (NEAT) paper. I understand the algorithm now, but one thing is still unclear to me.

When does the mutation occur and how does it take place? How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?

When does the mutation occur and how does it take place?

Finding a solution in NEAT algorithm is based on evolution strategy. It means that you have Neural Networks which are yours individuals, so mutations and crossing occurs in loop after phase of "fitnessing" (calculation fitness for every individual and removing bad ones).

  How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?

Randomly: just draw. You can read more about evolutionary algorithms here.

If it could somehow help, I include link to my repository with the implementation of NEAT.


Is text preprocessing really all that necessary for NLP? As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input.

For instance, this ACL paper seems to show that the result of text preprocessing is mixed, to say the least.

So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps!

It all depends on the quality of data. Due to old rule "Garbage in, garbage out" link
, if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular.

In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).

Shortly, preprocessing is optional, but highly advisable.


Algorithms can be greedy. What are some other algorithmic vices? Greedy algorithms are well known, and although useful in a local context for certain problems, and even potentially find general, global optimal solutions, they nonetheless trade optimality for shorter-term payoffs.

This seems to me a good analogue for human greed, although there is also the grey goo type of greed that is senseless acquisition of material (think plutocrats who talk about wealth as merely a way of "keeping score".)

Technical debt is an extension of development practices that fall under the algorithmic definition of greed (short-term payoff leads to trouble down the road.)  This may be further extended to any non-optimized code in terms of energy waste (flipping of unnecessary bits) which will only increase as everything becomes more computerized.

So my question is:

What are other vices that can arise in algorithms?

Algorithms can be racist, sexist, and otherwise bigoted. When we feed them data produced by systems that are biased against groups of people, the algorithm will learn to behave that way. We're used to garbage in garbage out, now we have to worry about racism in racism out.

See:
Facial Recognition Is Accurate, if You’re a White Guy
Rise of the Racist Robots – how AI is Learning all our Worst Impulses


Why is loss displayed as a parabola in mean squared error with gradient descent? I'm looking at the loss function: mean squared error with gradient descent in machine learning. I'm building a single-neuron network (perceptron) that outputs a linear number. For example:
Input * Weight + Bias > linear activation > output.
Let's say the output is 40 while I expect the number 20. That means the loss function has to correct the weights+bias from 40 towards 20.
What I don't understand about mean squared error + gradient descent is: why is this number 40 displayed as a point on a parabola?
Does this parabola represent all possible outcomes? Why isn't it just a line? How do I know where on the parabola the point "40" is?

Mean Square Error (MSE) is a quadratic function and the further you go away from your optimum the bigger (quadratic) the MSE gets. Take $o_{expected}=20$ and $o_{net}=40$ as example. Your MSE is then 400, because
$MSE = (o_{expected}-o_{net})^2$.
Just imagine $y = x^2$ with x being the output of your network.
If you want to shift the parabola with optimum at $20$ the formula you get is $y = (20-x)^2$.
For every new case you train the  net on, you get a different parabola with different parameters.


As a starter: what is the form of training data for image processing What we are doing in the image processing training.  We are storing some form of data which is going to act as the knowledge or experience of the system.

In which form can the system store it's training data?

For example, with the hand written recognition, we can represent the digits as combinations of curves and straight lines. For every round of training the recognition system stores data.  Is the data typically stored in a flat file (such as txt) or a database?

I have seen in Tesseract OCR that there is a text file that stores the x0,y0,x1,y1. They are the pixel points that represents the square on the training image that has the picture.

I need a efficient form of knowledge for Machine Learning, and would appreciate advice, context, or an explanation of the merits or downsides of different approaches.

  I need a form of knowledge that stored in a system. human brain evaluate '7' as 'horizontal line and vertical left crossed line start from right of the horizontal line'. like that machine must have some conceptual data to represent their knowledge.

Your question depends heavily on the method you are using for machine learning. It sounds like you want to extract certain features like "curves and straight lines" from your images and use them as training data. This step of extraction is usually not considered part of the training process but part of pre-processing. During pre-precessing you read your image in, extract certain features or perform some transformation and store the new data as your actual training samples or use them for training right away without intermediate storage. Usually storing this information is a good idea if you want to use the image for training more than once and you want to skip the pre-processing step in future training cycles.

Comparing File Storage and Database Storage

How you store your processed data for training is relatively independent from the application of machine learning and the general principles for data storage apply. Storing the data in a flat file is usually very convenient as disk storage is readily available and the APIs for storing and reading files are part of your programming language and OS. A database adds additional complexity to your architecture but has of course it's benefits, especially if you want to make the training data available to other instances or different learning engines. If you are working with a really huge amount of data, a well structured database can help you organize your data better and provides helpful functions for handling this data efficiently.

Keep in mind that the actual training of your AI takes place initially and once completed, you can roll out your AI without all the training data. So you only need to handle training data for a limited time in your application lifecycle. The speed of training is usually not very important, because it happens once and not during the daily use of the AI. Therefore the speed of file access during training can be neglected.

Conclusion

In conclusion, for most applications, the simpler implementation using just flat files is good enough. If you want to store each sample in an individual file or pack them together on bigger batches and use some meta information to identify the individual samples in the file is more a matter of taste than real technical relevance.

Storing data after training

Judging from your question, you sound like you understand machine learning in general. Just to clarify one potential misunderstanding for beginners - you don't have to store any of the "learned" information as data after training, as each training step just adapts weights and biases in the neural network. The actual training data can be thrown away after successful training, if you don't need it for future training cycles.

Further Information / References

To finish my answer I want to recommend a current course from Stanford University about CNNs and visual recognition (Dated Spring 2017). It is a great source of information for implementing neural networks for image recognition.


Was the Malaysian Airliner crash caused by a non-benign artificial intelligence system? I watched a youtube clip of Elon Musk talking about his view on the future of AI. He gave two examples. One of the examples was a benign scenario and the other example was a nonbenign scenario where he speculated the possibilities of future AI threats and what harm a deep intelligence could do.
According to Elon, a deep intelligence in the network could create fake news and spoof email accounts. "The pen is mightier than the sword". This non-benign scenario put forth by Elon was a hypothetical, but he went into detail about how it could have been possible that an AI, with the goal of maximising the portfolio of stocks, to go long on defense and short on the consumer, and start a war.
To be more specific, this could be achieved by hacking into the Malaysians Airlines aircraft routing server, and when the aircraft is over a warzone, send an anonymous tip that there is an enemy aircraft flying overhead which in turn would cause ground to air missiles to take down what was actually a "commercial" airliner.
Although this is a plausible hypothetical nonbenign scenario of AI, I'm wondering if this actually could have been the case regarding the Malaysian Airliner crash. The Stuxnet, for example, was a malicious computer worm, first uncovered in 2010. Thought to have been in development since at least 2005 and believed to be responsible for causing substantial damage to Iran's nuclear program. The Stuxnet wasn't even an AI.
The Stuxnet blew the world's minds when it was discovered. The sheer complexity of the worm and the amount of time it took to build was impressive, to say the least.
In conclusion, was the Malaysian Airliner crash caused by a non-benign artificial intelligence system?

Short answer: despite the incredible advances in AI via Machine Learning (and subfields) AI is nowhere near this kind of autonomous decision-making.

I can't prove a negative, but, the level of autonomy Musk is talking about is still on the horizon.

Doesn't mean Musk is wrong about the hypothetical.  imo I'm glad he keeps bringing these kinds of issues to the fore.

Engineers and mathematicians have a bit more credibility on this subject than laypeople.  If it seems alarmist, there are many in the scientific community who feel it is warranted.

What Musk is describing is an extension of what Asimov felt compelled to warn us about via his "Three Laws of Robotics" back in 1942, when computers sucked.  For an explication of this idea re: Machine Learning, see below.

JUST FOR FUN (SORT OF:)

Author and mathematical physicist Hannu Rajaniemi just published a story in the MIT Tech Review on this very subject.

Unchained: A Story of Love, Loss, and Blockchain

Warning: the story is both wickedly funny, surprisingly moving, and very likely prescient!


Why are all weights of a neural net updated and not just the weights of the first layer Why are all weights of a neural net updated and not only the weights of the first hidden layer?

The error-influence of the prediction by the weights of a neural net is calculated using the chain rule. However, the chain rule tells us how the first variable influences the second variable, and so on. Following that logic, we should only update the weights of the first hidden layer. My thought is, that if we backtrack the influence of the first variable but also change the values of the subsequent weights (of the subsequent hidden layer), there is no need to calculate the influence of the first weights in the first place. Where am I wrong?

However, the chain rule tells us how the first variable influences the second variable, and so on. Following that logic, we should only update the weights of the first hidden layer.

I don't see how the second statement follows from the first.

Each weight $w_i$ (not just the ones in the first layer) affects the loss $\mathcal{L}$ according to the partial derivative of $\mathcal{L}$ with respect to $w_i$, i.e. $\frac{\partial \mathcal{L}}{\partial w_i}$. Intuitively, the partial derivative with respect to parameter tells you how the function is changing with respect to that parameter.

  My thought is, that if we backtrack the influence of the first variable but also change the values of the subsequent weights (of the subsequent hidden layer), there is no need to calculate the influence of the first weights in the first place.

I am not sure I understand your reasoning, but, typically, you update the parameters only after having computed all the partial derivatives. In other words, first, you compute all partial derivatives, i.e. the gradient with back-propagation (a fancy name to denote the application of the chain rule), then you update the parameters.

Why do you do this? In this case, the loss function is multi-variable function, so it depends on multiple variables. The gradient $\nabla \mathcal{L} = \left[ \frac{\partial \mathcal{L}}{\partial w_1}, \dots, \frac{\partial \mathcal{L}}{\partial w_N} \right]$ represents the direction (note that the gradient is a vector and vectors have direction) towards which your function is increasing or decreasing (depending on the sign of the gradient).


Hyper-plane in logistic regression vs linear regression for same number of features Geometric interpretation of Logistic Regression and Linear regression is considered here.
I was going through Logistic regression and Linear regression. In the optimization equation of both following term is used. $$W^{T}.X$$
W is a vector which holds weights of the hyper-plane.
I realized following about the dimensions of the fitted hyper-plane. Want to confirm it.
Let,
d = Number of features for both Logistic Regression and Linear Regression.
Logistic Regression case:
Fitted hyper-plane is d-dimensional.
Linear Regression case:
Fitted hyper-plane is (d + 1) dimensions.
Example
d = 2
feature 1 : weight,
feature 2 : height
Logistic Regression:
Its a 2 class classification.
y : {obsess, normal)
Linear Regression:
y: blood pressure (real value)
Here,

Logistic Regression will fit a 2-D line.
Linear Regression will fit a 3-D plane.

Please confirm if this understanding is correct and same happens even in higher dimensions.

Mm, I understand what you want to say, but I think you're slightly wrong with terminology. If you consider a d-dimensional vector space, people usually a hyper-plane (d - 1)-dimensional, because it's a subspace of lower dimensionality. So for your example d = 2 the logistic regression would fit a 1-dimentional line (which would separate 2 classes) and the linear regression would fit a 2-dimentional plane (for every combination of 2 features the prediction would lay on some plane in 3-D space). So the general rule would be logreg -> (d-1)-dimensional hyperplane in d-dimensional vector space, linear regression -> d-dimensional hyperplane in (d+1)-dimensional vector space.


Why is the optimal policy for an infinite horizon MDP deterministic? Could someone please help me gain some intuition as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?

Suppose you learned your action-value function perfectly. Recall that the action-value function measures the expected return after taking a given action in a given state. Now, the goal when solving an MDP is to find a policy that maximizes expected returns. Suppose you're in state $s$. According to your action-value function, let's say actions $a$ maximizes the expected return. So, according to the goal of solving an MDP, the only action you would ever take from state $s$ is $a$. In other words $\pi(a'\mid s) = \mathbf{1}[a'=a]$, which is a deterministic policy.
Now, you might argue that your action-value function will never be perfect. However, this just means you need more exploration, which can manifest itself as stochasticity in the policy. However, in the limit of infinite data, the optimal policy will be deterministic since the action-value function will have converged to the optimum.


Is a state that includes only the past n-step price records partially observable? I'm currently working on a project to make an DQN agent that decides whether to charge or discharge an electric vehicle according to hourly changing price to sell or buy. The price pattern also varies from day to day. The goal of this work is to schedule the optimal charging/discharging actions so that it can save money.

The state contains past n-step price records, current energy level in battery, hour, etc., like below:

$$
s_t = \{ p_{t-5}, p_{t-4}, p_{t-3}, p_{t-2}, p_{t-1}, E_t, t \}
$$

What I'm wondering is whether this is a partial observable situation or not, because the agent can only observe past n-step prices rather than knowing every price at every time step.

Can anyone comment on this issue?

If this is the partial observable situation, is there any simple way to deal with it?

An environment is partially observable if the agent cannot fully observe the current state but it only partially observes it. More specifically, in fully observable MDPs (FOMDPs), the agent knows the current state of the environment, which can or not (for example, depending on whether the state is Markov or not) contain all the information required to theoretically take the optimal action. In a partially observable MDPs (POMDPs), the state is not available or observable, so the agent might not possess all the required information to theoretically take the optimal action.

For example, in the game of poker, there are hidden and observable cards. In this game, the state of the environment could be defined as all the hands of all players, the cards on the table and the next cards that will be drawn from the deck until the end of the round. If a player had access to all this information, then the environment would be fully observable, but, usually, this information is not available, so poker is usually considered a partially observable game.

You defined the state as follows

$$s_t = \{ p_{t:t-n}, E_t\}$$

So, if you have access to the past $n$ price records, $p_{1:n}$, and the energy level, $E_t$, then your environment is fully observable. However, this does not mean that it is a Markov environment, that is, intuitively, that your current state is sufficient to theoretically take the optimal action.

To conclude, partial observability depends on the definition of state in a specific problem. The concept of Markov property is also related to the concept of partial observability. However, in the case of MDPs, the concept of Markov property has a specific definition: given the current state and an action, the probability of the next state is conditionally independent of all previous states and actions or, more formally

$$
P(S_{t+1} \mid S_t, A_t) = P(S_{t+1} \mid S_t, S_{t-1}, \dots, S_1, A_t, A_{t-1}, \dots, A_1)
$$

where $S_{t+1}$ is the next state, $S_t$ the current state, $A_t$ the current action, $S_{t-1}, \dots, S_1$ the previous states and $A_{t-1}, \dots, A_1$ the previous actions.


Meaning of the statement $\forall x \exists y \forall z (z \neq y \iff f(x) \neq z)$ I need to understand the logic of below FOL statement.
Can someone help?

$$
\forall x \exists y \forall z (z \neq y \iff f(x) \neq z)
$$

Does this imply that x, y and z cannot be same or f(x) has no value?

The statement is

"for all $x$, there exists a value of $y$ such that for all $z$,
$z\neq y$ if and only if $z \neq f(x)$".

This can be simplified:
$$\begin{align}
& & \forall x \exists y \forall z (z\neq y \iff z \neq f(x))\\
&\implies & \forall x \exists y \forall z (z=y \iff z = f(x))\\
&\implies & \forall x \exists y \forall z (y = f(x))\\
&\implies & \forall x \exists y (y = f(x))\\
\end{align}$$

If we denote the set of all values of $x$ by $X$ and the set of all values of $y$ by $Y$, then this tells us that the function $f$ maps every $x$ in $X$ to a $y$ in $Y$.  That is, $f: X \to Y$.


Is it meaningful to give more weight to the result of monte carlo search with less turn win? I'm programming on Connect6 with MCTS.

Monte Carlo Tree Search is based on random moves. It counts up the number of wins in certain moves. (Whether it wins in 3 turns or 30 turns)

Is the move with less turns more powerful than the move with more turns?(as mcts just sees if it's win or not -- not considering the number of turns it took to win) And if so, is it meaningful to give bigger weight to the one with less turn win?

Traditionally (when not considering your idea), the evaluation function for terminal game states would be implemented to return $1$, $0$, or $-1$ for wins, draws, or losses, respectively.

Changing that in a naive/straightforward way to make short-term wins more rewarding, long-term wins less rewarding, short-term losses more negative, and long-term losses less negative can be dangerous, it may change the objective that your agent is ultimately optimizing for (i.e. may lose the guarantee of converging towards optimal play given an infinite amount of time) if not done very carefully.

There is definitely value in considering the idea though, especially because in the Play-Out phase of MCTS, trajectories of (semi-)random moves introduce uncertainty in the evaluations at the end of those simulations, and this uncertainty increases as the length of the trajectories increases (due to increased number of uninformed decisions being made along the trajectory). Note that it is especially important to take into consideration here the number of moves played in the Play-Out phase, not necessarily including the number of moves made in the Selection phase (which are selected according to a much more informed strategy).

One paper I know of that investigates ideas along these lines is "Quality-based Rewards for Monte-Carlo Tree Search Simulations".


LDA performs much better than other methods In a final project in diagnosing ADHD using Machine Learning we obtained parameters from real patients. We used this data and got much higher success rates in LDA than in SVM and Naive Base, we had only 100 examples in our training set. We are wondering why LDA specifically succeeded much more than the others?

It would be hard to tell if you don't provide what kind of data/problem you are working on, but LDA works well when data that are grouped in gaussian blobs surrounding centroids while vanilla SVM works well when the data is almost linearly separable and naive bayes works well when your features are relatively independent of each other.


How can I train a neural network if I don't have enough data? I have created a neural network that is able to recognize images with the numbers 1-5. The issue is that I have a database of 16x5 images which ,unfortunately, is not proving enough as the neural network fails in the test set. Are there ways to improve a neural network's performance without using more data? The ANN has approximately a 90% accuracy on the training sets and a 50% accuracy in the test ones.

Code:

clear
graphics_toolkit("gnuplot")
sigmoid = @(z) 1./(1 + exp(-z));
sig_der = @(y) sigmoid(y).*(1-sigmoid(y));

parse_image;   % This external f(x) loads the images so that they can be read.
%13x14
num=0;
for i=1:166
  if mod(i-1,10)<=5 && mod(i-1,10) > 0
    num=num+1;
    data(:,num) = dlmread(strcat("/tmp/",num2str(i)))(:);
  end
end

function [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i)
  mid_layer(:,1)=sum(w1.*data(:,i));
  mid_layer(:,2)=sigmoid(mid_layer(:,1));
  last_layer(:,1)=sum(mid_layer(:,2).*w2);
  last_layer(:,2)=sigmoid(last_layer(:,1));
  exp_res=rem(i,5);
  if exp_res==0
    exp_res=5;
  end
  exp_result=zeros(5,1); exp_result(exp_res)=1;
  cost = exp_result-last_layer(:,2);
end

function [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i)
  delta(1:5) = cost;
  delta(6:20) = sum(cost' .* w2,2);
  w2 = w2 + 0.05 .* delta(1:5) .* mid_layer(:,2) .* sig_der(last_layer(:,1))';
  w1 = w1 + 0.05 .* delta(6:20) .* sig_der(mid_layer(:,1))' .* data(:,i);
end

w1=rand(182,15)./2.*(rand(182,15).*-2+1);
w2=rand(15,5)./2.*(rand(15,5).*-2+1);

for j=1:10000
  for i=[randperm(85)]
    [cost, mid_layer, last_layer] = forward(w1,w2,data,sigmoid,i);
    [w1, w2] = backprop(w1,w2,mid_layer,last_layer,data,cost,sig_der,sigmoid,i);
    cost_mem(j,i,:)=cost;
  end
end

You can synthetically increase the number of samples. For example with augmentation or unsupervised adaption (Self-training). With augmentation you grant the system way more robustness so i would really recommend this. For example this github. The problem with such small database sizes is that your test-set is also very small and you cannot test properly if your network generalizes well, or just overfits.

You can try transfer learning with another larger network to adapt those feature extractors and use them on ur problem. That may work better than training a new one from scratch with so less labeled images. Hope i could help at least a little, stay tuned.


Is there a limit of minimum error for a particular training dataset in artificial Neural Network? In error-based learning using gradient descent, if I give you a training dataset, then can you find the minimum error after training? And the minimum error should be true for all architectures of a neural network? Consider you will use MSE for calculating the error. You can choose anything you want other than my specified condition. It's like no matter how you change your network you can never cross the limit.

In some sense yes, in some sense no. The "error" word you are using is not defined in the question. The network can have two type of errors, quite different - generalization error and loss (training error). The lower bound of loss on the dataset obviously depend on the network architecture, because loss is value produced by network as a whole, and some network will be able to produce exactly zero loss (which is likely be overfitting, wich is bad).

The generalization error is how well network generalize form training dataset to potentially infinite (or very big) amount of data samples "in the wild", so generalization error can not be calculated precisely, only estimated. The lower bound on generalization error is studied in Statistical Learning. It always depend on the complexity of the classifier (in our case architecture of network). Statistical Learning theory produce estimations like VC generalization bound which connect generalization error with size of the dataset and complexity of the classifer (netwroks in our case) space. However for specific dataset it probably don't make sense to talk about generalization bound outside of context of complexity of classifier space, because if there is no restriction we can always construct bigger network which would have lesser generalization error (for example by pretraning bigger network on bigger dataset)
To put it short you have to put restriction on complexity of network space to be able estimate minimum generalization error (and that probably wouldn't be useful anyway, because it wouldn't say anything about if gradient descent would reach that error)


Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter? Background
From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or classification layer) becomes more separate. As such, we can then apply a simple classifier(s) to the representation to chop up the regions where the different classes exist (as shown by this blog post).
If this is true and say we have some noisy data where the classes are not easily separable, would it make sense to push the input to a higher dimension, so we can more easily separate it later in the network?
For example, I have some tabular data that is a bit noisy, say it has 50 dimensions (input size of 50). To me, it seems logical to project the data to a higher dimension, such that it makes it easier for the classifier to separate. In essence, I would project the data to say 60 dimensions (layer out dim = 60), so the network can represent the data with more dimensions, allowing us to linearly separate it. (I find this similar to how SVMs can classify the data by pushing it to a higher dimension).
Question
Why, if the above is correct, do we not see many neural network architectures projecting the data into higher dimensions first then reducing the size of each layer thereafter?
I learned that if we have more hidden nodes than input nodes, the network will memorize rather than generalize.

To better understand this you should think in terms of capacity. Capacity is a theoretical notion that shows how much information your network can model.
The capacity of a network (given sufficient training) ties in directly with the bias/variance tradeoff:

too little capacity and your network isn't able to learn the complex relationships in the data.
too much capacity and your network has the capability of learning the noise in the dataset, besides the useful relationships.

At some point a network reaches a point where it has a high enough capacity to memorize the whole training set!
Now, by increasing the number of hidden neurons you essentially increase the capacity of your network. If the network already has enough capacity to learn the problem, then by increasing the neurons you are giving the network the capability of overfitting more easily.
Note: all the above assume that the network is trained sufficiently (i.e. no early stopping, etc).


How do LSTM or GRU gates learn to specialize in their desired tasks? While I was studying the equations for the computation inside GRU and LSTM units, I realized that although the different gates have different Weight matrices, their overall structure is the same. They are all dot products of a weight matrix and their inputs, plus bias, followed by a learned gating activation. Now, the difference between computation depends on the weight matrices being different from each other, that is, those weight matrices are specifically for specializing in the particular tasks like forgetting/keeping etc.

But these matrices are all initialized randomly, and it seems that there's no special tricks in the training scheme to make sure these weight matrices are learned in a manner that the associated gates specialize in their desired tasks. They are all random matrices that kept getting updated with gradient descent.

So how does, for example, a forget gate learn to function as a forgetting unit? Same question applies to others as well. Am I missing a part of the training for these networks? Can we ever say that these units learn truly disentangled functions from each other?

It comes down to the order they're computed in, and what they're used in. I will be referring to the LSTM in this answer.

Looking at the forget gate, you can see that it has the ability to manipulate the cell state. This gives it the ability to force a forget. Say (after training) it sees a super important input that means some previous data is irrelevant (say, like a full stop). This forget gate, while it might not force a forget, has the ability to force one, and will likely learn to.

The input gate ultimately adds to the cell state. This gate doesn't have direct influence over the cell state (it can't make it 0, like the forget gate can), but it can add to it and influence it that way. So it is an input gate.

The output gate is used to interpret the hidden state, and get it ready to be combined with the cell state for a final output at that time step.

While these gates all use sigmoid functions, are all initialised randomly and have the same dimensionality, what their output is used in, and the order they're computed in gives them a certain role to play. Initially, they won't conform to this role, but logically as they learn, they likely will.


Intuitively understanding translational invariance in CNNs I'm currently in the process of learning about using CNNs in image recognition. Many of the different resources I read that were explaining the motivation referred to the fact that these networks are (to some degree) translationally invariant. My understanding is as follows:

Fully connected networks are ill-suited to image recognition because of the high dimensionality of the data and especially because they do not preserve the spatial relationships between pixels. I care about the state of the pixels surrounding pixel X as well as X itself.
CNNs remedy this because they look at the image in 2 dimensions so that surrounding pixels are being considered as well (the kernel does this). However, while I care about what is around X, I don't care where X is. So, I apply the kernel the same everywhere and I make a bunch of layers so that I can get a bunch of kernels.
I also use pooling. This reduces the data dimensions but also adds some of that translation invariance.

So, my question is: What feature of the CNN is causing the invariance? I saw some explanations saying it was a result of the maps basically activating when a certain feature shows up, regardless of where. Other said that the pooling meant that if a horizontal line, for instance, showed up in one place vs. a pixel over, the pooling would activate the same for both. It seems like the first reason would be totally sufficient and the second not really adequate but I could also see it being both.

I also read a paper (https://arxiv.org/pdf/1606.09549.pdf) about fully-convolutional networks. In section 2.1, the authors explain that they intentionally chose to make it fully-convolutional because that allowed them to "compute the similarity at all translated sub-windows on a dense grid in a single evaluation."

That sentence makes me think that it is more the first explanation. Anyways, I hope to gain a better intuitive understanding of how the different parts of the CNN come together to work particularly well on images. Thanks!

The translational invariance comes from the fact that the kernels are applied everywhere. Basically the kernels are "translated", i.e. shifted, and that means the feature they detect are detected everywhere.

Pooling may help with that in some cases, because in the end you need a dog at the right side to activate the same neuron as a dog on the left, so there needs to be funnelling of the information, but it is not strictly necessary. You could also imagine a CNN with a dog kernel in a late layer, which just recognises dogs everywhere, with no pooling necessary.

Fully connected networks are wasteful in that they don't share weights. To get the necessary translational invariance you therefore need to learn pretty much the same weights at all positions. But they do preserve the spatial relationship between pixels.

Pooling reduces the dimensionality, but it also forgets the precise position of a feature. This might help with generalisation, but I'm not sure whether that's a big factor.


How can I model any structure for a neural network? Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the hidden layer of a network instead of a usual fully connected bipartite graph.
For instance my neural network would look something like this:

Basically I want the ability to create any structure in my hidden layer as long as it remains a DAG [add any edge between any node regardless of layers]. I have tried creating my own library to do so but it proved to be much more tedious than anticipated therefore I am looking for ways to do this on existing libraries such as Keras, pytorch, or tensorflow.

You seem to be seeking an implementation of a Residual Neural Network (https://en.m.wikipedia.org/wiki/Residual_neural_network), or ResNET for short. If you want some premade networks, the module tf.keras.applications.resnet from tensorflow (do check TF's documentation) might help you.


Representation of real numbers in Genetic Algorithm Take a look at section 2.2.2 of this book (from Page-15 to 16).

  2.2.2 Representation and Evaluation

  $$max f (x)= x sin(10πx)+2.0 ... ... ... (2.8)$$
  $$s.t. −1 ≤ x ≤ 2$$

  We can use a real number, in the range $[−1,2]$, to represent a solution in Eq. $2.8$ directly. Many operators can handle real number representation. But we use the binary code or binary representation here for two reasons. GAs were originally proposed to be binary code to imitate the genetic encoding of natural organisms. On the other hand, binary code is good for pedagogy. A binary chromosome is necessary to represent a solution x in the scale $[−1,2]$. The same holds for the binary representation of real numbers in a computer.

  In binary code, we cannot represent a real number completely correctly, so a
  trade-off is necessary. A tolerance needs to be deﬁned by the user, which means the errors below the tolerance are extraneous. If we divide the deﬁnition domain into $2^1 = 2$ parts evenly and select the smallest number in the parts to represent any number in the division, we can only represent $−1$ and $0.5$ by $0$ and $1$ respectively. $2^2 = 4$ divisions make the $00$, $01$, $10$, and $11$ represent $−1$, $−0.25$, $0.5$, and $1.25$, respectively. The larger division number we select, the less error there is in representing a real number on binary code. Suppose we use $100$ binary codes to represent a real number in the range $[−1,2]$; the maximum error is $\frac{3}{2100} ≈ {2.37}^{−30}$, which would be satisfactory for most users. In this way, we can represent a real number with any accuracy requirements.

  In this problem, we use $l = 12$ binary codes to represent one real number as follows, which constitutes a chromosome to be evolved.

Actually, I haven't understood this text. I know that binary numbers are already able to represent fractions.

So, what are they talking about?

The authors want to use a binary genome which contains specific interpretation/mapping to a fixed precision real number. They could use one or more existing schemes, but these typically won't represent the exact precision and range of real numbers they want to use in the example problem.

They use their own encoding scheme and explain it in the text for completeness. Any similarities between their scheme and other binary representations of real numbers are to be expected, and if you understand them, it will help understand what they are doing here too.

GAs don't have to use this sort of scheme for real numbers. Often they use standard floating-point representation as separate elements. But the authors want a binary code here to help teach some basic concepts, which is what they mean by "on the other hand, binary code is good for pedagogy"


Predict time series from initial non-time dependant parameters I'm trying to create an algorithm (neural network) that is able to predict a time series from a set of different parameters that are not given through time. Let's say I have a plane flying under the following conditions:

Parameters
Value

Angle of attack
8 degrees

Lateral angle
12 degrees

Wind speed
-20 m/s

Plane speed
200 m/s

From this point, I would like to predict the translational velocities in x-y-z axis for the next 2-3 seconds.
In order to train my model, I have a data base with different initial situations (input) and different motions of the plane (desired output) linked to their initial situation. Therefore, I want to train my model to predict these motions mentioned before, based only on the initial situation described.
In other words, the basics of what I'm trying to do could be summed up as the following:
Parameters describing the initial situation -> Model -> Time series of translational velocities.

I don't think you need a recurrent neural network for this. Why not just train a feedforward model with angle of attack etc as input and translation velocities as output? The size of the output will depend on how frequently you want updates to the velocities. e.g. if the update rate is 0.5 seconds and the network is predicting 2 seconds in advance it could have 12 outputs, being x,y,z values every 0.5 seconds.
You may need to massage the way you represent the outputs to find the best approach. For example, will the network be predicting absolute velocities or just the initial velocities plus changes to velocity over time? If the latter you can always get back to absolute velocities.


Why do we resize images before using them for object detection? In object detection, we can resize images by keeping the ratio the same as the original image, which is often known as "letterbox" resize.
My questions are

Why do we need to resize images? If we resize images to have all the same dimensions, given that some original images are too long vertically or horizontally, we will lose a lot of features in those images.

If the "letterbox" method is better than "normal resize" (i.e. without keeping the aspect ratio, e.g. the result of the application of OpenCV's resize function with the parameter interpolation set to cv2.INTER_AREA), why don't people apply it in the classification task?

There are different questions and even different lines of thought here. Let's go through them
On resizing

Why do we need to resize? To fit the network input which is fixed when nets are no Fully Convolutional Networks (FCN)
What if my net is FCN? Still makes sense to resize to bound the dimension of the input features you want to detect (a person on a small image VS big image). Take into account that the kernel sizes do not vary although the image size does.

On keeping aspect ratio (or letterbox as some people like to say)

Why to keep aspect ratio? This is more of a philosophical question. It is believed that keeping the aspect ratio helps the nets to learn the natural variability in object sizes (say a person bounding box can not be super hight and super thin, because that would be a street light).

Why not to keep aspect ratio? If you resize without keeping the aspect ratio and the aspect ratio distortion is not mega super very huge, the networks will still learn. In other words, if your input images don't have crazy aspect ratios, then there is no difference between adding or not a bit of distortion. In fact, sometimes it will even act as a regularization or augmentation.

Conlusion
As long as your application is not too specific and your input images aspect ratios are bounded (this is, if  you train with images from any regular camera), you should not worry too much about this.
When to worry about this? When you train with huge vertical or horizontal images or if you train with images taken from some very specific devices like geophisical, radio, or optical sensors. In these cases you should pay special attention on how to resize or split an image. For example, with a recording of a radio sensor, if you resize with aspect ratio deformation, a wave from a specific frequency would transform to another because of the sine wave warp)


What is eager learning and lazy learning? What is the difference between eager learning and lazy learning?
How does eager learning or lazy learning help me build a neural network system? And how can I use it for any target function?

What is eager learning or lazy learning?

Eager learning is when a model does all its computation before needing to make a prediction for unseen data. For example, Neural Networks are eager models.
Lazy learning is when a model doesn't require any training, but all of its computation during inference. An example of such a model is k-NN. Eager learning is also known as instance-based learning [1, 2, 3].

How does eager and lazy learning help me build a neural network system? And how can I use it for any target function?

To answer your second question, you can't employ lazy learning to train a neural network, because they are inherently eager models.


How to apply Q-learning when rewards is only available at the last state? I have a scheduling problem in which there are $n$ slots and $m$ clients. I am trying to solve the problem using Q-learning so I have made the following state-action model.
A state $s_t$ is given by the current slot $t=1,2,\ldots,n$ and an action $a_t$ at slot $t$ is given by one client, $a_t\in\{1,2,\ldots,m\}$. In my situation, I do not have any reward associated with a state-action pair $(s_t,a_t)$ until the terminal state which is the last slot. In other words, for all $s_t\in\{1,2,\ldots,n-1\}$, the reward is $0$ and for $s_t=n$ I can compute the reward given $(a_1,a_2,\ldots,a_n)$.
In this situation, the Q table, $Q(s_t,a_t)$, will contain only zeros except for the last row in which it will contain the updated reward.
Can I still apply Q-learning in this situation? Why do I need a Q table if I only use the last row?

Having only a non-zero reward at the very end is not uncommon. When rewards are sparse, it becomes a bit harder to learn compared to having lots of different rewards along the way, but for your problem, the goal state is always reached, so that should not be a problem. (The real problem with sparse rewards is that, if an agent can do a lot of exploration without every finding the goal, it essentially receives no feedback and will behave randomly, until it happens to stumble upon the very rare reward state.)
What concerns me more about your problem is that the final reward depends not just on the last state visited, but also on the chain of actions taken so far. That means that, to make this a proper MDP, you need to keep the chain of action in the state. So, your state would be something of the type $(s_k, [a_1, a_2, \ldots, a_{k-1}])$.
This kind of combinatorial problem is not what RL is really great at. RL is really good when the state and action together give a lot of information about the next state. Here it seems that, in your formulation, the next state is independent of the previous action.
Instead of seeing this as a RL problem, you might want to express this as sequences of actions with an associated reward, and look at it as a combinatorial optimization problem.


Is there a car detection software written in Tensorflow or Keras with Python? For a current project demo, I'm searching for a car detection neural network in Python written in TF/Keras (or any other type, as long as it has no C++ dependencies). Later on, I gonna write my own, but, for now, I need a working substitute. It should be easy to use and implement in my own Python code, even better if it's already trained. I searched the whole GitHub for this, but I can't find anything that would fit my needs.

Does someone know where to find this type of code? And are there ones for pedestrian detection, too?

I recall seeing several data sets on Kaggle on the topic of car images. Usually these data sets have associated with them user uploaded python kernels that do either classification( what make of car) or detection ( show a bounding box on the car).
I would recommend you go to Kaggle then select data sets and do a search for 'cars'. I did a quick look and found a data set at https://www.kaggle.com/jessicali9530/stanford-cars-dataset that has 18 kernels associated with it. There well may be many more data sets available. Good luck in your search!


Why do Bellman equations indirectly create a policy? I was watching a lecture on policy gradients and Bellman equations. And they say that a Bellman equation indirectly creates a policy, while the policy gradient directly learns a policy. Why is this?

Policy gradient methods directly learn parameters of a policy function, which is a mapping from states to actions. For example, $p(s, a)$ can denote a function which takes a state $s$ and an action $a$ as input, and returns a probability of taking action $a$ in state $s$ as output (equivalently it could just take $s$ as input, and output a vector or a distribution of probabilities for all actions). Such a function will often have some parameters (for example, a vector of weights if it's a linear function of the state, or matrices of weights in the case of a neural network). Policy gradient methods directly modify the function by modifying (learning) the parameters of that function.

The Bellman equation is typically used to learn a function that computes different kinds of values, such as $Q(s, a)$ values, which can intuitively be understood to denote all the future rewards that we expect to get if we take action $a$ in state $s$ (and follow a certain policy, typically "the optimal policy" afterwards until the end of time). Such a $Q(s, a)$ function that we learn is not a directly learned policy, because it is not a direct mapping from current states to actions, it doesn't directly tell us which action to take.

It can still be used indirectly to create a policy. For example, once such a $Q(s, a)$ function is learned, the greedy policy can be easily implemented by looping over all possible actions, and taking the action that gives the greatest $Q(s, a)$ output. This is indirect because the learned function does not directly give you the actions, but only provides information which you can convert into actions with some extra processing.


How to use LSTM to generate a paragraph A LSTM model can be trained to generate text sequences by feeding the first word. After feeding the first word, the model will generate a sequence of words (a sentence). Feed the first word to get the second word, feed the first word + the second word to get the third word, and so on.

However, about the next sentence, what should be the next first word? The thing is to generate a paragraph of multiple sentences.

Take the sentence that was generated by your LSTM and feed it back into the LSTM as input. Then the LSTM will generate the next sentence. So the LSTM is using it's previous output as it's input. That's what makes it recursive. The intial word is just your base case. Also you should consider using GPT2 by open AI to do this. It's pretty impressive. https://openai.com/blog/better-language-models/


Which hyperparameters in neural network are accesible to users adjustment I am new to Neural Networks and my questions are still very basic.
I know that most of neural networks allow and even ask user to chose hyper-parameters like:

amount of hidden layers
amount of neurons in each layer
amount of inputs and outputs
batches and epochs steps and some stuff related to back-propagation and gradient descent

But as I keep reading and youtubing, I understand that there are another important "mini-parameters" such as:

activation functions type

activation functions fine-tuning (for example shift and slope of sigmoid)

whether there is an activation funciton in the output

range of weights (are they from zero to one or from -1 to 1  or -100 to +100 or any other range)

are the weights normally distributed or they just random

etc...
Actually the question is:
Part a:
Do I understand right that most of  neural networks do not allow to change those "mini-parameters", as long as you are using "readymade" solutions?
In other words if I want to have an access to those "mini-parameters" I need to program the whole neural network by myself or there are  "semi-finished products"
Part b:(edited)
For someone who uses neural network as an everyday routine tool to solve problems(Like data scientist), How common and how often do those people deal with fine tuning things which I refer to as "mini-parameters"? Or those parameters are usually adjusted by a neural network developers who create the frameworks like pytorch, tensorflow etc?
Thank you very much

In general, many of the parameters you mentioned are called hyperparameters. All hyperparameters are user-adjusted (or user-programmed) in training phase. Some hyperparameters are:

learning rate,
batch size,
epochs,
optimizer,
layers,
activation functions etc.

To answer your (a) part of your question, there are obsiously many frameworks and libraries, for example in python; TensorFlow, pytorch and so on. You might never create a net from the very beginning; maybe only in order to understand the forward and backpropagation algorithms. When we call from scatch networks, we mean that these networks are trained from scratch, with learnable weights and chosen hyperparameters; with no transfer learning.
To answer your (b) part of your question, I can understand from it that you mean when a net is good enough. Dependently of your data, of course, a neural network is good enough, when it is trained adequately on them. That is, you should be aware of overfitting, underfitting, and in general of the model you are trying to train with all its parameters and hyperparameters.
Since, you are at the very beginning with Machine Learning, I propose you read some books, in order to get everything needed, in terms of Mathematical and Computer Science aspects.


Why is Standard Deviation based on L2 Variance and not L1 Variance Standard deviation and variance are in statistics but the formula for variance is somehow related to the L1 and L2.

Mathematically (L2 in machine learning sense),
$$Variance = \dfrac{(X_1-Mean)^2+..+(X_n-Mean)^2}{N}$$
and,
$$Standard\ Deviation= \sqrt(Variance)$$

Why shouldn't it be (L1 in machine learning sense):
$$Variance = \dfrac{|X_1-Mean|+..+|X_n-Mean|}{N}$$
and,
$$Standard\ Deviation= Variance$$

I've found the answer, the L2 is Standard Deviation, and the L1 is Mean Deviation. Standard deviation describes the variation better and the values are always different on different sets of X while the Mean Deviation gives the same values sometimes.

  *Footnote: Why square the differences? If we just add up the differences from the mean ... the negatives cancel the positives:

  standard deviation why a       (4 + 4 − 4 − 4) / 4  = 0 So that won't work.
  How about we use absolute values?

  standard deviation why a       (|4| + |4| + |−4| + |−4|) / 4  =  (4 + 4 + 4 +
  4) / 4 = 4 That looks good (and is the Mean Deviation), but what about
  this case:

  standard deviation why b       (|7| + |1| + |−6| + |−2|) / 4  =  (7 + 1 + 6 +
  2) / 4 = 4 Oh No! It also gives a value of 4, Even though the differences
  are more spread out.

  So let us try squaring each difference (and taking the square root at
  the end):

  standard deviation why a      √( (42 + 42 + 42 + 42) / 4 ) = √( 644) = 4
  standard deviation why b      √( (72 + 12 + 62 + 22) / 4 ) = √( 904) = 4.74...

Reference:
https://www.mathsisfun.com/data/standard-deviation.html


Can I calculate the training performance of GPUs by comparing their specification? I am currently using Nvidia GTX1050 with 640 CUDA cores and 2GB GDDR5 for Deep Neural Network training. I want to buy a new GPU for training, but I am not sure how much performance improvement I can get.

I wonder if there is a way to roughly calculate the training performance improvement by just comparing GPUs' specification?

Assuming all training parameters are the same. I wonder if I can roughly assume the training performance improvement is X times because the CUDA core number and memory size increased X times?

For example, Is RTX2070 with 2304 CUDA cores and 8GB GDDR6 roughly 4 times faster than GTX1050? And is RTX2080Ti with 4352 CUDA cores and 11GB GDDR6 roughly 7 times faster than GTX1050?

Thanks.

A lot matters when it comes to comparison of GPU's, I will give you a broad overview of the matter (it is not possible to go into exact details as huge number of factors are actually involved):

Cores - Number of CUDA cores increases means the parallelism has increased, thus multiple calculations can be done in parallel but is of no significance if your algorithm is inherently sequential. Then CUDA cores will not matter. Your library will parrallelize what it can parallelize and will take that many CUDA cores only, the rest will remain idle.
Memory - Memory is useful if you are working on data whose one instance requires huge memory (like pictures). Thus with greater memory you can load greater amount of data at the same time and the cores will process on that. If memory is too low then cores want data but it is not getting it (basically data available in RAM's is the fuel while cores are the engine, you cannot run jets on fuel tank the size of car, it will consume time to constantly refill the fuel). But according to convention of Machine Learning one should load only small mini-batches at a time.
Micro-architecture - Lastly, architecture matters. I do not exactly know how but NVIDIA's RTX is faster for deep learning than GTX. NVIDIA has two affordable architectures (Pascal - GTX and Turing - RTX). Thus even for exactly same specs Turing architecture will run faster for deep learning. But for more details you can explore NVIDIA's website on what architecture specialises in what. For example NVIDIA P-series is good for CAD purposes. Also there are some very high end GPU's using Tesla architecture.

So AFAIK, these are the factors that matter. The library you will be using also matters, as a lot depends on how the library unrolls your program and maps them on several GPU's. Also related these 2 answers I previously gave:

CPU preferences and specifications for a multi GPU deep-learning setup

Does fp32 & fp64 performance of GPU affect deep learning model training?

Hope this helps!


Can a second network take as input the weights of a first network and help training the first network? I understand that as a network learns about an output with regards to an input, weights are updated according to how wrong the guess was for that node. So, over time, the weights move in the "direction" towards the correct value.

Is it possible to use a separate neural network that takes as input the weights of the first network while it trains to try approximating that "direction" and, in effect, pushing the weights in that direction faster?

That is similar to a chicken-and-egg problem; If the second network could figure out the direction and learn faster, why couldn't the first network do that too and learn even faster.

However, I would recommend looking up Actor Critic Methods. Here there are two networks but one (the critic) evaluates the other (the actor) to head it in the right direction.


How to get more accuracy of the logistic regression model? I am working on a Baby Crying Detection model using logistic regression.
Out of $581$ audios, $222$ are of a baby crying. Each audio is of $5$ seconds.
what I have done is convert each audio into numbers. and those numbers go into a .csv file. so first I took $100$ samples from each audio, then $1000$ samples, and then all $110250$ samples into a .csv file, and at the end of each of them was a number 1 (crying) or 0 (not crying). Then I trained the model using logistic regression from that .csv file.
The Problem I m facing is that with $100$ samples the 64% accuracy on each audio,  while with 1000 samples and 110250 samples(Full dataset) it reaches to 66% accuracy only. How can I improve the accuracy of my model to upto 80% using logistic regression.
I can only use simple logistic regression because I have to deploy the model on Arduino.

Try Rectification
Improve the features available to your model, Remove some of the NOISE present in the data.

In audio data, a common way to do this is to smooth the data and then rectify it so that the total amount of sound energy over time is more distinguishable.

# Rectify the audio signal
audio_rectified = audio.apply(np.abs)

You can also calculate the absolute value of each time point. This is also called Rectification because you ensure that all time points are positive.

Smooth your data by taking the rolling mean in a window of say 50 samples

audio_rectified_smooth = audio_rectified.rolling(50).mean()

Calculating the envelope of each sound and smoothing it will eliminate much of the noise and you have a cleaner signal.

Calculate Spectrogram

Calculate a spectrogram of sound(i.e combining of windows Fourier transforms). This describes what spectral content (e.g., low and high pitches) are present in the sound over time. there is a lot more information in a spectrogram compared to a raw audio file. By computing the spectral features, you have a much better idea of what's going on.

this is similar to how we calculate rolling mean :

We calculate multiple fourier transforms in a sliding window to see how it changes over time.For each timepoint, we take a window of time around it, calculate a fourier transform for the window, then slide to the next window . The result is a description of the fourier transform as it changes throughout the time-series called a short-time fourier transform or STFT.

Choose a windows size and shape
At a timepoint, calculate the FFT for that window
Slide the window over by one
Aggregate the results

Calculating the STFT

We can calculate the STFT with librosa $\rightarrow$ import librosa as lr
There are several parameters we can tweak (such as window size)
For our purposes, we'll convert into decibels which normalizes the average values of all frequencies.
We can then visualize it with the specshow() function

This how you can calculate the STFT
# Calculating the STFT
# Import the functions we'll use for the STFT
from librosa.core import stft, amplitude_to_db
from librosa.display import specshow

# Calculate our STFT
HOP_LENGTH = 2**4
SIZE_WINDOW = 2**7
audio_spec = stft(audio, hop_length=HOP_LENGTH, n_fft=SIZE_WINDOW)

# Convert into decibels for visualization
spec_db = amplitude_to_db(audio_spec)

# Visualize
specshow(spec_db, sr=sfreq, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)

Try Spectral feature engineering
you can also perform the Spectral feature engineering on you baby audio data

since each time-series has a different spectral pattern.
We can calculate these spectral patterns by analyzing the spectrogram.
For example, spectral bandwidth and spectrum centroids describe where most of the energy is at each moment in time.

# Calculate the spectral centroid and bandwidth for the spectrogram
bandwidths = lr.feature.spectral_bandwidth(S=spec)[0]
centroids = lr.feature.spectral_centroid(S=spec)[0]

# Display these features on top of the spectrogram
ax = specshow(spec, x_axis='time', y_axis='hz', hop_length=HOP_LENGTH)
ax.plot(times_spec, centroids)
ax.fill_between(times_spec, centroids - bandwidths / 2,centroids + bandwidths / 2, alpha=0.5)

Now you can Combine spectral(spec $\rightarrow$ spectral dataframe) and temporal features in a classifier
centroids_all = []
bandwidths_all = []

for spec in spectrograms:
    bandwidths = lr.feature.spectral_bandwidth(S=lr.db_to_amplitude(spec))
    centroids = lr.feature.spectral_centroid(S=lr.db_to_amplitude(spec))
    # Calculate the mean spectral bandwidth
    bandwidths_all.append(np.mean(bandwidths))
    # Calculate the mean spectral centroid
    centroids_all.append(np.mean(centroids))
# Create our X matrix
X = np.column_stack([means, stds, maxs, tempo_mean, tempo_max, tempo_std, bandwidths_all, centroids_all])

for your logistic regression models

One of the way to improve accuracy is by optimising the prediction probability cutoff scores generated by your logit model.
You can Normalize all your features to the same scale before putting them in a machine learning model.
Look for class imbalance in your data.
You can Optimize on other Metrics also such as Log Loss and F1-Score.
Tune the hyperparameters of your model. In the case of LogisticRegression, parameter $C$ is a hyperparameter.


How is the depth of filters of hidden layers determined? I am a bit confused about the depth of the convolutional filters in a CNN.
At layer 1, there are usually about 40 3x3x3 filters. Each of these filters outputs a 2d array, so the total output of the first layer is 40 2d arrays.
Does the next convolutional filter have a depth of 40? So, would the filter dimensions be 3x3x40?

Does the next convolutional filter have a depth of 40? So, would the filter dimensions be 3x3x40?

Yes. The depth of the next layer $l$ (which corresponds to the number of feature maps) will be 40. If you apply $8$ kernels with a $3\times 3$ window to $l$, then the number of features maps (or the depth) of layer $l+1$ will be $8$. Each of these $8$ kernels has an actual shape of $3 \times 3 \times 40$. Bear in mind that the details of the implementations may change across different libraries.
The following simple TensorFlow (version 2.1) and Keras program
import tensorflow as tf

def get_model(input_shape, num_classes=10):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=input_shape))
    model.add(tf.keras.layers.Conv2D(40, kernel_size=3))
    model.add(tf.keras.layers.Conv2D(8, kernel_size=3))
    model.add(tf.keras.layers.Flatten())
    model.add(tf.keras.layers.Dense(num_classes))

    model.summary()

    return model

if __name__ == '__main__':
    input_shape = (28, 28, 1)  # MNIST digits have usually this shape.
    get_model(input_shape)

outputs the following
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d (Conv2D)              (None, 26, 26, 40)        400
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 24, 24, 8)         2888
_________________________________________________________________
flatten (Flatten)            (None, 4608)              0
_________________________________________________________________
dense (Dense)                (None, 10)                46090
=================================================================
Total params: 49,378
Trainable params: 49,378
Non-trainable params: 0
_________________________________________________________________

where conv2d has the output shape (None, 26, 26, 40) because there are 40 filters, each of which will have a $3\times 3 \times 40$ shape.
The documentation of the first argument (i.e. filters) of the Conv2D says

filters – Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution).

and the documentation of the kernel_size parameter states

kernel_size – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.

It doesn't actually say anything about the depth of the kernels, but this is implied from the depth of the layers.
Note that the first layer has $(40*(3*3*1))+40 = 400$ parameters. Where do these numbers come from? Note also that the second Conv2D layer has $(8*(3*3*40))+8 = 2888$ parameters. Try to set the parameter use_bias of the first Conv2D layer to False and see the number of parameters again.
Finally, note that this reasoning applies to 2d convolutions. In the case of 3d convolutions, the depth of the kernels could be different than the depth of the input. Check this answer for more details about 3d convolutions.


How do evolutionary algorithms have advantages over the conventional backpropagation methods? How does employing evolutionary algorithms to design and train artificial neural networks have advantages over using the conventional backpropagation algorithms?

Unlike backpropagation, evolutionary algorithms do not require the objective function to be differential with respect to the parameters you aim to optimize. As a result, you can optimize "more things" in the network, such as activation functions or number of layers, which wouldn't be possible in the standard backpropagation.

Another advantage is that by defining the mutation and crossover functions, you can influence how the parameter search space should be explored.


How does a GCN handle new input graphs? Quick questions to see whether I understand GCNs correctly.
Is it correct that, if I have trained a GCN, it can take arbitrary graphs as input, assuming the feature size is the same?
I can't seem to find explicit literature on this.

Graph neural networks, of which GCNs are a specific type, are able to handle arbitrary graphs as input. GNNs operate first over "neighborhoods" of nodes to compute individual node representations and then optionally apply a pooling function to reduce these to a single graph-level representation that can be used in classification. This means that GNNs work "locally" and do not contain implicit assumptions about the graph topology.
For example, the update equation for a given node's representation in a GCN can be written as
$$h_v^{t+1} = \sigma\left({\bf W}^{t+1} \sum_{u \in \mathcal{N}_v} {\bf L}_{uv}~ h_u^t \right) $$
where $h_v^l$ is the representation of node $v$ at update $t$, $\sigma$ is an activation function, ${\bf W^{t}}$ is a weight matrix, ${\bf L_{uv}}$ is the value of the graph Laplacian (which is a matrix) at nodes $u$ and $v$, and finally $\mathcal{N}_v$ is the neighborhood of $v$. Looking at this expression it becomes clear that the value of the summation is always of the same dimension as $h$, no matter how you define the neighborhood $\mathcal{N}_v$. So you are correct that as long as the node representation size is static, the network can take arbitrary graphs as input.
I highly recommend this tutorial paper on GNNs, which I used when first learning about them. Section 2.3 specifically answers your question and discusses how things like cycles and non-positional graphs are handled.


What is the current state of the art in animal facial recognition? As far as I can tell, most work on facial recognition has been done in relation to human faces. Has any significant work been done for dogs, and are there any special challenges that would make it difficult to adapt existing work to this use case?

To clarify: I don't mean image classification, but segmentation. In other words, I refer to a model that can take an image that can have any number, including zero, of dogs in it, and that will produce bounding boxes around each dog's face.

In case of animal classification - trying to specify the animal, trying to tell cat from a dog - it should be similar to human faces. An algorithm such as SVM could be used in that case. Trying to tell two breeds of dog apart would be more challenging. I have come across this article explaining how they use facial recognition software for animals news. And I have also come across  this article where they write about a novel method for identifying individual cetaceans using facial features and symmetry on dolphins.


What is easier or more efficient to summarize voice or text? [DP/RN] If possible consider the relationship between implementation difficulty and accuracy in voice examples or simply chat conversations.

And currently, what are the directions on algorithms like Deep Learning or others to solve this.

Summarizing text is always going to be 'easier or more efficient' than voice simply because voice requires the additional step of converting to text.  That doesn't tell you anything about accuracy.

From an article published on June 1, 2017, Google’s speech recognition is now almost as accurate as humans:
"According to Mary Meeker’s annual Internet Trends Report, Google’s machine learning-backed voice recognition — as of May 2017 — has achieved a 95% word accuracy rate for the English language. That current rate also happens to be the threshold for human accuracy."

If you need this kind of accuracy check out Google's Cloud Speech API.  There is even a speech to text feature on the web page.

Given a speech-to-text conversion accuracy of 95%, voice will be 5% less accurate than text if everything else is equal but it usually isn't.  People generally write better text, such as in documents or emails, than when they speak unless of course they are giving a formal lecture, or talking in a formal meeting.  If one is analyzing text messages, Tweets, or threads found in typical informal forums, you will find very poor quality in grammar, spelling, vocabulary, and punctuation.  The answer to your question will depend on the source of your text.

In another article, dated November 13, 2017, Why 100% Accuracy Is Not Available With Speech Recognition Software Alone, the author gives some reasons, albeit for transcription software which has a special purpose, why there will always be some errors due to:

Speech Patterns and Accents - Regional variations exist, for example English speakers in Boston sound different than Kentucky.  How does the software handle slurred speech or when a person blends their words?
Grammar and Punctuation - speech recognition software doesn't know where a period, comma, or semi-colon belongs
Homonyms and unusual words - "Speech processing software can only recognize words and phrases that it has specifically been trained to recognize."
Ambient Noise, Overlapping Speech, and Number of Speakers

To address your last question about where the technology is going...
Four days ago a paper by Tom Young, Devamanyu Hazarika, Soujanya Poria, and Erik Cambria entitled Recent Trends in Deep Learning Based
Natural Language Processing was published which gives some of the answers.

From the 'Conclusion' section:
With distributed representation, various deep models have
become the new state-of-the-art methods for NLP problems.
Supervised learning is the most popular practice in recent
deep learning research for NLP. In many real-world scenarios,
however, we have unlabeled data which require advanced
unsupervised or semi-supervised approaches. In cases where
there is lack of labeled data for some particular classes or the
appearance of a new class while testing the model, strategies
like zero-shot learning should be employed. These learning
schemes are still in their developing phase but we expect deep
learning based NLP research to be driven in the direction of
making better use of unlabeled data. We expect such trend to
continue with more and better model designs. We expect to
see more NLP applications that employ reinforcement learning
methods, e.g., dialogue systems. We also expect to see more
research on multimodal learning [167] as, in the real world,
language is often grounded on (or correlated with) other
signals.

Finally, we expect to see more deep learning models whose
internal memory (bottom-up knowledge learned from the data)
is enriched with an external memory (top-down knowledge
inherited from a KB). Coupling symbolic and sub-symbolic AI
will be key for stepping forward in the path from NLP to natural
language understanding. Relying on machine learning, in
fact, is good to make a ‘good guess’ based on past experience,
because sub-symbolic methods encode correlation and their
decision-making process is probabilistic.


Does fp32 & fp64 performance of GPU affect deep learning model training? I am purchasing Titan RTX GPU. Everything seems fine with that except float32 & float64 performance which seems lower vis-a-vis some of its counter parts. I wanted to understand if single precision and double precision performance of GPU affect deep learning training or efficiency ? We work mostly with images, however not limited to that.

First off I would like to post this comprehensive blog which makes comparison between all kinds of NVIDIA GPU's.

The most popular deep learning library TensorFlow by default uses 32 bit floating point precision. The choice is made as it helps in 2 causes:

Lesser memory requirements
Faster calculations

64 bit is only marginally better than 32 bit as very small gradient values will also be propagated to the very earlier layers. But the trade-off for the gain in performance vs (the time for calculations + memory requirements + time for running through so many epochs so that those small gradients actually do something) is not worth it. There are state of art CNN architectures, which insert gradients midpoint and has very good performance.

So overall 32 bit performance is the one which should really matter for deep learning, unless you are doing a very very high precision job (which still would hardly matter as small differences due to 64 bit representation is literally erased by any kind of softmax or sigmoid). So 64 bit might increase your accuracy classification by $<< 1 {\%}$ and will only become significant over very large datasets.

As far as raw specs go the TITAN RTX in comparison to 2080Ti, TITAN will perform better than 2080Ti in fp64 (as its memory is double than 2080Ti and has higher clock speeds, BW, etc) but a more practical approach would be to use 2 2080Ti's coupled together, giving a much better performance for price.

Side Note: Good GPU's require good CPU's. It is difficult to tell whether a given CPU will bottleneck a GPU as it entirely depends how the training is being performed (whether data is fully loaded in GPU then training occurs, or continuous feeding from CPU takes place.)
Here are a few links explaining the problem:

CPU and GPU Bottleneck: A Detailed Explanation

A Full Hardware Guide to Deep Learning


Could we code in Natural language once we pass the Turing test? Technically speaking, could we code in natural language once we pass the Turing test? Would passing the Turing test at least simplify programming languages' syntax?

No.  Humans pass the Turing Test but cannot express themselves with enough precision and little enough ambiguity to code well in English (natural language).  No machine will improve on that until it can greatly improve on human-level clarity in expressing their purpose when programming.

Is it possible to program in natural language?  Yes, I think so.  But it would require superhuman intelligence to anticipate all the possible confusions that might arise when a word is not sufficiently precise or accurate to represent an software activity or mechanism.  This probably would require adopting a rigorous convention in choosing a subset of unambiguous words to form a "natural" programming language (a kind of creole).  Coding consistently using only those words would require discipline that exceeds the ability of most humans, methinks.


Can Reinforcement Learning solve problems, where certain elements in the environement are randomly located? I want to solve a problem using Reinforcement Learning on a 20x20 board. An agent (a mouse) has to get the highest possible rewards as fast as possible by collecting cheese, which there are 10 in total. The agent has a fixed amount of time for solving this problem, namely 500 steps per game. The problem is, the cheeses will be randomly assigned to the fields on the board, the agent knows however, where the cheeses is located.

Is there any way how this could be solved using only Reinforcement Learning (and not training for an absurd amount of time)? Or is the only way to solve this problem to use algorithms like the A*-algorithm?

I've tried many different (deep)-q-learning models, but it always failed miserably.

Edit: I could not get any meaningful behavior after 6 hours of learning while using a GTX 950M. Maybe my implementation was off, but i don't think so.

Yes you can use RL for this. The trick is to include the location of the cheese as part of the state description. So as well as up to 400 states for the mouse location, you have (very roughly) $400^{10}$ possible cheese locations, meaning you have $400^{11}$ states in total.

So you are going to want some function approximation if you want to use RL - you would probably train using a convolutional neural network, with an "image" of the board including mouse and cheese positions, plus DQN to select actions.

Viewed like this, a game where the mouse tries to get the cheese in minimal time seems on the surface a lot simpler than many titles in the Atari game environments, which DQN has been shown to solve well for many games.

I would probably use two image channels - one for mouse location and one for cheese location. A third channel perhaps for walls/obstacles if there are any.

  Or is the only way to solve this problem to use algorithms like the A*-algorithm?

A* plus some kind of sequence optimisation like a travelling salesman problem (TSP) solver would probably be optimal if you have been presented the problem and asked to solve it any way you want. With only 11 locations to resolve - mouse start plus 10 cheese locations - then you can brute force the movement combinations in a few seconds on a modern CPU, so that part may not be particularly exciting (whilst TSP solvers can get more involved and interesting).

The interesting thing about RL is how it will solve the problem. RL is a learning algorithm - the purpose of implementing it is to see what it takes for the machine to gain knowledge towards a solution. Whilst A* and combinatorial optimisers are where you have knowledge of how to solve the problem and do so as optimally as possible based on a higher level analysis. The chances are high that an A*/optimiser solution would be more robust, quicker to code, and quicker to run than a RL solution.

There is nothing inherently wrong with either approach, if all you want to do is solve the problem at hand. It depends on your goals for why you are bothering with the problem in the first place.

You could even combine A* and RL if you really wanted to. A* to find the paths, then RL to decide best sequence using the paths as part of the input to the CNN. The A* analysis of routes would likely help the RL stage a lot - add them as one or more additional channels.


deep learning with kfold cross validation with epochs I am new into neural networks, I want to use K-fold cross-validation to train my neural network. I want to use 5 folds 50 epochs and a batch size of 64 I found a function in scikit for k-fold cross validation

model_selection.cross_val_score(model_kfold, x_train, y_train, cv=5)

and my code without cross validation is

history = alexNet_model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1,validation_data=(x_validation, y_validation))

I don't know how to implement this with batch size and epochs in python using keras and scikit. any idea?

Here's a good resource showing you the exact code for what you're trying to do.

Here's the particular snippet:

# evaluate a model using k-fold cross-validation
def evaluate_model(dataX, dataY, n_folds=5):
    scores, histories = list(), list()
    # prepare cross validation
    kfold = KFold(n_folds, shuffle=True, random_state=1)
    # enumerate splits
    for train_ix, test_ix in kfold.split(dataX):
        # define model
        model = define_model()
        # select rows for train and test
        trainX, trainY, testX, testY = dataX[train_ix], dataY[train_ix], dataX[test_ix], dataY[test_ix]
        # fit model
        history = model.fit(trainX, trainY, epochs=10, batch_size=32, validation_data=(testX, testY), verbose=0)
        # evaluate model
        _, acc = model.evaluate(testX, testY, verbose=0)
        print('> %.3f' % (acc * 100.0))
        # stores scores
        scores.append(acc)
        histories.append(history)
    return scores, histories

So you're using .fit in a loop over the folds set up by sklearn.model_selection.KFold.


If mankind can create artificial life in a machine, when would we define it's death? Can one actually kill a machine? Not only do we have problems in defining life, we also have problems in defining death. Will this also be true in artificial life and artificial intelligence?

Death as we know it for natural life is terminal. That is once dead, natural life cannot come back (at least in the current understanding and with current technologies---some people believe otherwise).

Death for AI is trickier. There may be only one scenario: Global destruction: Extreme scenario where everything supporting the existence of an AI disappears. This is equivalent to death in natural life, and low probability. It means all AIs die at once (as well as us).

We also do not know the degree and form of embodiment necessary for AGIs. We can assume now that hardware is replaceable indefinitely, thus "limiting" death to the above extreme scenario. But AGIs "body" may not be indefinitely replaceable. Then a definition closer to natural life death may be necessary.

We see arguments for two other scenarios, that I refute below:

"Static Death": An AI is still "defined" or "saved" somewhere (whatever it means actually), but it is not authorized or able to use resources. Assuming an AI is made of hardware and software, it is like a program stored on a disk, but without permission to run.

"Dynamic Death": Under the same characterization of AI as hardware and software, dynamic death is the invalidation of progress akin to strong liveness properties, where an AI is trapped in an infinite loop (or a void loop), in a form of "active death", as what happens to Sisyphus in Greek mythology. This is different from static death, as the AI still uses dynamic resources, although it cannot make progress. Continuing under the same assumptions, such AI could be "loaded" in main memory, or locked waiting for inputs or outputs to complete.

Note that in these two scenarios, rebirth is possible, and they also subsume that there is an entity that can decide conditions for rebirth, or preventing it completely. Would this entity be an "admin", a god, other AIs, or a human is another question, really.

The terms "death" and "rebirth" here could just be changed for "imprisoning", where the dynamic version would be like our human prisons, and the static version would be like SciFi cryogeny. This is a bit of a stretch, but we can see an equivalence, and no good reason to qualify these two scenarios as deaths.

In conclusion, death for AI seems to be an exceptional, singular scenario, so AI cannot die in practice, except if we are wrong on how we think we can make AGIs. AI can however be imprisoned forever.

Note: The terminology above is completely made-up for the post. I do not have citations to back some claims, but it is based on readings and personal work (including in software verification).


What is feature embedding in the context of convolutional neural networks? What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?

The term feature embedding appears to be a synonym for feature extraction, feature learning etc. I.e. a form of embedding/dimension reduction (with the caveat the goal may not be a lower dimensional representation but one of equal dimensionality, but more meaningfully expressed):

  Feature embedding is an emerging research area which intends to transform features from the original space into a new space to support effective learning.

Generalized Feature Embedding for Supervised, Unsupervised, and Online Learning Tasks (2019)

  Feature embedding aims to learn a low-dimensional vector
  representation for each instance to preserve the information
  in its features.

Large-Scale Heterogeneous Feature Embedding (2019)


What are some alternatives to "Papers with Code"? There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often not available.
Papers with Code (and the associated Github repo) already lists many research papers and often there is a link to the associated Github repo with the code, but sometimes the code is missing. So, are there alternatives to Papers with Code (for such cases)?

Recently arxiv.org added a Code Tab towards the end of paper descriptions. Which contains links to both the official and community code.

I don't know if this is the case for all the papers or not till know. But I'm sure it'll be extended to all the papers in a short while.